Professional Documents
Culture Documents
org/9781107001114
The Cambridge Handbook of Human
Affective Neuroscience
Neuroscientific research on emotion has developed dramatically over the past decade. The
cognitive neuroscience of human emotion, which has emerged as the new and thriving area
of “affective neuroscience,” is rapidly rendering existing overviews of the field obsolete. This
handbook provides a comprehensive, up-to-date, and authoritative survey of knowledge and
topics investigated in this cutting-edge field. It covers a range of topics, from face and voice
perception to pain and music, as well as social behaviors and decision making. The book
considers and interrogates multiple research methods, among them brain imaging and phys-
iology measurements, as well as methods used to evaluate behavior and genetics. Editors
Jorge Armony and Patrik Vuilleumier have enlisted well-known and active researchers from
more than 20 institutions across three continents, bringing geographic as well as method-
ological breadth to the collection. This timely volume will become a key reference work for
researchers and students in the growing field of neuroscience.
Jorge Armony holds the Canada Research Chair in Affective Neuroscience in the Depart-
ment of Psychiatry at McGill University. He is also a researcher at the Douglas Mental
Health University Institute and a member of the International Laboratory for Brain, Music,
and Sound Research (BRAMS). Dr. Armony’s research focuses on the neural mechanisms
of emotional processing across modalities, including the interactions of these mechanisms
with other cognitive functions in healthy individuals as well as in patients suffering from
psychiatric and neurological disorders.
Patrik Vuilleumier is a professor at the University of Geneva Medical School, where he leads
the Laboratory for Neurology and Imaging of Cognition and directs the Geneva Neuro-
science Center. Dr. Vuilleumier was part of the interdisciplinary team that helped shape the
Swiss National Center of Competence in Research in Affective Sciences. Dr. Vuilleumier’s
research focuses on the influence of emotion processing on perception, attention, and action,
using functional neuroimaging techniques and neuropsychological studies of brain-damaged
patients.
The Cambridge Handbook of Human
Affective Neuroscience
Edited by
JORGE ARMONY
McGill University
PATRIK VUILLEUMIER
University of Geneva
CAMBRIDGE UNIVERSITY PRESS
Cambridge, New York, Melbourne, Madrid, Cape Town,
Singapore, São Paulo, Delhi, Mexico City
Cambridge University Press
32 Avenue of the Americas, New York, NY 10013-2473, USA
www.cambridge.org
Information on this title: www.cambridge.org/9780521171557
C Cambridge University Press 2013
A catalog record for this publication is available from the British Library.
Cambridge University Press has no responsibility for the persistence or accuracy of URLs for external or third-party
Internet Web sites referred to in this publication and does not guarantee that any content on such Web sites is, or
will remain, accurate or appropriate.
To the memory of
David Servan-Schreiber, superb colleague and
generous friend who introduced me to, among many
other things, the field of Human Affective
Neuroscience (JA)
Jon Driver, a truly inspiring mentor whose
guidance and friendship have been a unique
experience that is still very much alive in my work (PV)
Contents
vii
viii CONTENTS
Index 663
xi
xii LIST OF CONTRIBUTORS
Understanding human emotion and the tive processes more “visible” and amenable
mechanisms underlying its generation or to experimental research in humans.
expression has been a central preoccupa- Largely building on previous research
tion of thinkers for millennia. Yet, its sci- in neurophysiology, human affective neu-
entific study, particularly from a biological roscience research began by focusing on
perspective, is quite recent, especially in the so-called basic emotions, particularly
comparison to that of other mental pro- fear, mostly through visual stimuli (e.g.,
cesses, such as vision, language, attention, facial expression). However, as illustrated
or memory. Despite this late start, neuro- in the wide range of topics covered here,
scientific approaches to emotion have expe- emotion research now covers different sen-
rienced a dramatic growth over the past sory modalities, processes, interactions with
decade. This has led to the birth of the other systems, as well as individual differ-
new area of affective neuroscience, which ences. Emotion is now an accepted compo-
has extended the field of cognitive research nent of many “unrelated” disciplines, such
initiated in the previous decade. This new as social psychology, economics, marketing,
development was in large part due to impor- politics, and philosophy.
tant advances in the use of noninvasive This book is intended to provide a wide
functional neuroimaging techniques – such yet comprehensive, up-to-date, and author-
as positron emission tomography (PET), itative review of the cognitive neuroscience
electroencephalography (EEG), magneto- of human emotion that is both rigorous and
encephalography (MEG), and, particu- accessible. Naturally, to keep the book man-
larly, functional magnetic resonance imaging ageable and of a reasonable size, we had to
(fMRI). Together with refinements in more make some difficult choices in terms of its
traditional methods, such as lesion stud- contents. Rather than choosing a few snip-
ies, behavioral measures, and physiological pets from the entire field of affective neuro-
recordings, the new techniques helped sci- science, we decided to focus on a specific
entists make subjective and “private” affec- area within the field. With this in mind,
1
2 INTRODUCTION
we explicitly left out nonhuman animal olfactory, and somatosensory) and different
work. This does not mean in any way that domains within a given modality (e.g., audi-
we underestimate the importance of this tory: voices and music; vision: faces and
research. Indeed, as stated in many of the bodies). We decided to take this approach
chapters, research in experimental animals rather than, say, dividing the section accord-
has been critical in providing the framework ing to the basic emotions, because most
in which human affective neuroscience has researchers, and thus their work, tend to
developed; the authors were encouraged to focus on one of these domains but often
highlight corresponding links with animal encompass several emotions and/or pro-
and biological sciences whenever possible. cesses. Thus, this structure, although some-
However, adding this perspective to the what arbitrary (because emotion is typically
book would have required substantial cover- multimodal) will be most helpful to readers
age of molecular and cellular techniques that and reflects the current mainstream direc-
go beyond the aims of a single book. We also tions in human affective neuroscience.
left out more clinically oriented research, Section IV follows with a description
such as emotional dysfunction in psychiatric of how emotion and cognition interact.
and neurological disorders, although several In this large and ever growing field, we
of the chapters, especially those in the Indi- focus on some of the most studied top-
vidual Differences section, are highly relevant ics; namely emotion-attention interactions,
to this important area of knowledge. emotion regulation, and decision making.
One of the key features of this volume is Because of its importance and the large
that all the invited authors are established, literature associated with it, interactions
yet young researchers – the Generation X between emotion and learning and mem-
of human affective neuroscience research – ory are covered in a separate section (Sec-
representing more than 20 institutions across tion V); its three chapters cover implicit and
three continents. They are some of the most explicit aspects of memory, aversive learn-
active researchers who have contributed to ing, and reward learning. Chapters in Sec-
the field and are still doing so. tion VI address recent research in the so-
The volume’s 28 chapters are organized called higher emotions, including morality,
into seven independent yet complementary empathy, and other social emotions. Finally,
sections. We believe that this organization Section VII covers some of the most stud-
of topics will help readers gain a broad and ied individual differences – namely sex and
structured view of the field. gender, anxiety, age, and genotype – in emo-
Section I provides an introduction to the tional processing.
study of emotion from a cognitive neuro- This book is particularly aimed at
science perspective. It is followed by a me- scientists and students of all levels (under-
thodology section (Section II) that presents graduate, graduate, and postdoctoral) from
some of the most effective and widely used psychology, neuroscience, and cognitive
approaches to measure emotional responses. science, as well as people from other
It describes the various techniques in a rig- disciplines – including medicine, biology,
orous yet accessible manner, with particular computer science, economics, sociology,
emphasis easy-to-follow on affective neuro- and political science – who have an interest
science research – highlighting the advan- in the relation between emotion and their
tages and limitations of each approach and area of study or research. In addition, this
providing concrete examples to help the book should be useful to more clinically
reader appreciate these issues. oriented professionals, including physi-
Section III consists of six chapters cov- cians and therapists, who are interested
ering emotional perception and expression in gaining a better understanding of the
across different modalities (visual, auditory, neurobiological bases of human emotions.
Section I
INTRODUCTION TO
HUMAN AFFECTIVE
NEUROSCIENCE
CHAPTER 1
Models of Emotion
The Affective Neuroscience Approach
David Sander
Since its emergence in the 1990s (e.g., David- In this context, the overall aim of this
son & Sutton, 1995; Panksepp, 1991), affec- chapter is to consider major current mod-
tive neuroscience has considerably extended els of emotion by using an affective neuro-
our knowledge of the emotional brain. How- science approach. It provides a global survey
ever, affective neuroscience research has of historical and conceptual issues that have
only started to influence interdisciplinary guided scientific inquiries about emotion
models of emotion. The scientific object and presents the major theoretical founda-
of affective neuroscience is “affect,” which tions for more experimental work described
many disciplines share. Yet the way that in the following chapters. Although the
affective neuroscience approaches affect and scope of affective neuroscience research is
emotion is unique. For historical reasons and not limited to emotion but includes other
because of epistemological boundaries, psy- affective phenomena such as moods, prefer-
chological, neuroscientific, computer-based, ences, and affective dispositions, this chap-
and philosophical models of emotion have ter examines models of emotion because
developed relatively independently from they are more typically the focus of affec-
each other during most of the 20th century. tive neuroscience research.
Today, however, there is hope that the inter- After having introduced what is implied
disciplinary nature of affective neuroscience by an affective neuroscience approach to
will be able to constrain such varied mod- models of emotion, I address terminologi-
els of emotion by bridging the gaps among cal and taxonomy-related issues and suggest
different disciplinary approaches to emo- what seems to be a relatively consensual def-
tion. Various debates that exist between and inition of emotion. Next, I outline the major
within disciplinary approaches to emotion models of emotion in modern research and
could also benefit from the search for con- the contrast in their focus on different phe-
verging behavioral, computational, and neu- nomena: expression, action tendencies, bod-
ral evidence that is characteristic of affective ily reaction, feeling, and cognition. Finally,
neuroscience. as a brief conclusion, I illustrate the potential
5
6 DAVID SANDER
definitions have evolved and whether emo- emotion found in the literature and cate-
tions have changed over historical time; gorized them in 10 specific lists that empha-
for example, whether today’s emotions of sized various aspects of emotion: (1) affec-
“shame” or “anger” are the same as those tive definitions (emphasizing feelings of
described by ancient Greeks, Mesopotami- arousal and/or hedonic value); (2) cognitive
ans, or other civilizations. definitions (emphasizing appraisal and/or
Of course, the difference mentioned here labeling processes); (3) external stimuli
in terms of time can also be investigated definitions (emphasizing external emotion-
in terms of space. Although the differences generating stimuli); (4) physiological defini-
found over the centuries cannot be directly tions (emphasizing internal physical mech-
investigated by affective neuroscience, the anisms of emotion); (5) expressive behavior
cultural differences that are observed today definitions (emphasizing externally observ-
are a classical topic of emotion psychology able emotional responses); (6) disruptive
(see e.g., Tsai, Knutson, & Fung, 2006) and definitions (emphasizing disorganizing or
have begun to be investigated from an affec- dysfunctional effects of emotion); (7) adap-
tive neuroscience approach – as suggested, tive definitions (emphasizing organizing or
for instance, by the publication of a spe- functional effects of emotion); (8) multi-
cial issue on Cultural Neuroscience in the aspect definitions (emphasizing several
journal Social Cognitive and Affective Neuro- interrelated components of emotion); (9)
science (see Chiao, 2010). Reviewing histor- restrictive definitions (distinguishing emo-
ical and cultural effects on emotion would tion from other psychological processes);
go far beyond the scope of this chapter, but, and (10) motivational definitions (emphasiz-
as it will be discussed later, the question ing the relationship between emotion and
of whether emotions are a universal phe- motivation).
nomenon or whether they vary as a function In affective neuroscience, scholars also
of time and space is fundamental for many disagree on how to define an emotion.
theories in affective sciences. For instance, let us consider the defini-
tions offered by two of the most influen-
tial scholars of current research on the emo-
The Variety of Definitions
tional brain, Damasio (1998) and LeDoux
of Emotion
(1994). LeDoux (1994, p. 291) highlighted the
“What is an emotion?” is not only the title fact that emotions cannot be unconscious
of one of the most widely cited articles on when stating that “in my view, emotions
emotion (James, 1884) but is also a current are affectively charged, subjectively expe-
conceptual question in emotion research rienced states of awareness. Emotions, in
that seems to correspond to a never end- other words, are conscious states.” Accord-
ing attempt to define emotion (see, e.g., ing to Damasio (1998, p. 84), “the term emo-
Duffy, 1934; Frijda, 2007; Gendron & Feld- tion should be rightfully used to designate a
man Barrett, 2009; Kleinginna & Kleinginna, collection of responses triggered from parts
1981; Russell & Barrett, 1999; Scherer, 2005). of the brain to the body, and from parts of
Certainly, what affective neuroscience can the brain to other parts of the brain, using
significantly contribute to this effort is an both neural and humoral routes.” There-
understanding of emotion as a scientific con- fore, Damasio certainly does not exclude
cept, in particular by offering functional the possibility that what he calls an emotion
architectures of emotional processes in the can be unconscious. Distinguishing between
form of explicit models. emotion and feeling, Damasio (1998, p. 84)
A necessary step in modeling emotion also stated that “the term feeling should be
is to acknowledge the variety of definitions used to describe the complex mental state
that scholars have given to emotion. In a that results from the emotional state.” It is
tour de force, Kleinginna and Kleinginna likely that such a mental state is conceptually
(1981) reviewed almost 100 definitions of closer to what LeDoux called an emotion,
MODELS OF EMOTION 9
although Damasio called it a feeling rather often considered causal for the elicitation of
than an emotion. emotion because events that are relevant for
major motivations of the individuals (e.g.,
needs or goals) are indeed those that typi-
The Specificity of Emotion
cally elicit emotions (see Moors, 2007). As a
The distinction between emotion and feel- constituent of emotion, motivation is often
ing mentioned earlier is just one of many considered as being expressed in action ten-
conceptually useful distinctions that can be dencies (e.g., approach or avoidance) that
made in the category of affective phenom- indeed motivate a change in the relation
ena. In fact, the term “emotion” is often between the individual and the event (see
considered in a framework that includes Frijda, 1986). A definition highlighting the
other less studied affective phenomena such specificity of emotion is suggested later.
as mood, motivation, drive, desire, pref-
erence, attitude, valenced reaction, pas-
sion, sentiment, affect, core affect, arousal, Taxonomies of Emotion
affective style, or affective reactivity. Some
of these concepts are more scientifically Not only can emotion be distinguished from
defined because they have been coined to other affective phenomenon but also within
refer to a specific new concept and there- the category of “emotion,” various subcate-
fore suffer less than others from having gories have been proposed. To the best of
a “folk” meaning (e.g., affective style, see my knowledge, no full taxonomy of emo-
Davidson, 1992; core affect, see Russell & tion has achieved consensus, but some cate-
Barrett, 1999). Attempts to define these con- gories are recognized as conceptually useful.
structs have sometimes led to extreme posi- Taxonomies of emotion are based on vari-
tions. For instance, Duffy questioned the ous features, and categories often overlap so
specificity of emotion and argued that “for that they should not be seen as describing
many years the writer has been of the opin- mutually exclusive categories of emotion,
ion that ‘emotion’ as a scientific concept but rather as describing ways in which emo-
is less than useless” (Duffy, 1941, p. 283). tions are categorized in various research tra-
She argued that, because emotion can be ditions. Indeed, a given emotion (e.g., anger)
reduced to other constructs, there is no can belong to many categories.
need to create a specific term for emotional
states.
Basic Emotions
More recently, Brehm (1999) argued that
emotion can be reduced to motivational As an example of a category that is defined
states. Indeed, some question the bound- by the type of emotion, the so-called basic
aries between emotion and motivation. For emotions category is very common in cur-
instance, because Rolls (1999) included thirst rent affective neuroscience research (for
or sexual behavior as emotions in his book review, see Ortony & Turner, 1990). This
The Brain and Emotion, Phillips (1999) pro- category, which is conceptually similar to
posed that it might have been more appro- the categories of “primary,” “discrete,” or
priate to title this book “The Brain and “fundamental” emotions, acknowledges the
Motivation.” fact that, according to many researchers, a
Motivation is typically considered as small set of emotions – typically between 2
being related to emotion, but most schol- and 10 – are more elementary than others.
ars would agree on the need to distin- This concept of “basic emotions” is key to
guish between these two constructs (for dis- the development of the basic emotion the-
cussion, see, e.g., Frijda, 1986, 2007). For ory that is discussed later (see the section,
instance, motivation can be considered both “Is Emotion an Expression?”). The follow-
as a determinant and as a constituent of ing emotions are often considered as being
emotion. As a determinant, motivation is “basic”: anger, disgust, fear, enjoyment,
10 DAVID SANDER
sadness, and surprise (see Matsumoto & 2005), it is often the feeling component that
Ekman, 2009). is considered: An emotion is positive when
In this theory, the adjective “basic” is “it feels pleasant” or negative when “it feels
used to express three postulates (see Ekman, unpleasant.” This valence-based distinction
1992). First, it is used to convey the notion has been key to the development of the
that “there are a number of separate emo- circumplex/bidimensional theories of emo-
tions which differ one from another in tion that are discussed later (see in partic-
important ways” (Ekman, 1992, p. 170). Sec- ular the section, “Is Emotion a Feeling?”).
ond, it is used to indicate that “evolution Of course, the valence dimension is not
played an important role in shaping both restricted to the feeling component; elicit-
the unique and the common features which ing events are sometimes categorized as pos-
these emotions display as well as their cur- itive or negative in terms of their appraised
rent function” (Ekman, 1992, p. 170). Finally, intrinsic pleasantness or goal conduciveness
the term also often refers to the notion that (e.g., Scherer, 2001). There is not always con-
nonbasic emotions are made up of blends of gruence between the appraised valence of
basic emotions (e.g., Tomkins, 1963). an event and the valence of the feeling. For
Of note, the notion of basic emotions instance, although the emotion of “interest”
is anchored in the philosophical history of is considered as positive in terms of feel-
psychology; for instance, Descartes (1649, ing, it can also be elicited by appraised nega-
Art. 69) distinguished among six primary tive stimuli (e.g., disgusting stimuli can elicit
emotions (admiration, love, hatred, desire, interest; see Silvia, 2006b).
joy, and sadness) and assumed that all other Although feelings are often considered to
emotions either belonged to these families be either positive or negative, some scholars
or were blends of these primary emotions. have argued that evaluations of events can
Most of the work in affective neuro- be ambivalent (see Cacioppo & Berntson,
science in the last decade has consisted of 1994). This means that one can feel both good
searching for discrete dedicated brain sys- and bad about an event, rather than good or
tems underlying each and every basic emo- bad about it (see Larsen, 2007). Depending
tion, using as evidence either neuropsy- on the aspect of the event that is appraised,
chological double dissociations (see Calder, the very same event can be appraised as
Lawrence, & Young, 2001) or brain imaging positive or negative, meaning that if two
results (see Vytal & Hamann, 2010). As dis- aspects are appraised simultaneously by two
cussed later (see the section, “Is Emotion an dissociated evaluative channels, both posi-
Expression?”), this view has been strongly tive and negative feelings could be elicited
challenged by both conceptual analyses and (for discussion, see Cacioppo & Berntson,
empirical results. 1994). For instance, having a sexual relation-
ship with someone else than one’s signifi-
cant other can be appraised as positive in
Positive versus Negative Emotions
the sense that it elicits pleasure, but as neg-
Another example of a category that is ative because it interferes with moral con-
defined by the type of emotion is the cerns of the individual. Ambivalent attitudes
common valence-based distinction between have been considered as evidence for the
“positive” and “negative” emotions. For separability of positive and negative sub-
instance, Tomkins’ (1963) influential con- strates and the view that mixed emotions
tribution to affective sciences was a book could be jointly elicited. For instance, it has
divided into two volumes, the first volume been suggested that individuals can feel both
concerning positive affects and the second happy and sad at the same time while view-
one on negative affects. Although the type ing tragicomic movies (Larsen, McGraw, &
of valence used to distinguish between so- Cacioppo, 2001).
called positive emotions and negative emo- In affective neuroscience, the notion
tions is often not clear (see Colombetti, that brain systems could be differentially
MODELS OF EMOTION 11
involved in the processing of positive and cesses are transferred into a common cur-
negative stimuli is grounded in various rency (see Chapter 17).
research traditions. For instance, under- Another research tradition that has influ-
standing the brain mechanisms involved enced an affective neuroscience account of
in a pain/aversion system versus a plea- the opposition between positive and neg-
sure/reward system (see Haber & Knutson, ative emotions is grounded in investiga-
2010; Lieberman & Eisenberger, 2009) has tions of functional hemispheric asymme-
been a major aim of affective neuroscience. try (see the next section). The so-called
Lieberman and Eisenberger (2009) suggested valence hypothesis of hemispheric asymme-
that the “pain network” consists of the dorsal try posits that there is a center for pos-
anterior cingulate cortex, insula, somatosen- itive feelings in the left hemisphere and
sory cortex, thalamus, and periaqueductal a center for negative feelings in the right
gray (see Chapter 9), whereas the “reward hemisphere (e.g., Ahern & Schwartz, 1979).
network” consists of the ventral tegmen- This hypothesis is highly debated, and sev-
tal area, ventral striatum, ventromedial pre- eral alternatives have been suggested in
frontal cortex, and the amygdala (see Chap- the literature (for discussions, see Gain-
ter 19). otti, 2000; Killgore & Yurgelun-Todd, 2007).
Another related tradition is illustrated For instance, the so-called right-hemisphere
by the work of Berridge and colleagues, hypothesis suggests that all emotion-related
who distinguish between brain mechanisms mechanisms are more lateralized in the right
subserving the processes of “liking” versus hemisphere.
“wanting” (see, e.g., Berridge & Robinson,
2003). This trend of research led to the
Approach-Related versus
suggestion that there exist several “hedonic
Avoidance-Related Emotions
hotspots” in the brain, including regions of
the nucleus accumbens and the ventral pal- An alternative to the “valence hypothesis”
lidum that are involved in the liking pro- described earlier is the account of “ante-
cess, whereas the dopamine system, often rior brain asymmetry,” as mainly tested
considered to mediate pleasure, would, in by Davidson and colleagues. Davidson and
fact, mediate a particular form of wanting Irwin (1999) proposed the existence of an
for reward called “incentive salience” (see approach system that facilitates appetitive
Berridge & Robinson, 2003). This distinc- behavior and generates some approach-
tion between liking and wanting could also related positive emotions such as pride or
lead, in principle, to ambivalent processes, enjoyment. These kinds of emotions would
given that, for instance, individuals could be generated in the context of moving
like what they do not want or want what toward a goal. Such a system would be lat-
they do not like, consistent with the notion eralized toward the left hemisphere. These
that people can come to both want some- authors also postulated a second system
thing more and like it less (Litt, Khan, & that would, however, be lateralized toward
Shiv, 2010). the right hemisphere. This system facilitates
Such a distinction between liking and withdrawal and generates some withdrawal-
wanting speaks in favor of the importance related negative emotions such as fear or
of distinguishing different types of valences disgust.
in affective neuroscience, as is the case An opposition between approach and
in other disciplines (see Colombetti, 2005). avoidance behaviors in the emotional
This approach can have considerable effects response is very often proposed and is
on the literature on preferences, values, and supposed to have a strong phylogenetic
decision making, as discussed, for instance, basis (see Schneirla, 1959). Most theories of
in the neuroeconomics literature, because it emotion acknowledge action tendencies to
may represent a complementary approach approach pleasure and avoid pain, and this
to the one advocating that all valenced pro- concept originates in philosophical writings.
12 DAVID SANDER
For instance, Hobbes (1651/1985, p. 119) in affective neuroscience, given the recent
suggested a dissociation between these rise of interest and research on nonbasic
two behaviors as they relate to desire emotions (e.g., Takahashi et al., 2009) on
when he wrote that “this Endeavour, one hand and on the self (e.g., Powell
when it is toward something which causes et al., 2009) on the other (see Chapter 21).
it, is called Appetitive. . . . And when the Affective neuroscience could certainly play
Endeavour is [away] from . . . something, a key role in understanding the specificities
it is generally called Aversion. These of subtypes of self-reflexive emotions (e.g.,
words Appetite, and Aversion . . . signify the Basile et al., 2011) by providing empirical
motions, one of approaching, the other of evidence that can constrain debates con-
retiring.” cerning the differences and similarities of
A particularly interesting aspect of the emotions such as shame and guilt (see
hemispheric asymmetry hypothesis is that Deonna & Teroni, 2008; Wagner, N’Diaye,
it does not overlap with the valence hypoth- Ethofer, & Vuilleumier, 2011).
esis, because it claims that a negative but
approach-related emotion such as anger
would be lateralized over the left hemi- Aesthetic Emotions
sphere (for a discussion and review of
A second example of a distinction that is
empirical evidence, see Carver & Harmon-
based on the type of object that elicits the
Jones, 2009). The “approach versus avoid-
emotion is the category of aesthetic emo-
ance” partition is important because it
tions. These emotions are typically elicited
allows the distinction of a valence-based dis-
when people engage with artworks or with
sociation from an action-tendencies-based
objects or scenes in nature (see Robinson,
dissociation.
2009). There is a debate whether aesthetic
emotions are a special type of emotion or
whether this category should be mainly
Self-Reflexive (or Self-Conscious)
defined by the fact that it is elicited by art-
Emotions
work (see Robinson, 2005).
A typical example of a distinction based on Some appraisal accounts of aesthetic
the type of object that elicits the emotion is emotions have been suggested (e.g., Silvia,
the category of so-called self-reflexive emo- 2006a). Such an approach is useful for ana-
tions such as shame, embarrassment, guilt, lyzing two critical issues in the study of aes-
or pride. A specific feature of this category thetic emotions. First, given that appraisal
is that the object of the emotion is the self, mechanisms are key to explaining individ-
rather than the eliciting event. For instance, ual differences, this approach can be used to
one feels ashamed about oneself, but one explain why people have different emotions
feels afraid about a snake. As discussed by in response to the same work of art. Sec-
Fontaine (2009), scholars often refer to this ond, it can be used to explain how exper-
category to describe emotions in which the tise determines aesthetic emotions (see
self rather than a survival concern is at Silvia 2006a, 2009).
stake. In this regard, in addition to the two The investigation of aesthetic emotions
particularly well studied self-reflexive emo- has somewhat favored positive emotions
tions of shame and guilt (Deonna & Teroni, over negative emotions, but the emo-
2008; Tangney & Dearing, 2002), other emo- tions elicited by works of art are not lim-
tions for which the self is at stake could ited to positive emotions (e.g., an emotion
therefore be included (e.g., humiliation, elicited by a work of art appraised as “ugly”).
gratitude, envy, or jealousy; see Fontaine, Although the growing field of neuroesthet-
2009). ics, which aims at understanding the neuro-
These emotions, sometimes also called biological basis of aesthetic experience (see
“self-conscious emotions” or “moral emo- Zeki, 2001), has focused on the visual arts,
tions,” are increasingly being investigated auditory art has also been investigated, with
MODELS OF EMOTION 13
in terms of differences between envy and with Haidt (2003) four types of moral emo-
jealousy, Parrott and Smith (1993) suggested tions: (1) those that are self-conscious (e.g.,
that envy is characterized by feelings of infe- shame and guilt); (2) those that are other-
riority, longing, resentment, and disapproval condemning (e.g., contempt, anger, and dis-
of the emotion but that jealousy is charac- gust), (3) those that are elicited by the suf-
terized by fear of loss, distrust, anxiety, and fering of others (e.g., compassion), and (4)
anger. those that are other-praising (e.g., gratitude
A related research question concerns how and elevation).
emotions are modulated by the social con- In addition to the study of moral emo-
text. For instance, fear, which is not a tions, affective neuroscience research has
“social emotion” – in the sense that it is considered moral judgments as a way to
not typically elicited by social events – can study the affective determinants of moral-
still be modulated by how other individu- ity; for instance, by using moral dilemmas
als appraise the danger that elicited fear. For (e.g., Greene et al., 2001; see Chapter 21).
example, if a baby is hesitating to cross a
visual cliff because he is afraid to fall and
Epistemic Emotions
his mother expresses fear on her face, then
the baby will be less likely to cross than if Some emotions such as interest, confusion,
his mother smiles (see Sorce, Emde, Cam- surprise, or awe particularly relate to knowl-
pos, & Klinnert, 1985). More generally, the edge and learning, and are, for this reason,
notion of “social appraisal” illustrates the sometimes called “epistemic” (or “knowl-
fact that the way an individual appraises edge”) emotions (Morton, 2010; Silvia, 2010;
an event can be modulated by how other de Sousa, 2008).
individuals appraise the very same event For instance, the emotion “interest” plays
(see Manstead & Fischer, 2001). Results sug- a key role in exploration, learning, growth of
gest, for instance, that social appraisal influ- knowledge, and the development of exper-
ences the recognition of facial expressions tise in many domains (Silvia, 2006b). The
of emotion (Mumenthaler & Sander, 2012). appraisal structure of interest has been stud-
Studies that have compared how the brain ied, and results suggest that the events that
computes social versus nonsocial emotional elicit interest are those that are appraised as
information suggest that some regions that novel and complex, but comprehensible (Silvia,
are critical for emotional processing are also 2006b).
involved in the processing of social relevance Although epistemic emotions have not
(see Norris et al., 2004; Scharpf, Wendt, been the focus of much research in affec-
Lotze, & Hamm, 2010; Vrticka et al., 2011). tive neuroscience, related aspects such as
novelty processing have indeed been exten-
sively investigated, with a key role attributed
Moral Emotions
to the amygdala for the processing of
Moral emotions are those emotions that are new and unfamiliar stimuli (e.g., Blackford,
elicited by moral evaluations. As described Buckholtz, Avery, & Zald, 2010) – possi-
by Mulligan (2009), such emotions possibly bly relating to a role of the amygdala in
rely on various moral phenomena such as uncertainty and ambiguity processing (see
moral norms (e.g., one ought not to tell Whalen, 1998).
lies), moral obligations (e.g., to look after As noted by Morton (2010), even emo-
one’s aged parents), moral right and wrong tions that are not directed at knowing (i.e.,
(e.g., murder), moral values (e.g., good- nonepistemic emotions) can lead to epis-
ness), and moral virtues (e.g., courage). Var- temic consequences. For instance, fear cer-
ious classifications of moral emotions have tainly leads to an increase in vigilance and
been suggested (see Haidt, 2003; Mulligan, attention to the threat, as well as the aim
2009; Tangney, Stuewig, & Mashek, 2007). of better knowing the situation to discover
For instance, one can distinguish together possibilities for escape.
MODELS OF EMOTION 15
To conclude this section on the tax- of emotion. This account focused on the
onomies of emotion, one can note that the general framework that was common to the
described categories are not mutually exclu- theories as proposed by James (1884) and
sive. For instance, anger can be described as Lange (1885). The main thesis defended by
a typically basic, negative, approach-related, these scholars can be summarized by James’s
often social, and sometimes moral emotion. (1884, p. 189) definition of emotion, which
These taxonomies of emotion are possibly certainly has been the most widely cited
useful ways to consider the various types definition of emotion since: “Bodily changes
of emotions and to simplify this variety. follow directly the PERCEPTION of the
Another, certainly more productive, way of exciting fact, and that our feeling of the
reducing the complexity is to develop mod- same changes as they occur IS the emotion.”
els of emotion aimed at representing the For instance, Cannon’s (1927) neuroscien-
current knowledge of the functional archi- tific theory of emotion was a direct reaction
tecture of emotion and to allow testable to James’s definition of emotion and gave
predictions to be derived. At the origin of rise to the very influential thalamic theory
the current models, one finds debates con- of emotion (see also Cannon, 1931), setting
cerning definitions of emotions. the stage for contemporary debates on the
respective roles of the central nervous sys-
tem and the peripheral nervous systems in
Definitions of Emotion and the Origin emotion (see Damasio, 1998; Chapter 3).
of Current Models Partly on the basis of Cannon’s work
in insisting on the role of the thalamus
The period that was arguably the most influ- and hypothalamus in emotion (see Lash-
ential for current models and debates in ley, 1938), Papez (1937) produced the first
affective sciences was the second part of explicit proposal of a cerebral circuit as a
the 19th century. Indeed, most of the cur- mechanism of emotion by also adding the
rent models of emotion can be traced to – at hippocampus and the cingulate cortex to
least – this period and to the work of Dar- it. Relying in particular on Cannon’s and
win, Dewey, Irons, James, Lange, Spencer, Papez’s works on brain regions involved
and Wundt to cite some of the most influ- in emotion and on Broca’s (1878) work on
ential scholars. the anatomical description of a “great lim-
This period was not only critical for bic lobe,” MacLean (1952) proposed a con-
debates concerning what an emotion is but cept that has been extremely influential in
it was also fundamental for preparing the the field: the limbic system. An important
emergence of neuroscientific accounts of extension of Papez’s model was MacLean’s
emotion. For instance, when Peper and inclusion of the amygdala as part of the lim-
Markowitsch (2001) considered the pioneers bic system, following, for instance, the work
of affective neuroscience, they pointed to of Kluver and Bucy (1939).
those who described early concepts of the However, the very notion of the lim-
emotional brain during that period: Exner’s bic system as being the unitary basis of
(1894) suggestion of an aversiveness process- the emotional brain has been strongly ques-
ing center in the brain, Freud’s (1895/1953) tioned and criticized (see Calder et al., 2001;
neuron network theory of emotional mem- LeDoux, 1991). Although the thalamic the-
ory, and Waynbaum’s (1907/1994) general ory, Papez’s circuit, and the limbic system
emotion center. are certainly no longer considered prevailing
In fact, the typical neuroscientific ac- brain-based models of emotion, they had a
counts of emotion that today are considered critical influence on what became affective
as classical (e.g., the existence of a limbic neuroscience a few decades later.
system) were launched by a reaction to a This contribution was particularly critical
very specific and controversial account of in the classical debate between the periph-
emotion, the so-called James-Lange theory eralists and the centralists. Indeed, whereas
16 DAVID SANDER
the so-called peripheralists relied on James’s ment of the last century that has strongly
(1884) definition and argued that the criti- shaped affective neuroscience is the cogni-
cal source of emotion was to be found in tive revolution. Although behaviorism never
the peripheral nervous system, the so-called dominated in Europe, the importance that
centralists relied on Cannon’s (1927) reaction this approach had in the United States
to James’s definition of emotion and argued meant that a cognitive approach to the mind
that the critical source of emotion was to be was considered as being a revolution (see
found in the central nervous system. Miller, 2003). This revolution, which took
In parallel with this debate, the age of place mainly in the 1950s, certainly influ-
behaviorism was flourishing, particularly in enced the emergence of models of emo-
the United States. The legacy of behaviorism tion that considered cognition as a criti-
for current definitions, models, and meth- cal aspect of emotion (e.g., Arnold, 1960;
ods is particularly strong in investigations of Lazarus, 1966; Schachter & Singer, 1962).
the emotional brain. For instance, the fear- The approach of considering the cogni-
conditioning experiments that Watson used tive mechanisms underlying the process of
to test conditioned emotional reactions in emotion elicitation (i.e., appraisal), as well
a human infant (Watson & Rayner, 1920) as the process of emotion categorization
have been the fundamental basis for the (i.e., labeling), allowed the development of
development of much of the work that has many influential cognitive models of emo-
led to our understanding of brain circuits tion in the 1980s (e.g., Frijda, 1986; Lazarus,
involved in fear learning in humans and ani- 1984; Ortony, Clore, & Collins, 1988; Rose-
mals, which has often been considered as man, 1984; Scherer, 1984). Such models were
a royal road for increasing knowledge con- considered cognitive because they defined
cerning the emotional brain more generally emotion as caused by cognitive appraisals
(Hartley & Phelps, 2010; LeDoux, 1996). In (e.g., Lazarus, 1966) or as labeled by cogni-
fact, the legacy of behaviorism to studies of tive categorization (e.g., Schachter & Singer,
the emotional brain goes beyond classical 1962).
conditioning to include operant condition-
ing with concepts that link emotion to moti-
A Typical Definition of Emotion
vation, such as those of “reward” and “pun-
ishment.” In that respect, many researchers Is there a minimal consensus for a defini-
who investigate emotions in the brain still tion of emotion? A review of recent major
use concepts and methods inherited from models of emotion indicates that there is
behaviorism, with a typical example being indeed consensus on four key criteria: (1)
the model of emotion suggested by Rolls Emotions are multicomponent phenomena;
that focuses on the role of reinforcement (2) emotions are two-step processes involv-
(see, e.g., Rolls, 2007). Indeed, according to ing emotion elicitation mechanisms that pro-
Rolls (2007, p. 72), “emotions may be defined duce emotional responses; (3) emotions have
as states elicited by reinforcers (rewards and relevant objects; and (4) emotions have a
punishers).” brief duration compared with other affec-
Understanding the brain mechanisms tive phenomena.
underlying reinforcement learning and, in
particular, identifying the reward circuit in EMOTIONS ARE MULTICOMPONENT
the brain have been major research ques- PHENOMENA
tions (see Haber & Knutson, 2010; Lieber- The three current dominant models of
man & Eisenberger, 2009) since the land- emotion – basic emotion models, circum-
mark study of Olds and Milner (1954), who plex/bidimensional models, and appraisal
showed that rats would work for electrical models – all seem to consider that an emo-
stimulation in several parts of their brain. tion is not unitary but rather has sev-
In addition to this behaviorist legacy, eral components. This idea is not partic-
another critical epistemological develop- ularly recent (e.g., Irons, 1897) but has
MODELS OF EMOTION 17
although one may not think of this first elic- mensions proposed by Lazarus (primary
itation step as being part of an emotion, one appraisal and secondary appraisal), a
feature of contemporary models of emotion detailed set of criteria has been devel-
is to consider emotion elicitation as not being oped by appraisal theorists to subserve the
antecedent to emotion, but as being consti- subjective interpretation of the personal
tutive of emotion. A useful analogy can be significance of events (for a review, see
made with memory. Indeed, memory could Scherer & Ellsworth, 2009). As described
easily be considered as corresponding mainly by Scherer and Ellsworth (2009), these cri-
to remembering: Memory is at work when teria include “the novelty or familiarity of
one remembers something. But according to objects or events; their intrinsic pleasantness
most contemporary models, there is more to or unpleasantness; their significance for the
memory than what is actually remembered: individual’s needs or goals; their perceived
There are also the encoding and consoli- causes (self, another person, or circum-
dating mechanisms. Although the process stances); the individual’s ability or power
of encoding could easily be seen as separate to influence or cope with the consequences
from – and antecedent to – memory, it is of the event, including the level of uncer-
typically considered as constitutive of mem- tainty; and the compatibility of the event
ory. Considering this analogy, elicitation is with social or personal standards, norms, or
to emotion what encoding is to memory: its values.”
first constitutive step. The number and order of these dimen-
Among the five components described sions (i.e., criteria) vary in different mod-
earlier, there are debates regarding which els. They assume that the level of pro-
components should be considered as belong- cessing of these dimensions is typically
ing to the emotion elicitation phase or to automatic (see Moors, 2009), but could
the emotional response phase. In fact, it also take place voluntarily. An affective
has been suggested that, as a process, emo- neuroscience approach to appraisal mech-
tion elicitation is modulated by rapid emo- anisms is relatively recent (see Brosch &
tional responses through feedback connec- Sander, in press; Sander et al., 2005) as com-
tions from the initial emotional responses pared with other theoretical traditions. For
that may further activate emotion elicita- instance, studies using electroencephalogra-
tion mechanisms (see Sander, Grandjean, & phy have tested the temporal order of some
Scherer, 2005). As depicted in Figure 1.1, the of these criteria (e.g., Grandjean & Scherer,
five components described earlier are typi- 2008). Other studies and conceptual anal-
cally categorized as being involved either in yses have pointed to a role of particular
emotion elicitation (appraisal component) brain structures in some of these criteria;
or in the emotional response (action ten- for instance, the amygdala is suggested to
dency, autonomic reaction, expression, and be critically involved in relevance detection
feeling components). (see Cristinzio et al., 2010; Sander, Graf-
Although appraisal dimensions (includ- man, & Zalla, 2003; and the concluding sec-
ing automatic appraisals) are considered in tion). In fact, some of the earlier mentioned
most theories to be the major determinants cognitive mechanisms (e.g., novelty detec-
of emotion elicitation (see the section, “Is tion) have been the focus of intense empir-
Emotion a Cognition?”), other mechanisms ical research in cognitive neuroscience (see,
have also been suggested to be involved e.g., Kumaran & Maguire, 2007), but typi-
in emotion elicitation. Indeed, some mod- cally without links being directly made to
els suggest that core relational themes, core emotion elicitation.
affects, embodied states, direct sensory trig- According to appraisal theories of emo-
gering, reflexes, instincts, or memory associ- tion, the appraisal component is a consti-
ations can elicit emotions. tutive (rather than antecedent) component
Appraisal dimensions. Mainly conceived of the emotional experience (see Moors,
as an extension of the two appraisal di- in press). The combination of appraisal
MODELS OF EMOTION 19
(e.g., James, 1894; see Ellsworth, 1994, for direct subcortical pathway from the audi-
discussion). tory thalamus to the amygdala, in addition
A very similar position was presumably to the more indirect cortical pathway (see
suggested by Lange (1885, p. 673): “If I begin LeDoux, 1996).
to tremble because I am threatened with a Although uncontroversial anatomical evi-
loaded pistol, does first a psychical process dence is lacking in humans, functional evi-
occur in me, does terror arise, and is that dence suggests that a colliculo-pulvinar-
what causes my trembling, palpitation of the amygdala subcortical pathway is involved in
heart, and confusion of thought; or are these coarse and fast processing of visual stim-
bodily phenomena produced directly by the uli in humans (e.g., Morris, DeGelder,
terrifying cause, so that the emotion consists Weiskrantz, & Dolan, 2001; Vuilleumier,
exclusively of the functional disturbances in Armony, Driver, & Dolan, 2003; see Vuilleu-
my body?” mier, 2005). A recent model has raised the
Here, again, most readers focus on the question of whether such a pathway might
notion that the bodily phenomena are now be better conceptualized as a two-stage
suggested to be the cause rather than architecture – rather than a dual route –
the consequence of emotion. When Lange according to which coarse and fast modes of
wrote that “bodily phenomena produced processing first occur in magnocellular path-
directly by the terrifying cause,” it is not fully ways and then are complemented by slower
clear whether the word “directly” referred parvocellular visual pathways (see Chap-
to sensory processing or to other automatic ter 14).
(possibly cognitive) processes. The notion of a very rapid and coarse
Because James’s and Lange’s definitions processing to the amygdala – whether it
insist on a direct elicitation of the bodily corresponds to a direct anatomical pathway
response, the study of which brain mech- or a first stage of processing – is typically
anisms are involved in emotion elicitation interpreted as accounting for the direct sen-
has led to results that have often been sory triggering of emotion. If one admits
interpreted as indicating a possible direct that direct sensory processing can elicit full-
sensory triggering of emotion. Zajonc was blown emotions, a key question is whether
particularly influential in that respect. For this is the rule or the exception (see Lev-
instance, describing the multilevel model enthal & Scherer, 1987; Robinson, 1998). It
of processing from the stimulus presenta- is possible that covert activation may not
tion to sensation to affect and to cold cog- induce any subjective feeling state, but may
nitions, Zajonc (1980, p. 171) claimed that still modulate indirect responses such as star-
“an affective reaction always directly follows tle (Anders, Weiskopf, Lule, & Birbaumer,
the sensory input.” In particular, from the 2004), or support intuition based on intero-
neuroanatomical work suggesting the exis- ceptive signals (Katkin, Wiens, & Öhman,
tence of a direct pathway from the retina to 2001).
the hypothalamus (see, e.g., Moore, 1973) in Reflexes. A similar perspective to that
the mammalian visual brain, Zajonc (1984, just described concerning the possible direct
p. 119) argued that “these findings would sensory triggering of an emotional response
imply that pure sensory input requiring no argues for the existence of emotional
transformation into cognition is capable of reflexes. For instance, Lang and Bradley
bringing about a full emotional response (2009, p. 334) wrote that “emotional reflexes
involving visceral and motor activity.” are physiological or behavioural reactions
More contemporary versions of this evoked automatically in humans by affec-
notion strongly rely on the presumed exis- tively evocative stimuli.” The very notion
tence of a subcortical “direct” pathway to of “reflexes” typically insists on the absence
the amygdala (see LeDoux, 1996). Indeed, of interpretation between the presence of a
experiments on auditory fear condition- stimulus and the production of an emotional
ing in rats have shown the existence of a response. Such reflexes are presumed to be
MODELS OF EMOTION 21
cortex is proposed to play a key role in the nificance but also refers to other types of
creation and activation of these associations concerns. For instance, Frijda (1986, 2007)
(see Damasio, 1994). has discussed the notion that emotions are
Of note, the role of associations in emo- elicited by events that are relevant to major
tion elicitation has been suggested not only concerns of an individual. Concerns are psy-
for vision but also for other sensory modal- chological representations that underlie or
ities, particularly for olfaction (see Chap- overlap with other motivational constructs
ter 10) and music (see Chapter 12). such as needs, goals, desires, and values. As
broadly defined, a concern is a disposition
From a theoretical perspective, a key con- to desire the occurrence or nonoccurrence
ceptual question is the type of eliciting of a given kind of situation. As proposed
mechanism that can be considered as being by Frijda, “source concerns” refer to general
constitutive of emotion. To the best of my kinds of goals and satisfaction (e.g., secu-
knowledge, multilevel appraisal dimensions, rity), whereas “surface concerns” are defined
core relational themes, core affects, and by such goals and satisfactions for a partic-
embodied states have been considered as ular person, object, or state of affairs (e.g.,
being components of emotions. Affective attachment to a political party).
neuroscience methods (see Chapters 2–6), A consensual definition of relevance
in particular those allowing high temporal detection that considers both the notions of
resolution, could be a great help in explain- “biological significance” (e.g., LeDoux, 1989)
ing how elicitation mechanisms shape the and of “primary appraisals” (e.g., Lazarus,
components of the emotional response 1991) would consider that an object or situa-
(i.e., action tendency, autonomic reaction, tion is appraised as relevant for an individual
expression, and feeling). if it increases the probability of satisfaction
or dissatisfaction toward a major concern of
EMOTIONS HAVE RELEVANT OBJECTS the individual. In parallel with a concern-
All research traditions on emotion – even based definition of relevance, some theories
those that are not based on appraisal theo- have focused more on the types of appraisal
ries – point to a link between an emotion criteria that could be involved in a relevance
and the significance (also called importance dimension. For instance, Scherer (2001) has
or relevance in its broad sense) of the elicit- suggested that the novelty check, the intrin-
ing situation for the organism. For instance, sic pleasantness check, and the goal/need
as LeDoux (1989) puts it, “The core of the relevance check all contribute to relevance
emotional system is a network that evalu- detection.
ates (computes) the biological significance As the relevance of the eliciting situation
of stimuli.” Of course, it is critical to define is supposed to be a sine qua non condition of
what is meant by “significance” in the vari- emotion elicitation, understanding its deter-
ous theories that highlight this concept. minants is critical for all theories of emo-
Most theories consider at least evolution- tion. Whether a theory refers to stimulus
ary significance as a key dimension, with the significance primarily in terms of (1) plea-
notion that events that are survival related sure and arousal (e.g., Bradley, 2009); (2) bio-
in terms of promoting secure conditions or logical and evolutionary considerations (e.g.,
avoiding aversive conditions are particularly LeDoux, 1989; Öhman & Mineka, 2001);
prone to eliciting emotions (e.g., Bradley, (3) primary appraisal (e.g., Lazarus, 1991);
2009; Ekman, 1992; Frijda, 1986; Lazarus, (4) dynamics of appraisal checks (e.g.,
1991; Ohman & Mineka, 2001; Panksepp, Scherer, 2001); or (5) concerns (e.g., Fri-
1991; Sander et al., 2003). jda, 2007), all approaches seem to agree that
The notion of relevance, as it has been emotions do indeed have relevant objects:
used in various theories of emotion, not only Only those specific events that are detected
captures the dimension of evolutionary sig- as relevant elicit emotions.
MODELS OF EMOTION 23
Surprise Fear
Contempt Disgust
Figure 1.2. The seven universal facial expressions of emotion according to Matsumoto and Ekman
(2009). This figure is reproduced by permission of Oxford University Press.
“each . . . qualitatively distinct from the oth- Cacioppo et al., 2000; Kreibig, 2010; Lev-
ers” (1897, p. 645). enson, 2011; Rainville, Bechara, Naqvi, &
Theorists differ on the number and nature Damasio, 2006; see Chapter 3).
of basic emotions that they propose, but The hypothesis that basic emotions rely
typically include the following six emotions: on specific brain activities has been explic-
anger, joy, sadness, fear, disgust, and surprise itly proposed by Ekman (1999, p. 50):
(see Ortony & Turner, 1990). The emotion “There must be unique physiological pat-
of contempt is also sometimes included in terns for each emotion, and these CNS pat-
the list (see Figure 1.2; Matsumoto & Ekman, terns should be specific to these emotions
2009). not found in other mental activity.” Based
Basic emotions are typically characterized (often implicitly) on this notion, a large
in this tradition as innate, easy, categorical, body of affective neuroscience research has
and immediate (see Russell, Bachorowski, attempted to identify specific brain regions
& Fernandez-Dols, 2003). As depicted in implementing distinct basic emotions. For
Figure 1.3, the basic emotion system is instance, with respect to sadness, Panksepp
responsible for both emotion elicitation (2003) suggested the existence of a “human
(through perception and schema produc- sadness system” that is based on those struc-
tion, as well as schema evaluation based tures subserving animal separation distress
on pattern matching with an emotion (the anterior cingulate, ventral septal, and
schema database) and emotional responses dorsal preoptic areas; the bed nucleus of
(emotional responding, cognition, physiol- the stria terminalis; the dorsomedial thala-
ogy, subjective experience, and expressive mus; and the periaqueductal central gray
behavior). area of the brainstem). Various investiga-
Although both Ekman (e.g., Ekman, 1972) tions also revealed a crucial involvement of
and Izard (e.g., Izard, 1971) have strongly the insula and basal ganglia in disgust (see
focused on facial expressions of emotion (see Calder et al., 2001). With respect to fear,
Figure 1.2), the study of responses in the Mineka and Öhman (2002) proposed that
autonomic nervous system was also consid- “the amygdala seems to be the central brain
ered relatively early on as indicating the exis- area dedicated to the fear module.” Of note,
tence of basic emotions (for discussion, see according to this “fear module” model, fear
MODELS OF EMOTION 25
Emotion
Schema
Database
Emotional
Responding
Cognition
Perception Appraisal
Possible
Match Physiology
Event Scanning Behavior
Schema Schema
Triggers
Production Evaluation Subjective
Experience
No Match
Expressive
Behavior
Figure 1.3. The basic emotion system according to Matsumoto and Ekman (2009). This figure is
reproduced by permission of Oxford University Press.
elicitation would not require any cognitive On the basis of another meta-analysis,
processing, and the only emotional response Vytal and Hamann (2010) explicitly argued
that would require a cognitive appraisal that neuroimaging results support the exis-
is the feeling component (experienced tence of discrete neural correlates of basic
fear). emotions. In this meta-analysis, the authors
Neuroimaging experiments and meta- considered activation likelihood maps rep-
analyses have further promoted the notion resenting regional activity that was consis-
of dedicated brain systems for basic emo- tently associated with each basic emotion
tions. For instance, Murphy, Nimmo-Smith, (see Vytal & Hamann, 2010, p. 2870). The
and Lawrence (2003) conducted a meta- authors concluded that the most prominent
analysis that provided partial support for clusters are located as follows: (1) for hap-
affect program emotion accounts. The acti- piness, in the right superior temporal gyrus
vation distributions associated with three (BA 22) and left anterior cingulate cortex
basic emotions differed significantly, with (BA 24); (2) for sadness, in the left cau-
fear being associated with the amygdala, dis- date head, left medial frontal gyrus (BA 9),
gust with the insula and globus pallidus, and and right inferior frontal gyrus (BA 9); (3)
anger with the lateral orbitofrontal cortex. for anger, in the left inferior frontal gyrus
However, no difference was found in the (BA 47) and right parahippocampal gyrus
activation distributions for happiness and (BA 35); (4) for fear, in the bilateral amyg-
sadness. dala, right cerebellum, and right insula; and
Phan, Wager, Taylor, and Liberzon (2002) (5) for disgust, in the bilateral insula (BA
also used a meta-analytic procedure to test 47). Of course, such findings do not exclude
for differential activations as a function of the possibility of finding more fundamental
basic emotions. Although many activations dimensions or components that constitute
were widely distributed over the brain, these the various emotions and are differentially
authors noted that sadness was particularly organized for specific emotions (for discus-
associated with the subcallosal anterior cin- sion, see Hamann, 2012).
gulate cortex, that both disgust and happi- A limitation of meta-analyses is the diffi-
ness were particularly associated with the culty in distinguishing among the emotional
basal ganglia, and that fear was particularly components in the series of studies included;
associated with the amygdala, but no evi- this limitation corresponds to the amalga-
dence was found for anger. mation between what is presented to the
26 DAVID SANDER
participant and the various components. For tial response pattern based on a series of
instance, when a participant is presented appraisal outcomes (see Sander, Grandjean,
with an angry face or an angry voice, one Kaiser, Wehrle, & Scherer, 2007).
can wonder whether it is the neural basis From another perspective – strongly con-
of anger or mainly the neural basis of fear trasting with the basic emotion theory – is
(as elicited by being confronted by an angry the proposal that specific emotions often
person) that is revealed. do not cause expressions; expressions are,
In addition, because most studies have rather, social messages directed to the
focused on the perception of facial expres- receiver that do not necessarily correlate
sions of emotion, very little is known about with specific emotions (see Russell et al.,
the brain mechanisms responsible for the 2003). This approach, which emphasizes
production of facial expressions (see Korb & the psychological construction of emotion
Sander, 2009; Morecraft, Stilwell-Morecraft, expression by relying on the existence of
& Rossing, 2004; Rinn, 1984). For instance, strong contextual and cultural effects, is par-
even the anatomical substrates of sponta- ticularly defended by scholars adopting cir-
neous versus voluntary facial expressions cumplex theories of emotion (e.g., Barrett,
remain unclear (see Korb & Sander, 2009). 2009; Russell et al., 2003; see the later section
Evidence from brain-damaged patients has “Is Emotion a Feeling?”).
suggested a double dissociation, with volun- Affective neuroscience research will be
tary expressions depending on cortical struc- critical in the future in comparing these
tures but spontaneous expressions depend- alternatives, in particular if robust meth-
ing on subcortical structures; however, other ods are developed to measure brain activ-
studies have indicated that areas of the cin- ity while participants produce expressions
gulate cortex may be relevant for sponta- of emotion. Techniques with a high tem-
neous facial movements as well (see Korb & poral resolution will certainly be particu-
Sander, 2009). larly suited for investigating the dynamics
When considering the production of an of expression production.
expression, a critical issue is to understand
what is expressed. Although this section on
Is Emotion an Action Tendency?
expression describes in detail the basic emo-
tion theory, other theories of emotion have Emotions prepare and guide actions (e.g.,
also considered the component of expres- Frijda, 1986). In his theory of emotion, Dewey
sion. For instance, researchers used predic- (1895, p. 17) considered emotions to imply
tions derived from appraisal theories of emo- “a readiness to act in certain ways” and
tion to develop an alternative approach to suggested that “anger means a tendency to
facial expression (e.g., Scherer, 1992; Smith, explode in a sudden attack, not a mere state
1989; Smith & Scott, 1997). The issue of men- of feeling.”
tal activity driving particular muscle activ- To some degree, emotion expressions can
ity was addressed by Darwin (1872/1998), be conceived of as particular types of actions
who interpreted the frown produced by the that are guided by motivations to act – with
innervation of the corrugator supercilii as a specific relational aim – on the world.
a sign of “something difficult or displeas- The emotion component that is supposed
ing encountered in a train of thought or in to serve action, in a coherent manner with
action” (p. 222; see also Pope & Smith, 1994). the expression component described earlier,
Moreover, Duchenne (1876/1990) attributed can be referred to as the “action tendency”
a special role in thought-related expressions component.
to the superior part of the musculus orbic- Arnold (1960) first explicitly used the
ularis oculi (which he called the “muscle notion of action tendency as a central ele-
of reflection”). Along the same line, it has ment of felt emotion; this concept was
been proposed that the facial expression then particularly developed in the emotion
of a given emotion expresses a differen- theory proposed by Frijda (1986, 2007) to
MODELS OF EMOTION 27
Figure 1.4. The emotion process as proposed by Frijda (2007). This figure is reproduced by permission
of Taylor & Francis.
describe the internal motive states that are the “positive versus negative” dimension.
hypothesized to underlie a felt urge, the Although such a distinction was not explicit
felt direction of that urge (e.g., toward or in the early work of Davidson, the view that
away from), and the “aboutness” of that Davidson’s model mainly reflected action
urge (Frijda, 2009b). Such action tenden- tendencies rather than the valence of feel-
cies (e.g., approach, avoidance, being with, ings has received support from the study
interrupting, dominating, submitting) are of anger. Indeed, evidence is accumulating
also thought to underlie overt behavior such that anger is an approach-related emotion
as running away or physically approaching a and that the left-sided approach system is
stimulus (Frijda, 2009b). consistently involved in anger (see Carver &
As a motivational process, an action ten- Harmon-Jones, 2009).
dency is a state of “action readiness” (see The view that these hemispheric asym-
Figure 1.4) that prepares the body to act with metry results are best accounted for as a
specific relational aims (e.g., exuberance, as function of approach and avoidance action
in joy, determines a diffuse openness to con- tendencies has been challenged with, for
tacts; hostility, as in anger, determines stop- instance, the hypothesis that a promotion
ping or hurting the antagonist; see Frijda, regulatory focus would be associated with
2009a). greater left frontal activity, whereas a pre-
Two opposing key action tendencies have vention regulatory focus would be asso-
been the focus of much research in affec- ciated with greater right frontal activity
tive neuroscience: approach versus with- (see Amodio, Shah, Sigelman, Brazy, &
drawal. As described by Gable and Harmon- Harmon-Jones, 2004). Another alternative
Jones (2008, p. 476), “approach motivation account relates the left anterior region to
refers to an urge or action tendency to go behavioral activation, independent of the
toward an object, whereas withdrawal moti- direction of behavior (approach or with-
vation refers to an urge or action tendency drawal), and the right anterior region to
to move away from an object” (see also goal-conflict-induced behavioral inhibition
the earlier section, “Approach-Related ver- (Wacker, Chavanon, Leue, & Stemmler,
sus Avoidance-Related Emotions”). 2008).
Consideration of an action tendency is It has been suggested that the distinction
extremely important conceptually because between “approach processes” and “avoid-
it allows a distinction between the “ap- ance processes” should not be confounded
proach versus avoidance” dimension and with the distinction between “positive
28 DAVID SANDER
Figure 1.6. Flow diagram involving emotion processing structures according to Damasio (1998). This
figure is reproduced by permission of Elsevier.
Of note, James himself claimed that both elicits the bodily reaction is considered
his and Lange’s theory concerned emo- instead to be limited to reflexes (for discus-
tional consciousness (rather than emotion; sion, see Lang & Bradley, 2009), instincts (for
see James, 1884, p. 516). Although most theo- discussion, see Lang, 1994, p. 217), pure sen-
retical approaches agree that the bodily reac- sory processes (see Zajonc, 1984), or seman-
tion is involved in the elicitation of a con- tic memory associations (e.g., Bower, 1981) –
scious feeling, the debate mainly concerns without room for evaluative interpretation
whether the bodily reaction is also more gen- in the elicitation process – then a substantial
erally fundamental for all other components theoretical debate remains.
of emotion (see Chapter 3). James’s approach was extremely influen-
If one accepts the view that it is the bod- tial for current research in affective neu-
ily reaction that elicits the emotion, then roscience, in particular for the notion that
one must explain what elicits the bodily bodily changes are primary to other emo-
reaction in the first place. If the mecha- tional components. This perspective has
nisms suggested to elicit the bodily reaction given rise to neo-Jamesian theories of emo-
are also those that typically elicit an emo- tion (e.g., Damasio, 1994; Prinz, 2004). For
tional response, the debate then seems to be instance, according to Damasio (1998) and
mainly a semantic one. However, if what as depicted in Figure 1.6, many brain systems
30 DAVID SANDER
are involved in the production and then the to basic emotions is not conclusive (for
representation of an “emotional body state.” reviews, see Cacioppo et al., 2000; Kreibig,
A critical role has been attributed to subcor- 2010; Levenson, 2003; Chapter 3). Most of
tical but also cortical (SI, SII, and the insular the research has compared bodily reac-
cortex) somatosensory maps as representing tions evoked for different basic emotions
the bodily response (see Chapter 9). and found that response patterns can dis-
The idea that specific emotional body criminate among those emotions, which
states could be represented in long-term is consistent with most theories of emo-
memory systems in the brain has been used tion (although not for some constructivist
as a counterargument to the classical crit- approaches to emotion such as the concep-
icisms according to which bodily feedback tual act model; Barrett, 2009). Of course,
is not sufficiently specific and, in particular, finding that two basic emotions can be dis-
is too slow to be primary to an emotional tinguished from each other by the pattern
experience. Indeed, as argued by Nieden- of bodily reaction they elicit does not con-
thal (2007), theories of embodied cognition ceptually show autonomic specificity nor
avoid such criticisms because they consider demonstrate that there are no more fun-
the modality-specific brain systems that damental dimensions underlying these dif-
store the representations of bodily reactions, ferences (e.g., the dimensions of approach
rather than possible “online” actual bodily versus avoidance could explain observed dif-
reactions. Such a notion is, for instance, ferences for anger versus fear).
central to Damasio’s model of emotion in Observing such differences is critical for
which the “as-if body loop” system imple- any functional perspective of emotion, but
ments “the simulation, in the brain’s body the question remains about what causal
maps, of a body state that is not actually mechanisms determine the (specific) bod-
taking place in the organism” (Damasio & ily reactions. For instance, both “affect pro-
Damasio, 2006, p. 19). According to these gram” and “appraisal” approaches predict
authors, such a simulation process indicates some functional specificities in the bodily
that the as-if loop system is what the mirror- reaction, but disagree on the underlying elic-
neuron system achieves conceptually. iting mechanisms.
It is worth noting that James’s definition Another consequence of James’s theory
of emotion was restricted to what he called relates to bodily feedback theories of emo-
standard emotions (e.g., surprise, curiosity, tion, in particular, the “facial feedback the-
rapture, fear, anger, lust, greed), which he ory,” according to which contracting facial
assumed to have distinct bodily expressions. muscles that are normally recruited in given
Therefore, other emotions – perhaps aesthetic, facial expressions (e.g., zygomaticus major)
epistemic, or self-reflexive emotions – do not can elicit or, at least, intensify the congruent
necessarily follow his definition. emotion (see Soussignan, 2002). This effect
This distinction raises an important impli- has been shown to occur even if participants
cation of James’s theory on bodily reactions are apparently not aware that their emotions
for current psychophysiological and affec- are under study and believe that they are
tive neuroscience research: Even according instructed to contract their muscles for other
to James, some emotions have a distinct reasons (see Soussignan, 2002).
bodily expression, whereas others do not. Indeed, a modulatory effect of such con-
This would mean that the bodily expres- tractions on emotion intensity seems to indi-
sion is primary only for some types of emo- cate that the bodily reactions are integrated
tion, which, in fact, is consistent with the in the brain during the emergence of an
basic emotion theories (see Levenson, 2011; emotional experience (Figure 1.7). Affective
see also the earlier section, “Is Emotion an neuroscience research into the role of bodily
Expression?”). reactions and representations in emotion has
Research concerning the existence of pat- considerably strengthened the explanatory
terns of bodily reactions that are specific power of embodiment and the models based
MODELS OF EMOTION 31
B D1 D2 F
happy
Enjoyment smile C E G
Figure 1.7. The Simulation of Smiles Core Model proposed by Niedenthal, Mermillod, Maringer, and
Hess (2010). This figure is reproduced by permission of Cambridge University Press.
on that concept (see, e.g., Niedenthal, 2007; For instance, Schachter and Singer (1962)
Niedenthal et al., 2010; Prinz, 2004). Fig- considered the states of physiological arousal
ure 1.7 illustrates the SIMS model proposed that are found in emotion to be general
by Niedenthal et al. (2010) for an enjoyment patterns of excitation of the sympathetic
smile, for which eye contact plays an impor- nervous system (see also Reisenzein, 2009).
tant role in leading to the recognition that When a stimulus elicits an arousal (as is typ-
the person feels happy (for details concern- ically the case during many emotions), the
ing this model, see Niedenthal et al., 2010, notion of “phasic” arousal can be applied to
p. 428). describe increases in arousal over a period
A key question concerning embodiment of a few seconds to a few minutes (Fowles,
and, more generally, the bodily reaction in 2009).
emotion concerns its very nature: Does this Although the concept of arousal is indeed
reaction correspond to specific patterns of tightly linked with bodily reactions, it is
bodily changes associated with particular in fact a much wider construct used for
situations or instead to a general arousal research not only on emotion (e.g., Bradley,
that is contextually interpreted? Jamesian 2009; Duffy, 1957; Russell & Barrett, 1999;
and basic emotion theories argue in favor Schachter & Singer, 1962) but also on atten-
of the existence of specific patterns for tion (see Robertson & Garavan, 2004), per-
some emotions (e.g., for “standard emo- formance (e.g., Yerkes & Dodson, 1908;
tions” [James, 1884] or for “basic emo- see Aston-Jones & Cohen, 2005), mem-
tions” [Levenson, 2011]). However, scholars ory (see McGaugh, 2006), and personal-
in affective neuroscience such as David- ity (e.g., Eysenck, 1967; Gray, 1987). For
son, Gray, LeDoux, and Panksepp have instance, Gray’s (1987) early model of the
argued that it is unlikely that autonomic ner- systems involved in emotion included a
vous system activity would show emotion- nonspecific arousal system (NAS), which,
specific response patterns, and so the focus together with the behavioral approach sys-
should be on research on the central ner- tem (BAS) and the behavioral inhibition sys-
vous system (see Ekman & Davidson, 1994, tem (BIS), is responsible for approach- and
p. 261). The difficulty of observing a specific “stop”-related behaviors. With respect to
bodily reaction for a specific emotion is con- personality, for instance, according to Wal-
sistent with the fact that many theories have lace, Bachorowski, & Newman (1991), the
the tendency to use the general construct of neuroticism construct can be conceived as
“arousal” to refer to the bodily reaction dur- reflecting the lability or reactivity of the non-
ing an emotion. specific arousal system proposed by Gray. As
32 DAVID SANDER
described in the next section, the usefulness ences (see Frijda, 2005) and their specific
of the notion of a general arousal system has content (for review, see Lambie & Marcel,
been strongly challenged over the years (see 2002) continue to be the subject of debate.
Neiss, 1988; Robbins, 1997), but it is still one For instance, Frijda (2005, p. 494) discussed
of the most widely used constructs in affec- the structure of emotional experience and
tive neuroscience. argued that “it generally contains con-
scious reflections of the four major noncon-
scious components of the process of emo-
Is Emotion a Feeling?
tions: affect, appraisal, action readiness, and
Until the 20th century, most theories of arousal. In addition, it may include the emo-
emotion – including those of James and tion’s felt ‘significance.” Panksepp (2005,
Wundt – were in fact theories of feeling (i.e., p. 32) considered that “primary-process
emotional experience or emotional con- affective consciousness emerges from large-
sciousness; see Reisenzein & Döring, 2009). scale neurodynamics of a variety of emo-
Many theories still equate emotion with tional systems that coordinate instinctual
feeling, but major efforts have been made to emotional actions.”
distinguish the two (see Frijda, 2007; Reisen- From another theoretical perspective,
zein & Döring, 2009; Scherer, 2005). Grandjean et al. (2008) have suggested that
Research on the feeling component has a feeling is a complex dynamic phenomenon
been considered as potentially important that implies neuronal synchronizations at
in affective neuroscience to separate the different levels; they have conceptualized
mechanisms involved in feeling from those “the process of an emergent conscious feel-
involved in other emotional components ing as the result of synchronizations of
(see, e.g., Damasio, 1998; LeDoux, 2007). different subsystems at different levels”
This distinction is consistent with various (p. 493; see also Scherer, 1984). Using a sim-
considerations that focus on the emo- ilar approach, Thagard and Aubie (2008,
tional experience rather than on other p. 811) argued that “conscious emotional
emotional reactions. As discussed earlier experience is produced by the brain as the
(see the section, “What Is an Emotion?”), result of many interacting brain areas coor-
this distinction can integrate James’s per- dinated in working memory.”
spective with other perspectives, because Some regions of the brain have been
James’s definition of emotion was, in fact, particularly linked to the feeling compo-
a definition of the emotional consciousness nent of emotion, such as the anterior cor-
(i.e., the feeling). That such an emotional tical midline structures (Heinzel, Moerth,
consciousness integrates the bodily reaction & Nothoff, 2010). Whether the amygdala is
is compatible with most current theories involved in the feeling component is a ques-
of emotion. The fact that the feeling tion that is still strongly debated (for dis-
component is necessarily conscious can be a cussion, see Feinstein, Adolphs, Damasio, &
matter of conceptual discussion (see Grand- Tranel, 2010).
jean, Sander, & Scherer, 2008), but the Most of the conceptualizations of the
terms “feeling,” “emotional consciousness,” feeling component in the literature are based
or “emotional experience” are typically on a dimensional perspective, largely on
used interchangeably in the literature. In Wundt’s model, which classifies feelings
addition, the literature tends to consider along three dimensions: (1) pleasantness–
emotional consciousness as emerging from unpleasantness, (2) excitement–inhibition,
the interaction among emotional compo- and (3) tension–relaxation (see, e.g., Wundt,
nents (typically appraisal, expression, action 1905). In a similar tradition, Osgood and col-
tendency, and the autonomic reaction) or leagues’ work on the dimensions underly-
dimensions (typically valence and arousal). ing affective meaning suggested the exis-
The relative importance of the different tence of three basic dimensions: arousal,
components in various emotional experi- valence, and potency (Osgood, May, &
MODELS OF EMOTION 33
AROUSAL Engagement
HIGH HIGH
Distress Excitement NEGATIVE POSITIVE
AFFECT AFFECT
Low Low
Depression Relaxation Negative
Positive
SLEEP Affect Disengagement Affect
Tense- Calm-
UNPLEASANT PLEASANT Tiredness Energy
Figure 1.8. Four types of affective circumplex models of emotion as described by Russell and Barrett
(2009). This figure is reproduced by permission of Oxford University Press.
Miron, 1975; see also Russell & Mehrabian, Intense debates concern whether arousal
1977). and valence should really be considered
Further research by Russell and oth- as the two major dimensions (see, e.g.,
ers concentrated on two dimensions of the Fontaine, Scherer, Roesch, & Ellsworth,
three, which led to the suggestion of a cir- 2007) and whether these dimensions should
cumplex model of affect (Russell, 1980). As be considered as unidimensional continua
depicted in Figure 1.8, various dimensions (see, e.g., Robbins, 1997, for arousal, and
have been suggested to anchor the so-called Cacioppo & Berntson, 1994, for valence).
affective circumplex. Such an approach As mentioned earlier (see the section,
reduces emotion to nonemotional elements “A Typical Definition of Emotion”), current
that can be felt on their own as distinc- leading versions of the circumplex model are
tive dimensions (see Russell, 2005). The two framed in the context of the construct called
dimensions of arousal and valence are typi- “core affect;” that is, the neurophysiologi-
cally considered constitutive of the circum- cal state that is always accessible as simply
plex model, but other related variants exist feeling good or bad, energized or enervated.
(see Figure 1.8 and Barrett & Russell, 2009). For instance, Russell (2005) argued that
Other traditions that have focused more core affect provides the emotional quality
on the physiological response than on the for any conscious state. Linking this model
feeling component have also highlighted to affective neuroscience research, Posner,
the role of arousal and valence as key Russell, and Peterson (2005) relied on Rus-
dimensions to organize the emotional poten- sell (1980)’s model in suggesting that “all
tial of stimuli (see Bradley & Lang, 2009). affective states arise from two fundamental
34 DAVID SANDER
research (see also Neiss, 1988). However, the responses in the anterior insula/operculum,
usefulness of a general arousal construct, at extending into the orbitofrontal cortex.
least as a “heuristic,” has been defended (see A few brain imaging experiments have
Robertson & Garavan, 2004), and the notion investigated the neural systems subserv-
of an arousal dimension is still central to ing the dimensions of valence and arousal
many theories of the emotional experience. with the same conceptual framework that is
The importance of the arousal construct at the origin of the circumplex model of
in affective neuroscience has been rein- emotion – that the arousal is a property of
forced by the intensive use of stimuli varying the feeling, but not of the presented stim-
in terms of subjective ratings of arousal (and ulus. Gerber et al. (2008) presented partici-
valence and domination; see Bradley & Lang, pants with faces that varied in valence and
2009). For instance, Anderson et al. (2003) arousal. Surprisingly, the authors found that
considered that an affective experience is activity in the amygdala and in the thalamus
determined by the two primary dimensions was stronger for low-arousal faces than for
of valence and arousal, and they investi- high-arousal faces. The authors interpreted
gated how the human brain processes olfac- this finding as being consistent with the
tory stimuli that vary in terms of valence fact that low-arousal faces are more ambigu-
and intensity. The authors used the term ous and salient than high-arousal faces. The
“intensity” to refer to “arousal” (Anderson authors also observed that the more nega-
& Sobel, 2003, p. 582) because they used tively valenced the faces, the more active
olfactory stimuli for which stimulus inten- the dorsal anterior cingulate cortex and the
sity appears to be strongly correlated with inferior parietal cortices.
indices of arousal (see Bensafi et al., 2002). Posner et al. (2009) presented participants
Anderson et al. (2003) showed that more with those emotion words that are typi-
intense odors activated the amygdala more, cally used in the circumplex model. These
independently of the valence of odors (i.e., authors were then able to identify regions
for both pleasant and unpleasant odors). In that correlated positively or negatively with
contrast, regions of the orbitofrontal cortex valence ratings and with arousal ratings. Of
were associated with valence, independent note, no particular relation was observed
of intensity. These findings were interpreted between activity in the amygdala and arousal
as suggesting that the affective representa- ratings.
tions of intensity and valence draw on dis- Using the same theoretical approach,
sociable neural substrates (see also Hamann, Colibazzi et al. (2010) investigated the neu-
2003, 2012). ral systems subserving valence and arousal
With a similar design – but adding the during the experience of induced emotions.
critical condition of so-called neutral olfac- In this study, rather than presenting isolated
tory stimuli – Winston, Gottfried, Kilner, faces, pictures, words, or odors, the authors
and Dolan (2005) questioned this conclusion explicitly requested participants to use a val-
by arguing that the amygdala is not sensitive idated scenario in order to feel emotions
to intensity per se, but is sensitive to inten- that, indeed, were reported by the partici-
sity only when stimuli are either positive or pants to vary as a function of felt valence
negative. Such results therefore challenged and felt intensity. The amygdala was found
the view that the amygdala is specifically to code for the arousal dimension, with more
involved in coding the intensity (or arousal) amygdala activity associated with more felt
dimension. arousal regardless of valence. Costa et al.
Although not explicitly equating inten- (2010) reported similar findings.
sity with arousal, Small et al. (2003) manip- In fact, evidence from various lines of
ulated the intensity and valence of taste; research indicates important similarities and
they observed that amygdala activity was overlaps in the brain mechanisms involved
driven by stimulus intensity regardless of in the processing of positive and nega-
valence and that variation in valence elicited tive stimuli. For instance, Leknes & Tracey
36 DAVID SANDER
consider and explain individual differences the complexity of the dichotomy issue high-
in emotion (see Chapters 22 and 24–28). lighted in the first question of Figure 1.9.
Appraisal is one type of cognition that The attribution-type of cognition here cor-
is much investigated in emotion research. responds to the interpretation of the state of
As depicted in Figure 1.9, a second type arousal in terms of knowledge. This inter-
of cognition – one that is different from pretation attributes the origin of the felt
appraisal – is often referred to in theories arousal to the appraised stimulus in context,
of emotion. First mainly considered in the which allows labeling of the felt arousal (see
framework of the so-called bifactorial theory Reisenzein, 2009). As phrased by Schachter
of emotion (Schachter & Singer, 1962), this and Singer (1962), “it is the cognition which
second type is today most evident in con- determines whether the state of physiolog-
structivist theories of emotion (e.g., Barrett, ical arousal will be labeled as ‘anger,’ ‘joy,’
2009; Russell, 2009). Although Schachter ‘fear,’ or whatever” (p. 380).
and Singer’s (1962) cognitive theory of emo- This view is a neo-Jamesian one, but
tion was based not on the typical process with a major difference in that accord-
of emotion generation, but rather on atyp- ing to James, the bodily response was spe-
ical cases in which arousal is unexplained cific for many emotions, whereas Schachter
(see Reisenzein, 1983), current constructivist instead considered a general undiffer-
theories have thought to extend this type entiated response (see Ellsworth, 1994).
of cognitive attribution and interpretation Among current constructivist approaches,
to typical cases of emotion elicitation (e.g., this notion was well captured by Posner et al.
Barrett, 2009; Russell, 2009). (2005), according to whom “the circumplex
As highlighted in the fourth question of model of affect proposes that all affective
Figure 1.9, the aim of this second type of states arise from cognitive interpretations of
cognition is to label an emotion as a func- core neural sensations that are the product
tion of both the emotional response and the of two independent neurophysiological sys-
context in which the stimulus appears. The tems” (p. 715).
emotional response corresponds mainly to Appraisal theories of emotion also
arousal in Schachter’s theory and to a blend acknowledge the importance of the context
of arousal and valence (i.e., core affect) in by considering that a stimulus is always
Russell and Barrett’s theory. appraised contextually and as a function
The two types of cognitions involved in of the currents needs, goals, and values
the process, and described in Figure 1.9, are of the individual. The appraisal process
well captured by the following example pro- is often thought to fully determine the
vided by Schachter and Singer: “Imagine a emotional response, including the feeling
man walking alone down a dark alley, a and the attached label (see Sander et
figure with a gun suddenly appears. The al., 2005). This perspective leaves room
perception-cognition ‘figure with a gun’ in for reappraisal and for interactive pro-
some fashion initiates a state of physiologi- cessing between components, without
cal arousal; this state of arousal is interpreted postulating an attribution-type of cogni-
in terms of knowledge about dark alleys and tion in addition to an appraisal-type of
guns and the state of arousal is labeled ‘fear’” cognition.
(p. 380). On the basis of philosophical approaches
The appraisal-type of cognition here cor- to emotion that have highlighted the role
responds to the perception-cognition “figure of subjective evaluation in emotion elicita-
with a gun” that initiates, in some fashion, a tion, scholars such as Arnold and Lazarus
state of physiological arousal. This fits with have pioneered the development of psy-
the proposed role of appraisal in the elic- chological models of emotion that consider
itation of an emotional response, and it is appraisal processes as the key determinants
likely that the use of the term “perception- of emotion (see contributions in Scherer
cognition” by Schachter and Singer reflects et al., 2001). Appraisal models of emotion
40 DAVID SANDER
Figure 1.10. The emotion process as proposed by Sander, Grandjean, and Scherer (2005). This figure is
reproduced by permission of Cambridge University Press.
typically consider that appraisal (1) is toward the realization of a goal” (for hap-
causal in emotion elicitation; (2) is con- piness).
stitutive of (not only antecedent to) emo- At a more dimensional or molecular level,
tion (i.e., the combination of appraisal process-oriented models have been sug-
outputs is key for emotional experience); gested to account for the functional dynam-
and (3) determines the response profiles ics of the emotion components. As an exam-
of the other components that are later ple, Figure 1.10 provides the component
integrated. process model of emotion – a functional
Because an individual’s appraisal of an architecture of emotion in which the emo-
event is a function of the specific rela- tion components and their interactions are
tion between the individual and the event, described (Scherer, 2001; see also Sander
the construct of a “core relational theme” et al., 2005).
has been proposed at a relatively molar According to this model, cognitive
level. As mentioned earlier, a core relational appraisal is seen both as a cause and as a
theme corresponds to the central person- component of emotion (on this issue, see
environment relationship that underlies Lazarus, 1991, pp. 172–74). The appraisal
each type of emotion (see e.g., Lazarus, component interacts with various cogni-
1991 pp. 121–24; Smith & Lazarus, 1990). As tive systems to drive variations in the
described in Lazarus (1991, p. 122), examples four components of the emotional response
of these themes are “a demeaning offense (autonomic physiology, action tendencies,
against me and mine” (for anger); “hav- motor expression, and subjective feeling
ing experienced an irrevocable loss” (for (see the section, “A Typical Definition of
sadness); or “making reasonable progress Emotion”).
MODELS OF EMOTION 41
In the component process model of emo- important topic of research in the field (for
tion, for instance, Scherer has proposed that a discussion, see Brosch, Pourtois, & Sander,
the appraisal dimensions (called stimulus 2010; Coppin et al., 2010; Cunningham &
evaluation checks) are organized in terms of Zelazo, 2007; Moors, 2010; Robinson, 1998).
achieving four appraisal objectives: The notion that emotional stimuli are pri-
oritized by the attention system has been
(1) How relevant is this event for me? Does supported by many research trends in affec-
it directly affect me or my social refer- tive neuroscience (see Vuilleumier, 2005;
ence group? (relevance); (2) What are the Chapter 14). Evidence suggests that the
implications or consequences of this event appraisal of relevance (rather than, e.g.,
and how do these affect my well-being and threat detection) might be critical for emo-
my immediate or long-term goals? (impli-
tional attention (e.g., Brosch, Sander, Pour-
cations); (3) How well can I cope with
or adjust to these consequences? (coping tois, & Scherer, 2008). Indeed, stimuli that
potential); (4) What is the significance of are appraised as relevant, in a potentially
this event with respect to my self-concept automatic way, would then drive attention
and to social norms and values (normative and may also modulate perception, mem-
significance (Sander et al., 2005, p. 319). ory, decision making, and moral judgment.
An in depth investigation of the “apprais-
These appraisal processes interact with ing brain” using affective neuroscience con-
many cognitive functions, and, as depicted cepts and methods would certainly allow a
in Figure 1.10, there is a bidirectional influ- new perspective to understanding the var-
ence between them. ious functions of the neural mechanisms
Electroencephalography has been used composing the emotional brain (see Brosch
to investigate the temporal dynamics of & Sander, in press). Most appraisal mod-
the appraisal processes, and results suggest els agree that relevance detection is a sine
that different appraisal checks have spe- qua non condition for emotion elicitation.
cific brain state correlates that occur rapidly. The suggestion of a process that evalu-
For instance, Grandjean and Scherer (2008) ates the affective relevance of an event is
manipulated three appraisal processes (nov- consistent with growing evidence indicating
elty, intrinsic pleasantness, and goal con- shared brain systems for the processing of
duciveness) and obtained results using topo- both pleasant and unpleasant stimuli (see
graphical and wavelet analyses suggesting Leknes & Tracey, 2008). The suggestion that
that the effects of these processes occur in the amygdala is critical for relevance detec-
sequential rather than parallel fashion. tion rather than being specific for threat pro-
These effects are predicted to differen- cessing, valence-based evaluation, or inten-
tially affect the emotional response compo- sity coding is an affective neuroscience
nents (for a review, see Scherer, 2009). For proposal that has directly linked appraisal
instance, Lanctôt and Hess (2007) found that models of emotion with brain systems
facial reactions to the intrinsic pleasantness (Sander et al., 2003).
manipulation were faster than facial reac- Evidence that the amygdala is critical for
tions to the goal conduciveness manipula- relevance detection is accumulating, with
tion. With respect to autonomic physiology, results indicating that this region is indeed
Delplanque et al. (2009) observed that ear- sensitive to novelty (e.g., Blackford et al.,
lier effects on heart rate occurred in response 2010), needs (e.g., Piech et al., 2009), goals
to novelty detection than in response to (e.g., Cunningham, Van Bavel, & Johnsen,
pleasantness manipulation. 2008), and values (e.g., Brosch, Coppin,
Most appraisal models of emotion sug- Scherer, Schwartz, & Sander, 2011). Because
gest that appraisal processes function at mul- the amygdala has also been considered in
tiple levels of processing (see Leventhal & other lines of research to be central for a fear
Scherer, 1987; Teasdale, 1999). In particular, module or for the arousal dimension, inves-
the automaticity of appraisal processes is an tigating the function of the amygdala seems
42 DAVID SANDER
Ahern, G. L., & Schwartz, G. E. (1979). Differ- nomic nervous system responses to odours:
ential lateralization for positive vs. negative The role of pleasantness and arousal. Chem-
emotion. Neuropsychologia, 17, 693–98. ical Senses, 27(8), 703–9.
Amodio, D. M., Shah, J. Y., Sigelman, J., Brazy, Berkowitz, L. (1990). On the formation and reg-
P. C., & Harmon-Jones, E. (2004). Implicit ulation of anger and aggression: A cognitive-
regulatory focus associated with resting frontal neoassociationistic analysis. American Psychol-
cortical asymmetry. Journal of Experimental ogist, 45(4), 494–503.
Social Psychology, 40, 225–32. Berridge, K. C., & Robinson, T. E. (2003). Parsing
Anders, S., Weiskopf, N., Lule, D., & Birbaumer, reward. Trends in Neurosciences, 26(9), 507–13.
N. (2004). Infrared oculography - validation Blackford, J. U., Buckholtz, J. W., Avery, S.
of a new method to monitor startle eyeblink N., & Zald, D. H. (2010) A unique role for
amplitudes during fMRI. Neuroimage, 22(2), the human amygdala in novelty detection.
767–770. Neuroimage, 50(3), 1188–93.
Anderson, A. K., Christoff, K., Stappen, I., Bower, G. H. (1981). Mood and memory. Ameri-
Panitz, D., Ghahremani. D.G., Glover, G., can Psychologist, 36, 129–48.
. . . Sobel, N. (2003). Dissociated neural repre- Bradley, M. (2009). Natural selective attention:
sentations of intensity and valence in human Orienting and emotion. Psychophysiology, 46,
olfaction. Nature Neuroscience, 6(2), 196–202. 1–11.
Anderson, A.K., & Sobel, N. (2003). Dissociating Bradley, M., & Lang, P. (2009). Eliciting stimulus
intensity from valence as sensory inputs into sets (for emotion research). In D. Sander &
emotion. Neuron, 39(4), 581–83. K. R. Scherer (Eds.), The Oxford companion to
Armony, J. L., Servan-Schreiber, D., Cohen, emotion and the affective sciences (pp.137–38).
J. D., & LeDoux, J. E. (1995). An anatomi- New York: Oxford University Press.
cally constrained neural network model of fear Brehm, J. W. (1999). The intensity of emotion.
conditioning. Behavioral Neuroscience, 109(2), Personality and Social Psychology Review, 3, 2–
246–57. 22.
Arnold, M. B. (1960). Emotion and personality. Broca, P. (1878). Anatomie comparée des circon-
New York: Columbia University Press. volutions cérébrales: Le grand lobe limbique
Aston-Jones, G., & Cohen, J. D. (2005). et la scissure limbique dans la série des mam-
An integrative theory of locus coeruleus- mifères. Revue d’Anthropologie, 7, 385–498.
norepinephrine function: Adaptive gain and Brosch, T., Coppin, G., Scherer, K. R., Schwartz,
optimal performance. Annual Review of Neu- S., & Sander, D. (2011). Generating value(s):
roscience, 28, 403–50. Psychological value hierarchies reflect context
Bar, M. (2009). A cognitive neuroscience hypoth- dependent sensitivity of the reward system.
esis of mood and depression. Trends in Cogni- Social Neuroscience, 6, 198–208.
tive Sciences, 13, 456–63. Brosch, T., Pourtois, G., & Sander, D. (2010).
Barrett, L. F. (2009). Variety is the spice of life: The perception and categorization of emo-
A psychological constructionist approach to tional stimuli: A review. Cognition and Emo-
understanding variability in emotion. Cogni- tion, 24(3), 377–400.
tion and Emotion, 23, 1284–1306. Brosch, T. & Sander, D. (in press). The apprais-
Barrett, L. F., Mesquita, B., Ochsner, K. N., & ing brain: Towards a neuro-cognitive model
Gross, J. J. (2007). The experience of emotion. of appraisal processes in emotion. Emotion
Annual Review of Psychology, 58, 373–403. Review.
Barrett, L. F., & Russell, J. A. (2009). The cir- Brosch, T., Sander, D., Pourtois, G., & Scherer,
cumplex model of affect. In D. Sander & K. K. R. (2008). Beyond fear: Rapid spatial orient-
Scherer (Eds.), The Oxford companion to emo- ing towards emotional positive stimuli. Psy-
tion and the affective sciences (pp. 85–88). New chological Science, 19(4), 362–70.
York: Oxford University Press. Cacioppo, J. T., & Berntson, G. G. (1994).
Basile, B., Mancini, F., Macaluso, E., Caltagirone, Relationship between attitudes and evalua-
C., Frackowiak, R. S., & Bozzali, M. (2011). tive space: A critical review, with emphasis
Deontological and altruistic guilt: Evidence on the separability of positive and negative
for distinct neurobiological substrates. Human substrates. Psychological Bulletin, 115, 401–23.
Brain Mapping, 32(2), 229–39. Cacioppo, J. T., Berntson, G. G., Larsen, J. T.,
Bensafi, M., Rouby, C., Farget, V., Bertrand, B., Poehlmann, K. M., & Ito, T. A. (2000). The
Vigouroux, M., & Holley, A. (2002). Auto- psychophysiology of emotion. In R. Lewis &
44 DAVID SANDER
Deonna, J. A., & Teroni, F. (2008). Differentiat- Ekman, P., Friesen, W. V., & Simons, R. C.
ing shame from guilt. Consciousness and Cog- (1985). Is the startle reaction an emotion? Jour-
nition 17(4), 1063–1400. nal of Personality and Social Psychology, 49(5),
Descartes, R. (1649). Les passions de l’âme. Paris. 1416–26.
de Sousa, R. (1987). The rationality of emotion. Ellsworth, P. (1994). William James and emotion:
Cambridge, MA: MIT Press. Is a century of fame worth a century of misun-
de Sousa, R. (2008). Epistemic feelings. In derstanding? Psychological Review, 101(2), 222–
G. Brun, U. Doguoglu, & D. Kuenzle (Eds.), 29.
Epistemology and emotions (pp. 185–204). Sur- Elster, J. (1996). Rationality and the emotions.
rey, UK: Ashgate. Economic Journal, 1386–97.
de Sousa, R. (2009). Rationality. In D. Sander & Erickson, K., & Schulkin, J. (2003). Facial expres-
K. R. Scherer (Eds.), The Oxford companion to sions of emotion: A cognitive neuroscience
emotion and the affective sciences (p. 329). New perspective. Brain and Cognition, 52(1), 52–60.
York: Oxford University Press. Eryilmaz, H., Van De Ville, D., Schwartz, S.,
Dewey. J. (1895). The theory of emotion. (2) The & Vuilleumier, P. (2011). Impact of transient
significance of emotions. Psychological Review, emotions on functional connectivity during
2, 13–32. subsequent resting state: A wavelet correla-
Dijk, E. van, & Zeelenberg, M. (2005). On tion approach. Neuroimage, 54(3), 2481–91.
the psychology of ‘if only’: Regret and the Exner, S. (1894). Entwurf zu einer physiologischen
comparison between factual and counterfac- Erklärung der psychischen Erscheinungen [Out-
tual outcomes. Organizational Behavior and line of a physiological explanation of mental
Human Decision Processes, 97(2), 152–60. phenomena]. Leipzig: Franz Deuticke.
Duchenne, G. B. A. (1990). Mécanisme Fehr, B., & Russell, J. A. (1984). Concept of
de la physionomie humaine: Ou, analyse emotion viewed from a prototype perspec-
électrophysiologique de l’expression des passions tive. Journal of Experimental Psychology: Gen-
[The mechanism of human facial expression] eral, 113, 464–86.
(R. A. Cuthbertson, Ed. & Trans.). Cam- Feinstein, J. S., Adolphs, R., Damasio, A., &
bridge: Cambridge University Press. (Original Tranel, D. (2010). The human amygdala and
work published 1876) the induction and experience of fear. Current
Duffy, E. (1934). Is emotion a mere term of con- Biology, 21(1), 34–38.
venience? Psychological Review, 41, 103–4. Fellous, J.-M., & Arbib, M.A. (2005). Who needs
Duffy, E. (1941). An explanation of “emotional” emotions? The brain meets the robot. New York:
phenomena without the use of the concept Oxford University Press.
“emotion.” Journal of General Psychology, 25, Fontaine, J. R. J. (2009). Self-reflexive emotions.
283–93. In D. Sander & K. R. Scherer (Eds.), The
Duffy, E. (1957). The psychological significance Oxford companion to emotion and the affective
of the concept of “arousal” or “activation.” Psy- sciences (pp. 357–59). New York: Oxford Uni-
chological Review, 64, 265–75. versity Press.
Eysenck, H. J. (1967). The biological basis of per- Fontaine, J. R., Scherer, K. R., Roesch, E. B., &
sonality. Springfield, IL: Charles C. Thomas. Ellsworth, P. (2007). The world of emotions
Ekman, P. (1972). Universals and cultural differ- is not two-dimensional. Psychological Science,
ences in facial expression of emotion. In J. R. 18(2), 1050–57.
Cole (Ed.), Nebraska symposium on motiva- Forgas, J. P. (2008). Affect and cognition. Perspec-
tion (pp. 207–83). Lincoln, NE: University of tives on Psychological Science, 3(2), 94–101.
Nebraska Press. Frank, R. (1988). Passions within reason. New
Ekman, P. (1992). An argument for basic emo- York: Norton.
tions. Cognition and Emotion, 6, 169–200. Fowles, D. C. (2009). Arousal. In D. Sander &
Ekman, P. (1999). Basic emotions. In T. Dalgleish K. R. Scherer (Eds.), The Oxford companion to
& M. Power (Eds.), Handbook of cognition and emotion and the affective sciences (p. 50). New
emotion (pp. 45–60). Chichester, UK: Wiley. York: Oxford University Press.
Ekman, P., & Davidson, R. J. (1994). After- Freud, S. (1953). Project for a scientific psychol-
word: Is there emotion-specific physiology? In ogy. In The standard edition of the complete
P. Ekman & R. J. Davidson (Eds.), The nature psychological works of sigmund Freud (Vol. 1).
of emotion: Fundamental questions (pp. 261–62). London: Hogarth Press. (Original work pub-
New York: Oxford University Press. lished 1895)
46 DAVID SANDER
Frijda, N. H. (1986). The emotions. Cambridge: Griffiths, P. E. (1997). What emotions really
Cambridge University Press. are: The problem of psychological categories.
Frijda, N. H. (2005). Emotion experience. Cogni- Chicago: University of Chicago Press.
tion and Emotion, 19, 473–98. Haber, S. N., & Knutson, B. (2010). The reward
Frijda, N. H. (2007). The laws of emotion. Mah- circuit: Linking primate anatomy and human
wah, NJ: Erlbaum. imaging. Neuropsychopharmacology, 35, 4–
Frijda, N. H. (2009a). Action readiness. In D. 26.
Sander & K. R. Scherer (Eds.), The Oxford Haidt, J. (2003). The moral emotions. In R. J.
companion to emotion and the affective sciences Davidson, K. R. Scherer, & H. H. Goldsmith
(p. 1). New York: Oxford University Press. (Eds.), Handbook of affective sciences (pp. 852–
Frijda, N. H. (2009b). Action tendencies. In D. 70). Oxford: Oxford University Press.
Sander & K. R. Scherer (Eds.), The Oxford Hamann, S. (2003). Nosing in on the emotional
companion to emotion and the affective sciences brain. Nature Neuroscience, 6(2), 106–8.
(pp. 1–2). New York: Oxford University Press. Hamann, S. (2012). Mapping discrete and dimen-
Gable, P. A., & Harmon-Jones, E. (2008). sional emotions onto the brain: controversies
Approach-motivated positive affect reduces and consensus. Trends in Cognitive Sciences,
breadth of attention. Psychological Science, 19, 16(9), 458–466.
476–82. Hareli, S., & Parkinson, B. (2009). Social emo-
Gainotti, G. (2000). Neuropsychological theories tions. In D. Sander & K.R. Scherer (Eds.), The
of emotion. In J. C. Borod (Ed.), The neuropsy- Oxford companion to emotion and the affective
chology of emotion (pp. 214–38). New York: sciences (pp. 374–75). New York: Oxford Uni-
Oxford University Press. versity Press.
Gendron, M., & Feldman Barrett, L. (2009). Hartley C. A., & Phelps E. A. (2010). Changing
Reconstructing the past: A century of ideas fear: The neurocircuity of emotion regulation.
about emotion in psychology. Emotion Review, Neuropsychopharmacology, 35(1), 136–46.
1(4), 316–39. Heinzel, A., Moerth, S., & Northoff, G. (2010).
Gerber, A. J., Posner, J., Gorman, D., Colibazzi, The central role of anterior cortical midline
T., Yu, S., Wang, Z., . . . Peterson, B.S. (2008). structures in emotional feeling and conscious-
An affective circumplex model of neural sys- ness. Psyche, 16(2), 23–47.
tems subserving valence, arousal, and cogni- Hilgard, E. R. (1980). The trilogy of mind: Cog-
tive overlay during the appraisal of emotional nition, affection, and conation. Journal of the
faces. Neuropsychologia 46, 2129–39. History of Behavioral Sciences, 16, 10717.
Gibson, J. (2009). Fiction and emotion. In D. Hobbes, T. (1985). Leviathan. England: Penguin
Sander & K. R. Scherer (Eds.), The Oxford Classics. (Original work published 1651)
companion to emotion and the affective sciences Irons, D. (1897). The primary emotions. Philo-
(pp. 184–85). New York: Oxford University sophical Review, 6, 626–45.
Press. Izard, C. E. (1971). The face of emotion. New York:
Grandjean, D., Sander, D., & Scherer, K. R. Appleton-Century-Crofts.
(2008). Conscious emotional experience James, W. (1884). What is an emotion? Mind, 9,
emerges as a function of multilevel, appraisal– 188–205.
driven response synchronization. Conscious- James, W. (1890). The principles of psychology.
ness & Cognition, 17(2), 484–95. New York: Dover Publications.
Grandjean, D., & Scherer, K. R. (2008). Unpack- James, W. (1894). The physical basis of emotion.
ing the cognitive architecture of emotion pro- Psychological Review, 1, 516–29.
cesses. Emotion, 8(3), 341–51. Jones, B. E. (2003). Arousal systems. Frontiers in
Gratch, J., & Marsella, S. (2005). Lessons from Bioscience, 8, 438–51.
emotion psychology for the design of lifelike Katkin, E. S., Wiens, S., & Ohman, A. (2001).
characters. Applied Artificial Intelligence, 19(3– Nonconscious fear conditioning, visceral per-
4), 215–33. ception, and the development of gut feelings.
Gray, J. A. (1987). The psychology of fear and stress. Psychological Science, 12(5), 366–70.
New York: Cambridge University Press. Killgore, W. D. S., & Yurgelun-Todd, D. A.
Greene, J. D., Sommerville, R. B., Nystrom, (2007). The right-hemisphere and valence
L. E., Darley, J. M., & Cohen, J. D. (2001). An hypotheses: Could they both be right (and
fMRI investigation of emotional engagement sometimes left)? Social, Cognitive and Affec-
in moral judgment. Science, 293, 2105–8. tive Neuroscience, 2(3), 240–50.
MODELS OF EMOTION 47
Kirman, A., Livet, P., & Teschl, M. (2010). Ratio- The classical psychologists, pp. 672–84, by B.
nality and emotions. Philosophical Transac- Rand (Ed.), 1912.
tions of the Royal Society B, 365(1538), 215– Larsen, J. T. (2007). Ambivalence. In R. F.
19. Baumeister & K. D. Vohs (Eds.), Encyclope-
Kleinginna, P. R. Jr., & Kleinginna, A. M. (1981). dia of social psychology. Thousand Oaks, CA:
A categorized list of emotion definitions with Sage.
suggestions for a consensual definition. Moti- Larsen, J. T., McGraw, A. P., & Cacioppo, J. T.
vation and Emotion, 5, 345–79. (2001). Can people feel happy and sad at the
Kluver, H., & Bucy, P. C. (1939). Preliminary same time? Journal of Personality and Social
analysis of the function of temporal lobe in Psychology, 81, 684–96.
monkeys. Archives of Neurology and Psychia- Lashley, K. S. (1938). The thalamus and emotion.
try, 42, 979–1000. Psychological Review, 45(1), 42–61.
Konstan, D. (2009). History of emotion. In D. Lazarus, R. S. (1966). Psychological stress and the
Sander & K. R. Scherer (Eds.), The Oxford coping process. New York: McGraw-Hill.
companion to emotion and the affective sciences Lazarus, R. S. (1982). Thoughts on the relations
(pp. 206–7). New York: Oxford University between emotion and cognition. American
Press. Psychologist, 37(9), 1019–24.
Korb, S., & Sander, D. (2009). Neural architec- Lazarus, R. S. (1984). On the primacy of cogni-
ture of facial expression. In D. Sander & K. R. tion. American Psychologist, 39, 124–29.
Scherer (Eds.), The Oxford companion to emo- Lazarus, R. S. (1991). Emotion and adaptation.
tion and the affective sciences (pp. 173–75). New New York: Oxford University Press.
York: Oxford University Press. LeDoux, J. E. (1989). Cognitive-emotional inter-
Kosslyn, S. M., & Koenig, O. (1992). Wet mind, actions in the brain. Cognition and Emotion, 3,
the new cognitive neuroscience. New York: Free 267–89.
Press. LeDoux, J. E. (1991). Emotion and the limbic sys-
Kreibig, S. D. (2010). Autonomic nervous system tem concept. Concepts in Neuroscience, 2, 169–
activity in emotion: A review. Biological Psy- 99.
chology, 84, 394–421. LeDoux, J. E. (1993). Cognition versus emotion,
Kreibig, S. D., Gendolla, G. H. E., & Scherer, again – this time in the brain: A response to
K. R. (2010). Psychophysiological effects of Parrott and Schulkin. Cognition and Emotion,
emotional responding to goal attainment. Bio- 7(1), 61–64.
logical Psychology, 84, 474–87. LeDoux, J. E. (1994). Emotional processing, but
Kumaran D., & Maguire E. A. (2007). Which not emotions, can occur unconsciously. In
computational mechanisms operate in the P. Ekman & R. J. Davidson (Eds.), The nature
hippocampus during novelty detection? Hip- of emotion: Fundamental questions (pp. 291–93).
pocampus, 17(9), 735–48. New York: Oxford University Press,
Lambie, J. A., & Marcel, A. J. (2002). Con- LeDoux, J. E. (1996). The emotional brain. New
sciousness and the varieties of emotion expe- York: Simon & Schuster.
rience: A theoretical framework. Psychological LeDoux, J. E. (2007). Unconscious and conscious
Review, 109(2), 219–59. contributions to the emotional and cognitive
Lanctôt, N., & Hess, U. (2007). The timing of aspects of emotions: A comment on Scherer’s
appraisals. Emotion, 7, 207–12. view of what an emotion is. Social Science
Lane, R., & Nadel, L. (2000). Cognitive neuro- Information, 46, 395–404.
science of emotion. New York: Oxford Univer- Leknes, S., & Tracey, I. (2008). A common neuro-
sity Press. biology for pain and pleasure. Nature Reviews
Lang, P. J. (1994). The varieties of emotional Neuroscience, 9, 314–20.
experience: A meditation on James–Lange Levenson, R. W. (2003). Autonomic specificity
theory. Psychological Review, 101(2), 211–21. and emotion. In R. J. Davidson, K. R. Scherer,
Lang, P., & Bradley, M. (2009). Reflexes (emo- & H. H. Goldsmith (Eds.), Handbook of affec-
tional). In D. Sander & K. R. Scherer (Eds.), tive sciences (pp. 212–24). New York: Oxford
The Oxford companion to emotion and the affec- University Press.
tive sciences (pp. 334–36). New York: Oxford Levenson, R. W. (2011). Basic emotion questions.
University Press. Emotion Review, 3, 379–86.
Lange, C. G. (1885). The mechanism of the emo- Leventhal, H., & Scherer, K. (1987). The rela-
tions (Benjamin Rand, Trans.) Reprinted from tionship of emotion to cognition: A functional
48 DAVID SANDER
the affective sciences (p. 140). New York: Parrott, W. G., & Schulkin, J. (1993b). What sort
Oxford University Press. of system could an affective system be? A
Niedenthal, P. M., Mermillod, M., Maringer, M., reply to LeDoux. Cognition & Emotion, 7, 65–
& Hess, U. (2010). The Simulation of Smiles 69.
(SIMS) model: Embodied simulation and the Parrott, W. G., & Smith, R. H. (1993). Distin-
meaning of facial expression. Behavioral and guishing the experiences of envy and jealousy.
Brain Sciences, 33(6), 417–33. Journal of Personality and Social Psychology, 64,
Nili, U., Goldberg, H., Weizman, A., & Dudai, 906–20.
Y. (2010). Fear thou not: Activity of frontal Paus, T. (2000). Functional anatomy of arousal
and temporal circuits in moments of real-life and attention systems in the human brain.
courage. Neuron, 66(6), 949–62. Progress in Brain Research, 126, 65–77.
Norris, C. J., Chen, E. E., Zhu, D. C., Small, S. Pelachaud, C. (2009). Embodied conversational
L., & Cacioppo, J. T. (2004). The interaction agent. In D. Sander & K. R. Scherer (Eds.),
of social and emotional processes in the brain. The Oxford companion to emotion and the affec-
Journal of Cognitive Neuroscience, 16, 1818–29. tive sciences (pp. 139–140). New York: Oxford
Ochsner, K. N., & Schacter, D. L. (2000). A social University Press.
cognitive neuroscience approach to emotion Peper, M., & Markowitsch, H.J. (2001). Pioneers
and memory. In J. C. Borod (Ed.), The neu- of affective neuroscience and early concepts
ropsychology of emotion (pp. 163–93). New of the emotional brain. Journal of the History
York: Oxford University Press. of the Neurosciences, 10, 58–66.
Öhman, A., & Mineka, S. (2001). Fears, phobias, Petta, P., & Gratch, J. (2009). Computational
and preparedness: Toward an evolved module models of emotion. In D. Sander & K. R.
of fear and fear learning. Psychological Review, Scherer (Eds.), The Oxford companion to emo-
108(3), 483–522. tion and the affective sciences (pp. 94–95). New
Olds, J., & Milner, P. (1954). Positive reinforce- York: Oxford University Press.
ment produced by electrical stimulation of Phan, K. L., Wager, T., Taylor, S. F., & Liberzon,
septal area and other regions of rat brain. Jour- I. (2002). Functional neuroanatomy of emo-
nal of Comparative Physiology and Psychology, tion: A meta-analysis of emotion activation
47, 419–27. studies in PET and fMRI. Neuroimage, 16(2),
Ortony, A., Clore, G. L., & Collins, A. (1988). 331–48.
The cognitive structure of emotions. New York: Phillips, A. G. (1999). “The Brain and Emotion”
Cambridge University Press. by Edmund T. Rolls. Trends in Cognitive Sci-
Ortony, A., & Turner, T. J. (1990). What’s basic ences, 3(7), 281–82.
about basic emotions? Psychological Review, Picard, R. (1997). Affective computing. Cambridge,
97, 315–31. MA: MIT Press.
Osgood, C. H., May, W. H., & Miron, M. S. Picard, R. W. (2009). Affective computing. In
(1975). Cross-cultural universals of affective D. Sander & K. R. Scherer (Eds.), The Oxford
meaning. Urbana: University of Illinois Press. companion to emotion and the affective sci-
Panksepp, J. (1991). Affective neuroscience: A ences (pp. 11–15). New York: Oxford Univer-
conceptual framework for the neurobiologi- sity Press.
cal study of emotions. In K. Strongman (Ed.), Piech, R. M., Lewis, J., Parkinson, C. H., Owen,
International reviews of emotion research (pp. A. M., Roberts, A. C., et al. (2009) Neural cor-
59–99). Chichester, UK: Wiley. relates of appetite and hunger-related evalua-
Panksepp, J. (2003). Feeling the pain of social loss. tive judgments. PLoS One, 4(8), e6581.
Science, 302, 237–39. Pope, L. K., & Smith, C. A. (1994). On the dis-
Panksepp, J. (2005). Affective consciousness: tinct meanings of smiles and frowns. Cognition
Core emotional feelings in animals and and Emotion, 8, 65–72.
humans. Consciousness and Cognition, 14(1), Posner, J., Russell, J. A., Gerber, A., Gorman, D.,
30–80. Colibazzi, T., Yu, S., . . . Peterson, B. S. (2009).
Papez, J. (1937). A proposed mechanism of emo- The neurophysiological bases of emotion: An
tion. Archives of Neurology and Psychiatry, 38, fMRI study of the affective circumplex using
725–43. emotion-denoting words. Human Brain Map-
Parrott, W. G., & Schulkin, J. (1993a). Neuropsy- ping, 30, 883–95.
chology and the cognitive nature of the emo- Posner, J., Russell, J. A., & Peterson, B. S. (2005).
tions. Cognition & Emotion, 7, 43–59. The circumplex model of affect: An inte-
50 DAVID SANDER
Sander, D., Grafman, J., & Zalla, T. (2003). The Scherer (Eds.), The Oxford companion to emo-
human amygdala: An evolved system for rele- tion and the affective sciences (pp. 45–49). New
vance detection. Reviews in the Neurosciences, York: Oxford University Press.
14(4), 303–16. Scherer, K. R., Shorr, A., & Johnstone, T. (Eds.).
Sander, D., Grandjean, D., Kaiser, S., Wehrle, (2001). Appraisal processes in emotion: Theory,
T., & Scherer, K. R. (2007). Interaction effects methods, research. New York: Oxford Univer-
of perceived gaze direction and dynamic facial sity Press.
expression: Evidence for appraisal theories of Schneirla, T. C. (1959). An evolutionary and
emotion. European Journal of Cognitive Psy- developmental theory of biphasic processes
chology, 19(3), 470–80. underlying approach and withdrawal. In
Sander, D., Grandjean, D., & Scherer, K. R. M. R. Jones (Ed.), Current theory and research
(2005). A systems approach to appraisal mech- in motivation (pp. 1–49). Lincoln: University of
anisms in emotion. Neural Networks, 18, 317– Nebraska Press.
52. Sergerie, K., Chochol, C., & Armony, J. L. (2008).
Sander, D., & Koenig, O. (2002). No inferior- The role of the amygdala in emotional pro-
ity complex in the study of emotion com- cessing: A quantitative meta-analysis of func-
plexity: A cognitive neuroscience compu- tional neuroimaging studies. Neuroscience and
tational architecture of emotion. Cognitive Biobehavioral Reviews, 32, 811–30.
Science Quarterly, 2, 249–72. Silvia, P. J. (2006a). Artistic training and inter-
Sander, D., & Scherer, K. R. (Eds.). (2009). The est in visual art: Applying the appraisal model
Oxford companion to emotion and the affective of aesthetic emotions. Empirical Studies of the
sciences. New York: Oxford University Press. Arts, 24, 139–61.
Schachter, S., & Singer, J. (1962). Cognitive, Silvia, P. J. (2006b). Exploring the psychology of
social, and physiological determinants of emo- interest. New York: Oxford University Press.
tional state. Psychological Review, 69(5), 379– Silvia, P. J. (2009). Aesthetic emotions (psycho-
399 logical perspectives). In D. Sander & K. R.
Scharpf, K. R., Wendt, J., Lotze, M., & Hamm, Scherer (Eds.), The Oxford companion to emo-
A. O. (2010). The brain’s relevance detection tion and the affective sciences (p. 9). New York:
network operates independently of stimulus Oxford University Press.
modality. Behavioural Brain Research, 210(1), Silvia, P. J. (2010). Confusion and interest: The
16–23. role of knowledge emotions in aesthetic expe-
Scherer, K. R. (1984). On the nature and rience. Psychology of Aesthetics, Creativity, and
function of emotion: A component process the Arts, 4, 75–80.
approach. In K. R. Scherer, & P. Ekman (Eds.), Simon, H. A. (1967). Motivational and emotional
Approaches to emotion (pp. 293–317). Hillsdale, controls of cognition. Psychological Review,
NJ: Erlbaum. 74(1), 29–39.
Scherer, K. R. (1992). What does facial expression Small, D. M., Gregory, M. D., Mak, Y. E., Gitel-
express? In K. Strongman (Ed.), International man, D., Mesulam, M. M., & Parrish, T.
review of studies on emotion, (Vol. 2, pp. 139– (2003). Dissociation of neural representation
65). Chichester, UK: Wiley. of intensity and affective valuation in human
Scherer, K. R. (2001). Appraisal considered as gustation. Neuron, 39, 701–11.
a process of multi-level sequential checking. Smith, C. A. (1989). Dimensions of appraisal and
In K. R. Scherer, A. Schorr, & T. Johnstone physiological response in emotion. Journal of
(Eds.), Appraisal processes in emotion: The- Personality and Social Psychology, 56, 339–53.
ory, methods, research (pp. 92–120). New York: Smith, C. A., & Kirby, L. D. (2009). Core rela-
Oxford University Press. tional themes. In D. Sander & K. R. Scherer
Scherer, K. R. (2005). What are emotions? And (Eds.), The Oxford companion to emotion and
how can they be measured? Social Science the affective sciences (pp. 104–5). New York:
Information, 44(4), 695–729. Oxford University Press.
Scherer, K. R. (2009). The dynamic architecture Smith, C. A., & Lazarus, R.S. (1990). Emo-
of emotion: Evidence for the component pro- tion and adaptation. In L. A. Pervin (ed.),
cess model. Cognition & Emotion, 23(7), 1307– Handbook of personality: Theory and research,
51. (pp. 609–37). New York: Guilford Press.
Scherer, K. R., & Ellsworth, P.C. (2009). Smith, C. A., & Scott, H. S. (1997). A com-
Appraisal theories. In D. Sander & K. R. ponential approach to the meaning of facial
52 DAVID SANDER
Wundt. W. (1905). Grundriss der Psychologie Zajonc, R. B. (1980). Feeling and thinking: Pref-
[Fundamentals of psychology] (7th rev. ed.). erences need no inferences. American Psychol-
Liepzig: Engelman. ogist, 35, 151–75.
Yerkes, R. M., & Dodson, J. D. (1908). The rela- Zajonc, R. B. (1984). On the primacy of affect.
tion of strength of stimulus to rapidity of American Psychologist, 39(2), 117–23.
habit-formation. Journal of Comparative and Zeki, S. (2001). Artistic creativity and the brain.
Neurological Psychology, 18, 459–82. Science, 293(5527), 51–52.
Section II
MEASURING EMOTIONAL
RESPONSES
CHAPTER 2
57
58 KATHERINE GARDHOUSE & ADAM K. ANDERSON
on examples from our own research that emotions, they require interpretation and
showcase the use of these response indices, categorization from the investigator. That
illustrating how objective and subjective being said, researchers strive to measure sub-
measures operate uniquely and offer oppor- jective states as objectively as possible so as
tunities to investigate the properties of emo- to assess the experiential content of another
tion from different vantage points. Finally, person’s mind with a high degree of insight
we address some limitations to measure- and consistency. Investigations carried out
ment and comment on innovative work that in affective neuroscience use methods such
is pushing research forward by combining as self-report, emotive language and expres-
measures with inventive methods. sions, descriptions of feelings and so on to
collect accounts of emotional experience.
We can record how participants feel dur-
Objective versus Subjective Measures ing experimentation and compare accounts
across experimental groups using standard-
Objective measurement is a cornerstone ized evaluations, correlate these findings
of science. Objectivity allows for precise with objective measures, and track fluctu-
measurement, analysis, and replication of ations in emotional experience to monitor
experiments to such a degree that when the inner emotional lives of our subjects.
the experimental context is duplicated, dif-
ferent investigators can observe the same
results. The science of psychology is the Objective Measures
application of the scientific method to men-
tal events such as emotion. In emotion A wide variety of objective measures
research there are many objective measures are used in the modern affective neuro-
that can be recorded in finite units and that science laboratory. Most were not avail-
require minimal interpretation on the part able or did not even exist a half-century
of the researcher. These include measures ago. The increasing quality of the tech-
of global behavior (e.g., approach, with- nology used to measure physiological reac-
drawal), skeletomotor activity (e.g., facial tions has essentially defined the field
expressions), autonomic physiology (e.g., of affective neuroscience to date. Dur-
electrodermal activity), and neurological ing episodes of emotional arousal we can
patterns (e.g., electrical current, blood oxy- monitor – via muscle and gland fluc-
gen supply). Objective measures expand our tuations – peripheral-physiological reac-
methodological horizon and assist with the tions, including heart rate, respiration, elec-
development of emotion theory, adding to tromyography (EMG) recordings during
our understanding of emotion by increas- facial expressions, transient dilation of the
ing our ability to identify and predict states. pupils, electrodermal activity through the
However, it is important to note that the skin (see Chapter 3), or even temperature
investigation will be limited if we do not changes of body parts through thermogra-
include measures of subjectivity. In this phy. We can also monitor neurological per-
regard, psychological sciences are unique in turbations using electroencephalographic
comparison to many of the “hard” sciences (EEG) rhythms and event-related potentials
in that they do not depend entirely on objec- (ERPs), magneto-encephalography (MEG),
tive measures, but measure such fanciful functional magnetic resonance imaging
matters as thoughts and emotions. (fMRI), and positron emission tomography
Subjective measures, unlike objective (PET; see Chapters 4–5). Objective mea-
measures, are not as easily quantified sures facilitate extraordinary examinations
because they consist of qualitative descrip- of emotional experience that could not be
tions collected from participants’ reflections obtained through subjective measures alone.
on their own feeling states. Because these Even when subjective reports are thor-
reports are self-reflecting inner accounts of oughly explored and divulged, the acting,
OBJECTIVE AND SUBJECTIVE MEASUREMENTS IN AFFECTIVE SCIENCE 59
feeling subject is not privy to a great deal environmental stimuli (Pieper, 1963). Using
of the underlying components comprising objective measures to investigate the pro-
a particular emotional experience. Objec- duction of emotional expressions can thus
tive measures can be instrumental in gain- shed light on the evolutionary basis of our
ing access to those components. Our lab very complex social display system.
has employed a number of objective mea- We chose to focus on fear and dis-
surement strategies to investigate the func- gust expressions because a statistical model
tional basis of emotional expression and of expression appearance (Dailey, Cottrell,
experience. Padgett, & Adolphs, 2002; Susskind, Little-
wort, Bartlett, Movellan, & Anderson, 2007)
revealed that fear and disgust have oppo-
Respiration
site shape and surface reflectance features
When our lab set out to investigate why (see Figure 2.1). Because fear expressions
specific facial expressions appear the way are associated with increases in behavioral
they do (Susskind et al., 2008), objective and neural markers of perception and atten-
measures provided important insights that tion (e.g., Anderson, Christoff, Panitz, De
would not have been accessible via sub- Rosa, & Gabrieli, 2003), it might be the
jective report alone. In particular, there is case that sensory acquisition is augmented
not a great deal that individuals can sub- during fearful events. Disgust expressions
jectively report when asked why they con- (eye closing, brow lowering), being opposite
tort their face the way they do during a in structural form to fear expressions (eye-
disgust reaction. Whether they make this lid opening, raised eyebrows; Susskind et
expression for social or other purposes, it al., 2008) and associated with sensory rejec-
is an automatic reaction about which we tion (Rozin & Fallon, 1987), may by con-
can only subjectively hypothesize. Or, like trast serve to diminish sensory exposure. To
many of our students did in response to test this hypothesis, we monitored air intake
that very question, you might just contort while participants posed fear and disgust
your face to display confusion and shrug facial expressions. We measured changes in
your shoulders. Although we produce facial nasal inspiratory capacity, one of the most
expressions all the time, they are not shaped basic and primitive forms of sensory intake
consciously. Even if there is a one-to-one (Zelano & Sobel, 2005), to test objectively
mapping between subjective experience and whether facial expressions alter the sensory
expression, which is highly debatable, sub- interface by augmenting or diminishing sen-
jective reports do not provide meaningful sory exposure.
information on the question of why expres- Participants were directed to make
sions appear as they do. facial expressions of fear and disgust
Using respiratory measures, however, while nasal respirometry, nasal temperature,
helped provide support for a theory origi- and abdominal-thoracic respiratory mea-
nally proposed by Darwin. Darwin hypoth- sures were collected during a controlled
esized that facial expressions evolved to instructed breathing cycle. We measured
regulate our perception before they served inspiratory volume by a flow meter attached
as nonverbal social communication signals to a facemask and placed a nasal ther-
(Darwin, 1872/1998). For example, although mistor under the right nostril to measure
the facial expression of fear has been temperature. Two strain gauges attached
adapted as a social signal to alert other con- around the chest and the abdomen mea-
specifics of danger, the role of fearful facial sured respiration. Expression configuration
expressions would have originally optimized had a significant effect on air intake, which
sensory exposure to the environment. If this increased linearly from disgust, to neutral,
is the case, it is likely that different facial to fear. Despite an equal duration of inspi-
expressions originally served to modify sen- ration, fear was associated with an increase
sory systems in order to control exposure to in air velocity and volume relative to neutral
60 KATHERINE GARDHOUSE & ADAM K. ANDERSON
Figure 2.1. (A) Disgust, (B) anger, (C) fear, and (D) surprise facial
expression prototypes. The facial action patterns depicted by the
arrows represent vector flow fields of the skin surface contortions
found from a mean group of each facial expression.
expressions. By contrast, disgust was associ- rience may not offer much insight into
ated with diminished air velocity and vol- how such innate physiological aspects of
ume relative to neutral. In addition to res- emotional response and display originated.
piratory measures, we also measured visual Indeed, fear and disgust are not oppo-
field size, finding an increase during fear- sites in the domain of subjective emotional
ful expressions and reduced size during dis- experience. In contrast, the objective mea-
gust expressions, as indexed by detection of sures provide insight into why these specific
eccentric targets. Taken together, these find- facial expressions are constructed in partic-
ings provide evidence of a multimodal sen- ular configurations. After completing these
sory augmentation during fear and a sensory experiments, our lab went on to explore
reduction during disgust. how these adaptive expressions have been
The overall augmentation found in inspi- adopted into the complex human socio-
ratory capacity and visual field size dur- emotional world using objective measures
ing expressions of fear compared to the to track facial muscle activity.
diminished sensory capacity associated with
disgust expressions suggests that emotional
Facial Muscle Activity
facial configurations do not only act as
communication signals but also may have This research lends support to the theory
evolved to alter sensory intake, thereby serv- that facial expressions had evolved to
ing an egocentric function regulating the regulate sensory intake (Darwin, 1872/1998;
receptors on the face. This link between Susskind et al., 2008), but then these sensory
form and function is not strongly supported gatekeepers were further adapted to serve
using subjective measures because expe- as social signals as well; the emotional states
OBJECTIVE AND SUBJECTIVE MEASUREMENTS IN AFFECTIVE SCIENCE 61
behind these expressions may also partici- ficult to determine what specific emotional
pate in complex socioemotional interactions state activates the insular cortex in response
(Ekman 1973; Marsh, Ambady, & Kleck, to a moral transgression, or any emotion for
2005). Our lab set out to address the ques- that matter.
tion of whether the complex social sense of In an attempt to directly test the relation-
morality is also based within the evolution- ship between morality and disgust we used
ary context of basic oral disgust (see Chap- measures that, unlike insular activation, are
man, Kim, Susskind, & Anderson, 2009). more uniquely correlated with the disgust
Oral disgust is a basic emotional response experience. Given our evidence for the sen-
that is found in some of the most ancient sory functional origins of facial expression
organisms on the planet. For example, sea appearance, we set out to find a precise facial
anemones will spit out toxic substances muscle or set of muscles that would respond
immediately after ingestion (Garcia & Han- uniquely during the consumption of disgust-
kins, 1975). More highly evolved animals ing substances. We used electromyography
such as rats have developed elaborate social (EMG) to assess the facial expression of par-
mechanisms to prevent ingesting dangerous ticipants while they drank unpleasant tasting
substances. If one rat is sick, other rats will (bitter, salty, sour) liquid (see Figure 2.2).
pick up the scent of recently consumed EMG provides an excellent objective mea-
foods and, by this social exchange, avoid sure because it precisely measures the mus-
that food in any future encounter (Galef, cle activity via electrodes attached to the
1985). Does such an ancient and engrained skin. We focused on measuring the acti-
response system influence our own complex vation of the levator labii muscle on the
sense of disgust in moral decision making? face, which is thought to be characteristic
Is morality based in the evolutionarily of the disgust expression (Ekman, Friesen, &
ancient precursor of the disgust response? Hager, 2002). This muscle raises the upper
Immoral acts undoubtedly elicit strong lip and is associated with nose wrinkling
negative emotions (Rozin, Lowery, Imada, that, as we discussed earlier, might have
& Haidt, 1999). Moreover, past research evolved to help reduce further chemosen-
has found that, when negative emotions sory exposure to toxic substances (Susskind
such as disgust are evoked, reactions to et al., 2008).
moral transgression are heightened (Schnall. We found that levator labii region activa-
Haidt, Clore, & Jordan, 2008). Yet does this tion was strongly correlated with responses
effect occur because disgust is a precursor to only the unpleasant tasting substances,
for responses to immoral acts, or is it just not to pleasant liquids over similar inten-
an effect of emotional arousal in general? sity, and thus it appeared to index aver-
Neurological evidence associates insular sive gustatory experience. We went on to
activation with oral disgust (Phillips et al., test levator labii activation while partici-
1997). Why not measure insular response pants viewed emotionally arousing photos
patterns during oral disgust and then com- and found that the degree of disgust to dis-
pare them with reactions to moral transgres- gusting pictures (related to contamination;
sions? Would a similar neurological pattern for example, vomit, bathroom images), but
in these two experiences inform us about not the degree of sadness to sad pictures, was
the relationship of these two phenomena? associated with activation of the levator labii
As much as these types of findings are of muscle. This objective measure proved to be
interest, they do not enlighten this particular a useful index to test basic oral disgust and
discussion in a concrete manner. The insula then further to monitor responses to more
has been associated with numerous cog- complex reactions to photographs. As such,
nitive and emotional responses, including the levator labii response demonstrated both
interoceptive awareness and anxiety, anger, specificity for disgust and continuity across
and uncertainty (e.g., Simmons, Matthews, disgust triggers, from basic distaste to more
Paulus, & Stein, 2008). At present, it is dif- complex contamination disgust.
62 KATHERINE GARDHOUSE & ADAM K. ANDERSON
(a) (b) 25
18
10
6
5
0 0
Neutral Sweet Unpleasant 0 5 10 15 20 25
Less unpleasant More unpleasant
Taste Rank of Valence Rating
(c)
Figure 2.2. (A) Mean levator labii region EMG response evoked by ingestion of neutral, sweet, and
bitter liquids. (B) Correlation between valence (positive vs. negative) ratings and levator labii region
EMG response. Points on the plot show this average EMG response by rank; higher rank indicates
greater unpleasantness. (C) The five most expressive individuals’ facial expressions were averaged to
prepare a model-generated expression during tasting neutral, sweet, and bitter solutions. The upper
lip and nose areas are highlighted to show the action of the levator labii muscle (upper lip raise and
nose wrinkle) across conditions.
Having demonstrated the reliability of mond, 2010) we examined the facial motor
the levator labii as an indicator of the dis- activity associated with violations of fairness
gust experience, we extended our research using this paradigm. We found that levator
to test sociomoral transgressions. We used labii region EMG activity was elicited sig-
the ultimatum game, which requires par- nificantly by unfair offers. Specifically, there
ticipants to split a sum of money between was a marked increase in self-reported dis-
themselves and the other player. The pro- gust to unfair offers, which was paralleled by
poser decides how she would like to divide a parametric increase in levator labii region
the money between herself and the respon- activity as offers became increasingly unfair.
der. If the responder rejects an offer, then Furthermore, both the degree of experi-
neither player gets any money. Players typ- enced disgust and the corresponding leva-
ically reject offers of less than 30%. One tor labii activity predicted rejection of unfair
possible explanation is that we all have a offers, suggesting that our sense of morality
strong sense of fairness (Rabin, 1993). It feels and moral behavior has a basis in disgust.
unfair to offer the other player no money, However, other research examining ex-
but our self-interest ensures that we keep a pression recognition has proposed that there
bit more than half. Likewise it seems unfair are three subtypes of the disgust facial
if we receive much less than half, and there- expression: one in response to oral disgust,
fore some moral principle causes the respon- the second to nasal disgust, and the third
der to reject unfair offers even at his own to moral violations. The third is associated
expense (i.e., he receives nothing rather than strongly with the upper lip curl produced
the offered amount). Given that fairness is by the levator labii, but this expression
central to human morality (Sokol & Ham- was also judged as signaling anger (Rozin,
OBJECTIVE AND SUBJECTIVE MEASUREMENTS IN AFFECTIVE SCIENCE 63
Lowery, & Ebert, 1994). However, it is ering the mechanisms responsible for such
known that verbal reports often confuse a memory boost has been more challeng-
anger and disgust (Nabi, 2002). Therefore ing. In this research, the question being
it is important to note that, throughout asked was geared toward understanding the
our investigation we measured subjective role of the amygdala in emotional mem-
reports of disgust in response to distaste, ory enhancement. To begin our investiga-
visual disgust, and moral transgressions using tions, we relied on subjective reports of
facial expression matching rather than ver- affect (arousal and valence) and memory
bal labels. At each level of our investiga- to evaluate stimuli known to recruit amyg-
tion the subjective measures confirmed that dala activity. Although subjective measures
disgust was the emotion that participants only brought us so far in this investigation,
associated with each experience, not anger. by adding objective measures of EDA we
Moreover, the subjective reports were con- were able to gain insight regarding the role
sistent with the objective data, such that of physiological activity in relation to sub-
higher ratings of disgust correlated with lev- jective experience, autonomic activity, and
ator labii movement, whereas other emo- amygdala activation.
tional ratings did not. So why bother with Past neuroimaging research has shown
the objective measures if subjective reports that greater amygdala activation during
can confirm our findings so easily? In this emotional, relative to neutral, events is
particular case, self-reports may have been associated with enhanced episodic mem-
insufficient to make strong conclusions. Lan- ory (e.g., Adolphs, Tranel, & Buchanan,
guage can both limit and extend the mean- 2005; Kensinger & Corkin 2004). In con-
ing of a situation, and in the case of the word sidering the role of the amygdala in EEM,
“disgust,” it is not always clear whether this we wanted to use stimuli known to reli-
linguistic term incorporates other emotions, ably recruit the amygdala. We know that
such as anger, into its meaning as well (Nabi, the identification of fearful faces is dis-
2002). Thus, verbal self-report measures of rupted in patients with amygdala dam-
subjective experience alone would not be age (e.g., Adolphs, Gosselin, et al. 2005),
diagnostic of “pure” disgust. Together par- and because research has found fearful
ticipants revealed both objective and sub- faces, compared to neutral, happy, angry,
jective signs of disgust that were associated sad, or disgusted faces, to be especially
with the degree of unfairness in the Ulti- linked to amygdala activation (Anderson
matum Game. From this, we proposed that et al., 2003), we used fearful faces to test
our moral capacity is based in ancient evo- the involvement of the amygdala in EEM. If
lutionary precursors related to disgust and amygdala activation is sufficient to enhance
distaste. The objective measures allowed us episodic memory, then EEM should be
to draw a direct link between facial motor present for fearful faces relative to neu-
activity evoked by bad tastes to highly com- tral faces because only fearful faces engage
plex social and moral judgments. the amygdala. Moreover, because fear faces
have been shown to induce even greater
activation of the amygdala than negatively
Autonomic Activity
arousing scenes, we might predict even more
We used electrodermal activity (EDA) to pronounced memory for fearful faces.
assist in an investigation of the psycholog- However, we did not find that fearful
ical and physiological underpinnings of the faces were subject to reliable EEM across
emotional enhancement of memory (EEM). a variety of delays relative to aversive scenes
Although humans are able to report that (e.g., photographs of mutilated bodies, hor-
emotional events hold privileged positions rible car crashes, etc.), which were better
in memory, whether it is a graduation, a remembered (Anderson, Wais, & Gabrieli,
surprise birthday party, a wedding, or a 2006). Relative to fearful faces, aversive
birth (e.g., Berntsen & Rubin, 2004), uncov- scenes specifically enhanced the experience
64 KATHERINE GARDHOUSE & ADAM K. ANDERSON
descriptors – happy, nervous, fearful, and pants then completed an fMRI scan. During
so on – that subjects can use to indicate the scan, sad and neutral video clips were
their feelings. These checklists can be used shown and subjective measures were admin-
with different time frames in mind (e.g., istered intermittently throughout. Partici-
present, over the last week, in general) pants rated their mood after watching each
to help assess moods, attitudes, and dis- clip on a 5-point Likert-type scale (1 = not
positions, respectively. Subjective experi- sad at all; 5 = extremely sad). This proce-
ence can also be measured by asking open- dure provided us with an online index of
ended questions that allow participants to each participant’s mood and allowed us to
answer in detail using their own words track fluctuations of mood with correspond-
and descriptions through speech or writing. ing neural changes in response to the film
These open-ended questionnaires and inter- clips. Indeed sad film clips reliably induced
views (e.g., the Structured Clinical Inter- a dysphoric mood when compared to neu-
view for DSM-IV; First, Gibbon, Hilsen- ral clips. Correlation analyses of these ques-
roth, & Segal, 2004) require more time and tionnaires (BDI, BAI, and SCL-90-R) found
interpretation on the part of the investigator, that only BDI scores signaling depressive
which renders them less available to objec- traits were reliably correlated with right
tive scrutiny, but provide deeper insight to insula activation. You can see in Figure 2.3
experience. that higher BDI scores were associated with
Questionnaires (e.g., the Positive Affect greater deactivations in these regions in
Negative Affect Schedule [Watson, Clark response to sadness provocation (Farb et al.,
& Tellegen, 1988], Beck Depression Inven- 2010). Although both the experimental
tory [Beck, Steer, & Brown, 1996]) are useful group and the control group responded with
tools because they have been meticulously similar reports of sad mood to the film clips,
developed to address specific situations or those who underwent mindfulness training
groups and are used to assess a wide range demonstrated less neural reactivity to sad-
of affect-related states and traits. In con- ness provocation than the control group,
junction with other objective neural mea- in particular in cortical midline activations
sures they make for a powerful suite of tools as well as less deactivations of right mid-
to investigate affective functions. (For other insular regions related to objective body
issues related to anxiety questionnaires, see states.
Chapter 24.) The self-report indices used in the scan-
For example, when investigating the ner (i.e., sadness ratings) correlated with
influence of mindfulness meditation training activation along the posterior and ante-
on emotional experience in individuals with rior regions of the cortical midline, as well
subsyndromal complaints of depression and as in left-lateralized language and concep-
anxiety, we assessed acute emotional states tual processing centers (Farb et al., 2010).
and neural reactivity during sadness provo- These neural patterns have been previ-
cation, as well as used the Beck Depres- ously associated with increased self-focus,
sion Inventory (BDI), the Beck Anxiety cognitive elaboration, and problem solv-
Inventory (BAI), and the Symptom Check- ing that would be typical of reappraisal
list 90 Revised (SCL-90-Revised) to eval- processes, which support healthy assess-
uate individual traits of depressive affect, ment and self-reflection (Ray, Wilhelm, &
anxiety, and psychopathology, respectively Gross, 2008). The significant neural dif-
(Farb el al., 2010). Analysis of these set ferences found after mindfulness training
of questionnaires confirmed that all partic- provide support that this practice might
ipants showed moderate levels of depres- reduce rumination and cognitive reactiv-
sion and anxiety. The experimental group ity, processes that are known to predict
completed an 8-week mindfulness train- depressive relapse (Segal et al., 2006). The
ing course, whereas the control group was use of questionnaires and self-report in
waitlisted for the course, and all partici- this study provided critical information
OBJECTIVE AND SUBJECTIVE MEASUREMENTS IN AFFECTIVE SCIENCE 67
regarding the emotional state of partici- ferences between the mindfulness group
pants before and after mindfulness train- and the control group to changes in cog-
ing (see Table 2.1), and they allowed online nitive processes in response to sadness
tracking of emotional responses while the appraisals, because emotional reactivity was
participants were in the scanner. As sad- similar between groups. The subjective
ness ratings were held equal across groups, measures thus painted a more complex
it was possible to attribute neural dif- picture.
Self-Report of One’s Own Emotional When the dummy hand and their own out-
Awareness of-view hand are touched or moved, the par-
ticipant experiences an illusion of ownership
Another example of how subjective mea-
over the dummy hand (Slater, Spanlang,
sures significantly inform investigations of
Sanchez-Vives, & Blanke, 2010). Emotional
affect is the role that subjective measures
contagion also occurs unconsciously, causing
have played in the longstanding debate
humans to adopt emotional states from oth-
about the degree to which emotionally
ers without being aware of this exchange,
salient stimuli require awareness to be pro-
like when one unconsciously mimics the
cessed. At one point the prevailing notion
facial expression of someone else in pain
was that they did not (for reviews see
(Bavelas, Black, Lemery, & Mullett, 1986).
Tsuchiya, 2009). But this notion has been
These conundrums highlight the wide
challenged by reexamining the measures
variation in levels of processing that blurs
typically used to gauge awareness of a stim-
the lines between aware and unaware (see
ulus. When more sensitive measures are
also Chapter 14). The awareness of an organ-
used many participants are actually aware
ism can range from alert wakefulness, to
to some degree, for example, of fearful faces
drowsiness, sleep, dream-states, anesthetic
flashed at latencies previously thought to be
awareness, and comatose states, all of which
outside of awareness (Pessoa, Japee, Stur-
maintain some level of awareness. Measur-
man, & Ungerleider, 2006).
ing the critical point when awareness is
Until recently a stimulus was only con-
turned on is a difficult challenge. Although
sidered consciously perceived when partici-
we can use objective measures to gauge
pants were able to assess their current men-
awareness by measuring correct behavioral
tal state and report affirmatively on it. It was
responses and neurological patterns when
assumed that if there is any awareness, par-
stimuli are flashed on a screen, these mea-
ticipants should be able to introspect and
sures have led neuroimaging researchers to
provide confirmation on whether or not they
conclude that emotional faces are uncon-
have become aware of a stimulus’ presence.
sciously perceived because the amygdala is
By this definition anything that does not
activated when participants do not respond
permeate the introspective barrier, and is
behaviorally to the emotional faces pre-
thus not reported, is not consciously per-
sented (e.g., Liddell et al., 2005; Whalen,
ceived (Merikle, Smilek, & Eastwood, 2001).
1998). This research suggests that partici-
However, levels of consciousness are not so
pants are unaware of backwardly masked
cleanly divided. At what point does one
emotional faces shown for 30 ms or less. This
become aware of a stimulus in the envi-
might be considered the threshold of visual
ronment or of percolating sensations from
awareness. Further research has suggested
inside the body? Blindsight patients have no
that this is not the case, however, because
conscious experience of seeing and they do
the objective measures used were not sen-
not engage with their environment naturally
sitive enough to tease apart the aware-
(Pegna, Khateb, Lazeyras, & Seghier, 2004).
ness level of participants (Pessoa, Japee, &
Yet we know from blindsight studies that
Ungerleider, 2005; Pessoa et al., 2006).
these patients do in fact “see.” They can cor-
In one study, participants were asked to
rectly use information gained from vision
first respond objectively whether or not a
(e.g., avoid walking into an object), although
fearful face was present on the display screen
they will report that they do not see any-
(correct or incorrect behavioral response),
thing. There are also examples of when con-
and second, to subjectively rate the confi-
scious experiences of our own bodies are not
dence of their answer (Pessoa et al., 2005).
even true in reality. For example, in dummy
They found 7 of 11 participants detected
hand experiments participants wear virtual
fearful-face targets at 33 ms and 25 ms, and 2
reality goggles and stare at a dummy hand,
participants were even able to detect 17-ms
while their real hand is hidden from view.
latencies based on confidence ratings. This
OBJECTIVE AND SUBJECTIVE MEASUREMENTS IN AFFECTIVE SCIENCE 69
use of confidence ratings arguably provides associated with subjective awareness (Pessoa
a subjective index of the participants’ own et al., 2006).
assessment of their performance (Kolb & Complicating matters is research that
Braun, 1995). Participants indicated whether suggests the amygdala plays some mod-
they were guessing or not. Response con- ulatory role in awareness itself (see also
fidence and accuracy should provide an Chapter 14). For instance, we have shown
index of awareness such that higher confi- that emotionally arousing events are asso-
dence ratings should correlate with correct ciated with enhanced perceptual awareness
responses more often than with incorrect (Anderson, 2005). That is, under condi-
responses if participants have some aware- tions of reduced attention during dual tar-
ness of the face (Pessoa et al., 2006). Using get rapid serial visual presentation, there is
these measures together, a “hit” is recorded a transient “attentional blink,” lasting a few
with a correct response and high confidence, hundred milliseconds, during which incom-
and a “miss” when there is a correct response ing stimuli have difficulty reaching aware-
with low confidence. A “false alarm” refers ness. When emotionally arousing events are
to an incorrect trial with high confidence, presented within the blink period, they
and a “correct rejection” refers to an incor- tend to survive the blink (Anderson, 2005).
rect trial with low confidence (Pessoa Patients with bilateral and unilateral amyg-
et al., 2006). These can then be used to calcu- dala lesions have been shown to exhibit
late the distribution of correct and incorrect other impairments associated with amygdala
responses to both fear faces and nontarget damage, such as impaired fear conditioning
neutral faces. and reduced EEM, but they did not demon-
One extremely innovative advantage pro- strate the arousal modulation of the atten-
vided by this use of subjective measure is tional blink (Anderson & Phelps, 2001). That
that it provides a method we can use to is, emotionally arousing stimuli did not sur-
assess the content of awareness as well as the vive the blink any more than did neutral
experience of the response (Pessoa, 2008). ones. Further, it was shown when stimuli
At shorter durations (< 30 ms) participants were made more perceptually salient, they
are able to reliably detect fearful faces, and also survived the blink more readily, and
at longer presentation intervals (>30 ms) this finding was intact following amygdala
participants were able to produce accu- damage, suggesting that the amygdala plays
rate discriminations consistently. This divi- an important role in emotional salience.
sion provides a more accurate understanding Rather than concluding that the amygdala
of awareness, outlining a zone when sub- does not depend on attention or awareness
jects are objectively aware but subjectively for the extraction of emotional/motivational
unaware (Kolb & Braun, 1995). They are salience, these results indicate that feedfor-
likely able to make correct responses above ward perceptual representations arrive at
chance, but not have confidence to indicate the amygdala which in turn influence aware-
that they know the response was accurate. ness. Thus the amygdala is not just depen-
This could be referred to as the unaware per- dent on awareness but also modulates it.
ception zone (e.g., Pessoa et al., 2006), and Therefore amygdala activation may be both
it delineates a spectrum of awareness that is a cause and a consequence of awareness.
not based on an absolute threshold, dividing
awareness from unawareness (Macmillan &
Classification of Emotions
Creelman, 2005). This dissociation increases
our ability to understand different levels of With subjective indices of emotion comes
emotional processing, providing a realistic unavoidable categorization. The feeling
index of perception that is not accessible via state of an individual ultimately has to be
objective measures. Under these more rig- pegged down with some description, and
orous testing conditions, it has been shown using words to describe such complex expe-
that amygdala activation to fear faces is riences is no simple matter. For the sake
70 KATHERINE GARDHOUSE & ADAM K. ANDERSON
of scientific rigor, how do we best capture tively, whereas the withdrawal experience
and define emotional states? There are the- of fear can become more or less intense to
ories for both basic and complex categories form terror or shyness, respectively. How
of emotion. Basic emotions are considered these semantic definitions map out to match
biologically determined, whereas complex objective neurological measures is a very
emotions may be susceptible or related to complicated matter and one that we return
cultural climates. Paul Ekman (1973) uncov- to later in our discussion. Are these cate-
ered a set of universal facial expressions that gories doing a sufficient job in isolating an
correspond to the same emotional experi- experience that is a unique emotion, or are
ence across cultures and used this set to elements of the experience being lost in
establish a model for emotion classification. between categories?
His model included six basic emotion states To complicate the categorization of emo-
of anger, disgust, happiness, fear, surprise, tions further there are states and traits to
and sadness. These facial expressions pro- consider as well; time is an interesting vari-
vide an objective index of the underlying able when thinking about emotional classifi-
experiences, but only as they pertain to cation. Are you showing a momentary emo-
those six emotional states. As you are well tional state or a more prolonged mood, or
aware, we have many more emotional words are your feelings attributed to your disposi-
to describe how we feel and many more tion or personality? Should transient emo-
feeling states then just those six, but they tions be measured and regarded differently
do not all come with their own facial signa- from lasting experiences like that of depres-
ture. Some expressions too, may be adapted sion or love? Regardless of the nature of
to other feelings because of cultural norms. emotions being experienced, when it comes
Later in his career, Ekman extended the to subjective measurement of emotional
basic model to include complex emotional states, it is imperative that these details
states that do not necessarily have corre- are definitively considered and controlled
sponding facial expressions (Ekman, 1992). for in experimentation. Carefully defining
Whether complex emotions such as hatred, your region/emotion of interest and focus-
shame, anxiety, or jealousy are unique unto ing on it while eliminating and control-
themselves, or whether they are combina- ling for other possible confounds will help
tions of the six basic categories is a discussion with the murky waters of emotions and
of much interest, and choosing the appro- our corresponding terminology. Doing so
priate model to develop an experiment is can ultimately allow researchers to have
paramount. To best capture the nature of a meaningful conversations about investiga-
state many researchers opt not to use dis- tions of the same topic. For example, as
crete categories, but rather adopt dimen- is the case for disgust research, disgust and
sional approaches using ratings of arousal anger can easily be intertwined or confused.
(e.g., intensity) and valence (i.e., pleas- This is evident when following Russell’s
ant vs. unpleasant), for example (Russell, two-dimensional model of affect (Russell,
1980). Robert Plutchik developed the three- 1980) that focuses on arousal and valence.
dimensional model that combines basic and When assessed with these dimensions in
complex categories of emotion with dimen- mind, disgust and anger are very closely
sional theories of emotions. In his model related and could meet the same criterion
eight basic emotions that are evolutionarily during analysis. As a result it is important to
based and cause either approach or with- find unique markers of each in order to dis-
drawal can be influenced by different inten- sociate the two and to find ways to ensure
sities to produce the complex and primarily that participants can indicate this distinc-
human-based, emotions (Conte & Plutchik, tion accurately. Some of these considera-
1981). For example, the basic approach expe- tions reemerge in the following discussion
rience of joy can become more or less of current limitations to measurement in the
intense to form ecstasy or serenity respec- field.
OBJECTIVE AND SUBJECTIVE MEASUREMENTS IN AFFECTIVE SCIENCE 71
objective measures to increase the predic- tions (e.g., sad, happy, angry, and disgusted)
tive power of results and support relation- because it is often the case that more than
ships found between mental and neural one label applies to an emotional state,
activity. or more than one emotional state can be
described with the same label. To circum-
vent this issue, in our lab we typically do
Linguistic and Phenomenological
not ask participants to pick only one cate-
Limitations of Subjective Measures
gory to match the feeling they are experienc-
Although humans have the ability to self- ing. Instead participants are presented with
reflect, raw emotional subjectivity is ineffa- seven canonical emotional facial expressions
ble. We cannot fully describe the feeling of (disgust, fear, sadness, surprise, and happi-
grief when a loved one dies, the joy that a ness) and are asked to rate, using a Lik-
parent experiences holding his or her new- ert scale, how their own feelings matched
born child, the hurt of a break-up or divorce, each of the photographs (Chapman et al.,
or the anger of abuse. When describing all of 2009). This method helps tease apart the
the details of an emotional experience there interrelated categorical system and provides
are limitations to how effectively we can participants with a much more versatile
communicate our abstract feelings when index.
we are required to do so using concrete In addition to linguistic challenges, it is
language. Even the most carefully designed also difficult to measure the experience of
self-reports used in scientific studies reduce affect because there exist so many con-
emotional experience to finite terms. When tributing factors to any given experience.
self-reports are subjected to statistical For example, the most detailed description
analyses, the depth of the emotion being of how someone feels pain cannot ensure a
described is limited. A major concern with comprehensive understanding of that expe-
most self-report measures is the unavoidable rience of pain because it is influenced by
reliance on language (Eriksen, 1960). In using one’s own pain tolerance, past experience
semantically based tools, we are confronted of pain, expectation, attention toward the
with a convoluted and complex linguistic infliction, and distress (Moseley, 2007). Even
challenge because not only does language when two individuals share the exact same
fail to directly communicate experience experience, how they perceive and respond
but also the questions asked and answers to it will differ because of a vast num-
given can be interpreted in many ways by ber of differences in personality, history,
both the participant and the researcher. It emotional state, and the like, and these
is not always easy to express a feeling, and countless contributing influences make it
sometimes it is not even clear to the subjects impossible to know exactly how perceptions
how they themselves feel. Language has vary.
evolved to describe a vast array of specific Although subjective experience is
feeling states, but it is worth considering unique, the feelings we have are not like
whether some things cannot be said. “Any anything else we would quantitatively mea-
patch of sunlight in a wood will show you sure. It is not like anything else at all – except
something about the sun which you could that is, the next person’s subjective experi-
never get from reading books on astron- ence. It is at least similar to other subjective
omy” (C. S. Lewis, 1963, Letters to Malcolm, experiences. Humans make the assump-
p. 91). tion that others experience the world –
Emotional states are often described with have passions, motivations, thoughts, and
terminology using discrete categories (e.g., reflections – in similar ways to their own.
arousal: sleepiness vs. wide awake, valence: All humans share this thing that cannot
pleasure vs. displeasure). Difficulty arises be said, cannot be fully described, but can
in using categorical labels of specific emo- only be experienced. Humans have the
OBJECTIVE AND SUBJECTIVE MEASUREMENTS IN AFFECTIVE SCIENCE 75
ability to relate to the minds of others, lar intensity and frequency in their reports
to empathize another’s experience (see of daily emotional events as that of control
Chapter 23). We can know to some degree participants.
how it feels to be in the shoes of another. One complicated possibility is that
This adds power to subjective measures. patients with amygdala damage are describ-
Empathy allows subjective reports to have ing “as if” experiences, so that their self-
deeper insight because, although the mind report is inaccurate. They may be describ-
is not physically measurable at present, it ing their experiences in a prototypical way
is not forbidden territory because we can that reflects how they ought to feel, instead
share experiences, histories, recollections, of how they really feel, or they may lack
concerns and outlooks between minds. It is the ability to gauge the intensity of their
our hope that we can use our experiential emotions the same way as controls might.
findings in tandem with objective measures We wanted our self-report measure to have
to verify intuitions we have regarding indices that are not accessible to subjects’
the emotional state of participants with explicit knowledge of how emotions work,
fact. making it more difficult to “fake” a genuine
In our own work, we have attempted emotional experience. To do so we exam-
to find some objective measures of sub- ined the covariance structure of emotional
jective self-reports of emotional experi- experiences. Lay knowledge suggests that
ence. Given the central role of the amyg- positive and negative valence load oppo-
dala in human affective neuroscience, one sitely on a single dimension. We found, like
of the most under-investigated questions others, that positive and negative experi-
is whether amygdala lesions are associ- ences are independent dimensions (Watson,
ated with altered emotional experience. Clark, & Tellegen, 1988), and this under-
Although there is ample evidence that lying emotional covariance structure was
amygdala activation is correlated with emo- present in patients with amygdala damage
tional experience, whether the amygdala as well, suggesting intact emotional expe-
is necessary for the subjective experi- riential structure. This covariance structure
ence of emotions is unclear because amyg- may represent an objective index of subjec-
dala recruitment may represent one of tive emotional experience, which was found
many information-processing functions cor- to be intact following amygdala lesions. In a
related with emotional experience, such separate study, we also assessed objectively
as memory modulation or motivational the facial expressions of amygdala-lesioned
salience. Indeed, if emotional experience patients who displayed internally felt emo-
was substantially altered following amygdala tions on their face through a relived emo-
damage, then any effects on information tions paradigm (Anderson & Phelps, 2000).
processing, such as the emotional mod- Under those circumstances, these patients
ulation of memory and attention, could demonstrated robust and reliably decoded
and likely would be secondary to this pri- expressions of various emotions including
mary deficit. In our own examinations of fear, yet could not recognize that expres-
this issue, we assessed emotional states sion on another person’s face. Although
and traits in individuals with bilateral and more data and studies on a variety of
right or left amygdala damage, follow- patients are clearly needed on this issue,
ing unilateral temporal lobectomy (Ander- these findings suggest that the amygdala
son & Phelps, 2000). In our most detailed may be important for some aspects of emo-
examinations, we assessed subjective expe- tional awareness but not others. Getting
rience through a daily “diary” in which to the bottom of such a question takes
individuals made reports on positive, neg- creative consideration as we strive to ver-
ative, and arousing everyday experiences. ify subjective measures and ensure their
Patients with amygdala damage had simi- accuracy.
76 KATHERINE GARDHOUSE & ADAM K. ANDERSON
Bridging the Gap between Objective and temporoparietal cortex regions (Mason,
and Subjective Observations Van Horn, Wegner, Grafton, & Macrae,
2007). Although mind wandering is not
Merging Measures:
strictly speaking an affective process, it most
Neurophenomenology
certainly can involve emotional thoughts.
Research investigations in affective neu- Rumination in depression is an obvious
roscience are often designed to mini- example of when mind wandering may be
mize dependence on subjective reports by heavily riddled with affect. In addition,
attempting to control participants’ affective research by Eryilmaz and colleagues (2011)
experiences using paradigms that reliably has found that emotional arousal does in
evoke emotional states of interest. How- fact affect the default network by increasing
ever, this approach has limitations when activity of the ventral ACC, insular cortex,
we attempt to assess spontaneous emotional and their coupling. After emotion induc-
experience, and it can be too strict when tion by watching fearful, joyful, and neu-
measuring complex emotional experiences. tral movies, participants showed subsequent
There are times when subjective reports are changes in self-referential thoughts during
less informative to the paradigm, and it is rest.
entirely appropriate to manipulate and mea- But how does one measure mind wander-
sure emotional responses objectively; for ing effectively? Christoff, Gordon, Small-
example, when mapping out the receptive wood, Smith, and Schooler (2009) used sub-
field properties of amygdala neurons using jective measures in conjunction with fMRI
fearful faces as stimuli, it is not obvious from to investigate the neurophenomenology of
introspection that there is crucial impor- mind wandering and found that activation
tance in the eyes (Whalen et al., 2004) and of the default network is more pronounced
thus subjective reports lack relative input. when participants are unaware that their
But when we start to look at high-level thoughts were drifting away from a task they
emotional experiences, such as trauma or were completing while in the scanner. This
depression, too great a focus on measur- awareness of one’s own mental state is often
ing experience objectively and the need to referred to as meta-awareness (Schooler &
be completely formulaic in our assessments Schreiber, 2004). As such, when participants
can prevent us from assessing real, deeply did not have meta-awareness of the fact that
passionate experience. Subjective measures they were mind wandering (i.e., they were
can assist in allowing us more flexibility in not consciously aware that they were mind
emotional paradigms of this sort (Lutz & wandering), the default network was even
Thompson, 2003). more pronounced than when participants
An interesting challenge for affective were consciously aware of their mind wan-
neuroscience is to find new ways to integrate dering. “Experience sampling,” an online
experiential measures with objective phys- self-report regarding current mental states
iological measures. Recent research assess- (Kahneman, Krueger, Schkade, Schwarz, &
ing mind wandering integrated both metrics Stone 2004), was used to investigate the cog-
using a neurophenomenological approach. nitions of participants throughout the task.
Mind wandering is generally thought to To measure experience sampling while
consist of internally focused reflections on in the scanner, participants performed a
future endeavors and unresolved issues relatively simple go/no-go task by indicat-
(Gusnard & Raichle, 2001). A series of neu- ing when a target appeared on screen or
roimaging studies have unveiled what is not. The task chosen was previously found
referred to as the default network, which to correlate with a high degree of mind
appears to be activated when individuals let wandering (Smallwood, Baracaia, Lowe, &
their thoughts wander away from the task Obonsawin, 2003). While they were doing
at hand. The default network includes ven- this task, experience sampling was admin-
tral ACC, posterior cingulate/precuneus, istered intermittently by presenting the
OBJECTIVE AND SUBJECTIVE MEASUREMENTS IN AFFECTIVE SCIENCE 77
participants with thought probes (Antrobus, in earlier studies, as well as the default
1968). These thought probes asked partic- network during reported periods of mind
ipants to rate, on a 7-point Likert scale, wandering. The researchers argued that
whether or not they were on- or off-task designs that merge self-report and behav-
and whether they were aware that they were ioral/neurological measures will play a crit-
on- or off-task. Participants were trained to ical role in advancing our understanding
do experience sampling before the exper- of the neurophenomenology of subjectiv-
iment began, and awareness was defined ity because they provide a more direct link
as the experience of recognizing that their to experience during online measures of
thoughts were drifting away from the task brain function (Christoff et al., 2009). Future
before they were asked (Schooler, Reichle, research that investigates deeply experien-
& Halpern, 2004). If they did not recognize tial aspects of the human emotional life will
that their thoughts were drifting until the benefit from employing a network of mea-
thought probe inquired, then they could sures to provide a more in-depth under-
respond that they were unaware. Impor- standing of the neurophenomenology of
tantly, this self-report method provided a emotion.
deep relationship between neural networks
and experience because the subjective mea- Conclusion: Objective and Subjective
sures targeted mind wandering directly. The Measures
researchers correlated “off-task” and “on-
task” moments with neural activity by ana- A central challenge for theory and research
lyzing the 10-second interval immediately in emotion continues to be the development
preceding the probe. Behavioral measures of comprehensive methods that account for
were also analyzed by tracking the number the array of activity among numerous sys-
of incorrect responses on the task, which tems and, ultimately, that integrate these
was previously found to correlate with mind recorded measures. Of course, it would be
wandering (Cheyne, Carriere, & Smilek, optimal to measure every aspect of emotion.
2006). As mind wandering increases, errors Although emotional complexity currently
increase. defies exhaustive measurement, it is impera-
The use of experience sampling diverged tive that researchers not limit themselves by
from previous work in mind wandering focusing on only one type of measurement,
because most prior neuroimaging stud- because objective and subjective measures
ies indirectly equated mind wandering offer unique insight and can be very pow-
with resting periods between tasks. It was erful when used in parallel. At present, one
assumed that during task performance, par- cannot measure activity in brain structure
ticipants were less likely to let their thoughts (e.g., the amygdala), and diagnose a particu-
drift, and once the task had ended, the inci- lar affective experience. Similarly, one can-
dence of mind wandering would increase not rely solely on reports of affective experi-
(Mason et al., 2007). However, by using ence to characterize underlying neural con-
experience sampling these researchers were figurations. The challenge before affective
able to index mind wandering more accu- neuroscientists is to include and integrate
rately while it occurred during an online multiple indices from both subjective and
task, and they could even assess when objective categories of measurement to best
subjects had a meta-awareness that they capture the mental and neural bases of emo-
were off-task or not. This innovative use tional life.
of a subjective measure of meta-awareness,
in combination with behavioral and neu-
roimaging data, allowed the researchers to Outstanding Questions
identify activation of the executive net-
r Will evidence presented from fMRI data
work, which includes regions of lateral pre-
frontal cortex and dorsal ACC, not detected be able to tease apart different models of
78 KATHERINE GARDHOUSE & ADAM K. ANDERSON
emotions, including whether basic emo- tions and experience. Social, Cognitive, &
tions exist or whether they are supported Affective Neuroscience, 2, 71–72.
by overlapping systems described best by Anderson, A. K., Christoff, K., Panitz, D., De
dimensional accounts? Rosa, E., & Gabrieli, J. D. (2003). Neural cor-
r Can fMRI reveal the constituent appraisal relates of the automatic processing of threat
facial signals. Journal of Neuroscience, 23,
processes thought to underlie the gener-
5627–33.
ation of emotions?
r Anderson, A. K., & Phelps, E. A. (2000). Expres-
What possibilities do new technologies sion without recognition: Contributions of the
that record daily experiences offer future human amygdala to emotional communica-
studies of real-world emotional experi- tion. Psychological Science, 11, 106–11.
ences? Anderson, A. K., & Phelps, E. A. (2001). Lesions
r Although much of the research we dis- of the human amygdala impair enhanced per-
cussed in this chapter is based on group ception of emotionally salient events. Nature,
mean data, are there potentially more 411, 305–9.
important individual differences in emo- Anderson, A. K., Wais, P. E., & Gabrieli, J. D.
tions that have not yet been explored? (2006). Emotion enhances remembrance of
neutral events past. Proceedings of the National
How do culture and genetics interact to
Academy of Sciences, 103, 1599–1604.
make up an individual’s emotional com-
Antrobus, J. S. (1968). Information theory and
petence? stimulus-independent thought. British Journal
r In what new ways might we use computer of Psychology, 59, 423–30.
modeling to explore emotion space, and Bavelas, J. B., Black, A., Lemery, C. R., &
how will these analyses modify psycho- Mullett, J. (1986). “I show how you feel”:
logical theories of emotion? Motor mimicry as a communicative act. Jour-
nal of Personality and Social Psychology, 50 (2),
322–29.
Beck, A., Steer, R. A., & Brown, G. K. (1996).
References Manual for the Beck Anxiety Inventory-ii. San
Antonio, TX: Psychological Corporation.
Adolphs, R., Damasio, H., Tranel, D., Cooper, Bensafi, M., Rouby, C., Farget, V., Bertrand,
G., & Damasio, A. R. (2000). A role for B., Vigoroux, M., & Holley, A. (2002). Auto-
somatosensory cortices in the visual recog- nomic nervous system responses to odors: The
nition of emotion as revealed by three- role of pleasantness and arousal. Chemical
dimensional lesion mapping. Journal of Neu- Senses, 27, 703–9.
roscience, 20(7), 2683–90. Berntsen, D., & Rubin, D.C. (2004). Cultural life
Adolphs, R., Gosselin, F., Buchanan, T. W., scripts structure recall from autobiographical
Tranel, D., Schyns, P., & Damasio, A. R. memory. Memory and Cognition, 32, 427–42.
(2005). A mechanism for impaired fear recog- Blair, R. J., Morris, J. S., Frith, C. D., Per-
nition after amygdala damage. Nature, 433, rett, D. I., & Dolan, R. J. (1999). Dissociable
68–72. neural responses to facial expressions of sad-
Adolphs, R., Tranel, D., & Buchanan, T. W. ness andanger. Brain: A Journal of Neurology,
(2005a). Amygdala damage impairs emotional 122(Pt. 5), 883–93.
memory for gist but not details of complex Cacioppo, J. T., & Bernston, G. G. (1994). Rela-
stimuli. Nature Neuroscience, 8, 512–18. tionships between attitudes and evaluative
Alter, T. (2005). The knowledge argument against space: A critical review with emphasis on
physicalism. In the Internet Encyclopedia of Phi- the separability of positive and negative sub-
losophy. Retrieved from http://www.iep.utm. strates. Psychological Bulletin, 115, 401–23.
edu. Calder, A. J., Keane, J., Lawrence, A. D., &
Anderson, A. K. (2005). Affective influences on Manes, F. (2004). Impaired recognition of
the attentional dynamics supporting aware- anger following damage to the ventral stria-
ness. Journal of Experimental Psychology: Gen- tum. Brain, 127(9), 1958–69.
eral, 134(2), 258–81. Chalmers, D. J. (1995). Facing up to the prob-
Anderson, A. K. (2007). Seeing and feeling emo- lem of consciousness. Journal of Consciousness
tion: The amygdala links emotional percep- Studies, 2(3), 200–219.
OBJECTIVE AND SUBJECTIVE MEASUREMENTS IN AFFECTIVE SCIENCE 79
Chalmers, D. J. (2000). What is a neural correlate Eriksen, C. W. (1960). Discrimination and learn-
of consciousness? In T. Metzinger (Ed.), Neu- ing without awareness: A methodological
ral correlates of consciousness: Empirical and survey and evaluation. Psychological Review,
conceptual questions (pp. 17–40). Cambridge, 67(5), 279–300.
MA: MIT Press. Eryilmaz, H., Van De Ville, D., Schwartz, S.,
Chapman, H. A., Kim, D. A., Susskind, J. M., & & Vuilleumier, P. (2011). Impact of tran-
Anderson, A. K. (2009). In bad taste: Evidence sient emotions on functional connectivity dur-
for the oral origins of moral disgust. Science, ing subsequent resting state: A wavelet cor-
323(5918), 1222–26. relation approach. Neuroimage, 54(3), 2481–
Cheyne, J. A., Carriere, J., & Smilek, D. 91.
(2006). Absent-mindedness: Lapses of con- Farb, N. A., Anderson, A. K., Mayberg, H.,
scious awareness and everyday cognitive fail- Bean, J., McKeon D., & Segal, Z. V. (2010).
ures. Consciousness and Cognition: An Interna- Minding one’s emotions: Mindfulness training
tional Journal, 15(3), 578–92. alters the neural expression of sadness. Emo-
Christoff, K., Gordon, A. M., Smallwood, J., tion, 10(1), 25–34.
Smith, R., & Schooler, J. W. (2009). Expe- First, M. B., Gibbon, M., Hilsenroth, M. J., &
rience sampling during fMRI reveals default Segal, D. L. (2004). The Structured Clini-
network and executive system contributions cal Interview for DSM-IV Axis I Disorders
to mind wandering. Proceedings of the National (SCID-I) and the Structured Clinical Inter-
Academy of Sciences, 106(21), 8719–24. view for DSM-IV Axis II Disorders (SCID-II).
Conte, H. R., & Plutchik, R. (1981). A circum- In M. J. Hilsenroth, D. L. Segal, & M. Hersen
plex model for interpersonal personality traits. (Eds.), Comprehensive handbook of psycholog-
Journal of Personality and Social Psychology, ical assessment, Vol. 2: Personality assessment
40(4), 701–11. (pp. 134–43). Hoboken, NJ: Wiley.
Crick, F., & Koch, C. (1990). Towards a neurobi- Galef, B. G. (1985). Direct and indirect behav-
ological theory of consciousness. Seminars in ioral pathways to the social transmission
Neuroscience, 2, 263–75. of food avoidance: Experimental assessments
Dailey, M. N., Cottrell, G. W., Padgett, C., & and clinical applications of conditioned food
Adolphs, R. (2002). EMPATH: A neural net- aversions. Annals of the New York Academy of
work that categorizes facial expressions. Jour- Sciences, 443, 203–15.
nal of Cognitive Neuroscience, 14, 1158–73. Garcia, J., & Hankins, W.G. (1975). The evolution
Darwin, C. (1998). The expression of the emotions of bitter and the acquisition of toxiphobia. In
in man and animals. New York: Oxford Uni- D. A. Denton & J. P. Coghlan (Eds.), Olfaction
versity Press. (Original work published 1872) and taste V (pp. 39–45). New York: Academic
D’Esposito, M., Ballard, D., Aguirre, G. K., & Press.
Zarahn, E. (1998). Human prefrontal cortex is Gusnard, D. A., & Raichle, M. E. (2001). Search-
not specific for working memory: A functional ing for a baseline: Functional imaging and the
MRI study. Neuroimage, 8, 274–82. resting human brain. Nature Reviews Neuro-
Doty, R.L. (1975). An examination of relation- science, 2, 685–94.
ships between the pleasantness, intensity and Hamann, S. B., & Adolphs, R. (1999). Normal
concentration of 10 odorous stimuli. Perception recognition of emotional similarity between
and Psychophysiology, 17(5), 492–96. facial expressions following bilateral amygdala
Downing, P. E., Jiang, Y., Shuman, M., & Kan- damage. Neuropsychologia, 37(10), 1135–41.
wisher, N. (2001). A cortical area selective for Haxby, J. V., Gobbini, M. I., Furey, M. L., Ishai,
visual processing of the human body. Science, A., Schouten, J. L., & Pietrini, P. (2001). Dis-
293(5539), 2470–73. tributed and overlapping representations of
Ekman, P. (1973). Darwin and facial expression: A faces and objects in ventral temporal cortex.
century of research in review. New York: Aca- Science, 293(5539), 2425–30.
demic Press. Holender, D. (1986). Semantic activation with-
Ekman, P. (1992). An argument for basic emo- out conscious identification in dichotic listen-
tions. Cognition and Emotion, 756(3–4), 169– ing, parafoveal vision, and visual masking: A
200. survey and appraisal. Behavioral and Brain Sci-
Ekman, P., Friesen, W., & Hager, J. C. (2002). ences, 9(1), 1–66.
Facial action coding system. Salt Lake City: Kahneman D., Krueger A. B., Schkade D. A.,
Research Nexus. Schwarz, N., & Stone, A. A. (2004). A survey
80 KATHERINE GARDHOUSE & ADAM K. ANDERSON
method for characterizing daily life experi- Okuda, S., Roozendaal, B., & McGaugh, J. L.
ence: The day reconstruction method. Science, (2004). Glucocorticoid effects on object recog-
306, 1776–80. nition memory require training-associated
Kensinger, E. A., & Corkin, S. (2004). Two routes emotional arousal. Proceedings of the National
to emotional memory: Distinct neural pro- Academy of Sciences, 101, 853–58.
cesses for valence and arousal. Proceedings of Pegna, A. J., Khateb, A., Lazeyras, F., & Seghier,
the National Academy of Sciences, 101, 3310–15. M. L. (2004). Discriminating emotional faces
Kolb, F. C., & Braun, J. (1995). Blindsight in nor- without primary visual cortices involves the
mal observers. Nature, 377, 336–38. right amygdala. Nature Neuroscience, 8, 24–25.
Lang, P. J., Greenwald, M. K., Bradley, M. M., Pessoa, L. (2008). On the relationship between
& Hamm, A. O. (1993). Looking at pictures: emotion and cognition. Nature Reviews Neu-
Affective, facial, visceral and behavioral reac- roscience, 9(2), 148–58.
tions. Psychophysiology, 30, 261–73. Pessoa, L., Japee, S., Sturman, D., & Ungerleider,
Lawrence, A. D., Goerendt, I. K., & Brooks, L. (2006). Target visibility and visual aware-
D. J. (2006). Impaired recognition of facial ness modulate amygdala responses to fearful
expressions of anger in Parkinson’s disease faces. Cerebral Cortex, 16(3), 366–75.
patients acutely withdrawn from dopamine Pessoa, L., Japee, S., & Ungerleider, L. (2005).
replacement therapy. Neuropsychologia, 45(1), Visual awareness and the detection of fearful
65–74. faces. Emotion, 5(2), 243–47.
Lewis, C. S. (1963). Letters to Malcolm chiefly on Phillips, M. L., Young, A. W., Senior, C., Bram-
prayer. New York: Harcourt. mer, M., Andrews, C., Calder, A. J., . . .
Liddell, B. J., Brown, K. J., Kemp, A. H., Bar- David, A. S. (1997). A specific neural sub-
ton, M. J., Das, P., Evian, A., Williams, G., strate for perceiving facial expressions of dis-
& Williams, L. M. (2005). A direct brainstem– gust. Nature, 389(6650), 495–98.
amygdala–cortical ‘alarm’ system for sublimi- Phillips, M. L., Young, A. W., Scott, S. K.,
nal signals of fear. Neuroimage, 24(1), 235–43. Calder, A. J., Andrew, C., Giampietro, V., et
Lindquist, K. A., Wager, T. D., Kober, H., Bliss- al. (1998). Neural responses to facial and vocal
Moreau, E., & Feldman Barrett, L. (2011). The expressions of fear and disgust. Proceedings of
brain basis of emotion: A meta-analytic review. the Royal Society of London. Series B: Biological
Cambridge, MA: Harvard University Press. Sciences, 265(1408), 1809–17.
Lutz, A., & Thompson, E. (2003). Neurophe- Pieper, A. (1963). Cerebral function in infancy and
nomenology: Integrating subjective experi- childhood. New York: Consultants Bureau.
ence and brain dynamics in the neuroscience Rabin, M. (1993). Incorporating fairness into
of consciousness. Journal of Consciousness game theory and economics. American Eco-
Studies, 10, 31–52. nomic Review, 83(5), 1281–1302.
Macmillan, N. A., & Creelman, C. D. (2005). Ray, R. D., Wilhelm, F. H., & Gross, J. J. (2008).
Detection theory: A user’s guide (2nd ed.). Mah- All in the mind’s eye? Anger rumination and
wah, NJ: Erlbaum. reappraisal. Journal of Personality and Social
Marsh, A. A., Ambady, N., & Kleck, R. E. (2005). Psychology, 94, 133–45.
The effects of fear and anger facial expressions Rolls, E. T. (2001). The rules of formation of
on approach- and avoidance-related behav- the olfactory representations found in the
iors. Emotion, 5, 119–24. orbitofrontal cortex olfactory areas of pri-
Mason, M. F., Van Horn, J. D., Wegner, D. mates. Chemical Senses, 26, 595–604.
M., Grafton, S. T., & Macrae, C. N. (2007). Rozin, P., & Fallon, A. E. (1987). A perspective
Wandering minds: The default network and on disgust. Psychological Review, 94, 23–41.
stimulus-independent thought. Science, 315, Rozin, P., Lowery, L., & Ebert, R. (1994). Vari-
393–95. eties of disgust faces and the structure of dis-
Merikle, P. M., Smilek, D., & Eastwood, J. D. gust. Journal of Personality and Social Psychol-
(2001). Perception without awareness: Per- ogy, 66, 870–81.
spectives from cognitive psychology [Special Rozin, P., Lowery, L., Imada, S., & Haidt, J.
issue]. Cognition, 79(1–2), 115–34. (1999). The CAD triad hypothesis: A mapping
Moseley, G. L., (2007). Reconceptualising pain between three moral emotions (contempt,
according to its underlying biology. Physical anger, disgust) and three moral codes (com-
Therapy Review, 12, 169–78. munity, autonomy, divinity). Journal of Per-
Nabi, R. L. (2002). Cognition. Emotion, 16, 695. sonality and Social Psychology, 76, 574.
OBJECTIVE AND SUBJECTIVE MEASUREMENTS IN AFFECTIVE SCIENCE 81
A Two-Way Road
Efferent and Afferent Pathways of Autonomic
Activity in Emotion
82
A TWO-WAY ROAD 83
Anatomy and Central Control of the through the third (oculomotor), seventh
Autonomic Nervous System (facial), and ninth (glossopharangeal) cranial
nerves to synapse with postganglionic neu-
The anatomical structure of the ANS and rons in ganglia associated with the intraocu-
its central control constitute the basis for lar smooth muscles and glands of the head;
understanding autonomic effects of emo- via the tenth (vagus) nerve to ganglia asso-
tion. ciated with thoracic and abdominal organs;
and through the pelvic splanchnic nerves to
ganglia associated with pelvic organs.
Anatomy of the Autonomic Nervous
These anatomical differences, however,
System
should not be confused with differences in
To understand the functional complexity of specificity of function. For example, despite
the ANS and appreciate the basis of earlier being considerably longer than parasym-
misinterpretations of its functional architec- pathetic postganglionic neurons, sympa-
ture requires a brief overview of its con- thetic postganglionic neurons show a simi-
ceptual origins. Langley (1900) coined the lar degree of specificity and do not branch
term autonomic nervous system to refer to the in their projection course to target tissue
system of motor neurones and their axonal (Pick, 1970). Instead, both sympathetic and
connections that carry signals from the cen- parasympathetic neurons display tight target
tral nervous system (CNS) to all innervated specificity, multiply branching only when
tissues and organs, except striated muscles. close to their specific target cells (Pick, 1970).
Langley also initiated the current division As shall be seen in the section, “The Auto-
of the ANS into sympathetic, para- (mean- nomic Nervous System and Patterned Phys-
ing by the side of or alongside) sympa- iological Responses,” although use of the
thetic, and enteric components, a differenti- terms “sympathetic” and “parasympathetic”
ation that importantly was based predomi- nervous systems has utility on an anatomi-
nantly on neuroanatomical, not functional, cal basis, their use in a global sense to define
considerations. Following this innovation, sympathetic and parasympathetic functions
preganglionic cell bodies originating in the has the potential to generate misunderstand-
intermediate zone of the thoraco-lumbar ings and create a false impression of the
spinal cord (and their axonal connections) functional architecture of the ANS. It also
were assigned to the sympathetic nervous demonstrates the fallacy of using individ-
system, whereas preganglionic cell bod- ual convenience measures, such as skin con-
ies originating in the distal zones of the ductance change, as unitary indices of over-
cranio-sacral spinal cord (including brain- all sympathetic tone, which is discussed in
stem dorsal motor nucleus of the vagus, more detail later.
nucleus ambiguus, superior and inferior sali-
vary nuclei, and Edinger-Westphal nuclei
Central Autonomic Network
and their axonal connections) were assigned
to the parasympathetic nervous system. Homoeostatic control and integration across
Sympathetic and parasympathetic ner- these functionally discrete autonomic path-
vous systems are also differentiated accord- ways are supported by a hierarchy of
ing to several other anatomical features. For direct and indirect central synaptic con-
example, sympathetic preganglionic axons nections, called the central autonomic net-
are typically short and project through the work, which has been identified princi-
spinal cord ventral roots to synapse with pally through the application of modern
postganglionic neurons in either the para- or tract tracing techniques (Janig, 2006; Saper,
prevertebral ganglia or through the hypogas- 2002). Broadly, this network includes direct
tric nerves to the pelvic splanchnic gan- connections with autonomic premotor neu-
glia. Parasympathetic preganglionic axons, rons within the ventromedial and ventrolat-
by contrast, are typically long and project eral medulla, ventrolateral pons, and lateral
84 NEIL A. HARRISON, SYLVIA D. KREIBIG, & HUGO D. CRITCHLEY
functionally distinct channels have helped This architecture can be best illustrated
solidify Levenson’s conclusion (Janig, 2006) by closer inspection of a few of these discrete
and have led to a reconceptualization of the functional subunits. For example, two dis-
ANS as a system consisting of multiple func- tinct classes of sympathetic vasoconstrictor
tionally discrete pathways, rather than as a neurons have been identified. The first, sym-
bipolar system operating in an undifferen- pathetic muscle vasoconstrictor neurons, are
tiated all-or-nothing manner. Additionally, noradrenergic axons associated with small
autonomic regulation of mechanical aspects and large arteries, which play a critical role
of sensory organs, such as the eye and ear, in modulating blood pressure in response
assists attentional focusing on emotionally to ongoing environmental demands. They
salient environmental stimuli, whereas reg- are spontaneously active and under pow-
ulation of immune responses to host infec- erful inhibitory control by arterial barore-
tion or bodily injury (Tracey, 2002) impli- ceptors, shown by their rhythmic inhibition
cates the ANS in the regulation of emotional with each heart beat (Wallin & Fagius,
and behavioral sickness responses (Harrison 1988). They are activated by nociceptors
et al., 2009). on the body surface and distention-sensitive
receptors in the bladder and colon and are
inhibited by low-threshold mechanorecep-
Functional Architecture of the
tive afferents from the skin. Their activity
Autonomic Nervous System
is also modulated by breathing, but is unaf-
Empirical data from anatomical, neurophys- fected by activation of central thermorecep-
iological (Janig, 2006), and neurochemi- tors (Janig, Sundlof, & Wallin, 1983). Cuta-
cal (Gibbins, 1995) studies in animals and neous vasoconstrictor neurons, by contrast,
humans have shown that pre- and post- play a role in thermoregulation. They too
ganglionic neurons of both the sympathetic are spontaneously active; however, unlike
and parasympathetic nervous system link muscle vasoconstrictor neurons they show
together in functionally distinct pathways absent or only weak modulation by arte-
that exert precise actions on their selec- rial baroreceptors (Blumberg, Janig, Rieck-
tive target tissues. Collectively these data mann, & Szulczyk, 1980). Unlike muscle
have helped undermine a number of previ- vasoconstrictor neurons, they are inhibited
ous false assumptions; for example, that (1) by stimulation of nociceptors and other bod-
sympathetic preganglionic neurons diverge ily afferents (e.g., trigeminal nasal afferents),
widely and synapse with postganglionic neu- whereas stimulation of low-threshold skin
rons with multiple diverse functions and (2) mechanoreceptor afferents leads to their
that adrenaline and noradrenaline released excitation (Blumberg et al., 1980). These
physiologically from the adrenal medulla act neurons are not sensitive to breathing, but
to uniformly enhance the effects of sym- do show inhibition in response to warm-
pathetic postganglionic neurons. These data ing of the spinal cord (Janig, 2006), consis-
have helped dispel the belief that the sym- tent with their role in thermoregulation (see
pathetic nervous system operates in a mono- Figure 3.2).
lithic all-or-nothing fashion (Cannon, 1931) In addition to these two distinct classes
and have instead enabled its reconceptual- of sympathetic vasoconstrictor neurons, a
ization as a system composed of multiple number of other, functionally distinct sym-
discrete functional subunits, each of which pathetic non-vasoconstrictor pathways have
displays a characteristic pattern of discharge also been described. For example, two sep-
that is dependent on the structure of cen- arate sympathetic pathways to the adrenal
tral circuits and the synaptic connections medulla have been identified, one mediat-
of these circuits with different groups of ing the release of adrenaline and the other
afferent input. Taken together, this struc- the release of noradrenaline (Vollmer, 1996).
ture makes a large degree of functional speci- These pathways again show specific func-
ficity possible. tional fingerprints. For example, adrenergic
A TWO-WAY ROAD 87
BP
150 mm Hg
PHR
mechanical stimulation 5s
of nasal mucosa
Figure 3.2. Different subclasses of autonomic neurons show distinct patterns of functional reflex
activity. The figure shows changes in blood pressure and firing rate in three different types of
sympathetic preganglionic neurons across time and in response to mechanical stimulation of the nasal
mucosa (labeled at the bottom). The top row shows changes in diastolic and systolic blood pressure.
The middle row shows activity in three neuronal types – cutaneous vasoconstrictor (CVC),
inspiratory (INSP), and muscle vasoconstrictor (MVC) neurons – in an anaesthetized cat. Each
neuron can be distinguished by its distinctive discharge shape. The bottom row shows activity in the
phrenic nerve (active during inspiration). Before stimulation the CVC neuron is active in expiration
but not inspiration, the MVC neuron is active in inspiration but not expiration, and the INSP neuron
is almost silent. Noxious mechanical stimulation of the nasal mucosa can be seen to inhibit activity in
the CVC neuron, activate the MVC neuron in inspiration as well as expiration (which continues even
after nasal stimulation has ceased), and activate the INSP neuron only in inspiration. Note the
increase in blood pressure (BP) correlated with the continuous muscle vasoconstrictor discharge.
Reproduced with permission from W. Janig, The integrative action of the autonomic nervous system,
Cambridge University Press, 2006.
autonomic neurons (defined by their tar- Conversely, Stemmler has argued that
get tissue) may, on occasion, also be dif- given the specific functions of individual
ferentiated by the profile of neuropeptides emotions for human adaptation it would be
co-localized with their classical neurotrans- improbable for there not to be emotion-
mitter. This finding has led to the concept specific activation patterns (Stemmler,
of neurochemical coding of autonomic neu- 2004). Specifically, he has reasoned that
rons (Furness, Morris, Gibbins, & Costa, because emotions are associated with dis-
1989). Together these multiple, functionally tinct goals, they require a differentiated pat-
discrete pathways enable the ANS to sup- terning of autonomic activity to enable dis-
port the differing allostatic and metabolic tinct behavioral preparation and protection
requirements of individual organs across the of the organism. Further, given that the
broad repertoire of human behavior, pro- CNS is organized to produce integrated pat-
viding the hardware necessary for emotion- terns of response rather than single isolated
specific autonomic responding. changes (Hilton, 1975), it is likely that indi-
vidual physiological variables will contribute
to several such patterns; he has therefore
What Is the Relationship between stressed the importance of studying patterns
Physiological Responses and Emotion? of physiological response, rather than single
isolated variables (Stemmler, 2004).
Conceptually, the range of potential rela- Results of meta-analyses of physiological
tionships between emotion and discrete responding in emotion, however, have sug-
physiological responses extends from a tight gested an intermediate position (Cacioppo
one-to-one mapping to a null-relationship et al., 2000). Although these studies have
(i.e., complete independence; Cacioppo & identified some reliable differences in the
Tassinary, 1990). Although these extreme pattern of ANS responses to specific emo-
positions are now rarely endorsed, several tions, they have also revealed context-
contrary positions on the issue of emotion- specific effects. In other words, patterns of
specific physiological patterning are still cur- ANS activity differ not only between emo-
rently presented in the literature (Kreibig, tions but also according to the particular
2010). The diversity of these positions emotion induction paradigm used. In one
can perhaps best be appreciated through particularly influential review, Cacioppo
closer inspection of a few examples. First, et al. found that valence-specific patterning
on the basis of the great heterogeneity was more consistent than emotion-specific
of emotion-specific physiological responses patterning. In particular, negative emotions
found in meta-analytical studies (Cacioppo, were associated with greater autonomic
Berntson, & Larsen, 2000), Feldman-Barrett responses than positive emotions. However,
has argued that “it is not possible to confi- as has been recently noted (Kreibig, 2010),
dently claim that there are kinds of emotion this meta-analysis had an unequal represen-
with unique and invariant autonomic signa- tation of positive (one) and negative (five)
tures” (Feldman-Barrett, 2006: 41). Instead, emotions, which likely significantly biased
she has proposed that patterns of physio- these findings. Supporting this critique, a
logical response follow more general dimen- recent comprehensive review of the liter-
sions of threat and challenge and posi- ature on physiological responding in emo-
tive versus negative affect. Further, she has tion, which did include multiple positive
argued that because ANS activity is mobi- emotions (affection, amusement, content-
lized in response to the metabolic demands ment, happiness, joy, pleasure, pride, and
associated with actual or expected behav- surprise), also identified a greater degree
ior and because different behaviors have of emotion-specific physiological patterning
been shown to be neither emotion spe- (Kreibig, 2010). This more recent finding is
cific nor context invariant (Lang, Bradley, & also consistent with those of several other
Cuthbert, 1993), emotion-specific autono- review articles that typically acknowledge
mic patterns are therefore improbable. that discrete emotions may differ in patterns
A TWO-WAY ROAD 89
of autonomic responses evoked, even if not pupil dilation, core temperature, and gastric
on individual variables (Mauss & Robinson, rhythmicity. Given the frequently signif-
2009). icant context-specific effects on patterns
Thus, current positions range from highly of observed autonomic responses, we have
specific ANS patterning to distinct emo- also included a description of the induction
tions, through intermediate degrees of pat- paradigms used in each study discussed and
terning (in which emotion-specific pattern- their likely influence on the pattern of ANS
ing of responses is also sensitive to the responses observed. This section does not
method of emotion induction), to a dimen- attempt to provide an exhaustive review of
sional organization of autonomic patterns. the literature but rather summarizes and
Common to each of these positions, how- highlights the most salient studies.
ever, is an acknowledgment that patterns of
physiological response do show differences
Fear
across either emotional categories or emo-
tional dimensions. We now turn to review- Studies of fear have used a number of
ing the empirical evidence for patterned different fear-induction paradigms, includ-
autonomic responses between distinct emo- ing presentation of threatening pictures or
tions. film clips, personalized recall, standardized
imagery, and even real-life manipulations,
such as threat of electric shock. Over-
Patterned Autonomic Responses to all, studies on autonomic responses to fear
Distinct Emotions have identified a broad pattern of sympa-
thetic activation, including increased heart
In this section we review the literature relat- rate, myocardial contractility, vasoconstric-
ing to physiological responses to the five tion, and electrodermal activity (Kreibig,
basic emotions of fear, anger, disgust, sad- 2010). Of note, however, in contrast to
ness, and happiness, based on a recent com- responses to anger, the majority of stud-
prehensive review of the literature (Kreibig, ies investigating changes in peripheral resis-
2010). Where possible, we have attempted to tance have demonstrated a fear-associated
synthesize the literature and start each sub- reduction in peripheral vascular resistance
section with a summary of the characteris- (Stemmler, Heldmann, Pauls, & Scherer,
tic patterns of autonomic responses associ- 2001). This broad increase in cardiac and
ated with that distinct emotion. It should, electrodermal measures of sympathetic acti-
however, be noted that many of the phys- vation has also been shown to be accom-
iological responses reported here represent panied by a decrease in vagal influences
the modal findings across published stud- on the heart (Rainville, Bechara, Naqvi, &
ies. These responses are rarely unanimously Damasio, 2006) and by an increase in respi-
reported in the literature. We also discuss ratory activity associated with a decrease in
conflicting findings that prevent identifica- blood carbon dioxide levels (Kreibig, Wil-
tion of a modal response or that illumi- helm, Roth, & Gross, 2007); in particular,
nate potential influences of induction condi- the increase in respiratory rate is secondary
tions. For a discussion of response patterns to a reduction in expiration time (Etzel,
for other emotions (e.g., anxiety, embar- Johnsen, Dickerson, Tranel, & Adolphs,
rassment, amusement, joy, pride, and relief) 2006; Kreibig et al., 2007).
and a comprehensive listing of studies, see A large number of studies investigat-
Kreibig (2010). ing physiological responses to fear (and
The measures reported in the literature indeed to other emotions) have recorded
are largely based on cardiovascular, respira- only one or two channels of physiologi-
tory, and skin conductance responses. How- cal responses, typically effects on heart rate
ever, where available, we have also inte- and skin conductance responses. Most stud-
grated the smaller literature on physiological ies have reported an increase in heart rate
responses in other autonomic axes, such as and/or electrodermal activity (measured as
90 NEIL A. HARRISON, SYLVIA D. KREIBIG, & HUGO D. CRITCHLEY
an increase in evoked skin conductance expiration time (Etzel et al., 2006; Kreibig
responses, nonspecific skin conductance et al., 2007). Supporting these findings, vol-
responses, and skin conductance level), sug- umetric measures have shown an increase in
gesting a more general activation response. minute ventilation (Kreibig et al., 2007), and
However, studies that have simultaneously gas exchange analyses have found a reduc-
assessed combinations of multiple cardiovas- tion in pCO2 (Kreibig et al., 2007), both of
cular and/or cardiorespiratory parameters which are consistent with an increase in ven-
have yielded a more complete description tilation rate. Increased variability of respira-
of the pattern of fear-related physiological tory parameters, such as respiratory ampli-
responses. These studies have reported an tude, inspiratory flow rate, and pCO2 levels,
increase in vasoconstriction (measured as a has also been described (Rainville et al.,
decrease in finger temperature, finger pulse 2006).
amplitude, and/or reduced pulse transit In animal behavioral studies the concept
time) in conjunction with increases in heart of a “predatory imminence continuum” is
rate. Associated increases in blood pressure frequently used to investigate responses to
(systolic, diastolic, and mean arterial pres- threatening or fear-inducing stimuli. The
sure) have also been frequently reported imminence of fearful stimuli has been pro-
(Kreibig et al., 2007). In contrast, stud- posed to modulate consequent fight, flight,
ies measuring peripheral vascular resistance or freeze behaviors (Blanchard & Blanchard,
typically demonstrate a decrease in this mea- 1990). It is interesting therefore to explore
sure (Stemmler et al., 2001). the context in which fear was elicited in
Where measured, fear-associated increa- those human studies that reported a fear-
ses in heart rate co-occur with increases in associated decrease rather than an increase
myocardial contractility (e.g., an increase in heart rate. In several of those studies,
in ejection speed, shortened pre-ejection “real-life” fear-induction paradigms (e.g., a
period, and/or decreased left ventricular sudden and unexpected light outage) were
ejection time; Kreibig et al., 2007; Stemm- used, whereas another study used a film
ler et al., 2001). These changes in cardiac clip eliciting fear of falling as the fear-
contractility have also been associated with eliciting stimulus (Fredrickson & Levenson,
consequent changes in cardiac pump func- 1998; Stemmler & Fahrenberg, 1989). It is
tion; however, the effect of these changes on therefore possible that participants in these
stroke volume and/or cardiac output is less studies were further along the fear immi-
clear. Increased sympathetic cardiac control nence continuum, and their responses were
is also indicated by increases in P-wave and characterized more by an immobilization
decreases in T-wave amplitude (Stemmler than an active coping response associated
et al., 2001). with sympathetic inhibition. The impor-
Evidence of fear-associated vagal with- tance of considering threat imminence in
drawal has come from a number of studies studying fear responses is further supported
showing a reduction in heart rate variabil- by a recent human neuroimaging study that
ity using a range of measures; for exam- investigated the neural structures sensitive
ple, mean difference between successive RR to threat imminence in humans (Mobbs
intervals, root mean square of successive et al., 2007). This study demonstrated a
RR interval differences, and spectral respira- shift from the vmPFC to PAG activity that
tory sinus arrhythmia (Gilissen, Bakermans- may mediate the activation of inhibitory cir-
Kranenburg, van Ijzendoorn, & van der cuits on sympathetic outflow with increas-
Veer, 2008). Studies measuring both car- ing threat imminence.
diovascular and respiratory measures have
reported increases in both heart rate and res-
Anger
piratory activity to fear stimuli (Levenson,
Heider, Ekman, & Friesen, 1992; Rainville Studies investigating the pattern of physio-
et al., 2006), in particular, a reduction in logical responses to anger have used a wide
A TWO-WAY ROAD 91
rate and α-adrenergic responses in anger been reported to static pictures of core dis-
and that various subtypes of anger may gust and no change in nonspecific skin con-
exist, which are differentiated by motiva- ductance responses to core disgust-related
tional direction. film clips (Kreibig, 2010).
In contrast, autonomic responses to body-
boundary-violation disgust-inducing stim-
Disgust
uli are characterized by reduced sympa-
Influenced by the pioneering work of thetic cardiac activation (reduced heart
Paul Rozin on the variety of different rate), increased electrodermal activity, faster
forms of disgust (Rozin, Lowery, & Ebert, breathing (and reduced tidal volume), and
1994), most studies of autonomic respond- unchanged stroke volume and total periph-
ing to disgust have used stimuli that elicit eral resistance (Rohrmann & Hopp, 2008).
either core/contamination–related/ingestive Increased T-wave amplitude has also been
or extended/body-boundary-violation forms described. Several studies have addition-
of disgust. Examples of core disgust- induc- ally described an unusual temporal pattern
ing stimuli include pictures or videos of of heart rate responses, including one that
unpalatable food, maggots on food, foul showed a triphasic heart rate response with
smells, facial expressions of expelling food, early and late decelerations separated by a
and pictures of dirty toilets or cock- brief acceleration period (Winton, Clark,
roaches. Body-boundary-violation disgust- & Edelmann, 1995). Together, these find-
inducing stimuli, by contrast, include images ings suggest decreased cardiac and increased
of injections, mutilation scenes, bloody electrodermal sympathetic activity together
injuries and surgical videos. These broad with unchanged or decreased cardiac
classes of disgust-inducing stimuli have also vagal activity (Harrison, Gray, Gianaros,
been associated with characteristic facial & Critchley, 2010). Increased skin conduc-
expressions. For example, whereas core tance response for body-boundary-violation
disgust is characterized by nose wrinkle, versus core/contamination disgust has also
mouth gape, and tongue protrusion, body- been reported (Bradley, Codispoti, Cuth-
boundary-violation disgust is instead associ- bert, & Lang, 2001).
ated with upper lip retraction (Rozin et al., Interestingly, the reduction in cardiac
1994). The following discussion differenti- output seen across disgust forms distin-
ates physiological responses associated with guishes it from the other negative emo-
each of these discrete disgust forms. tions, which are typically associated with
Broadly, autonomic responses to core an increase in cardiac output. Furthermore,
disgust-inducing stimuli are character- core disgust, unlike the other negative emo-
ized by increases in both sympathetic tions, is associated with an increase in heart
and parasympathetic indices. Core disgust rate variability. Reports of disgust-associated
is associated with either increased or un- changes in other cardiovascular measures
changed heart rate, increased heart rate (e.g., systolic, diastolic, and mean blood
variability, and decreased stroke volume pressure; pre-ejection period; left ventric-
(Kreibig, 2010; Prkachin, Williams-Avery, ular ejection time; and pulse transit time),
Zwaal, & Mills, 1999). Total peripheral resis- however, appear more variable, with no dis-
tance is also raised. Effects on breathing cernible pattern emerging (Kreibig, 2010).
include an increase in respiration rate sec- Across emotion-induction paradigms (e.g.,
ondary to a reduction in inspiration time picture viewing, film clips, directed facial
and in respiratory volume (e.g., decreased action, and personalized recall), and regard-
tidal and minute volumes; Kreibig, 2010). less of disgust form, increased electroder-
This finding is noteworthy because it is mal activity (increased skin conductance
at variance with the marked increase in response and nonspecific skin conductance
inspiration time typically seen before vom- responses) is consistently reported (Lang et
iting. Of note, unchanged or sometimes al., 1993). However, there is some suggestion
decreased skin conductance responses have that the character of these electrodermal
A TWO-WAY ROAD 93
changes may vary according to the induc- sadness and sympathetic and parasympa-
tion paradigm used. For example, disgust- thetic withdrawal associated with noncry-
ing odors may be associated with longer ing sadness (Gross, Frederickson, & Lev-
duration skin conductance responses than enson, 1994). Interestingly, those studies in
observation of facial expressions of disgust which the presence or absence of crying has
(Kreibig, 2010). not been reported also suggest the existence
Given the lay association between feel- of two broad patterns of sadness-associated
ings of disgust and gastrointestinal respon- physiological activity (Kreibig, 2010). The
ses, it is surprising that few studies to first, an activating response, partially over-
date have investigated changes in gastric laps with physiological responses to crying-
contractility or electrical activity associated associated sadness and is characterized by
with elicitors of disgust. In the few stud- an increase in sympathetic cardiovascular
ies that have done so (using electrogastro- control and unchanged respiratory activity.
graphic recordings), core disgust appears to The second, a deactivating response, par-
be related either to a decrease in normogas- tially overlaps with physiological responses
tric (normal frequency of stomach electri- to noncrying sadness and is characterized
cal activity of three cycles per minute; Jok- by sympathetic withdrawal. A further char-
erst, Levine, Stern, & Koch, 1997; Stern, Jok- acteristic of deactivating/noncrying sadness
erst, Levine, & Koch, 2001) or an increase compared to all other negative emotions is a
in tachygastric responses (an increase in decrease in electrodermal activity (Christie
the rate of stomach electrical activity in & Friedman, 2004; Gross & Levenson, 1997).
the stomach to more than four cycles Physiological responses to crying-associa-
per minute; Harrison et al., 2010). In the ted sadness include a ubiquitously reported
one study comparing gastric responses to increase in heart rate (Gross et al., 1994) and
both core and body-boundary-violation dis- in skin conductance level and/or nonspecific
gust, tachygastric responses appeared to be skin conductance responses (Gross et al.,
greater to core than to body-boundary- 1994). Increased vasoconstriction (reduction
violation disgust (Harrison et al., 2010). Rel- in finger pulse amplitude and temperature)
evant to this field are a small number of and respiratory rate have also been reported
studies that have suggested an association (Gross et al., 1994). Similarly, activating
between tachygastric responses in the stom- sadness elicited by film clips, personalized
ach and feelings of nausea associated with recall, or directed facial action is typically
motion sickness (Levine, 2005), suggest- associated with an increase in heart rate, skin
ing a potential association between mecha- conductance level, and measures of vasocon-
nisms underlying the experience of nausea striction and respiratory rate (Ekman, Lev-
and core disgust. A recent study has also enson, & Friesen, 1983). Studies using per-
shown an increase in salivary inflammatory sonalized recall to induce sadness have also
cytokines in response to visual disgust stim- reported an increase in systolic and diastolic
uli (Stevenson, Hodgson, Oaten, Barouei, & blood pressure and total peripheral resis-
Case, 2011). tance (Prkachin et al., 1999). Cardiac out-
put, stroke volume, and finger temperature
(a measure of vasoconstriction) are reported
Sadness
to decrease or remain unchanged (Prkachin
Broadly, autonomic responses to sadness et al., 1999) with no discernible pattern of
demonstrate a heterogeneous pattern of effects on heart rate variability. Effects on
sympathetic and parasympathetic coactiva- respiration include an increase in respira-
tion. This pattern may be further defined tion period and respiration period variability
by subclassifying the sadness induced into (Rainville et al., 2006).
crying or noncrying forms (Kreibig, 2010). In contrast, noncrying sadness has been
Those few studies that have considered associated with a reduction in heart rate,
crying status in this way show uncoupled reduced electrodermal activity (measured as
sympathetic activation in crying-associated a reduction in skin conductance level and
94 NEIL A. HARRISON, SYLVIA D. KREIBIG, & HUGO D. CRITCHLEY
smaller nonspecific skin reaction responses), physiological responses observed for crying
reduced heart rate variability, and an and noncrying sadness. It is therefore inter-
increase in respiratory rate (Gross et al., esting to consider the contexts of sadness
1994; Rottenberg, Wilhelm, Gross, & Gotlib, induction that result in either an activat-
2003). Interestingly, the majority of stud- ing or a deactivating sadness response. In
ies using film clips, music, or standardized her recent review of this literature, Kreibig
imagery to induce sadness report a deactivat- (2010) noted a potential dissociation on the
ing sadness response. Film-clip-induced sad- basis of the imminence of the sadness-
ness is characterized by a reduction in car- associated loss. For example, the activating
diac activation and decreased electrodermal pattern typically occurs in response to film
activity; in particular, a reduction in heart clips that depict scenes related to impending
rate, increased or unchanged heart rate vari- loss (e.g., a man talking to his dying sister). In
ability, decreased or unchanged diastolic and contrast, the deactivating pattern typically
mean arterial blood pressure, and reduction occurs in response to clips that depict scenes
in measures of vasoconstriction (Kreibig, related to a loss that has already occurred
2010). Some also report a reduction in res- (e.g., a young boy crying over his father’s
piratory activity, indicated by a reduction death). It is possible that anticipatory sad-
in respiratory rate and consequent increase ness as opposed to the sadness experienced
in pCO2 (Kreibig et al., 2007). Similarly, after a loss or during grieving, together with
reports of sadness induced by music or stan- the crying status, may play a role in differ-
dardized imagery find a reduction in heart entiating physiological responses in sadness
rate and respiratory rate (Etzel et al., 2006). (Kreibig, 2010).
In addition to these studies reporting pat-
terns of cardiovascular and electrodermal
Happiness
responses to the observation or experience
of sadness, a recent study has also identified Happiness has been induced with a wide
the role of a visual physiological response, variety of emotion-elicitation paradigms,
pupil size, in the perception of facial expres- including directed facial action, person-
sions of sadness. In this study, Harrison and alized recall, standardized imagery, film
colleagues suggested that, in the context clips, music, and pictures. The pattern of
of expressions of sadness, smaller observed physiological response in happiness can
pupils are interpreted as signaling a greater be characterized by an increase in cardiac
intensity of experienced sadness (Harri- activity secondary to vagal withdrawal,
son, Singer, Rotshtein, Dolan, & Critchley, vasodilation, and increased electrodermal
2006; Harrison, Wilson, & Critchley, 2007). and respiratory activity. This pattern sug-
Pupillary constriction can be indicative of gests a differentiated sympathetic response
increased parasympathetic and/or decreased characterized by a decrease in α- and
sympathetic influences. This study implied β-adrenergic and increase in cholinergically
that a reduction in pupil size may contribute mediated effects. Similar to many of the
to the pattern of physiological responses negative emotions, happiness is associated
associated with sadness. This position has with cardiac activation secondary to vagal
recently been strengthened by a single report withdrawal. However, in contrast to many
of a woman who despite having a light- of the negative emotions, it is associated
unresponsive pupil (due to Holmes Adie with peripheral vasodilation.
syndrome) showed a marked reduction in More specifically, cardiovascular respon-
pupil size during crying-associated sadness ses reported in happiness typically include
(del Valle & Garcia Ruiz, 2009). an increase in heart rate and decreased or
To summarize, the published literature unchanged heart rate variability. Studies
suggests two broad patterns of sadness- recording blood pressure responses also typ-
associated physiological responses – an acti- ically indicate an increase in systolic, diasto-
vating and a deactivating response – that lic, and/or mean arterial pressure (Prkachin
substantially overlap with the patterns of et al., 1999). Vasodilation – inferred from
A TWO-WAY ROAD 95
an increase in finger temperature, increased feelings, let us now consider the anatomy
finger pulse amplitude, and finger and ear and physiology of visceral afferent neurons
pulse transit time – has also been described and their potential role in the generation of
in a number of studies (Levenson et al., feelings.
1992; Stemmler & Fahrenberg, 1989). Incre-
ased electrodermal activity (skin conduc-
Visceral Afferent Neurons
tance level and nonspecific skin conduc-
tance response rate) is also sometimes seen, Both physiological outflow, as discussed ear-
though not uniformly reported. Increased lier, and physiological inflow have been pro-
respiratory activity is evidenced by an posed to play a role in emotion. Although
increase in respiratory rate associated with the physiological feedback hypothesis of
a reduction in both inspiratory and expira- emotional feelings has received less atten-
tory time and in postexpiratory pause time tion than physiological (efferent) effects of
(Kreibig, 2010). emotion, the underlying physiological archi-
Whereas we consider here only one pos- tecture warrants discussion. In addition to
itive emotion, it appears important to care- the tissue-specific efferent control of the
fully define and distinguish among positive internal organs via the ANS, the brain
emotions (e.g., amusement, contentment, also receives continuous afferent feedback
joy, pride, relief) when studying and char- from the viscera through pathways using
acterizing physiological responses (Kreibig, neuronal, hormonal, chemical, and physi-
Gendolla, & Scherer, 2010). It has also been cal mediators (Janig, 2006). This continuous
recently suggested that happiness may not afferent feedback provides information
be the only positive “basic emotion” and that about the state of homeostatic parameters,
pride may also be universally expressed and such as levels of blood oxygen, bicarbon-
recognized (Tracy & Robins, 2004). ate, and glucose, as well as the size of
fat stores. It signals the filling state of the
internal organs, systemic arterial blood pres-
Summary of Findings
sure, activity of endocrine glands, and the
Table 3.1 summarizes these modal changes state of the immune system. This section
in physiological responses in the heart, focuses on neural feedback mechanisms,
lungs, stomach, skin, peripheral blood ves- although hormonal, chemical, and physi-
sels, pupil, and immune systems across these cal feedback mechanisms may also operate
five primary emotions. This table shows that through interactions with central compo-
use of a single or a limited number of phys- nents of this neural pathway. For example,
iological measures will often fail to reveal effects of circulating feeding-related pep-
emotion-specific patterns of physiological tides, such as glucagon-like peptide 1 and
responses, which are identified only when pancreatic polypeptide, on gastric motility,
a broader set of measures are recorded. It acid secretion, or feelings of satiety are likely
also illustrates the importance of the context to be mediated via direct actions on the area
in which individual emotions are elicited postrema (Travagli & Rogers, 2001). This
for the patterns of associated physiological brain region has a weak blood-brain barrier
responses (e.g., elicitors of core or body- that is neurally connected both to the vagal
boundary-violation disgust and activating or afferent nucleus (nucleus tractus solitarius)
deactivating sadness). and the vagal efferent dorsal motor nucleus
(Rogers, McTigue, & Hermann, 1995).
Visceral organs in the thoracic, abdom-
Afferent Effects of Autonomic Nervous inal, and pelvic cavities are innervated by
System Activity in Emotion vagal and spinal visceral afferent neurons
that convey physical and chemical events
Taking the alternative causal direction of the in the visceral organs to the brainstem and
influence of physiology on emotion, specif- spinal cord (Janig, 2006). These afferent
ically the experiential aspect of emotional feedback signals connect to all levels of the
96 NEIL A. HARRISON, SYLVIA D. KREIBIG, & HUGO D. CRITCHLEY
Notes: HR – heart rate, HRV – high-frequency components of heart rate variability (an index of
parasympathetic influences on the heart), EDA – electrodermal activity, TNF – tumor necrosis factor.
Disgust: c – core disgust, b – body-boundary-violation disgust. Sadness: a – activated sadness, d –
deactivated sadness.
autonomic motor hierarchy, including cor- et al., 1997). Their activation generates “sick-
tical centers such as the insula, that are ness behaviors” (Bluthe et al., 1994; Harrison
believed to be responsible for conscious sen- et al., 2009) – the collection of cognitive,
sations (Craig, Chen, Bandy, & Reiman, emotional, behavioral, and motivational
2000) and to regions regulating behavioral change associated with infection or inflam-
states (Swanson, 2000). Afferents traveling mation.
in the vagus nerve represent about 80–85% of Vagal afferents project viscerotopically
all vagus nerve fibers and are subdivided into to the medullary nucleus tractus solitarius
several functional types. With a few excep- (NTS). Sensations generated by the activa-
tions, they are not involved in visceral pain, tion of vagal afferents are represented in the
but they are involved in other visceral sensa- middle part of the dorsal insula (Craig &
tions, such as hunger, satiety, thirst, nausea, Blomqvist, 2002). In primates, the sensations
and respiratory sensations (Morley, Levine, elicited by vagal afferent neurons are medi-
Kneip, & Grace, 1982). Subgroups of vagal ated by a viscerotopically organized path-
afferents are activated by inflammatory pro- way, projecting from the NTS either directly
cesses and by pro-inflammatory cytokines, or indirectly via the external medial nucleus
such as tumor necrosis factor (TNF-alpha) of the parabrachial nucleus to the basal part
and interleukin 1beta (IL-1beta; Goehler of the ventromedial nucleus (VMb) of the
A TWO-WAY ROAD 97
thalamus and from there to the dorsal insula James posed this question more than a cen-
(Craig, 2002). Taste is represented rostrally tury ago (what is an emotion) (James, 1894),
with the gastrointestinal tract, and respira- yet the notion that afferent visceral signals
tory and cardiovascular systems are repre- are essential for the unique experiences of
sented more caudally. distinct emotions has remained a key unre-
In addition to vagal afferent fibers, the solved question at the heart of affective neu-
viscera are also innervated by spinal vis- roscience (Cacioppo et al., 2000; Rainville
ceral afferent neurons. Spinal afferent fibers et al., 2006). Cannon then later Hess’ early
project to lamina I, lamina V, and deeper challenges to the James-Lange position, that
lamina of the spinal gray matter (Craig & emotional feelings result from central rep-
Blomqvist, 2002) and are involved in visceral resentations of changes in body physiology,
pain and nonpainful visceral sensations. argued that autonomic outflow was both
Sensations associated with spinal visceral too slow and also too poorly differentiated
afferent neurons (e.g., thermal sensations to enable emotion-specific response patterns
and pain) probably originate in neurons (Cannon, 1931).
ascending in lamina I, which is composed Each of these criticisms to the James-
of functionally distinct neural channels that Lange theory has received focused atten-
are activated by afferent neurons innervat- tion in the intervening years. For example,
ing skin, deep somatic tissues, and viscera. In Damasio (1999) has argued for the existence
primates, lamina I neurons project directly of an “as if loop,” conceptually similar to
to the posterior part of the ventrome- notions of efference copy in motor theory. It
dial nucleus of the thalamus (VMpo). The would allow for a real-time central represen-
VMpo, which is caudal to VMb, projects tation of centrally generated physiological
topographically to the dorsal insula and cau- motor commands and further provide a tem-
dally to the projection field of the VMb porally sensitive system to detect peripher-
(Craig, 2002). ally generated changes in physiological activ-
The dorsal insula is the primary sensory ity. Similarly, with regard to differentiation
cortex of interoception and represents sen- of autonomic activity, the demonstration
sations related to the states of the tissues that both the sympathetic (Morrison, 2001)
of the body, including the visceral organs and the parasympathetic (Porges, 2007)
(Craig et al., 2000). It has been proposed nervous systems possess exquisite organ-
that re-representation and integration of the specific regulation has powerfully countered
physiological state of all bodily tissues with this argument and contributed to a revival of
other contextual information within the somatic theories (Damasio, 1994).
anterior insula provide a substrate for con- As detailed in the section, “Patterned
scious awareness of changes in one’s internal Autonomic Responses to Distinct Emo-
physiological state (interoception) and emo- tions,” emotion-specific patterns of periph-
tional feelings (Craig, 2002; Critchley et al., eral autonomic activity have been reported
2004; see Figure 3.3). These structures might to many of the basic emotions. Similarly,
provide a physiological basis for proposed emotion-specific patterns of neural activity
autonomic feedback mechanisms that could have also been demonstrated in association
support a role of autonomic sensations in with discrete feeling states (Damasio et al.,
emotion generation and the mental repre- 2000). Recently, using a combination of
sentation of emotional feelings. fMRI and simultaneous recording of cardiac
and gastric activity during the experience of
core and body-boundary-violation disgust,
Role of Peripheral-Physiological two of the authors addressed the question
Responses in Emotional Feelings of whether central neural representations
of emotion-specific peripheral-physiological
Do we run from a bear because we are afraid patterning also contribute to reported
or are we afraid because we run? William subjective feeling states (Harrison et al.,
lat.
ventr.
rostral
taste
cir/inflamation
respir./card. vasc dorsal thalamus
pain insula
2 temperature, itch
VMb visceral sensations
caudal
1
VMpo
caudal
blood borne
factors
NTS Medulla
AP
DMNX
CC
NA
CN
Figure 3.3. Schematic representation of visceral afferent pathways. Spinal visceral afferents project to
lamina I of the spinal cord and afferents traveling with the facial, glossopharangeal, and vagus nerves
project to the nucleus tractus solitarius (NTS). Blood-borne mediators (e.g., ghrelin, angiotensin) act
via the adjacent area postrema (AP). The transverse section at the level of the medullary obex
demonstrates the close anatomical relationship and connectivity of the afferent NTS and AP and
efferent vagus dorsal motor nucleus (DMNX) and nucleus ambiguus (NA). Visceral afferent fibers
from lamina I and NTS then project viscerotopically either directly or via the external medial nucleus
of the parabrachial nucleus to the basal and posterior parts, respectively, of the thalamic ventromedial
nucleus (VMb VMpo). Fibers then project to the dorsal posterior/mid-insula while maintaining their
viscerotopic organization. Courtesy: We are grateful to Dr. Sarah Garfinkel for producing this figure.
A TWO-WAY ROAD 99
measures, such as frequency domain analy- tion induction influences the pattern of
ses of cardiac interbeat intervals, are used in observed autonomic responses. It has pre-
their place. Although many of these indices viously been argued that the effects of phys-
(e.g., heart rate) have the advantage of ical and psychological contextual factors,
being relatively easy to measure, their value such as physical activity and body posture,
in discriminating activity within individual attention, and mental effort, on patterns of
functional subunits or even in distinguish- physiological activation may cause the low
ing sympathetic from parasympathetic consistency found in meta-analyses of phys-
influences on individual organs is more iological patterning observed to individual
equivocal. Similarly, the use of individual emotions (Stemmler et al., 2001). Because
convenience measures, such as skin conduc- physical and nonemotional psychological
tance responses, as a summary measure of contexts vary widely across studies of emo-
sympathetic arousal, is, at best, naive and, at tion, a low degree of consistency of emotion-
worst, may lead to false conclusions about specific physiological configurations may be
emotion-associated changes in patterned expected.
sympathetic activity. Consequently, future Concepts of context-deviation specificity
studies aiming to discriminate emotion- view emotion specificity as a conditional
specific physiological responses will benefit concept, in which an emotional stimu-
from a careful selection of multiple physi- lus modifies a context-bound physiologi-
ological indices sensitive to functionally dis- cal pattern. Thus, emotion and contextual
tinct autonomic pathways, both within and influences on patterns of observed physi-
between individual organ systems. Thus, ological responses are usually confounded.
until we arrive at a clear understanding of It has been argued that emotion speci-
which response channels are most indicative ficity can therefore only be demonstrated
of a particular emotion and most discrimi- once this confound has been pulled apart.
native between certain emotion contrasts, a Emotion specificity therefore may only be
comprehensive sampling will be necessary. found when there are systematic and spe-
Of note, a recent study found that as few cific deviations of the emotion-plus-context
as five measures – pre-ejection period, skin pattern of physiological reactivity from the
conductance response rate, end-tidal pCO2 , context-alone pattern. Currently, relatively
and the two facial muscles corrugator super- few studies have systematically investi-
cilii and zygomatic major – may be ade- gated either context–emotion confounds or
quate to distinguish between two emotional the validity of context-deviation specificity
states (fear and sadness), if they are selected (Kreibig, Wilhelm, Gross, & Roth, 2005;
such that they are maximally discriminant Stemmler et al., 2001). This will need to be
(Kolodyazhniy, Kreibig, Gross, Roth, & Wil- considered in future studies.
helm, 2011). However, to identify such max-
imally discriminant response measures, a
What Is the Role of Physiological
broad assessment of ANS functioning is
Context?
necessary, because response measures that
might be discriminative of one emotion con- In addition to psychological context, the
trast (e.g., fear and sadness) need not to be processing of environmental stimuli is also
discriminating of another emotion contrast influenced by the organism’s visceral state
(e.g., fear and anger; Kolodyazhniy et al., at the time of stimulus presentation. As
2011; Stemmler, 2004). discussed in the section on visceral affer-
ents, the brain receives continuous affer-
ent information about internal bodily func-
What Are the Roles of Physical and
tions. Cardiovascular homeostasis depends
Psychological Context?
on such feedback, notably phasic signals
Throughout the earlier discussion on emo- from aortic arch and carotid sinus barore-
tion-specific physiological patterning, it was ceptors activated by blood-volume disten-
repeatedly noted that the context of emo- sion during cardiac systole. Baroreceptor
A TWO-WAY ROAD 101
firing during cardiac systole is known to ent pathways, through which central emo-
modulate efferent autonomic responses in tional processes may organize autonomic
an organ-specific manner; for example, responding, and, vice versa, from the per-
inhibiting muscle sympathetic nerves while spective of afferent pathways through which
leaving skin sympathetic outflow unaltered changes in peripheral visceral state may
(Wallin & Fagius, 1988). Interestingly, the feed back and contribute to the experi-
central processing of painful stimuli and ence of emotional feelings. The review of
consequently the perception of pain are also the anatomy and central control of the
influenced by physiological state, includ- ANS illustrated a system consisting of mul-
ing resting blood pressure and time during tiple functionally distinct units, controlled
the cardiac cycle (i.e., systole or diastole; by a hierarchy of direct and indirect cen-
Gray, Minati, Paoletti, & Critchley, 2010) tral synaptic connections known as the
and experimentally increasing aortic barore- central autonomic network that together
ceptor firing by applying pressure to the provide the hardware for emotion-specific
neck at specific phases of the cardiac cycle. autonomic responding. Consistent with this
Similarly, presentation of unexpected anatomical perspective our review of pat-
somatosensory stimuli early in cardiac sys- terned autonomic physiological responses
tole also inhibits muscle sympathetic nerve to distinct emotions identified separable
activity, whereas presenting stimuli before response patterns for fear, anger, disgust,
cardiac systole does not. Intriguingly, this sadness, and happiness.
pattern is exaggerated in patients with blood The existence of such differentiated auto-
phobia and syncope, suggesting that feed- nomic patterning in emotion leads to the
back mechanisms supporting cardiac home- related question of whether and if so
ostasis shape an individual’s affective style. to what extent visceral afferent informa-
Recent data from our laboratory (Gray tion contributes to subjectively experienced
et al., 2012) suggest that the presentation emotional feelings. Various neural feed-
of brief visual emotional stimuli at differ- back mechanisms exist that allow for the
ent points in the cardiac cycle may also transmission of information on homeostatic
influence the pattern of subsequent physi- parameters. Re-representation and integra-
ological responses and associated subjective tion of the physiological state of all bod-
experience. Together these data suggest that ily tissues with other contextual informa-
physiological context (e.g., timing in the tion within the anterior insula may provide a
cardiac cycle or even blood pressure more substrate for conscious awareness of changes
broadly) influences both the processing of in physiological state and emotional feel-
motivationally significant nociceptive stim- ings. However, the exact nature and role of
uli applied directly to the body and also visu- peripheral physiological responses to emo-
ally presented emotional stimuli. Whether tional feelings remain to be determined.
physiological context, such as resting blood Based on our review of the literature,
pressure, or timing of stimuli presentation we considered practical implications and
relative to recurrent autonomic events, such future challenges, highlighting the need for
as cardiac and respiratory cycles, influences a comprehensive sampling of physiological
physiological and subjective responses to response measures at least until we develop
emotional stimuli more broadly remains an a better understanding of which channels
open empirical question and will need to be are most indicative of a particular emo-
addressed in future studies. tion and most discriminative between dis-
tinct emotions. Furthermore, we noted the
potential influences of physical, psycholog-
Conclusion ical, and physiological context variables on
physiological responses on emotion. Taken
In this chapter, we have considered the together, we hope that this chapter inspires
physiology of emotion responses from two new and innovative research on the con-
perspectives: from the perspective of effer- tribution of efferent outflows and afferent
102 NEIL A. HARRISON, SYLVIA D. KREIBIG, & HUGO D. CRITCHLEY
Critchley, H. D. (2009). Psychophysiology Furness, J. B., Morris, J. L., Gibbins, I. L., &
of neural, cognitive and affective integra- Costa, M. (1989). Chemical coding of neu-
tion: fMRI and autonomic indicants. Inter- rons and plurichemical transmission. Annual
national Journal of Psychophysiology, 73(2), Review of Pharmacology and Toxicology, 29,
88–94. 289–306.
Critchley, H. D., Corfield, D. R., Chandler, M. Gianaros, P. J., Derbyshire, S. W., May, J. C.,
P., Mathias, C. J., & Dolan, R. J. (2000). Cere- Siegle, G. J., Gamalo, M. A., & Jennings, J. R.
bral correlates of autonomic cardiovascular (2005). Anterior cingulate activity correlates
arousal: A functional neuroimaging investi- with blood pressure during stress. Psychophys-
gation in humans. Journal of Physiology, 523 iology, 42(6), 627–35.
(Pt. 1), 259–70. Gibbins, I. L. (1995). Chemical neuroanatomy
Critchley, H. D., Elliott, R., Mathias, C. J., & of sympathetic ganglia. In E. M. McLachlan
Dolan, R. J. (2000). Neural activity relating to (Ed.), Autonomic ganglia (pp. 73–122(. Luxem-
generation and representation of galvanic skin burg: Harwood Academic Publishers.
conductance responses: A functional magnetic Gilissen, R., Bakermans-Kranenburg, M. J., van
resonance imaging study. Journal of Neuro- Ijzendoorn, M. H., & van der Veer, V. (2008).
science, 20(8), 3033–40. Parent-child relationship, temperament, and
Critchley, H. D., Wiens, S., Rotshtein, P., physiological reactions to fear-inducing film
Ohman, A., & Dolan, R. J. (2004). Neural clips: Further evidence for differential suscep-
systems supporting interoceptive awareness. tibility. Journal of Experimental Child Psychol-
Nature Neuroscience, 7(2), 189–95. ogy, 99(3), 182–95.
Damasio, A. R. (1994). Descartes’ error: Emotion, Goehler, L. E., Relton, J. K., Dripps, D.,
reason and the human brain. New York: Gros- Kiechle, R., Tartaglia, N., Maier, S. F.,
set/Putnam. et al. (1997). Vagal paraganglia bind biotiny-
Damasio, A. R. (1999). The feeling of what hap- lated interleukin-1 receptor antagonist: A pos-
pens: Body and emotion in the making of con- sible mechanism for immune-to-brain com-
sciousness. New York: Harcourt Brace. munication. Brain Research Bulletin, 43(3),
Damasio, A. R., Grabowski, T. J., Bechara, A., 357–64.
Damasio, H., Ponto, L. L. B., Parvizi, J., et al. Gray, M. A., Beacher, F. D., Minati, L., Nagai,
(2000). Subcortical and cortical brain activity Y., Kemp, A. H., Harrison, N. A., et al. (2012).
during the feeling of self-generated emotions. Emotional appraisal is influenced by cardiac
Nature Neuroscience, 3(10), 1049–56. afferent information. Emotion 12(1), 180–91.
del Valle, L. M., & Garcia Ruiz, P. J. (2009). A Gray, M. A., Minati, L., Paoletti, G., & Critchley,
new clinical sign in Holmes-Adie syndrome. H. D. (2010). Baroreceptor activation attenu-
Journal of Neurology, 256(1), 127–28. ates attentional effects on pain-evoked poten-
Dimberg, U., & Thunberg, M. (2007). Speech tials. Pain, 151(3), 853–61.
anxiety and rapid emotional reactions to angry Gross, J. J., Frederickson, B. L., & Levenson,
and happy facial expressions. Scandinavian R. W. (1994). The psychophysiology of crying.
Journal of Psychology, 48(4), 321–28. Psychophysiology, 31(5), 460–68.
Ekman, P., Levenson, R. W., & Friesen, W. Gross, J. J., & Levenson, R. W. (1997). Hiding
V. (1983). Autonomic nervous-system activ- feelings: The acute effects of inhibiting nega-
ity distinguishes among emotions. Science, tive and positive emotion. Journal of Abnormal
221(4616), 1208–10. Psychology, 106(1), 95–103.
Etzel, J. A., Johnsen, E. L., Dickerson, J., Tranel, Harrison, N. A., Brydon, L., Walker, C., Gray,
D., & Adolphs, R. (2006). Cardiovascular and M. A., Steptoe, A., Dolan, R. J., et al. (2009).
respiratory responses during musical mood Neural origins of human sickness in intero-
induction. International Journal of Psychophys- ceptive responses to inflammation. Biological
iology, 61(1), 57–69. Psychiatry, 66(5), 415–22.
Feldman-Barrett, L. (2006). Are emotions natural Harrison, N. A., Gray, M. A., Gianaros, P. J.,
kinds? Perspectives on Psychological Science, 1, & Critchley, H. D. (2010). The embodiment
28–58. of emotional feelings in the brain. Journal of
Fredrickson, B. L., & Levenson, R. W. (1998). Pos- Neuroscience, 30(38), 12878–84.
itive emotions speed recovery from the cardio- Harrison, N. A., Singer, T., Rotshtein, P., Dolan,
vascular sequelae of negative emotions. Cog- R. J., & Critchley, H. D. (2006). Pupillary
nition and Emotion, 12(2), 191–220. contagion: Central mechanisms engaged in
104 NEIL A. HARRISON, SYLVIA D. KREIBIG, & HUGO D. CRITCHLEY
sadness processing. Social Cognitive and Affec- Kreibig, S. D., Wilhelm, F. H., Gross, J. J.,
tive Neuroscience, 1(1), 5–17. & Roth, W. T. (2005). Specific emotional
Harrison, N. A., Wilson, C. E., & Critchley, H. responses as deviations from the experimental
D. (2007). Processing of pupil size modulates context. Psychophysiology, 42(s1), s77.
perception of sadness and predicts empathy. Kreibig, S. D., Wilhelm, F. H., Roth, W. T., &
Emotion, 7(4), 724–29. Gross, J. J. (2007). Cardiovascular, electroder-
Hilton, S. M. (1975). Ways of viewing the central mal, and respiratory response patterns to fear-
nervous control of the circulation – old and and sadness-inducing films. Psychophysiology,
new. Brain Research, 87, 213–19. 44(5), 787–806.
Izard, C. E. (2007). Basic emotions, natural kinds, Lang, P. J., Bradley, M. M., & Cuthbert, B.
emotion schemas, and a new paradigm. Per- N. (1993). Emotion, arousal, valence, and the
spectives on Psychological Science, 2(3), 260–80. startle reflex. In N. Birbaumer & A. Ohman
James, W. (1894). Physical basis of emotion. Psy- (Eds.), The structure of emotion: Psychophys-
chological Review, 1, 516–29. iological, cognitive and clinical aspects (pp.
Janig, W. (2006). The integrative action of the 243–51). Seattle: Hogrefe & Huber.
autonomic nervous system: Neurobiology of Langley, J. N. (1900). The sympathetic and other
homeostasis. Cambridge: Cambridge Univer- related systems of nerves. In Textbook of Phys-
sity Press. iology. London: Young J. Pentland.
Janig, W., Sundlof, G., & Wallin, B. G. (1983). Levenson, R. W. (1988). Emotion and the auto-
Discharge patterns of sympathetic neurons nomic nervous system: A prospectus for
supplying skeletal muscle and skin in man and research on autonomic specificity. In H. L.
cat. Journal of the Autonomic Nervous System, Wagner (Ed.), Handbook of affective sciences
7(3–4), 239–56. (pp. 212–24). Chichester: Wiley.
Jokerst, M. D., Levine, M., Stern, R. M., & Levenson, R. W., Heider, K., Ekman, P., &
Koch, K. L. (1997). Modified sham feeding Friesen, W. V. (1992). Emotion and autonomic
with pleasant and disgusting foods: Cephalic- nervous-system activity in the Minangkabau
vagal influences on gastric myoelectric activ- of West Sumatra. Journal of Personality and
ity. Gastroenterology, 112(4), A755. Social Psychology, 62(6), 972–88.
Jonsson, P., & Sonnby-Borgstrom, M. (2003). Levine, M. E. (2005). Sickness and satiety: Phys-
The effects of pictures of emotional faces on iological mechanisms underlying perceptions
tonic and phasic autonomic cardiac control in of nausea and stomach fullness. Current Gas-
women and men. Biological Psychology, 62(2), troenterology Reports, 7(4), 280–88.
157–73. Mangina, C. A., & Beuzeron-Mangina, J. H.
Kaada, B. R. (1951). Somato-motor, autonomic (1996). Direct electrical stimulation of specific
and electrocorticographic responses to elec- human brain structures and bilateral electro-
trical stimulation of rhinencephalic and other dermal activity. International Journal of Psy-
structures in primates, cat, and dog; a study chophysiology, 22(1–2), 1–8.
of responses from the limbic, subcallosal, Matsunaga, M., Isowa, T., Kimura, K.,
orbito-insular, piriform and temporal cor- Miyakoshi, M., Kanayama, N., Murakami,
tex, hippocampus-fornix and amygdala. Acta H., et al. (2008). Associations among central
Physiological Scandinavia Supplement, 24(83), nervous, endocrine, and immune activities
1–262. when positive emotions are elicited by look-
Kolodyazhniy, V., Kreibig, S. D., Gross, J. J., ing at a favorite person. Brain, Behavior, and
Roth, W. T., & Wilhelm, F. H. (2011). An Immunity, 22(3), 408–17.
affective computing approach to physiolog- Mauss, I. B., & Robinson, M. D. (2009). Measures
ical emotion specificity: Toward subject- of emotion: A review. Cognition and Emotion,
independent and stimulus-independent 23(2), 209–37.
classification of film-induced emotions. Mobbs, D., Petrovic, P., Marchant, J. L., Has-
Psychophysiology, 48(7), 908–22. sabis, D., Weiskopf, N., Seymour, B., et al.
Kreibig, S. D. (2010). Autonomic nervous system (2007). When fear is near: Threat imminence
activity in emotion: A review. Biological Psy- elicits prefrontal-periaqueductal gray shifts in
chology, 84(3), 394–421. humans. Science, 317(5841), 1079–83.
Kreibig, S. D., Gendolla, G. H., & Scherer, K. Morley, J. E., Levine, A. S., Kneip, J., & Grace,
R. (2010). Psychophysiological effects of emo- M. (1982). The effect of vagotomy on the sati-
tional responding to goal attainment. Biologi- ety effects of neuropeptides and naloxone. Life
cal Psychology, 84(3), 474–87. Sciences, 30(22), 1943–47.
A TWO-WAY ROAD 105
Morrison, S. F. (2001). Differential control of disgust. Journal of Personality and Social Psy-
sympathetic outflow. American Journal of chology, 66(5), 870–81.
Physiology-Regulatory Integrative and Compar- Saper, C. B. (2002). The central autonomic ner-
ative Physiology, 281(3), R683–R698. vous system: Conscious visceral perception
Morrison, S. F., & Cao, W. H. (2000). Differ- and autonomic pattern generation. Annual
ent adrenal sympathetic preganglionic neu- Review of Neuroscience, 25, 433–69. Retrieved
rons regulate epinephrine and norepinephrine from PM:12052916
secretion. American Journal of Physiology- Scherer, K. R. (2004). Feelings integrate the
Regulatory Integrative and Comparative Phys- central representation of appraisal-driven
iology, 279(5), R1763–R1775. response organisation in emotion. In A.S.R.
Nagai, Y., Critchley, H. D., Featherstone, E., Manstead, N. H. Frijda, & A. H. Fischer
Trimble, M. R., & Dolan, R. J. (2004). Activ- (Eds.), Feelings and emotions: The Amsterdam
ity in ventromedial prefrontal cortex covaries Symposium (pp. 136–57). Cambridge: Cam-
with sympathetic skin conductance level: A bridge University Press.
physiological account of a “default mode” of Scherer, K. R. (2005). What are emotions? And
brain function. Neuroimage, 22(1), 243–51. how can they be measured? Social Science
Nakayama, K., Goto, S., Kuraoka, K., & Naka- Information, 44(4), 693–727.
mura, K. (2005). Decrease in nasal tempera- Scherer, K. R. (2009). The dynamic architec-
ture of rhesus monkeys (Macaca mulatta) in ture of emotion: Evidence for the component
negative emotional state. Physiology & Behav- process model. Cognition & Emotion, 23(7),
ior, 84, 783–90. 1307–51.
Pick, J. (1970). The autonomic nervous system. Shimazu, T., & Fukuda, A. (1965). Increased
Philadelphia: Lippincott. activities of glycogenolytic enzymes in liver
Porges, S. W. (2007). The polyvagal perspective. after splanchnic-nerve stimulation. Science,
Biological Psychology, 74(2), 116–43. 150(703), 1607–8.
Prkachin, K. M., Mills, D. E., Zwaal, C., & Sinha, R., Lovallo, W. R., & Parsons, O. A.
Husted, J. (2001). Comparison of hemody- (1992). Cardiovascular differentiation of emo-
namic responses to social and nonsocial stress: tions. Psychosomatic Medicine, 54(4), 422–35.
Evaluation of an anger interview. Psychophys- Stemmler, G. (2004). Physiological processes dur-
iology, 38(6), 879–85. ing emotion. In P. Philippot & R. S. Feldman
Prkachin, K. M., Williams-Avery, R. M., Zwaal, (Eds.), The regulation of emotion. Mahwah, NJ:
C., & Mills, D. E. (1999). Cardiovascular Erlbaum.
changes during induced emotion: An applica- Stemmler, G., Aue, T., & Wacker, J. (2007).
tion of Lang’s theory of emotional imagery. Anger and fear: Separable effects of emotion
Journal of Psychosomatic Research, 47(3), and motivational direction on somatovisceral
255–67. responses. International Journal of Psychophys-
Rainville, P., Bechara, A., Naqvi, N., & Dama- iology, 66(2), 141–53.
sio, A. R. (2006). Basic emotions are associ- Stemmler, G., & Fahrenberg, J. (1989). Psy-
ated with distinct patterns of cardiorespiratory chophysiological assessment: Conceptual,
activity. International Journal of Psychophysiol- psychometric, and statistical issues. In G.
ogy, 61, 5–18. Turpin (Ed.), Handbook of clinical psychophys-
Rogers, R. C., McTigue, D. M., & Hermann, iology (pp. 71–104). Chichester,: Wiley.
G. E. (1995). Vagovagal reflex control of Stemmler, G., Heldmann, M., Pauls, C. A., &
digestion: Afferent modulation by neural and Scherer, T. (2001). Constraints for emotion
“endoneurocrine” factors. American Journal of specificity in fear and anger: The context
Physiology, 268(1 Pt. 1), G1–10. counts. Psychophysiology, 38(2), 275–91.
Rohrmann, S., & Hopp, H. (2008). Cardiovascu- Stern, R. M., Jokerst, M. D., Levine, M.
lar indicators of disgust. International Journal E., & Koch, K. L. (2001). The stomach’s
of Psychophysiology, 68(3), 201–8. response to unappetizing food: Cephalic-
Rottenberg, J., Wilhelm, F. H., Gross, J. J., & vagal effects on gastric myoelectric activ-
Gotlib, I. H. (2003). Vagal rebound during ity. Neurogastroenterology and Motility, 13(2),
resolution of tearful crying among depressed 151–54.
and nondepressed individuals. Psychophysiol- Stevenson, R. J., Hodgson, D., Oaten, M. J.,
ogy, 40(1), 1–6. Barouei, J., & Case, T. I. (2011). The effect of
Rozin, P., Lowery, L., & Ebert, R. (1994). Vari- disgust on oral immune function. Psychophys-
eties of disgust faces and the structure of iology, 48(7), 900–7.
106 NEIL A. HARRISON, SYLVIA D. KREIBIG, & HUGO D. CRITCHLEY
Andreas Keil
The conscious stream has fuzzy water in niques to novel experimental paradigms and
it, as well as driftwood, swimming fish, sophisticated analysis techniques. It is now
floating leaves and grass, and of course, possible to record from high-density sen-
careful scientific thinking. sor arrays at hundreds of extracranial loca-
– William James, Principles of tions, providing impressive spatial sampling
Psychology along with high temporal accuracy. Combi-
nation of electrophysiological methods with
other imaging modalities has been partic-
ularly fruitful. Modeling studies of meso-
Affective neuroscience studies using electro- and macroscopic brain activity (i.e., activ-
encephalography (EEG) or magneto-encep- ity in small and large groups of neurons)
halography (MEG) frequently highlight have provided a constantly improving con-
these methods’ exquisite temporal resolu- ceptual framework for interpreting the mea-
tion for capturing the electrophysiology1 of sured signals.
emotion. Although this is already a desir- Because of this rich potential for address-
able property, electrophysiological measure- ing pertinent questions in the affective
ments have additional advantages: They are neurosciences, human electrophysiology has
more affordable than hemodynamic imag- been a popular avenue for studying emotion.
ing techniques, are widely available, and Over the last decade, this field has seen high
impose little if any discomfort on research productivity, with increasing numbers of
participants. As a consequence, electrophys- published research reports, now accounting
iological methods have found wide appli- for hundreds of papers per year. In fact, elec-
cation in the cognitive and affective neu- trophysiological measures have been used to
rosciences. Recent years have seen exciting address questions centered on human emo-
developments in virtually every aspect of tional experience and behavior since their
EEG/MEG research of emotion, ranging inception in the late 1920s. It is thus help-
from innovation in data-recording tech- ful to begin by reviewing a few key events
107
108 ANDREAS KEIL
that have shaped the present state of the arousal, particularly startle, causes pronoun-
field. ced heightening and slowing of the 90 σ
[waves with a duration of approximately
90 ms, i.e, alpha waves around 10 Hz; AK]
A (Very) Short History of EEG and waves.” Later, he revised this view to suggest
MEG in Affective Neuroscience that “I tend to the conclusion that during
arousal, distress, and mental work, the big-
Hans Berger conducted the first series of ger 90 σ waves . . . fade, and the smaller 35 σ
human EEG recordings using scalp elec- waves [that is, waves at roughly 20–30 Hz;
trodes in the city of Jena, Germany, dur- presently referred to as upper beta, AK] get
ing the years 1910 to 1929 (see Berger, 1969, more abundant” (Borck, 2005).
for an English version of his first report For the next 30 years, most researchers
on these experiments). He initially used agreed that the spectral composition of
silver-chloride needle electrodes placed in ongoing EEG during emotion was best inter-
the scalp tissue together with a delicate gal- preted along the lines proposed by Berger.
vanometer apparatus to record the voltage For instance, in 1950, Donald Lindsley, a pio-
oscillations reflective of electrocortical pro- neer of EEG research in the United States,
cesses. These efforts were characterized by summarized the state of the art as follows:
setbacks, doubts, and very small samples
of subjects, mostly consisting of Berger’s Two principal kinds of changes are reflected
laboratory assistant, Fräulein Von Bülow in the EEG under conditions of emo-
(whom he married in 1911), his son Klaus, tional arousal, like apprehension, unex-
and Berger himself, who also attempted pected sensory stimulation, and anxiety
states: reduction or suppression of alpha
the first neurofeedback procedure, trying
rhythm and increase in amount of beta-
to monitor his own EEG using a mirror like fast activity. EEG studies now provide
system. From the beginning, the focus of data on cortical diencephalic relationships
the work of Berger and other early EEG and thus may resolve the problem of cen-
researchers was on characterizing the fre- tral and autonomic factors in relation to
quency content of EEG in response to a task emotion. (Lindsley, 1950)
or event. Most prominently, Berger identi-
fied “first-order waves” oscillating at a rate We see later that the use of alpha as a
of 10 cycles/second (10 Hz, later referred measure of cortical/behavioral idling and
to as alpha waves) and faster “second-order low arousal is still an important con-
waves,” oscillating at about 20 to 30 Hz (i.e., cept in contemporary affective neuro-
beta waves). The protocols of the record- science, although more recent work has
ing sessions during those years were scripted demonstrated that certain tasks and stimuli
by Berger in great detail and contain pre- heighten, rather than reduce, alpha power in
cise instructions for the presentation of the EEG (Klimesch, Sauseng, & Hanslmayr,
events to the subject; for example, “count 2006). Early recordings with intracranial
to ten (watch clock!), then gently touch electrodes indicated that oscillatory brain
Fräulein Von Bülow’s cheek with the glass activity is relevant for behavior, for instance
probe [underline by Berger], simultaneously when epilepsy patients showed heightened
pressing trigger key for the wave recorder. beta and gamma oscillations in the amygdala
N.B. No physical contact is to be made during fear conditioning (Lesse, 1957).
with Fräulein Von Bülow, or the reclining With the advent of computer technology,
chair.” extracting representative waveforms from
In the context of this chapter, it is of the EEG by averaging across many simi-
interest that this choreography regularly lar events became possible. This procedure
included the elements of startle and dis- yields event-related brain potentials (ERPs)
tress. For instance, on the first of Novem- that reflect the typical response to a given,
ber 1927, Berger noted in his diary “affective repeated event – sensory, behavioral, or
ELECTRO- AND MAGNETO-ENCEPHALOGRAPHY IN THE STUDY OF EMOTION 109
to be neglected for all practical purposes pret the data in the light of that particular
(Olejniczak, 2006), although specific oscil- reference. Thus, a major advantage of MEG
latory events contained in the EEG/MEG over EEG recordings is their greater spa-
may be reflective of nonsynaptic events tial specificity. Further differences exist with
when properly recorded and extracted from regard to the orientation of the generating
the raw signal (Whittingstall & Logothetis, tissue: Because a magnetic field is oriented
2009). orthogonal to the generating electric field,
Pronounced differences between EEG and because the brain has a near-spherical
and MEG emerge in terms of their sensi- shape, sources oriented radially (i.e., per-
tivity to these intracranial events: To arrive pendicular) to the scalp surface generate
at EEG scalp electrodes, currents gener- only very weak magnetic field gradients out-
ated by the cortical processes outlined ear- side the skull. Thus, MEG studies tend to
lier need to travel through the tissue of the contain a strong bias toward activity in the
human body. Thus, relatively strong cur- cortical sulci, which results in sources that
rents originating close to a given sensor elec- are oriented tangentially to the scalp surface.
trode are spatially well represented in the Many authors have emphasized that MEG
human EEG (Nunez & Srinivasan, 2006). might not be as sensitive to activity in deep
However, currents are conducted through brain regions, given that the magnetic field
the body and brain and may be measured at strength declines with the square of the dis-
remote electrodes, impeding spatial speci- tance to the source (Wennberg, Valiante, &
ficity. Thus, the fact that the original extra- Cheyne, 2011).
cellular voltage gradient is reduced and To summarize, scalp-recorded human
distorted in the process of volume conduc- electrophysiology reflects neural mass activ-
tion favors registration of EEG from near- ity in near-surface gray matter. Specifically,
surface cortex oriented parallel to the scalp, it is highly sensitive to the integrated synap-
over that from deep brain regions with less tic afferent processes at dendritic trees of
favorable orientation. Moreover, because pyramidal neurons to the extent that they
the different types of tissue that the current are synchronously active, thus generating
passes through (e.g., brain, cerebrospinal sufficiently strong electromagnetic fields.
fluid, skull, skin) each have different electric The orientation of these fields differentially
properties, the original electric field is not affects EEG and MEG, with MEG being
veridically represented. Volume conduction more sensitive to tangential generators and
effects may also vary across the recording EEG more sensitive to radial generators.
period, thus affecting the EEG as a function Some of these properties vary as a func-
of time. Overall, the scalp-recorded EEG tion of the recording parameters and record-
represents the true underlying voltage gradi- ing apparatus employed, and I review these
ents as they are altered by properties of the aspects next.
tissues between the electrical source and the
recording electrode on the scalp, conduc-
tive properties of the electrode itself, and The Recording Process
the orientation of the cortical generator to
the recording electrode. Excellent overviews EEG and MEG are safe and noninvasive
of the methods and physiology of the EEG procedures that are associated with mini-
are given by Nunez and Srinivasan (2006) mum discomfort to the participant. Over
and in the textbook by Regan (1989), which the last two decades, the application of sen-
are highly recommended reading. sors to the scalp has become even more com-
In contrast, MEG is directly sensitive to fortable with technical innovations such as
the original cellular events not mediated by all-in-one EEG electrode caps, which can
volume conduction; this property eliminates hold hundreds of sensors. MEG systems like-
the problem of spatial distortion of the neu- wise have grown to contain dense arrays
ral signal (Cohen & Cuffin, 1987). There is of sensors in an expensive and intricate
also no need to select a reference and inter- piece of engineering: A large vacuum flask
ELECTRO- AND MAGNETO-ENCEPHALOGRAPHY IN THE STUDY OF EMOTION 111
(Dewar) shaped as a helmet-like structure those sites and distort the measured voltage.
contains superconducting quantum interfer- For instance, it has been proposed that use
ence devices (SQUIDs) immersed in liquid of a nose reference may enhance sensitiv-
helium, at a temperature of 4◦ Kelvin. When ity of the EEG to certain noncerebral (ocu-
cooled to the temperature of liquid helium, lar) signals, which then may be represented
superconducting elements carry electricity at frontal electrodes and misinterpreted as
without resistance. This lack of resistance oscillatory brain activity (Yuval-Greenberg,
allows a SQUID to measure the magnetic Tomer, Keren, Nelken, & Deouell, 2008).
interference induced at detection coils also Alternatively, the EEG may be recorded
embedded in the MEG helmet, close to the using any one of the sensors on the scalp
participant’s head. In contemporary EEG itself (for instance, the vertex electrode at
recordings, electrodes are placed over many the top of the head) as the reference site
different scalp locations. The most fre- during recording, and the so-called average
quently used sensors involve a silver-silver reference is computed offline as the mean
chloride contact that is filled with conduc- difference across all electrodes. Use of this
tive paste, but other technologies are now average reference avoids biases due to the
available that provide excellent electric con- specific reference location, but requires
tact between the scalp and the amplifier. For extensive coverage over the entire head for
instance, electrode contacts can be placed in accurate estimates. Simulation studies are
a sponge, which conducts current when it is available to guide the choice of the reference
bathed in a saline solution prior to record- for a given electrode montage (Junghöfer,
ing (Ferree, Luu, Russell, & Tucker, 2001). Elbert, Tucker, & Braun, 1999).
Active and shielded electrodes are also avail- In this context, one major advantage of
able now, further improving the quality of MEG technology becomes clear: There is
the recorded signal and reducing the impact no need to define or use a reference sensor,
of extracranial noise. because both magnetometer-based (measur-
Because voltage is a relative measure – ing the strength of the magnetic field) and
that is, a difference in electrical potential gradiometer-based (measuring the local gra-
between two sites – EEG voltage at any spe- dient; i.e., the difference in magnetic flux)
cific sensor is measured with respect to a ref- MEG systems measure the magnetic field
erence site. Clinical EEG studies often use (or the gradient, respectively) at a given
bipolar recordings in which a voltage gradi- location, with no external reference needed.
ent is measured between paired electrodes. MEG recordings, however, pose stronger
Most frequently used today is the com- requirements than EEG with respect to
mon reference, in which all of the record- the shielding of the recording environment.
ing electrodes measure the voltage gradient Whereas in EEG research a Faraday cage is
at their location against the same metric, considered state-of-the-art shielding against
derived from one or more reference elec- stray electric currents, MEG requires a mag-
trodes. The location of the common ref- netically shielded participant room. Such a
erence electrode(s) is critical to the inter- room excludes almost all of the fluctuating
pretation of the resulting scalp distribution external fields, which are several orders of
of the measured voltage. Ideally one would magnitude greater than the neuromagnetic
place the reference at a site that is elec- fields (Cohen, 1972). The degree of shield-
trically silent, allowing an estimate of the ing tends to increase with the number of
neural activity at each of the recorded sen- wall layers, made of non-ferromagnetic met-
sor locations. Thus, distant and presumably als such as aluminum.
electrically “quiet” regions have been used as In the majority of cases, electrophysiolog-
reference sites, such as a sensor mounted on ical signals are continuously recorded and
the tip of the nose, the front of the forehead, digitized at an even rate – the sampling
or on the earlobes. Nonetheless, both cra- frequency. For many applications in affec-
nial currents and extracranial sources (e.g., tive neuroscience, sampling frequencies of
muscle activity) can affect current flow at 250 Hz allow accurate reconstruction of
112 ANDREAS KEIL
brain activity over time. By contrast, study- ongoing raw waveforms during task perfor-
ing high-frequency oscillatory activity and mance are prima facie properties of EEG
very brief transient responses requires higher and MEG – they are obvious when visu-
sampling rates (e.g., 1,000 Hz or higher), ally inspecting the ongoing electrophysio-
which afford sufficient temporal resolu- logical time series. Because communication
tion for monitoring very fast neural events. within and between coupled neuronal pop-
Knowing the fundamental physics of sam- ulations of neurons involves excitatory as
pling and digital filtering is certainly benefi- well as inhibitory connections, the overall
cial to affective neuroscientists interested in activity of the network in time is likely to
using electrophysiology, but is outside the be oscillatory in nature. Oscillatory activ-
scope of this chapter. Readers are referred ity represents a general feature of other bio-
to the excellent overviews and tutorials in logical large-scale systems as well, such as
Cook and Miller (1992) and in Nitschke, motor or cardiovascular systems (Haken,
Miller, & Cook (1998). 1983). Given its salience even with basic
After the electrophysiological time series recording setups, the oscillatory character of
are recorded, several sources of noise may EEG and the relationship between different
still be present in the signal. Extracranial types of oscillations and mental processes
electrical activity that leads to artifacts in the were the focus of pioneering work in EEG
measured EEG include eyeblinks, eye move- (Berger, 1969) and MEG research (Cohen,
ments, muscular activity, external electro- 1972). In the affective neurosciences, spec-
magnetic noise, sweating, and heart rate tral analyses of extended periods of resting
potentials. Such artifacts are often identi- and task-related EEG have been particularly
fied in a process of visual inspection and fruitful, leading to the establishment of an
then discarded or corrected by appropriate entire research field concerned with later-
algorithms. In traditional approaches to arti- alized alpha band power as a predictor of
fact correction, EEG segments (or sensors) affective states or traits (Davidson, 1995).
containing artifacts or contaminated sensors Open-source software packages that include
are completely removed from the data, lead- sophisticated ways of calculating and quan-
ing to data attrition. More recent methods tifying frequency-domain aspects of elec-
try to retain as much of the recorded data trophysiological time series include the
as possible by mathematically modeling cer- fieldtrip toolbox (http://fieldtrip.fcdonders.
tain types of artifacts (e.g., eyeblinks), and nl/start), EEGLAB (http://sccn.ucsd.edu/
then correcting (i.e., recalculating) a version eeglab/), and EMEGS (http://www.emegs.
of the original EEG/MEG that estimates org/), all written in Matlab code and avail-
the true signal without the targeted arti- able to researchers for download at no
fact (Ille, Berg, & Scherg, 2002; Junghofer, cost.
Elbert, Tucker, & Rockstroh, 2000). Inde- The frequency spectrum of a sufficiently
pendent component analysis (Makeig, Jung, long data segment can be obtained with a
Bell, Ghahremani, & Sejnowski, 1997; dis- host of different methods, but traditional
cussed later in this chapter) is increasingly Fourier-based methods represent the most
used for this purpose and has shown the popular and most prevalent approach. These
capability to identify, isolate, and remove methods are applied to time-domain data
unwanted signals from the data. (where data points along the x-axis repre-
sent a temporal sequence) and transform
them into a spectral representation (where
The Measures: What to Record and data points represent different frequencies),
What to Look for in Studies of Emotion called the frequency domain. In princi-
ple, the discrete Fourier transform (DFT,
Spectral Analyses of Ongoing Activity
often implemented as the fast Fourier trans-
As evident from the historic overview, form, FFT, in commercial analysis packages)
changes in the spectral properties of the can be illustrated by a simple sequence of
ELECTRO- AND MAGNETO-ENCEPHALOGRAPHY IN THE STUDY OF EMOTION 113
steps: First, sine and cosine waves at the consequence: The data epoch that is submit-
frequencies of interest are multiplied to the ted to DFT is regarded as a segment of an
measured digitized signal, and the resulting infinite-length time series with stable peri-
values are summed over time. This leads odic properties over time. Hence, DFT will
to greater weighting of aspects of the time result in a summary spectrum of the entire
series that follow the shape of a given sine signal that can be regarded as an average over
or cosine wave, and the resulting sums (one all the periodic and nonperiodic processes
for the sine, one for the cosine) are thus a in the time segment of interest that match
measure of how much variance there is in the sine and cosine templates employed.
the original signal for each frequency. The Information as to whether any nonstation-
information contained in the sine and cosine ary change in the original time series con-
parts of the signal can be used to estimate tributed to the resulting frequency spec-
the spectral power, by calculating the mod- trum will be lost, as illustrated in Figure 4.1.
ulus (here, the square root of the sum of the Electrophysiological time series from affec-
squares) of the sine and cosine parts. In addi- tive neuroscience studies are often not sta-
tion, the phase of the measured signal can tionary. For instance, the presentation of an
be derived from the relationship between emotional stimulus may result in a transient,
the sine and cosine parts, using the arctan- nonperiodic change in the signal, which does
gent function. Often, the phase is consid- not meet the stationarity assumption when
ered a measure of latency or temporal dis- submitted to DFT. Interpretation of fre-
placement of the oscillation relative to the quency spectra recorded during emotional
sine function, again at a specific frequency. engagement (e.g., when listening to music
Alternative approaches use different basis or viewing film clips) should take this limi-
functions other than sine and cosine waves tation into account. Methods – experimental
and therefore have different weighting rules, and computational – are available to avoid
but rely on the same principle of model- nonstationarity. They include the selection
ing the desired oscillation by analytic time of stationary segments as evident by for-
series resembling the oscillation of interest. mal statistical testing. For instance, the aug-
Although widely used, not all of the prob- mented Dickey-Fuller test (see e.g., Elliott,
lems relevant for estimating spectral proper- Rothenberg, & Stock, 1996) has been estab-
ties of EEG and MEG by means of Fourier lished as a means to examine electrocortical
transform are known to affective neurosci- time series for (non)stationarity. In addition,
entists or reported in research reports. Let us de-trending methods such as the subtrac-
consider a few interesting issues that inform tion of a linear regression line across sam-
but also constrain the use of frequency- ple points are often used to remove slow
domain techniques in affective neuroscience aspects of nonstationary change (e.g., slow
research. drifting signals) from a time series. Exper-
imental approaches for obtaining station-
STATIONARITY OF THE TIME SERIES ary neurophysiological time series include
One important premise underlying all steady-state stimulation (see “Applications”)
Fourier-based procedures is the assumption and prolonged stimulus exposure.
that the signal is stationary. The concept of
temporal stationarity implies that a series SPECTRAL LEAKING AND THE
of values has a stable mean and variance UNCERTAINTY PRINCIPLE
over time, which on the level of physiol- Because of the properties outlined earlier,
ogy may mean that the underlying neural the quality of the Fourier-based frequency
process does not change qualitatively dur- spectrum of EEG and MEG depends on
ing the epoch of interest. In the formulation the periodicity and duration of the signal
of Fourier analysis, sine and cosine waves – submitted to the DFT. As an important
which are of infinite length – are used to constraint, the frequency resolution of the
describe the signal. This has an important spectrum is inversely related to the time
114 ANDREAS KEIL
Figure 4.1. Fourier analysis of electrophysiological time series. Middle Panel (A): A noisy time domain
signal was constructed to resemble an EEG/MEG with strong alpha contribution. To this end, a 10-Hz
sine wave was superimposed with real MEG noise, filtered between 12 and 50 Hz. The first 500 ms of
the signal contain the 10-Hz oscillation plus noise; the second 500 ms contain an offset of .5 microV
and an additional 25-Hz oscillation (sine wave) having half the amplitude of the 10-Hz signal. Note
that this second oscillation is not visible to the unaided eye at the noise level present in the data. This
time series looks quite regular, but significantly fails to be stationary (Augmented Dickey-Fuller Test).
The cosine-square function used for windowing is shown overlaid to the nonwindowed signal (thick
gray lines). Top Panel (B): Discrete Fourier transform (dashed line) of the entire time series (1,000-ms
duration) yields 1-Hz frequency resolution (see text). The addition of a cosine-square taper window
function reduces the amount of noise in the signal (solid line) as it attenuates on- and offset artifacts.
This analysis fails to detect that the 25-Hz oscillation is present only in the second half of the time
domain signal. Bottom Panels (C): Discrete Fourier transforms of the first and second half of the time
domain signal. This analysis yields a frequency resolution of 2 Hz (each epoch is 500 ms in duration).
Thus, the 25-Hz oscillation in the second part of the signal is detected, but is shown at 24 and 26 Hz
(leakage into neighboring frequency bins), due to the absence of a 25-Hz bin. Also shown with the
shorter data segments in panel C: Windowing reduces leaking and the offset at 0 Hz (i.e., the so-called
DC component).
resolution. Submitting short epochs of tainty principle, which states that time local-
EEG/MEG to DFT (high time resolution) ization trades off against frequency local-
leads to large steps in the frequency-domain ization. For instance, submitting a time
representation of the signal (poor frequency segment of 500 ms to DFT results in a
resolution), whereas submitting longer time frequency resolution of 2 Hz, with the max-
epochs allows for precise identification of a imum frequency meaningfully represented
given frequency at the cost of temporal reso- being the Nyquist frequency (i.e., 50% of the
lution. Thus, this problem can be regarded a sampling rate). Formally, the spectral reso-
variant of the Heisenberg (or Fourier) uncer- lution, measured as the size of the available
ELECTRO- AND MAGNETO-ENCEPHALOGRAPHY IN THE STUDY OF EMOTION 115
frequency steps f, is a simple function of signals have discontinuities at the ends of the
the signal duration: f (in Hz) = 1,000/dura- measured time. When the FFT assumes that
tion (ms). In practice, this means that to the signal repeats, it will therefore introduce
achieve a given resolution in the frequency discontinuities at the edges, which are not
domain, researchers should aim to use sta- really present in the signal (see Figure 4.1).
tionary time segments of sufficient length. Because sharp discontinuities have broad
For instance, time segments with a 500-ms frequency spectra, they cause the signal’s
duration will not enable one to make distinc- frequency spectrum to spread, resulting in
tions between the 25-Hz and 26-Hz activity increased leakage and/or misrepresentation
in a signal, because only frequency steps at of frequencies.
0, 2, 4, 6 Hz, and so forth, will be available, This problem is addressed by so-called
given the frequency resolution of 1,000/500 window functions that attenuate the signal
= 2 Hz. As shown in Figure 4.1, true oscilla- of interest at the edges. “Windowing” a sig-
tory activity at a frequency not represented nal thus means multiplication of the signal
in the spectrum (e.g., 25 Hz in the previ- with a weighting function that has low val-
ous example) shows at the neighboring fre- ues at the beginning and end. Most window
quencies in the spectrum (24 and 26 Hz), functions provide greater weight to events
thus smearing the appearance of the signal at the center of the signal, with decreas-
in the frequency domain. If a researcher is ing sensitivity to events at the edges. Thus,
interested in a given a priori frequency, par- they tend to have a symmetrical, tapered
ticular care should be taken when selecting shape. In Figure 4.1, we used a simple cosine-
the duration of the to-be-analyzed signal to square function (illustrated as thick gray
ensure that it is covered accurately with the lines in the middle panel) to address this
resulting frequency spectrum. issue, leading to reduction of noise over-
all (solid lines). Other window functions,
WINDOWS AND OVERLAPPING SEGMENTS such as Hanning, Hamming, Kaiser, and
Method sections of affective neuroscience so forth, differ in shape and have spe-
manuscripts on spectral properties of cific applications in digital signal process-
EEG/MEG often refer to windowed FFTs, ing, but they serve the same purpose of
tapers, and zero-padding, but the reasons providing desired weighting functions along
for selecting these parameters in a partic- a time series. Hence, windowing can be
ular manner are often opaque. In fact, when harnessed to enhance temporal localization
using convenient interactive analysis tools, when many adjacent Fourier transforms are
novice users often face difficulties when they computed for one time series, leading to a
are asked by a pop-down menu to decide spectrogram – the representation of spectral
whether they would like to use “Hamming,” power as it changes over time in several fre-
“Hanning,” “Welch,” or other types of win- quency bands. Often, windows are shifted
dows/procedures. Although a complete dis- along the signal with an overlap of 50% to
cussion of these issues is outside the scope ensure sensitivity to each spectral event in
of this chapter, let us consider a few basic the data and avoid exclusion of processes
issues that affect the looks and the inter- around the edges of the shifted windows.
pretation of spectral domain data in the A similar approach is used in the so-called
affective neuroscience. For a more compre- Welch periodogram method, in which over-
hensive description, readers are referred to lapping windows are moved over the sig-
Nitschke et al. (1998). Because Fourier anal- nal and the resulting spectra are averaged
ysis is designed to assume infinite repetitions to increase the signal-to-noise ratio and fur-
of the signal, any short segment of data that ther attenuate effects of noise and edges.
is cut out of the ongoing signal and submit- An excellent overview of window functions
ted to DFT is represented as repetitive; that and frequency-domain issues in EEG/MEG
is, the segment within the measured time research is given by Nitschke and colleagues
repeats for all time. In practice, most real (1998).
116 ANDREAS KEIL
Paralleling fMRI research, difference wave- nent analysis techniques with real and sim-
forms between two experimental conditions ulated data and has provided specific rec-
are sometimes used to highlight a neural dif- ommendations along with a tutorial descrip-
ference between situations and events that tion and open source code allowing their
were kept identical, except for one aspect of application. Likewise, the EEGLab toolbox
interest that was manipulated by the exper- (Delorme & Makeig, 2004) contains useful
imenter. applications to conduct ICA and combine
Related to this approach, researchers it with time-frequency analysis, single-trial
have conducted statistical tests for each sen- visualization, and source estimation.
sor (and possibly time point) available, com-
paring experimental conditions or groups. LATENCY MEASURES
The resulting parameters are then displayed In addition to amplitude measures, the
in a color-coded fashion as a scalp map or latency at which specific ERP components
projected to a brain volume. This mapping occur may be relevant to hypotheses in
approach does not reduce the information affective neuroscience research. Component
contained in the ERP/ERF matrix because latency is often defined as the point in
it can be conducted for each sensor-sample time at which the peak of the deflec-
pair in the n x t matrix, but it is associ- tion occurs or as the time at which a
ated with alpha error accumulation. Thus, certain criterion is reached (e.g., 50% of
false-positive tests must be expected with the maximum amplitude). More elaborate
large matrices, and means for adjusting the approaches involve use of lagged cross-
alpha level are needed. Recent advances correlations, which estimate the relative
in the field of statistics have led to novel displacement of an entire waveform with
applications such as permutation tests for respect to a standard waveform by calcu-
scalp topographies and time points (Blair & lating the cross-correlation function at dif-
Karniski, 1993), in which a test statistic is ferent lags. Accurate estimates of latency
evaluated against an empirical distribution are highly dependent on the signal-to-noise
resulting from large numbers of tests con- ratio of the individual ERP, and thus sev-
ducted over the same data, but with shuf- eral methods have been proposed to obtain
fled (permutated) conditions. Two recent information regarding statistical significance
reviews (Groppe, 2011; Maris, 2012) give of a given latency difference. In many sit-
an overview of the current thinking in uations researchers may consider use of
ERP/ERF statistical testing and are recom- the jackknife approach (Miller, Patterson, &
mended for the interested reader. Ulrich, 1998) to examine latency differences
In addition to neurophysiological rea- between experimental conditions or groups.
soning and mass testing of all information Jackknifing is often preferred over individual
available, multivariate statistical approaches peak-picking methods because it has been
such as principal component analysis (PCA) shown to be more sensitive to real latency
or independent component analysis (ICA; differences than single-subject-based scoring
see Makeig, Debener, Onton, & Delorme, methods while at the same time being less
2004) have been used to extract time points affected by noise. This feature is achieved
and scalp regions that are sensitive to the by using averages rather than individual par-
experimental manipulation (Foti, Hajcak, & ticipant data as the observation variable:
Dien, 2009). Both PCA and ICA are fam- Jackknife-based statistics involve recompu-
ilies of procedures that aim to statistically tation of the desired test statistic, leaving
de-correlate or factorize the spatiotemporal out one observation at a time from the
EEG (or MEG) data and then extract tem- sample set. New averaged waveforms are
poral and spatial features that are unique formed to replace each of the participants’
and specific, so they may be attributed to a individual waveforms or time-varying spec-
given brain process or artifact. Dien (2010) tra, or other event, for each experimental
has explicitly compared different compo- condition. Each of these waveforms then
118 ANDREAS KEIL
represents a grand mean across all partici- ERP source estimation and animal models
pants but one. The latency of each event of vision.
of interest can be scored as the point in
time when a criterion is reached. Jackknifed STEADY-STATE EVOKED
statistics are then calculated and appropri- POTENTIALS/FIELDS
ate corrections are applied to account for A special case of ERPs are steady-state
the averaging (Miller et al., 1998). potentials, elicited by oscillating stimuli –
visual, auditory, or somatosensory. In the
TOPOGRAPHY visual domain, steady-state visual evoked
With the advent of dense-array EEG/ERP potentials (ssVEPs) can be used as a con-
recordings, the interest in localization of the tinuous measure of visual cortical engage-
generators underlying the scalp-recorded ment when processing a visual stimulus
EEG has dramatically increased. Although (Müller, Teder-Salejarvi, & Hillyard, 1998).
the spatial sensitivity of ERPs is poor com- They represent responses to stimuli modu-
pared to hemodynamic imaging techniques lated in luminance (i.e., flickered), in which
such as fMRI or PET, the spatial informa- the frequency of the electrocortical response
tion inherent in multichannel ERP record- recorded from the scalp equals that of the
ings can be harnessed, using appropriate driving stimulus. Of significant advantage,
algorithms, to guide careful estimates as to the oscillatory cortical response has a known
their cerebral origin. I address the topic of frequency and can thus be reliably separated
source estimation later in more detail. An from noise and quantified in the frequency
alternative use of the topographical informa- domain. Another advantage of this paradigm
tion from multichannel ERPs is to inform so- is that multiple stimuli flickering at different
called microstate analysis (Pascual-Marqui, frequencies can be presented simultaneously
Michel, & Lehmann, 1995). Microstates are to the visual system, but their electrocorti-
hypothetical constructs extracted from the cal signature can be separated (a technique
ERP n x t matrix that reflect periods of referred to as “frequency tagging”). With
time in which the topographical distribu- respect to emotional stimulus features, it
tion is stable, thus potentially reflecting a has been demonstrated that highly arous-
specific state of the central nervous system. ing affective stimuli generate greater ssVEPs
These states can be variable in duration, but than neutral stimuli in visual and frontopari-
are conceptualized to be non-overlapping etal networks (Keil, Gruber, et al., 2003),
in time. Thus, microstate analysis results in pointing to facilitation of attention and per-
a temporal sequence of topographies char- ception when processing visual objects that
acteristic of each state. In affective neuro- bear high motivational relevance (Moratti,
science, microstate analysis has been used to Keil, & Stolarova, 2004).
demonstrate the temporal and spatial locus
of ERP activity related to hedonic valence
Time-Frequency Analyses
and the emotional intensity of affective pic-
tures (Gianotti et al., 2008). Other work We discussed earlier that EEG/MEG record-
has used this approach to segment the ERP ings mirror the activity at hundreds of thou-
time course of face processing as a function sands of cortical neurons, which are engaged
of expression and observers’ affective traits in an interdependent and oscillatory fash-
into meaningful temporal regions (i.e., the ion. This activity may change its frequency
microstates), which were then submitted composition over time, as a function of
to source estimation (Pizzagalli, Lehmann, subjective state or external demands. Spec-
Koenig, Regard, & Pascual-Marqui, 2000). In tral analyses as shown earlier cannot fully
support of the microstate concept, the pre- address the issue of overlapping and rapidly
viously mentioned visual ERP work showed changing neural oscillations during emo-
that the sequence of microstates converged tional processing. To fill this gap, time-
with early processing stages as suggested by frequency analyses have been developed
ELECTRO- AND MAGNETO-ENCEPHALOGRAPHY IN THE STUDY OF EMOTION 119
(see Tallon-Baudry & Bertrand, 1999, for a (Moratti, Clementz, Ortiz, & Keil, 2007).
review). They allow researchers to study the Importantly, they also allow follow-up anal-
changes of the signal spectrum over time, yses using algorithms that are defined best in
taking into account the power (or ampli- the frequency domain, such as phase-locking
tude) of the signal at a given frequency, as across single trials or phase-locking across
well as changes in the phase (latency). Such recording sites – indices that are sensitive to
a comprehensive analysis of neural popu- the connectivity and coherence underlying
lation activity is helpful given that not all the electrophysiological data recorded. The
meaningful neural activity is exactly time- spectral range of time-frequency techniques
and phase-locked to an event. For instance, is only constrained by the sampling rate. As
based on recordings in experimental ani- opposed to band-pass filtering as used in ear-
mals, Galambos (1992) proposed to dis- lier studies, most time-frequency transforms
tinguish among (1) spontaneous rhythms, do not require that a frequency range of
which are not related to external stimuli; interest is known a priori, thus yielding a
(2) evoked responses, which are elicited by rich and unconstrained database.
and precisely time-locked to the onset of an How are time-frequency analyses imple-
external stimulus; (3) emitted oscillations, mented? The uncertainty principle dictates
which are time-locked to a stimulus that that time and frequency cannot be measured
has been omitted; and (4) induced oscilla- both at arbitrary accuracy, but that there is a
tions that are initiated by but not time- and tradeoff between the two domains. Wavelet
phase-locked to the onset of a stimulus. transform represents a group of methods
As we saw earlier, the most widely used that allow the researcher to use variable
electrocortical parameters in the affective temporal (and thus frequency) resolution
neurosciences, the ERP and the ERF, are across different frequencies (Tallon-Baudry
based on time-domain averaging across stim- & Bertrand, 1999). Appropriate compro-
ulus events. In Galambos’ terms, the ERP mises between time and frequency resolu-
thus measures time- and phase-locked pro- tion can be defined for different frequencies,
cesses (i.e., evoked activity). In contrast, resulting in a near-optimum representation
measures involving frequency-domain aver- of the signal. A popular wavelet in human
aging across events are sensitive to induced, electrophysiology has been the continu-
in addition to evoked neuronal activity, ous Morlet wavelet, first suggested for use
because the single trials are first trans- in geophysics and introduced into neuro-
formed into the time-frequency plane and science by Olivier Bertrand in the mid 1990s
then averaged. This procedure prevents (Bertrand, Bohorquez, & Pernier, 1994). The
attenuation of neural activity that is not Morlet wavelet is a sine wave segment
aligned with the affective event in the multiplied by a Gaussian window function
time domain. Further advantages of time- that is dilated and extended to be sensi-
frequency approaches to the study of oscil- tive to different frequencies, also resulting
latory activity lie in their more immediate in shorter/longer duration of the wavelet,
relationship with neurophysiological pro- depending on the dilation/extension. As a
cesses and their ability to retain single-trial consequence, the width of the wavelets
information. For instance, time-frequency in the frequency domain changes as a
representations of EEG/MEG signals have function of the analysis frequency, while
allowed a description of temporal dynam- having a constant ratio between the fre-
ics that overlap in the time domain, but quency resolution and the time resolution.
are located in different frequency ranges With standard parameters, time resolution
(Kranczioch, Debener, Maye, & Engel, for high frequencies is superior compared
2007). They have enabled researchers to to low-frequency ranges, where frequency
characterize the effects of prestimulus and resolution is high, but time resolution is
ongoing activity on evoked activity, which is coarse. Conversely, spectral resolution in
difficult with time-domain approaches alone higher frequency ranges is low, as is the time
120 ANDREAS KEIL
resolution in the lower bands. Importantly, Ray, 2007). The PLI is bounded between
these properties are consistent with the 0 and 1; 0 indicating random distribution
behavior of oscillatory activity as observed of phase across trials and 1 indicating per-
in the mammalian cortex: Bursts of high- fect identity of the phase across trials, at
frequency oscillatory activity tend to appear a given time and frequency. In the domain
reliably after a given stimulus, but may vary of visual perception, time-frequency analy-
in terms of center frequency, phase, and sis of signals collected at occipital sensors
exact latency (Singer et al., 1997). Thus, has converged to show an early, evoked,
the high temporal resolution at the cost lower frequency burst around 80 ms that
of frequency smearing at high frequen- is followed by a late, induced, higher fre-
cies – as afforded by wavelet analysis – quency oscillation. Figure 4.2 illustrates this
matches the physiological properties of cor- for the oscillatory activity in response to
tical networks. The high temporal sensitiv- a conditioned stimulus (here: a gray-and-
ity of Morlet wavelets in the upper spectral white grating stimulus) predicting a loud
range helps identify brief epochs of high- noise. Both the early evoked and the late
frequency oscillations (above 20 Hz), which induced gamma response have been related
are assumed to occur during formation of to emotional processing; for instance, during
percepts or memories, activation of learned classical conditioning of brief visual stimuli
associations, and preparation and execution (Keil, Stolarova, et al., 2007).
of actions. These high-frequency phenom-
ena have often been referred to as gamma-
band activity. Gamma-band and other oscil- Source Analysis
lations are a potentially useful tool for
examining the network aspects of emotional As noted previously, knowing the topogra-
processing in the brain (see, e.g., Keil, Sto- phy of a specific ERP component does not
larova, Moratti & Ray, 2007). Parameters provide specific information regarding the
of lower frequency electrocortical dynamics original brain generators (i.e., the sources)
are often seen as necessary to complete this of the electrical activity that have been mea-
picture, and of course they provide impor- sured over the scalp. Nonetheless, since the
tant information on their own. For instance, advent of dense array EEG/MEG record-
modern views of alpha oscillations (8–12 Hz) ings, there have been computational efforts
relate these rhythms not only to “idling” to estimate the location of brain electric
states of the brain but also to diverse func- sources. It helps to be familiar with a few
tions comprising affective, motor, and mem- conceptual issues that affect the use of
ory processes (Klimesch et al., 2006). source analyses techniques in affective neu-
As mentioned earlier, wavelet analysis roscience.
of single trials allows researchers to quan- The inverse problem. EEG and MEG in
tify the amount of phase-locking over many principle can be regarded as two-dimensio-
stimulus repetitions. In addition to time- nal projections of a three-dimensional reality
varying spectral power, it is possible to quan- to a surface (the scalp or the MEG helmet),
tify the intertrial phase-locking of the neu- and it has long been known that an infinite
ral oscillations measured with the wavelet number of three-dimensional source configu-
family. To this end, the normalized, com- rations exists that will produce a given two-
plex representations of the time-frequency dimensional projection. This is referred to as
matrices are averaged according to the algo- the inverse problem: It is not possible to
rithm described (e.g., in Tallon-Baudry & unambiguously infer the three-dimensional
Bertrand, 1999). This procedure results in origin of a two-dimensional representation,
a measure of phase stability across trials, a fact that is often illustrated by the shadow
for each time point and frequency, which cast by an object on a screen. A given rectan-
is referred to as the phase-locking index gular shadow can be the result of someone
(PLI; see e.g., Keil, Stolarova, Moratti, & holding up a teacup or a pocket calculator.
ELECTRO- AND MAGNETO-ENCEPHALOGRAPHY IN THE STUDY OF EMOTION 121
40 0
−.1
20
−100 0 100 200 300 400 500 600 700 ms
40 .2
.1
20
−100 0 100 200 300 400 500 600 700 ms
Figure 4.2. Time-frequency analysis. Top panel: Grand mean (n = 16) baseline-corrected time by
frequency plots obtained at a posterior EEG electrode when participants viewed a visual cue (the
CS+) predicting an unpleasant noise (the US) during a classical conditioning paradigm. Values were
obtained by wavelet analyses of the single-trial source waveforms. The plots show time running from
left to right and frequency from bottom to top. The amount of spectral amplitude change at a given
time and frequency is coded by gray scale, with increase over the baseline level coded in increasingly
dark shades. A pre-stimulus segment was used as the baseline, and amplitude values indicate relative
change after subtracting the baseline mean. Note the two oscillatory responses to the visual cue: an
early low-frequency response and a late higher-frequency response Bottom panel: Grand mean time
by frequency representation of the phase-locking index for the same experiment. Phase-locking
indices reflect the amount of phase stability across trials for a given time and frequency. Again, the
amount of spectral phase-locking across trials is coded by grayscale. High phase-locking is evident for
the first low-frequency oscillatory response and is lacking for the late, high-frequency response.
In practice, this means that there is no source density (CSD) mapping approach
mathematically unambiguous way to calcu- that is mathematically based on the Laplace
late the neural sources of an ERP/ERF field operator (Tenke & Kayser, 2005). The Lapla-
distribution. Nonetheless, several methods cian can be conceived of as the second spatial
have been developed that use additional derivative of the voltage map, thus showing
knowledge to constrain the infinite solution the change of the voltage gradient across the
space or to infer gross source configurations scalp. Assuming that there are no sources in
by means of simple data transformations. the scalp itself, the CSD map will be propor-
Mapping techniques. The extent of the tional to the radial current flow in or out of
spatial sampling of scalp voltages affects the the skull at a given site. Because it is reason-
ability of researchers to use topographical able to assume that most generators of the
information of scalp distributions to infer ERP/ERF are located in near-surface cortex,
sources. By taking into account the physics this type of data transformation allows a first
of current flow in a conducting volume approximation of the underlying electrocor-
together with sufficient spatial sampling, tical sources.
transformations of the topographical map Source estimation techniques. More elabo-
are available that highlight the current flow rate source analysis techniques harness the
on the scalp and allow one to infer possible spatial and temporal information inherent
sources of brain-electric activity. Among the in multichannel recordings using different
most widely used techniques is the current algorithms to guide estimates as to their
122 ANDREAS KEIL
cerebral origin. One of the first techniques early visual cortex engagement (Pourtois,
used was to fit the surface potential distri- Grandjean, Sander, & Vuilleumier, 2004).
bution by a single equivalent current dipole The development of distributed source
(Scherg & Von Cramon, 1986). In this proce- approaches to address the inverse problem
dure, the assumption is that the voltage map is ongoing (Ding & He, 2008), and current
at the scalp reflects the activity of a single discussions include methods for including
neural source with a positive and negative neurophysiological rather than physical con-
pole. Knowledge about the possible distor- straints to guide selection from the infinite
tion of this signal is used together with other solution space (Grave de Peralta, Hauk, &
neuroanatomical information to systemati- Gonzalez, 2009).
cally fit a source dipole that is most simi- In addition to the methods outlined ear-
lar to the measured voltage distribution. To lier, spatial filtering techniques such as
this end, a forward calculation is performed beamforming have been implemented in
many times, which computes various volt- MEG, and increasingly in EEG research,
age distributions based on many different typically in conjunction with realistic mod-
dipole locations and orientations, and the els of a participant’s head and brain as
dipolar source with the location, strength, obtained from magnet resonance images.
and orientation that lead to the distribution These head models are then filled with a
most similar to the measured signal is iden- grid of voxels at meaningful locations, used
tified. Generally, MEG has been shown to as reference points for the source estimation.
have slightly better spatial sensitivity than Beamforming does not attempt to model the
EEG (Cohen & Cuffin, 1991), although puta- entire scalp field or minimize residual vari-
tive differences in the depth sensitivity are ance. By contrast, a beamformer is a col-
still subject to empirical testing (Cohen lection of spatial filters, each being opti-
et al., 1990). mally tuned to a voxel in the source volume.
So-called distributed source modeling Raw MEG (and EEG) data can be projected
procedures have been proposed as a means through these spatial filters to obtain an esti-
for enhancing the resolution of electro- mate of electrical activity at each voxel.
physiological data without constraining the Correlated electrocortical activity at two
inferred electric activity to point sources different locations is suppressed by this tech-
or single dipoles. In these procedures, hun- nique, further illustrating that the choice of
dreds of potential dipoles at various loca- any source estimation technique must fol-
tions in a three-dimensional volume are low consideration of its suitability for a given
included in the inverse modeling process, type of data and experimental design.
and a unique solution is achieved by adopt- As evident in this volume, work in
ing certain border conditions, such as favor- the animal model as well as studies with
ing the solution with minimum power hemodynamic imaging have highlighted the
of the source currents (Hämäläinen & importance of deep brain structures such
Ilmoniemi, 1984). In EEG-based affective as the amygaloid bodies, insulae, nucleus
neuroscience research, LORETA (Pascual- accumbens, the bed nucleus of the stria
Marqui, Michel, & Lehmann, 1994), has terminalis, and others for affective behav-
been popular; it is a distributed source esti- ior and experience. An ongoing debate in
mation approach that maximizes smooth- EEG/MEG research on emotions in humans
ness and minimizes localization error under concerns the extent to which electrophysio-
ideal conditions. Such approaches have been logical methods can specifically identify and
employed to identify the sequence of cor- quantify neural activity in those deep struc-
tical activation during perception of an tures. A number of authors have reported
aversive conditioned stimulus (Pizzagalli, for instance that recording activity from the
Greischar, & Davidson, 2003) or to test amygdala during emotional engagement is
whether the orienting of attention to emo- possible with MEG (see, e.g., Bayle, Henaff,
tional face stimuli is accompanied by & Krolak-Salmon, 2009; Luo et al., 2009) and
ELECTRO- AND MAGNETO-ENCEPHALOGRAPHY IN THE STUDY OF EMOTION 123
even EEG (Homma, Nakajima, Toma, Ito, ticipants with larger amplitudes on one trial
& Shibata, 1998), but others have doubted also have larger amplitudes on subsequent
these claims. Because of the nature of the trials, compared to other participants) or
inverse problem, validation of deep source other repetitions of the task (i.e., blocks or
findings is difficult without a reference data conditions). It can easily be assessed using
set from another measurement modality. Cronbach’s coefficient alpha. Coefficients
Such data sets are often given in studies may be calculated for each sensor and/or
of epileptic spike activity, in which find- brain location, resulting in topographical
ings obtained with MEG/EEG-based source maps that reflect the reliability of the
estimation can be cross-validated against measure under consideration in a topo-
intracranial recordings, neuroimaging, and graphic fashion. Such efforts exist for oscil-
the surgical findings. Although a system- latory activity in the gamma range (Frund,
atic large-scale validation study capitaliz- Schadow, Busch, Korner, & Herrmann,
ing on this methodology has not yet been 2007; Keil, Stolarova, Heim, Gruber, &
conducted, this work strongly suggests a Muller, 2003), the steady-state visual evoked
cautious approach to deep source local- potential (Keil et al., 2008), and the P300
ization with noninvasive electrophysiology: component of the ERP (Ravden & Polich,
Although epileptic spikes are stronger elec- 1999) – all pointing to surprisingly high sta-
tric signals than those evoked during emo- bility and consistency of electrophysiological
tional processing, numerous reports have measures. The potential of these properties
failed to correctly identify known epilep- is yet to be fully employed.
tic foci located in the medial temporal The validity of electrophysiological para-
regions, based on extracranial MEG/EEG meters – their ability to index the intended
(e.g., Wennberg et al., 2011). brain process – is typically studied using
cross-correlation with external validation
variables, whether continuous or discrete. In
Psychometric Properties of EEG and the affective neurosciences, such validation
MEG Parameters has shown that the emotional modulation of
late positive ERP potentials in response to
The reliability and validity of electrophysio- affective pictures covaries with widespread
logical measures are not routinely reported cortical activation as measured by fMRI
or discussed in human neuroscience stud- (Sabatinelli, Lang, Keil, & Bradley, 2007).
ies. As an exception, the affective neuro- Other examples include establishing corre-
sciences have seen some discussions of the lations between electrophysiology and clini-
topic; for instance, as it relates to the relia- cal or trait measures (Davidson, Pizzagalli,
bility of affective trait measures extracted Nitschke, & Putnam, 2002). A substantial
from resting EEG (Tomarken, Davidson, body of work has adopted this strategy and
Wheeler, & Kinney, 1992). Although not has converged to suggest that brain-electric
tested for many variables, the reliability of and neuromagnetic parameters can be used
electrophysiological indices of affect is a as valid indicators of emotional engagement
topic of increasing importance, particularly (Cahn & Polich, 2006).
in the context of potential clinical applica- Similar to the question of reliability and
tions. As Simons and Miles point out in their validity, the sensitivity and specificity of
comprehensive review of reliability issues EEG/MEG derived measures for address-
in ERP research (Simons & Miles, 1990), ing issues in affective neurosciences have
indices of psychometric quality can be eas- not been widely studied empirically. It is
ily obtained when considering the consis- important to emphasize that an ERP com-
tency of a given brain measure from trial ponent does not exist independently of the
to trial or from block to block. Reliability is specific experimental context in which it
said to be high if the rank ordering of sub- is measured. Thus sensitivity of an electro-
jects remains stable across trials (e.g., par- physiological index to an affective process
124 ANDREAS KEIL
of interest is dependent on the way it was methods in the context of different research
measured. If an electrophysiological index questions.
with similar timing and topography is found
in different experimental contexts, care-
Event-Related Potentials/Fields When
ful experimentation is needed to conclude
Processing Transient Emotional
that the same latent process is responsible.
Stimuli
Such experimentation will involve system-
atic comparisons of the contexts prompt- A host of electrophysiological studies have
ing that ERP component within the same capitalized on the ERP or ERF elicited
participants. For instance, if a study finds by the onset of an emotional stimu-
that the amplitude of a visual P1 component lus, whether visual or auditory. Typically,
(which is often sensitive to spatial attention) researchers compare ERP/ERF waveforms
is enhanced with lateralized fear faces, this obtained by separate time-domain averaging
finding cannot be taken to indicate that spa- for different contents (e.g., pleasant, neu-
tial attention was paid to the fear face stim- tral, unpleasant). They then draw conclu-
uli. Rather, P1 amplitude can be affected by sions regarding the amount of discrimination
whether the stimulus is brighter, closer, and among contents, its timing, and its locus in
more salient in another context; has greater the brain. In the visual domain, early ERP
spatial contrast; or has other properties that work (Johnston & Wang, 1991; Radilova,
may lead to the observed change. 1982) has suggested that late positive poten-
tials (LPPs), appearing at around 300 ms
after stimulus onset, are enhanced when par-
Applications: Typical Paradigms for ticipants view emotionally arousing visual
MEG and EEG in the Affective stimuli, compared to neutral exemplars.
Neurosciences In the last decades, pictures taken from
the International Affective Picture System
As outlined earlier, the versatility of electro- (IAPS; Lang, Bradley, & Cuthbert, 2005)
physiological measures allows researchers to have frequently been used in ERP studies
address a host of experimental questions in of visual emotion processing. The IAPS is a
affective neuroscience, ranging from studies collection of more than 1,000 pictures and
of perception to attention to social cognition provides normative ratings regarding emo-
paradigms addressing high-level constructs tional arousal and hedonic valence associ-
such as ethnic biases or empathy. Accord- ated with each picture. Affectively arousing
ingly, the measures extracted from electro- IAPS pictures, pleasant and unpleasant,
physiological time series are diverse and are have been shown to engage motivational
used for different purposes in the experi- reflex physiology reflective of appetitive
mental process: They may provide timing or aversive/defensive response tendencies
information, complementing a behavioral or (Lang, 1994). Electrophysiological work has
physiological measure. They may index the established that passive viewing of affec-
amount of electrocortical activation during tively arousing, compared to nonarousing
different tasks or in different situations, and IAPS pictures, is associated with amplifica-
they can help generate hypotheses about tion (enhancement) of the LPP (Cuthbert,
neuroanatomical localization. To avoid fish- Schupp, Bradley, Birbaumer, & Lang, 2000).
ing for significant results in the wealth of This centroparietal ERP effect has been
multidimensional result matrices contain- related to heightened activity in occipi-
ing sensors, time points, and/or frequen- tal cortex (Keil et al., 2002) and has also
cies, researchers should select theoretically been observed in MEG recordings (Peyk,
meaningful dependent variables in an a pri- Schupp, Elbert, & Junghofer, 2008). The
ori fashion. The following section describes modulation of the LPP closely correlates
some of the most widely used paradigms, with other physiological measures of affec-
addressing the use of particular EEG/MEG tive processing, such as skin conductance;
ELECTRO- AND MAGNETO-ENCEPHALOGRAPHY IN THE STUDY OF EMOTION 125
it also has a strong positive linear relation- affective stimuli already at initial stages of
ship with the emotional arousal as indicated perception and has been reported for other
in self-report questionnaire data (Cuthbert, stimulus types as well, including faces and
Schupp, Bradley, Birbaumer, & Lang, 2000). words. For references, the reader may want
Figure 4.3 shows the typical scalp topogra- to read Chapter 13 where this response is
phy and time course of the posterior LPP extensively discussed.
that is enhanced for emotionally engaging, A large body of work has examined pro-
compared to neutral pictures. cessing of emotional facial expressions with
Other ERP/ERF components continue ERP/ERF methods (see Chapter 7). Over-
to be explored regarding their systematic all, these results have been mixed, but
relationship with the emotional content robust patterns of findings have emerged
of a given stimulus. For instance, several when studying participants high and low in
studies have reported that an early differ- social phobia (Bar-Haim, Lamy, Pergamin,
ence between ERPs to emotional (pleasant Bakermans-Kranenburg, & van Ijzendoorn,
and unpleasant) and neutral IAPS pictures 2007). For instance, several studies sug-
develops around 120–150 ms after stimulus gest that social anxiety is associated with
onset and extends up to 300 ms (Junghöfer, enhanced visual occipitotemporal P1 com-
Bradley, Elbert, & Lang, 2001; Schupp, ponents to faces in general, regardless of
Junghofer, Weike, & Hamm, 2003). Picture expression, and with heightened right tem-
complexity and potentially other physical poroparietal N170 components to angry and
properties of the stimulus contribute to this happy faces (Kolassa, Kolassa, Musial, &
modulation, raising the question of validity Miltner, 2007; Kolassa & Miltner, 2006;
(Bradley, Hamby, Low, & Lang, 2007). This Wieser, Pauli, Reicherts, & Muhlberger,
early differential ERP response has been 2010). Such findings are consistent with the
taken to reflect a processing advantage of notion that the fear relevance of a given
126 ANDREAS KEIL
stimulus is associated with facilitation in the co-workers (2008), ssVEPs were recorded
cortices specialized to process the features of to rapidly flickering squares (here used as
that stimulus (Keil, Stolarova, et al. 2007). a probe stimulus) superimposed on neutral
and emotionally highly arousing pictures,
and variations in ssVEP amplitude over
Paradigms with Probe Stimuli
time were calculated. As reflected in ssVEP
The abrupt presentation of a brief, intense amplitude and target detection rates, arous-
acoustic stimulus (e.g., a burst of white ing emotional background pictures with-
noise) elicits a reflexive startle response that drew processing resources from the detec-
is modulated by emotion. One component tion task compared to neutral exemplars for
of the startle response – the reflexive eye- several hundred milliseconds after stimulus
blink – is modulated by hedonic valence, onset. The amplitude was found to bear a
with larger eyeblinks elicited when process- close temporal relationship with accurate
ing unpleasant stimuli and smaller blinks target detection as a function of time after
when processing pleasant content. In con- stimulus onset.
trast, the electrocortical response to the The ssVEP approach also allows exam-
same startling probe varies with emotional ining the competition between two emo-
arousal, rather than hedonic valence: The tional and/or neutral stimuli, each of which
amplitude of the P3 component of the ERP can then serve as a probe or as a target
is smaller when viewing either pleasant or stimulus, if tagged with different frequen-
unpleasant, compared to neutral, pictures cies. This technique was adopted in a study
(Cuthbert et al., 1998; Schupp, Cuthbert, examining competition between two spa-
Bradley, Birbaumer, & Lang, 1997; Schupp tially separated facial expressions in high and
et al., 2004). The P3 is a positive compo- low socially anxious individuals (Wieser,
nent, which is prompted by task-relevant McTeague, & Keil, 2011). Two facial expres-
stimuli, and is also present in response to sions were flickered at different frequencies
novel or infrequent stimuli (Linden, 2005). (14 and 17.5 Hz) to separate the electrocor-
Using dense-array EEG recordings and a tical signals evoked by the competing stim-
source estimation procedure, Keil, Bradley, uli (“frequency-tagging”). Angry faces com-
et al. (2007) observed that P3 amplitude was pared to happy and neutral expressions were
indeed smaller when startle probes were linked to greater electrocortical facilitation
presented during emotional, compared to over visual areas in the high but not the
neutral, stimuli for both sound and pic- low socially anxious individuals. Using the
ture foregrounds. Source modeling indi- time information inherent in the ssVEP, it
cated a common frontocentral maximum of was found that heightened electrocortical
P3 modulation by affect. These data sup- engagement in socially anxious participants
port the notion that emotionally arousing was present after 200 ms and was sustained
stimuli transmodally attract resources, lead- for the entire presentation period. These
ing to optimized processing of the affective results, based on a continuous measure of
stimulus at the cost of processing concurrent attentional resource allocation, support the
information. view that stimuli of high personal signifi-
Sharing of limited resources between cance are associated with early and sustained
an emotional stimulus and a concurrent prioritized sensory processing.
probe within a modality and even between
spatially overlapping stimuli can also be
Classical Conditioning
assessed by means of frequency-tagged
ssVEPs. This approach allows researchers Learning the contingencies between threat
to continuously measure the time course of signals and the potential dangers that they
competition for processing resources arising predict is considered a ubiquitous adap-
from emotionally salient but task-irrelevant tive behavior across species (LeDoux, 1993).
stimuli while performing a foreground tar- Pavlovian fear conditioning is probably the
get detection task. In a study by Müller and best studied laboratory model for this type
ELECTRO- AND MAGNETO-ENCEPHALOGRAPHY IN THE STUDY OF EMOTION 127
of learning. During fear conditioning, a neu- regarding US delivery in a given trial, subse-
tral stimulus becomes an effective trigger quent work (Moratti & Keil, 2009) has sug-
(i.e., the conditioned stimulus, CS+) of gested that the cortical facilitation of fear
a conditioned fear response after having cue processing is determined by associative
been paired with an aversive event (i.e., strength and previous exposure to learning
the unconditioned stimulus, US), compared contingencies, rather than by cognitive pro-
with a stimulus that signals safety (CS-). It cesses such as the anticipation of the US.
has been consistently shown that the acti-
vation of primary sensory cortices related to
the modality of the CS+ increases after suc- Conclusion
cessful learning of the CS-US relationship
(Büchel, Coull, & Friston, 1999; Moratti, Large-scale brain electric activity can be
Keil, & Miller, 2006). measured using EEG/MEG and is a use-
The ERP technique was used in numer- ful index of specific processes related to
ous studies to study deep cortical (Pizza- emotional engagement, when considered in
galli et al., 2003) or sensory cortical (Sto- a meaningful experimental context. Elec-
larova, Keil, & Moratti, 2006) processes trophysiological work converges with ani-
associated with acquisition of the condi- mal studies, suggesting a central role of
tioned response. This work has demon- dynamic cortical networks in establishing
strated enhancement of the very early C1 widespread connectivity during affective
visual event-related component for threat- processing. These widespread networks may
related stimuli, compared to neutral stim- rely on oscillatory activity to provide short-
uli, and that this selective amplitude dif- term links between sensory and motor sys-
ference increases as learning progresses. In tems and to induce long-term changes in
terms of early oscillatory activity during clas- neuronal architecture. Multimodal record-
sical conditioning, a similar gradual increase ings and computational models of human
in amplitude and synchrony was seen specif- electrophysiology suggest that deep struc-
ically for the CS+ across conditioning ses- tures such as the amygdala and insula form
sions in the visual (Keil, Stolarova, et al. rapid and flexible links with neocortical
2007) and auditory modalities (Heim & areas during emotional engagement. Differ-
Keil, 2006). Converging evidence was found ences in emotional behavior and experience
when examining driven oscillatory activity may ultimately be determined by which
in the auditory system elicited by amplitude- structures link to particular areas in the
modulated acoustic CSs (Weisz, Kostadi- neocortex, as well as by the spatiotempo-
nov, Dohrmann, Hartmann, & Schlee, ral dynamics of cortical activation and inte-
2007), where sensory enhancement was gration. Dramatic technological and con-
observed for the CS+, particularly in par- ceptual advances have taken the field to a
ticipants with a greater fear response. This point where the physiology underlying the
parallels findings obtained with steady- modulation of extracranial fields is increas-
state evoked magnetic fields recorded using ingly understood. Thus, electrophysiologi-
MEG: Analysis of the heart rate response cal studies of emotion may move beyond
during classical conditioning separated sub- a descriptive analysis of waveforms to con-
jects into those who showed either acceler- sider the data as measures of in-vivo neuro-
ation or deceleration when presented with physiology.
the flickering CS+. Heart rate accelerators
showed increased ssVEF amplitude in visual
and parietal cortex during CS+ presenta- Outstanding Issues and Future
tion when awareness of the CS-US contin- Directions
gencies was not controlled (Moratti & Keil,
2005) as well as when fully aware (Moratti r To better exploit the rich information
et al., 2006). Manipulating both the num- in electrophysiological data, multidisci-
ber of learning trials and the expectation plinary research is needed that brings
128 ANDREAS KEIL
Blair, R. C., & Karniski, W. (1993). An alternative K. Hugdahl (Eds.), Brain asymmetry (pp. 361–
method for significance testing of waveform 87). Cambridge, MA: MIT Press.
difference potentials. Psychophysiology, 30(5), Davidson, R. J., Pizzagalli, D., Nitschke, J. B., &
518–24. Putnam, K. (2002). Depression: Perspectives
Borck, C. (2005). Hirnströme – Eine Kul- from affective neuroscience. Annual Review
turgeschichte der Elektroenzephalographie. of Psychology, 53, 545–74.
Göttingen: Wallstein. Delorme, A., & Makeig, S. (2004). EEGLAB: An
Bradley, M. M., Hamby, S., Low, A., & Lang, P. J. open source toolbox for analysis of single-trial
(2007). Brain potentials in perception: Pic- EEG dynamics including independent compo-
ture complexity and emotional arousal. Psy- nent analysis. Journal of Neuroscience Methods,
chophysiology, 44(3), 364–73. 134(1), 9–21.
Büchel, C., Coull, J. T., & Friston, K. J. (1999). Dien, J. (2010). Evaluating two-step PCA of ERP
The predictive value of changes in effec- data with Geomin, Infomax, Oblimin, Pro-
tive connectivity for human learning. Science, max, and Varimax rotations. Psychophysiology,
283(5407), 1538–41. 47(1), 170–83.
Cahn, B. R., & Polich, J. (2006). Meditation states Ding, L., & He, B. (2008). Sparse source imaging
and traits: EEG, ERP, and neuroimaging stud- in electroencephalography with accurate field
ies. Psychology Bulletin, 132(2), 180–211. modeling. Human Brain Mapping, 29(9), 1053–
Cohen, D. (1968). Magnetoencephalography: 67.
Evidence of magnetic fields produced by Elliott, G., Rothenberg, T. J., & Stock, J. (1996).
alpha-rhythm currents. Science, 161(843), 784– Efficient tests for an autoregressive unit root.
86. Econometrica, 64(4), 813–36.
Cohen, D. (1972). Magnetoencephalography: Ferree, T. C., Luu, P., Russell, G. S., & Tucker, D.
Detection of the brain’s electrical activity M. (2001). Scalp electrode impedance, infec-
with a superconducting magnetometer. Sci- tion risk, and EEG data quality. Clinical Neu-
ence, 175(22), 664–66. rophysiology, 112(3), 536–44.
Cohen, D., & Cuffin, B. N. (1987). A method for Flor, H., Braun, C., Elbert, T., & Birbaumer, N.
combining MEG and EEG to determine the (1997). Extensive reorganization of primary
sources. Physics in Medicine and Biology, 32(1), somatosensory cortex in chronic back pain
85–89. patients. Neuroscience Letters, 224(1), 5–8.
Cohen, D., & Cuffin, B. N. (1991). EEG Foti, D., Hajcak, G., & Dien, J. (2009) Differ-
versus MEG localization accuracy: Theory entiating neural responses to emotional pic-
and experiment. Brain Topography, 4(2), tures: evidence from temporal-spatial PCA.
95–103. Psychophysiology, 46, 521–30.
Cohen, D., Cuffin, B. N., Yunokuchi, K., Frund, I., Schadow, J., Busch, N. A., Korner, U.,
Maniewski, R., Purcell, C., Cosgrove, G. R., et & Herrmann, C. S. (2007). Evoked gamma
al. (1990). MEG versus EEG localization test oscillations in human scalp EEG are test-retest
using implanted sources in the human brain. reliable. Clinical Neurophysiology, 118(1), 221–
Annals of Neurology, 28(6), 811–17. 27.
Cook, E. W., III, & Miller, G. A. (1992). Digi- Galambos, R. (1992). A comparison of certain
tal filtering: Background and tutorial for psy- gamma-band (40 Hz) brain rhythms in cat
chophysiologists. Psychophysiology, 29(3), 350– and man. In E. Basar & T. Bullock (Eds.),
67. Induced rhythms in the brain (pp. 103–22).
Cuthbert, B. N., Schupp, H. T., Bradley, M. Berlin: Springer.
M., Birbaumer, N., & Lang, P. J. (2000). Gianotti, L. R., Faber, P. L., Schuler, M., Pascual-
Brain potentials in affective picture process- Marqui, R. D., Kochi, K., & Lehmann, D.
ing: Covariation with autonomic arousal and (2008). First valence, then arousal: The tempo-
affective report. Biological Psychology, 52(2), ral dynamics of brain electric activity evoked
95–111. by emotional stimuli. Brain Topography, 20(3),
Cuthbert, B. N., Schupp, H. T., Bradley, M., 143–56.
McManis, M., & Lang, P. J. (1998). Probing Grave de Peralta, R., Hauk, O., & Gonzalez,
affective pictures: Attended startle and tone S. L. (2009). The neuroelectromagnetic
probes. Psychophysiology, 35(3), 344–47. inverse problem and the zero dipole localiza-
Davidson, R. J. (1995). Cerebral asymmetry, emo- tion error. Computational Intelligence and Neu-
tion, and affective style. In R. J. Davidson & roscience, 659247. doi:10.1155/2009/659247
130 ANDREAS KEIL
Groppe, D. (2011). Mass univariate analysis stimuli: Evidence from event-related poten-
of event-related brain potentials/fields I: tials. Cognitive, Affective, & Behavioral Neuro-
A critical tutorial review. Psychophysiology, science, 7(1), 18–24.
48(12),1726–37. Keil, A., Gruber, T., Muller, M. M., Moratti,
Hajcak, G., Weinberg, A., MacNamara, A., & S., Stolarova, M., Bradley, M. M., et al.
Foti, D. (2011). ERPs and the study of emotion. (2003). Early modulation of visual perception
In S. J. Luck & E. Kappenman (Eds.), Oxford by emotional arousal: Evidence from steady-
handbook of event-related potential components state visual evoked brain potentials. Cognitive,
(pp. 441–74). New York: Oxford University Affective, & Behavioral Neuroscience, 3(3), 195–
Press. 206.
Haken, H. (1983). Synergetics: An introduction. Keil, A., Smith, J. C., Wangelin, B., Sabatinelli,
Berlin: Springer. D., Bradley, M. M., & Lang, P. J. (2008).
Hämäläinen, M., & Ilmoniemi, R. (1984). Inter- Electrocortical and electrodermal responses
preting measured magnetic fields of the brain: co-vary as a function of emotional arousal:
Estimates of current distributions. (Technical A single-trial analysis. Psychophysiology, 45(4),
Report No. TKK-F-A559). Helsinki: Helsinki 511–15.
University of Technology. Keil, A., Stolarova, M., Heim, S., Gruber, T.,
Heim, S., & Keil, A. (2006). Effects of classi- & Muller, M. M. (2003). Temporal stabil-
cal conditioning on identification and corti- ity of high-frequency brain oscillations in the
cal processing of speech syllables. Experimen- human EEG. Brain Topography, 16(2), 101–10.
tal Brain Research, 175, 411–24. Keil, A., Stolarova, M., Moratti, S., & Ray, W. J.
Homma, S., Nakajima, Y., Toma, S., Ito, T., & (2007). Adaptation in visual cortex as a mecha-
Shibata, T. (1998). Intracerebral source local- nism for rapid discrimination of aversive stim-
ization of mental process-related potentials uli. Neuroimage, 36, 472–79.
elicited prior to mental sweating response in Klimesch, W., Sauseng, P., & Hanslmayr, S.
humans. Neuroscience Letters, 247(1), 25–28. (2007). EEG alpha oscillations: The inhibition-
Ille, N., Berg, P., & Scherg, M. (2002). Artifact timing hypothesis. Brain Research Reviews,
correction of the ongoing EEG using spa- 53(1), 63–88.
tial filters based on artifact and brain signal Knott, J. R., & Irwin, D. A. (1968). Anxi-
topographies. Journal of Clinical Neurophysi- ety, stress and contingent negative variation
ology, 19(2), 113–24. (CNV). Electroencephalography and Clinical
Johnston, V. S., & Wang, X. T. (1991). The rela- Neurophysiology, 24(3), 286–87.
tionship between menstrual phase and the P3 Kolassa, I., Kolassa, S., Musial, F., & Miltner,
component of ERPs. Psychophysiology, 28(4), W. H. (2007). Event-related potentials to
400–9. schematic faces in social phobia. Cognition and
Junghöfer, M., Bradley, M. M., Elbert, T. R., & Emotion, 21(8), 1721–44.
Lang, P. J. (2001). Fleeting images: A new look Kolassa, I. T., & Miltner, W. H. (2006). Psy-
at early emotion discrimination. Psychophysi- chophysiological correlates of face processing
ology, 38(2), 175–78. in social phobia. Brain Research, 1118(1), 130–41.
Junghöfer, M., Elbert, T., Tucker, D. M., & Kranczioch, C., Debener, S., Maye, A., & Engel,
Braun, C. (1999). The polar average reference A. K. (2007). Temporal dynamics of access to
effect: A bias in estimating the head surface consciousness in the attentional blink. Neu-
integral in EEG recording. Clinical Neurophys- roimage, 37(3), 947–55.
iology, 110(6), 1149–55. Lang, P. J. (1994). The motivational organiza-
Junghofer, M., Elbert, T., Tucker, D. M., & Rock- tion of emotion: Affect-reflex connections. In
stroh, B. (2000). Statistical control of artifacts S. H. M. Van Goozen, N. E. Van de Poll, & J.
in dense array EEG/MEG studies. Psychophys- E. Sergeant (Eds.), Emotions: Essays on emotion
iology, 37(4), 523–32. theory (pp. 61–93). Hillsdale, NJ: Erlbaum.
Keil, A., Bradley, M. M., Hauk, O., Rockstroh, Lang, P. J., Bradley, M. M., & Cuthbert, B.
B., Elbert, T., & Lang, P. J. (2002). Large-scale N. (2005). International Affective Picture Sys-
neural correlates of affective picture process- tem: Technical manual and affective ratings.
ing. Psychophysiology, 39(5), 641–49. Gainesville, FL: NIMH Center for the Study
Keil, A., Bradley, M. M., Junghoefer, M., Russ- of Emotion and Attention.
mann, T., Lowenthal, W., & Lang, P. J. (2007). LeDoux, J. E. (1993). Emotional networks in the
Cross-modal attention capture by affective brain. In J. M. H. Lewis (Ed.), Handbook of
ELECTRO- AND MAGNETO-ENCEPHALOGRAPHY IN THE STUDY OF EMOTION 131
emotions. (pp. 109–18). New York: Guilford Moratti, S., Keil, A., & Miller, G. A. (2006). Fear
Press. but not awareness predicts enhanced sensory
Lesse, H. (1957). Amygdoloid electrical activity dur- processing in fear conditioning. Psychophysiol-
ing a conditioned response. Paper presented ogy, 43(2), 216–26.
at the International Congress of Electroen- Moratti, S., Keil, A., & Stolarova, M. (2004).
cephalography and Clinical Neurophysiology, Motivated attention in emotional picture pro-
Brussels. cessing is reflected by activity modulation in
Linden, D. E. (2005). The p300: Where in the cortical attention networks. Neuroimage, 21(3),
brain is it produced and what does it tell us? 954–64.
Neuroscientist, 11(6), 563–76. Müller, M. M., Andersen, S., & Keil, A. (2008).
Lindsley, D. B. (1950). Emotions and the elec- Time course of competition for visual process-
troencephalogram. In M. Reymert (Ed.), Feel- ing resources between emotional pictures and
ings and emotions; The Mooseheart Symposium a foreground task. Cerebral Cortex, 18, 1892–99.
(pp. 238–46). New York: McGraw-Hill. Müller, M. M., Teder-Salejarvi, W., & Hillyard,
Luck, S. J. (2005). An introduction to the event- S. A. (1998). The time course of cortical facil-
related potential technique. Cambridge, MA: itation during cued shifts of spatial attention.
MIT Press. Nature Neuroscience, 1(7), 631–34
Luo, Q., Mitchell, D., Cheng, X., Mondillo, K., Nitschke, J. B., Miller, G. A., & Cook, E. W.
McCaffrey, D., Holroyd, T., et al. (2009). (1998). Digital filtering in EEG/ERP analy-
Visual awareness, emotion, and gamma band sis: Some technical and empirical compar-
synchronization. Cerebral Cortex, 19(8), 1896– isons. Behavior Research Methods Instruments
1904. & Computers, 30(1), 54–67.
Makeig, S., Debener, S., Onton, J., & Delorme, Nunez, P. L., & Srinivasan, R. (2006). Electric
A. (2004). Mining event-related brain dynam- fields of the brain (2nd ed.). New York: Oxford
ics. Trends in Cognitive Sciences, 8(5), 204–10. University Press.
Makeig, S., Jung, T. P., Bell, A. J., Ghahremani, Nunez, P. L., Srinivasan, R., Westdorp, A. F.,
D., & Sejnowski, T. J. (1997). Blind separa- Wijesinghe, R. S., Tucker, D. M., Silberstein,
tion of auditory event-related brain responses R. B., et al. (1997). EEG coherency. I: Statis-
into independent components. Proceedings of tics, reference electrode, volume conduction,
the National Academy of Sciences, 94, 10979–84. Laplacians, cortical imaging, and interpre-
Maris, E. (2012). Statistical testing in electrophys- tation at multiple scales. Electroencephalog-
iological studies. Psychophysiology, 9(4), 549– raphy and Clinical Neurophysiology, 103(5),
65. 499–515.
Melcher, J. R., & Cohen, D. (1988). Dependence Olejniczak, P. (2006). Neurophysiologic basis
of the MEG on dipole orientation in the rab- of EEG. Journal of Clinical Neurophysiology,
bit head. Electroencephalography and Clinical 23(3), 186–89.
Neurophysiology, 70(5), 460–72. Pascual-Marqui, R. D., Michel, C. M., &
Miller, J., Patterson, T., & Ulrich, R. (1998). Lehmann, D. (1994). Low resolution elec-
Jackknife-based method for measuring LRP tromagnetic tomography: A new method for
onset latency differences. Psychophysiology, localizing electrical activity in the brain. Inter-
35(1), 99–115. national Journal of Psychophysiology, 18(1), 49–
Moratti, S., Clementz, B., Ortiz, T., & Keil, A. 65.
(2007). Neural mechanisms of evoked oscilla- Pascual-Marqui, R. D., Michel, C. M., &
tions: Stability and interaction with transient Lehmann, D. (1995). Segmentation of brain
events. Human Brain Mapping, 28(12), 1318–33. electrical activity into microstates: Model esti-
Moratti, S., & Keil, A. (2005). Cortical acti- mation and validation. IEEE Transactions of
vation during Pavlovian fear conditioning Biomedical Engineering, 42(7), 658–65.
depends on heart rate response patterns: An Peyk, P., Schupp, H. T., Elbert, T., & Junghofer,
MEG study. Brain Research: Cognitive Brain M. (2008). Emotion processing in the visual
Research, 25(2), 459–71. brain: A MEG analysis. Brain Topography,
Moratti, S., & Keil, A. (2009). Not what you 20(4), 205–215.
expect: Experience but not expectancy pre- Pizzagalli, D. A., Greischar, L. L., & David-
dicts conditioned responses in human visual son, R. J. (2003). Spatio-temporal dynamics of
and supplementary cortex. Cerebral Cortex, brain mechanisms in aversive classical condi-
19(12), 2803–9. tioning: High-density event-related potential
132 ANDREAS KEIL
and brain electrical tomography analyses. signature and detectability. Trends in Cognitive
Neuropsychologia, 41(2), 184–94. Sciences, 1, 252–61.
Pizzagalli, D., Lehmann, D., Koenig, T., Regard, Stolarova, M., Keil, A., & Moratti, S. (2006).
M., & Pascual-Marqui, R. D. (2000). Face- Modulation of the C1 visual event-related
elicited ERPs and affective attitude: Brain component by conditioned stimuli: Evidence
electric microstate and tomography analyses. for sensory plasticity in early affective percep-
Clinical Neurophysiology, 111(3), 521–31. tion. Cerebral Cortex, 16(6), 876–87.
Pourtois, G., Grandjean, D., Sander, D., & Tallon-Baudry, C., & Bertrand, O. (1999). Oscil-
Vuilleumier, P. (2004). Electrophysiologi- latory gamma activity in humans and its role
cal correlates of rapid spatial orienting in object representation. Trends in Cognitive
towards fearful faces. Cerebral Cortex, 14(6), Sciences, 3(4), 151–62.
619–33. Tenke, C. E., & Kayser, J. (2005). Reference-free
Radilova, J. (1982). The late positive compo- quantification of EEG spectra: Combining
nent of visual evoked response sensitive to current source density (CSD) and frequency
emotional factors. Act Nerv Super (Praha), principal components analysis (fPCA). Clini-
3(Suppl., Pt. 2), 334–37. cal Neurophysiology, 116(12), 2826–46.
Ravden, D., & Polich, J. (1999). On P300 measure- Tomarken, A. J., Davidson, R. J., Wheeler, R. E.,
ment stability: Habituation, intra-trial block & Kinney, L. (1992). Psychometric properties
variation, and ultradian rhythms. Biological of resting anterior EEG asymmetry: Temporal
Psychology, 51(1), 59–76. stability and internal consistency. Psychophys-
Regan, D. (1989). Human brain electrophysiology: iology, 29(5), 576–92.
Evoked potentials and evoked magnetic fields in Walter, W. G. (1967). The analysis, synthesis and
science and medicine. New York: Elsevier. identification of evoked responses and con-
Sabatinelli, D., Lang, P. J., Keil, A., & Bradley, tingent negative variation (CNV). Electroen-
M. M. (2007). Emotional perception: Corre- cephalography and Clinical Neurophysiology,
lation of functional MRI and event related 23(5), 489.
potentials. Cerebral Cortex, 17, 1066–73. Weisz, N., Kostadinov, B., Dohrmann, K., Hart-
Scherg, M., & Von Cramon, D. (1986). Evoked mann, T., & Schlee, W. (2007). Tracking short-
dipole source potentials of the human audi- term auditory cortical plasticity during classi-
tory cortex. Electroencephalography and Clini- cal conditioning using frequency-tagged stim-
cal Neurophysiology, 65(5), 344–60. uli. Cerebral Cortex, 17(8), 1867–76.
Schupp, H. T., Cuthbert, B. N., Bradley, M. M., Wennberg, R., Valiante, T., & Cheyne, D. (2011).
Birbaumer, N., & Lang, P. J. (1997). Probe P3 EEG and MEG in mesial temporal lobe
and blinks: Two measures of affective startle epilepsy: Where do the spikes really come
modulation. Psychophysiology, 34(1), 1–6. from? Clinical Neurophysiology, 122(7), 1295–
Schupp, H. T., Cuthbert, B. N., Bradley, M. M., 1313.
Hillman, C. H., Hamm, A. O., & Lang, P. J. Whittingstall, K., & Logothetis, N. K. (2009).
(2004). Brain processes in emotional percep- Frequency-band coupling in surface EEG
tion: Motivated attention. Cognition and Emo- reflects spiking activity in monkey visual cor-
tion, 18(5), 593–611. tex. Neuron, 64(2), 281–89.
Schupp, H. T., Junghofer, M., Weike, A. I., & Wieser, M. J., McTeague, L. M., & Keil, A.
Hamm, A. O. (2003). Emotional facilitation (2011). Sustained preferential processing of
of sensory processing in the visual cortex. Psy- social threat cues: Bias without competition?
chological Science, 14(1), 7–13. Journal of Cognitive Neuroscience, 23(8), 1973–
Simons, R. F., & Miles, M. A. (1990). Nonfamil- 86.
ial strategies for the identification of subjects Wieser, M. J., Pauli, P., Reicherts, P., &
at risk for severe psychopathology: Issues of Muhlberger, A. (2010). Don’t look at me in
reliability in the assessment of event-related anger! Enhanced processing of angry faces in
potentials (ERP). In J. W. Rohrbaugh, R. Para- anticipation of public speaking. Psychophysiol-
suraman, & R. J. Johnson (Eds.), Event-related ogy, 47(2), 271–80.
brain potentials: Basic issues and applications Yuval-Greenberg, S., Tomer, O., Keren, A. S.,
(pp. 343–63). Amsterdam: Elsevier. Nelken, I., & Deouell, L. Y. (2008). Transient
Singer, W., Engel, A. K., Kreiter, A. K., Munk, induced gamma-band response in EEG as a
M. H. J., Neuenschwander, S., & Roelfsema, manifestation of miniature saccades. Neuron,
P. R. (1997) Neuronal assemblies: Necessity, 58(3), 429–41.
CHAPTER 5
133
134 JORGE ARMONY & JUNG EUN HAN
photon pairs, one can trace the location (rCBF) that, as mentioned earlier, can be
of the decays and thus build a quantitative used as a marker of neural activity.
image of the distribution of the radionuclei- The majority of rCBF PET studies
des at a given time (interval). Such a proce- employ oxygen15-labeled water molecules
dure constitutes the basis of PET. ([15 O]H2 O) as the radiotracer. This tech-
For this technique to be useful for cog- nique was instrumental, particularly dur-
nitive neuroscience research, the isotopes ing the 1980s and 1990s, in advancing our
employed need to have a half-life (the time understanding of the neural correlates of
that it takes for 50% of the nuclei in a sam- a variety of mental processes in healthy
ple to decay) long enough to allow them humans, including emotional processing,
to reach the target area, in this case the such as perception of emotional faces (Mor-
brain, and short enough to generate a suf- ris et al., 1996) or pictures (Kosslyn et
ficient number of decays for an accurate al., 1996), and mood induction (George et
image reconstruction and identification of al., 1995). In addition, PET has been used
the source of activity associated with an to identify functional abnormalities associ-
experimental condition, typically over the ated with neurological and psychiatric dis-
course of a few minutes. In addition, they orders by, for example, comparing resting-
have to be able to be introduced easily into state rCBF between individuals suffering
the body and incorporated into substances from panic disorder and healthy controls
that are involved in the process of interest (Reiman, Raichle, Butler, Herscovitch, &
(e.g., metabolism, blood flow, neurorecep- Robins, 1984). However, this technique’s
tor pharmacology). Fortunately, several iso- temporal resolution presents an important
topes fulfill these criteria; the most widely limitation, because the different experimen-
used are oxygen-15 (t1/2: 2 min), fluorine-18 tal conditions need to be presented in a
(t1/2: 110 min), nitrogen-13 (t1/2: 10 min), and so-called block design to allow for a suffi-
carbon-11 (t1/2: 20 min). However, because of cient number of photon pairs to be detected.
the short half-life of these radionucleides, Block designs typically involve the repeated
the cyclotron that produces them needs to presentation of trials of the same experimen-
be located close to the PET scanner. tal category (e.g., fearful faces) in one block
Importantly, there are several issues that lasting a minute or so, alternating with a sec-
limit the spatial resolution that can be ond condition, usually the control one (e.g.,
achieved with PET. First, positrons are capa- neutral faces). In such a design, any tem-
ble of traveling a non-negligible distance poral information about the hemodynamic
before their collision with an electron. This (and therefore neural) response is limited to
mean free path, which depends on the the duration of the block.
energy of the particle and the medium in
which is traveling, can be up to 5 mm long
[18 F]deoxyglucose PET
and thus lead to an inaccurate localization
of the original beta decay. In addition, the Another index of brain activity, namely local
emitted photons can interact with electrons changes in cerebral glucose metabolism,
in the medium, losing energy and chang- can be assessed with the PET tracer
ing their path. These collisions, known as [18 F]deoxyglucose (FDG). When adminis-
Compton scattering, also introduce errors in tered, FDG is distributed to active brain
the image reconstruction, because the true regions in need of glucose, where it is phos-
location of the decay will no longer be on the phorylated to FDG-6-phosphate, which gets
line connecting the two detected photons. trapped in those brain areas. The quan-
tity of radioactivity of trapped FDG-6-
phosphate reflects regional glucose uptake
Cerebral Blood Flow PET
and metabolism. FDG PET is less suit-
PET has proven particularly appropriate able for imaging ongoing neuronal activ-
for measuring regional cerebral blood flow ity, because of its low time resolution: One
136 JORGE ARMONY & JUNG EUN HAN
scan image represents the average brain glu- oligand affinity constant, ultimately repre-
cose metabolic activity of about 40 min- senting receptor density. To account for the
utes. Regional cerebral metabolic rate of glu- specificity of binding, the model requires the
cose (rCMRGLU ) during the resting state is use of a reference region devoid of, or low
most frequently explored in the FDG PET in, the binding sites of investigation (e.g.,
literature. Most often, this “baseline” glu- generally the cerebellum for dopamine and
cose metabolism measure is used to differ- serotonin, the occipital cortex for opioids).
entiate clinical populations (e.g., depressed) Differences in the BPs between the control
from healthy controls (Baxter et al., 1989) and experimental conditions are assumed
or to examine treatment-related changes to be caused by the task-induced change
(Goldapple et al., 2004). The use of FDG in the levels of endogenous neurotransmit-
PET can contribute to our knowledge of ter. When the BP cannot be determined,
emotion in the healthy brain as well, by a related measure, the distribution volume
localizing glucose utilization at rest in rela- (DV), is used, which is the ratio of the radi-
tion to personality traits (Volkow et al., oligand concentration in the region of inter-
2011). Finally, resting rCMRGLU PET can est to its plasma concentration at equilib-
test the intraindividual stability of the base- rium.
line neuronal activity in different brain areas One of the most crucial and difficult
across time, which is an underlying assump- aspects in performing neurotransmission
tion of typical fMRI and PET activation studies with PET is the synthesis of an
studies that require repeated measurements appropriate radiotracer for the neurotrans-
(Schaefer et al., 2000). mitter system of interest. PET tracers need
to fulfill several criteria for reliable and accu-
rate neurotransmission mapping. Ideal radi-
Ligand PET
oligands are manufactured to (1) yield highly
In addition to measuring task-induced specific radioactivity and to have (2) good
changes in CBF or cerebral metabolism, PET permeability through the blood-brain bar-
imaging can be used to directly and selec- rier (e.g., having appropriate lipophilicity),
tively assess the action of different neuro- (3) sufficient affinity and high selectivity for
transmitters in the human brain in vivo. The the target receptor system, (4) low non-
basic principles of ligand-based PET involve specific binding, and (5) minimal binding
competition for occupancy of the receptor of their metabolites in the brain. Despite
sites between the endogenous neurotrans- continuous effort to develop PET ligands,
mitter and the injected PET ligand, both of only a subset of molecules can be investi-
which target the same receptors in the brain. gated today. A description and some exam-
Ligand PET can be used to quantify baseline ples of the use in emotion research of two
receptor density (e.g., at rest), which can be of the most studied neurotransmitters with
then correlated with other measures (e.g., this technique, dopamine and serotonin, are
neuropsychological measures, hormone lev- presented next.
els, etc.), as well as task-induced changes
in the levels of endogenous neurotransmit- DOPAMINERGIC SYSTEM
ter. In the latter case, comparison of the Dopamine is by far the most extensively
regional radioactivity concentrations (either studied neuromodulator of emotion in the
increases or decreases) during control and molecular PET literature, particularly in
experimental conditions provides spatial reward and addiction research. The stria-
and temporal information about the task- tum is the brain region known to have
induced neurotransmission. the highest density of dopamine recep-
Most often, the outcome parameter is tors (see Chapter 19) and is therefore the
the binding potential (BP) of specific tracer region typically targeted in dopamine PET
binding, determined by the receptor con- studies, usually with [11 C]raclopride as the
centration available for binding and the radi- tracer. With relatively low affinity, injected
PET AND FMRI 137
[11 C]raclopride binds to dopamine D2/D3 was observed with D2 receptor availability,
receptors in high-receptor density areas such measured with [11C]FLB457.
as the striatum within a few minutes and These postsynaptic dopamine receptors
reliably detects changes in endogenous levels are not the only system probed by ligand-
of striatal dopamine. A [11 C] raclopride PET based PET. Available PET ligands for
study (Pruessner, Champagne, Meaney, & dopamine synthesis and dopamine trans-
Dagher, 2004) used a stress task to exam- porter (DAT) have also enabled the inves-
ine the stress-induced release of endogenous tigation of presynaptic dopamine function.
dopamine (i.e., reduced [11 C]raclopride BP) Despite complicated analysis and interpre-
in the ventral striatum in healthy individu- tation because of its peripheral metabolism
als, which was also significantly correlated and labeled metabolites, [18 F]fluorodopa
with cortisol levels. ([18 F]FDOPA), a radiolabeled dopamine
Emotion processing also engages brain precursor, is still widely used to explore
regions other than the striatum where dopamine synthesis capacity. Because the
dopamine may play a significant role. How- [18 F]FDOPA PET signal extracted from
ever, the low affinity of [11 C]raclopride extrastriatal regions is not likely to be spe-
makes it unsuitable for regions with low cific to the dopaminergic system (Brown
dopamine receptor density. The dopamin- et al., 1999), [18 F]FDOPA is typically used to
ergic system in the extrastriatal regions target the striatum. The uptake of this ligand
can be better explored using the tracer reflects the decarboxylation of fluorodopa to
[18 F]fallypride. Shortly after injection, this fluorodopamine, which is stored in synaptic
high-affinity ligand binds to available vesicles. Laakso and colleagues (2003) used
dopamine receptors (D2/D3) in brain [18F]FDOPA PET to assess the relationship
areas with low dopamine receptor density, between presynaptic dopamine synthesis
whereas binding of this tracer in the stria- and various personality traits. They observed
tum takes hours. Using [18F]fallypride PET, that high scores on somatic anxiety, muscu-
Badgaiyan and colleagues (2009) observed lar tension, and irritability were associated
increase in dopaminergic activity in the with low [(18)F]fluorodopa uptake in the
amygdala, medial temporal lobe, and infe- caudate. Using the DAT ligand [18F]CFT
rior frontal gyrus when subjects were pre- ([18F]WIN 35,428), the group also found a
sented with emotional, compared to neu- negative correlation between DAT binding
tral, words. Although [11 C]raclopride and in the putamen and detachment personality
[18 F]fallypride bind to both D2 and D3 scores.
receptors, because the ratio of D2 to D3
receptors is high in most brain regions, D2 is SEROTONINERGIC (5-HT) SYSTEM
assumed to be the target of these tracers. Despite the prominent role that the sero-
In addition, a relatively small number of tonergic system plays in affective disor-
PET ligands are currently available for in ders such as mood disorders, aggression,
vivo imaging of D1 receptors. The two most and anxiety, little attention has been given
commonly used ligands are [11 C]SCH23390 to the study of serotonin and emotion in
and [11 C]NNC112, both of which have high healthy individuals in the neuroreceptor
affinity for D1 receptors and are appropri- PET literature. Moreover, among numer-
ate tracers to target the striatum, where ous subtypes of serotonergic receptors and
the highest concentration of D1 receptor is their PET tracers available today, emo-
found, although they can be used for other tion research has so far focused mainly
regions as well. For instance, Takahashi and on 5-HT1A and 5-HT2A receptors, serotonin
colleagues (2010) used [11C]SCH23390 PET, uptake transporter (SERT), and 5-HT syn-
in combination with fMRI (in a separate ses- thesis in healthy subjects.
sion), to show that amygdala activation to 5-HT1A receptors are distributed in
fearful faces was significantly related to D1 high densities in the amygdala, hippocam-
receptor availability. In contrast, no relation pus, hypothalamus, septum, and neocortex.
138 JORGE ARMONY & JUNG EUN HAN
With selectivity and high affinity, [carbonyl- cessful PET tracers, [11 C]DASB is used
11
C]WAY-100635 is the tracer of choice for 5- most widely, and it is the only one docu-
HT1A receptors. For instance, Tauscher and mented in previous studies on emotion in
colleagues (2001) were able to explore, using healthy populations. [11 C]DASB is known to
this tracer, the significant negative associa- have high selectivity and affinity, be highly
tion between baseline 5-HT1A receptor avail- reproducible and reliable, and require sim-
ability in the dorsolateral prefrontal, occipi- ple quantification. One [11 C]DASB study
tal and parietal cortices and anxiety. (Kupers et al., 2011) explored the relation-
5-HT2A receptors are found in high ship between baseline SERT binding in sev-
density in the neocortex and are vir- eral brain regions and pain tolerance ratings
tually absent in the cerebellum. Previ- to noxious heat stimuli, further corroborat-
ous studies in healthy individuals have ing the role of the serotonergic system in
explored these receptors with respect to pain processing and modulation.
emotion processing using [18 F]setoperone In summary, ligand PET constitutes a
and [18 F]altanserin, both of which show powerful tool to examine neurotransmitter
good reproducibility. [18 F]setoperone binds function during emotional processing, espe-
to both striatal dopamine D2 receptors cially when combined with complemen-
and cortical 5-HT2A receptors. Although tary techniques. For instance, a recent study
less lipophilic than the parent compound, Salimpoor and colleagues (2011) investigated
this tracer also produces radiometabolites. reward responses to pleasurable music using
By comparing [18 F]setoperone binding in [11 C]raclopride PET and blood-oxygen-
the low receptor density area (cerebel- level-dependent (BOLD) fMRI measures,
lum) to that in the rich receptor density in separate scans but using the same stim-
areas (orbitofrontal cortex [OFC] and ante- uli. In the PET experiment, they observed a
rior cingulate cortex [ACC]), and corre- significant release of dopamine in the stria-
lating the receptor availability to several tum associated with emotional arousal while
questionnaire scores, Gerretsen and col- listening to highly pleasant music. Then,
leagues (2010) were able to provide evidence using fMRI, the authors explored the time
for the potential modulatory role of the course of the response and found that the
serotonergic system in OFC and ACC on caudate and the nucleus accumbens were
the desire for social relationships. Today, more active during the anticipation and
[18 F]altanserin is the most widely used 5- experience, respectively, of peak emotional
HT2A PET ligand, despite the lipophilic responses to music.
radiometabolites that the tracer produces,
which necessitates the use of complex
kinetic modeling or a 2-hour bolus-infusion Functional MRI
paradigm. [18 F]altanserin contains longer
lived 18 F label and is more selective than Although early neuroimaging studies of
[18 F]setoperone. emotion mostly used PET, over the last
As the target of a number of antide- decade the field has experienced a shift in
pressant medications (e.g., selective sero- methodology, and currently most emotion-
tonin reuptake inhibitors; SSRIs), SERT has related experiments employ functional mag-
been investigated extensively in the liter- netic resonance imaging. Advances in
ature of depression, bipolar disorder, and acquisition techniques and methodologi-
social phobia, among many other psychiatric cal approaches, together with the dramatic
conditions, using various methods includ- increase in MR scanner availability, have
ing ligand-based PET imaging. Although dis- made this technique ubiquitous in cogni-
tributed widely throughout the brain, the tive and affective neuroscience research.
highest concentrations of SERT are found in Functional MRI has several advantages over
the amygdala, hippocampus, striatum, tha- PET, such as increased temporal and spatial
lamus, and midbrain. Among a few suc- resolution, described later, as well as the
PET AND FMRI 139
important fact that it does not involve ion- In contrast, when the object is placed in a
izing radiation. strong magnetic field B0, spins will assume
one of a few specific energy levels. In the
case of the spin 1/2 hydrogen nuclei, spins will
Physical Basis of MRI
be either in a low-energy spin-up (parallel)
MRI is based on nuclear magnetic reso- state or a high-energy spin-down (antipar-
nance (NMR). As its name suggests, this allel) state. Because the parallel state has
phenomenon is related to the interaction, lower energy than the antiparallel one, there
through resonance, between atomic nuclei will be slightly more spins in the parallel
and magnetic fields. Simple NMR behav- state (about 3 per million protons, for an
ior can be discussed using classical physics, external field of 1 Tesla at room temper-
whereas a more complete and accurate view ature). It is this excess of protons point-
requires quantum physics. We present next ing “up,” resulting in a net macroscopic
a very brief and simplified description of the magnetization of the object being studied,
principles of NMR. For a more thorough which underlies the generation of the MRI
account of the physical basis of MRI, the signal.
interested reader is referred to the numer- Given the appropriate amount of energy,
ous excellent reviews and textbooks on the spins in the lower energy state (spin-up) can
topic (e.g., Bernstein, King, & Zhou, 2004; jump to the higher energy level. The amount
Haacke, Brown, Thompson, & Venkatesan, of energy required for this transition is equal
1999; Huettel, Song, & McCarthy, 2009). to the difference in energy between levels
In addition to mass and charge, atomic and can be delivered by an electromagnetic
nuclei can have intrinsic angular momen- field that oscillates with a frequency that
tum, or spin. Although spin is a quantum is tuned to the required energy value (the
mechanical property of particles, which can- energy of an electromagnetic wave is pro-
not be explained by classical physics, it may portional to its frequency). This resonant fre-
be helpful to think of it as representing a quency, known as the Larmor frequency, is
physical rotation of the nucleus around its equal to the product of the gyromagnetic
axis, like a top. Nuclei with an odd number ratio γ – a property of the spinning nucleus
of protons or neutrons have nonzero spin. (approximately 42 MHz per Tesla for hydro-
In particular, the nucleus of hydrogen – the gen nuclei) – and the external field B0. If this
most abundant element in the universe and, electromagnetic field is left on for a sufficient
critically, in the human body (mostly in the time, enough spins will jump to the higher
form of water) – is composed of a single pro- energy level so that the number of parallel
ton and thus has a nonzero (1/2) spin. A key and antiparallel spins becomes equal, and
property of nuclei with nonzero spin is that therefore there is no longer a net magneti-
they possess a magnetic moment, related to zation along the direction of the B0 field.
the nuclear spin through a nucleus-specific In the classical view, the electromagnetic
multiplicative constant known as the gyro- field, applied in a direction perpendicular to
magnetic ratio γ . The nuclear magnetic the static main magnetic field, tips the spins
moment is a vector, and hence it has, in 90 degrees so that they are now lying on the
addition to a magnitude, a defined direc- orthogonal plane. After the radiofrequency
tion in space. In the absence of an external (RF) pulse is turned off, spins go back to the
magnetic field, there is no preferred orienta- original equilibrium state, with an excess of
tion for a magnetic moment so that different spin-up protons, so that the bulk magne-
spins will be pointing in different, random tization along the axis of B0 is recovered.
directions. This means that the vector sum This process, in which the energy acquired
of the individual magnetic moments of a col- through the radiofrequency pulse is released
lection of nuclear spins (e.g., those consti- (and captured by the receiver coil), is known
tuting an object) will be zero and there will as longitudinal relaxation or recovery and is
be no net macroscopic magnetic moment. governed by a time constant T1 that depends
140 JORGE ARMONY & JUNG EUN HAN
on the medium in which the protons are constant. As explained later, the most com-
embedded. Differences in T1 values between mon type of functional MRI relies on T2 *-
gray matter, white matter, and cerebrospinal weighted images.
fluid provide the basis for the identification A large proportion of current MRI proto-
of these tissue types in standard anatomical, cols for T1 -, T2 -, or T2 *-weighted images con-
T1 -weighted MR images. sist of the sequential acquisition of a series
Individual spins do not completely align of two-dimensional images, or slices. The
in the direction of the external magnetic thickness of the slices, typically on the order
field. Instead, they “precess” around this axis of 2–5 mm, is normally a selectable param-
at the Larmor frequency. In the classical eter of the acquisition. Each slice, in turn,
view, this motion is similar to the wob- is divided into a grid of pixels (e.g., 64x64,
bling of a top around the direction of grav- 192x192), which, together with the field of
ity. This precession means that, in addition view (FOV, region of space being imaged),
to a longitudinal component parallel to the determine the in-plane resolution, much
external magnetic field, each spin’s magnetic like a digital camera. Thus, the final recon-
moment has a rotating transverse compo- structed 3D image is composed of three-
nent. Because the direction of this compo- dimensional pixels, or voxels, which consti-
nent at any given time is different for the tute the resolution elements of the image.
different spins (they are out of phase), the The chosen voxel size usually represents a
net bulk magnetic moment in the trans- tradeoff between spatial resolution, signal-
verse plane is zero. However, the RF pulse to-noise ratio, and scan duration.
that tips the spins also causes them to start
precessing in phase, creating a net macro-
BOLD fMRI: Principles and First
scopic magnetization in the plane perpen-
Studies
dicular to the static magnetic field. Once
the RF pulse is turned off, spins begin to Materials can have magnetic properties;
lose coherence due to their interaction and that is, they can behave in specific ways
gradually become out of phase and the net when placed in a magnetic field. The most
transverse magnetization disappears. This well-known type of interaction is ferromag-
transverse relaxation or decay process also netism, through which compounds such as
depends on the medium in which the pro- iron become strongly attracted to magnets
tons precess, as does the longitudinal relax- and can even become magnetic themselves.
ation, but it has a different time constant, Most materials, however, do not show any
known as T2 . MR images based on trans- noticeable behavior when exposed to an
verse relaxation (T2 -weighted) also provide external magnetic field. Yet, the British
important information about brain structure physicist Michael Faraday observed, in 1845,
and are particularly useful for clinical appli- that a large variety of common “nonmag-
cations (e.g., detection of tumors). netic” compounds, from glass to wood to
Importantly, local magnetic inhomo- beef, were slightly repelled by magnets
geneities will cause spins in different spa- (Faraday, 1846). He termed this small but
tial locations to experience slightly differ- measurable effect diamagnetism. Interest-
ent magnetic fields. This in turn will result ingly, Faraday also tested blood and found
in spins precessing at different frequencies it to be diamagnetic.
(because of different Larmor frequencies; About 80 years later, Linus Pauling
see the earlier discussion), leading to a more and Charles Coryell (Pauling & Coryell,
rapid decay of transverse relaxation than 1936) reexamined the magnetic prop-
that predicted by the spin-spin interactions erties of blood and made a startling
process just described. The combined effects discovery that would have profound impli-
of spin-spin interactions and local magnetic cations for the development of fMRI a half-
inhomogeneities on transverse magnetiza- century later. They found that, whereas oxy-
tion decay are characterized by a T2 * time hemoglobin (oxygenated blood) is mildly
PET AND FMRI 141
diamagnetic, as originally reported by Fara- activity functional maps and could there-
day, deoxy-hemoglobin (i.e., hemoglobin fore become an alternative to other hemo-
without oxygen, dHb) is paramagnetic; that dynamic techniques, notably PET, with the
is, it possesses a positive magnetic suscep- key advantage that it did not require the
tibility and therefore is attracted by mag- injection of an external tracer or contrast
netic fields.1 Thus, the magnetic properties agent. Indeed, very soon thereafter, three
of blood are drastically different depend- groups published, in the same year, studies
ing on whether oxygen is attached to the using BOLD fMRI to track neural-related
hemoglobin molecules. The relevance of this activity in humans while viewing visual
finding for fMRI becomes apparent when we stimuli (Kwong et al., 1992; Ogawa et al.,
put it together with what was described in 1992) or performing a motor task (Bandet-
the previous sections: (1) Increased neural tini, Wong, Hinks, Tikofsky, & Hyde, 1992).
activity in a given brain region is associated Although the first fMRI studies mostly
with an increase in (oxygenated) blood flow focused on relatively simple visual and
in that area, (2) oxy- and deoxy-hemoglobin motor responses, researchers quickly real-
have different magnetic properties, and (3) ized the potential of this new technique
T2 * values depend on the local magnetic to explore a variety of cognitive pro-
properties of the region where the hydrogen cesses, including language (Rao et al., 1992),
spins reside. Thus, areas in which there is a memory (Stern et al., 1996), and mental
change in neural activity – for instance when imagery (Le Bihan et al., 1993). Emotion
comparing an “active” versus “rest” condi- was certainly not left out: A study by Bre-
tion – will be associated with a change in the iter and colleagues on symptom provoca-
oxy- and deoxy-hemoglobin concentrations, tion in patients suffering from obsessive-
which will alter the local magnetic proper- compulsive disorder was presented at an
ties of the medium and thus result in dif- international conference in 1993 (Breiter et
ferences in signal intensity in T2 *-weighted al., 1993) and published about three years
images. later (Breiter et al., 1996; Breiter & Rauch,
The hypothesis that differences in the 1996).2 In 1995, Grodd and colleagues pub-
relative concentration of oxygen in blood lished a study in Der Radiologe reporting
could be quantitatively measured through significant left amygdala activation (i.e., sig-
T2 * values in MRI was confirmed by Thul- nal intensity differences) during sad, but not
born and colleagues in 1982. They observed happy, mood induction, compared to base-
a quadratic relation between 1/T2 values and line (Grodd, Schneider, Klose, & Nagele,
the fraction of deoxygenated hemoglobin 1995). These studies were followed by many
in blood samples, so that T2 values were more, using more sophisticated acquisition
smaller for larger concentrations of dHb, techniques, analysis methods, and experi-
consistent with the paramagnetic proper- mental paradigms to explore an impressive
ties of this molecule and its effects on the variety of mental processes – so much so that
local magnetic field. They also reported that fMRI has arguably become the most popu-
the 1/T2 values, and hence the sensitivity lar technique in cognitive neuroscience in
to detect relative differences in dHB con- general and in affective neuroscience in par-
centrations, increased quadratically with the ticular, as exemplified in most chapters in
strength of the static magnetic field. the present volume.
A few years later, Ogawa and colleagues
showed that this technique could be used to
The BOLD Response
view the brain vasculature in vivo in rodents
(Ogawa, Lee, Nayak, & Glynn, 1990). They The change in the BOLD signal triggered by
termed this effect blood-oxygenation-level- a brief neural event is known as the hemo-
dependent (BOLD) contrast (Ogawa, Lee, dynamic response (HDR). Neural activity
Kay, & Tank, 1990). They suggested that it causes an initial increase in dHb concen-
could be used to produce real-time brain tration due to local oxygen consumption,
142 JORGE ARMONY & JUNG EUN HAN
signal can be observed even if there is an research, two of the most important areas
underlying increase in neuronal response in in emotional processing, the amygdala and
that region. ventromedial PFC, are particularly affected
Still, there is now compelling evidence by these susceptibility artifacts. Indeed, it
suggesting that, in some cases, negative was not uncommon in the early days of fMRI
BOLD responses do indeed reflect actual to question any reported amygdala activa-
decreases in neural responses. For instance, tion because of this issue (Merboldt, Frans-
using simultaneous electrophysiological son, Bruhn, & Frahm, 2001). Fortunately,
recordings and fMRI in anesthetized several modifications to reduce susceptibil-
macaque monkeys, Shmuel and colleagues ity artifacts and improve image quality have
(2006) showed that negative BOLD been developed over the years, both for the
responses in the visual cortex were directly amygdala (Chen, Dickey, Yoo, Guttman, &
associated with decreases in neural activity Panych, 2003; Morawetz et al., 2008; Rick
to levels below their spontaneous rate. et al., 2010) and ventromedial PFC (Preston,
Furthermore, Devor et al. (2007) confirmed, Thomason, Ochsner, Cooper, & Glover,
in experiments of rat forepaw stimulation, 2004; Truong & Song, 2008). Nonetheless,
that whereas increased blood oxygena- these artifacts remain an issue that deserves
tion and volume were mainly associated careful attention, and it is therefore wise to
with neuronal excitation and arteriolar routinely examine the raw functional images
vasodilation, decreases in blood oxygena- to ensure that adequate signal is obtained in
tion were observed in areas of neuronal these regions.
inhibition and arterioral vasoconstriction.
Categorical versus Parametric Designs
Some Issues Related to fMRI Because the absolute magnitude of the
Acquisition, Design, and Analysis BOLD signal is arbitrary (i.e., has no direct
physical or physiological meaning), an
When designing, running, or analyzing an fMRI experiment typically requires the
fMRI experiment, there are a number of comparison of one (or more) experimental
choices that need to be made and poten- conditions against a control one. Researchers
tial pitfalls to be avoided. Although most usually spend most of their time and effort
of these considerations apply to any study designing the perfect experimental trials
in cognitive neuroscience, they are partic- so that they engage the mental process of
ularly relevant to emotion-related research. interest; for example, perception of fearful
In this section we present a few of the more faces, empathic responses, or remembered
important issues in our view. emotional events. Selection of an appro-
priate control condition, however, is also
critical if one is to draw the appropriate
Susceptibility Artifacts
conclusions from the observed activation,
As described in the “Physical Basis” section, because this approach relies on the notion
T2* contrast is sensitive to local inhomo- of cognitive subtraction and the principle of
geneities in magnetic susceptibility, such as pure insertion (Donders, 1969). That is, it is
those induced by the presence of deoxy- assumed that the only difference between
hemoglobin. However, macroscopic field the experimental and control trials is the
inhomogeneities, like those that arise in factor of interest (e.g., emotionality) and
tissue-air interfaces (which have different that the mathematical subtraction of the
magnetic susceptibilities), also affect the sig- responses it generates will isolate those, and
nal and introduce important artifacts, usu- only those, areas specifically involved in
ally in the form of image distortion and sig- that process. As an illustration of this issue,
nal loss in the voxels near air-filled cavities. we observed in a recent meta-analysis of
Unfortunately for affective neuroscience brain responses to visual emotional stimuli
PET AND FMRI 145
(Sergerie, Chochol, & Armony, 2008) that of design, Anderson and colleagues (2003)
effect sizes of amygdala activation are larger reported that amygdala activation correlated
when comparing an emotional visual stim- with subjects’ ratings of emotional inten-
ulus with a low-level condition, such as a sity, but not valence, of olfactory stimuli,
fixation cross or a scrambled stimulus, than whereas the orbitofrontal cortex exhibited
when using a neutral version of the same the opposite pattern; that is, a correla-
stimulus (e.g., fearful vs. neutral faces). tion with valence but not intensity. This
Although this finding suggests that the use approach, however, assumes that the sub-
of a simpler control is preferable in terms of jects and their brains do process the factor of
statistical power, because it increases sensi- interest in a parametric fashion, which may
tivity, this is done at the expense of speci- not always be the case (e.g., a morphed face
ficity. In other words, an activation obtained with 20% of fear may be perceived as neu-
in the contrast fearful faces versus fixation tral, whereas one with 30% may be consid-
cross may reflect an involvement of that ered to be fearful). Indeed, a recent study by
region in emotional (fear) processing, in the Zaretsky, Mendelsohn, Mintz, and Hendler
processing of faces, or more generally in the (2010) showed significant amygdala activa-
processing of complex visual stimuli, with tion associated with the perceived level of
little or no relation to their emotional value. threat from dynamic facial expressions of
Another important issue that deserves various levels of fear (obtained through mor-
attention when contrasting two trial types is phing different percentages of neutral and
that of apparent deactivations. Specifically, fear expressions of the same individual), but
if the control condition elicits a stronger not with the actual degree of fear present in
response than the experimental one, the the stimuli.
standard comparison will yield negative val-
ues, which could be interpreted as deac-
Block versus Event-Related Designs
tivations (representing neural inhibition)
associated with the condition of interest. Early fMRI studies employed a block
Although this result will typically not arise design, similar to those used in PET. How-
in the case of a simple, low-level baseline ever, as acquisition and analysis techniques
(such as a fixation cross), it is quite com- improved, it became possible to present
mon when the comparison involves two sim- stimuli in any given order and model indi-
ilarly complex or demanding conditions. For vidual events within the framework of the
instance, a deactivation associated with the GLM. These event-related designs, named
contrast of fearful minus happy faces could after ERP studies in electroencephalogra-
represent a true decrease in the BOLD signal phy (see Chapter 4), are very flexible and
for the fearful faces or a stronger response allow for the study of a variety of phenom-
for the happy ones. Introducing a third con- ena in a way that is not possible (or at least
dition (e.g., neutral faces or a low-level far from optimal) with block designs; these
baseline condition) may sometimes help phenomena include learning, oddball, and
in distinguishing between these different priming effects, as well as paradigms requir-
possibilities. ing an a posteriori assignment of stimuli to
An alternative approach to the subtrac- a particular category based on the subject’s
tion method, which does not require an response, such as those exploring the sub-
explicit control condition, is the use of a sequent memory effect (Dolcos, LaBar, &
parametric design. In this case, the fac- Cabeza, 2004; Sergerie, Lepage, & Armony,
tor of interest (e.g., emotional valence or 2006) or the effects of stimulus awareness
intensity) is varied along a given scale, and (Pessoa, Japee, Sturman, & Ungerleider,
a regression analysis is then performed, 2006; Vuilleumier et al., 2002). In addition,
instead of a categorical comparison (e.g., event-related designs are less susceptible to
t-test between experimental and control habituation, an issue of particular relevance
conditions). For instance, using this type in the case of the amygdala, where neurons
146 JORGE ARMONY & JUNG EUN HAN
rapidly decrease their response to repeated In this case, each event is characterized by
stimuli (Bordi & LeDoux, 1992; Wilson & a set of parameters (one for each function
Rolls, 1993). Indeed, several fMRI studies used in the model), and the resulting HDR
have shown rapid amygdala habituation to is built by the linear combination of the basis
repeated presentations of emotional stimuli functions, each multiplied by its parameter
(Breiter et al., 1996; Fischer et al., 2003), an estimate. The obvious advantage of such an
effect that appears to be more pronounced approach is that it can capture a wide range
in the right hemisphere (Sergerie et al., of HDRs with this flexible model. In con-
2008). Nonetheless, it should be noted that trast, its drawback is that in this case each
amygdala activation has been reported with parameter estimate is difficult to interpret
block designs in both PET and fMRI stud- on its own, and perhaps more importantly,
ies, even when using blocks of 17.5 minutes some of the results may correspond to “non-
in an FDG PET experiment (Fernandez- physiological” HDRs (i.e., their shapes are
Egea et al., 2009). However, block designs unlikely to correspond to a real response,
do have the advantages of being easier to for example, by failing to follow the known
analyze and not requiring an accurate model hemodynamic slow temporal course).
of the elicited hemodynamic response (see An often used compromise between
the later discussion). In addition, they have these two extreme options is to use the
much higher statistical power than event- standard or canonical HRF together with its
related or any other type of designs, even temporal derivative. Functions that, like the
in the case of the amygdala (Sergerie et al., HRF, go to zero at plus and minus infin-
2008). ity, have the property of being orthogonal
to their temporal derivatives. Thus, these
two basis functions explain different, non-
Modeling the HDR
overlapping components of the variance of
There are several underlying assumptions in the data (see the section, “Independence
the GLM approach, two of which are impor- of Explanatory Variables”). In addition, the
tant enough to have made it into its name. shape of the temporal derivative is such
As mentioned earlier, the expected response that, when linearly combined with the HRF,
for each event needs to be modeled; that it produces HDRs with different delays,
is, an assumption of its shape is required. depending on the relative weight of each
The particular model used can be very spe- function. Thus, such combinations can be
cific, such as when a “synthetic” hemody- used to examine differences in the latency
namic response function (HRF) is used. This of the response as a function of the stim-
has the advantages that only one parameter ulus type. Reinders et al. (2006) employed
needs to be estimated and its value is easy this approach to show that responses in the
to interpret: It directly represents the mag- amygdala-hippocampus junction occurred
nitude of the HDR for that particular event earlier for fearful than for neutral faces. The
type. Its disadvantage, however, is that this authors interpreted this finding as provid-
model will only capture HDRs that behave ing support for a preconscious, subcorti-
like those of the model and will fail to cap- cally mediated amygdala response to threat-
ture responses that have a different behavior related stimuli.
(e.g., longer latencies, shorter duration, and
so on). This disadvantage is important given
Linearity of the HDR
that, as previously mentioned, the exact
shape of the HDR is likely to vary as a func- A second assumption of the GLM is that
tion of regions, individuals, and stimuli or of the linearity of the responses, in accor-
tasks. To account for this potential variabil- dance with the superposition principle. That
ity in the HDR, one can use a general set of is, if a stimulus elicits a specific HDR, pre-
basis functions, such as Fourier, gamma, or senting the same (type of) stimulus twice
boxcar (“finite impulse response”) functions. within a short time interval should result
PET AND FMRI 147
in an overall response that is simply the (e.g., age or anxiety scores; see Chapter 24).
sum of the individual responses that each One can then identify brain regions whose
stimulus would elicit independently of the activity is modulated by that explanatory
other. Although this seems to be the case variable. In addition, one can enter variables
for intertrial intervals (ITI) longer than a of no interest, included simply to remove
couple of seconds (the specific cutoff time variance that has no experimental interest
depends on various factors), the linearity but that can explain some of the variabil-
assumption no longer holds for very short ity in the data. Introducing these confound-
ITIs (Birn & Bandettini, 2005; de Zwart ing variables has two main objectives: to
et al., 2009). The nonlinear effect usually reduce the residuals (“error”) of the model
manifests as a reduced amplitude of the and hence increase statistical power, and to
HDR for the second stimulus and there- account for some of the noise in the data that
fore can, if not taken into account, lead to otherwise could be assigned to the experi-
an under-estimation of the true response to mental conditions.
the stimulus. Interestingly, this limitation Typical examples of these variables are
can be turned into an advantage in studying the movement parameters obtained in the
the stimulus specificity of neural responses realignment preprocessing step, which cap-
in a particular region, as evidenced by the ture the signal changes associated with the
growing use of the so-called fMRI adapta- subject’s movement during the scan; these
tion technique (for a description and review parameters can be nonspecific (e.g., slowly
of this approach, see Krekelberg, Boynton, adjusting the position of the head to improve
& van Wezel, 2006). Rotshtein and col- comfort), or they can be related to the task,
leagues (2001) employed an fMRI adaptation such as when they are caused by button
paradigm to explore amygdala and cortical pressing, speech, startle caused by loud audi-
responses to unpleasant faces obtained by tory stimuli or sudden changes in luminos-
an “expressional transfiguration” (ET) effect ity of the visual stimulus, and so on. At
(inverting eyes and mouth). They observed the level of the subject, typical examples
that the lateral occipital complex was not are age, overall performance in the task, or
affected by valence, but exhibited differ- some other factor that can influence brain
ential patterns of adaptation depending on responses.
the stimulus value. In contrast, the amyg- One could be tempted to include as many
dala responded differently to ET and normal potential confounding variables as one can
faces, but its adaptation was not affected. think of. However, it is important to keep in
One interpretation provided by the authors mind that the GLM is essentially a multiple
is that the emotional valence of the stimuli regression model, and the parameter esti-
prevented neurons in the visual cortex from mates represent the values of the semipartial
adapting to repeated presentations of the ET correlation of the data and each covariate,
stimuli. after the variance explained by all the other
regressors in the model has been removed.
Therefore, if two covariates are correlated
Independence of Explanatory
(i.e., they explain some of the same vari-
Variables
ance in the data), the contribution of each
In addition to the basis functions represent- of them will be reduced. As an illustra-
ing the hemodynamic response (e.g., canon- tion, if one were to enter the exact same
ical HRF and its the temporal derivative), it regressor twice, the parameter estimate for
is possible to include in the model or design each instance of that regressor would be
matrix other variables that can explain some half of the original value. One alterna-
of the variance in the data. These can be tive to avoid this issue is to sequentially
entered at the level of each scan (e.g., physi- orthogonalize each new covariate that is
ological measures; see Chapter 3), trial (e.g., entered in the model with respect to the oth-
reaction times or valence ratings), or subject ers, although in this case the interpretation
148 JORGE ARMONY & JUNG EUN HAN
of their corresponding parameter estimates method, there are limitations and poten-
becomes more difficult. Sometimes it is tial pitfalls that one needs to consider when
desirable to enter an additional covariate designing, analyzing, or interpreting experi-
because it is correlated with the one of inter- ments using PET or fMRI.
est, to show that the results obtained are
not due to this other (usually uninterest- Outstanding Questions
ing) factor. For instance, Dickie, Bruner,
Akerib, and Armony (2008) reported a cor- r The amygdala is composed of several sub-
relation between amygdala and ventrome- nuclei with unique cytoarchitectonic and
dial PFC activations for successful mem- physiological properties. Yet, to date, the
ory encoding of fearful faces and symptom
spatial resolution of most fMRI systems
severity in individuals suffering from post- does not allow for an unambiguous local-
traumatic stress disorder (PTSD). Because ization of the observed activations to spe-
depression is often a comorbid condition in
cific regions within this structure. Will
PTSD and is likely to influence emotional high-field (e.g., 7T) fMRI hold the key
processing, it was important to rule out to solve this problem?
the possibility that the results were due to r Functional neuroimaging techniques are
depression rather than PTSD symptomatol- inherently correlational and thus do not
ogy. By entering both variables in the anal-
speak to causality. Will further develop-
ysis, the researchers showed that, although
ments in analysis techniques (e.g., con-
the strength of the correlation with PTSD nectivity), as well as the combination of
symptom severity was reduced, as expected
neuroimaging with other methods (e.g.,
because both variables are typically highly
lesions, electrical or magnetic stimula-
correlated, the results were still statistically tion), help better understand the role of
significant. In contrast, the correlations with
the different brain regions in emotional
depression scores did not reach the thresh- processing?
old of statistical significance and therefore r Most reported findings from neuroimag-
could not explain the observed activations.
ing studies are based on group averages.
Whereas results from single-subject data
are fairly robust and easy to interpret
Conclusion for simple sensory or motor tasks, that
is not the case for most of the emotional
In summary, functional neuroimaging tech-
paradigms used in the literature. Could
niques, particularly fMRI, have allowed
further advances in acquisition and anal-
researchers to make great advances in affec-
ysis methods solve this problem and even-
tive neuroscience research. Although ear-
tually pave the way for the use of fMRI
lier studies largely confirmed findings previ-
as a diagnostic tool in psychiatry?
ously obtained in experimental animals and
with neurological patients, more recently
researchers have begun using sophisticated Acknowledgments
and innovative paradigms and analysis tech-
niques to advance our understanding of We are grateful to Bruce Pike for comments
the neural basis of emotional processing on an earlier version of this chapter. JLA was
in the healthy brain and to shed light on supported by the Canada Research Chairs
the dysfunction of the emotional brain in a Program.
variety of psychiatric disorders. The integra-
tion of these techniques with EEG, tran-
scranial magnetic stimulation (TMS), and Notes
other approaches should pave the way for
even further advances in this exciting field. 1 On a historical note, Pauling and Coryell
Nonetheless, it is important to keep in mind began their seminal article by quoting Fara-
that, as is the case with any experimental day as mentioning that he had examined the
PET AND FMRI 149
magnetic properties of dried blood and that frontal cortex glucose metabolism common to
he needed to (but never did) try “recent fluid three types of depression. Archives of General
blood.” However, it seems that Faraday did Psychiatry, 46(3), 243–50.
try fluid blood and still found it to be diamag- Bernstein, M. A., King, K. F., & Zhou, Z. J. (2004).
netic. In fact, he argued that “if a man could Handbook of MRI pulse sequences. Amsterdam:
be suspended . . . and placed in the magnetic Elsevier Academic Press.
field, he would point equatorially [perpen- Birn, R. M., & Bandettini, P. A. (2005). The effect
dicular to the magnetic field lines]; for all the of stimulus duty cycle and “off” duration on
substances of which he is formed, including BOLD response linearity. Neuroimage, 27(1),
the blood, possess this property [of diamag- 70–82.
netism].” Bordi, F., & LeDoux, J. (1992). Sensory tuning
2 The authors mention that the study actually beyond the sensory system: An initial analysis
begun before the publication of the first fMRI of auditory response properties of neurons in
articles in 1992 and was submitted for publi- the lateral amygdaloid nucleus and overlying
cation in May 1994, but that it took more than areas of the striatum. Journal of Neuroscience,
two years for it to finally appear in press (Bre- 12(7), 2493–2503.
iter & Rauch, 1996). Breiter, H. C., Etcoff, N. L., Whalen, P. J.,
Kennedy, W. A., Rauch, S. L., Buckner,
R. . . . Rosen, B. R. (1996). Response and habit-
References uation of the human amygdala during visual
processing of facial expression. Neuron, 17(5),
Abraham, T., & Feng, J. (2011). Evolution of brain 875–87.
imaging instrumentation. Seminars in Nuclear Breiter, H. C., Kwong, K. K., Baker, J. R., Stern, J.
Medicine, 41(3), 202–19. W., Belliveau, J. W., Davis, T. L., . . . Rosen, B.
Aguirre, G. K., Zarahn, E., & D’Esposito, M. R. (1993). Functional magnetic resonance imag-
(1998). The variability of human, BOLD ing of symptom provocation in patients with
hemodynamic responses. Neuroimage, 8(4), obsessive-compulsive disorder versus controls.
360–69. Paper presented at the International Society
Anderson, A. K., Christoff, K., Stappen, I., for Magnetic Resonance in Medicine.
Panitz, D., Ghahremani, D. G., Glover, Breiter, H. C., & Rauch, S. L. (1996). Functional
G., . . . Sobel, N. (2003). Dissociated neural MRI and the study of OCD: From symptom
representations of intensity and valence in provocation to cognitive-behavioral probes
human olfaction. Nature Neuroscience, 6(2), of cortico-striatal systems and the amygdala.
196–202. Neuroimage, 4(3 Pt. 3), S127–38.
Anderson, C. D. (1932). The apparent exis- Breiter, H. C., Rauch, S. L., Kwong, K. K.,
tence of easily deflectable positives. Science, Baker, J. R., Weisskoff, R. M., Kennedy,
76(1967), 238–39. D. N., . . . Rosen, B. R. (1996). Functional
Attwell, D., & Iadecola, C. (2002). The neu- magnetic resonance imaging of symptom
ral basis of functional brain imaging signals. provocation in obsessive-compulsive disorder.
Trends in Neurosciences, 25(12), 621–25. Archives of General Psychiatry, 53(7), 595–606.
Attwell, D., & Laughlin, S. B. (2001). An energy Brown, W. D., Taylor, M. D., Roberts, A.
budget for signaling in the grey matter of D., Oakes, T. R., Schueller, M. J., Holden,
the brain. Journal of Cerebral Blood Flow & J. E., . . . Nickles, R. J. (1999). FluoroDOPA
Metabolism, 21(10), 1133–45. PET shows the nondopaminergic as well as
Badgaiyan, R. D., Fischman, A. J., & Alpert, dopaminergic destinations of levodopa. Neu-
N. M. (2009). Dopamine release during rology, 53(6), 1212–18.
human emotional processing. Neuroimage, Buxton, R. B. (2002). Introduction to functional
47(4), 2041–45. magnetic resonance imaging: Principles and
Bandettini, P. A., Wong, E. C., Hinks, R. S., techniques. Cambridge: Cambridge University
Tikofsky, R. S., & Hyde, J. S. (1992). Time Press.
course EPI of human brain function dur- Chen, N. K., Dickey, C. C., Yoo, S. S.,
ing task activation. Magnetic Resonance in Guttmann, C. R., & Panych, L. P. (2003).
Medicine, 25(2), 390–97. Selection of voxel size and slice orientation
Baxter, L. R., Jr., Schwartz, J. M., Phelps, M. for fMRI in the presence of susceptibility field
E., Mazziotta, J. C., Guze, B. H., Selin, C. gradients: Application to imaging of the amyg-
E., . . . Sumida, R. M. (1989). Reduction of pre- dala. Neuroimage, 19(3), 817–25.
150 JORGE ARMONY & JUNG EUN HAN
de Zwart, J. A., van Gelderen, P., Jansma, ing signals. European Journal of Neuroscience,
J. M., Fukunaga, M., Bianciardi, M., & Duyn, 33(4), 577–88.
J. H. (2009). Hemodynamic nonlinearities Fischer, H., Wright, C. I., Whalen, P. J., McIner-
affect BOLD fMRI response timing and ney, S. C., Shin, L. M., & Rauch, S. L. (2003).
amplitude. Neuroimage, 47(4), 1649–58. Brain habituation during repeated exposure
Devor, A., Tian, P., Nishimura, N., Teng, I. C., to fearful and neutral faces: A functional MRI
Hillman, E. M., Narayanan, S. N., . . . Dale, study. Brain Research Bulletin, 59(5), 387–92.
A. M. (2007). Suppressed neuronal activ- Genovese, C. R., Lazar, N. A., & Nichols, T.
ity and concurrent arteriolar vasoconstric- (2002). Thresholding of statistical maps in
tion may explain negative blood oxygena- functional neuroimaging using the false dis-
tion level-dependent signal. Journal of Neu- covery rate. Neuroimage, 15(4), 870–78.
roscience, 27(16), 4452–59. George, M. S., Ketter, T. A., Parekh, P. I.,
Dickie, E. W., Brunet, A., Akerib, V., & Armony, Horwitz, B., Herscovitch, P., & Post, R. M.
J. L. (2008). An fMRI investigation of mem- (1995). Brain activity during transient sadness
ory encoding in PTSD: Influence of symptom and happiness in healthy women. American
severity. Neuropsychologia, 46(5), 1522–31. Journal of Psychiatry, 152(3), 341–51.
Dirac, P. A. M. (1928). The quantum theory of Gerretsen, P., Graff-Guerrero, A., Menon, M.,
the electron. Proceedings of the Royal Society of Pollock, B. G., Kapur, S., Vasdev, N., Houle,
London A, 117, 15. S., & Mamo, D. (2010). Is desire for social rela-
Dolcos, F., LaBar, K. S., & Cabeza, R. (2004). tionships mediated by the serotonergic system
Dissociable effects of arousal and valence in the prefrontal cortex? An [(18)F]setoperone
on prefrontal activity indexing emotional PET study. Social Neuroscience, 5(4),
evaluation and subsequent memory: An 375–83.
event-related fMRI study. Neuroimage, 23(1), Goldapple, K., Segal, Z., Garson, C., Lau, M.,
64–74. Bieling, P., Kennedy, S., & Mayberg, H.
Donders, F. C. (1969). Over de snelheid van (2004). Modulation of cortical-limbic path-
psychische processen [On the speed of psy- ways in major depression: Treatment-specific
chological processes]. In W. G. Koster & effects of cognitive behavior therapy. Archives
Instituut voor Perceptie Onderzoek (Eind- of General Psychiatry, 61(1), 34–41.
hoven Netherlands) (Eds.), Attention and per- Grodd, W., Schneider, F., Klose, U., & Nagele,
formance II: Proceedings of the Donders Cen- T. (1995). [Functional magnetic resonance
tenary Symposium on Reaction Time, held in tomography of psychological functions exem-
Eindhoven, July 29-August 2, 1968. Amsterdam: plified by experimentally induced emotions.
North-Holland Publishing. Radiologe, 35(4), 283–89.
Ernst, T., & Hennig, J. (1994). Observation of a Gusnard, D. A., & Raichle, M. E. (2001). Search-
fast response in functional MR. Magnetic Res- ing for a baseline: Functional imaging and the
onance in Medicine, 32(1), 146–49. resting human brain. Nature Reviews Neuro-
Faraday, M. (1846). Experimental researches in science, 2(10), 685–94.
electricity. Nineteenth series. On the magne- Haacke, E. M., Brown, R. W., Thompson,
tization of light and the illumination of mag- M. R., & Venkatesan, R. (1999). Magnetic reso-
netic lines of force. Philosophical Transactions nance imaging: Physical principles and sequence
of the Royal Society of London, 136, 1–20. design. New York: Wiley-Liss.
Fermi, E. (1934). Versuch einer Theorie der β- Harel, N., Lee, S. P., Nagaoka, T., Kim, D. S., &
Strahlen [Towards the Theory of β-Rays]. Z. Kim, S. G. (2002). Origin of negative blood
Phys., 88, 17. oxygenation level-dependent fMRI signals.
Fernandez-Egea, E., Parellada, E., Lomena, F., Journal of Cerebral Blood Flow & Metabolism,
Falcon, C., Pavia, J., Mane, A., . . . Bernardo, 22(8), 908–17.
M. (2009). A continuous emotional task acti- Hu, X., Le, T. H., & Ugurbil, K. (1997). Eval-
vates the left amygdala in healthy volunteers: uation of the early response in fMRI in indi-
(18)FDG PET study. Psychiatry Research, vidual subjects using short stimulus duration.
171(3), 199–206. Magnetic Resonance in Medicine, 37(6), 877–84.
Figley, C. R., & Stroman, P. W. (2011). The Huettel, S. A., & McCarthy, G. (2001). Regional
role(s) of astrocytes and astrocyte activity differences in the refractory period of the
in neurometabolism, neurovascular coupling, hemodynamic response: An event-related
and the production of functional neuroimag- fMRI study. Neuroimage, 14(5), 967–76.
PET AND FMRI 151
Huettel, S. A., Song, A. W., & McCarthy, trol (Vol. 1235, pp. 137–44). Amsterdam: Else-
G. (2009). Functional magnetic resonance imag- vier.
ing (2nd ed.). Sunderland, MA: Sinauer Asso- Logan, B. R., & Rowe, D. B. (2004). An evaluation
ciates. of thresholding techniques in fMRI analysis.
Jezzard, P., Matthews, P. M., & Smith, S. M. Neuroimage, 22(1), 95–108.
(2001). Functional MRI: An introduction to Logothetis, N. (2000). Can current fMRI tech-
methods. Oxford: Oxford University Press. niques reveal the micro-architecture of cor-
Kannurpatti, S. S., & Biswal, B. B. (2004). Nega- tex? Nature Neuroscience, 3(5), 413–14.
tive functional response to sensory stimulation Logothetis, N. K., Pauls, J., Augath, M., Trinath,
and its origins. Journal of Cerebral Blood Flow T., & Oeltermann, A. (2001). Neurophysio-
& Metabolism, 24(6), 703–12. logical investigation of the basis of the fMRI
Kosslyn, S. M., Shin, L. M., Thompson, W. L., signal. Nature, 412(6843), 150–57.
McNally, R. J., Rauch, S. L., Pitman, R. K., Logothetis, N. K., & Wandell, B. A. (2004). Inter-
& Alpert, N. M. (1996). Neural effects of preting the BOLD signal. Annual Review of
visualizing and perceiving aversive stimuli: A Physiology, 66, 735–69.
PET investigation. Neuroreport, 7(10), 1569– Merboldt, K. D., Fransson, P., Bruhn, H., &
76. Frahm, J. (2001). Functional MRI of the
Krekelberg, B., Boynton, G. M., & van Wezel, human amygdala? Neuroimage, 14(2), 253–
R. J. (2006). Adaptation: From single cells to 57.
BOLD signals. Trends in Neurosciences, 29(5), Morawetz, C., Holz, P., Lange, C., Baudewig, J.,
250–56. Weniger, G., Irle, E., & Dechent, P. (2008).
Kupers, R., Frokjaer, V. G., Erritzoe, D., Improved functional mapping of the human
Naert, A., Budtz-Joergensen, E., Nielsen, F. amygdala using a standard functional mag-
A., . . . Knudsen, G. M. (2011). Serotonin trans- netic resonance imaging sequence with sim-
porter binding in the hypothalamus corre- ple modifications. Magnetic Resonance Imag-
lates negatively with tonic heat pain ratings ing, 26(1), 45–53.
in healthy subjects: A [11C]DASB PET study. Morris, J. S., Frith, C. D., Perrett, D. I., Rowland,
Neuroimage, 54(2), 1336–43. D., Young, A. W., Calder, A. J., & Dolan,
Kwong, K. K., Belliveau, J. W., Chesler, D. A., R. J. (1996). A differential neural response
Goldberg, I. E., Weisskoff, R. M., Poncelet, in the human amygdala to fearful and happy
B. P., . . . Rosen, B. R. (1992). Dynamic mag- facial expressions. Nature, 383(6603), 812–15.
netic resonance imaging of human brain activ- Mosso, A. (1881). Uber den Kreislauf des Blutes im
ity during primary sensory stimulation. Pro- menschlichen Gehirn. Leipzig: Veit.
ceedings of the National Academy of Sciences, Mosso, A. (1896). Fear. London: Longmans,
89(12), 5675–79. Green.
Laakso, A., Wallius, E., Kajander, J., Bergman, Mukamel, R., Gelbard, H., Arieli, A., Hasson,
J., Eskola, O., Solin, O., . . . Hietala, J. (2003). U., Fried, I., & Malach, R. (2005). Cou-
Personality traits and striatal dopamine syn- pling between neuronal firing, field potentials,
thesis capacity in healthy subjects. American and FMRI in human auditory cortex. Science,
Journal of Psychiatry, 160(5), 904–910. 309(5736), 951–54.
Lauritzen, M. (2005). Reading vascular changes Ogawa, S., Lee, T. M., Kay, A. R., & Tank, D.
in brain imaging: Is dendritic calcium the key? W. (1990). Brain magnetic resonance imaging
Nature Reviews Neuroscience, 6(1), 77–85. with contrast dependent on blood oxygena-
Le Bihan, D., Turner, R., Zeffiro, T. A., Cuenod, tion. Proceedings of the National Academy of
C. A., Jezzard, P., & Bonnerot, V. (1993). Acti- Sciences, 87(24), 9868–72.
vation of human primary visual cortex during Ogawa, S., Lee, T. M., Nayak, A. S., & Glynn,
visual recall: A magnetic resonance imaging P. (1990). Oxygenation-sensitive contrast in
study. Proceedings of the National Academy of magnetic resonance image of rodent brain at
Sciences, 90(24), 11802–5. high magnetic fields. Magnetic Resonance in
Lindauer, U., Royl, G., Leithner, C., Kühl, Medicine, 14(1), 68–78.
M., Gethmann, J., Kohl-Bareis, . . . Dirnagl, U. Ogawa, S., Tank, D. W., Menon, R., Ellermann,
(2002). Neural activation induced changes in J. M., Kim, S. G., Merkle, H., & Ugurbil,
microcirculatory haemoglobin oxygenation: K. (1992). Intrinsic signal changes accompa-
To dip or not to dip. In M. Tomita, I. Kanno, & nying sensory stimulation: Functional brain
E. Hamel (Eds.), Brain activation and CBF con- mapping with magnetic resonance imaging.
152 JORGE ARMONY & JUNG EUN HAN
Proceedings of the National Academy of Sci- ipation and experience of peak emotion to
ences, 89(13), 5951–55. music. Nature Neuroscience, 14(2), 257–62.
Pauling, L., & Coryell, C. D. (1936). The magnetic Schaefer, S. M., Abercrombie, H. C., Lindgren,
properties and structure of hemoglobin, oxy- K. A., Larson, C. L., Ward, R. T., Oakes, T.
hemoglobin and carbonmonoxyhemoglobin. R., . . . Davidson, R. J. (2000). Six-month test-
Proceedings of the National Academy of, 22(4), retest reliability of MRI-defined PET measures
210–16. of regional cerebral glucose metabolic rate in
Penny, W. D., Friston, K. J., Ashburner, J. T., selected subcortical structures. Human Brain
Kiebel, S. J., & Nichols, T. E. (2007). Statistical Mapping, 10(1), 1–9.
parametric mapping: The analysis of functional Sergerie, K., Chochol, C., & Armony, J. L. (2008).
brain images. Amsterdam: Elsevier. The role of the amygdala in emotional pro-
Pessoa, L., Japee, S., Sturman, D., & Ungerleider, cessing: A quantitative meta-analysis of func-
L. G. (2006). Target visibility and visual aware- tional neuroimaging studies. Neuroscience &
ness modulate amygdala responses to fearful Biobehavioral Reviews, 32(4), 811–30.
faces. Cerebral Cortex, 16(3), 366–75. Sergerie, K., Lepage, M., & Armony, J. L. (2006).
Preston, A. R., Thomason, M. E., Ochsner, K. A process-specific functional dissociation of
N., Cooper, J. C., & Glover, G. H. (2004). the amygdala in emotional memory. Journal
Comparison of spiral-in/out and spiral-out of Cognitive Neuroscience, 18(8), 1359–87.
BOLD fMRI at 1.5 and 3 T. Neuroimage, 21(1), Shmuel, A. (2010). Locally measured neu-
291–301. ronal correlates of functional MRI signals. In
Pruessner, J. C., Champagne, F., Meaney, C. Mulert & L. Lemieux (Eds.), EEG-fMRI:
M. J., & Dagher, A. (2004). Dopamine release Physiological basis, technique, and applications
in response to a psychological stress in humans (pp. 63–82). Heidelberg: Springer.
and its relationship to early life maternal Shmuel, A., Augath, M., Oeltermann, A., &
care: A positron emission tomography study Logothetis, N. K. (2006). Negative functional
using [11C]raclopride. Journal of Neuroscience, MRI response correlates with decreases in
24(11), 2825–31. neuronal activity in monkey visual area V1.
Rao, S., Bandettini, P. A., Wong, E. C., Nature Neuroscience, 9(4), 569–77.
Yetkin, F. Z., Hammeke, T. A., Mueller, Shmuel, A., & Grinvald, A. (1996). Functional
W. M., . . . Hyde, J. S. (1992). Gradient echo organization for direction of motion and its
EPI demonstrates bilateral superior temporal relationship to orientation maps in cat area 18.
gyrus activation during passive word presenta- Journal of Neuroscience, 16(21), 6945–6964.
tion. Paper presented at the 11th Annual Meet- Smith, A. J., Blumenfeld, H., Behar, K. L., Roth-
ing of the Society of Magnetic Resonance in man, D. L., Shulman, R. G., & Hyder, F.
Medicine, Berlin. (2002). Cerebral energetics and spiking fre-
Reiman, E. M., Raichle, M. E., Butler, F. K., Her- quency: The neurophysiological basis of fMRI.
scovitch, P., & Robins, E. (1984). A focal brain Proceedings of the National Academy of Sci-
abnormality in panic disorder, a severe form ences, 99(16), 10765–70.
of anxiety. Nature, 310(5979), 683–85. Smith, S. M., Miller, K. L., Salimi-Khorshidi, G.,
Reinders, A. A., Glascher, J., de Jong, J. Webster, M., Beckmann, C. F., Nichols, T.
R., Willemsen, A. T., den Boer, J. A., & E., . . . Woolrich, M. W. (2011). Network mod-
Buchel, C. (2006). Detecting fearful and elling methods for FMRI. Neuroimage, 54(2),
neutral faces: BOLD latency differences in 875–91.
amygdala-hippocampal junction. Neuroimage, Stern, C. E., Corkin, S., Gonzalez, R. G.,
33(2), 805–14. Guimaraes, A. R., Baker, J. R., Jennings, P.
Rotshtein, P., Malach, R., Hadar, U., Graif, M., & J., . . . Rosen, B. R. (1996). The hippocampal
Hendler, T. (2001). Feeling or features: Differ- formation participates in novel picture encod-
ent sensitivity to emotion in high-order visual ing: Evidence from functional magnetic res-
cortex and amygdala. Neuron, 32(4), 747–57. onance imaging. Proceedings of the National
Roy, C. S., & Sherrington, C. S. (1890). On the Academy of Sciences, 93(16), 8660–65.
regulation of the blood-supply of the brain. Takahashi, H., Takano, H., Kodaka, F., Arakawa,
Journal of Physiology, 11(1–2), 85–158. R., Yamada, M., Otsuka, T., . . . Suhara, T.
Salimpoor, V. N., Benovoy, M., Larcher, K., (2010). Contribution of dopamine D1 and D2
Dagher, A., & Zatorre, R. J. (2011). Anatomi- receptors to amygdala activity in human. Jour-
cally distinct dopamine release during antic- nal Neuroscience, 30(8), 3043–47.
PET AND FMRI 153
Takano, T., Tian, G. F., Peng, W., Lou, N., Volkow, N. D., Tomasi, D., Wang, G. J., Fowler,
Libionka, W., Han, X., & Nedergaard, M. J. S., Telang, F., Goldstein, R. Z., . . . Alexoff,
(2006). Astrocyte-mediated control of cere- D. (2011). Positive emotionality is associated
bral blood flow. Nature Neuroscience, 9(2), with baseline metabolism in orbitofrontal cor-
260–67. tex and in regions of the default network.
Tauscher, J., Bagby, R. M., Javanmard, M., Molecular Psychiatry, 16(8), 818–25.
Christensen, B. K., Kasper, S., & Kapur, Vuilleumier, P., Armony, J. L., Clarke, K.,
S. (2001). Inverse relationship between sero- Husain, M., Driver, J., & Dolan, R. J. (2002).
tonin 5-HT(1A) receptor binding and anxi- Neural response to emotional faces with and
ety: A [(11)C]WAY-100635 PET investigation without awareness: Event-related fMRI in a
in healthy volunteers. American Journal of Psy- parietal patient with visual extinction and spa-
chiatry, 158(8), 1326–28. tial neglect. Neuropsychologia, 40(12), 2156–66.
Thulborn, K. R., Waterton, J. C., Matthews, Wilson, F. A., & Rolls, E. T. (1993). The effects of
P. M., & Radda, G. K. (1982). Oxygenation stimulus novelty and familiarity on neuronal
dependence of the transverse relaxation time activity in the amygdala of monkeys perform-
of water protons in whole blood at high ing recognition memory tasks. Experimental
field. Biochimica et Biophysica Acta, 714(2), Brain Research, 93(3), 367–82.
265–70. Worsley, K. J., Marrett, S., Neelin, P., Vandal,
Truong, T. K., & Song, A. W. (2008). Single-shot A. C., Friston, K. J., & Evans, A. C. (1996).
dual-z-shimmed sensitivity-encoded spiral- A unified statistical approach for determining
in/out imaging for functional MRI with significant signals in images of cerebral activa-
reduced susceptibility artifacts. Magnetic Res- tion. Human Brain Mapping, 4(1), 58–73.
onance in Medicine, 59(1), 221–27. Yacoub, E., & Hu, X. (1999). Detection of the
Vanzetta, I., & Grinvald, A. (2001). Evidence and early negative response in fMRI at 1.5 Tesla.
lack of evidence for the initial dip in the anes- Magnetic Resonance in Medicine, 41(6), 1088–
thetized rat: Implications for human func- 92.
tional brain imaging. Neuroimage, 13(6 Pt. 1), Zaretsky, M., Mendelsohn, A., Mintz, M.,
959–67. & Hendler, T. (2010). In the eye of the
Viswanathan, A., & Freeman, R. D. (2007). beholder: Internally driven uncertainty of dan-
Neurometabolic coupling in cerebral cortex ger recruits the amygdala and dorsomedial
reflects synaptic more than spiking activity. prefrontal cortex. Journal of Cognitive Neuro-
Nature Neuroscience, 10(10), 1308–12. science, 22(10), 2263–75.
CHAPTER 6
Lesley K. Fellows
154
LESION STUDIES IN AFFECTIVE NEUROSCIENCE 155
allow testing of necessity claims: that is, and Wernicke about how language is dis-
they can provide evidence to support the rupted after brain injury. The clinical obser-
hypothesis that a particular brain region is vations that language production could be
necessary for a given process or component impaired, and comprehension spared (and
of behavior (Fellows et al., 2005; Rorden & vice versa), first made more than a century
Karnath, 2004). Such loss-of-function evi- ago, ground our understanding of how lan-
dence is an important complement to the guage is organized in the brain still today
activation studies common with fMRI or (Caplan, 2003).
EEG methods. The latter two approaches Lesion studies thus offer particular
instead relate measures of regional brain inferential strengths, complementing other
activation to aspects of behavior. Thus, approaches (Chatterjee, 2005; Fellows et al.,
fMRI or EEG studies can identify brain 2005; Rorden & Karnath, 2004). As with
regions that participate in a given cogni- any method, the quality of the evidence
tive or affective process, but cannot establish provided depends on thoughtful and careful
that these regions are critical for the process. experimental design. Advances in anatom-
This point may be particularly impor- ical neuroimaging, systematic patient rec-
tant in affective and social neuroscience, a ruitment, and analytic methods “borrowed”
newer area of study that aims to under- from fMRI all provide new opportunities to
stand complex processes that may be partic- ensure that modern experimental neuropsy-
ularly challenging to dissect. For example, chology can be effectively brought to bear
recognizing an emotional facial expression to inform our understanding of human brain
may also invoke the experience of a similar functions.
emotion in the subject, with attendant auto-
nomic changes. All of these processes may
be correlated with emotion recognition per- Experimental Design
formance, although only some are critical to
General Considerations
recognition itself. It can be very challeng-
ing to disentangle the processes of interest Human lesion studies are at best quasi-
from epiphenomena as they relate to fMRI experimental. For obvious reasons, the
activation patterns (Fusar-Poli et al., 2009). lesions are not under experimental control,
Converging lesion evidence can be very which means that the extent of brain injury
helpful in this regard. We might choose a is at the whim of nature, as is the “selection”
candidate region of the brain from an fMRI of the individuals who suffer those injuries.
study of emotion recognition, such as the The researcher using these methods is carry-
ventromedial prefrontal cortex, for exam- ing out an observational study, akin to a case-
ple, and ask whether damage to that region control study in epidemiology. The funda-
disrupts emotion recognition. Finding that mentally observational nature of this work
this is the case reassures us that this region means that the experimenter can choose
participates in the process of interest, rather whether to consider the brain (i.e., the lesion
than in some correlated additional pro- location) or the behavior as the indepen-
cess (Heberlein, Padon, Gillihan, Farah, & dent variable. Thus, studies can enroll sub-
Fellows, 2008; Hornak et al., 2003). jects with a particular behavioral deficit –
A second, related application of lesion for example, patients with depression after
methods is in testing the dissociability of stroke – and then ask whether they have
(putative) processes. Closely linked aspects common locations of damage (the tenta-
of behavior may rely on distinct neuro- tive answer, it turns out, is “no”; Carson
anatomical circuits: Work in patients with et al., 2000). Alternatively, one can ask
focal damage can demonstrate this involve- about the behavioral effects of damage to
ment in ways that correlative methods such a particular region: for example, selecting
as fMRI cannot. A classic example comes patients with amygdala damage and ascer-
from observations stretching back to Broca taining their behavioral deficits (Adolphs
156 LESLEY K. FELLOWS
et al., 2005; Adolphs & Spezio, 2006; Adolphs ing the lesion to begin with and so will be
& Tranel, 2003; Heberlein & Adolphs, 2004). over-represented in the lesion group com-
Finally, some studies are concerned not with pared to a healthy control group. A classic
structure-function relationships, but with example is risk-taking, which might plau-
understanding the relationship between psy- sibly predispose to head injury rather than
chological processes. If two abilities rely on being its consequence.
identical underlying psychological (and neu- There may be other reasons for the
ral) processes, then it should not be possi- behavioral deficit not to be caused by the
ble to disrupt one ability and not the other. focal brain injury itself. Common contrib-
Thus, studies in clinical populations can utors to poor performance include non-
test whether processes are shared or dis- specific additional injury that may or may
sociable, with or without considering their not be identifiable on brain imaging. Exam-
neurobiological substrates (Johnsen, Tranel, ples of such coincident injury include
Lutgendorf, & Adolphs, 2009; Robinson & diffuse axonal injury in traumatic brain
Sahakian, 2009). injury, hydrocephalus complicating cerebral
hemorrhage, or periventricular ischemic
changes in patients with a large-vessel
Patient Recruitment and Selection
stroke. Other important and common clin-
Although a study may be behavior driven or ical confounds include psychoactive medi-
lesion driven, it certainly should not be both. cation use, depression, or past or present
Selecting patients on the basis of both a clin- drug or alcohol abuse. There are two main
ical deficit and a particular lesion will intro- approaches to controlling for these issues:
duce a strong bias toward finding a (possibly One is to exclude participants for these
spurious) structure-function relationship, a reasons, and the other is to ensure that
kind of “cherry-picking” that undermines these factors are also present in the refer-
any conclusions (Rorden, Fridriksson, & Kar- ence group. A third approach is to include
nath, 2009). Other forms of selection bias one or more of these variables as a covari-
may be less obvious, but still need consider- ate in the analyses. The covariate approach
ation in interpreting lesion studies. In prin- can be useful, but often is not, because the
ciple, one should study every patient with small sample sizes of most lesion studies
either the lesion or behavior of interest. In bias toward failing to detect (statistically)
practice, there are nonrandom limitations to the effects of confounding clinical variables
this goal: At the least patients must con- even when they exist.
sent to be studied and be willing and able Lesion chronicity also needs considera-
to participate in the experiments. This form tion. The acute effects of stroke clearly differ
of selection bias may be a particular issue from the chronic effects, examined weeks or
for affective neuroscience. Patients suffering months later (see, e.g., Ochfeld et al., 2011).
from loss of motivation, depression, impair- For feasibility reasons, most lesion stud-
ments in empathy, or social deficits may be ies are carried out when patients are “sta-
less likely to volunteer for studies, meaning ble,” typically at least several months (and
that those who do participate might have often many years) after the brain injury.
less of the behavior of interest, biasing the Hyperacute studies can be done in the
results toward the null. On similar grounds, right setting (see, e.g., Marsh & Hillis, 2008;
lesions that lead to very difficult behavioral Newhart, Ken, Kleinman, Heidler-Gary, &
changes – impulsivity, social inappropriate- Hillis, 2007). It is obviously important to
ness, or aggression – will similarly be under- consider why there are differences between
represented in most studies. Even if such acute and chronic lesion effects. Some of
patients consent, the researcher may find the reasons are definitely relevant to inter-
it impossible to carry out the experiments. preting structure-function claims: Chronic
Finally, certain personality or demographic injury likely leads to compensation, whe-
characteristics may be risk factors for suffer- ther through plastic changes in the brain,
LESION STUDIES IN AFFECTIVE NEUROSCIENCE 157
2002). Case studies have been important in reference group: It should be large enough
affective neuroscience as well: As examples, to definitively establish that the deficit in
a detailed case report describing the social the patient is not within the range of nor-
and decision-making deficits following ven- mal, and it should be appropriately matched
tromedial frontal lobe damage in patient E. on relevant variables, whether clinical or
V. R. set the scene for much of the work demographic. Statistical tests that address
on the functions of this region in the last 20 the common issues that arise when compar-
years (Eslinger & Damasio, 1985), and a set ing individual performance to small refer-
of interesting studies on emotion recogni- ence groups should be applied as relevant
tion has been carried out in a unique patient (for a review of these important statistical
with bilateral amygdala injury (Adolphs analysis issues, see Crawford & Garthwaite,
et al., 2005; Adolphs, Tranel, Damasio, & 2012). Even if the experiment passes these
Damasio, 1994; Adolphs, Tranel, Damasio, standards with flying colors, the result can
& Damasio, 1995; Heberlein & Adolphs, be considered interesting or provocative, but
2004). not definitive in isolation.
Although single cases have been a rich
source of inspiration in the history of neu-
Group Studies
ropsychology, in principle they are weak evi-
dence taken alone: It is impossible to gener- Many of the drawbacks of single case stud-
alize any finding with confidence. Observed ies can be addressed with group designs. At
deficits in a single patient may be due to their most basic, such experiments can be
premorbid differences in function, so that seen simply as a series of single cases. If
normal individual differences may thus be each individual patient with a similar lesion
misattributed to the lesion. This attribu- shows a similar behavioral deficit, then the
tion, however, may be implausible for some generality of the structure-function claim is
deficits: Common sense dictates that major obviously much stronger: Individual differ-
hemiparesis or visual field defects are out- ences, whether premorbid or lesion related,
side the range of “normal” variation and can become less and less likely explanations for
generally be safely linked to the brain injury. the observed effects.
Yet other aspects of behavior, particularly Group studies can do better than this,
in emotional or social domains, may dif- however, by providing opportunities to fur-
fer substantially across healthy individuals, ther refine lesion analysis. Lesions are often
making it more likely that such a difference more extensive, or less precisely located,
will be found by chance in a brain-injured than is ideal for testing a given structure-
patient. Idiosyncrasies in brain organiza- function hypothesis. If the function is dis-
tion, structure-function mapping, or recov- rupted but the lesion is large, the conclu-
ery from brain injury can also contribute to sions cannot be very specific. If a group
“exceptional” performance in a single case. of patients with lesions varying in extent,
Finally, observations in a single case may be but overlapping in some smaller area, are
confounded by lesion-related factors other found to have a common impairment in
than the site of damage, such as lesion etiol- function, one can tentatively infer that the
ogy, comorbidities, or medication use. function relies on the region of overlap that
The critical reader of a single case study is common across patients. Recent methods
should consider the following issues: How have built on this logic to allow statistical
well defined is the lesion? How precisely tests of structure-function relations at the
characterized is the behavior being studied? voxel cluster level, and they are discussed
Are there clinical or demographic features in more detail later. Thus, lesion extent can
that might be confounding factors? Were limit structure-function mapping in single
control tasks administered to rule out alter- case studies, but can at least be addressed
native explanations for an observed deficit? and maybe turned to an advantage in group
Careful attention should also be paid to the studies.
LESION STUDIES IN AFFECTIVE NEUROSCIENCE 159
or other (eye movements, autonomic mea- ties, such as language comprehension or vig-
sures, and so on). It is important to ask ilance, are not the explanation for deficits on
whether these behavioral measures will be the experimental tasks of interest. It is bet-
influenced by nonspecific or correlated dis- ter still to develop specific control tasks that
abilities in the patient group. For exam- closely mimic the demands of the experi-
ple, insula damage is often accompanied mental tasks without requiring the partic-
by damage to adjacent motor pathways, so ular process of interest. An example might
patients may have slower reaction times be judging age as a control task for judging
because of simple motor dysfunction. Auto- trustworthiness from faces, using the same
nomic measures have been used to interest- stimulus set for both tasks.
ing effect in lesion studies in affective neuro-
science (see, e.g., Bechara, Damasio, Tranel,
& Damasio, 1997). Some patients may be Lesion-Symptom Mapping
more likely to take medications (for hyper-
tension, for example) that might affect such With the brain lesions characterized and
measures, requiring extra care in reference the behavior of interest precisely measured,
group selection and interpretation of the we are in position to test brain structure-
results. function relationships. There are three com-
In general, ideal behavioral measures mon approaches that are considered in
will have good psychometric properties: turn.
no ceiling or floor performance, good test-
retest reliability, and with minimal influ-
Behavior-Driven Approaches
ence of demographic or educational factors
(Laws, 2005). Measurement variation (i.e., A major challenge in cognitive neuroscience
the extent to which task performance will is to define the architecture of behavior; that
vary if the same subject is tested repeat- is, to parse complex behavior into analyz-
edly) is a source of noise that, in principle, able constituents, whether conceptualized
is under the experimenter’s control. It may as modules, processes, or interacting net-
have important influences on the analysis works (Dunn & Kirsner, 2003). The chal-
and should be minimized to the extent pos- lenge is to identify the appropriate con-
sible (Bates, Appelbaum, Salcedo, Saygin, stituent parts and then to understand how
& Pizzamiglio, 2003). When developing they interact from both a psychological and
novel tasks, consider that patients tend to a neural point of view. This enterprise obvi-
be older and less educated, on average, ously requires data gathered with a variety
than the samples of undergraduate students of methods. Studies in patients with brain
often conveniently available for piloting. injury can provide important insights into
Pilot work in healthy older subjects is usually the biologically relevant lines of cleavage for
an excellent investment during task devel- a given (complex) behavior by helping iden-
opment, although such subjects may be at tify associations and dissociations between
least as hard to find as subjects with brain putative component processes.
injury. When behavior is treated as the indepen-
dent variable, patients are selected based on
the presence of some behavioral manifes-
Control Tasks
tation – either a clinical syndrome or per-
Brain injury rarely affects only a single pro- formance on a particular task. Additional
cess, so control tasks are important in sup- behavioral measures aiming to isolate puta-
porting almost any claim. At the least, tive component processes are then adminis-
patients should generally undergo a thor- tered to determine whether these processes
ough neuropsychological screening assess- are, in fact, distinct (i.e., dissociable). A
ment to reassure researcher and readers alike single dissociation refers to a situation in
that difficulties with obviously crucial abili- which subjects are impaired on a task that
LESION STUDIES IN AFFECTIVE NEUROSCIENCE 161
ROI-damaged group differs from the brain- Converging evidence argues that
damaged control group, which provides structure-function relations are consider-
much stronger support for the existence ably more discrete than is captured by
of a specific structure-function relationship examining hemispheric, or even lobar,
(Nieuwenhuis, Forstmann, & Wagenmak- effects, but there are practical limits to the
ers, 2011). regional specificity that can be attained with
It is important to consider what an ROI ROI designs. If the study is restricted to
design does not do, because it imposes an patients with damage to some very specific
anatomical boundary that may or may not be and small brain area, an adequate sample
optimal. Obviously, nothing will be learned is unlikely to be recruited in a reasonable
about the potential contributions of brain time. An alternative is to enroll patients
regions outside that boundary. Perhaps less with variable damage to a relatively broad
obviously, there can be a risk of not detect- region – even one hemisphere or the whole
ing effects that are, in fact, related to dam- brain – and then undertake analyses to
age within the boundary. This can happen establish which subregion contributes to
if the defined ROI is much larger than the the observed deficits in function.
actually critical brain area; effects due to There are three main approaches to ana-
damage in the smaller area are diluted by lyzing data from patients who have variable
normal performance in those with damage damage in a large brain region. The one with
affecting the larger, but not critical, area. the longest history involves a secondary anal-
The group as a whole will have variable per- ysis in a standard ROI study. Having estab-
formance, and the statistical analyses may lished that some anatomically defined group
fail to detect effects. Even effects that are is impaired, and observing the usual variabil-
detected with a given ROI may neverthe- ity in that impairment, one may ask whether
less have been better captured by a different there is an anatomical basis to that variabil-
anatomical boundary. ity; that is, whether damage to a specific
subregion is a main determinant of task per-
formance. This question can be addressed
Voxel-Based Methods
qualitatively by examining the pattern of
Regions of interest can, in principle, be any lesions in the impaired and unimpaired sub-
size. In practice, there is a lower limit of groups, essentially carrying out a behavior-
resolution imposed by the volume of brain driven analysis nested in the original ROI
tissue that is injured in individual subjects, study. Lesion overlap methods are often
the extent to which those volumes over- used to this end: The lesion overlap image
lap in a given sample, or the resolution of for impaired and unimpaired subgroups can
the imaging methods that are used to char- be examined visually, or lesion extent can be
acterize the injury. Lesion volume, rather subtracted across these groups in an effort
than imaging resolution, is typically the lim- to identify the potentially critical subregion.
iting factor in the MRI era. The upper Alternative but analogous methods include
limit of resolution is determined by con- tabulating the presence or absence of injury
ceptual issues; determining that some func- to Brodmann areas and comparing the out-
tion is related to the integrity of the whole come in patients with and without behav-
brain, for example, is likely to be of lim- ioral impairment.
ited interest. That said, many core concepts There are drawbacks to pursuing sub-
in neuropsychology began with regions of group analyses in ROI studies. First, it is
interest encompassing entire cerebral hemi- important to realize that these analyses are
spheres, and defining such broad structure- usually undertaken post hoc, and any finding
function relations may still be impor- needs confirmation in a new experiment that
tant as cognitive neuroscience tackles new is designed to test the specific ROI a priori.
areas of study, such as in social or affective Selection bias and confounding factors can
domains. easily influence the results. Such analyses
LESION STUDIES IN AFFECTIVE NEUROSCIENCE 163
usually involve very small sample sizes, and These advantages come with tradeoffs. As
it can be impossible to properly account with fMRI analysis, this massively univari-
for other contributors to observed effects, ate analysis requires very conservative cor-
such as demographic variables. Results from rection for multiple comparisons, which in
such analyses should be treated with partic- turn demands a substantial sample size. The
ular caution when the a priori, ROI-based number of subjects is not the only considera-
analysis did not establish significant differ- tion; lesion overlap and distribution are also
ences between groups. A “multiple ROI” important determinants of a study’s power.
approach can be applied a priori, of course. Methods exist to estimate the anatomical
Several studies have taken this tack (Picton extent of adequate power in a given sam-
et al., 2007; Stuss et al., 2003). The main ple, and their use is important for interpret-
difficulty, in addition to the perennial ing VLSM analyses (Kimberg et al., 2007;
limitation of sample size, is determining Rudrauf et al., 2008).
how to appropriately correct for multiple
comparisons.
Recently, statistical methods that were White Matter Damage and
developed for fMRI have been adapted for Disconnection Effects
examining structure-dysfunction relation-
ships at a voxel-by-voxel level. This is a nat- With the exception of certain neurosurgical
ural extension of multi-ROI designs, with resections, lesions are rarely confined to a
the advantage of principled control of mul- single structure and often disrupt the white
tiple comparisons. Once lesions volumes are matter leading into or away from a given
registered to a common template, univari- gray matter region, or fibers of passage (i.e.,
ate statistics can be applied to test whether adjacent tracts that may have nothing to
the performance of patients with damage to do with the damaged gray matter beyond
a given voxel differs from performance of physical proximity). This damage can pose
patients with damage that spares that voxel. challenges in interpreting lesion studies.
This results in a statistical map showing the Observed behavioral effects might be due to
strength of association between damage and the white matter damage, which would be
dysfunction in anatomical space. particularly misleading if it involves fibers
This approach, commonly referred to of passage. Modern neuroimaging can assess
as voxel-based lesion-symptom mapping the extent of white matter injury, either
(VLSM), does not require imposing poten- with standard structural scans, or by using
tially arbitrary ROI boundaries and allows tract-specific imaging such as diffusion ten-
task performance to be considered either as sor imaging. Further, white matter atlases
a dichotomous (intact/impaired) or contin- are becoming increasingly sophisticated.
uous variable. The use of continuous behav- Thus, methods exist to address possible
ioral measures avoids having to impose a sec- white matter contributions and are begin-
ond potentially arbitrary boundary on the ning to be applied to structure-function
data. VLSM also has the potential to map mapping (Catani, Jones, & ffytche, 2005;
networks (i.e., to identify several regions Karnath, Rorden, & Ticini, 2009; Philippi,
that may contribute to task performance Mehta, Grabowski, Adolphs, & Rudrauf,
within a single experiment). Several vari- 2009; Rudrauf, Mehta, & Grabowski, 2008;
ations of this method, using different sta- Thiebaut de Schotten et al., 2008; Urbanski
tistical approaches, have been developed et al., 2008).
(Bates et al., 2003; Chen, Hillis, Pawlak, Developments in image analysis to study
& Herskovits, 2008; Kinkingnehun et al., network properties of the brain, whether
2007; Rorden et al., 2009; Rorden & Kar- captured by structural or functional mea-
nath, 2004; Rorden, Karnath, & Bonilha, sures, may prove useful as adjuncts to the
2007; Solomon, Raymont, Braun, Butman, lesion approaches discussed so far (Dosen-
& Grafman, 2007). bach, Fair, Cohen, Schlaggar, & Petersen,
164 LESLEY K. FELLOWS
2008; He, Dagher, et al., 2009; He, Wang, quate? Null findings (especially null find-
et al., 2009). At the least, these techniques ings meant to reassure the reader about
draw attention to network-oriented con- potential confounders) should be consid-
ceptual frameworks and are likely to be ered critically whenever sample size is lim-
important in providing a more complete ited. With these basic questions in mind,
description of the brain basis of complex the alert reader should be well prepared to
behaviors. integrate lesion findings with other forms
of evidence and is encouraged to do so
to move the field of affective neuroscience
Clinical Conditions with Diffuse forward with all the rigor that can be
Damage mustered.
In summary, group lesion studies provide r How can advances in neuroimaging best
stronger support for the generalizability of be combined with behavioral measures to
a given brain-structure function claim than understand how focal lesions disrupt the
do single cases. The critical reader of such functions of brain networks?
a study should first consider the appro- r What are the mechanisms underlying
priateness of the behavioral measures and improvement after an acute focal brain
whether the reference group is suitable for injury, and how should these changes be
testing the stated hypothesis. Are alterna- accommodated in the inferences about
tive explanations for group differences, such causal structure-function relationships
as demographic, clinical, or task-related fac- that are drawn from lesion studies?
tors, ruled out? How focal is the brain r What is the best research platform to
injury, and how consistent is the damage ensure that nonclinicians have access to
(in terms of extent, etiology, and chronic- suitable, well-characterized patients so
ity) across subjects? What selection biases that lesion studies can continue to in-
might have been introduced by the patient form cognitive and affective neuroscience
recruitment method? Is the sample size ade- theories?
LESION STUDIES IN AFFECTIVE NEUROSCIENCE 165
Gross, C. G. (1995). Aristotle on the brain. Neu- Quantification of brain lesions using interac-
roscientist, 1, 245. tive automated software. Behavioral Research
He, Y., Dagher, A., Chen, Z., Charil, A., Zij- Methods, Instruments, & Computers, 34(1), 6–
denbos, A., Worsley, K., & Evans, A. (2009). 18.
Impaired small-world efficiency in structural Marsh, E. B., & Hillis, A. E. (2008). Dissociation
cortical networks in multiple sclerosis asso- between egocentric and allocentric visuospa-
ciated with white matter lesion load. Brain, tial and tactile neglect in acute stroke. Cortex,
132(Pt. 12), 3366–79. 44(9), 1215–20.
He, Y., Wang, J., Wang, L., Chen, Z. J., Yan, Nachev, P., Coulthard, E., Jager, H. R., Kennard,
C., Yang, H., . . . Evans, A.C. (2009). Uncov- C., & Husain, M. (2008). Enantiomorphic nor-
ering intrinsic modular organization of spon- malization of focally lesioned brains. Neuroim-
taneous brain activity in humans. PLoS One, age, 39(3), 1215–26.
4(4), e5226. Newhart, M., Ken, L., Kleinman, J. T., Heidler-
Heberlein, A. S., & Adolphs, R. (2004). Impai- Gary, J., & Hillis, A. E. (2007). Neural net-
red spontaneous anthropomorphizing despite works essential for naming and word compre-
intact perception and social knowledge. Pro- hension. Cognitive and Behavioral Neurology,
ceedings of the National Academy of Sciences, 20(1), 25–30.
101(19), 7487–91. Nieuwenhuis, S., Forstmann, B. U., & Wagen-
Heberlein, A. S., Padon, A. A., Gillihan, S. J., makers, E. J. (2011). Erroneous analyses of
Farah, M. J., & Fellows, L. K. (2008). Ventro- interactions in neuroscience: a problem of sig-
medial frontal lobe plays a critical role in facial nificance. Nature Neuroscience, 14(9), 1105–7.
emotion recognition. Journal of Cognitive Neu- Ochfeld, E., Newhart, M., Molitoris, J.,
roscience, 20(4), 721–33. Leigh, R., Cloutman, L., Davis, C., . . . Hillis,
Hornak, J., Bramham, J., Rolls, E. T., Morris, R. A. E. (2011). Ischemia in Broca area is associ-
G., O’Doherty, J., Bullock, P. R., & Polkey, ated with Broca aphasia more reliably in acute
C.E. (2003). Changes in emotion after circum- than in chronic stroke. Stroke, 41(2), 325–30.
scribed surgical lesions of the orbitofrontal and Philippi, C. L., Mehta, S., Grabowski, T.,
cingulate cortices. Brain, 126(Pt. 7), 1691–1712. Adolphs, R., & Rudrauf, D. (2009). Damage
Johnsen, E. L., Tranel, D., Lutgendorf, S., & to association fiber tracts impairs recognition
Adolphs, R. (2009). A neuroanatomical disso- of the facial expression of emotion. Journal of
ciation for emotion induced by music. Interna- Neuroscience, 29(48), 15089–99.
tional Journal of Psychophysiology, 72(1), 24–33. Picton, T. W., Stuss, D. T., Alexander, M. P.,
Karnath, H. O., Rorden, C., & Ticini, L. F. (2009). Shallice, T., Binns, M. A., & Gillingham,
Damage to white matter fiber tracts in acute S. (2007). Effects of focal frontal lesions on
spatial neglect. Cerebral Cortex, 19(10), 2331– response inhibition. Cerebral Cortex, 17(4),
37. 826–38.
Kimberg, D. Y., Coslett, H. B., & Schwartz, M. F. Robertson, L. C., Knight, R. T., Rafal, R., & Shi-
(2007). Power in Voxel-based lesion-symptom mamura, A. P. (1993). Cognitive neuropsy-
mapping. Journal of Cognitive Neuroscience, chology is more than single-case studies. Jour-
19(7), 1067–80. nal of Experimental Psychology, Learning, Mem-
Kinkingnehun, S., Volle, E., Pelegrini-Issac, M., ory, and Cognition, 19(3), 710–17; discussion
Golmard, J. L., Lehericy, S., du Boisgue- 718–34.
heneuc, F., . . . Dubois, B. (2007). A novel Robinson, O. J., & Sahakian, B. J. (2009). A dou-
approach to clinical-radiological correlations: ble dissociation in the roles of serotonin and
Anatomo-Clinical Overlapping Maps (Ana- mood in healthy subjects. Biological Psychia-
COM): method and validation. Neuroimage, try, 65(1), 89–92.
37(4), 1237–49. Rorden, C., & Brett, M. (2000). Stereotaxic dis-
Laws, K. R. (2005). “Illusions of normality”: A play of brain lesions. Behavioral Neurology, 12,
methodological critique of category-specific 191–200.
naming. Cortex, 41(6), 842–51; discussion 852– Rorden, C., Fridriksson, J., & Karnath, H. O.
43. (2009). An evaluation of traditional and novel
Macmillan, M. (2000). An odd kind of fame. Cam- tools for lesion behavior mapping. Neuroim-
bridge, MA: MIT Press. age, 44(4), 1355–62.
Makale, M., Solomon, J., Patronas, N. J., Danek, Rorden, C., & Karnath, H. O. (2004). Using
A., Butman, J. A., & Grafman, J. (2002). human brain lesions to infer function: A relic
LESION STUDIES IN AFFECTIVE NEUROSCIENCE 167
from a past era in the fMRI age? Nature software for the analysis of brain lesions
Reviews Neuroscience, 5(10), 813–19. (ABLe). Computer Methods and Programs in
Rorden, C., Karnath, H. O., & Bonilha, L. (2007). Biomedicine, 86(3), 245–54.
Improving lesion-symptom mapping. Journal Stuss, D. T., Murphy, K. J., Binns, M. A., &
of Cognitive Neuroscience, 19(7), 1081–88. Alexander, M. P. (2003). Staying on the job:
Rudrauf, D., Mehta, S., Bruss, J., Tranel, D., The frontal lobes control individual perfor-
Damasio, H., & Grabowski, T. J. (2008). mance variability. Brain, 126(Pt. 11), 2363–80.
Thresholding lesion overlap difference maps: Teuber, H. L. (1955). Physiological psychology.
Application to category-related naming and Annual Review of Psychology, 6, 267–96.
recognition deficits. Neuroimage, 41(3), 970– Thiebaut de Schotten, M., Kinkingnehun, S.,
84. Delmaire, C., Lehericy, S., Duffau, H.,
Rudrauf, D., Mehta, S., & Grabowski, T. J. Thivard, L., et al. (2008). Visualization of
(2008). Disconnection’s renaissance takes disconnection syndromes in humans. Cortex,
shape: Formal incorporation in group-level 44(8), 1097–1103.
lesion studies. Cortex, 44(8), 1084–96. Urbanski, M., Thiebaut de Schotten, M.,
Shallice, T. (1988). From neuropsychology to men- Rodrigo, S., Catani, M., Oppenheim, C.,
tal structure. New York: Cambridge University Touze, E., et al. (2008). Brain networks of spa-
Press. tial awareness: Evidence from diffusion tensor
Solomon, J., Raymont, V., Braun, A., Butman, imaging tractography. Journal of Neurology,
J. A., & Grafman, J. (2007). User-friendly Neurosurgery, & Psychiatry, 79(5), 598–601.
Section III
EMOTION PERCEPTION
AND ELICITATION
CHAPTER 7
Nathalie George
With its fine musculature and its (mostly) cates impeding danger. These communica-
hairless nature, the human face has evolved tion functions seem to be at their acme
as a major signaling and communication with the facial expressions of emotions in
channel for emotions, through facial expres- humans.
sions. This evolution took place in parallel In this chapter, we review how the facial
with the acquisition of bipedalism, which expressions of emotions are produced and
placed the face in a fully exposed-to-the- which physical features of the face convey
view position and favored the development emotional expressions. We then turn to the
of vision as a prominent sensory modal- neural substrates of the perception of emo-
ity to gather information on the environ- tion from faces, addressing the brain regions
ment – including conspecifics. The facial involved and the temporal dynamics of their
expressions of emotions form an impor- responses. Finally, we discuss other aspects
tant part of nonverbal communication. Sig- of faces that are related to emotion percep-
naling and communication are among the tion and elicitation.
basic functions of emotion expressiveness
in general: Emotional reactions – as adap-
tive reactions to eliciting events (see Chap- The Production of Emotional Facial
ter 1), which may include facial expressions Expressions
as well as body movements (see Chapter
The Movements of Facial Muscles
8), and vocal demonstrations (see Chap-
Produce the Facial Expressions of
ter 11) – inform others about one’s needs
Emotions
or intention of action, such as the intents
of fleeing, fighting, or friendly approach- The scientific approach to the facial expres-
ing. Furthermore, the emotional reactions sions of emotion goes back to the 19th
of others give indications about surround- century with the research about the mus-
ing events; for instance, fear expression indi- cular command of the facial expressions
171
172 NATHALIE GEORGE
(a)
(b)
(c)
Figure 7.1. The facial expression of emotions. (a) Example faces displaying a neutral expression and
the six basic emotions (from left to right: neutral, happy, angry, fearful, disgusted, surprised, and
sad). These faces are taken from the NimStim face stimulus set, available freely for research purposes
only at http://www.macbrain.org/resources.htm. (b) Low-pass filtered faces with happy (on the left)
and angry (on the right) expression. The low spatial frequencies convey important information for
emotion categorization. (c) Distinct face parts as well as different frequency bands convey diagnostic
information for different emotions. These faces represent the combination of the diagnostic
information in different frequency bands used by subjects to categorize the face as – from left to
right – happy, surprised, angry, disgusted, sad, fearful, or neutral; adapted from Smith and Schyns
(2009).
includes wide eye opening, which may favor independent from that on the perception of
the acquisition of visual information neces- facial expressions, because such emotions do
sary for finding a way out of danger. In con- not correspond to well-identified patterns of
trast, a disgusted expression would corre- facial movements.
spond to a reduction of sensory exposure. In addition, although bearing some uni-
These prototypical patterns of emotional versality, the facial expressions of (even
facial expressions seem to appear across basic) emotions show both interindividual
early child development, probably in paral- variability and context dependency, and
lel with the differentiation and enrichment their display is contingent on cultural codes.
of infant emotional life, as well as with the Thus, there seems to be better recognition
maturation of neuromuscular systems. of emotions expressed by members of a
Of course, basic emotions form only a pri- common social – geographical, ethnical, or
mary set of the emotions that the human national – group (e.g., Jack, Blais, Scheepers,
face can express. Facial expressions can con- Schyns, & Caldara, 2009). The interpretation
vey a large (possibly infinite?) number of of facial expressions of emotions has also
subtler and/or mixed emotions whose mean- been shown to be dependent on the situa-
ing is dependent on culture (Russell, 1991). tional and perceptual context in which they
Other emotions, including guilt, shame, arise (for a recent review, see Aviezer, Has-
envy, gratefulness, pride, vanity, and so sin, Bentin, & Trope, 2008). Overall, these
forth, may involve more complex social cog- influences underline the social dimension
nition processes as well as moral cognition of emotional expressions as communication
processes, as presented in the section on signals. However, although long debated,
social emotions. The study of moral emo- the idea of universals in the facial expres-
tions in particular, as detailed in Chapter 21, sions of emotions and of a correspond-
constitutes a distinct area of research, largely ing set of basic emotions has proven very
174 NATHALIE GEORGE
influential in the field of research on the neu- sus the nondelivery of an expected pun-
ropsychological mechanisms of emotional ishment; fear and enjoyment would be
facial expression perception. on the two sides of a second, orthogonal
axis opposing the delivery of the expected
punishment versus that of the expected
Models of Emotional Facial
reward. The dimension of approach ver-
Expressions
sus withdrawal is another important dimen-
Two main types of models have been pro- sion in motivated behavior along which the
posed to classify the facial expressions of facial displays of emotions can be classi-
emotions. These models are detailed in fied. Under this latter framework, anger and
Chapter 1 and are here reviewed only briefly. happiness are considered as negatively and
The categorical models derive directly from positively valenced approach-related emo-
the theory of universals in emotional expres- tions, respectively, whereas fear and dis-
sions. Indeed, these models concern the gust are both negative withdrawal-related
basic emotions that have been shown to be emotions.
associated with specific and highly recog- Mixed models have also been proposed:
nizable facial displays. They propose that They postulate discrete boundaries between
the facial displays of basic emotions are a few basic or primary emotions together
perceived and recognized as discrete cate- with a continuous, orthogonal dimension of
gories. This proposal has received empiri- intensity or arousal level along which these
cal support following the development of emotions can vary, giving rise to the vari-
morphing methods1 : The use of faces mor- ety of human emotional feelings and their
phed across emotions has indicated that the displays. Overall, the multiplicity of these
six basic emotions are perceived and recog- models reveals the wealth of human emo-
nized in a categorical manner (Young et al., tions and of their facial expressions.
1997).
There are, however, opposing models
that argue for low-dimensional accounts of Which Visual Information from the
the perception of facial emotional expres- Face Conveys Emotions?
sions. These models put into question
Emotional Expressions Are Encoded
the existence of prototypical patterns of
Separately from Other Facial
emotional facial expression. They argue
Information such as Identity
that emotions vary continuously along a
few underlying dimensions and that facial Which physical features of the face con-
expressions vary accordingly and can be vey emotions? One piece of knowledge
systematically located or differentiated in in this domain is that the facial infor-
the low-dimensional space engendered (for mation relative to emotional expression is
a review, see Young et al., 1997). Two processed largely independently from the
main dimensions of emotions are usually facial information relative to the person’s
considered: one that is related to valence, identity. This is illustrated by the facts that
with pleasant and unpleasant emotions at you may rather easily recognize someone’s
both ends, and the other one that is identity under different emotions and recip-
related to intensity or arousal with low rocally have no problem recognizing simi-
and high values at both ends. Rolls (2005) lar emotions on the faces of different indi-
also proposed a model focused on the viduals. However, it is interesting to go
delivery versus nondelivery of expected beyond this trivial evidence to examine what
reward and punishment. According to this experimental psychology has learned in this
model, anger and relief would be located domain.
on the two sides of an axis opposing the It was initially thought that the facial
nondelivery of an expected reward ver- signs of emotion and identity might be
THE FACIAL EXPRESSION OF EMOTIONS 175
related because both identity and emo- was recently put in question (Calder &
tion recognition are markedly affected by Young, 2005; Pourtois, Spinelli, Seeck, &
face inversion. However, this phenomenon Vuilleumier, 2010a; Tsuchiya, Kawasaki,
instead reflects that inversion disrupts the Oya, Howard, & Adolphs, 2008). Calder and
processing of configural information (con- Young proposed a principal-component-
stituted by the fine spatial arrangement of analysis framework in which a single mul-
features) from the faces and that an impor- tidimensional coding system can analyze
tant amount of information about both both identity and expression, thus chal-
individual identity and emotional expres- lenging the view of the independent per-
sion is contained in this information. Fur- ceptual analysis of these facial attributes.
thermore, inversion also disrupts the ori- This model accounts for the interference
entation of facial features, which is crucial between emotion and identity recognition
for certain emotions (see, for example, the that has been reported in some studies. It
raised-up corners of a happy mouth). How- also fits with the finding that single neu-
ever, the information essential for emotion rons of the macaque temporal cortex can
and that for identity recognition are dif- code for different facial attributes (such as
ferent and processed independently from global configural information, facial iden-
each other. Evidence for this dissociation tity, and emotional expression) in different
comes in particular from neuropsycholog- time intervals of their response (for a review,
ical studies: some brain-lesioned patients see Calder & Young, 2005); this finding was
show selective deficits in the perception recently extended to humans through the
and recognition of emotional expressions, use of intracerebral recordings2 (Pourtois et
with spared processing of other aspects of al., 2010a; Tsuchiya et al., 2008). Importantly,
the face (e.g., Young, Newcombe, de Haan, however, although Calder and Young (2005)
Small, & Hay, 1993); other patients show a argue that there may be a common percep-
reverse clinical pattern with impaired iden- tual representation stage for both facial iden-
tity recognition but preserved emotion per- tity and expression, they emphasize that this
ception from faces (e.g., Tranel, Damasio, common representation would entail differ-
& Damasio, 1988). Another stream of evi- ent facial information coding for facial iden-
dence comes from cell recordings that have tity and expression. In other words, their
revealed the existence of “face neurons” in model emphasizes that the cues about emo-
different temporal areas of the macaque tional expressions and identity seem to be
brain. Face neurons respond selectively to conveyed by different visual information.
faces in comparison with a multitude of Furthermore, there would be different opti-
other objects whether biologically rele- mal facial cues as well as distinct neural
vant or not or meaningful or meaningless. systems for perceiving different emotional
Some of these neurons respond selectively expressions.
to only one aspect of faces, such as facial Neuroanatomical versions of Bruce and
expression, identity, or gaze direction (for a Young’s model have been proposed. Gob-
review, see Rolls, 2007). bini and Haxby (2007) postulate a core
system for the perceptual analysis of faces,
comprising the inferior occipital gyrus, the
Functional Models of Face Processing
lateral fusiform gyrus, and the superior tem-
In line with these findings, the most influ- poral sulcus (STS; Figure 7.2). We come
ential model of face perception and recog- back later to the involvement of these
nition has proposed that the processing of regions in emotional processing. What is
facial emotional expressions and that of interesting to mention here is that this
other attributes of faces, including iden- model postulates distinct neural routes
tity, rely on distinct functional routes (Bruce for the perceptual analysis of invariant
& Young, 1986). Interestingly, this model attributes of faces – such as identity – and
176 NATHALIE GEORGE
that the brain regions responsive to the emo- rely on the region of the face bridging the
tional facial expressions are selectively sen- lower part of the nose and the mouth; the
sitive to the low spatial frequencies of the diagnostic region for both happiness and sur-
face (e.g., Vuilleumier, Armony, Driver, & prise is the mouth but with a distinct phys-
Dolan, 2003) and that early brain responses ical structure of that feature for these two
to faces are enhanced for fearful relative to expressions (open mouth for surprise, smile
neutral faces containing low spatial frequen- for happiness). In addition, faces seem to
cies only (Vlamings, Goffaux, & Kemner, be categorized as neutral on the basis of
2009). distributed information over the eyes, nose,
Moreover, different emotions and differ- and mouth, with a bias toward the right eye
ent tasks may be supported by different (Figure 7.1c). In agreement with this behav-
spatial frequency bands (Smith & Schyns, ioral evidence, Schyns’ group has also shown
2009). For example, the explicit identifica- with EEG (see Chapter 4) that the temporal
tion of happy emotional expressions seems dynamics of early brain responses correlate
to rely on broad-band information from the with the dynamics of face feature integra-
face encompassing both high and low spatial tion during emotional face perception.
frequencies (comprised between 7.5 and 120 In conclusion, these data support the idea
cycles per face). By contrast, fear appears of distinct categories of emotions relying on
to be conveyed by high spatial frequen- discrete information from the face, which
cies (more than 30 cycles per face), and is distinct from the information conveying
lower bands may convey diagnostic infor- the person’s identity. Whereas the discrim-
mation for anger and surprise. The ori- ination of emotional expressions may rely
gin and the functional role of such dif- preferentially on low spatial frequencies, the
ferences in the optimal frequency bands diagnostic features of the facial expressions
for recognizing basic emotions are unclear. of different emotions seem to lie in different
Because the range of spatial frequencies per- frequency bands and to involve distinct face
ceived gets confined to lower frequencies parts.
with increasing viewing distance, spatial fre-
quency may in particular influence the range
The Neural Network of the Perception
of distances over which the facial expres-
of Emotional Facial Expressions
sion of a given emotion constitutes an effec-
tive communication signal (Smith & Schyns,
Faces, and particularly emotional faces, are
2009).
among the most frequently used stimuli in
the field of social and affective neuroscience.
Distinct Face Parts Are Important
This has enabled substantial insight to be
for Different Emotions
gained into the neural system involved in
In addition to spatial frequencies conveying the perception of emotional facial expres-
different visual information, distinct parts sions. The research in this domain has
of the face may be important for different been markedly influenced by two streams
emotional expressions. This issue was inves- of data. First the finding of universals in
tigated by Schyns’ group using spatial filter- facial expressions and the definition of basic
ing techniques combined with the “bubbles” emotions have led to a focus on the neu-
method3 (e.g., Smith & Schyns, 2009). These ral substrates of the perception of these
authors demonstrated that fear lies selec- basic emotions, mostly adopting a categori-
tively in the eyes. Accordingly, differences cal approach. Second, the long tradition of
in gaze fixation onto the eyes of seen faces animal studies that have concentrated on
have been shown to account for some gen- fear-related emotional reactions has led to
der differences in fear recognition from faces a focus on the perception of fearful faces
(Halla, Huttona, & Morgana, 2010). By con- in humans. It is mostly for this emotion,
trast, the identification of disgust seems to as well as for disgust, that specific neural
178 NATHALIE GEORGE
substrates have been described and dis- These early findings about amygdala
cussed. This emphasis has led to a major involvement in emotional face perception
focus on the amygdala (in relation to fear) were consistent with the knowledge accu-
and secondarily on the anterior insula and mulated on amygdala function from ani-
basal ganglia (in relation to disgust). We mal studies, particularly its well-established
thus first review the evidence regarding role in the processing of emotional – espe-
the involvement of these regions in emo- cially fear-related – and social stimuli (for a
tional face perception. The neural network recent review, see Adolphs, 2010). However,
involved in the perception of emotion facial the focus on fearful faces raised the ques-
expressions is, however, extended (for a tion of the specificity of amygdala involve-
recent review, see Fusar-Poli et al., 2009), ment in fear perception, as discussed in
and the other regions involved are then detail in Chapter 1. In this regard, several
described. points of the seminal and later studies
may be emphasized. First, although the
deficit of patient SM seemed to be par-
The Amygdala
ticularly marked for fearful faces, she was
The amygdala is a structure of the medial also impaired in decoding mixed emotions
temporal lobe situated just in front of the from possibly less prototypical yet custom-
hippocampus, within the so-called limbic ary facial expressions (Adolphs et al., 1994).
system (Figure 7.2). It has attracted a great Other patients with partial or total amyg-
deal of attention in the field of emotion dalectomy showed some global impairment
study in general and of emotional face per- in the recognition of emotional facial expres-
ception in particular. A now classic study sions or at least deficits in the recognition
in this domain examined a patient with a of negative emotions such as anger, disgust,
selective, bilateral lesion of the amygdala and sadness, as well as impaired perception
(Adolphs, Damasio, & Damasio, 1994). This of gaze direction. Moreover, brain imag-
patient, SM, was strikingly impaired in her ing studies have found involvement of the
ability to recognize the facial expression of amygdala not only during fearful face per-
fear. However, she had not lost the concept ception but also during the perception of
of fear as such (although she did not appear angry, disgusted, sad, surprised, and happy
to experience fear in a normal way); neither faces, as compared to neutral faces (e.g.,
did she have some visuoperceptual deficit Breiter et al., 1996; Derntl et al., 2009). Some
that could have accounted for her impair- authors even found no difference between
ment. In particular, she was able to describe the levels of amygdala activation for differ-
situations likely to elicit fear in people ent emotional expressions including positive
and how this feeling of fear would mani- ones (e.g., Winston, O’Doherty, & Dolan,
fest in terms of bodily reaction and behavior. 2003). Thus, the role of the amygdala is not
She was also unimpaired in recognizing the limited to the perception of fearful expres-
persons she knew from their faces as well sion or of negative emotion expressions.
as in learning new faces and discriminating In addition, the role of the amygdala
among unfamiliar faces. Thus, she seemed extends to the perception of emotion from
selectively impaired in fearful face recogni- other cues than faces, as discussed in detail
tion. Other patients with an amygdala lesion in the other chapters in this book section.
and a similar pattern of deficits have been Indeed, although inconsistently observed,
examined since then. Moreover, the devel- it seems that the loss of fear recognition
opment of functional brain imaging tech- following amygdala damage can extend to
niques has complemented these data with vocal expressions. With fMRI, the amygdala
the finding of greater amygdala activation has been shown to be involved in the per-
in response to fearful than neutral and/or ception of facial, vocal, and bodily expres-
happy faces in normal subjects (Breiter sions of fear, as well as in the cross-modal
et al., 1996; Morris et al., 1996). integration between facial and vocal signals
THE FACIAL EXPRESSION OF EMOTIONS 179
of fear. One study, however, showed a dif- the inconsistent findings on amygdala acti-
ferential impairment of the recognition of vation to different facial emotional expres-
anger-related scenes when the facial expres- sions. Interestingly, they could also con-
sions of the scene’s protagonists were vis- tribute to the important variability of the
ible as opposed to concealed, suggesting a results regarding the type and the selectiv-
special role of the amydgala in the percep- ity of the deficits associated with amygdala
tion of some emotions from facial expres- lesions, although etiology, disease history
sion (Adolphs & Tranel, 2003). This notwith- (particularly onset age), precise delineation
standing, the amygdala clearly receives of the lesion, and compensatory mechanisms
multisensory inputs, and it is involved in are bound to be essential factors in this case.
emotion perception from multiple sensory Overall, this overview of the findings on
channels. Its dense and reciprocal connec- amygdala involvement in emotional process-
tions with the visual system, which is a ing raises two important questions: First,
prevailing sensory modality in the human what exactly is the role of this key struc-
primates, may explain why it has been ture in emotional face perception? Second,
particularly implicated in the perception of to which facial cues is the amygdala sensi-
facial displays of emotions. tive?
As detailed in Chapters 14 and 15, amyg- A general view has emerged over recent
dala involvement in emotional, particularly years that allows accommodation of the var-
fear-related, perception has been found in ious results regarding amygdala involvement
conditions where the processing of facial in face and person perception under a unify-
expression was implicit as well as when ing theory. The amygdala is a structure that
the faces were unattended or not con- is at the crossroad of (1) the perceptual anal-
sciously perceived. In contrast, some stud- ysis of incoming stimuli, (2) the evaluation
ies reported that amygdala involvement of stimuli and events in relation to (innate
was dependent on attention, task relevance, and acquired) endogenous knowledge, and
and/or awareness. It is thus likely that the (3) emotional reactions. It plays a key role in
amygdala is involved in multiple emotion- the interaction between emotion and mem-
related processes, some of which are auto- ory: It is involved in the binding of events to
matic and largely unconscious and others an affective value and in the representation
that depend on attention and/or awareness and memorization of this link, as detailed
(for reviews, see Adolphs, 2010, and Chap- in the section on emotional learning and
ters 14–15). memory (for recent reviews, see Morrison
Moreover, there seems to be impor- & Salzman, 2010; Murty, Ritchey, Adcock,
tant interindividual variability in amygdala & LaBar, 2010). The amygdala is thus ideally
involvement for the perception of different placed to play a pivotal role in the appraisal
emotions. For example, amygdala respon- of stimuli and in the processing of behav-
siveness to happy faces has been reported to iorally relevant and salient stimuli: It evalu-
depend on the personality trait of extraver- ates and combines information from incom-
sion (Canli, Sivers, Whitfield, Gotlib, & ing stimuli with information related to prior
Gabrieli, 2002). Gender and anxiety are experiences as well as to current goals of
other sources of interindividual differences the perceiver, allowing adaptive responses
that may modulate amygdala reactivity to to the stimuli. As discussed in Chapter 1,
different types of emotional expressions, the amygdala may thus be viewed as a rele-
including fear, as detailed in Chapters 24 vance detector or, in other words, as a key
and 26. In addition, culture and the famil- structure for the appraisal of events that are
iarity with the social group of an individ- relevant to the organism.
ual whose fearful face is perceived can also This view of the amygdala allows us
modulate the amygdala response to that face to understand its notable involvement in
(e.g., Chiao et al., 2008). These modulatory the perception of emotional faces and its
variables could at least partly account for more general role in social cognition (for
180 NATHALIE GEORGE
a recent review, see Adolphs, 2010). Social uation to decipher their meaning and rele-
stimuli have become among the most rele- vance.
vant stimuli through human evolution, and The second important question raised
faces may form the most relevant class of earlier asked to which visual information
social stimuli. This holds true from birth, from the face is the amygdala sensitive. Two
because faces convey information about oth- types of findings appear of particular inter-
ers’ identity, emotions, and intentions, and est. First, the amygdala seems to be par-
the decoding and integration of facial signals ticularly sensitive to the low spatial fre-
guide our interpersonal behavior. Among quency content of threat-related faces (e.g.,
the information conveyed by faces, emo- Vuilleumier et al., 2003). This is consistent
tional expressions are particularly salient with the finding that low spatial frequencies
and relevant because they often signal an contribute preferentially to the categoriza-
intention of action (such as in anger or hap- tion of emotional facial expressions (Schyns
piness, which are both approach related but & Oliva, 1999). Yet, it is contradictory to
convey very different intents) and/or the the earlier described finding of Smith and
presence of a danger in the environment Schyns (2009) about the selective contribu-
(such as in fear). Furthermore, the mean- tion of high spatial frequencies to fearful
ing of emotional expressions depends both face recognition. These effects might, how-
on the sender and on the perceiver, as well as ever, depend on the task, and further fMRI
on the context of emotion elicitation. Thus, studies will be necessary to clarify these dis-
the specific role of the amygdala in the per- crepant findings.
ception of emotional faces, the variability Second, the amygdala seems to be partic-
of its involvement across subjects and stud- ularly sensitive to the information conveyed
ies, and its more general role in social cog- by the eyes. It has been shown to be partic-
nition may be understood in the light of ularly responsive to fearful eyes, even when
its general role in the appraisal of stimulus they are perceived unconsciously. This
relevance. This role may also account for may in fact explain the finding of selective
its implication in various pathologies char- amygdala responses to fearful expression,
acterized by impairments in social inter- because, as mentioned earlier, fear lies
actions, such as autism, schizophrenia, or especially in the eyes, and the widening of
social phobia, where impairments in emo- the eyes typical of fearful expression makes
tional face perception and/or in amygdala the eye region very salient. In agreement
responses have been described (for review, with this idea, it was recently shown that
see Schumann, Bauman, & Amaral, 2011). In the patient SM of Adolphs and colleagues’
addition, appraisal processes are multistage (1994) study scanned faces abnormally,
processes, including automatic and noncon- neglecting the eye region, which is the first
scious mechanisms as well as more con- and most frequently fixated region of the
trolled and awareness-dependent processes, face in normal subjects. Her impairment in
which fits with the view of the multistage recognizing the facial expression of fear was
involvement of the amygdala in emotion in fact nonexistent when she was explicitly
perception. instructed to fixate on the eyes (Adolphs
The amygdala has also been proposed to et al., 2005). In the same vein, it has been
be involved in processing stimulus ambi- reported that the level of amygdala acti-
guity, particularly for threat-related stim- vation to faces correlated with the amount
uli and in relation with decision mak- of gaze fixation onto the eye region of
ing (Adams, Gordon, Baird, Ambady, & seen faces in autistic patients, who have
Kleck, 2003; Hsu, Bhatt, Adolphs, Tranel, core deficits in social interactions (Dalton
& Camerer, 2005). Such a proposal is, how- et al., 2005). Amygdala activation was also
ever, not in contradiction with the role of found to predict fixation onto the eyes of
amygdala in relevance appraisal, because fearful faces in healthy subjects (Gamer &
ambiguous stimuli call for deepened eval- Buchel, 2009).
THE FACIAL EXPRESSION OF EMOTIONS 181
tion of the facial expression of anger (e.g., pling between these brain regions during
Calder, Keane, Lawrence, & Manes, 2004). the processing of unconsciously perceived
Future studies will thus have to determine threat-related faces (for a review, see Tami-
whether the processing of anger and disgust etto & de Gelder, 2010). This was recently
may actually be segregated within differ- complemented by effective and anatomi-
ent regions of the striatum. It is also pos- cal connectivity studies (Garrido, Barnes,
sible that disgust processing implicates dis- Sahani, & Dolan, 2012; Tamietto, Pullens,
proportionately the anterior insula, whereas de Gelder, Weiskrantz, & Goebel, 2012).
anger processing selectively recruits some This route would rely on the magnocellular
part of the ventral striatal system. In addi- pathway of the visual system that is sensi-
tion, these findings will have to be artic- tive to low spatial frequencies and has the
ulated with those implicating the ventral property of conveying coarse visual infor-
striatum in reward and positive emotions. mation but with fast response dynamics. It
would thus subtend the amygdala respon-
siveness to the low spatial frequency con-
Other Brain Regions
tent of fearful faces, and it would sub-
The network of brain regions involved in serve the rapid and automatic detection of
the perception of the facial expressions of fear-related facial expressions and the sub-
emotions is in fact quite extended (Figure sequent enhanced perceptual processing of
7.2). This is likely related to the fact that these stimuli (e.g., Vuilleumier et al., 2003).
the perception of facial expressions triggers However, the involvement of this subcor-
multiple elementary processes related to tical route in the perception of emotional
emotion processing (such as detection, faces remains debated, and an alternative
attention orienting, appraisal, emotion route, also involving the pulvinar, may play
recognition processes), as well as to person a central role (for a recent review, see Pessoa
perception (such as inference of the other’s & Adolphs, 2010).
mental state and intention, personality trait The pulvinar is densely and reciprocally
attribution, empathy). This section, which connected to other regions involved in emo-
reviews the role of the main regions involved tions, such as the anterior cingulate and the
in the perception of emotions from faces, is orbitofrontal and insular cortices; hence, it
organized into three parts. I first discuss the also reaches the amygdala indirectly. Its pro-
pulvinar, which in close interaction with the jections further include the superior tem-
amygdala may play an important role in sev- poral sulcal and gyral regions, which are
eral routes of emotion processing. Second, involved in the perception of emotional
I describe the involvement of the regions expressions, as detailed later. Finally, the
classically associated with the perceptual pulvinar has also connections with the pos-
analysis of faces. Finally, I briefly mention terior parietal regions and, together with the
other cortical regions that have general roles superior colliculus, pertains to the attention-
in emotion. orienting system of the human brain; it may
thus play a role in rapid orienting toward
THE PULVINAR emotionally relevant stimuli.
The pulvinar is a thalamic nucleus that has In sum, the pulvinar has rich and recip-
been proposed to play a special role in rocal connections with many cortical areas
the processing of fear-related expressions, subtending the multiple processes involved
particularly when they are unconsciously in the detection and recognition of the facial
perceived. This role may involve several expressions of emotion, and it may play a
pathways. First, there is a route from the general role in cortical integration, as well as
superior colliculus to the pulvinar that, in be a point of convergence of the cortical and
turn, projects to the amygdala. Evidence subcortical routes involved in emotion pro-
for this route came initially from functional cessing. Thus, it may be a key structure of
imaging studies that showed functional cou- emotional face processing, as discussed by
THE FACIAL EXPRESSION OF EMOTIONS 183
Pessoa and Adolphs (2010). The extended extrastriate visual cortex and the amygdala.
cortical-pulvinar connectivity may account Indeed, the enhancement of face-selective
for the selective disruption of fear recog- fusiform responses to fearful relative to neu-
nition from faces following damage to the tral faces has been shown to be driven by the
medial part of the pulvinar, and it may also low spatial frequency content of the faces,
explain why evidence for intact fearful face which also drives the amygdala responses to
detection was recently found in the patient those faces (Vuilleumier et al., 2003). Fur-
SM, despite the bilateral lesion of her amyg- thermore, an amygdala lesion abolishes the
dala. It is, however, unclear why such a func- increase in fusiform responses to fearful rel-
tional pathway would be restricted to fear- ative to neutral faces (Vuilleumier, Richard-
ful face perception. There is some evidence son, Armony, Driver, & Dolan, 2004).
of pulvinar involvement in the perception of Moreover, the pattern of relative atten-
other emotions expressed by faces or bodies. tion independency of the response to emo-
Future research will have to determine the tional faces seems very similar for both
precise extent of the functional involvement the fusiform gyrus and amygdala, although
of the pulvinar in emotional face processing. the activity of visual regions may be over-
all more liable to the influence of atten-
THE CORE SYSTEM INVOLVED IN THE tion and/or awareness, as discussed in Chap-
PERCEPTUAL ANALYSIS OF FACES ters 14–15. Thus, the dominant view is that
According to Gobbini and Haxby’s (2007) coarse visual information first reaches the
model of face processing, the core system amygdala via either the subcortical colliculo-
for the perceptual analysis of faces com- pulvinar pathway or a rapid visual cortical
prises two sets of regions: (1) the fusiform route and that the amygdala then exerts
and inferior occipital regions involved in the top-down influence onto the network of
processing of the invariant features of faces cortical areas involved in emotion process-
that is necessary for identity recognition and ing, including the visual regions responsi-
(2) the posterior region of the superior ble for the perceptual analysis of faces. This
temporal sulcus (STS) that subtends the process would be constitutive of a mecha-
processing of the dynamic features of faces nism of emotional attention allowing atten-
including emotional expressions (Figure 7.2). tional capture by and enhanced process-
The activity throughout the visual pathway ing of highly behaviorally relevant stimuli
has been shown to be modulated by the such as emotional faces (see Chapter 14).
emotions conveyed by faces, but the mod- The precise temporal dynamics of the inter-
ulation of fusiform and STS responses has play among amygdala, subcortical, and cor-
been scrutinized most extensively. tical responses to different facial expressions
It is well established that the activity of emotions remain, however, to be estab-
of the fusiform regions selectively respon- lished.
sive to faces is influenced by the emo- The modulation of STS activity by the
tion conveyed by the faces (for a review, facial expressions of emotions is usually
see Vuilleumier & Pourtois, 2007). The interpreted quite differently from that of
emotional enhancement of face-selective the fusiform activity. STS involvement in
fusiform responses has been most com- the processing of facial expressions was
monly obtained for fearful faces compared first shown with intracerebral recording in
to neutral faces. However, it was also macaque monkeys (for a review, see Rolls,
reported for the facial display of other emo- 2007). According to Gobbini and Haxby’s
tions. In addition, the emotional enhance- (2007) model of face recognition, the pos-
ment of stimulus-selective perceptual pro- terior STS would be primarily involved in
cesses does not seem limited to faces and the encoding of changeable facial features;
extends to bodies as well as scenes. These hence, of emotional facial expressions. This
modulations seem to be tightly linked to would be related to the more general role
the anatomo-functional coupling between of the STS in the perception of biological
184 NATHALIE GEORGE
motion (for a recent review, see Beauchamp, have evolved as an ideal substrate for the
2011). decoding of emotional facial expressions,
Surprisingly, however, the evidence for and even when facial expressions are pre-
the involvement of the STS (and adjacent sented in isolation, their multidimensional
superior temporal gyrus regions) in the per- aspect may be implied. Furthermore, the
ception of the facial expression of emo- STS may have a general role in the analy-
tion in humans is somewhat more limited sis of others’ intentions from the combina-
than the evidence regarding the emo- tion of their emotional expression, gaze, and
tional modulation of ventral occipitotem- action (e.g., Wyk, Hudac, Carter, Sobel, &
poral responses (for a review, see Calder Pelphrey, 2009).
& Young, 2005). This may be related to
the fact that most studies have focused on OTHER CORTICAL REGIONS
static stimuli. Indeed, it seems that the STS Two frontal regions have been consistently
responses to emotional faces are enhanced reported as activated during the percep-
for dynamic relative to static displays of tion of emotional faces: the orbitofrontal
facial expressions (e.g., Pitcher et al., 2011; and ventromedial prefrontal cortices
Trautmann et al., 2009). (OFC/vmPFC), which have been partic-
STS responses to facial expressions do ularly involved in the processing of angry
not seem to be specific to a particu- faces and of positively valenced emotional
lar emotion. Furthermore, these responses faces, and the anterior cingulate cortex
have been observed under conditions where (ACC), which has been shown to be
the faces were not consciously perceived, responsive to various emotions from faces
although the responses seem to be enhanced including happiness, anger, sadness, disgust,
under conditions of explicit processing of and fear (for a review, see Fusar-Poli
the emotional expression. Importantly, STS et al., 2009). Both regions appear to play
responses may not be specific to the facial a general role in emotion processing. The
signals of emotion, because they have also OFC/vmPFC is quite generally involved
been observed for vocal expressions of emo- in emotion identification from multiple
tions, as detailed in Chapter 11. In addition, sensory channels as well as in emotion expe-
the STS is known to be involved in the rience, as revealed by the classical clinical
processing of other facial attributes, such as description of Phineas Gage’s symptoms, an
gaze, and it seems to play a role in the inte- early case of an accidental ventromedial pre-
gration of gaze and facial expression cues, as frontal lesion. It would play a role in linking
well as in the integration of facial and vocal a stimulus with its affective or emotional
emotion (for a review, see Beauchamp, 2011). value and adapting this link as a function of
It is also involved in the processing of motion the changes in incoming events. It may also
from various face and body cues. be involved in the top-down control and
Altogether, these findings have led to the regulation of behavior, as well as in decision
proposal that the role of the STS in the making on the basis of the affective values
perception of the facial expression of emo- of stimuli and of the predicted outcomes
tion may be related to its sensitivity to mul- of actions or events (for a recent review,
tisensory inputs. Indeed, emotional signals see Roy, Shohamy, & Wager, 2012). The
in real-life situations are essentially multi- ACC is also implicated in the perception of
dimensional: They combine multiple chan- different types of emotional stimuli and in
nels of information such as vocal and facial the subjective experience of emotion. It is
expression, for example, or gaze and facial a region of integration of emotional, atten-
expression cues (the signification of a fear- tional, cognitive, autonomic, and visceral
ful face will certainly not be the same if information, which is involved in conflict
that face is looking at you rather than at monitoring and the regulation of moti-
something behind you; Calder & Young, vated behavior, as well as in the conscious
2005). Thus, some regions of the STS would experience of emotion. Thus, the ACC,
THE FACIAL EXPRESSION OF EMOTIONS 185
particularly in its ventral part, would play this region is unclear, it may be implicated in
a general role in emotion (for a review, see biographical knowledge, particularly affect-
Rushworth, Behrens, Rudebeck, & Walton, laden autobiographical memory, as well as in
2007). Both the OFC/vmPFC and ACC various aspects of semantic memory includ-
regions have strong anatomical connections ing conceptual knowledge of social behav-
with the amygdala and show functional ior. Activations in discrete parts of the infe-
coupling with it during the perception of rior, middle, and superior temporal gyri have
emotional faces. also been found during sad, disgusted, and
Moreover, premotor regions of the fron- happy face perception, respectively. These
tal lobe have been reported to be activated activations may reflect some aspects of per-
during the perception of emotional faces. ceptual learning and/or perceptual analysis
A discrete region of the pre- supplemen- of emotional expressions.
tary motor area (pre-SMA) showed selec-
tive response to the perception of happy CONCLUSION
faces during intracerebral recording in an A large network of cortical and subcorti-
epileptic patient. The contact of the elec- cal regions is involved in the perception of
trode that showed this selective response the facial expressions of emotions, among
elicited laughter and merriment when elec- which the amygdala seems to play a central
trically stimulated (Krolak-Salmon, Henaff, role. Although it is likely that the percep-
Vighetto, et al., 2006). This finding adds tion of different emotions engages preferen-
to the body of evidence that the percep- tially distinct parts of this network, it seems
tion of others’ emotions involves the same also clear that there are common regions
regions as those implicated in one’s own engaged by different emotions and that the
experience of emotions. This is indeed the regions involved depend on task, context,
case for the amygdala and anterior insula, and individual variables. Moreover, part of
which are involved in the experience and in this network is engaged in the perception of
the perception of fear and disgust, respec- emotions from face as well as from nonface
tively; it seems to be also the case for the stimuli.
ventral striatum and dopaminergic system,
which are involved in the feeling and in
the perception of anger. This network of The Temporal Dynamics of the Brain
regions could participate in an “emotional Responses to Emotional Faces
mirror neuron system,” which assumes that
the recognition of the emotion expressed by How quickly are the facial expressions of
a conspecific is subtended by an activation emotions encoded by the human brain, and
of the structures that mediate the feeling which stages of face processing are affected
of that emotion in ourselves. The avail- by emotion? The importance of the per-
able evidence for involvement of the right ception and recognition of others’ emo-
somatosensory cortices in the recognition of tional facial expressions for adaptive behav-
seen facial expressions is also in line with ior, particularly when they signal danger,
this view (e.g., Adolphs, Damasio, Tranel, has led to the classical hypothesis that there
Cooper, & Damasio, 2000; Winston et al., should be very rapid brain responses to emo-
2003). tional faces, at least to threat-related ones,
In addition, several neocortical regions of in the course of face processing. The best
the temporal lobe have been reported to illustration of this view is provided by an
be activated during the perception of emo- early model of the unfolding in time of
tional faces, although somewhat more anec- emotional face processing (Adolphs, 2002).
dotally (for a review, see Fusar-Poli et al., According to this model, a coarse represen-
2009). For example, the temporal pole was tation of the emotion expressed, conveyed
found to be activated during angry and sad by the low spatial frequency content of the
face perception. Although the exact role of face, would be extracted either through the
186 NATHALIE GEORGE
subcortical colliculo-pulvinar pathway to may last up to 500 ms. They are sometimes
the amygdala or through a fast feedforward preceded by a posterior N2 (a negative com-
cortical sweep of information from visual ponent peaking between about 200 and 300
areas to the OFC and amygdala. Thus, such ms) that is enhanced to the emotional faces
coarse information would be extracted by (e.g., Schupp et al., 2004).
about 100 ms and fed back to the visual These late effects, particularly the
regions responsible for the perceptual analy- P300/LPP effects, seem to be most clearly
sis of faces, resulting in enhanced processing seen when the task involves an explicit pro-
of the seen emotional face around 150 ms cessing of the emotional facial expression
poststimulus onset. Then the distributed (e.g., Krolak-Salmon et al., 2001). In agree-
cortical stage of emotional face processing ment with this idea, intracerebral recordings
would allow extracting the meaning of and in epileptic patients have shown a selec-
conceptual knowledge about the emotion tive response to disgusted faces (as com-
expressed, from about 300 ms after stimu- pared to fearful, happy, surprised, and neu-
lus onset. tral faces) in the anterior insula, which began
However, the experimental evidence for 300 ms postface onset and was dependent on
such a model is largely indirect, particularly the explicit processing of emotion. Further-
concerning its first part, which draws mainly more, selective amygdala responses to fear-
on fMRI and lesion studies of conscious and ful faces dependent on the explicit nature
nonconscious emotional face perception of the task with regard to the seen emo-
(e.g., Vuilleumier et al., 2004). Such studies tions were reported between 200 and 800 ms
lack the temporal resolution necessary for (for a review, see Krolak-Salmon, Henaff,
assessing the time course of face processing. Bertrand, Vighetto, & Mauguiere, 2006). A
However, time-resolved brain responses can late amygdala response (starting from 700
be recorded with human electrophysiolog- ms) selective to fearful relative to neutral
ical methods, including electroencephalog- faces has also been reported to be dependent
raphy (EEG) and magnetoencephalography on visuospatial attention in another recent
(MEG) techniques in healthy subjects, as intracerebral EEG study (Pourtois, Spinelli,
well as intracerebral EEG recordings in Seeck, & Vuilleumier, 2010b). Finally, emo-
epileptic patients, as detailed in Chapter 4. tional expression has been shown to mod-
These methods have uncovered some ulate the electrical responses specific to
aspects of the temporal dynamics of the faces from 200 to 1,000 ms in the fusiform
neural responses to emotional faces (for a region (Pourtois et al., 2010a). Thus the
review, see Vuilleumier & Pourtois, 2007). late effects recorded at the scalp surface
may reflect late stages of emotional face
processing in a distributed network of
Brain Responses to Emotional Facial
regions, including amygdala, insula, and
Expressions Have Often Been Found
extrastriate regions selective for faces. This
in a Late Time Range
is also supported by source localization of
A large number of event-related potential MEG and EEG responses, which has shown
(ERP) studies have reported late, sustained that these effects implicate the extended
effects of the emotion conveyed by faces network of temporal and frontal cortical
(Figure 7.3a,b). These effects are typically regions known to be involved in emo-
observed after the face stimulus onset in tional face processing (e.g., Esslen, Pascual-
the form of enhanced late positive com- Marqui, Hell, Kochi, & Lehmann, 2004).
ponents, with greater P300 or late positive Emotion influence on late brain responses
potentials (LPPs) in response to emotional to faces has been reported for different
than neutral faces (e.g., Krolak-Salmon, Fis- types of emotions, including positive (e.g.,
cher, Vighetto, & Mauguiere, 2001; Schupp happy) and negative (e.g., fearful, threaten-
et al., 2004). These late positive effects start ing, or disgusted) facial expressions. Thus,
between 250 and 400 ms poststimulus and although sometimes debated, these late
THE FACIAL EXPRESSION OF EMOTIONS 187
P1 P300 / LPP
(a) μV
5 C1
−5
N170
0 + 6 μV 0 400 800
MEG fT
(c) +22
40-50 ms −22
+ 4 μV
(d)
+ 9 μV
P7
−100 0 100 300
fearful
− 9 μV happy
neutral
Figure 7.3. The dynamics of brain responses to emotional faces. (a) The brain responses to emotional
faces as reflected in evoked potentials or magnetic fields comprise a succession of components,
including the C1, P1, N170, and P300 or LPP (following the EEG nomenclature). These components
are here illustrated on an overlay of the time course of evoked potentials recorded on 64 electrodes
distributed over the scalp (with potentials in ordinate, in μV, and time in abscissa, in ms). Note that
the C1 is best recorded with stimuli displayed peripherally (see Rauss et al., 2011, for a recent review).
Here, faces were presented foveally, thus eliciting a very small C1. Data from Morel et al. (2009). (b)
Effects of emotion have typically been found in the late time range. On the left: grand mean
topographical map of the P300 averaged between 300 and 600 ms for fearful, happy, and neutral faces,
shown on a back view of the head. On the right: time course of the evoked potential on CPz, the
centro-parietal electrode site where the P300 culminated, showing enhanced P300 for fearful relative
to neutral and happy faces between 300 and 600 ms. Data from Morel et al. (2009). (c) There is
growing evidence for early to very early influence of emotion on brain responses to faces, including in
the time range of the C1. Here, MEG data are presented. There was a differentiated repetition effect
(difference in the brain responses for the second relative to the first presentation of stimuli) for
emotional versus neutral faces, between 40 and 50 ms. A grand mean topographical map of the
repetition effect averaged between 40 and 50 ms is shown on the left and right side views of the head,
highlighting the sensors where the repetition effect peaked for the fearful (black squares), the happy
(gray circles), and the neutral faces (white squares). Data from Morel et al. (2009). (d) Modulation of
the face-specific N170 by emotional expressions. On the left: a grand mean topographical map of the
N170 in response to emotional and neutral faces is shown at the latency of its peak (160 ms), on left
and right side views of the head. On the right: time course of the evoked potential on a left temporal
electrode, P7; on the first presentation of fearful, happy, and neutral faces, the amplitude of the N170
was found to be greater for the emotional than the neutral faces. Data from Morel et al. (2009).
188 NATHALIE GEORGE
effects do not seem to reflect a processing get discrimination task (Pourtois, Grand-
stage selective for a particular emotion. It jean, Sander, & Vuilleumier, 2004). This
is, however, possible that the precise net- effect does not seem to be specific to emo-
work of regions subtending such generally tional faces, because it has also been shown
widespread effects varies as a function of the that the C1 in response to laterally presented
seen emotion, which may explain why selec- gratings can be modulated when these grat-
tive or differentiated effects have sometimes ings acquire an affective value through con-
been reported (for a review, see Vuilleu- ditioning (Stolarova, Keil, & Moratti, 2006).
mier & Pourtois, 2007). Overall, the late In addition, in a study combining EEG and
effects have been proposed to reflect post- MEG, the emotional expression of centrally
perceptual stages of processing related to the viewed faces has been shown to modulate
decoding of the meaning of emotions, global the repetition effects elicited by those faces
activation processes elicited by the appraisal between 40 and 50 ms, over posterior occip-
of the seen emotional faces, and/or corti- itotemporal regions (Morel, Ponz, Mercier,
cal integration mechanisms associated with Vuilleumier, & George, 2009; Figure 7.3c).
emotional processing. They are not specific In a follow-up study, the prior association
to the perception of emotions from faces, of neutral faces with an emotional auditory-
because they have been reported for other verbal context was found to modulate the
types of emotional stimuli such as scenes and brain responses to these faces in a simi-
words. lar time range (Morel, Beaucousin, Perrin,
& George, 2012). Altogether, these findings
provide evidence for very early differenti-
Accumulating Evidence for Early to
ated processing of emotional stimuli, which
Very Early Effects of Emotions
could reflect not only coarse categoriza-
Are the early stages of the perceptual analy- tion mechanisms of low-level visual features
sis of faces really immune to the influence of predictive of emotions (such as local con-
the emotion displayed by the face? The stud- trast variations produced by the widened
ies that addressed this question have concen- eyes of fearful faces), but also the influ-
trated on two types of emotional influences ence of acquired salience or emotional sig-
on brain responses to faces: (1) the emotional nificance on the earliest stages of visual
modulations that may occur at the earliest processing.
stages of visual processing, which take place The neural origin of these very early
within about the first 100 ms and are not effects is unclear. Although long held to
specific to faces, and (2) the emotional influ- originate mainly from the primary visual
ences on the early brain responses selective cortex, the C1 is now known to reflect neural
to faces, that is, on N170 (in EEG) and M170 activity from the striate and extrastriate cor-
(in MEG; Figure 7.3a,c,d). tices (for a review, see Rauss, Schwartz, &
An increasing number of studies have Pourtois, 2011). It might also be influenced by
shown that emotional modulation of the some neural activity from a subcortical ori-
brain responses to seen faces could occur gin. Indeed, intracerebral recordings in the
within the first 100 ms of face processing pulvinar and in mutiple visual cortical areas
(for reviews, see George, Morel, & Conty, in the cat have shown that the initial pro-
2008; Vuilleumier & Pourtois, 2007). Such cessing of information in the visual system is
modulations have been reported in the time parallel, with response latencies in the pul-
range of the C1 (or N70m in the MEG vinar comprised between 50 and 85 ms for
vocabulary), which is the earliest ERP com- simple stimuli. More surprisingly, thalamic
ponent in response to visual stimuli, peak- sources of activity have been localized with
ing between 50 and 90 ms. The C1 was MEG as early as 10 to 20 ms after the onset
found to be greater in response to fearful of the display of fearful faces (Luo, Holroyd,
than happy faces that were presented lat- Jones, Hendler, & Blair, 2007). In addition, in
erally and were incidental to a main tar- this study, selective responses to fearful faces
THE FACIAL EXPRESSION OF EMOTIONS 189
were shown to originate in the amygdala cific to faces, because they have also been
from 20 to 30 ms, earlier than the first neural observed in response to nonhumanoid emo-
sources of MEG signals in the visual cor- tional robots, emotional pictures, and grat-
tex (between 40 and 50 ms). A recent MEG ings predictive of painful shocks, as well
study has also shown activities differenti- as in association with the integration of
ating the perception of fearful, happy, and emotions from faces and bodies or from
neutral faces in the STS, medial prefrontal faces and scenes (e.g., Dubal, Foucher, Jou-
cortex, and amygdala between 35 and 96 vent, & Nadel, 2011; Righart & de Gelder,
ms (Liu & Ioannides, 2010; see also, Garrido 2008).
et al., 2012 and Morel et al., 2012), and an EEG The emotional effects on the P1 may
study indicated differentiated responses to in particular be conveyed by the low spa-
emotional and neutral faces from 70 ms in tial frequency content of the face stimuli
the superior temporal polysensory area in (e.g., Vlamings et al., 2009). They could
a patient with bilateral destruction of the reflect attentional modulation of stimulus
primary visual cortex (Andino, Menendez, processing induced by emotion (or so-called
Khateb, Landis, & Pegna, 2009). Although emotional attention processes, discussed in
these results need further confirmation, they Chapter 14). Indeed, P1 amplitude mod-
raise the interesting possibility that (1) the ulations have often been associated with
subcortical colliculo-pulvinar route to the selective attention processes, and the emo-
amygdala may effectively be involved in tional modulation over frontocentral record-
the perception of emotional faces, in paral- ing sites described by Eimer and colleagues
lel with or with a small temporal precedence in the time range of the P1 was dependent on
over the feedforward visual cortical route, the presentation of the faces at the attended
and (2) this route may involve much faster location, being abolished for ignored faces.
information flow than previously thought. It However, some of the emotional effects
is, however, also possible that the very early on the P1 were obtained in conditions
effects of emotional face perception reflect where the faces were presented centrally
purely a tuning of the responses of neuronal and the processing of emotional expressions
assemblies to relevant categories of stimuli was incidental to the task at hand (e.g.,
(defined either by overlearned visual cues Bayle & Taylor, 2010; for reviews, see Eimer
or by associative – including emotional – & Holmes, 2007; Vuilleumier & Pourtois,
learning, for example), or that they might 2007). The attentional processes associated
be associated with predictive coding mech- with emotional face processing thus seem
anisms (Rauss et al., 2011). to encompass an automatic component
Moreover, several studies have shown related to attentional capture by emotional
that the emotional expression of seen faces faces.
can modulate the P1 component that fol- These emotional modulations of the
lows the C1 and peaks typically between P1 may be mainly of extrastriate visual
80 and 130 ms (for a review, see Vuilleu- origin, reflecting gain control and amplifi-
mier & Pourtois, 2007). The P1 shows some cation mechanisms induced by emotional
general face sensitivity; however, it is not attention. However, the contribution of
considered to be a face-specific compo- other brain regions cannot be excluded.
nent. P1 modulation by facial expression Indeed, with intracerebral recordings, early
has been observed for the different basic responses to aversive visual stimuli have
emotions (compared to neutral condition), been reported as early as from 120 ms post-
as well as for faces yielding positive ver- stimulus onset in the OFC (Kawasaki et al.,
sus negative affective judgment or whose 2001). Differentiated amygdala responses to
emotional value was reinforced by aversive fearful versus neutral faces have also recently
conditioning. Thus, it seems to reflect a been reported from about 140 ms, a time
global emotional effect. Furthermore, the range compatible with some P1 effect (Pour-
emotional effects on the P1 are not spe- tois et al., 2010b). The top-down influence of
190 NATHALIE GEORGE
these regions may be responsible for the pro- influence of facial expression of emotions on
cesses of emotional attention reflected in P1 the N170 and M170 could reflect differences
modulations. Furthermore, the localization in configural information for distinct emo-
of the brain sources of EEG/MEG signal has tional expressions, which may be extracted
confirmed the involvement of a distributed from a perceptual representation stage com-
cortical network in the emotional modula- mon with that of face identity, even if the
tion of the P1, including insular, amygdalar, configural cues relative to emotional expres-
orbitofrontal, lateral frontal, and somatosen- sion and identity differ, as mentioned earlier.
sory cortex regions (e.g., Esslen et al., It is also possible that the emotional modu-
2004; Hung et al., 2010). This involvement lation of N170 and M170 reflects emotional
is consistent with the early establishment attention processes driven by the amygdala,
of a large-scale network during emotional similarly to the emotional modulation of the
processing. face-selective fusiform responses observed
Another type of early emotional influ- with fMRI.
ence concerns the N170 in response to faces. This raises the question of the neural ori-
This component follows the P1, and it peaks gin of the N170 and M170 modulations by
typically around 150–170 ms. The N170 (or emotional expression. Note first that these
M170 for its MEG counterpart) was his- EEG and MEG components may reflect at
torically characterized as associated with least partly different neural activity. The
an early stage of face-selective perceptual N170 has been proposed to reflect STS activ-
analysis (e.g., George, Evans, Fiori, David- ity (e.g., Henson et al., 2003), whereas the
off, & Renault, 1996). Following Bruce and M170 would reflect activity in the infe-
Young’s (1986) model, this stage was ini- rior occipital and fusiform regions (e.g.,
tially considered to be immune to the influ- Itier, Herdman, George, Cheyne, & Tay-
ences of emotional and/or cognitive factors. lor, 2006). It seems, however, more plau-
Accordingly, a number of studies manipu- sible that both components reflect brain
lating the emotional expressions of faces did responses within a distributed and partially
not find any impact of this manipulation on overlapping network of ventral and lateral
the N170 (e.g., Krolak-Salmon et al., 2001). occipitotemporal regions. Overall, the local-
However, more and more studies have put ization of the brain sources of EEG and
this view into question, reporting an influ- MEG signals has revealed that the emo-
ence of the facial expression of emotion on tional modulation of the N170 and M170
the N170 or on its magnetic counterpart, reflected activity of extrastriate regions,
the M170, in response to faces (e.g., Morel including the fusiform gyrus region (e.g.,
et al., 2009; Figure 7.3d). This N/M170 mod- Pegna et al., 2008). Moreover, nonvisual
ulation does not seem to be specific either to regions could contribute to N170 and M170
a given emotion or to emotion valence, but emotional modulations, because intracere-
it may vary with the intensity of the emo- bral recordings in epileptic patients have
tion expressed (Sprengelmeyer & Jentzsch, shown that an extended network of regions
2006). It is unclear whether it is conveyed is activated in response to faces by 150 ms,
by some particular spatial frequencies of the including medial temporal structures such
face because inconsistent results have been as the amygdala and extending into pari-
obtained in this regard (e.g., Pourtois, Dan, etal and frontal – including premotor –
Grandjean, Sander, & Vuilleumier, 2005; regions (e.g., Barbeau, et al., 2008; Krolak-
Vlamings et al., 2009). In any case, it seems Salmon, Henaff, Vighetto, et al., 2006; Pour-
to reflect an automatic influence of emo- tois et al., 2010b). Accordingly, a recent
tion, because the processing of emotional study with EEG source localization found
expressions was incidental in many stud- an extended set of regions showing differ-
ies and one study reported an N170 mod- entiated responses to distinct facial emo-
ulation for masked fearful versus nonfearful tional expressions in the time range of the
faces (Pegna, Landis, & Khateb, 2008). The N170. These regions included the STS, the
THE FACIAL EXPRESSION OF EMOTIONS 191
inferior frontal gyrus, the OFC, the ACC, that these early stages may not be asso-
and the right amygdala (Andino et al., 2009). ciated with any conscious access, explicit
Furthermore, several studies concentrated appraisal, or conceptual knowledge related
on the localization of amygdala activity to the perceived emotional expression.
from MEG signals, and they found amyg- These processes may rather be reflected by
dala responses sensitive to emotional expres- later ERP components, starting from about
sions in the time range of the M170 (for a 200 ms.
review, see Dumas, Attal, Dubal, Jouvent, &
George, 2011). Thus, it is likely that the mod-
ulation of the N170 and M170 responses to Beyond the Facial Expression of
faces by emotional expressions reflects dis- Emotion: The Many Faces of Emotions
tributed and possibly recurrent activity in a
Neutral Faces Can Elicit
network of interconnected regions involved
Emotion-Related Processes
in emotional face processing.
Finally, it is interesting to underline that Facial expressions of emotions constitute
the emotional modulations of the C1, P1, the main aspect of emotion perception
and N170 have sometimes been found inde- and elicitation obtained from face informa-
pendently of each other (for a review, see tion. However, even when not displaying
Vuilleumier & Pourtois, 2007). This finding any particular emotion, faces may trigger
emphasizes that some information about the emotion-related processes. In particular, it
emotional value of a face may be extracted has been shown that several processes asso-
before the brain “knows” it is a face (that is, ciated with social perception and social cog-
before face-selective mechanisms are trig- nition are rooted in emotion-related process-
gered). Moreover, it suggests that these ing. For example, the evaluation of some
effects reflect different types of emotional personality traits of neutral faces, such as
influences. In brief, some coarse aspects of trustworthiness, seems to be tightly linked
emotional expressions, possibly conveyed to general processes of valence appraisal;
by low spatial frequencies, may be pro- it involves overgeneralization of emotion-
cessed very early, independently of the face- related facial features, based on similarity to
selective perceptual analysis. It is possible expressions that signal approach or avoid-
that this very early processing engages a ance behavior. Such processes seem to take
fast subcortical route to the amygdala, or place automatically, because evidence for
that it reflects a rapid feedforward flow of amygdala involvement has been found in
information through cortical areas whose a paradigm of implicit evaluation of face
neurons are tuned to diagnostic features of trustworthiness (for a review, see Todorov,
emotional expressions. It is unclear whether 2008).
a first sweep of the top-down influence In the same vein, the emotional value
from anterior regions (such as the amyg- of contextual information concomitant with
dala and/or OFC) onto posterior visual a face encounter seems to be very easily
regions may also take place as early as by attached to that face, influencing both the
∼50 ms. Furthermore, more detailed, pos- affective judgment and the neural represen-
sibly configural, information about emo- tation of the face, and also biasing the eval-
tional expression would be extracted at uation of new faces having some likeness to
a face-selective perceptual representation the previously encountered one (e.g., Morel
stage that may be common to identity et al., 2012; Verosky & Todorov, 2010). In
processing, although relying on a differ- other words, even neutral faces may have
ent type of information. At this stage, the or easily acquire an intrinsic affective value.
functional interaction and coupling between This may be why the mere perception of
amygdala and perceptual regions, as well as neutral faces activates a distributed network
among a wider set of regions, are clearly of regions, including the amygdala (e.g.,
taking place. It is important to mention Ishai, Schmidt, & Boesiger, 2005).
192 NATHALIE GEORGE
directed or averted gaze, will remain among allows the transformation of a happy expres-
the most powerful stimuli for research in sion of a face into an angry expression of that
human affective neuroscience. same face, and intermediate images showing
varying degrees of emotion on the continuum
between enjoyment and anger can then be
Outstanding Questions and Future extracted.
Directions 2 Intracerebral electrodes are implanted for
presurgical diagnostic purposes in pharmaco-
r Emotions are dynamic in nature, but logically intractable epilepsy.
most of the knowledge accumulated so 3 This method allows applying various masks
far on the perception of emotional faces over faces, with each mask being generated
has been obtained with static images. as random holes or “bubbles” in an otherwise
What are the neural pathways and the black screen. The bubbles through which
the stimulus can be seen have different sizes
time course of the perception of dynamic
according to the different frequency bands
emotional expressions?
r The positive side of emotion is likely to explored. This technique allows the genera-
tion of numerous incomplete stimuli that can
be particularly important in regard with then be used in various detection, discrimina-
social stimuli such as faces, but it has tion, recognition, or categorization tasks. The
been somewhat neglected so far. Is the diagnostic visual cues used by the subjects to
same spatially and temporally distributed perform the task can then be computed by
network of activity involved in the pro- averaging the stimuli as a function of the task
cessing of the facial displays of positive responses.
affects?
r Numerous studies in social psychology
have shown the impact of the physical References
presence of others on human perceptual
and cognitive processes. How can emo- Adams, R. B., Jr., Gordon, H. L., Baird, A. A.,
tion perception and elicitation from faces Ambady, N., & Kleck, R. E. (2003). Effects of
be studied in a real-life setup? Would gaze on amygdala sensitivity to anger and fear
faces. Science, 300(5625), 1536.
such study lead to the uncovering of dif-
Adolphs, R. (2002). Neural systems for recogniz-
ferent mechanisms for the perception of ing emotion. Current Opinions in Neurobiol-
emotion in the context of live social inter- ogy, 12(2), 169–177.
actions? Adolphs, R. (2010). What does the amygdala con-
r Emotion perception from faces and social
tribute to social cognition? Annals of the New
perception are tightly interrelated. What York Academy of Sciences, 1191, 42–61.
are the relative weights of nature, culture, Adolphs, R., Damasio, H., & Damasio, A. R.
and individual differences on the neural (1994). Impaired recognition of emotion in
network and temporal dynamics of emo- facial expressions following bilateral dam-
tional face perception? age to the human amygdala. Nature, 372,
r What exactly is the role of the pulvinar 669–72.
Adolphs, R., Damasio, H., Tranel, D., Cooper,
in the different stages of emotional face
G., & Damasio, A. R. (2000). A role for
perception? And to what extent is a fast somatosensory cortices in the visual recog-
subcortical colliculo-pulvinar-amydgalar nition of emotion as revealed by three-
route actually involved in the perception dimensional lesion mapping. Journal of Neu-
of emotional faces? roscience, 20(7), 2683–90.
Adolphs, R., Gosselin, F., Buchanan, T. W.,
Tranel, D., Schyns, P., & Damasio, A. R.
Notes (2005). A mechanism for impaired fear
recognition after amygdala damage. Nature,
1 Morphing allows computing the smooth and 433(7021), 68–72.
continuous transformation that turns a stim- Adolphs, R., & Tranel, D. (2003). Amygdala dam-
ulus into another stimulus. For example, it age impairs emotion recognition from scenes
194 NATHALIE GEORGE
only when they contain facial expressions. Craig, A. D. (2011). Significance of the insula for
Neuropsychologia, 41(10), 1281–89. the evolution of human awareness of feel-
Andino, S. L., Menendez, R. G., Khateb, A., ings from the body. Annals of the New York
Landis, T., & Pegna, A. J. (2009). Electrophysi- Academy of Sciences,1225, 72–82.
ological correlates of affective blindsight. Neu- Dalton, K. M., Nacewicz, B. M., Johnstone, T.,
roimage, 44(2), 581–89. Schaefer, H. S., Gernsbacher, M. A., Gold-
Aviezer, H., Hassin, R. R., Bentin, S., & Trope, smith, H. H., et al. (2005). Gaze fixation
Y. (2008). Putting facial expressions back in and the neural circuitry of face processing in
context. In N. Ambady & J. J. Skowronski autism. Nature Neuroscience, 8(4), 519–26.
(Eds.), First impressions (pp. 255–86). New Derntl, B., Habel, U., Windischberger, C.,
York: Guilford Press. Robinson, S., Kryspin-Exner, I., Gur, R. C.,
Barbeau, E. J., Taylor, M. J., Regis, J., Mar- et al. (2009). General and specific responsive-
quis, P., Chauvel, P., & Liegeois-Chauvel, ness of the amygdala during explicit emotion
C. (2008). Spatiotemporal dynamics of face recognition in females and males. BMC Neu-
recognition. Cerebral Cortex, 18(5), 997–1009. roscience, 10, 91.
Bayle, D. J., & Taylor, M. J. (2010). Attention Dubal, S., Foucher, A., Jouvent, R., & Nadel,
inhibition of early cortical activation to fearful J. (2011). Human brain spots emotion in non
faces. Brain Research, 1313, 113–23. humanoid robots. Social, Cognitive, and Affec-
Beauchamp, M. S. (2011). Biological motion and tive Neuroscience, 6(1), 90–97.
multisensory integration: The role of the supe- Dumas, T., Attal, Y., Dubal, S., Jouvent, R., &
rior temporal sulcus. In R. B. Adams, N. George, N. (2011). Detection of activity from
Ambady, K. Nakayama, & S. Shimojo (Eds.), the amygdala with magnetoencephalography.
The science of social vision (pp. 409–20). New IRBM, 32(1), 42–47.
York: Oxford University Press. Eimer, M., & Holmes, A. (2007). Event-related
Breiter, H. C., Etcoff, N. L., Whalen, P. J., brain potential correlates of emotional face
Kennedy, W. A., Rauch, S. L., Buckner, R. processing. Neuropsychologia, 45(1), 15–31.
L., et al. (1996). Response and habituation of Ekman, P., Sorenson, E. R., & Friesen, W. V.
the human amygdala during visual process- (1969). Pan-cultural elements in facial displays
ing of facial expression. Neuron, 17(5), 875– of emotion. Science, 164(3875), 86–88.
87. Esslen, M., Pascual-Marqui, R. D., Hell, D.,
Bruce, V., & Young, A. (1986). Understanding Kochi, K., & Lehmann, D. (2004). Brain areas
face recognition. British Journal of Psychology, and time course of emotional processing. Neu-
77, 305–27. roimage, 21(4), 1189–1203.
Calder, A. J., Keane, J., Lawrence, A. D., & Frischen, A., Bayliss, A. P., & Tipper, S. P. (2007).
Manes, F. (2004). Impaired recognition of Gaze cueing of attention: visual attention,
anger following damage to the ventral stria- social cognition, and individual differences.
tum. Brain, 127(Pt. 9), 1958–69. Psychology Bulletin, 133(4), 694–724.
Calder, A. J., & Young, A. W. (2005). Under- Fusar-Poli, P., Placentino, A., Carletti, F., Landi,
standing the recognition of facial identity P., Allen, P., Surguladze, S., et al. (2009).
and facial expression. Nature Reviews Neuro- Functional atlas of emotional faces processing:
science, 6(8), 641–51. A voxel-based meta-analysis of 105 functional
Canli, T., Sivers, H., Whitfield, S. L., Gotlib, magnetic resonance imaging studies. Journal
I. H., & Gabrieli, J. D. (2002). Amygdala of Psychiatry and Neuroscience, 34(6), 418–32.
response to happy faces as a function of Gamer, M., & Buchel, C. (2009). Amygdala acti-
extraversion. Science, 296(5576), 2191. vation predicts gaze toward fearful eyes. Jour-
Chiao, J. Y., Iidaka, T., Gordon, H. L., Nogawa, nal of Neuroscience, 29(28), 9123–26.
J., Bar, M., Aminoff, E., et al. (2008). Cultural Garrido, M.I., Barnes, G.R., Sahani, M., & Dolan,
specificity in amygdala response to fear faces. R.J. (2012). Functional evidence for a dual
Journal of Cognitive Neuroscience, 20(12), 2167– route to amygdala. Current Biology, 22, 129–
174. 134.
Conty, L., Russo, M., Loehr, V., Hugueville, L., George, N., & Conty, L. (2008). Facing the gaze
Barbu, S., Huguet, P., et al. (2010). The mere of others. Neurophysiologie Clinique/Clinical
perception of eye contact increases arousal Neurophysiology, 38(3), 197–207.
during a word-spelling task. Social Neuro- George, N., Driver, J., & Dolan, R. J. (2001).
science, 5(2), 171–86. Seen gaze-direction modulates fusiform
THE FACIAL EXPRESSION OF EMOTIONS 195
activity and its coupling with other brain areas Kampe, K. K., Frith, C. D., Dolan, R. J., & Frith,
during face processing. Neuroimage, 13(6 Pt. 1), U. (2001). Reward value of attractiveness and
1102–12. gaze. Nature, 413(6856), 589.
George, N., Evans, J., Fiori, N., Davidoff, J., & Kawasaki, H., Kaufman, O., Damasio, H., Dama-
Renault, B. (1996). Brain events related to nor- sio, A. R., Granner, M., Bakken, H., et al.
mal and moderately scrambled faces. Brain (2001). Single-neuron responses to emotional
Research: Cognitive Brain Research, 4(2), 65– visual stimuli recorded in human ventral pre-
76. frontal cortex. Nature Neuroscience, 4(1), 15–
George, N., Morel, S., & Conty, L. (2008). Vis- 16.
ages et electrophysiologie. In E. Barbeau, S. Krolak-Salmon, P., Fischer, C., Vighetto, A.,
Joubert, & O. Felician (Eds.), Traitement et & Mauguiere, F. (2001). Processing of facial
reconnaissance des visages: du percept à la per- emotional expression: spatio-temporal data
sonne (pp. 113–42). Marseille: Solal (collection as assessed by scalp event-related potentials.
Neuropsychologie). European Journal of Neuroscience, 13(5), 987–
Gobbini, M. I., & Haxby, J. V. (2007). Neural 94.
systems for recognition of familiar faces. Neu- Krolak-Salmon, P., Henaff, M. A., Bertrand, O.,
ropsychologia, 45(1), 32–41. Vighetto, A., & Mauguiere, F. (2006). Part
Halla, J. K., Huttona, S. B., & Morgana, M. J. II: Recognising facial expressions. Revue Neu-
(2010). Sex differences in scanning faces: Does rologique (Paris), 162(11), 1047–58.
attention to the eyes explain female superior- Krolak-Salmon, P., Henaff, M. A., Vighetto, A.,
ity in facial expression recognition? Cognition Bauchet, F., Bertrand, O., Mauguiere, F., et
and Emotion, 24(4), 629–37. al. (2006). Experiencing and detecting happi-
Hennenlotter, A., Schroeder, U., Erhard, P., ness in humans: The role of the supplementary
Haslinger, B., Stahl, R., Weindl, A., et motor area. Annals of Neurology, 59(1), 196–99.
al. (2004). Neural correlates associated Lamm, C., & Singer, T. (2010). The role of ante-
with impaired disgust processing in pre- rior insular cortex in social emotions. Brain
symptomatic Huntington’s disease. Brain, Structure and Function, 214(5–6), 579–91.
127(Pt. 6), 1446–53. Liu, L., & Ioannides, A. A. (2010). Emotion sep-
Henson, R. N., Goshen-Gottstein, Y., Ganel, T., aration is completed early and it depends
Otten, L. J., Quayle, A., & Rugg, M. D. (2003). on visual field presentation. PLoS One, 5(3),
Electrophysiological and haemodynamic cor- e9790.
relates of face perception, recognition and Luo, Q., Holroyd, T., Jones, M., Hendler,
priming. Cerebral Cortex, 13(7), 793–805. T., & Blair, J. (2007). Neural dynamics for
Hsu, M., Bhatt, M., Adolphs, R., Tranel, D., facial threat processing as revealed by gamma
& Camerer, C. F. (2005). Neural systems band synchronization using MEG. Neuroim-
responding to degrees of uncertainty in human age, 34(2), 839–47.
decision-making. Science, 310(5754), 1680–83. Mason, M. F., Tatkow, E. P., & Macrae, C. N.
Hung, Y., Smith, M. L., Bayle, D. J., Mills, T., (2005). The look of love: Gaze shifts and per-
Cheyne, D., & Taylor, M. J. (2010). Unat- son perception. Psychological Science, 16(3),
tended emotional faces elicit early lateral- 236–39.
ized amygdala-frontal and fusiform activa- Morel, S., Beaucousin, V., Perrin, M., & George,
tions. Neuroimage, 50(2), 727–33. N. (2012). Very early modulation of brain
Ishai, A., Schmidt, C. F., & Boesiger, P. (2005). responses to neutral faces by a single prior
Face perception is mediated by a distributed association with an emotional context: Evi-
cortical network. Brain Research Bulletin, dence from MEG. Neuroimage, 61, 1461–70.
67(1–2), 87–93. Morel, S., Ponz, A., Mercier, M., Vuilleumier, P.,
Itier, R. J., Herdman, A. T., George, N., Cheyne, & George, N. (2009). EEG-MEG evidence for
D., & Taylor, M. J. (2006). Inversion and early differential repetition effects for fearful,
contrast-reversal effects on face processing happy and neutral faces. Brain Research, 1254,
assessed by MEG. Brain Research, 1115(1), 108– 84–98.
20. Morris, J. S., Frith, C. D., Perrett, D. I., Row-
Jack, R. E., Blais, C., Scheepers, C., Schyns, P. land, D., Young, A. W., Calder, A. J., et al.
G., & Caldara, R. (2009). Cultural confusions (1996). A differential neural response in the
show that facial expressions are not universal. human amygdala to fearful and happy facial
Current Biology, 19(18), 1543–48. expressions. Nature, 383(6603), 812–15.
196 NATHALIE GEORGE
Morrison, S. E., & Salzman, C. D. (2010). Re- Neuroscience and Biobehavioral Reviews, 35(5),
valuing the amygdala. Current Opinions in 1237–53.
Neurobiology, 20(2), 221–30. Righart, R., & de Gelder, B. (2008). Rapid influ-
Murty, V. P., Ritchey, M., Adcock, R. A., & ence of emotional scenes on encoding of facial
LaBar, K. S. (2010). fMRI studies of successful expressions: An ERP study. Social, Cognitive,
emotional memory encoding: A quantitative and Affective Neuroscience, 3(3), 270–78.
meta-analysis. Neuropsychologia, 48(12), 3459– Rolls, E. T. (2005). Emotion explained. Oxford:
69. Oxford University Press.
N’Diaye, K., Sander, D., & Vuilleumier, P. Rolls, E. T. (2007). The representation of infor-
(2009). Self-relevance processing in the mation about faces in the temporal and frontal
human amygdala: Gaze direction, facial lobes. Neuropsychologia, 45(1), 124–43.
expression, and emotion intensity. Emotion, Roy, M., Shohamy, D., & Wager, T.D. (2012).
9(6), 798–806. Ventromedial prefrontal-subcortical systems
Pegna, A. J., Landis, T., & Khateb, A. (2008). and the generation of affective meaning.
Electrophysiological evidence for early non- Trends in Cognitive Sciences, 16, 147–156.
conscious processing of fearful facial expres- Rushworth, M. F. S., Behrens, T. E. J., Rudebeck,
sions. International Journal of Psychophysiol- P. H., & Walton, M. E. (2007). Contrasting
ogy, 70(2), 127–36. roles for cingulate and orbitofrontal cortex in
Pessoa, L., & Adolphs, R. (2010). Emotion pro- decisions and social behaviour. Trends in Cog-
cessing and the amygdala: From a ‘low road’ nitive Sciences, 11(4), 168–76.
to ‘many roads’ of evaluating biological sig- Russell, J. A. (1991). Culture and the categoriza-
nificance. Nature Reviews Neuroscience, 11(11), tion of emotions. Psychological Bulletin, 110(3),
773–83. 426–50.
Pitcher, D., Dilks, D. D., Saxe, R. R., Triantafyl- Sato, W., Kochiyama, T., Uono, S., & Yoshikawa,
lou, C., & Kanwisher, N. (2011). Differential S. (2010). Amygdala integrates emotional
selectivity for dynamic versus static informa- expression and gaze direction in response
tion in face-selective cortical regions. Neu- to dynamic facial expressions. Neuroimage,
roimage, 56(4), 2356–63. 50(4), 1658–65.
Pourtois, G., Dan, E. S., Grandjean, D., Sander, Schumann, C. M., Bauman, M. D., & Amaral,
D., & Vuilleumier, P. (2005). Enhanced extras- D. G. (2011). Abnormal structure or func-
triate visual response to bandpass spatial fre- tion of the amygdala is a common component
quency filtered fearful faces: Time course of neurodevelopmental disorders. Neuropsy-
and topographic evoked-potentials mapping. chologia, 49(4), 745–59.
Human Brain Mapping, 26(1), 65–79. Schupp, H. T., Ohman, A., Junghofer, M.,
Pourtois, G., Grandjean, D., Sander, D., & Weike, A. I., Stockburger, J., & Hamm, A. O.
Vuilleumier, P. (2004). Electrophysiological (2004). The facilitated processing of threaten-
correlates of rapid spatial orienting towards ing faces: An ERP analysis. Emotion, 4(2), 189–
fearful faces. Cerebral Cortex, 14(6), 619–33. 200.
Pourtois, G., Spinelli, L., Seeck, M., & Vuilleu- Schyns, P. G., & Oliva, A. (1999). Dr. Angry and
mier, P. (2010a). Modulation of face process- Mr. Smile: When categorization flexibly mod-
ing by emotional expression and gaze direc- ifies the perception of faces in rapid visual pre-
tion during intracranial recordings in right sentations. Cognition, 69(3), 243–65.
fusiform cortex. Journal of Cognitive Neuro- Senju, A., & Johnson, M. H. (2009). The eye
science, 22(9), 2086–2107. contact effect: Mechanisms and development.
Pourtois, G., Spinelli, L., Seeck, M., & Vuilleu- Trends in Cognitive Sciences, 13(3), 127–34.
mier, P. (2010b). Temporal precedence of Smith, F. W., & Schyns, P. G. (2009). Smile
emotion over attention modulations in the lat- through your fear and sadness: transmitting
eral amygdala: Intracranial ERP evidence from and identifying facial expression signals over
a patient with temporal lobe epilepsy. Cog- a range of viewing distances. Psychological Sci-
nitve Affective and Behavioral Neuroscience, ence, 20(10), 1202–8.
10(1), 83–93. Sprengelmeyer, R., & Jentzsch, I. (2006). Event
Rauss, K., Schwartz, S., & Pourtois, G. (2011). related potentials and the perception of inten-
Top-down effects on early visual processing sity in facial expressions. Neuropsychologia,
in humans: A predictive coding framework. 44(14), 2899–2906.
THE FACIAL EXPRESSION OF EMOTIONS 197
Sprengelmeyer, R., Young, A. W., Calder, A. J., Vlamings, P. H., Goffaux, V., & Kemner, C.
Karnat, A., Lange, H., Homberg, V., et al. (2009). Is the early modulation of brain activ-
(1996). Loss of disgust. Perception of faces ity by fearful facial expressions primarily
and emotions in Huntington’s disease. Brain, mediated by coarse low spatial frequency
119(Pt.5), 1647–65. information? Journal of Vision, 9(5), 12.1–13.
Stolarova, M., Keil, A., & Moratti, S. (2006). Vuilleumier, P., Armony, J. L., Driver, J., &
Modulation of the C1 visual event-related Dolan, R. J. (2003). Distinct spatial frequency
component by conditioned stimuli: Evidence sensitivities for processing faces and emo-
for sensory plasticity in early affective percep- tional expressions. Nature Neuroscience, 6(6),
tion. Cerebral Cortex, 16(6), 876–87. 624–31.
Susskind, J. M., Lee, D. H., Cusi, A., Feiman, Vuilleumier, P., & Pourtois, G. (2007). Dis-
R., Grabski, W., & Anderson, A. K. (2008). tributed and interactive brain mechanisms
Expressing fear enhances sensory acquisition. during emotion face perception: Evidence
Nature Neuroscience, 11(7), 843–50. from functional neuroimaging. Neuropsycholo-
Tamietto, M., & de Gelder, B. (2010). Neural gia, 45(1), 174–94.
bases of the non-conscious perception of emo- Vuilleumier, P., Richardson, M. P., Armony, J.
tional signals. Nature Reviews Neuroscience 11, L., Driver, J., & Dolan, R. J. (2004). Distant
697–709. influences of amygdala lesion on visual cortical
Tamietto, M., Pullens, P., de Gelder, B., activation during emotional face processing.
Weiskrantz, L., & Goebel, R. (2012). Sub- Nature Neuroscience, 7(11), 1271–78.
cortical connections to human amygdala and Wicker, B., Keysers, C., Plailly, J., Royet, J. P.,
changes following destruction of the visual Gallese, V., & Rizzolatti, G. (2003). Both of
cortex. Current Biology, 22, 1449–55. us disgusted in My insula: The common neu-
Todorov, A. (2008). Evaluating faces on ral basis of seeing and feeling disgust. Neuron,
trustworthiness: an extension of systems 40(3), 655–64.
for recognition of emotions signaling Wicker, B., Perrett, D. I., Baron-Cohen, S.,
approach/avoidance behaviors. Annals of & Decety, J. (2003). Being the target of
the New York Academy of Sciences, 1124, another’s emotion: A PET study. Neuropsy-
208–24. chologia, 41(2), 139–46.
Tranel, D., Damasio, A. R., & Damasio, H. Winston, J. S., O’Doherty, J., & Dolan, R. J.
(1988). Intact recognition of facial expression, (2003). Common and distinct neural responses
gender, and age in patients with impaired during direct and incidental processing of mul-
recognition of face identity. Neurology, 38, tiple facial emotions. Neuroimage, 20(1), 84–
690–96. 97.
Trautmann, S. A., Fehr, T., & Herrmann, M. Wyk, B. C., Hudac, C. M., Carter, E. J.,
(2009). Emotions in motion: Dynamic com- Sobel, D. M., & Pelphrey, K. A. (2009).
pared to static facial expressions of disgust and Action understanding in the superior tempo-
happiness reveal more widespread emotion- ral sulcus region. Psychological Science, 20(6),
specific activations. Brain Research, 1284, 771–77.
100–15. Young, A. W., Newcombe, F., de Haan, E. H. F.,
Tsuchiya, N., Kawasaki, H., Oya, H., Howard, Small, M., & Hay, D. C. (1993). Face percep-
M. A., III, & Adolphs, R. (2008). Decod- tion after brain injury: Selective impairments
ing face information in time, frequency and affecting identity and expression. Brain, 116,
space from direct intracranial recordings of the 941–59.
human brain. PLoS One, 3(12), e3892. Young, A. W., Rowland, D., Calder, A. J., Etcoff,
Verosky, S. C., & Todorov, A. (2010). Generaliza- N. L., Seth, A., & Perrett, D. I. (1997). Facial
tion of affective learning about faces to per- expression megamix: Tests of dimensional
ceptually similar faces. Psychological Science, and category accounts of emotion recognition.
21(6), 779–85. Cognition, 63(3), 271–313.
CHAPTER 8
Anthony P. Atkinson
Emotions are often expressed or signaled via from body posture and movement stimuli.
postures and movements of the whole body The subsequent two sections, consider the
or its parts. Although faces are body parts, different visual cues that humans use to
the face and the rest of the body are dis- perceive bodies per se and emotional bod-
tinct means for expressing, or channels for ies in particular. I then survey the current
signaling, emotions. There is a large corpus state of knowledge about how the brain pro-
of research on facial expressions and their cesses visual information relating to other
perception, surveyed elsewhere in this book people’s bodies and bodily expressions of
(see Chapter 7). A central aim of this chap- emotion. In these latter two sections we
ter is to survey the smaller but neverthe- see that viewing other people’s bodies and
less important corpus of research devoted bodily expressions engages not only neu-
to investigating the perception of bodily ral mechanisms with primarily visual func-
expressed emotions and its neural substrate. tions but also neural mechanisms involved in
Its particular focus is on the visual cues planning and executing actions and in elic-
underlying body and bodily emotion percep- iting emotional responses and representing
tion. the changes in body state that are impor-
The chapter begins with a considera- tant and probably essential components of
tion of what constitutes a bodily expres- emotions (see Chapter 1). In the last section,
sion or signal of emotion, highlighting a we see that some of this evidence points to
difference between actions that directly simulation or “shared substrates” hypotheses
convey an emotion and actions that do not of emotion recognition, according to which
but are nevertheless performed in an emo- recognizing another’s emotional expres-
tional way. This is followed by a brief sum- sion recruits neural mechanisms in the
mary of studies that have demonstrated the perceiver responsible for generating his
ability of human observers to distinguish or her own emotional experience and
between and identify a range of emotions behavior.
198
BODILY EXPRESSIONS OF EMOTION 199
What Constitutes a Bodily Expression p. 34). Such actions, being habitual, can
or Signal of Emotion? become reliable predictors of a person’s sub-
sequent actions and clues to his or her state
I begin with a note on terminology. It is of mind. Furthermore, Darwin suggested
important to distinguish between “expres- that the attempt to exert voluntary con-
sions” and “signals” (or “displays”) of emo- trol over such habitual actions nevertheless
tion. The former term captures the idea that often leaves behind or may itself directly
what are primarily internal changes in bodily result in still visible expressive movements.
states and feelings nevertheless often have The principle of antithesis expresses the idea
external manifestations visible to the naked that some actions that are not themselves
eye (or audible to the naked ear); one’s emo- serviceable (i.e., do not or did not serve
tional state is expressed or revealed to the an adaptive function) are nevertheless per-
world via one’s body, face, and voice. This is formed because they are opposite in nature
what some authors have called the “readout” to actions that are serviceable. For exam-
hypothesis (e.g., Buck, 1994). By contrast, ple, shrugging of the shoulders, Darwin sug-
calling something an emotion signal signi- gested, is the antithetical action of a confi-
fies its intentionally communicative nature, dent or aggressive stance. The principle of
whether the signaler is experiencing the dis- direct action of the nervous system is essen-
played emotion or not. Fridlund (1991), for tially the idea that emotions involve some
example, argued that displays of emotion are direct and usually automatic activity of cer-
specific to the intentions of the displayer and tain parts of the nervous system, reflected in
the social context of the display and rarely, if movements and physiological changes char-
ever, directly reflect underlying emotional or acteristic of particular emotions that are
motivational states. For present purposes I independent of the will and are largely habit-
am simply going to note my agreement with ual. Examples include physiological changes
Buck (1994) – and, I suspect, many other evident in the skin or internal organs, the
emotion researchers and affective neurosci- trembling associated with fear or rage, and
entists – that people sometimes unintention- apparently purposeless movements such as
ally express emotions and sometimes inten- clapping hands and jumping in joy.
tionally signal emotions even when they With respect to the visual perception
are not experiencing them, but in many of bodily expressed emotions, more recent
instances emotions involve mixtures of both research has focused on bodily postures
unintentional expression and intentional sig- and movements, leaving aside other visible
nal. I will also ride roughshod over the dis- bodily changes, such as color of the skin
tinction I have just drawn by henceforth and sweating. An examination of contempo-
using “expression” as shorthand to refer both rary research on emotion perception reveals
to unintentionally expressed and intention- a distinction, often implicit, between two
ally signaled emotions. Nonetheless, the dis- ways in which emotions can be expressed
tinction will rear its head again shortly when via bodily postures and movements, which
we consider symbolic gestures. cuts across Darwin’s three principles. On
In Darwin’s (1872/1998) seminal work, the one hand, there are what we can call
emotional expressions were characterized expressive actions – actions that are direct
in terms of three principles: those of “ser- manifestations of internal emotional states,
viceable habits,” “antithesis,” and “direct such as fleeing or freezing in fear, retching
action of the nervous system.” Serviceable in disgust, or standing erect with expanded
habits are actions with direct or indirect chest and clenched fists in anger. On the
adaptive significance that are reliably asso- other hand, there are everyday actions that
ciated with “certain states of the mind, are performed in an emotional way. In such
in order to relieve or gratify certain sen- cases, the emotion in some sense “leaks out”
sations, desires, etc.” (Darwin, 1872/1998, of one’s action. For example, one might
200 ANTHONY P. ATKINSON
perform a transitive action, such as pick- such as anger, fear, happiness, and disgust,
ing up an object or knocking on or clos- which are distinguished from more com-
ing a door, with a sharp, aggressive move- plex social and moral emotions, such as jeal-
ment that could readily be interpreted as ousy, guilt, and embarrassment. One such
an expression of anger, whether or not one reason is that basic emotions are in part
intended it to be so. Or one’s slow, slumped defined by characteristic facial expressions,
walk might suggest a sad, depressed state. consisting in distinct sets of facial muscle
Both expressive actions and nonemo- actions (see Chapter 1 for further discus-
tional actions performed in an emotional sion). It is less clear whether these basic
way can be and often have been adopted emotions can be defined by distinct sets
or adapted as conventional gestures with of bodily as well as facial actions (an issue
symbolic meaning. In many cultures, for that is ripe for further investigation). In any
example, raised fists are symbolic of anger, case, considerable research has shown that
and a bowed head with the face buried human observers are readily able to distin-
in one’s hands or the miming of sniffing guish or identify at least a limited set of emo-
and wiping away tears can signal sadness. tions – particularly basic emotions – from
There are also bodily gestures whose origin bodily expressions in the absence of facial
may not lie either with expressive actions and vocal cues. Evidence comes from sev-
or nonemotional actions; namely, arbi- eral different types of tasks, but most com-
trary signals with culture-specific meanings. monly from forced-choice emotion-labeling
There are many examples of such gestures, tasks, in which observers typically show sig-
particularly involving the hands, in sign nificantly above-chance performance when
languages (e.g., Poizner, 1981). Following asked to select from a limited list the sin-
Ekman and Friesen (1969) and other emo- gle word that best describes the viewed
tion researchers (e.g., Buck, 1984; Fridlund, bodily expression. Accurate forced-choice
1991), I refer to the general class of symbolic emotion classification has been demon-
gestures, whether they are arbitrary or orig- strated with static images depicting peo-
inate from expressive or everyday nonemo- ple intentionally portraying emotions with
tional actions, as emblems. The class of bod- static body postures (e.g., Atkinson, Heber-
ily emblems consisting of arbitrary gestures lein, & Adolphs, 2007), computer-generated
has been studied little, if at all, by affective mannequins manipulated to reflect descrip-
neuroscientists, perhaps because of the cul- tions of emotional postures (Coulson, 2004),
tural specificity of such gestures and of how and single frames extracted from movie
few of them signal specific emotional states, clips of people intentionally portraying emo-
particularly when unaccompanied by facial tions (e.g., Atkinson, Dittrich, Gemmell, &
gestures. Nonetheless, it is worth bearing in Young, 2004; Atkinson, Heberlein, et al.,
mind that specific examplars of nonemo- 2007; Hadjikhani & de Gelder, 2003). Exam-
tional actions and particularly of expres- ples of such images are shown in Figure 8.1.
sive actions used by affective neuroscientists Accurate forced-choice emotion classi-
can vary in how symbolic or conventional fication has also been demonstrated with
they are. moving images of whole-body expressions
in the form of expressive actions (Atkinson
et al., 2004; Atkinson, Heberlein, et al., 2007;
The Visual Perception of Bodily Atkinson, Tunstall, & Dittrich, 2007), dance
Expressed Emotions movements intended to portray specific
emotions (e.g., Dittrich, Troscianko, Lea, &
For theoretical and practical reasons, Morgan, 1996; Hejmadi, Davidson, & Rozin,
research on emotion perception and recog- 2000), combinations of specific body move-
nition has focused predominantly on the ments that were not deliberate expressions
ability of people to discriminate or iden- of particular emotions (de Meijer, 1989),
tify “basic” emotions (e.g., Ekman, 1992), and walking movements either intended to
BODILY EXPRESSIONS OF EMOTION 201
Figure 8.1. a–d. Still images extracted from short movie clips of body movements intended to express
anger (a) and fear (b–d). Images a and b were extracted from full-light or fully illuminated displays, in
which the whole body is visible (with the face covered). Images c and d are stills from, respectively,
patch-light and point-light displays of the same movement sequence and time point as for image b.
For details of how these stimuli were constructed, see Atkinson et al. (2004) and Atkinson et al. (2012).
e–f: Photographs of emotional postures intended to express anger (e) and fear (f), drawn from a set
used by Atkinson, Heberlein, and Adolphs (2007) (created by A. S. Heberlein). g: A still frame from a
point-light display of a human walking (with no intention to express any emotion). Point-light and
patch-light stimuli are useful for studies of movement-based emotion perception because both face
and morphological cues are absent or minimal, but kinematic cues are preserved.
portray specific emotions (e.g., Heberlein, suggesting that observers make use of the
Adolphs, Tranel, & Damasio, 2004; Mon- motion or multiple form cues (or both)
tepare, Goldstein, & Clausen, 1987) or available in the moving images to help them
that reflected self-induced emotional states identify the emotions. Observers also use
(Roether, Omlor, Christensen, & Giese, motion information to aid their judgments
2009; Roether, Omlor, & Giese, 2008). of the emotional intensity of whole-body
Drinking and knocking arm movements expressive actions (Atkinson et al., 2004).
alone, performed with the intention of Accurate classification of emotions in
expressing specific emotional states, are whole-body movements has also been
sufficient for observers to distinguish demonstrated using emotion-rating tasks,
among those emotions (Pollick, Paterson, in which observers are required to rate
Bruderlin, & Sanford, 2001). Forced-choice either how compatible the viewed move-
classification accuracy is greater for moving ments are with the emotion that the actor
whole-body expressive actions than from intended to portray (Sawada, Suda, & Ishii,
single static frames extracted from the 2003) or how much of each particular emo-
same video footage (Atkinson et al., 2004), tion is in each display (i.e., its intensity)
202 ANTHONY P. ATKINSON
without being told what emotion the actor impairs the recognition of their identity. It is
was intending to portray (Atkinson, Heber- generally considered that face inversion dis-
lein, et al., 2007). Even when asked sim- rupts configural processing, specifically the
ply to describe what they see, observers coding of second-order relational informa-
are able to identify intended expressions of tion; that is, the metric distances amongst
some emotions at greater than chance levels features (e.g., Diamond & Carey, 1986). In
in individual displays of whole-body dance Reed et al.’s (2006) study, participants were
movements (Hejmadi et al., 2000) and in impaired in discriminating pairs of inverted
expressive action portrayals (Hubert et al., compared to upright whole-body postures
2007; Moore, Hobson, & Lee, 1997). but not houses. However, the matching of
isolated body parts (arms, legs, heads) was
unaffected by inversion, indicating that, as
The Visual Cues That Humans Use to with isolated facial features, individual body
Perceive Bodies and Their Motion parts do not evoke configural processing.
Disrupting first-order spatial relations, by
What form and motion information does the rearranging the body parts around the trunk
human visual system extract from others’ (by, for example, putting the arms in the leg
body postures and movements that might and head positions), abolished the inversion
be used as a basis for judging their emo- effect, indicating that such first-order con-
tional states? In this section, I consider what figural cues do not contribute to body pos-
form and motion information our visual sys- ture recognition. Presenting half-body pos-
tems extract from bodies per se. In a later tures that were divided along the vertical
section this discussion is extended to emo- midline (i.e., left or right halves), which pre-
tional bodies. serves the structural hierarchy of body parts
The form of the human body could but disrupts holistic template matching, did
be represented in several different ways, not abolish the body inversion effect. In
demarcating points on a configural-pro- contrast, presenting half-body postures that
cessing continuum from part-based to were divided along the horizontal midline
holistic processing. Thus, bodies could be (the waist), which preserves salient parts
represented in terms of individual body (e.g., both arms) but disrupts structural hier-
parts or features, the relative positions of archy information, did not produce an inver-
those parts (i.e., first-order spatial relations), sion effect. Thus, the particular form of con-
the structural hierarchy of body parts (i.e., figural processing critical to body posture
first-order configuration plus information recognition, as indexed by the presence of
about the relative position of features with an inversion effect, appears to be the struc-
respect to the whole body), or whole-body tural hierarchy of body parts; that is, the
posture templates (Reed, Stone, Grubb, & positions of body parts relative to them-
McGoldrick, 2006). Unlike faces, the rela- selves and to the whole body. Nonetheless,
tive positions of body parts change as people there is no body inversion effect for head-
move, which suggests the need for a rela- less bodies (Minnebusch, Suchan, & Daum,
tively fine-grained structural description of 2009; Yovel, Pelc, & Lubetzky, 2010), imply-
the spatial relationships among body parts. ing that configural information about the
A series of experiments by Reed and head is critical to body posture processing.
colleagues (Reed et al., 2006) suggests that There are three main classes of informa-
the recognition of (nonemotional) body pos- tion pertaining to the movements of human
tures depends on the processing of the struc- bodies: the changes of structural or form
tural hierarchy of body parts. This study information over time (including motion-
drew on the well-known inversion effect in mediated structural information), kinemat-
face recognition: that turning faces upside ics (e.g., velocity, acceleration, displace-
down impairs the ability to recognize their ment) and dynamics (motion specified in
identity more than inverting nonface objects terms of mass and force). Considerable
BODILY EXPRESSIONS OF EMOTION 203
attention has been given to the role of kine- the whole body, either from static form
matics in specifying cues for action and or motion-mediated structural cues (e.g.,
person perception (e.g., Westhoff & Troje, Casile & Giese, 2005; Mather, Radford, &
2007). Typically, these studies employ point- West, 1992). Nevertheless, neuropsychologi-
light or patch-light displays of human or cal and neurophysiological evidence demon-
other biological motion (see Figure 8.1), in strates that form information can indeed
which static form information is minimal or subserve biological motion perception from
absent but motion information (kinematics point-light displays (e.g., McLeod, Dittrich,
and dynamics) and motion-mediated struc- Driver, Perrett, & Zihl, 1996; Vaina, Cowey,
tural information are preserved (Johansson, LeMay, Bienfang, & Kikinis, 2002). The pro-
1973). Point-light displays of body move- cessing of changes in the form of the body
ments provide a sufficient basis for observers over time may be particularly important
to discriminate biological motion from other (e.g., Beintema & Lappe, 2002), especially
types of motion and to make accurate judg- in the context of more sophisticated tasks,
ments about the people making the move- such as recognizing emotional states or com-
ments, including sex from gait, identity plex actions (Giese & Poggio, 2003).
from gait or actions, and complex individ- This conclusion gains some support from
ual or social actions from whole-body move- inversion effects in biological motion per-
ments (reviewed by Blake & Shiffrar, 2007). ception. The spontaneous identification of
Some of this evidence shows equivalent or point-light motion displays as biological
near equivalent performance with point- motion is impaired when they are shown
light compared to full-light (or solid-body) upside down (e.g., Bertenthal & Pinto, 1994;
displays, in which the whole body is visi- Shipley, 2003). Moreover, neural activation
ble (e.g., Runeson & Frykholm, 1981), which characteristic of upright biological motion
suggests that static form cues are less impor- displays is attenuated or absent when such
tant than motion cues and may often be displays are inverted (Grossman & Blake,
unnecessary for successful judgments about 2001; Pavlova, Lutzenberger, Sokolov, &
people and their actions based on their visi- Birbaumer, 2004). Inversion of point-light
ble behavior. displays also disrupts the ability to distin-
Evidence for the relative importance of guish the identity of the actors from their
kinematic cues comes from studies that actions (Loula, Prasad, Harber, & Shiffrar,
measure the effects on recognition of 2005), and sex judgments based on gait
changes in certain kinematic or structural tend to be reversed (Barclay, Cutting, &
dimensions of point-light stimuli. For ex- Kozlowski, 1978). Although it is likely that
ample, accuracy in judging the sex of inversion of biological motion disrupts the
point-light walkers was influenced more by processing of dynamic cues related to move-
“body sway” than by the ratio of shoulder ment within the earth’s gravitational field
to hip width in Mather and Murdoch’s (e.g., Barclay et al., 1978), there is also
(1994) study, and it was greater when some evidence to suggest that inversion of
point-light walkers were normalized with whole-body movements impairs the pro-
respect to their size (thus providing only cessing of configural information (e.g., Pinto
motion information) than when they were & Shiffrar, 1999).
normalized with respect to their motion Finally, recent research provides evidence
information (thus providing only size cues) that the relative contributions of form and
in Troje’s (2002) study. motion cues to biological motion perception
It has been argued that the ability to dis- depend on the body parts in question and
criminate at least simple biological move- the stimulus duration (Thurman, Giese &
ments in point-light displays may be based Grossman, 2010): Observers relied on form
on relatively low-level or mid-level visual information in the upper body (head and
processing that does not involve the recon- shoulder posture) and on dynamic informa-
struction of the form of body parts or of tion in the lower body (the relative motion
204 ANTHONY P. ATKINSON
a role for static form cues for both face and – attests to the importance of form cues
body stimuli. Bassili (1978) reported greater in emotion perception; conversely, the fact
emotion classification accuracy for full-light that emotion classification performance was
compared to point-light facial movements still substantially above chance, even in the
(except for happy expressions), and Dit- spatially inverted, reversed patch-light dis-
trich (1991) reported equivalent emotion plays, attests to the importance of kinemat-
recognition performance for point-light face ics in providing cues for emotion perception.
stimuli in which the dots demarcated key Although it is likely that spatial inversion
facial structures (e.g., eyes, mouth) and of biological motion disrupts the process-
those in which the dots were positioned ing of dynamic cues related to movement
randomly on the face. This latter result within the earth’s gravitational field, if that
contrasts with Hill, Jinno, and Johnston’s were all that spatial inversion impaired, then
(2003) finding that sex judgments from facial we should not have seen a greater effect of
movements were more accurate with spa- orientation for the patch-light compared to
tially normalized than pseudo-random dot full-light stimuli.
placement and thus highlights the relation- What specific form-related cues are used
ship between form and motion information in emotion perception from body expres-
in specifying cues for emotion perception. sions? One suggestion is that the over-
For bodily expressed emotions, emotion all shape of particular body postures, such
recognition accuracy tends to be lower with as their angularity or roundedness, informs
point-light (Dittrich et al., 1996) and patch- emotion judgments (Aronoff, Woike, &
light (Atkinson et al., 2004) compared to Hyman, 1992). The spatial inversion effects
full-light displays of body movements. that we found (Atkinson, Tunstall, et al.,
Building on this earlier work, my col- 2007) highlight the importance of relational
leagues and I have demonstrated robust or configural cues, adding weight to previous
effects of stimulus inversion and motion claims that configural information plays an
reversal on the classification of basic important role in subserving emotion per-
emotions from patch-light and full-light ception from bodily expressions (Dittrich et
movie clips of bodily expressions (Atkin- al., 1996; Stekelenburg & de Gelder, 2004). In
son, Tunstall, et al., 2007). Spatially invert- contrast, the effects of motion reversal ten-
ing the 3-second-long movies significantly tatively suggest a possible role for spatiotem-
impaired emotion recognition accuracy, but poral cues (changes in form over time)
did so more in the patch-light than in in emotion recognition. Given the conven-
the full-light displays, indicating that spa- tional and sometimes symbolic (Buck, 1984)
tial inversion disrupts the processing of nature of our actors’ movements (see Atkin-
form cues more than it does the pro- son et al., 2004, for details), we speculate
cessing of kinematic and dynamic cues. that configurations of static form and their
Playing the movies backward also signifi- changes over time are more closely associ-
cantly impaired emotion recognition accu- ated with representations of what people
racy, but this effect was only marginally do with their bodies than with how they
greater for the patch-light than for the full- move them, the latter being specified mostly
light displays, providing qualified support by kinematics (see also Giese & Poggio,
for the importance of the sequencing of 2003).
changes in form to judgments of emotions A recent and comprehensive attempt to
from body gestures. Although we cannot extract critical motion and posture features
be certain that our stimulus manipulations for the perception of bodily expressed emo-
completely eliminated all cues other than tions is the work of Giese, Roether, and col-
kinematics, even when in combination, the leagues (Roether et al., 2008, 2009). First,
substantial reduction in emotion classifica- these researchers used motion-capture tech-
tion performance – especially for the spa- nology to record the movements of 25 indi-
tially inverted, reversed patch-light displays viduals as they walked across the recording
206 ANTHONY P. ATKINSON
space after having self-induced one of energy (defined in terms of change in joint
four emotional states (anger, fear, hap- angles over time) and consequently was
piness, and sadness) by recalling a past judged as more emotionally expressive than
situation in which they had experienced the right side (Roether et al., 2008).
that emotion. Emotionally neutral walks
were also recorded before the emotion-
induction procedure. Then, using machine- The Neural Processing of Bodily Form
learning techniques, Roether and colleagues and Motion
extracted the average flexion angles of the
joints (shoulders, elbows, hips, and knees) The form of the human (or primate) body
and the joint-angle trajectories that best is a category of visual object for which there
distinguished among gaits of the differ- appears to be both selectivity and func-
ent emotions (Roether et al., 2009). (This tional specialization in higher level visual
approach is reminiscent of work done with cortices. By selectivity, I mean the extent to
facial expressions of emotion; see Chap- which a mechanism is activated by or oper-
ter 2 and Susskind, Littlewort, Bartlett, ates over a particular stimulus class, such
Movellan, & Anderson, 2007.) Subsequent as faces or bodies, as compared to other
experiments with human observers who stimulus classes. By functional specialization
classified and rated computer avatars ani- (or function for short), I mean a mecha-
mated with the same motion-capture infor- nism’s specificity for performing a particular
mation confirmed a close match between process. Evidence for body-selective visual
the features used by these observers and mechanisms comes from studies of both
the machine-learning algorithm or “ideal humans and nonhuman primates (reviewed
observer” (Roether et al., 2009). For exam- by Peelen & Downing, 2007). In humans,
ple, leg-movement changes and postural the evidence points to two distinct regions,
cues defined in terms of limb flexion were dubbed the extrastriate body area (EBA),
found to be important for the perception of located in the lateral occipitotemporal cor-
anger and fear, whereas head inclination was tex (Downing, Jiang, Shuman, & Kanwisher,
particularly important for the perception of 2001), and the fusiform body area (FBA),
sadness. located in the fusiform gyrus (Peelen &
Speed of gait was another important cue Downing, 2005; Schwarzlose, Baker, & Kan-
that human observers used to distinguish wisher, 2005). The EBA and FBA respond
among the different emotions. Moreover, selectively to human bodies and body parts
comparison of the motion parameters compared with objects, faces, and other con-
between emotionally expressive gaits and trol stimuli, despite considerable anatomi-
velocity-matched neutral gaits showed cal overlap between the FBA and the face-
that quantitative aspects of movement selective fusiform face area (FFA; Peelen &
activity varied independently of gait veloc- Downing, 2005; Schwarzlose et al., 2005)
ities (Roether et al., 2009). For emotions and between the EBA, motion-processing
associated with higher gait velocities there area V5/MT, and object-form-selective lat-
was more movement activity relative to eral occipital complex (Downing, Wiggett,
velocity-matched neutral gaits, particularly & Peelen, 2007). These functional imag-
in the arms, whereas for emotions associ- ing findings are complemented by intracra-
ated with lower gait velocities there was nial recordings in humans (McCarthy, Puce,
less movement activity. Emotion-specific Belger, & Allison, 1999; Pourtois, Peelen,
body postures were found to vary largely Spinelli, Seeck, & Vuilleumier, 2007) and
independently of gait velocity. Finally, single-cell recordings in monkeys (e.g., Desi-
for expressions of anger, happiness, and mone, Albright, Gross, & Bruce, 1984) show-
sadness, the left side of the body was found ing responses specific to bodies or body parts
to contain significantly greater maximum compared to faces, other objects, or com-
joint-angle amplitudes and movement plex shapes.
BODILY EXPRESSIONS OF EMOTION 207
With respect to functional specialization, ies – perhaps the EBA and FBA – in repre-
the EBA represents the static structure of senting the three-dimensional orientation of
viewed bodies (Downing, Peelen, Wiggett, those bodies (Jackson & Blake, 2010).
& Tew, 2006; Peelen, Wiggett, & Downing, A recent study by my colleagues and me
2006), although these representations appear (Atkinson, Vuong, & Smithson, 2012) pro-
to be at the level of individual body parts vides evidence that body-selective EBA and
rather than at the level of the whole-body FBA and face-selective FFA are sensitive to
configuration (Taylor, Wiggett, & Down- specific motion-related cues – particularly
ing, 2007; Urgesi, Calvo-Merino, Haggard, form-from-motion cues – characteristic of
& Aglioti, 2007). As discussed earlier, con- their proprietary stimulus categories, rather
figural cues in body perception include the than to human motion per se (i.e., to motion
relative positions of body parts and the posi- regardless of whether it is from the face or
tions of those parts with respect to the whole body). Using fMRI, we directly contrasted
body (Reed et al., 2006), and there is evi- responses to point-light face and body move-
dence indicating that the processing of one ments, which provides a stronger test of
or other or both of these configural cues selectivity than contrasts against some base-
is more a function of the FBA than of the line stimulus condition such as scrambled
EBA (Taylor et al., 2007). More recent work point-light displays, as performed in previ-
suggests that both the EBA and FBA also ous studies (e.g., Grossman & Blake, 2002;
process body motion cues. Initial reports Peelen et al., 2006). By statistically control-
showed selectivity to whole-body point- ling for differences in perceived emotional
light motion in the posterior inferior tem- intensity based on kinematics, we focused
poral sulcus/middle temporal gyrus (e.g., particularly on the contribution of form-
Grossman & Blake, 2002; Saygin, Wilson, from-motion information.
Hagler, Bates, & Sereno, 2004), which Standard region-of-interest (ROI) anal-
reflected activation of body-selective pop- yses revealed that point-light body move-
ulations of neurons constituting the EBA ments activated body-selective regions in
rather than of motion-selective neuronal the lateral occipitotemporal cortex (right
populations in the overlapping V5/MT (Pee- and left EBA) and fusiform gyrus (right but
len et al., 2006). There are also reports of not left FBA), regardless of whether partici-
selectivity to whole-body movements in the pants were judging the expressed emotion or
fusiform cortex (e.g., Grossman & Blake, the color change of the stimulus dots (Atkin-
2002; Jastorff & Orban, 2009), which reflects son et al., 2012). Point-light face move-
activation of body-selective rather than face- ments activated face-selective FFA bilat-
selective neuronal populations (Peelen et al., erally, although this greater activation to
2006). It has been suggested that these find- point-light faces than to point-light bodies
ings reflect the extraction by the EBA and was evident in the right hemisphere only
FBA of “snapshots” that represent the vari- when participants were explicitly judging
ous static postures comprising a movement the expressed emotion. Voxelwise correla-
sequence (Giese & Poggio, 2003; Peelen tion analyses revealed that, even in bilateral
et al., 2006). However, the findings of a more regions of fusiform cortex containing over-
recent study suggest instead that both the lapping populations of body-selective and
EBA and FBA integrate form and motion face-selective neurons, the patterns of activ-
(kinematic) information (Jastorff & Orban, ity elicited by point-light bodies were pos-
2009). Nonetheless, that study also indicated itively correlated with voxelwise selectiv-
that the EBA has a relatively greater role ity for static bodies but not for static faces
in processing kinematics, whereas the FBA’s (which was also the case in right and left
role is more in processing body configura- EBA), whereas activity elicited by point-
tion (Jastorff & Orban, 2009). Recent behav- light faces was positively correlated with
ioral evidence implicates regions responsive voxelwise selectivity for static faces but not
to both the form and motion of human bod- for static bodies. We further demonstrated
208 ANTHONY P. ATKINSON
enhanced activation of several body- and ferent body parts (for reviews, see Blake &
face-selective regions for happy or angry Shiffrar, 2007; Puce & Perrett, 2003). Dis-
relative to emotionally neutral movements, ruption of the activity of the right poste-
in some regions depending on the task set rior STS using TMS has confirmed a critical
(Atkinson et al., 2012). role for this region in perceiving body move-
The EBA appears to constitute a critical ment (Grossman, Battelli, & Pascual-Leone,
early stage in the perception of other peo- 2005). More recently, a lesion-overlap study
ple (Chan, Peelen, & Downing, 2004), rather with 60 brain-damaged subjects showed that
than a later processing stage via, for exam- impairments in the ability to discriminate
ple, top-down effects related to imaginary whole-body from nonbiological motion in
gestures and movement (de Gelder, 2006). point-light displays were most reliably asso-
Evidence in support of this claim comes ciated with lesions in the posterior temporal
from recent studies using either intracranial and ventral premotor cortices, which corre-
recordings or transcranial magnetic stimula- sponded with the regions whose activity in
tion (TMS). Pourtois et al. (2007) recorded neurologically intact subjects was selective
highly body-selective visual evoked poten- for the same point-light whole-body move-
tials over the EBA of a patient that started ments (Saygin, 2007). The critical involve-
approximately 190 ms and peaked 260 ms ment of the ventral premotor cortex in
after stimulus onset. Consistent with this this study confirms earlier studies show-
finding are reports of selectively impaired ing selectivity in this region for point-light
perception of body form following applica- whole-body movements (e.g., Saygin et al.,
tion of TMS over the EBA at 150–250 ms 2004).
(Urgesi, Berlucchi, & Aglioti, 2004; Urgesi, The distribution of responses in the STS
Calvo-Merino, et al., 2007) and at 150–350 ms and surrounding cortex to the motion of dif-
(Urgesi, Candidi, Ionta, & Aglioti, 2007) ferent body parts suggests a functional orga-
poststimulus onset. Despite this evidence, nization in which distinct but overlapping
however, it is entirely possible that, in addi- patches of the cortex extract body-part spe-
tion to its role in the early visual process- cific representations of biological motion,
ing of body form and motion, the EBA also with a posterior region of the STS, especially
plays a role in later processing stages of per- in the right hemisphere, thereby encoding
son perception. Little is yet known about the a higher level representation of biological
timing of the FBA and ventral premotor cor- motion that is not dependent on the particu-
tex involvement in body and person percep- lar body part generating that motion. There
tion, although given that they preferentially is considerable evidence for an important
represent configural over body-part cues it role for areas of the STS in the integration
is likely that their initial involvement occurs of motion and form (as well as auditory)
subsequent to that of the EBA. Nonetheless, information (e.g., Beauchamp, 2005), espe-
as Taylor et al. (2007) comment, a strictly cially that related to social perception (Puce
serial model is probably too simplistic, given & Perrett, 2003). Thus, as Jastorff and Orban
the widespread bidirectional connectivity in (2009) suggest, the contribution of the pos-
the visual cortex. terior STS is probably particularly impor-
Human lesion, electrophyshiological, and tant when movement complexity or task
neuroimaging studies confirm an important demands require a detailed analysis of the
role for the superior temporal cortex, par- action for which the more basic processing
ticularly its posterior aspects, in the per- in the EBA and FBA of articulated human
ception of body and facial movement com- movement per se is insufficient.
pared to static images of the same body parts There is some debate over whether the
and to nonbiological motion. Neuroimaging posterior STS analyzes local image motion
studies in humans have also revealed dis- and higher level optic flow (e.g., Giese &
tinct regions of the superior temporal sulcus Poggio, 2003) or some more global motion
(STS) selective for the movements of dif- of the whole figure (e.g., Lange & Lappe,
BODILY EXPRESSIONS OF EMOTION 209
2006). Nonetheless, it is possible that the tral premotor cortex, particularly in the left
posterior STS both analyzes local image hemisphere, is known for its role in both
motion and (probably at a later stage) the the planning of motor actions (Johnson &
more global motion information related to Grafton, 2003) and in the visual discrimi-
changes in body and body-part configura- nation of such actions (e.g., Urgesi, Can-
tions over time. Furthermore, it has recently didi, et al., 2007) and has a critical role
been shown that the STS as well as regions in processing configural body cues (Urgesi,
in the inferior parietal lobe and in inferior Calvo-Merino, et al., 2007). Such evidence
frontal gyrus respond to human move- has helped spawn a variety of simulation
ments, regardless of whether those move- accounts of action understanding, accord-
ments comply with or violate normal kine- ing to which observing another perform an
matic laws of motion. Only regions of the action triggers in the observer an offline sim-
dorsal premotor and dorsolateral prefrontal ulation of the viewed action (e.g., Blake-
cortex in the left hemisphere and a region of more & Decety, 2001; Gallese, Keysers, &
the ventromedial prefrontal cortex showed Rizzolatti, 2004).
greater activity to movement complying Work on motor control indicates ways
with kinematic laws of human movement in which such simulations may be com-
(Casile et al., 2010). putationally instantiated, in the form of
So far I have emphasized the use of forward models, inverse models, or both
structural form and motion information in (Grush, 2004; Wolpert, Doya, & Kawato,
the perception and identification of bodies, 2003). Forward models use copies of the
body postures, and movements. There is also motor commands to map the current sen-
likely to be a role for processes that rely on sory states and motor commands to the
visual semantics about body postures and future sensory and motor states that will
movements related to emotion categories as result once the current motor commands
stored in long-term memory (Dittrich, 1991), have been executed. Inverse models per-
especially for symbolic gestures or emblems. form the opposite transformations, by map-
Yet is such an image-processing account, ping sensory representations associated with
or an extension of it, sufficient for explain- the intended action to the motor commands
ing our ability to perceive and understand to execute the action. One suggestion, for
bodily actions? The majority of human pos- example, is that proposed by Wolpert et
tures and movements are not aimless, but al. (2003): When observing another’s action,
are directed toward some purpose or goal the observer’s brain generates a set of motor
and thus reflect the person’s intentions and commands that would be produced given
may also or instead reflect his or her emo- the observed movements and the current
tional and other internal states. Moreover, state of the observed person. Rather than
humans are not passive observers but, like driving the observer’s own motor behav-
the people whose postures and movements ior, these motor commands are used to pre-
they are observing, have intentions and emo- dict the sensory and motor consequences of
tions and act in a purposive, goal-directed the observed action, which are then com-
manner. pared with the observed new state of the
Research and theory over the past decade actor.
or so suggest that one (and perhaps the
main) route to understanding others’ actions
depends on the observer’s own action capa- The Neural Processing of Emotional
bilities. Neuroimaging studies, for exam- Bodies
ple, have shown that ventral premotor and
intraparietal cortices respond with differen- The neural substrates of bodily emotion
tial selectivity to the motion of different perception have only recently begun to be
body parts, in a somatotopic manner (e.g., revealed. This section briefly summarizes
Buccino et al., 2001). In addition, the ven- that work.
210 ANTHONY P. ATKINSON
Right Hemisphere
Inferior Frontal
Gyrus
left right
movements, regardless of task. Yet, although who was previously demonstrated to have
emotional face and particularly body move- impairments recognizing fearful faces (for
ments modulated activity in the fusiform this latter evidence, see, e.g., Adolphs,
gyrus and emotional body movements mod- Tranel, Damasio, & Damasio, 1994). In
ulated activity in the right posterior STS, one of these studies, photographs of dra-
there was no evidence that emotional mod- matic, emotionally charged movie scenes
ulation in these regions occurred in a stim- were altered so that the facial expressions
ulus category-selective manner. Thus, our were not visible. Although this alteration
findings (Atkinson et al., 2012) suggest an significantly reduced normal subjects’ abil-
asymmetry in emotional modulation of neu- ity to recognize all emotions, including fear,
ral responses by body and face motion in from the characters, SM recognized fear nor-
body- and face-selective regions. This asym- mally in a forced-choice paradigm – in fact,
metry constrains the claim that expressive her performance was better for these stimuli
movements modulate neuronal populations than for the intact photographs (Adolphs &
that code for the viewed stimulus category Tranel, 2003). (Note that three other partic-
(Peelen et al., 2007). Further research is ipants with bilateral damage that included
required to test whether these findings the amygdala as well as other medial tempo-
extend to emotional expressions other than ral lobe structures were similarly normal on
of anger and happiness and to fully illumi- both the masked and intact photographs.)
nated displays of faces and bodies, in which In the second study (Atkinson, Heberlein,
static form is visible. et al., 2007), SM was tested with four differ-
So far we have seen imaging evidence for ent whole-body stimulus sets: a set of emo-
a role for the amygdala in processing bod- tional body posture photographs with the
ily expressions of emotion. Although more faces blurred; dynamic stimuli in which the
work is required to uncover the exact nature actors faced forward and expressed emotion
of that and any other roles the amygdala with a full-body gesture; the same stimuli
might play in bodily emotion perception, edited to be patch-light (similar to point-
evidence from lesion studies is not consis- light: Atkinson et al., 2004); and a set of
tent with the amygdala being necessary for emotional point-light walkers in which the
emotional body perception. Sprengelmeyer actors were filmed walking, creeping, danc-
et al. (1999) found impaired fearful posture ing, or otherwise locomoting across the field
recognition in a subject with bilateral amyg- of view (Heberlein et al., 2004; Heberlein &
dala damage, consistent with this individ- Saxe, 2005). Another bilaterally amygdala-
ual’s impaired recognition of fear from faces damaged subject, AP, was tested on two
and vocal expressions. However, this sub- of these sets (the posture photographs and
ject’s lesion was not entirely restricted to the point-light walkers). Both participants
the amygdala, notably including some dam- recognized fear normally, in forced-choice
age to the left thalamus, nor did the lesion tasks, in all the stimulus sets that they judged
encompass the entire extent of both amyg- (Atkinson, Heberlein et al., 2007). These
dalae, the damage to the left amygdala being two findings make it impossible to claim that
incomplete and smaller than that to the the amygdala is necessary for normal recog-
right amygdala. Thus, it is possible that the nition of emotional body movements.
impaired recognition of fear from body pos- Consistent with the data of de Gelder and
tures in this subject was not entirely a con- colleagues (e.g., de Gelder et al., 2004), it
sequence of amygdala damage. may be the case that in intact brains the
Two other studies have demonstrated amygdala serves to associate perceived bod-
normal emotion recognition from sev- ily expressions of fear with relevant motor
eral differing whole-body stimulus sets in plans. By this view, SM – although she is
another subject with bilateral amygdala able to know that the perceived individual is
damage, SM, whose brain damage encom- afraid – would not prepare escape behavior
passed all of the amygdala bilaterally and in response to seeing another person’s fearful
BODILY EXPRESSIONS OF EMOTION 213
bodily expression. Potential future experi- subjects, three separate studies found
ments examining the role of the amygdala that impaired recognition of a range of
in responding to whole-body expressions emotions in static faces (Adolphs, Damasio,
might include an examination of evoked Tranel, Cooper, & Damasio, 2000), prosody
motor responses or motor-related activity in (Adolphs, Damasio, & Tranel, 2002), and
patients with bilateral amygdala damage as body movements represented in point-light
well as further studies of amygdala responses stimuli (Heberlein et al., 2004) correlated
to dynamic and static whole-body emotional best with lesions in the right somatosensory
expressions, ideally with the inclusion of cortex. The region of maximal lesion over-
individual differences measures. lap among impaired patients in these studies
So far, I have been emphasizing the was in the right posterior postcentral gyrus,
roles of configural form cues and kinemat- bordering on the supramarginal gyrus, thus
ics in the recognition of emotions from including not just primary but also more
body movements. Yet, as with action under- posterior secondary somatosensory regions.
standing in general, purely image-processing Impaired emotion recognition performance
accounts may not be sufficient for emo- was also associated with damage to the
tional expression understanding or at any insula and left frontal operculum. Given
rate may not detail the only means by that the lesion method can reveal critical
which we can understand others’ emotional roles for structures only when lesions are
expressions. One way in which we might confined to those structures, it is significant
be able to recognize the emotional state of that in two of these studies (Adolphs et al.,
another is via our perception of an emotional 2002; Heberlein et al., 2004) a small number
response within ourselves (e.g., Adolphs, of people had lesions restricted to the right
2002; Gallese et al., 2004; Heberlein & Atkin- somatosensory cortex and were impaired at
son, 2009). One version of this idea is that recognizing emotions, whereas people with
a visual representation of another’s expres- lesions that spared the right somatosensory
sion leads us to experience what that per- cortex – including damage to nearby motor
son is feeling (i.e., emotional contagion), regions in the postcentral gyrus – tended
which allows us then to infer that per- not to have impaired emotion recognition.
son’s emotional state. That is, the grounds Several studies using functional neuroimag-
for inferring the viewed person’s emotional ing have corroborated these lesion-based
state is knowledge from the “inside;” expe- findings, of which I mention two. (Figure 8.3
riencing the emotion for oneself (even in provides an illustrative example of regions
an attenuated or unconscious form) is an activated when observers judge the emo-
important, perhaps necessary step to mak- tions expressed in body and face stimuli.)
ing accurate judgments about the other’s Winston, O’Doherty, and Dolan (2003)
emotion. A different but conceivably com- found that the activity of the right
patible idea is that coming to know what somatosensory cortices, as well as the ven-
another is feeling involves simulating the tromedial prefrontal cortex (which also
viewed emotional state via the generation represents somatic states), was enhanced
of a somatosensory image of the associated when participants were judging the emo-
body state (Adolphs, 2002) or simulating the tion as compared to the masculinity of
motor programs for producing the viewed faces. Heberlein and Saxe (2005) found that
expression (e.g., Gallese et al., 2004; Leslie, a region at the border of the right post-
Johnson-Frey, & Grafton, 2004). central and supramarginal gyri was more
An important source of evidence for active when subjects made emotion judg-
simulation accounts of emotion recognition ments about point-light walkers (given one
is research showing somatosensory cortex of a known set of emotion words, rating how
involvement in the explicit judgment of well it fit the stimulus) than when they made
emotional expressions. Using lesion overlap personality trait judgments (a comparable
analyses with large groups of brain-damaged task with trait words) based on the same
214 ANTHONY P. ATKINSON
Right Hemisphere
Inferior Frontal
Superior Temporal Somatosensory Gyrus
Sulcus (STS) Cortex
Inferior Frontal
Gyrus
Superior Temporal
Sulcus (STS)
stimuli. Further, a region of interest based judgments in both the lesion and fMRI stud-
on the maximal lesion overlap reported in ies (Heberlein & Saxe, 2005).
Heberlein et al. (2004) – that is, the poste- In addition, three studies using TMS
rior right somatosensory cortex – was signif- have confirmed the critical role of the right
icantly more active for emotion judgments somatosensory cortex in emotion recogni-
as compared to personality trait judgments. tion from faces (Pitcher, Garrido, Walsh, &
A separate region, in the left inferior frontal Duchaine, 2008; Pourtois et al., 2004) and
gyrus, was associated with personality trait voices (van Rijn et al., 2005) in the healthy
BODILY EXPRESSIONS OF EMOTION 215
brain. The important contribution of Pitcher tures activated cortical regions involved in
et al.’s (2008) study was its demonstration of motor planning and action representation,
different critical periods for the involvement relative to emotionally neutral postures,
of the right somatosensory cortex and right including the supplementary motor area and
occipital face area (OFA) in emotion dis- inferior frontal and precentral gyri bilat-
crimination: The right OFA’s involvement erally. They proposed that this enhance-
was pinpointed to a window of 60–100 ms ment of action representation regions by
from stimulus onset, whereas the involve- fearful, but not happy or neutral, static
ment of the right somatosensory cortex was bodies reflected direct engagement of fear-
pinpointed to a later processing stage, 100– related motor plans in the observer, as if the
170 ms from stimulus onset. TMS has not yet motion implied in such images were initi-
been used to confirm whether the involve- ating preparations for flight. Yet although
ment of the right somatosensory cortex is Grèzes, Pichon, and de Gelder (2007) found
critical for neurologically intact individu- that fearful compared to neutral bodies,
als to recognize bodily expressed emotions. regardless of dynamic or static information,
Further studies using TMS on the right and activated a small number of action represen-
left somatosensory regions will help estab- tation regions, the only such region prefer-
lish the role of somatosensory cortices in entially activated by fearful vs. neutral body
emotion recognition based on facial or bod- movements was the right premotor cor-
ily expressions. tex. Grèzes et al. concluded that it remains
Adolphs and Spezio (2006) have pro- unclear whether this increased fear-related
posed a role also for the amygdala in sim- response in the premotor cortex reflects the
ulation accounts of emotion recognition. engagement of motor simulation routines
They suggest that the amygdala modu- as a means of action and emotion under-
lates somatosensory and insula cortices, in standing, rather than the preparation of fear-
an analogous fashion to its modulation of ful motor responses. A more recent study
higher level visual cortices. The amygdala’s by the same group showed that viewing
action might thus increase the sensitivity threat-related (fearful or angry) compared
of somatosensory cortex and insula to sig- to neutral whole-body actions activated a
nals received from the observer’s own body, network of action-related regions, including
enhance the selectivity of these regions to the premotor cortex, regardless of task set or
particular types of input, or reactivate pre- “attentional control” – specifically, whether
viously learnt associations between stim- participants were naming the emotion por-
uli and bodily responses. Although this trayed by the action or the color of a dot
proposal is supported by direct and indi- that appeared briefly during the video clip
rect anatomical connections from the amyg- (Pichon, de Gelder, & Grèzes, 2011; for more
dala to somatosensory cortices and insula on the interactions between emotion and
in the monkey (reviewed by Adolphs & top-down attention, see Chapter 14).
Spezio, 2006), further evidence is required
from neuroimaging and single-cell record-
ing studies that examine functional connec- Conclusion
tivity between these regions when human
observers are required to judge the emotions This chapter has surveyed the burgeoning
expressed in bodies, faces, and voices. body of research devoted to investigating
Finally, neural regions known to be the perception of bodily expressed emo-
involved in motor planning and execution tions and its neural substrate. This review
have also been implicated in the process- has been necessarily brief; complementary
ing of bodily expressions of emotion. For reviews of the visual and neural process-
example, de Gelder et al. (2004) reported ing of bodily expressed emotions are pro-
that fearful but not happy static body pos- vided by de Gelder (2006, 2009) and de
216 ANTHONY P. ATKINSON
Gelder et al. (2010) and of bodies per se by interpreted to mean the same thing, that
Peelen and Downing (2007) and de Gelder someone is fearful or happy for example.
et al. (2010). In this chapter, we have seen Thus when evaluating how someone feels,
that the ability to discriminate or identify our representation of another person’s emo-
different bodily expressed emotions relies tional state must be abstracted away from
on a variety of visual cues: bodily kinemat- the specific sensory input. A recent study
ics, the sequencing of changes in body form provides evidence that two high-level brain
over time, and static form and particularly areas previously implicated in affective pro-
configural form cues. Cortical regions that cessing, mental state attribution, and “the-
extract bodily form and motion information ory of mind” – the medial prefrontal cortex
show enhanced activity to emotional bodies (MPFC) and left superior temporal sulcus
and thus play roles in emotion perception, (STS) – represent perceived emotions at an
although exactly what those roles are is still abstract level, independent of the modality
being investigated. This emotional modula- (body, face, voice) of the expressed emotion
tion of visual regions is facilitated at least in (Peelen, Atkinson, & Vuilleumier, 2010). In
part by feedback from the amygdala, yet the this fMRI study, participants evaluated the
amygdala itself is not necessary for successful intensity of emotions perceived from fully
identification of bodily expressed emotions. illuminated body movements, face move-
The perception of emotional bodies is not ments, or vocal intonations. Using multi-
solely a visual capacity, however. Viewing voxel pattern analysis, Peelen and colleagues
emotional bodies also sometimes engages found stimulus modality-independent but
regions involved in planning and executing emotion category-specific activity patterns
motor actions, suggesting preparation for an in the MPFC and left STS. Multivoxel pat-
appropriate action by the observer or simu- terns in these regions contained information
lation of the observed action to allow it to about the category of the perceived emo-
be understood (or perhaps both). Further- tions (anger, disgust, fear, happiness, sad-
more, explicitly judging expressed emotions ness) across all modality comparisons (face–
engages regions involved in representing body, face–voice, body–voice) and inde-
somatic changes associated with the viewed pendently of the perceived intensity of
emotional states, suggesting that coming to the emotions. These results suggest that
know what another is feeling involves sim- the MPFC and left STS play key roles in
ulating the viewed emotional state via the the understanding and categorization of oth-
generation of a somatosensory image of the ers’ emotional mental states. Future research
associated body state. will need to investigate such issues as
Given that in everyday life we do not rely whether supramodal emotion-specific rep-
solely on one source of cues to other peo- resentations exist for emotions other than
ple’s emotional states, it is important also those tested by Peelen and colleagues and
to consider the perception of multimodal when observers are not explicitly evaluat-
emotion signals (i.e., as expressed in two ing the emotional content of, or attending
or more of the body, face, and voice). On to, the stimuli. Another interesting avenue
this matter the interested reader might like for future research will be to investigate
to begin with Van den Stock, Righart, and whether the MPFC and left STS are similarly
de Gelder (2007). Furthermore, it is impor- activated by perceived and self-experienced
tant to consider that we are able to eval- emotions, particularly given other evidence
uate how someone feels despite the dis- that the MPFC is involved in both the per-
parate sensory nature of emotions signaled ception and experience of emotions (Kober
by the body, face, and voice. Those disparate et al., 2008; Lee & Siegle, 2009). The MPFC
sensory cues – movements and postures might be involved in evaluating others’
of body parts; changes in the size, shape, emotions in virtue of its role in the gen-
and relations between facial features; and eration of the perceiver’s own emotional
acoustic changes in voices – are all typically responses.
BODILY EXPRESSIONS OF EMOTION 217
Outstanding Questions and Future the amygdala modulate the activity of the
Directions somatosensory cortex and insula and thus
increase the sensitivity of these regions to
r How does the brain process the spe- signals received from the observer’s own
cific kinematic and configural form and body, enhance the selectivity of these
motion cues that behavioral research has regions to particular types of input, or
shown observers use to discriminate and reactivate previously learned associations
identify bodily expressed emotions? between stimuli and bodily responses?
r What specific aspects of visual process- r Problems in identifying bodily expressed
ing are enhanced when one views emo- emotions in autism are associated with
tional as compared to emotionally neutral impairments in perceiving global coher-
bodies? Does this emotional modulation ent motion (Atkinson, 2009). Individuals
of visual cortical regions reflect enhanced with autism also show deficient activity
processing of cues specific to bodies or of in and abnormal functional connections
more general visual cues, at least some of between brain regions involved in the
which are contained in body stimuli? visual processing of bodies and in the
r What specific aspects of motor planning generation and understanding of bodily
and action preparation are engaged when actions (e.g., Grèzes, Wicker, Berthoz, &
one views emotional as compared to emo- de Gelder, 2009; Hadjikhani et al., 2009).
tionally neutral bodies? Do all bodily What are the relationships between visual
expressed emotions engage neural regions and attentional impairments in autism
involved in motor planning and action and impaired emotion recognition? How
preparation or just certain specific emo- are these deficits related to individual
tions? Does such activity reflect prepara- differences in autism symptomatology
tion for action appropriate to the viewed and comorbid conditions such as atten-
emotion (e.g., freezing or fleeing in fear tion deficits and low IQ? How are these
when one sees another being fearful or deficits explained by compromised func-
angry) or some offline simulation of the tioning of and connectivity between par-
viewed action, perhaps to aid understand- ticular brain regions? What training tech-
ing of the emotion or action? niques or other interventions might be
r Does the critical involvement of somato- able to ameliorate the effects of such
sensory cortex and insula in emotion deficits on everyday social functioning?
judgments from bodies (and faces) reflect
interoception – specifically, a simulation References
of changes in internal body state asso-
ciated with the viewed emotion – or Adolphs, R. (2002). Recognizing emotion from
proprioception, specifically a simulation facial expressions: Psychological and neuro-
of the viewed bodily movements and logical mechanisms. Behavioral and Cognitive
postures? Neuroscience Reviews, 1, 21–62.
r What are the functional relationships Adolphs, R., Damasio, H., & Tranel, D. (2002).
among the various regions implicated Neural systems for recognition of emotional
in processing bodies per se and emo- prosody: A 3-D lesion study. Emotion, 2,
23–51.
tional bodies in particular, including the
Adolphs, R., Damasio, H., Tranel, D., Cooper,
body-selective EBA and FBA, STS, amyg- G., & Damasio, A. R. (2000). A role for
dala, somatosensory cortex, insula, and somatosensory cortices in the visual recog-
regions of frontal cortex? For example, nition of emotion as revealed by three-
does the amygdala function to associate dimensional lesion mapping. Journal of Neu-
perceived bodily expressions of emotions roscience, 20, 2683–90.
such as fear with relevant motor plans, in Adolphs, R., & Spezio, M. (2006). Role of the
order to facilitate action preparation or amygdala in processing visual social stimuli.
action/emotion understanding? Or does Progress in Brain Research, 156, 363–78.
218 ANTHONY P. ATKINSON
Adolphs, R., & Tranel, D. (2003). Amygdala dam- Bertenthal, B. I., & Pinto, J. (1994). Global pro-
age impairs emotion recognition from scenes cessing of biological motions. Psychological Sci-
only when they contain facial expressions. ence, 5, 221–25.
Neuropsychologia, 41, 1281–89. Blake, R., & Shiffrar, M. (2007). Perception of
Adolphs, R., Tranel, D., Damasio, H., & Dama- human motion. Annual Review of Psychology,
sio, A. (1994). Impaired recognition of emo- 58, 47–73.
tion in facial expressions following bilateral Blakemore, S. J., & Decety, J. (2001). From the
damage to the human amygdala. Nature, 372, perception of action to the understanding of
669–72. intention. Nature Reviews Neuroscience, 2, 561–
Aronoff, J., Woike, B. A., & Hyman, L. M. (1992). 67.
Which are the stimuli in facial displays of Buccino, G., Binkofski, F., Fink, G. R., Fadiga, L.,
anger and happiness? Configurational bases Fogassi, L., Gallese, V., et al. (2001). Action
of emotion recognition. Journal of Personality observation activates premotor and parietal
and Social Psychology, 62, 1050–66. areas in a somatotopic manner: An fMRI
Atkinson, A. P. (2009). Impaired recognition of study. European Journal of Neuroscience, 13,
emotions from body movements is associated 400–4.
with elevated motion coherence thresholds in Buck, R. (1984). The communication of emotion.
autism spectrum disorders. Neuropsychologia, New York: Guilford Press.
47, 3023–29. Buck, R. (1994). Social and emotional functions
Atkinson, A. P., Dittrich, W. H., Gemmell, A. J., in facial expression and communication: The
& Young, A. W. (2004). Emotion perception readout hypothesis. Biological Psychology, 38,
from dynamic and static body expressions in 95–115.
point-light and full-light displays. Perception, Casile, A., Dayan, E., Caggiano, V., Hendler,
33, 717–46. T., Flash, T., & Giese, M. A. (2010). Neu-
Atkinson, A. P., Heberlein, A. S., & Adolphs, R. ronal encoding of human kinematic invariants
(2007). Spared ability to recognise fear from during action observation. Cerebral Cortex, 20,
static and moving whole-body cues following 1647–55.
bilateral amygdala damage. Neuropsychologia, Casile, A., & Giese, M. A. (2005). Critical fea-
45, 2772–82. tures for the recognition of biological motion.
Atkinson, A. P., Tunstall, M. L., & Dittrich, Journal of Vision, 5, 348–60.
W. H. (2007). Evidence for distinct contribu- Chan, A. W., Peelen, M. V., & Downing, P. E.
tions of form and motion information to the (2004). The effect of viewpoint on body rep-
recognition of emotions from body gestures. resentation in the extrastriate body area. Neu-
Cognition, 104, 59–72. roreport, 15, 2407–10.
Atkinson, A. P., Vuong, Q. C., & Smithson, Coulson, M. (2004). Attributing emotion to static
H. E. (2012). Modulation of the face- and body- body postures: Recognition accuracy, confu-
selective visual regions by the motion and sions, and viewpoint dependence. Journal of
emotion of point-light face and body stimuli. Nonverbal Behavior, 28, 117–39.
Neuroimage, 59, 1700–12. Darwin, C. (1998). The expression of the emotions
Barclay, C. D., Cutting, J. E., & Kozlowski, in man and animals (3rd ed.). London: Harper
L. T. (1978). Temporal and spatial factors in Collins. (Original work published in 1872)
gait perception that influence gender recogni- de Gelder, B. (2006). Towards the neurobiology
tion. Perception and Psychophysics, 23, 145–52. of emotional body language. Nature Reviews
Bassili, J. N. (1978). Facial motion in the per- Neuroscience, 7, 242–49.
ception of faces and of emotional expression. de Gelder, B. (2009). Why bodies? Twelve rea-
Journal of Experimental Psychology: Human sons for including bodily expressions in affec-
Perception and Performance, 4, 373–79. tive neuroscience. Philosophical Transactions
Beauchamp, M. S. (2005). See me, hear me, of the Royal Society B: Biological Sciences, 364,
touch me: Multisensory integration in lateral 3475–84.
occipital-temporal cortex. Current Opinion in de Gelder, B., Snyder, J., Greve, D., Gerard, G.,
Neurobiology, 15, 145–53. & Hadjikhani, N. (2004). Fear fosters flight: A
Beintema, J. A., & Lappe, M. (2002). Percep- mechanism for fear contagion when perceiv-
tion of biological motion without local image ing emotion expressed by a whole body. Pro-
motion. Proceedings of the National Academy ceedings of the National Academy of Sciences,
of Sciences, 99, 5661–63. 101, 16701–6.
BODILY EXPRESSIONS OF EMOTION 219
de Gelder, B., Van den Stock, J., Meeren, H. K. Grèzes, J., Pichon, S., & de Gelder, B. (2007).
M., Sinke, C. B. A., Kret, M. E., & Tamietto, Perceiving fear in dynamic body expressions.
M. (2010). Standing up for the body: Recent Neuroimage, 35, 959–67.
progress in uncovering the networks involved Grèzes, J., Wicker, B., Berthoz, S., & de Gelder,
in the perception of bodies and bodily expres- B. (2009). A failure to grasp the affective
sions. Neuroscience & Biobehavioral Reviews, meaning of actions in autism spectrum dis-
34, 513–27. order subjects. Neuropsychologia, 47, 1816–25.
de Meijer, M. (1989). The contribution of general Grossman, E. D., Battelli, L., & Pascual-Leone,
features of body movement to the attribution A. (2005). Repetitive TMS over posterior
of emotions. Journal of Nonverbal Behavior, 13, STS disrupts perception of biological motion.
247–68. Vision Research, 45, 2847–53.
Desimone, R., Albright, T. D., Gross, C. G., & Grossman, E. D., & Blake, R. (2001). Brain activity
Bruce, C. (1984). Stimulus-selective properties evoked by inverted and imagined biological
of inferior temporal neurons in the macaque. motion. Vision Research, 41, 1475–82.
Journal of Neuroscience, 4, 2051–62. Grossman, E. D., & Blake, R. (2002). Brain areas
Diamond, R., & Carey, S. (1986). Why faces are active during visual perception of biological
and are not special: An effect of expertise. motion. Neuron, 35, 1167–75.
Journal of Experimental Psychology: General, Grush, R. (2004). The emulation theory of repre-
115, 107–17. sentation: motor control, imagery, and per-
Dittrich, W. H. (1991). Das Erkennen von ception. Behavioral and Brain Sciences, 27,
Emotionen aus Ausdrucksbewegungen des 377–96; discussion 396–442.
Gesichts. Psychologische Beitrage, 33, 366–77. Hadjikhani, N., & de Gelder, B. (2003). Seeing
Dittrich, W. H., Troscianko, T., Lea, S., & Mor- fearful body expressions activates the fusiform
gan, D. (1996). Perception of emotion from cortex and amygdala. Current Biology, 13, 2201–
dynamic point-light displays represented in 5.
dance. Perception, 25, 727–38. Hadjikhani, N., Joseph, R. M., Manoach, D.
Downing, P. E., Jiang, Y., Shuman, M., & Kan- S., Naik, P., Snyder, J., Dominick, K., et al.
wisher, N. (2001). A cortical area selective for (2009). Body expressions of emotion do not
visual processing of the human body. Science, trigger fear contagion in autism spectrum dis-
293, 2470–73. order. Social Cognitive and Affective Neuro-
Downing, P. E., Peelen, M. V., Wiggett, A. J., science, 4, 70–78.
& Tew, B. D. (2006). The role of the extras- Heberlein, A. S., Adolphs, R., Tranel, D., &
triate body area in action perception. Social Damasio, H. (2004). Cortical regions for judg-
Neuroscience, 1, 52–62. ments of emotions and personality traits from
Downing, P. E., Wiggett, A. J., & Peelen, M. V. point-light walkers. Journal of Cognitive Neu-
(2007). Functional magnetic resonance imag- roscience, 16, 1143–58.
ing investigation of overlapping lateral occipi- Heberlein, A. S., & Atkinson, A. P. (2009). Neu-
totemporal activations using multi-voxel pat- roscientific evidence for simulation and shared
tern analysis. Journal of Neuroscience, 27, 226– substrates in emotion recognition: Beyond
33. faces. Emotion Review, 1, 162–77.
Ekman, P. (1992). An argument for basic emo- Heberlein, A. S., & Saxe, R. R. (2005). Dissoci-
tions. Cognition & Emotion, 6, 169–200. ation between emotion and personality judg-
Ekman, P., & Friesen, W. (1969). The reper- ments: Convergent evidence from functional
toire of nonverbal behavior: Categories, neuroimaging. Neuroimage, 28, 770–77.
origins, usage, and coding. Semiotica, 1, Hejmadi, A., Davidson, R. J., & Rozin, P. (2000).
49–98. Exploring Hindu Indian emotion expressions:
Fridlund, A. J. (1991). Evolution and facial action Evidence for accurate recognition by Ameri-
in reflex, social motive, and paralanguage. Bio- cans and Indians. Psychological Science, 11, 183–
logical Psychology, 32, 3–100. 87.
Gallese, V., Keysers, C., & Rizzolatti, G. (2004). Hill, H., Jinno, Y., & Johnston, A. (2003). Com-
A unifying view of the basis of social cognition. paring solid-body with point-light animations.
Trends in Cognitive Sciences, 8, 396–403. Perception, 32, 561–66.
Giese, M. A., & Poggio, T. (2003). Neural mech- Hubert, B., Wicker, B., Moore, D. G., Mon-
anisms for the recognition of biological move- fardini, E., Duverger, H., Da Fonseca, D.,
ments. Nature Reviews Neuroscience, 4, 179–92. et al. (2007). Recognition of emotional and
220 ANTHONY P. ATKINSON
non-emotional biological motion in individu- Proceedings of the Royal Society of London Series
als with autistic spectrum disorders. Journal of B: Biological Sciences, 249, 149–55.
Autism and Developmental Disorders, 37, 1386– McCarthy, G., Puce, A., Belger, A., & Allison, T.
92. (1999). Electrophysiological studies of human
Jackson, S., & Blake, R. (2010). Neural integra- face perception. II: Response properties of
tion of information specifying human struc- face-specific potentials generated in occipi-
ture from form, motion, and depth. Journal of totemporal cortex. Cerebral Cortex, 9, 431–44.
Neuroscience, 30, 838–48. McLeod, P., Dittrich, W., Driver, J., Perrett,
James, W. (1932). A study of the expression of D., & Zihl, J. (1996). Preserved and impaired
bodily posture. Journal of General Psychology, detection of structure from motion by a
7, 405–36. “motion-blind” patient. Visual Cognition, 3,
Jastorff, J., & Orban, G. A. (2009). Human func- 363–91.
tional magnetic resonance imaging reveals Minnebusch, D. A., Suchan, B., & Daum, I.
separation and integration of shape and (2009). Losing your head: Behavioral and
motion cues in biological motion processing. electrophysiological effects of body inversion.
Journal of Neuroscience, 29, 7315–29. Journal of Cognitive Neuroscience, 21, 865.
Johansson, G. (1973). Visual perception of bio- Montepare, J., Goldstein, S. B., & Clausen, A.
logical motion and a model for its analysis. (1987). The identification of emotions from
Perception and Psychophysics, 14, 201–11. gait information. Journal of Nonverbal Behav-
Johnson, S. H., & Grafton, S. T. (2003). From ior, 11, 33–42.
‘acting on’ to ‘acting with’: The functional Moore, D. G., Hobson, R. P., & Lee, A. (1997).
anatomy of object-oriented action schemata. Components of person perception: An investi-
Progress in Brain Research, 142, 127–39. gation with autistic, non-autistic retarded and
Kober, H., Barrett, L. F., Joseph, J., Bliss-Moreau, typically developing children and adolescents.
E., Lindquist, K., & Wager, T. D. (2008). Func- British Journal of Developmental Psychology, 15,
tional grouping and cortical-subcortical inter- 401–23.
actions in emotion: A meta-analysis of neu- Pavlova, M., Lutzenberger, W., Sokolov, A.,
roimaging studies. Neuroimage, 42, 998–1031. & Birbaumer, N. (2004). Dissociable cor-
Laban, R., & Ullmann, L. (1988). The mastery tical processing of recognizable and non-
of movement (4th ed.). Plymouth, England: recognizable biological movement: Analysing
Northcote House. gamma MEG activity. Cerebral Cortex, 14, 181–
Lange, J., & Lappe, M. (2006). A model of biolog- 88.
ical motion perception from configural form Peelen, M. V., Atkinson, A. P., Andersson, F., &
cues. Journal of Neuroscience, 26, 2894–2906. Vuilleumier, P. (2007). Emotional modulation
Lee, K. H., & Siegle, G. J. (2009). Common and of body-selective visual areas. Social Cognitive
distinct brain networks underlying explicit and Affective Neuroscience, 2, 274–83.
emotional evaluation: A meta-analytic study. Peelen, M. V., Atkinson, A. P., & Vuilleumier,
Social Cognitive and Affective Neuroscience. P. (2010). Supramodal representations of per-
doi: 10.1093/scan/nsp001 ceived emotions in the human brain. Journal
Leslie, K. R., Johnson-Frey, S. H., & Grafton, of Neuroscience, 30, 10127–34.
S. T. (2004). Functional imaging of face and Peelen, M. V., & Downing, P. E. (2005). Selectiv-
hand imitation: Towards a motor theory of ity for the human body in the fusiform gyrus.
empathy. Neuroimage, 21, 601–7. Journal of Neurophysiology, 93, 603–8.
Loula, F., Prasad, S., Harber, K., & Shiffrar, Peelen, M. V., & Downing, P. E. (2007). The
M. (2005). Recognizing people from their neural basis of visual body perception. Nature
movement. Journal of Experimental Psychol- Reviews Neuroscience, 8, 636–48.
ogy: Human Perception and Performance, 31, Peelen, M. V., Wiggett, A. J., & Downing, P.
210–20. E. (2006). Patterns of fMRI activity dissoci-
Mather, G., & Murdoch, L. (1994). Gender ate overlapping functional brain areas that
discrimination in biological motion displays respond to biological motion. Neuron, 49, 815–
based on dynamic cues. Proceedings of the 22.
Royal Society of London, Series B: Biological Sci- Pessoa, L., McKenna, M., Gutierrez, E., & Unger-
ences, 258, 273–79. leider, L. G. (2002). Neural processing of emo-
Mather, G., Radford, K., & West, S. (1992). Low- tional faces requires attention. Proceedings of
level visual processing of biological motion. the National Academy of Sciences, 99, 11458–63.
BODILY EXPRESSIONS OF EMOTION 221
Pichon, S., de Gelder, B., & Grèzes, J. (2011). Saygin, A. P. (2007). Superior temporal and
Threat prompts defensive brain responses premotor brain areas necessary for biological
independently of attentional control. Cerebral motion perception. Brain, 130, 2452–61.
Cortex. doi: 10.1093/cercor/bhr060 Saygin, A. P., Wilson, S. M., Hagler, D. J., Bates,
Pinto, J., & Shiffrar, M. (1999). Subconfigurations E., & Sereno, M. I. (2004). Point-light biolog-
of the human form in the perception of bio- ical motion perception activates human pre-
logical motion displays. Acta Psychologica, 102, motor cortex. Journal of Neuroscience, 24, 6181–
293–318. 88.
Pitcher, D., Garrido, L., Walsh, V., & Duchaine, Schwarzlose, R. F., Baker, C. I., & Kanwisher, N.
B. (2008). TMS disrupts the perception and (2005). Separate face and body selectivity on
embodiment of facial expressions. Journal of the fusiform gyrus. Journal of Neuroscience, 25,
Vision, 8, 700. 11055–59.
Poizner, H. (1981). Visual and “phonetic” coding Shallice, T. (1988). From neuropsychology to mental
of movement: Evidence from American Sign structure. Cambridge: Cambridge University
Language. Science, 212, 691–93. Press.
Pollick, F. E., Paterson, H. M., Bruderlin, A., & Shipley, T. F. (2003). The effect of object and
Sanford, A. J. (2001). Perceiving affect from event orientation on perception of biological
arm movement. Cognition, 82, B51–61. motion. Psychological Science, 14, 377–80.
Pourtois, G., Peelen, M. V., Spinelli, L., Seeck, Sprengelmeyer, R., Young, A. W., Schroeder, U.,
M., & Vuilleumier, P. (2007). Direct intracra- Grossenbacher, P. G., Federlein, J., Buttner,
nial recording of body-selective responses in T., et al. (1999). Knowing no fear. Proceedings
human extrastriate visual cortex. Neuropsy- of the Royal Society of London. Series B: Biolog-
chologia, 45, 2621–25. ical Sciences, 266, 2451–56.
Pourtois, G., Sander, D., Andres, M., Grandjean, Stanislavski, K. (1936). An actor prepares (E. R.
D., Reveret, L., Olivier, E., et al. (2004). Dis- Hopgood, Trans.). New York: Theatre Arts
sociable roles of the human somatosensory Books.
and superior temporal cortices for processing Stekelenburg, J. J., & de Gelder, B. (2004). The
social face signals. European Journal of Neuro- neural correlates of perceiving human bodies:
science, 20, 3507–15. An ERP study on the body-inversion effect.
Puce, A., & Perrett, D. (2003). Electrophysiol- Neuroreport, 15, 777–80.
ogy and brain imaging of biological motion. Sugase, Y., Yamane, S., Ueno, S., & Kawano, K.
Philosophical Transactions of the Royal Society (1999). Global and fine information coded by
of London, Series B: Biological Sciences, 358, single neurons in the temporal visual cortex.
435–45. Nature, 400, 869–73.
Reed, C. L., Stone, V. E., Grubb, J. D., & Susskind, J. M., Littlewort, G., Bartlett,
McGoldrick, J. E. (2006). Turning configural M. S., Movellan, J., & Anderson, A. K. (2007).
processing upside down: Part and whole body Human and computer recognition of facial
postures. Journal of Experimental Psychology: expressions of emotion. Neuropsychologia, 45,
Human Perception and Performance, 32, 73–87. 152–62.
Roether, C. L., Omlor, L., Christensen, A., & Taylor, J. C., Wiggett, A. J., & Downing, P. E.
Giese, M. A. (2009). Critical features for the (2007). Functional MRI analysis of body and
perception of emotion from gait. Journal of body part representations in the extrastriate
Vision, 9, 1–32. and fusiform body areas. Journal of Neurophys-
Roether, C. L., Omlor, L., & Giese, M. A. (2008). iology, 98, 1626–33.
Lateral asymmetry of bodily emotion expres- Thurman, S. M., Giese, M. A., & Grossman, E. D.
sion. Current Biology, 18, R329–R330. (2010). Perceptual and computational analy-
Runeson, S., & Frykholm, G. (1981). Visual per- sis of critical features for biological motion.
ception of lifted weight. Journal of Experimen- Journal of Vision, 10(12):15, 1-14. doi: 10.1167/10.
tal Psychology: Human Perception and Perfor- 12.15
mance, 7, 733–40. Troje, N. F. (2002). Decomposing biological
Sawada, M., Suda, K., & Ishii, M. (2003). motion: A framework for analysis and synthe-
Expression of emotions in dance: Relation sis of human gait patterns. Journal of Vision, 2,
between arm movement characteristics and 371–87.
emotion. Perceptual and Motor Skills, 97, Urgesi, C., Berlucchi, G., & Aglioti, S. M. (2004).
697–708. Magnetic stimulation of extrastriate body area
222 ANTHONY P. ATKINSON
impairs visual processing of nonfacial body Vuilleumier, P., & Driver, J. (2007). Modulation
parts. Current Biology, 14, 2130–34. of visual processing by attention and emo-
Urgesi, C., Calvo-Merino, B., Haggard, P., & tion: Windows on causal interactions between
Aglioti, S. M. (2007). Transcranial magnetic human brain regions. Philosophical Transac-
stimulation reveals two cortical pathways for tions of the Royal Society B: Biological Sciences,
visual body processing. Journal of Neuro- 362, 837–55.
science, 27, 8023–30. Vuilleumier, P., Richardson, M. P., Armony, J.
Urgesi, C., Candidi, M., Ionta, S., & Aglioti, S. M. L., Driver, J., & Dolan, R. J. (2004). Distant
(2007). Representation of body identity and influences of amygdala lesion on visual cortical
body actions in extrastriate body area and ven- activation during emotional face processing.
tral premotor cortex. Nature Neuroscience, 10, Nature Neuroscience, 7, 1271–78.
30–1. Wallbott, H. G. (1998). Bodily expression of emo-
Vaina, L. M., Cowey, A., LeMay, M., Bienfang, tion. European Journal of Social Psychology, 28,
D. C., & Kikinis, R. (2002). Visual deficits in 879–96.
a patient with ‘kaleidoscopic disintegration of Westhoff, C., & Troje, N. F. (2007). Kinematic
the visual world’. European Journal of Neurol- cues for person identification from biological
ogy, 9, 463–77. motion. Perception & Psychophysics, 69, 241–53.
Van den Stock, J., Righart, R., & de Gelder, B. Winston, J. S., O’Doherty, J., & Dolan, R. J.
(2007). Body expressions influence recognition (2003). Common and distinct neural responses
of emotions in the face and voice. Emotion, 7, during direct and incidental processing of mul-
487–94. tiple facial emotions. Neuroimage, 20, 84–97.
van Rijn, S., Aleman, A., van Diessen, E., Berck- Wolpert, D. M., Doya, K., & Kawato, M.
moes, C., Vingerhoets, G., & Kahn, R. S. (2003). A unifying computational framework
(2005). What is said or how it is said makes for motor control and social interaction. Philo-
a difference: Role of the right fronto-parietal sophical Transactions of the Royal Society B:
operculum in emotional prosody as revealed Biological Sciences, 358, 593–602.
by repetitive TMS. European Journal of Neu- Yovel, G., Pelc, T., & Lubetzky, I. (2010). It’s
roscience, 21, 3195–3200. all in your head: Why is the body inversion
Vuilleumier, P. (2005). How brains beware: Neu- effect abolished for headless bodies? Journal
ral mechanisms of emotional attention. Trends of Experimental Psychology: Human Perception
in Cognitive Sciences, 9, 585–94. and Performance, 36, 759–67.
CHAPTER 9
Pierre Rainville
The somatosensory system is at the heart (e.g., Craig, 2002, 2009; Damasio, 1999; Met-
of emotion. First and foremost, this sen- zinger, 2000).
sory system provides the most immedi- Although the somatosensory system pro-
ate and direct information to the cen- vides the most basic form of emotional
tral nervous system (CNS) about potential inducers, somatosensory inputs are not
shifts toward or away from biophysio- distinctly perceived as pertaining to the
logical stability (homeostasis/allostasis); it self, because some can also be considered
thereby constitutes the fundamental mon- exteroceptive signals. Interoceptive percep-
itoring component essential for the inte- tion refers to somatosensory experiences
grated neural regulation of basic physio- attributed to the state or function of the
logical functions (e.g., Craig, 2003). This body itself and constitutes a fundamental
integrated regulation is especially critical component of self-representation. Accord-
when local physiological processes are insuf- ing to this general conception, interocep-
ficient to respond adequately to threatening tion may include not only visceral sensa-
conditions that require the mobilization of tion but also all sensory signals conveying
additional resources involving coordinated information about the state of the body,
emotional systems ranging from low-level including the skin, the musculoskeletal sys-
motor and autonomic responses to higher tem, and humoral state. In contrast, extero-
order brain processes. Furthermore, from ceptive somatosensory perception refers to
psychological and experiential perspectives, experiences attributed to external objects
whereas signals originating from other sen- that are in direct contact with the body.
sory modalities are normally perceived as In the somatosensory system, exteroceptive
properties of external objects, signals pro- information is typically conveyed by compo-
cessed in the somatosensory system provide nents of the skin senses (e.g., low-threshold
information about the body itself, a core mechanoreceptors), as well as other sub-
component of self-representation in influ- systems (e.g., proprioceptive) involved in
ential theories of emotion and consciousness body representation (i.e., the position of the
223
224 PIERRE RAINVILLE
fingers inform about the shape of an object ant sensory and emotional experience asso-
held in one’s hand). Although they are most ciated with actual or potential tissue dam-
proximal to the body, these external objects age, or described in terms of such dam-
do not have intrinsic properties sufficient to age” (Merskey & Spear, 1967). Price (1999,
be considered primordial emotional induc- pp. 1–2) has proposed a slightly different def-
ers because their emotional impact is depen- inition in which pain is “a somatic percep-
dent on the interpretation of their potential tion containing (1) a bodily sensation with
effects on the body-self. qualities like those reported during tissue
Although there is no strict separation damaging stimulation, (2) an experienced
between interoceptive and exteroceptive threat associated with this sensation, and (3)
pathways in the somatosensory system, a feeling of unpleasantness or other negative
exteroceptive information is generally asso- emotions based on this experienced threat.”
ciated with lemniscal function, whereas In both definitions, negative emotions, at
the multiple extra-lemniscal pathways con- least in their most basic expression, are a
tribute primarily, but not exclusively, to constituent of the pain experience. Impor-
interoceptive function (for a major excep- tantly, this definition distinguishes pain from
tion involving visceral input from the lower nociception, with the latter notion referring
abdomen transmitted through the dorsal to the biological processes associated with
column, see Willis, Al-Chaer, Quast, & tissue damage. Of course, pain may result
Westlund, 1999). In this respect, acute from nociception (i.e., nociceptive pain),
cutaneous pain is particularly interesting but in several conditions, pain may be expe-
because the sensory signals convey infor- rienced without evidence of tissue damage
mation about both the state of the body and nociception may be observed without
(i.e., actual or potential tissue damage) and resulting in a pain experience.
the properties of the nociceptive stimulus. Price (1999) has further described stages
Experientially, this information translates of pain processing that distinguish between
into experiences of states of the self (“I” am primary pain affect and secondary emotions.
in pain) and perceived properties of external The primary affective stage refers to the
objects (“this burner” is hot). immediate unpleasantness that is integral to
In addition to emotional induction, the the pain experience and is intimately related
somatosensory system also provides key to the sense of immediate threat. The second
information about the emotional responses stage is characterized by emotions related to
elicited in the body. Somatovisceral and the broader meaning of pain and the eval-
skeletomotor activation are fundamental uation of the future consequences of pain.
constituents of emotional responses, and Although the sensory experience and the
sensory feedback from the body is central first stage of pain affect are necessary and
to classical theories of emotions (Dama- sufficient for an experience to be character-
sio, 1994, 1996; James, 1994; see Chap- ized as painful, the emotions associated with
ter 1). Somatosensory feedback may also the secondary stage of pain complement the
contribute to self-perception of emotional experience in relation to its broader signifi-
states and provides meaningful psychophys- cance and future implications.
iological information relevant to cogni-
tive processes, particularly decision making
(Bechara, 2004; Bechara, Damasio, Tranel, & Is Pain an Emotion?
Damasio, 1997).
There are several features that charac-
terize emotions and are given different
Nociception and Pain weights in theories of emotions (see Chap-
ter 1). Emotions are triggered by objects
The International Association for the Study or events that are immediately present or
of Pain (IASP) defines pain as a “an unpleas- evoked mentally (“emotion inducer”). The
PAIN AND THE EMOTIONAL RESPONSES TO NOXIOUS STIMULI 225
Figure 9.1. Hierarchical organization of the nociceptive system. The stimulation of peripheral
nociceptors activates spinal neurons and ascending pathways that transmit the nociceptive signal to
multiple targets at the bulbar, reticular, mesencephalic, diencephalic, and telencephalic level. At each
of those levels (1 to 6), new integration processes (1 to 6 ) are possible that can be indexed by output
measures and/or modulatory effects (right column). In addition to the ascending pathways,
interactions between levels are possible through ascending and descending connections between levels
(curved arrows).
Figure 9.2. This study shows very clearly that control. In contrast, the insular cortex has
the spinothalamocortical ascending system been associated with autonomic and home-
targets the primary (SI) and secondary (SII) ostatic regulation (Augustine, 1996; Craig,
somatosensory areas, the insular cortex, and 2003). Although all cortical targets receive
the anterior cingulate cortex (ACC). Impor- information about the intensity of the nox-
tantly, this study further demonstrates that ious input, the somatosensory areas, includ-
the primary target territory in the ACC cor- ing part of the insula, have been associ-
responds to the caudal part of the supra- ated more specifically, although perhaps not
callosal ACC, also associated with motor exclusively, with the conscious perception
Figure 9.2. Spinothalamocortical pathways (left) convey the nociceptive signal from the dorsal horn
of the spinal cord to four thalamic nuclei and four target cortical areas (Dum et al., 2009).
Thalamocortical activation is consistently observed during acute pain as demonstrated by the
activation likelihood estimation (ALE) map (right) showing highly significant responses found across
117 brain imaging studies in healthy individuals published before 2008 (Duerden et al., 2008).
PAIN AND THE EMOTIONAL RESPONSES TO NOXIOUS STIMULI 227
and memory of the sensory properties of lum, hypothalamus, and brainstem, includ-
pain (spatiotemporal and intensity) and ing the region of the mesencephalic PAG.
with sensory-discriminative functions (e.g., The relation between the activity in these
Albanese, Duerden, Rainville, & Duncan, various areas and the level of pain felt by the
2007; Hofbauer, Rainville, Duncan, & 2001; participants was also examined using vari-
Ploner, Freund, & Schnitzler, 1999). ous experimental approaches. These obser-
In addition to those more classical pain vations generally supported the notion of
pathways, the prefrontal cortex is also a a “pain (neuro)matrix,” a distributed cere-
target of ascending projections activated via bral network in which activity is associ-
additional relay nuclei (thalamus, nucleus ated with pain. However, this idea, fun-
parabrachial, and amygdala; e.g., Bernard & damental to the pioneer theorization of
Villanueva, 2009; Bourgeais, Monconduit, Melzack (1990), raised important questions
Villanueva, & Bernard, 2001; Monconduit, about the relation of identity between pain
Bourgeais, Bernard, Le Bars, & Villanueva, and the activity in this cerebral network.
1999). Important anatomical connections It recently nourished an animated debate
are also present between the amygdala, the on the specificity of pain-related brain acti-
insula (Augustine, 1996), and the prefrontal vation.
cortices (Sah, Faber, Lopez de, & Power,
2003) that could play an important part in
Sensitivity and Specificity of
the regulation of pain responses (e.g., Ji
Pain-Related Brain Activity
et al., 2010). In summary, the nociceptive
transmission channels involve disynaptic The sensitivity of a test corresponds to
spinothalamocortical pathways that are its capacity to detect the presence of a
generally associated with the perception of given condition, as determined by a univer-
pain sensation and pain affective responses, sally recognized criterion (i.e., the gold stan-
as well as several multisynaptic pathways dard). In the context of pain neuroimagery,
that include additional sites of integration this measurement of reference generally
at the bulbar, reticular, mesencephalic, and corresponds to subjective reports of pain,
diencephalic levels, before reaching various consistent with the experiential emphasis
cortical territories. put on pain phenomena by the IASP. A
coordinate-based meta-analysis carried out
on the results of 117 pain imaging studies
Functional Brain Imaging of Acute Pain published from 1991 to 2008 confirmed the
reliable activation of the thalamus, as well as
The results of human functional neuroimag- the SI, SII, insula, and ACC (Duerden, Fu,
ing studies have been compiled by Apkar- Rainville, & Duncan, 2008; see Figure 9.2).
ian in an important review article (Apkar- Other sites where the probability of ob-
ian, Bushnell, Treede, & Zubieta, 2005). This serving1 an activation exceeded chance were
article describes the activations induced by several prefrontal and subcortical sites, par-
the application of acute noxious stimula- ticularly in the putamen and the cerebel-
tions (e.g. thermal, mechanical, and so on) in lum. This coordinate-based meta-analysis
each cortical area targeted by the spinotha- confirmed that activation sites in separate
lamic pathways, including SI, SII, the insula, studies were likely superimposed spatially
and the ACC. Activations were also noted on the target structures. Results also con-
in the prefrontal cortex, the motor and firmed the greatest sensitivity of the insula
premotor areas (including the supplemen- and the ACC to the presence of acute pain.
tary motor areas, near the caudal ACC), However, the observation of high sensitiv-
and the posterior parietal cortex. Subcor- ity to the presence of pain is insufficient to
tical activation was also reported in some conclude that there is a “relation of iden-
studies in the thalamus and the basal ganglia tity” between these activations and pain
and, more rarely, in the amygdala, cerebel- experiences.
228 PIERRE RAINVILLE
Although the unique nature of painful pain reflects the activation of a functional
experiences is generally recognized, as com- network involved in the detection and the
pared to other sensory or emotional expe- evaluation of biologically or psychologically
riences, it is not completely clear that it relevant sensory information. Activation of
results in a unique and distinctive pattern this network may further reflect the sponta-
of brain activation in functional brain imag- neous mobilization of higher order resources
ing studies. Brain areas activated by painful to respond adaptively to biologically and
stimuli receive spinothalamocortical inputs psychologically relevant inputs (Shackman
(Dum et al., 2009) and contain nociceptive et al., 2011). Thus, the notion of a pain
neurons (e.g., Kenshalo, Iwata, Sholas, & matrix should refer to functional systems
Thomas, 2000; Shyu, Sikes, Vogt, & Vogt, reliably involved in pain (i.e., high sensi-
2010; Treede, Kenshalo, Gracely, & Jones, tivity) but not necessarily specific to pain.
1999; Vogt, 2005); however, in some con- The relative neurofunctional specificity of
ditions, innocuous (and nonpainful) stimu- pain processes may be found at the level
lation may also produce robust activation of the neural networks (Shyu et al., 2010)
in somatosensory areas as well as in the within each area that are activated by painful
insula and ACC (Legrain, Iannetti, Plaghki, stimuli and reflect the underlying factors
& Mouraux, 2011). The ACC and the insula that make pain highly salient based on
are considered multisensory areas associ- its strong intensity and inherent affective
ated with cognitive and emotional pro- valence.
cesses (Augustine, 1996; Shackman et al.,
2011; Vogt, 2005). These areas are also acti-
Beyond the Stimulus Model of
vated by salient but nonpainful stimuli
Pain-Related Activation
in the auditory, visual, and somatosentory
modalities (Baliki, Geha, & Apkarian, 2009; The functional analysis of the cerebral net-
Downar, Crawley, Mikulis, & Davis, 2000; works implied in pain extends beyond the
Legrain et al., 2011). Some studies relying simple examination of brain activations
on cortical evoked potential measurements evoked by noxious stimuli. Indeed, func-
have arrived at similar conclusions, stress- tional brain imaging studies generally try to
ing that the pain responses can be explained relate these activations to some aspect of the
by a somatosensory activation evoked by participants’ experience or responses. In the
a very salient stimulation (Legrain et al., vast majority of studies, the self-report of
2011). High saliency being an intrinsic prop- pain intensity on a visual or numerical scale
erty of painful stimuli, one may there- constitutes the principal behavioral mea-
fore interpret the strongest activation found sure. This choice is justified by the crucial
to painful versus nonpainful stimulation in importance of this measurement in clinical
functional brain imaging studies as reflecting studies of pain, consistent with the experi-
somatosensory processing of highly salient ential definition of pain, as discussed earlier.
stimuli (Legrain et al., 2011).2 Because this subjective experience of pain is
The factors that contribute to saliency generally considered to reflect the activation
include stimulus intensity, contrast, novelty, of the spinothalamocortical pathway(s), the
and intrinsic or extrinsic affective value, and primary focus is largely on the activations
each of these influences may reflect at least observed within the cortical targets of this
partly different neural mechanisms. Accord- system. These self-report measurements are
ing to this perspective, pain is inherently the result of the multiple stages of integra-
salient based on its strong sensory inten- tion of the noxious signal at the various lev-
sity and intrinsic aversive valence, two basic els of the CNS, and not simply the passive
properties that translate into experiential spinal and thalamic transmission of nocicep-
dimensions of pain intensity and unpleas- tive signals.
antness. These observations suggest that the At each level of integration of the noxious
pattern of brain activation associated with signal in the CNS, a variety of additional
PAIN AND THE EMOTIONAL RESPONSES TO NOXIOUS STIMULI 229
output responses can be initiated (Figure involved in the coding of stimulus inten-
9.1). The integration of nociceptive signals sity and those involved more specifically in
from the primary afferents in the dorsal perceived intensity, this research confirms
horn of the spinal cord activates not only that self-reports of pain are generally pro-
the ascending nociceptive pathways but also portional to the stimulus-driven increase in
the spinal networks involved in the produc- the cortical responses within the areas SI
tion of spinal reflexes that can be measured and SII, the insula, and the ACC (and adja-
by the activation of the target peripheral cent SMA). These results thereby demon-
effectors. At the level of the brainstem, strate a robust correspondence between the
bulbar integration is essential for the reg- level of activation of the spinothalamocorti-
ulation of the hetero-segmental somato- cal pathways by the stimulus and the sub-
somatic interactions (e.g., noxious coun- jective experience of pain. However, some
terstimulation analgesia), whereas reticular subareas present monotonic increases to
sites are critical for the production of non- increasing stimulus intensity that do not
specific systemic responses, such as systemic necessarily code specifically or linearly for
autonomic activation and general arousal the perceived intensity of pain. For exam-
(see reviews by Bandler & Shipley, 1994; ple, whereas the ACC generally presents a
Villanueva & Bourgeais, 2009; Villanueva response proportional to pain, some sectors
& Fields, 2004). Mesencephalic and dien- show a gradual increase to both nonpainful
cephalic nuclei provide yet another stage and painful stimulations (consistent with the
of integration, allowing for the production coding of stimulus intensity not specific to
of coordinated systemic activation involving pain). Other subsectors of the ACC dis-
complex musculoskeletal, visceral, and hor- play an increase mainly at the level of the
monal responses. Lastly, the diencephalon pain threshold, with no additional increases
and the telencephalon allow for the inter- to suprathreshold intensities (Büchel et al.,
modal integration of nociceptive informa- 2002; see a similar stimulus-response func-
tion, a process essential to various forms of tion analysis of other pain-activated areas in
associative learning (e.g., the amygdala, the Bornhövd et al., 2002). Thus, some subareas
thalamus, and the ACC in aversive condi- could be primarily sensitive to the occur-
tioning; Gabriel, 1993; LeDoux, 2007); they rence of pain (threshold), whereas others
provide the additional mechanisms for the may code more generally for the intensity
psychological regulation of the experience of the stimulus or more specifically for the
and the communication of pain. Impor- intensity of pain.
tantly, these higher order sites of integration This relation between cerebral activity
also exert regulatory influences on the lower and pain experiences is also critical to
order levels. the understanding of interindividual differ-
ences commonly observed in self-reports
and the variations observed spontaneously
Stimulus Intensity and Self-Reports
or induced by various experimental manip-
of Pain
ulations in intraindividual paradigms using
The simplest and most direct manipulation constant stimuli. Coghill and his collabo-
of the magnitude of the pain experience is rators have clearly shown that individuals
generally based on a precise and controlled reporting greater painful sensitivity (i.e., a
modification of the intensity of the stimulus higher pain rating) during the application of
to verify if the cerebral responses are indeed stimulations of constant intensity also dis-
proportional to the stimulus and the pain play stronger activation in cortical targets of
reported by the participants (Coghill, Sang, the spinothalamocortical pathway (e.g., S1
Maisog, & Iadarola, 1999; Derbyshire et al., and the ACC; Coghill, McHaffie, & Yen,
1997; Porro, Cettolo, Francescato, & Baraldi, 2003). These results are extremely impor-
1998). Although these studies do not gener- tant because they indicate that the interindi-
ally allow for a separation between processes vidual differences in the subjective reports
230 PIERRE RAINVILLE
of pain are not simply a consequence of brospinal systems affecting noxious activity
response biases, but rather imply that these in the dorsal horn, as postulated by Melzack,
differences reflect, at least in part, individual Wall, and Casey more than 40 years ago
differences in the neurophysiological pro- (Melzack & Casey, 1968; Melzack & Wall,
cessing of the nociceptive input in cerebral 1965).
areas and/or at earlier stages of nocicep-
tive processes in the spinothalamocortical
Spinal Nociceptive Responses
pathways.
and Brain Activity
The multiple experimental studies inves-
tigating the psychological modulation of The integration of noxious information at
pain also support these conclusions on the the spinal level can trigger a reflex with-
role of various cerebral areas in the subjec- drawal response simultaneous with the acti-
tive experience of pain. Indeed, modulation vation of the ascending nociceptive path-
of the pain by hypnosis, attention, expec- ways that will in turn activate the higher
tations (e.g., placebo), and emotions gener- order brain centers. Experimentally, senso-
ally produces changes in cerebral activation rimotor spinal activation can be induced
in one or more target sectors of the nox- by a brief electric stimulation to the skin
ious ascending pathways, in particular in the or to a cutaneous nerve (e.g., the sural
territories receiving the spinothalamocorti- nerve), which will induce a fast activation
cal input (see table 3 in Apkarian et al., of the flexor muscle of the correspond-
2005). These studies also suggest that vari- ing limb measured in the EMG (e.g., the
ous psychological manipulations can act on biceps femoris). The latency of this noci-
various sectors of this network (e.g., Hof- ceptive flexion reflex (RIII reflex) is 80–
bauer et al., 2001; Rainville, Duncan, Price, 90 ms, and its duration is 60–90 ms. Over
Carrier, & Bushnell, 1997) that are thought the last 35 years, this experimental model
to be involved more specifically, although has been used to study spinal nociception
not necessarily exclusively, in the sensory and has been applied to various clinical and
(e.g., SI) or affective (e.g., ACC) dimensions experimental contexts to assess the involve-
of pain. ment of descending regulatory systems
In addition to the modulation of the (Sandrini et al., 2005). Including this addi-
noxious responses, the studies interested in tional measure of nociception in fMRI stud-
the psychological modulation of pain also ies recently allowed us to further explore
examine the areas potentially implicated the relation between spinal and cerebral
in the production of analgesic/hyperalgesic responses.
effects. These analyses have revealed pre- The earlier discussion of interindividual
frontal activation during hypnotic modula- differences in cerebral activation empha-
tion (e.g., Faymonville et al., 2003; Rainville sized the interindividual differences in the
et al., 1999), distraction (e.g., Frankenstein, subjective experience of pain as reflected
Richter, McIntyre, & Remy, 2001; Valet in pain reports and brain activity. Some of
et al., 2004), or placebo (e.g., Bingel, Lorenz, these cerebral differences may also reflect,
Schoell, Weiller, & Buchel, 2006; Wager more or less directly, some differences in
et al., 2004; Watson et al., 2009). Moreover, the integration of noxious signals at lower
several of these studies also show mesen- levels of the neuraxis, including the spinal
cephalic activation during analgesia. These cord. Indeed, when examining the cere-
brainstem responses are generally thought to bral responses to the acute painful elec-
reflect the activation of descending pain con- trical stimulations used to elicit the RIII
trol mechanisms (Tracey & Mantyh, 2007). reflex, one finds that the amplitude of the
This interpretation is certainly plausible, response in some areas is proportional to
but the cerebral imaging results are insuffi- the gain in the spinal responses, as measured
cient to determine whether these activations by the amplitude of the RIII, whereas other
indeed reflect the activation of the cere- areas appear to correlate more specifically
PAIN AND THE EMOTIONAL RESPONSES TO NOXIOUS STIMULI 231
need for incorporating multiple pain-related nociceptive ascending pathways (e.g., SI,
responses in our experimental paradigms to SII, insula, ACC, and medulla) presented
better appreciate the potential interactions activity levels proportional to the sympa-
among multiple regulating mechanisms act- thetic reactivity of the participants. How-
ing at the various levels of the CNS. Much ever, the activation of SI and the insula
remains to be done in this multidimensional correlated better with the brain responses
exploration to better account for the com- in the nonpainful condition (warm). This
plexity of pain modulation reflected in vari- finding was suggested to reflect a role of
ous pain responses. these structures in the monitoring and pos-
sibly the top-down regulation of autonomic
activity that was masked in the pain condi-
Brain Correlates of the Autonomic tion when the predominant ascending nox-
Responses to Pain ious signals overshadowed the less impor-
tant effects associated with sympathetic
In addition to the spinal reflex responses, reactivity. In contrast, a more robust rela-
acute pain is generally accompanied by a tion with the SCR was observed in the
robust autonomic activation. Contrary to painful condition compared to the not-
the RIII reflect, however, this autonomic painful condition in the ACC, amygdala,
activation does not reflect specifically a noci- thalamus, and hypothalamus (and to a lesser
ceptive response, because it can be produced extent in SII and the medulla). Thus, the
under various non-nociceptive physiological SCR could reflect the spinothalamocortical
conditions (e.g., cardiorespiratory arrhyth- activity during innocuous thermal stimula-
mia regulated by the baroreflex system) or tions, as well as the activation of brainstem
by an intense or unexpected innocuous stim- responses including the spinoparabrachial
ulation in any sensory modality (cf. the pathway activating the hypothalamus and
saliency effect as discussed earlier). In spite amygdala in the painful condition. This
of their poor specificity, these autonomic interpretation is compatible with the role of
responses are generally very sensitive to the this pathway in the regulation of arousal and
occurrence of acute pain and likely reflect emotional responses evoked by nociceptive
the nonspecific alerting induced through inputs and is highly consistent with the cen-
the activation of various brainstem nuclei. tral role of the amygdala in aversive leaning
Given these properties, autonomic mea- involving noxious unconditioned stimuli.
sures are certainly not the tools of choice In a separate study, the cerebral responses
to evaluate pain. Nevertheless, they provide produced by noxious electrical stimulations
highly relevant means to improve our under- evoking an RIII reflex were examined in
standing of the mechanisms involved in the relation with the SCR. In this case, the
integration and the regulation of nocicep- analysis aimed at exploring individual dif-
tive responses, some of which are obviously ferences in activation specifically associated
based on networks that play functional roles with the SCR, after having controlled for (by
nonspecific to pain. covariance) effects explained by the varia-
The relation between cerebral activity tions in the RIII and in the self-reports of
and autonomic activity evoked by acute pain. Intraindividual fluctuations in the SCR
pain stimuli has been examined in two were correlated with the activation of the
recent studies combining fMRI and skin insula and the ACC, consistent with a cen-
conductance responses (SCRs). In the first, tral role of these structures in autonomic
the interindividual differences in cerebral monitoring and regulation (see Chapter 3).
responses were assessed in relation to However, the examination of the interindi-
interindividual differences in the amplitude vidual differences in the SCR revealed a
of the SCR evoked by painful and non- very intriguing pattern of cerebral activa-
painful thermal stimuli (Dube et al., 2009). tion. Indeed, the response evoked by the
Several areas targeted by the thermal and electric stimuli in the orbitofrontal cortex
234 PIERRE RAINVILLE
(among others) was slightly negative; more 2002). However, this important expressive
importantly, it was negatively correlated channel is often neglected in pain research
with the interindividual differences in the because it is subject to contextual, social,
amplitude of the SCR (i.e., stronger SCRs in and cultural rules that complicate its inter-
subjects showing larger decreases in OFC), pretation.
while being positively correlated with the We recently examined the spontaneous
interindividual differences in the ampli- facial expression of pain in an fMRI study
tude of RIII response (i.e., stronger RIII involving acute thermal stimulation. In this
in subjects showing smaller decreases or study, the intensity of the stimuli was
some increase in OFC). A multiple regres- adjusted individually to control for differ-
sion model further demonstrated that the ences in pain sensitivity as reported in the
combination of interindividual differences subjective evaluations of the participants,
in the motor (RIII) and autonomic (SCR) thus enabling us to examine more specif-
responses predicted the orbitofrontal acti- ically the cerebral correlates of the facial
vation to the stimulus with a very high level expression of pain (Kunz, Chen, Lauten-
of precision (R2 = 0.93). This unexpected bacher, Vachon-Presseau, & Rainville, 2011).
relation certainly deserves a thorough exam- In addition to foreseeable activations in the
ination in future studies. region of the face in the primary motor cor-
These effects may be relevant to the tex two important effects were observed.
pattern of nociceptive responses observed First, the intraindividual analysis com-
in mild cognitive impairment and demen- parison of trials with versus without spon-
tia, where an increase in motor reflex taneous facial responses showed that the
responses (RIII) is associated with a reduc- occurrence of such an expression reflects a
tion in autonomic activation (Kunz, Mylius, more important activation in several cere-
Scharmann, Schepelman, & Lautenbacher, bral sites such as the thalamus, SI, SII, insula,
2009; Kunz, Mylius, Schepelmann, & Laut- and the ACC (see Figure 9.3). A response
enbacher, 2009). These clinical physiologi- proportional to the expression of pain was
cal signs could be explained by prefrontal also observed across individuals in SI and
damage involving orbitofrontal sectors and the insula (i.e., stronger responses in more
leading to a motor disinhibition, combined expressive participant). Again, these effects
with reduced sympathetic reactivity. These were observed even though the subjective
studies clearly illustrate the advantage of reports of pain did not show significant
considering multiple physiological responses differences between trials with and with-
evoked during pain to better detect and out expression, or across subjects displaying
interpret the functional role of the vari- weak/no or strong facial responses. These
ous areas activated by noxious stimulations. observations indicate that the facial expres-
These responses may be complemented sion of pain conveys unique information that
by the assessment of complex behavioral is at least partially independent from the
responses. subjective report and that these two output
response channels of communication reflect
at least in part the activation of the cortical
The Nonverbal Expression of Pain targets of the spinothalamic pathways. This
finding contributes to the validation of the
At a higher level in the hierarchy of noxious facial expression for the evaluation of pain
and pain-related responses are mechanisms (at least in within-subject conditions). How-
involved in the regulation of the behavioral ever, the absence of facial expression should
expression of pain. Among these responses, not be interpreted simply as a reduced sen-
facial expression constitutes one of the priv- sitivity to pain.
ileged output channels for the communica- The second important finding in this
tion of pain experiences (Craig et al., 2001; study relates to the stronger cerebral acti-
Hadjistavropoulos & Craig, 2002; Williams, vations to the painful thermal stimulations
PAIN AND THE EMOTIONAL RESPONSES TO NOXIOUS STIMULI 235
in the absence of spontaneous facial expres- account the sociocontextual factors typically
sion. Such enhanced brain responses were leading to the suppression of pain expres-
observed mainly in the prefrontal cortices sion. The nonverbal behavioral expression,
(including the perigenual ACC) of the less being free of inhibitory influences, may con-
expressive individuals and in the trials with- stitute more direct measurement of activity
out facial expression in the more expressive within the nociceptive-responsive brain sys-
participants (see Figure 9.3). Moreover, tems and thereby constitute a more valid
this enhanced prefrontal response was sig- method of pain assessment in these popula-
nificantly decreased when nonexpressive tions.
participants were asked to voluntarily com-
municate their painful experience by their
facial expression. Consistent with classical Conclusion
studies showing lower thresholds for “winc-
ing” and withdrawal responses to acute heat Understanding the functional roles of the
pain stimuli in prefrontal lobotomy patients higher order centers of the brain in pain
(Chapman, Rose, & Solomon, 1950), these relies primarily on the anatomical and phys-
results suggest that “stoicism” during pain iological knowledge gained from animal
reflects the active suppression of the facial studies on the nociceptive and homeostatic
expression by prefrontal cortices. regulatory systems. Research in functional
From a clinical perspective, these results neuroimaging further provides indispens-
are important because they imply that the able information about areas implicated in
spontaneous presence and the magnitude the perception and the modulation of pain
of pain facial expression signal the activa- in humans. This analysis relates cerebral
tion and magnitude of the neurophysiologi- activation to subjective responses reported
cal responses within brain systems involved by participants on pain-rating scales. This
in the experience of pain. However, the approach is certainly essential to understand
absence of expression in a patient who nev- the mechanisms underlying pain processes,
ertheless reports an intense pain experience but it does not provide a comprehensive
on self-report pain scales likely reflects the account of the complexity of nociceptive
inhibition of the facial expressive channel. processes. The analysis of neurophysiologi-
Reciprocally, the spontaneous and intense cal activity underlying nociception and pain
facial expression of pain in a patient who must also consider the multiple responses
is unable to use self-report scales could that accompany painful experiences and are
signal intense pain that is not adequately not necessarily encoded adequately in sub-
communicated voluntarily. This is partic- jective responses.
ularly important in populations present- Noxious neurophysiological activation is
ing some limitations in verbal communica- expressed in various ways at each level of
tion and for whom nonverbal expression is integration of the central nervous system.
obviously an alternative way to assess pain In this brief review, I highlighted the cere-
(Hadjistavropoulos, von Baeyer, & Craig, bral activations associated with spinal with-
2001). Recent research further demonstrates drawal reflexes, the SCR, and the facial
that, for comparable levels of self-reports expression. Each of these response chan-
of pain, patients presenting with symp- nels provides information that complements
toms of dementia or even mild cognitive the subjective evaluation of pain and is
impairments may more adequately commu- more strongly associated with the activa-
nicate pain by their facial expression (Had- tion of partly distinct subsectors of pain-
jistavropoulos, Voyer, Sharpe, Verreault, & responsive brain networks. This approach
Aubin, 2008; Kunz, Scharmann, Hemme- to pain research may contribute to the
ter, Schepelmann, & Lautenbacher, 2007; advancement of emotion research to the
Kunz et al., 2009). Their facial expressions extent that pain-related responses can be
may reflect a reduced ability to take into conceived as emotional responses to direct
236 PIERRE RAINVILLE
threats to the integrity of the body. This cephalic components, reflect the com-
multivariate approach could also contribute plex pattern of responses in these sepa-
to a better understanding of the interindi- rate output systems?
vidual differences in emotion responses and r Individuals vary greatly in the pattern
help refine the physiopathological analy- of responses (e.g., facial expression, self-
sis of abnormal pain states and emotional report, autonomic) they exhibit dur-
responses across various clinical popula- ing pain and emotion. How does this
tions. variability reflect differences in brain
activity, and how are those differences
Outstanding Questions related to biopsychosocial factors ranging
from genetic predispositions to cultural
r Pain reflects potential injury to the context?
body (i.e., interoceptive signal) and, at
least in some circumstances, properties
of nociceptive stimuli (i.e., exterocep- Acknowledgments
tive signal). What are the brain mecha-
This research is funded mainly by the
nisms involved in the interoceptive versus
exteroceptive functions associated with “Fonds de la recherche en santé du Québec,”
the Canadian Institutes of Health Research
pain?
r The brain activation associated with (CIHR), and the Natural Science and Engi-
acute pain is very similar to that produced neering Council of Canada (NSERC).
by other salient sensory experiences. Is
there a pattern of brain activity specif- Notes
ically associated with pain experiences?
What are the distinctive features of pain- 1 Note that coordinate-based meta-analyses are
related brain responses that make pain vulnerable to report biases where a pri-
experiences unique? ori hypotheses (including valid ones) often
r Although pain is not typically considered guide the search and report of activation
an emotion, it does share almost all of its peaks using lower statistical threshold. This
defining features with emotions (with the increases the probability of reporting an acti-
exception of being necessarily triggered vation peak and leads to higher activation
likelihood estimates in target areas. This is not
by a somatic inducer with specific sen-
to say that these estimates do not reflect true
sory qualities). Shared features include activation but simply emphasizes that areas
behavioral and autonomic responses as outside these target sites are less likely to be
well as affective experiential dimensions. reported, given the weakest a priori hypothe-
How much convergence is there between ses for these additional areas and the corre-
brain systems involved in pain and sponding more stringent activation threshold
emotions? used to detect these responses.
r Human affective neuroscience in general 2 Note nevertheless that this is not incompati-
and neuroimaging studies in particular ble with a neuro-functional specificity of pain
have largely focused on the conditions processes at the level of the neural networks
that lead to pain and emotion induc- within each area, as acknowledged by Legrain
et al. (2011).
tion. However, pain and emotion are
also defined by patterns of responses that
reflect the engagement of several neu- References
rological subsystems involved in behav-
ioral, autonomic, and hormonal regu- Albanese, M. C., Duerden, E. G., Rainville, P., &
lation. How does the pattern of brain Duncan, G. H. (2007). Memory traces of pain
activity observed during pain and emo- in human cortex. Journal of Neuroscience, 27,
tion, including the diencephalic and tele- 4612–20.
PAIN AND THE EMOTIONAL RESPONSES TO NOXIOUS STIMULI 237
Apkarian, A. V., Bushnell, M. C., Treede, R. D., ciable neural responses related to pain inten-
& Zubieta, J. K. (2005). Human brain mech- sity, stimulus intensity, and stimulus aware-
anisms of pain perception and regulation in ness within the anterior cingulate cortex: A
health and disease. European Journal of Pain, parametric single-trial laser functional mag-
9, 463–84. netic resonance imaging study. Journal of Neu-
Augustine, J. R. (1996). Circuitry and functional roscience, 22, 970–76.
aspects of the insular lobe in primates includ- Chapman, W. P., Rose, A. S., & Solomon,
ing humans. Brain Research: Brain Research H. C. (1950). A follow-up study of motor with-
Reviews, 22, 229–44. drawal reaction to heat discomfort in patients
Baliki, M. N., Geha, P. Y., & Apkarian, A. V. before and after frontal lobotomy. American
(2009). Parsing pain perception between noci- Journal of Psychiatry, 107, 221–24.
ceptive representation and magnitude estima- Coghill, R. C., McHaffie, J. G., & Yen, Y. F.
tion. Journal of Neurophysiology, 101, 875–87. (2003). Neural correlates of interindividual
Bandler, R., & Shipley, M. T. (1994). Colum- differences in the subjective experience of
nar organization in the midbrain periaqueduc- pain. Proceedings of the National Academy of
tal gray: Modules for emotional expression? Sciences, 100, 8538–42.
Trends in Neuroscience, 17, 379–89. Coghill, R. C., Sang, C. N., Maisog, J. M., &
Bechara, A. (2004). The role of emotion in Iadarola, M. J. (1999). Pain intensity process-
decision-making: Evidence from neurological ing within the human brain: A bilateral, dis-
patients with orbitofrontal damage. Brain and tributed mechanism. Journal of Neurophysiol-
Cognition, 55, 30–40. ogy, 82, 1934–43.
Bechara, A., Damasio, H., Tranel, D., & Dama- Craig, A. D. (2002). How do you feel? Interocep-
sio, A. R. (1997). Deciding advantageously tion: The sense of the physiological condition
before knowing the advantageous strategy. of the body. Nature Reviews Neuroscience, 3,
Science, 275, 1293–95. 655–66.
Bernard, J. F., Bester, H., & Besson, J. M. (1996). Craig, A. D. (2003). A new view of pain as a
Involvement of the spinoparabrachio amyg- homeostatic emotion. Trends in Neuroscience,
daloid and hypothalamic pathways in the 26, 303–7.
autonomic and affective emotional aspects Craig, A. D. (2009). How do you feel – now? The
of pain. Progress in Brain Research, 107, anterior insula and human awareness. Nature
243–55. Reviews Neuroscience, 10, 59–70.
Bernard, J. F. & Villanueva, L. (2009). Architec- Craig, K. D., Prkachin, K. M., & Grunau, R. V. E.
ture fonctionnelle des systèmes nociceptifs. In (2001). The facial expression of pain. In D. C.
D. Bouhassira & B. Calvino (Eds.), Douleurs: Turk & R. Melzack (Eds.), Handbook of pain
Physiologie, physiopathologie et pharmacologie assessment (2nd ed., pp. 153–69). New York:
(pp. 1–29). Paris: Arnette. Guilford Press.
Bingel, U., Lorenz, J., Schoell, E., Weiller, C., Damasio, A. R. (1994). Descartes’ error: Emotion,
& Buchel, C. (2006). Mechanisms of placebo reason and the human brain. New York: Avon
analgesia: rACC recruitment of a subcor- Books.
tical antinociceptive network. Pain, 120, Damasio, A. R. (1996). The somatic marker
8–15. hypothesis and the possible functions of the
Bornhövd, K., Quante, M., Glauche, V., Bromm, prefrontal cortex. Philosophical Transactions of
B., Weiller, C., & Büchel, C. (2002). Painful the Royal Society of London Series B: Biological
stimuli evoke different stimulus-response Sciences, 351, 1413–20.
functions in the amygdala, prefrontal, insula Damasio, A. R. (1999). The feeling of what
and somatosensory cortex: A single-trial fMRI happens: Body and emotion in the mak-
study. Brain, 125, 1326–36. ing of consciousness. New York: Hartcourt
Bourgeais, L., Monconduit, L., Villanueva, L., Brace.
& Bernard, J. F. (2001). Parabrachial internal Derbyshire, S. W., Jones, A. K., Gyulai, F., Clark,
lateral neurons convey nociceptive messages S., Townsend, D., & Firestone, L. L. (1997).
from the deep laminas of the dorsal horn to Pain processing during three levels of noxious
the intralaminar thalamus. Journal of Neuro- stimulation produces differential patterns of
science, 21, 2159–65. central activity. Pain, 73, 431–45.
Büchel, C., Bornhovd, K., Quante, M., Glauche, Dostrovsky, J. O., & Craig, A. D. (2006). Ascend-
V., Bromm, B., & Weiller, C. (2002). Disso- ing projection systems. In S. B. McMahon &
238 PIERRE RAINVILLE
Melzack, R. (1990). Phantom limbs and the con- methods of pain control: Basic science and clini-
cept of a neuromatrix. Trends in Neurosciences, cal perspectives (pp. 117–41). Seattle WA: IASP
13, 88–92. Press.
Melzack, R., & Casey, K. L. (1968). Sensory, moti- Rainville, P., Bao, Q. V., & Chretien, P. (2005).
vational, and central control determinants of Pain-related emotions modulate experimen-
pain: A new conceptual model. In D. Ken- tal pain perception and autonomic responses.
shalo (Ed.), The skin senses (pp. 423–443). Pain, 118, 306–18.
Springfield, IL: Thomas. Rainville, P., Carrier, B., Hofbauer, R. K., Bush-
Melzack, R., & Wall, P. D. (1965). Pain mecha- nell, M. C., & Duncan, G. H. (1999). Dissoci-
nisms: A new theory. Science, 150, 971–78. ation of pain sensory and affective dimensions
Merskey, H., & Spear, F. G. (1967). The concept using hypnotic modulation. Pain, 82, 159–71.
of pain. Journal of Psychosomatic Research, 11, Rainville, P., Duncan, G. H., Price, D. D., Car-
59–67. rier, B., & Bushnell, M. C. (1997). Pain affect
Metzinger, T. (2000). The subjectivity of subjec- encoded in human anterior cingulate but not
tive experience: A representationalist analy- somatosensory cortex. Science, 277, 968–71.
sis of the first-person perspective. In T. Met- Rainville, P., Hofbauer, R. K., Paus, T., Duncan,
zinger (Ed.), Neural correlates of consciousness: G. H., Bushnell, M. C., & Price, D. D. (1999).
Empirical and conceptual questions (pp. 285– Cerebral mechanisms of hypnotic induction
306). Cambridge, MA: MIT Press. and suggestion. Journal of Cognitive Neuro-
Monconduit, L., Bourgeais, L., Bernard, J. F., Le science, 11, 110–25.
Bars, D., & Villanueva, L. (1999). Ventrome- Rhudy, J. L., Williams, A. E., McCabe, K. M.,
dial thalamic neurons convey nociceptive sig- Nguyen, M. A., & Rambo, P. (2005). Affec-
nals from the whole body surface to the dorso- tive modulation of nociception at spinal and
lateral neocortex. Journal of Neuroscience, 19, supraspinal levels. Psychophysiology, 42, 579–
9063–72. 87.
Piché, M., Arsenault, M., & Rainville, P. (2010). Ridderinkhof, K. R., Ullsperger, M., Crone, E.
Dissection of perceptual, motor and auto- A., & Nieuwenhuiss, S. (2004). The role of
nomic components of brain activity evoked the medial frontal cortex in cognitive control.
by noxious stimulation. Pain, 149, 453–62. Science, 306, 443–47.
Ploner, M., Freund, H. J., & Schnitzler, A. (1999). Roy, M., Piche, M., Chen, J. I., Peretz, I., &
Pain affect without pain sensation in a patient Rainville, P. (2009). Cerebral and spinal mod-
with a postcentral lesion. Pain, 81, 211–14. ulation of pain by emotions. Proceedings of the
Plutchik, R. (1980). A general psychoevolution- National Academy of Sciences, 106(49), 20900–
ary theory of emotion. In R. Plutchik & H. 5.
Kellerman (Eds.), Emotion: Theory, research, Sah, P., Faber, E. S., Lopez de, A. M., & Power,
and experience: Vol. 1. Theories of emotion J. (2003). The amygdaloid complex: Anatomy
(pp. 3–33). New York: Academic Press. and physiology. Physiology Review, 83, 803–34.
Porro, C. A., Cettolo, V., Francescato, M. P., & Sandrini, G., Serrao, M., Rossi, P., Romaniello,
Baraldi, P. (1998). Temporal and intensity cod- A., Cruccu, G., & Willer, J. C. (2005). The
ing of pain in human cortex. Journal of Neuro- lower limb flexion reflex in humans. Progress
physiology, 80, 3312–20. in Neurobiology, 77, 353–95.
Price, D. D. (1999). Psychological mechanisms of Sato, A., Sato, Y., & Schmidt, R. F. (1997). The
pain and analgesia. Seattle, WA: IASP Press. impact of somatosensoty input on autonomic
Price, D. D., & Barrell, J. J. (1984). Some gen- functions. In M. P. Blaustein, H. Grunicke,
eral laws of human emotion: Interrelation- D. P. Konstanz, G. Schultz, & M. Schweiger
ships between intensities of desire, expec- (Eds.), Reviews of physiology biochemistry and
tation, and emotional feeling. Journal of pharmacology (pp. 1–310). Berlin: Springer-
Personality, 52, 389–409. Verlag.
Price, D. D., Barrell, J. J., & Gracely, R. H. Shackman, A. J., Salomons, T. V., Slagter, H. A.,
(1980). A psychophysical analysis of exper- Fox, A. S., Winter, J. J., & Davidson, R. J.
imental factors that selectively influence (2011). The integration of negative affect, pain
the affective dimension of pain. Pain, 8, and cognitive control in the cingulate cortex.
137–49. Nature Reviews Neuroscience, 12, 154–67.
Rainville, P. (2004). Pain and emotions. In D. D. Shyu, B. C., Sikes, R. W., Vogt, L. J., & Vogt, B.
Price & M. C. Bushnell (Eds.), Psychological A. (2010). Nociceptive processing by anterior
240 PIERRE RAINVILLE
cingulate pyramidal neurons. Journal of Neu- Villemure, C., Slotnick, B. M., & Bushnell,
rophysiology, 103, 3287–3301. M. C. (2003). Effects of odors on pain per-
Simon, D., Craig, K. D., Gosselin, F., Belin, P., & ception: Deciphering the roles of emotion and
Rainville, P. (2008). Recognition and discrimi- attention. Pain, 106, 101–8.
nation of prototypical dynamic expressions of Vogt, B. A. (2005). Pain and emotion interactions
pain and emotions. Pain, 135, 55–64. in subregions of the cingulate gyrus. Nature
Tracey, I., & Mantyh, P. W. (2007). The cerebral Reviews Neuroscience, 6, 533–44.
signature for pain perception and its modula- Wager, T. D., Rilling, J. K., Smith, E. E., Soko-
tion. Neuron, 55, 377–91. lik, A., Casey, K. L., Davidson, R. J., et al.
Treede, R. D., Kenshalo, D. R., Gracely, R. H., (2004). Placebo-induced changes in FMRI in
& Jones, A. K. P. (1999). The cortical repre- the anticipation and experience of pain. Sci-
sentation of pain. Pain, 79, 105–11. ence, 303, 1162–67.
Valet, M., Sprenger, T., Boecker, H., Willoch, Watson, A., El-Deredy, W., Iannetti, G. D.,
F., Rummeny, E., Conrad, B., et al. (2004). Lloyd, D., Tracey, I., Vogt, B. A., et al. (2009).
Distraction modulates connectivity of the Placebo conditioning and placebo analgesia
cingulo-frontal cortex and the midbrain modulate a common brain network during
during pain–an fMRI analysis. Pain, 109, pain anticipation and perception. Pain, 145,
399–408. 24–30.
Villanueva, L., & Bourgeais, L. (2009). Systèmes Williams, A. C. (2002). Facial expression of pain:
de modulation de la douleur. In D. Bouhas- An evolutionary account. Behavioral Brain
sira & B. Calvino (Eds.), Douleurs: Physiologie, Science, 25, 439–55.
physiopathologie et pharmacologie (pp. 30–45). Willis, W. D., Al-Chaer, E. D., Quast, M. J.,
Paris: Arnette. & Westlund, K. N. (1999). A visceral pain
Villanueva, L., & Fields, H. L. (2004). Endoge- pathway in the dorsal column of the spinal
nous central mechanisms of pain modula- cord. Proceedings of the National Academy of
tion. In L. Villanueva, A. Dickenson, & Sciences, 96, 7675–79.
H. Ollat (Eds.), The pain system in normal and Willis, W. D., & Westlund, K. N. (1997). Neu-
pathological states: A primer for clinicians. Vol. roanatomy of the pain system and of the path-
31: Progress in pain research and management ways that modulate pain. [Review] Journal of
Seattle: IASP Press. Clinical Neurophysiology, 14, 2–31.
CHAPTER 10
Madhavika parimala lalite naba malati mood (Schiffman & Miller, 1995). In con-
jati sugandhau trast to vision, audition, and somesthesia,
Munimanasampi mohanakarini taruna which involve early cortical processing in
karana bandhau sensory unimodal brain areas, chemosen-
sory processing initially occurs in limbic
The strong scent of the flowers Madhavi, and paralimbic heteromodal regions that are
Malati and Jati heavily implicated in emotional processing
pervade the air, enchanting, even the (Carmichael, Clugnet, & Price, 1994; Got-
meditating tfried, 2006). Indeed, many of these cen-
hermits (Munis), who feel again the ters function as a key hub linking smells,
passion of intimate tastes, emotions, and behavior. Nonetheless,
bonds of youth, which they had despite the demonstration of strong percep-
abandoned tual and anatomical links between emotion
–from Gitagovinda, 12th-century and olfaction, the visuo-centric perspective
Sanskrit poetry by Jayadeva of human sensory experience has long dom-
inated the field of affective psychology and
neuroscience – as evidenced by the over-
Historically, odors have been recognized for whelming reliance on visual stimuli such
their power to evoke strong emotional reac- as the International Affective Picture Sys-
tions. This view is well substantiated in the tem (Lang, Bradley, & Cuthbert, 2008) and
scientific literature, showing that odor per- the Ekman and Friesen series of Pictures of
ception is commonly associated both with Facial Affect (Ekman & Friesen, 1976) – with
emotional reactions measured at the verbal the general effect of crowding out olfactory-
(Berglund, Berglund, Engen, & Ekman, 1973; based models of emotional processing.
Schiffman, 1974), behavioral (Bensafi et al., One bias that may have contributed to
2003), and physiological levels (Bensafi et al., the under-utilization of olfactory stimuli
2002a) and with more long-lasting changes in in affective neuroscience research is that
241
242 APRAJITA MOHANTY & JAY A. GOTTFRIED
humans are assumed to have a bad sense of anisms that support different aspects of
smell. Genetic evidence showing a decline in emotion coding and emotion-related learn-
the number of functional olfactory receptor ing in the amygdala, orbitofrontal cortex,
genes in the course of evolution from mice and olfactory cortices. Another important
to humans certainly supports this viewpoint emphasis of the chapter is on the mal-
(Shepherd, 2004). Yet, contradictory behav- leability of olfactory hedonic perception.
ioral evidence suggests that humans have By highlighting this aspect of olfactory per-
a surprisingly good sense of smell despite ception, we can delineate the psycholog-
their reduced receptor gene repertoire. For ical and neural mechanisms of contextual
example, ethyl mercaptan, an odorant that modulation and plasticity of emotional pro-
is often added to alert the presence of odor- cessing. Finally, we discuss odor memory as
less propane gas, can be detected at con- an example of how olfactory research has
centrations as low as 0.2 parts per billion brought new insights into mechanisms of
(Yeshurun & Sobel, 2010). A series of stud- emotion-cognition interactions. This chap-
ies by Laska and colleagues show that not ter begins with a discussion of basic proper-
only can humans detect minuscule quanti- ties of olfactory perception to illustrate how
ties of odor but they can also tell apart one the anatomy and function of olfaction and
odorant from another, based on molecular emotion are closely intertwined.
identity or concentration. Humans can dis-
criminate aliphatic odorants equal in num-
ber of carbons but differing in functional What Constitutes an Olfactory
group, odorants differing in chain length Stimulus?
by one carbon, and odors of enantiomer
(mirror-image) molecules like (+) and (−) Brains have evolved to perceive sensory
carvone (Laska & Seibt, 2002). Furthermore, events that hold behavioral relevance. The
humans can detect certain odorants with sensory events that are relevant to one
better acuity than can rats (Laska & Seibt, species may entirely differ from those rel-
2002) and have the ability to track scents evant to another species, but for each and
and roughly mimic the tracking patterns of every creature its sensory perceptual reper-
dogs (Porter et al., 2007). In fact, contrary toire is constrained by the natural real-world
to popular opinion, human olfactory dis- form of the sensory stimulus. This holds
crimination ability is in the same range as for all sensory systems. In the case of an
vision and audition (Mueller, 1951). Hence, odor stimulus, a defining characteristic is
by virtue of its acuity in humans, its ability its multimolecular complexity. For exam-
to evoke strong emotional reactions, and its ple, the odor of chocolate contains many
intrinsic overlap with the limbic emotional hundreds of volatile organic compounds,
system, the olfactory modality provides a yet the olfactory system seamlessly inte-
unique window into emotional processing. grates these disparate elements into a per-
In recent years, the development of ceptual whole, resulting in configural, rather
sophisticated imaging technologies has sig- than elemental odor perception (Gottfried,
nificantly advanced our understanding of 2006). This stimulus composite of differ-
both human olfactory perception and affec- ent odorous molecules comprises an odor
tive neuroscience. This chapter focuses on object. Thus, just as humans perceive the
olfactory hedonic perception and through visual object “chocolate,” a synthetic percept
this focus highlights mechanisms of basic fabricated out of many component features,
emotional processing as well as more com- they also perceive the odor object “choco-
plex aspects such as emotion-cognition late” (Gottfried, 2010).
interactions. It describes how an exami- Odor objects possess many of the same
nation of limbic brain areas involved in properties described for visual objects (Got-
odor perception has resulted in consider- tfried, 2010). For example, when walking
able progress in delineating the neural mech- into a bakery, our olfactory system is capable
EXAMINING EMOTION PERCEPTION AND ELICITATION VIA OLFACTION 243
of filtering out the irrelevant (background) tory nerve) that synapse onto the dendritic
smells so that the chocolate odor stands out endings of mitral and tufted cells within
as an object. This property, known as figure– spheroid units called glomeruli located in
ground segmentation, is evident in visual the olfactory bulb. Each olfactory sensory
and auditory object perception, and its prin- neuron innervates just one or two glomeruli,
ciples equally apply to odor object percep- and each glomerulus is innervated by sen-
tion (Linster, Henry, Kadohisa, & Wilson, sory neurons expressing the same type of
2007). Moreover, although the aroma of receptor (Firestein, 2001). Thus, this pro-
freshly baked chocolate brownies is different cessing stage facilitates a powerful conver-
from the scent of chocolate liqueur, we are gence of input from same-type receptors.
able to retain the “objectness” of chocolate, Patterns of activity within the olfactory bulb
a perceptual property referred to as object grossly reflect discrimination of one odorant
constancy. Extraction of perceptual same- from another, although a finer scale chemo-
ness across different stimuli, or object cate- typic organization has not been well estab-
gorization, is an important aspect of object lished (Gottfried, 2010). Olfactory informa-
constancy and is balanced by the ability to tion is then conveyed via axonal projections
discriminate among individual objects (for from the mitral and tufted cells that form
example, white versus dark chocolate). Both the lateral olfactory tract, terminating in sev-
odor categorization and odor discrimination eral areas in the basal frontal and medial
are well-established properties of olfactory temporal lobes, including the anterior olfac-
perception in the human brain (Howard, tory nucleus, olfactory tubercle, anterior
Plailly, Grueschow, Haynes, & Gottfried, and posterior piriform cortices, medial and
2009; Li, Luxenberg, Parrish, & Gottfried, cortical nuclei of the amygdala, and ros-
2006). In summary, although the visual and tral entorhinal cortex (Figure 10.1). These
olfactory systems have evolved under differ- areas are sometimes collectively referred
ent ecological constraints, the basic princi- to as the “primary olfactory cortex.” It
ples underlying visual object perception are is assumed that here olfactory categori-
also applicable to olfactory object percep- cal information is represented in the form
tion. Like the visual or auditory system, the of odor objects (Gottfried, 2010). Higher
olfactory system is optimized to detect and order projections arising from these olfac-
encode behaviorally salient events (objects) tory structures relay to the orbitofrontal cor-
that are encountered in the real world. How- tex, agranular insula, other amygdala subnu-
ever, compared to visual or auditory objects, clei, thalamus, hypothalamus, basal ganglia,
an overwhelming majority of odor objects and hippocampus (Carmichael et al., 1994).
tend to be salient by virtue of their associa- This complex network of connections pro-
tion to appetitive or aversive consequences. vides the basis for odor-guided modulation
of behavior, feeding, emotion, autonomic
states, and memory. In addition, each region
Anatomy of Olfaction: A Direct of the primary olfactory cortex (apart from
Pathway to Emotion the olfactory tubercle) sends dense feedback
projections to the olfactory bulb (Gottfried,
Olfaction in mammals starts with a sniff. 2006) through which central or “top-down”
This simple motor action primes the olfac- modulation of olfactory information pro-
tory system for the arrival of odors (Sobel cessing can occur as early as the second-
et al., 1998) and physically transports odor- order neurons.
ant molecules to the nose. Binding of an
odorant to the receptors of olfactory sen-
Piriform Cortex
sory neurons in the nasal epithelium initi-
ates the process of signal transduction. Neu- The piriform cortex, named for its pear-
ral information is relayed through the axons shaped structure, is a three-layer paleocor-
of the olfactory sensory neurons (i.e., olfac- tex that lies at the medial junction of the
244 APRAJITA MOHANTY & JAY A. GOTTFRIED
(a) (b)
ob
Ant. ofc
OLF
L
R
lot
otu aon
apc
ob
Figure 10.1. Anatomy of the human olfactory brain. (a) A ventral view of the human brain in which
the right anterior temporal lobe has been resected in the coronal plane to expose the limbic olfactory
areas depicted in panel (b). Afferent output from the olfactory bulb (OB) passes through the lateral
olfactory tract (LOT) and projects monosynaptically to numerous regions, including the anterior
olfactory nucleus (AON), olfactory tubercle (OTUB), anterior piriform cortex (APC), posterior
piriform cortex (PPC), amygdala (AM), and entorhinal cortex (EC). Downstream relays include the
hippocampus (HP) and the putative olfactory projection site in the human orbitofrontal cortex
(OFColf ). As noted in the inset, information is not transferred serially through this circuit.
Monosynaptic projections from the lateral olfactory tract reach numerous downstream regions in
parallel, and these regions are then reciprocally interconnected (not shown). Adapted from Gottfried
(2010). See color plate 10.1.
frontal and temporal lobes. It is the largest that the piriform cortex functions as a sen-
recipient of projections from the olfactory sory association cortex where representa-
bulb. In contrast to the coarse chemotypic tions of individual components are assem-
modular spatial architecture in the olfactory bled into holistic odor objects (Gottfried,
bulb, the piriform cortex shows a more dis- 2010; Howard et al., 2009).
tributed pattern of connectivity (Stettler &
Axel, 2009). It is reciprocally and extensively
Amygdala
connected with several high-order areas of
the cerebral cortex, including the prefrontal, Projections from the olfactory bulb termi-
amygdaloid, perirhinal, and entorhinal cor- nate in several discrete subnuclei situated
tices (Carmichael et al., 1994; Gottfried, in the dorsomedial margin of the amyg-
2006). Some piriform neurons project to dala, including the periamygdaloid region,
more than one of these areas, and neigh- anterior and posterior cortical nuclei, the
boring piriform cells can have highly dis- nucleus of the lateral olfactory tract, and the
similar projection targets. This widely dis- medial nucleus (Carmichael et al., 1994).
tributed pattern of connectivity, with direct The olfactory amygdala is continuous ros-
links to brain regions that mediate cognition, trally with the piriform cortex, and in addi-
emotion, memory, and behavior, suggests tion to sending projections back to the bulb,
EXAMINING EMOTION PERCEPTION AND ELICITATION VIA OLFACTION 245
it provides direct input to other portions of tures that distinguish it from other sen-
the amygdala, including lateral, basolateral, sory modalities. In contrast to the visual
and central amygdaloid nuclei (Pitkänen, and olfactory modalities, odor processing
2000), as well as to the basal ganglia, remains ipsilateral from the nasal periph-
thalamus, hypothalamus, and prefrontal ery all the way to the primary olfactory cor-
cortex. In fact, olfaction is the only sensory tex. Also unlike visual and auditory modal-
modality possessing direct bidirectional pro- ities, transmission of olfactory information
jections between the amygdala and primary to central brain regions, including the pri-
sensory cortex. Not surprisingly, during mary olfactory cortex and neocortical (pre-
inhalation of odorants, the medial amygdala frontal) areas, is achieved without thalamic
shows increased firing rates measured elec- modulation. For example, monosynaptic
trophysiologically in conscious monkeys and projections from the piriform cortex to the
humans, and odor stimulation induces both OFC (Carmichael et al., 1994) ensure that
evoked potentials and oscillatory activity in odor information has access to the neocor-
the human amygdala (Gottfried, 2006). tex without first passing through the tha-
lamus. The absence of a thalamic node
in the processing hierarchy may help pre-
Orbitofrontal Cortex
serve the fidelity of the original olfactory
The orbitofrontal cortex (OFC) is the main percept in the context of unpredictable
neocortical projection of the olfactory cor- shifts in stimulus concentration, background
tex. The OFC is located along the basal smells, and respiratory patterns. Finally, as
surface of the caudal frontal lobes. It noted earlier, there is substantial anatomi-
receives direct afferent input (without thala- cal overlap between the structures involved
mic relay) from the primary olfactory areas, in olfactory processing and those devoted
including the piriform cortex, amygdala, and more generally to emotional processing.
entorhinal cortex. In turn, the OFC pro- The manifestation of this anatomical over-
vides direct feedback projections to each of lap is evident in the role odorous stimuli
these regions. Physiological (Tanabe, Iino, play in motivating almost every aspect of
& Takagi, 1975) and anatomical (Carmichael animal behavior, including maternal bond-
et al., 1994) data in monkeys suggest that, ing, kinship recognition, food search, mate
within the OFC, it is the posterior orbital selection, predator avoidance, and territorial
cortex in areas Iam, Iapm, and 13a that marking.
receives maximum input from the olfactory
areas. However, a meta-analysis of human
neuroimaging studies indicates that the sec- Does Olfaction Function via Emotion?
ondary olfactory areas in OFC appear to be
located more rostral (near area 11l) than the Based on a review of the human olfaction
corresponding sites in monkeys (Gottfried literature, Stevenson has classified olfactory
& Zald, 2005). Finally, it is important to functions into three major categories relat-
note that adjacent, non-overlapping regions ing to (1) ingestive behavior, (2) avoidance
of the OFC receive sensory input from gus- of environmental hazards, and (3) social
tatory and visual centers, as well as informa- communication (Stevenson, 2010). All three
tion about visceral states, providing a neural functions are inextricably linked with emo-
substrate for associative learning and cross- tional evaluation, emphasizing the strong
modal integration (Rolls, 2004) that supports link between olfaction and emotion.
feeding and odor-related behaviors. With regard to ingestive behavior, smell
plays a very important role in the percep-
tion of flavor and the formation of cognitive
Unique Features
and emotional responses to foods. Olfaction
The central organization of the olfactory is described as having a unique “dual nature”
system has several unique anatomical fea- (Rozin, 1982), because it involves sensing
246 APRAJITA MOHANTY & JAY A. GOTTFRIED
externally (orthonasal) and internally (ret- threats signal the presence of pathogens, and
ronasal) originating signals. Orthonasal stim- exposure to disgust-eliciting odors may help
ulation involves sniffing odors in the outside in priming the immune system for a poten-
environment through the external nares of tial microbial attack. In animals and humans,
the nose and is useful in identifying a food’s odors have been shown to function as con-
suitability for ingestion. Retronasal stimu- ditioned cues that stimulate or suppress
lation occurs during food ingestion, when immune responses (see Stevenson, 2010, for
volatile odorous molecules released from a review). The facial-motor response elicited
the food in the mouth are channeled from by disgusting odors, including drawing in the
the back of the oral cavity up through the nostrils, constricting the nostril airway diam-
nasopharynx to the olfactory epithelium. eter, and pinching together the lips, all serve
This retronasal food molecule-laden air is to minimize any further entry of poten-
what imparts “flavor” to the food (Shep- tial toxins into the oronasal cavity (Susskind
herd, 2006). Retronasal stimulation is impor- & Anderson, 2008). The insula appears to
tant in regulating food intake via changes in play an important role in processing dis-
hedonic responsiveness to flavors in a state- gust as evidenced by increased insular acti-
specific manner. An example of these regu- vation after inhaling disgust-eliciting odors
latory effects in the Stevenson (2010) review (Royet, Plailly, Delon-Martin, Kareken, &
includes the “appetizer effect,” when brief Segebarth, 2003; Zald & Pardo, 1997).
exposure to a palatable food makes partic- In Salman Rushdie’s novel Midnight’s
ipants hungrier, resulting in greater subse- Children (Rushdie, 1981), the main protag-
quent consumption of the food. Another onist Saleem inherits an acute sense of
regulatory effect that involves a strong affec- smell that enables him to detect other peo-
tive component is that of sensory-specific ple’s emotions such as “the acrid stench of
satiety. Hedonic ratings of the odor of a his mother’s embarrassment” (p. 20), “the
food eaten to satiety, relative to a control whiff of things concealed mingling with
(uneaten) food odor, show significant nega- the odours of burgeoning romance and the
tive shifts changing from like at the start to sharp stink of my grandmother’s curiosity
dislike at the end of a meal (Rolls & Rolls, and strength” (p. 52), and “the heady but
1997), and human imaging studies have con- quick-fading perfume of new love, and also
sistently revealed parallel changes in the the deeper, longer lasting pungency of hate”
magnitude of activation in the orbitofrontal (p. 298). In contrast to this fictional account,
cortex (Gottfried, O’Doherty, & Dolan, the scientific evidence for the social com-
2003). munication function of odors in humans
Detection of olfactory signals that indi- remains debatable. The role of pheromones
cate danger is very well developed in as a means of interspecies communication
humans. This is evidenced by their impres- in invertebrates is well established; how-
sive ability to detect minute amounts of ever, the existence of pheromonal commu-
volatile agents that signal the presence of nication among mammals, especially human
odorless airborne hazards or by their ability mammals, is not well substantiated in the
to detect and avoid sources that emit chem- scientific literature (Doty, 2010). The emo-
icals resulting from the biological decay pro- tional and behavioral effects of androsta-
cess (Stevenson, 2010). Stevenson divided dienone and estratetraenol, putative human
chemical hazard signals into two functional pheromones found in sweat and saliva,
categories, each associated with a differ- are well studied. Administration of these
ent emotion – nonmicrobial hazards (e.g., steroids has been shown to result in greater
predators, fire, degraded air, and poisons), distraction from emotional information on
which are typically associated with a fear attention tasks and increased sympathetic
response, and microbial threats (e.g., feces, arousal and reports of positive mood in
vomit, bodily odors and organic decay), women and decreased positive mood in
which are associated with disgust. Microbial men (Hummer & McClintock, 2009; Jacob,
EXAMINING EMOTION PERCEPTION AND ELICITATION VIA OLFACTION 247
discriminate individual odor objects (for a CS) was repeatedly paired with a mild elec-
review, see Gottfried, 2010). tric shock (the unconditioned stimulus; Li
There appears to be a functional het- et al., 2008; Sacco & Sacchetti, 2010). After
erogeneity with regard to representation of conditioning, subjects were better able to
valence in the anterior and posterior subdi- discriminate between the previously indis-
visions of the piriform cortex. fMRI stud- tinguishable enantiomers (CS+), but this
ies have shown that the anterior segment was not the case for a control pair of
of the piriform cortex (which also incorpo- odor enantiomers (CS–; Figure 10.2). Olfac-
rates the frontal piriform cortex) is sensitive tory aversive learning was associated with
to valence, showing increased activity for a reorganization of fMRI ensemble activ-
unpleasant and pleasant odors compared to ity patterns in the PPC, specifically for the
neutral odors (Gottfried, Deichmann, Win- enantiomer pair that was used during con-
ston, & Dolan, 2002), as well as increased ditioning.
activity for unpleasant versus pleasant odor- This spatial reorganization of sensory
ants during odor imagery (Bensafi, Sobel, & coding in the piriform cortex may reflect
Khan, 2007) and odor perception (Zelano, changes in olfactory receptive-field tuning,
Montag, Johnson, Khan, & Sobel, 2007). leading to improved perception of odor
Coding of odor valence at such an early stage cues, such that piriform representations
of processing is consistent with behavioral “tagged” with salience might gain privileged
findings showing that humans react faster to access to networks mediating behaviorally
aversive and dangerous odors than to appet- relevant actions. In fact, a recent study in rats
itive odors (Bensafi et al., 2003), which may reported that excitotoxic lesions of the PPC
reflect the survival value in deciding very impaired remote, but not recent, fear mem-
quickly whether the environmental stimu- ories (Sacco & Sacchetti, 2010). Although
lus is noxious or dangerous. It is likely that similar results were found for secondary
this early coding of valence is mediated by auditory and visual cortices, the memory
strong reciprocal connections between ante- impairment was modality specific and was
rior piriform cortex and orbitofrontal struc- not due to an interference with sensory or
tures (Gottfried, Deichmann, et al., 2002). emotional processes. Furthermore, lesions
The receptiveness of the anterior piriform of the same areas left intact the memory
cortex to the hedonic quality of odors stands of sensory stimuli not associated with fear.
in contrast to the insensitivity of the poste- This study further highlights the role of
rior piriform cortex (PPC) to valence; the the piriform cortex (and secondary associa-
PPC seems to be broadly tuned to differ- tive cortices in other modalities) in memory
ent odors irrespective of valence (Gottfried, storage and retrieval of sensory stimuli that
Deichmann, et al., 2002). Although the PPC have acquired behavioral salience as a con-
does not seem to be involved in coding emo- sequence of experience.
tional aspects of odors, by virtue of its exten-
sive connectivity with other limbic regions,
Amygdala
it serves as an important substrate for encod-
ing emotional learning and memory-related A dominant model in emotion research
changes in odor object coding. Its par- posits that emotion is best represented on
ticipation in emotional learning has been a circumplex, or two orthogonal axes (Rus-
demonstrated in animals and humans (Li, sell, 1980). The first axis, referred to as
Howard, Parrish, & Gottfried, 2008; Sacco & the valence axis, represents the pleasant-
Sacchetti, 2010). For example, in an olfactory ness/unpleasantness of emotion, whereas
fMRI study of aversive learning, subjects the second axis represents the degree of
smelled two odor enantiomers, or mirror- arousal associated with the emotion. These
image molecules, before and after an aver- two dimensions are hypothesized to map
sive conditioning session in which one of the onto two independent neurophysiological
two enantiomers (the conditioned stimulus, systems, and every affective experience is
EXAMINING EMOTION PERCEPTION AND ELICITATION VIA OLFACTION 249
0.8 0.9
Pre
(tirangular test)
Pre
0.6
Post 0.8
0.4
Post
0.7
0.2
0.0 0.6
CS+ pair CS– pair CS+ pair CS– pair
Figure 10.2. Role of olfactory cortical areas in aversive learning. In an fMRI version of olfactory fear
learning, subjects smelled pairs of odor enantiomers that were perceptually indistinguishable. One of
these enantiomers, designated the target CS+ (tgCS+), was subsequently paired with mild footshock.
Neither its chiral counterpart (chCS+) nor a control pair (CS- and chCS-) was paired with shock.
From preconditioning to postconditioning, perceptual discrimination between tgCS+ and chCS+
selectively increased (a), in parallel with fMRI ensemble decorrelation (less pattern overlap) in PPC
(b). Ensemble activation maps across PPC voxels in one subject (c) show increasing pattern divergence
between tgCS+ and chCS+ after learning. Adapted from Li et al. (2008). See color plate 10.2.
Strong 0.5
% signal change
Intensity
0.4
0.3
0.2
Moderate
0.1
0
Weak –0.1
Barely
–0.2
detectable
Neutral
Dislike
Like
Str. dislike
Str. like
Mod. dislike
Sl. dislike
Sl. like
Mod. like 0 10 20 0 10 20 0 10 20
Time (seconds)
Valence — High concentration — Low concentration
Figure 10.3. Amygdala codes an interaction of valence and intensity. Experimental design and
behavioral data (A) show that that valence (three levels: pleasant, neutral, or unpleasant) and intensity
(two levels: high or low) were independently manipulated in a 3 × 2 factorial design. Behavioral
ratings are depicted in the plot of psychophysical odor space: valence on the abscissa, intensity on the
ordinate. Sl., Slightly; Mod., moderately; Str., strongly. Dark symbols, high concentration odors; light
symbols, low-concentration odors. Error bars show SEM; vertical bars represent error in intensity
dimension; horizontal bars represent error in valence ratings. (B) fMRI findings show differential
effects of odor valence on intensity coding in the amygdala. Response time courses of amygdala
activation for each level of odor concentration and odor type show the selective effects of intensity on
amygdala activity only at the extremes of odor valence. Adapted from Winston et al. (2005).
and intensity were similarly dissociated, but more, it is debatable whether odor inten-
a neutrally valenced odor was added to sity is the same as odor-elicited arousal. For
the pleasant and unpleasant odor conditions example, peppermint odor was associated
(Winston, Gottfried, Kilner, & Dolan, 2005). with increases in self-rated alertness whereas
Hence, high- and low-intensity versions of ylang-ylang was associated with decreases in
pleasant, neutral, and unpleasant odors were alertness and increases in self-rated calm-
delivered to subjects (Figure 10.3). The ness, despite the odors being equated for
study then examined whether the amyg- intensity (Moss, Hewitt, Moss, & Wesnes,
dala coded intensity regardless of valence 2008). In another study, lemon oil adminis-
(greater activation for high versus low inten- tration resulted in elevated norepinephrine
sity for all three odor types) or a combina- levels compared to water or lavender
tion of valence and intensity (greater activa- (Kiecolt-Glaser et al., 2008) indicating that
tion for high versus low intensity for pleasant odor-elicited arousal may not be equivalent
and unpleasant odors, but not for neutral to odor intensity. Furthermore, the rela-
odor). The study findings were consistent tionship between stimulus-related arousal
with the latter hypothesis, suggesting that and intensity may differ in the chemosen-
the amygdala codes an interaction between sory modality where arousal is linked more
intensity and valence, caring mainly about closely to the physical characteristics of the
the overall behavioral salience of an stimulus (Bensafi et al., 2002a), as opposed
odor. to the auditory and visual domain where the
The problem with the Anderson et al. two are not as closely related (Junghofer,
study (and the Winston et al. study) is Bradley, Elbert, & Lang, 2001). For exam-
that they are not a reflection of real-world ple, judgments of the calming or arous-
valence-arousal relationships. To dissoci- ing nature of visual scenes do not depend
ate valence and intensity, Anderson et al. greatly on the contrast, luminance, or hue
selected odors in a narrow valence and inten- of the scenes. Given the close anatom-
sity range (bordering on neutral). Further- ical connections between the amygdala
EXAMINING EMOTION PERCEPTION AND ELICITATION VIA OLFACTION 251
and olfactory cortices, increased amygdala ple, after a neutral face (conditioned stimu-
activation related to physical stimulus inten- lus, CS) was repetitively paired with either
sity across differently valenced stimuli or a pleasant or unpleasant odor UCS, pre-
specifically for unpleasant or pleasant stim- sentation of the conditioned face by itself
uli (Anderson et al., 2003; Winston et al., elicited dissociable activations in human
2005) accords well with increased neural OFC according to whether the learning
activity in response to physically intense mode was appetitive or aversive, indicat-
visual or auditory stimuli in primary and sec- ing that this region is involved in the
ondary sensory areas (e.g., Mohamed, Pinus, valence-specific establishment of picture-
Faro, Patel, & Tracy, 2002). odor contingencies (Gottfried, O’Doherty,
et al., 2002). Related studies have shown
that the OFC encodes olfactory predic-
Orbitofrontal Cortex
tive value. In an fMRI version of a rein-
The OFC receives inputs from sev- forcer inflation paradigm, subjects were con-
eral modalities, including taste, smell, ditioned to associate two neutral faces with
somatosensory, and visual modalities, and two different aversive odors (Gottfried &
is believed to play a critical role in repre- Dolan, 2004). Subsequently, the aversive
senting the identity and reward value of value of one of the two odors was enhanced
tastes and odors. This places the OFC in the or “inflated” by presenting it at increased
unique position of integrating information, intensity after conditioning. A final scan-
some of which is rewarding or punishing, ning session was conducted during extinc-
from different sensory modalities – a process tion, with the presentation of faces in the
that is critical for associative learning and absence of any odor. Imaging data from
emotional and motivational behavior. The the extinction session revealed greater acti-
OFC has been implicated in olfactory pro- vation in the lateral OFC in response to
cessing in both monkey electrophysiological the face associated with the inflated odor
(Critchley & Rolls, 1996b; Tanabe et al., value, compared to the control face, sug-
1975) and human lesion and imaging (Li et gesting that representations of the original
al, 2010; Zatorre, Jones-Gotman, Evans, & conditioned value persist even as extinction
Meyer, 1992) studies. Single-cell recording ensues. Finally, in a selective satiety fMRI
during olfactory information encoding in study, two arbitrary visual stimuli were pre-
the OFC of monkeys showed that, among sented with different pleasant food smells,
odor-responsive neurons, representation both before and after an olfactory deval-
of the olfactory stimulus depended on its uation procedure in which subjects ate a
association with taste in 35% of neurons, food corresponding to one of the odors until
whereas in the remaining 65%, the represen- satiation. Amygdala and orbitofrontal cor-
tation of the odor stimulus identity was not tex responses evoked by a predictive tar-
influenced by the associated taste (Critchley get stimulus were decreased after devalua-
& Rolls, 1996b). Furthermore, OFC neurons tion, whereas responses to the nondevalued
decreased their responses to the odor of the stimulus were maintained (Gottfried et al.,
food with which the monkey was fed to 2003), indicating that these regions encode
satiety (Critchley & Rolls, 1996a), indicating the current predictive value of reward
that not only do OFC neurons represent representations.
the rewarding stimuli but also that the In summary, by showing that the OFC
reward value of each stimulus is continually is involved in representing odors and their
updated, albeit in a sensory-specific manner. reward value, as well as in learning and
The OFC has also been implicated in modulating associations between visual and
classical (Pavlovian) conditioning in which odor stimuli, olfactory learning studies have
odors are used as unconditioned stimuli helped delineate its role in controlling
(UCS; Gottfried & Dolan, 2004; Gottfried, and adjusting emotional and motivational
O’Doherty, & Dolan, 2002, 2003). For exam- behavior.
252 APRAJITA MOHANTY & JAY A. GOTTFRIED
2.0
∗∗ ∗∗ ∗∗ Minty odors
∗
Woody odors
Citrusy odors
1.5
Mean rating
1.0
0.5
∗
0.0
rry
y
dy
ty
nt
nt
ra
ic
tr u
in
sa
sa
oo
be
flo
sp
m
ci
ea
ea
w
pl
pl
un
Figure 10.4. Familiar odorants are differentiated along their perceptual characteristics rather than
their perceived pleasantness. Ratings of odorants using the Dravnieks 146-item questionnaire were
averaged across specific odor-quality categories and across each of the odorants belonging to a given
category. Minty odors (white bars) and citrusy odors (dark-gray bars) were easily discriminated with
regard to their minty and citrus ratings, but were indistinguishable in terms of their unpleasant and
pleasant ratings.
of differential responses to pleasant and nic responses that are under evolutionary
unpleasant odors across age groups and control (e.g., pheromones and predator or
species. As reviewed in Yeshurun and Sobel host odors) in animals, a large portion of
(2010), neonate facial expressions (e.g., wrin- these responses are acquired via experi-
kling nose, raising upper lip) discrimi- ence and learning. For example, the odor-
nate between a pleasant vanillin odor and ant eugenol (the “clove” odor used in den-
unpleasant butyric acid, and there is some tal cement) was evaluated negatively and
agreement in adult and child ratings of var- elicited autonomic fear responses in patients
ious pure odorants and personal odors. Fur- who were fearful of dental procedures,
thermore, odorants that are rated pleasant but not in patients who were not fearful
by humans are also the ones that mice (Robin, Alaoui-Ismaili, Dittmar, & Vernet-
investigate longer and human subjects sniff Maury, 1998). A novel ambient odor associ-
longer, and odor pleasantness ratings in ated with a negative emotional state had a
humans and investigation time in mice are detrimental impact on behavior when pre-
both correlated with the physicochemical sented at a later time, suggesting that the
properties of the molecules. However, a pre- hedonic evaluation of a novel odor changed
ponderance of the behavioral and neural in accordance with the emotional experi-
data, as reviewed next, indicates that odor ence with which it had been associated
hedonics are highly plastic and depend on (Herz, 2005). An unfamiliar pleasant odor
learning, sensory context, and past expe- paired with a negative emotional experi-
rience (Rouby, Pouliot, & Bensafi, 2009; ence was subsequently perceived as less
Stevenson & Wilson, 2007). pleasant, whereas an unfamiliar unpleas-
ant odor paired with a positive emotional
experience became more acceptable – again
Olfactory Hedonic Plasticity
demonstrating the effect of associative learn-
Although there is evidence of physicochem- ing on odor-hedonic perception (Herz,
ically driven, hard-wired olfactory hedo- 2005).
EXAMINING EMOTION PERCEPTION AND ELICITATION VIA OLFACTION 255
There is evidence from the animal and odorants such as prior exposure, familiar-
human literature indicating that neural ity, and intensity. Pleasantness and famil-
representations of odors can be modified iarity are usually highly correlated. As
through associative learning with appeti- reviewed in Herz (2005), studies show that
tive or aversive stimuli. For example, in infants of mothers who consumed dis-
honeybees learning to discriminate an odor tinctive smelling substances (e.g., garlic,
associated with reward (sucrose) from an alcohol, cigarette smoke) during pregnancy
unrewarded odor, associative learning trans- or lactation showed preferences for these
formed the odor representations, such that odors compared to infants who had not
activity for only the rewarded odor increased been exposed. An odorant, methyl salicy-
in the antennal lobe (olfactory bulb equiv- late (wintergreen), was rated less pleasant in
alent in insects), and there was a decorrela- the United Kingdom, where it is commonly
tion in the activity patterns for the rewarded associated with medication. By contrast, it
and unrewarded odor, making them less was rated more pleasant in United States
similar (Faber, Joerges, & Menzel, 1999). As where it was associated with a candy mint
discussed earlier, following olfactory aver- smell. Generally, familiar odors are rated
sive learning (odor-footshock conditioning) as more pleasant than unfamiliar odors,
human subjects were able to discriminate and pleasant odors are perceived as being
odors that had been originally perceived as more familiar (Ayabe-Kanamura, Saito, Dis-
identical. These behavioral changes were tel, Martinez-Gomez, & Hudson, 1998;
associated with changes in ensemble pat- Moskowitz, Dravnieks, & Klarman, 1976).
terns for the shocked and unshocked odors The relation between familiarity and
in the posterior piriform cortex (cf. Figure pleasantness has been attributed to the
10.2), suggesting that associative learning has exposure effect, which involves an enhance-
a direct influence on how perceptual infor- ment of one’s attitude toward a stimulus
mation about an odor stimulus is repre- after its repeated exposure (Zajonc, 1968).
sented in sensory-specific cortex (Li et al., However, researchers have argued that sim-
2008). ple exposure is not enough to increase liking.
Context also has an important modula- When different odors were exposed equally
tory effect on olfactory hedonic perception. but some as designated targets and others
According to Rouby et al. (2009), odor- as nontargets in an attentional task, only
hedonic judgments are influenced by non- target odors were liked more after expo-
hedonic characteristics of the odor (e.g., sure (Prescott, Kim, & Kim, 2008). Further-
odor intensity and familiarity), character- more, the relationship between pleasant-
istics of the perceiver (e.g., sex, hormonal ness and familiarity may not be as clear-
status, age, emotional state, physiological cut and can vary based on the pleasantness
state), and context of the stimulus or of the odors. For example, one study found
perceiver (e.g., stimulation of other modal- that after 30 minutes of exposure, a pleas-
ities, accompanying verbal information, ant lemon odor was rated less pleasant,
experimental task or instructions, semantic whereas an unpleasant rancid odor was
knowledge, and cultural background). rated as less unpleasant (Cain & Johnson,
The concept of an odor stimulus being 1978). Delplanque and colleagues showed
endowed with intrinsic emotional value that the positive correlation between pleas-
becomes more difficult to reconcile given antness and familiarity was found to be
the influences of stimulus, perceiver, and specific for pleasant, but not unpleasant
context on odor affect. odors. Furthermore, appraisal processes of
novelty/familiarity and pleasantness appear
to proceed in a sequential manner, in which
Stimulus Characteristics
an odor is detected as novel or famil-
Hedonic evaluations of odorants are strongly iar before it is evaluated as pleasant or
influenced by other characteristics of the unpleasant (Delplanque et al., 2009). Finally,
256 APRAJITA MOHANTY & JAY A. GOTTFRIED
another way of examining familiarity and induced via film clips followed by admin-
exposure-related effects is by examining istration of different kinds of odors (Chen
how cultural context affects odor hedonics. & Dalton, 2005). The study found that,
Ayabe-Kanamura et al. (1998) found differ- regardless of emotional state or person-
ences between German and Japanese popu- ality, women responded to the emotion-
lations in pleasantness ratings for the same ally valenced (i.e., pleasant) odorants faster
set of odorants, as well as a positive relation- than the neutral ones. Personality modu-
ship between pleasantness and edibility rat- lated reaction time and perceived intensity,
ings, suggesting that culture-specific expe- such that neurotic and anxious individuals
riences regarding foods may significantly were more responsive to both pleasant and
influence odor perception. unpleasant odorants. Odor hedonics is also
Hedonic judgment and intensity also bear modulated by the physiological state of the
a complex relationship, with odorants show- perceiver; for example, when a food is eaten
ing a positive correlation, negative corre- to satiety (as discussed earlier in this chap-
lation, inverted U-shaped function, or no ter), the reward value of its odor decreases,
relationship at all between pleasantness and an effect that does not generalize to other
intensity (Bensafi et al., 2002a; Distel et al., foods (O’Doherty et al., 2000; Rolls & Rolls,
1999; Doty, 1975; Henion, 1971; Moskowitz, 1997).
et al., 1976). Thus, how intense an odor
is and how pleasant it is judged to be are
Contextual Characteristics
dependent on the particular odor (Figure
10.5). For example, methyl salicylate is neu- Odor perception is strongly influenced by
tral at low intensity, but becomes pleasant accompanying visual and verbal cues. In an
with increasing intensity (positive correla- example of visual cues affecting olfactory
tion); furfural is neutral at low intensity and perception, a strawberry-smelling solution
becomes unpleasant with increasing inten- was rated as smelling stronger when col-
sity (negative correlation); weak benzalde- ored (e.g., red) than when colorless (Zell-
hyde is neutral at low intensity, pleasant at ner & Kautz, 1990). White wine artifi-
medium intensity, and unpleasant at strong cially colored red with an odorless dye
intensity (inverted U-shaped function); and was described as possessing olfactory prop-
geraniol is pleasant throughout most of its erties of red wine (Morrot, Brochet, &
intensity range (no correlation; Doty, 1975). Dubourdieu, 2001). Gottfried and colleagues
used functional imaging to examine the
neural mechanisms involved in the visual
Perceiver Characteristics
modulation of olfactory perception (Got-
Odor-hedonic judgment is influenced by tfried & Dolan, 2003). Behaviorally, they
the characteristics not only of the stimu- found that olfactory detection was faster
lus but also of the perceiver. For exam- and more accurate when odors appeared
ple, hormonal status influences the pleas- in the context of semantically congruent
antness of androstenone odor such that visual cues. This behavioral advantage was
women rate this odor as less unpleasant dur- associated with greater neural activity in
ing their ovulatory phase but as unpleas- the anterior hippocampus and rostrome-
ant during the other phases (Hummel, Gol- dial orbitofrontal cortex, implicating these
lisch, Wildt, & Kobal, 1991). When different regions in the cross-modal integration of
moods were induced via films, androsta- visual and olfactory information.
dienone and estratetraenol (putative human Verbal information also affects hedonic
sex pheromones) increased sexual arousal judgments of the odor. For example, pleas-
after a sexually arousing film, but no effect antness, familiarity, and intensity of com-
was observed after a neutral film (Bensafi, mon odors were rated higher when names
Brown, Khan, Levenson, & Sobel, 2004). were provided compared with when the
In another study, different moods were same odors were evaluated without names
EXAMINING EMOTION PERCEPTION AND ELICITATION VIA OLFACTION 257
0 r = .86 INTENSITY 20
16
PLEASANTNESS (n = 38) INTENSITY
+4 (n = 20) 16
12 (n = 33)
0 12
8
BENZYL 6 8
16 ACETATE 0 4
r = .99
+2
12 INTENSITY 0 0 r = .94
−2 +2
(n = 37) 0
8
−2
−6
4 PLEASANTNESS
−10 (n = 37) −6 PLEASANTNESS
0 r = .75
PLEASANTNESS −10 (n = 32)
+4 (n = 36)
−14
0 GERANIOL
16
INTENSITY
BENZALDEHYDE 12 PHENYL ETHYL
(n = 37)
12 ALCOHOL
INTENSITY 8 8
INTENSITY
8 (n = 39)
4 4 (n = 38)
4 r = .97
0 r = .09 0
0 r = .89 PLEASANTNESS PLEASANTNESS
+2 +4 (n = 37) +4 (n = 35)
0 PLEASANTNESS (n = 36)
–2 0 0
C –40 –35 –30 –25 –20 –15 –10 C–40 –35 –30 –25 –20 –15 –10 C –40 –35 –30 –25 –20 –15 –10
LOG CONCENTRATION
Figure 10.5. Relationship between pleasantness and intensity of odorants. Mean pleasantness and
intensity magnitude ratings (on the ordinate) and log-volume concentration in propylene glycol
diluent (on the abscissa) are depicted for 10 odorants. Pearson product-moment correlations between
pleasantness and intensity estimates across data points differ across odorants. All correlations are
significant beyond the p < .01 level except those for anethole (p > 0.20), eugenol (p < 0.06), and
geraniol (p > 0.20). Adapted from Doty (1975).
(Ayabe-Kanamura et al., 1998; Distel et al., without verbal labels, olfactory evaluations
1999). Unfamiliar odors were rated as more were based on sensation and experiential
pleasant when they were given positive familiarity (Herz, 2003). The same odor was
names (Ayabe-Kanamura, Kikuchi, & Saito, perceived as more pleasant (e.g., “parme-
1997). When verbal information regarding san cheese”) or unpleasant (e.g., “vomit”)
odors was provided to subjects, hedonic depending on the associated verbal label
evaluations of odors were in agreement (Herz & von Clef, 2001). These behavioral
with the connotation of the label regard- findings are substantiated by imaging results.
less of whether an odorant was present. Compared to control air, a test odor (iso-
However, when odorants were presented valeric acid, with a cheesy, sweaty type
258 APRAJITA MOHANTY & JAY A. GOTTFRIED
prior learning and context. Are olfac- nomic nervous system responses to odours:
tory stimuli more susceptible to such The role of pleasantness and arousal. Chem-
higher order modulation than visual and ical Senses, 27(8), 703–9.
auditory stimuli, and if so, what are the Bensafi, M., Rouby, C., Farget, V., Bertrand, B.,
neural mechanisms that implement this Vigouroux, M., & Holley, A. (2002b). Influ-
ence of affective and cognitive judgments on
heightened plasticity?
autonomic parameters during inhalation of
pleasant and unpleasant odors in humans.
Neuroscience Letters, 319(3), 162–66. doi: Pii
Acknowledgments S0304–3940(01)02572–1
Bensafi, M., Rouby, C., Farget, V., Bertrand,
We would like to acknowledge NIH grant B., Vigouroux, M., & Holley, A. (2003).
support from the National Institute on Deaf- Perceptual, affective, and cognitive judg-
ness and Other Communications Disor- ments of odors: Pleasantness and handedness
ders (5R01DC010014 and 5K08DC007653–05 effects. Brain and Cognition, 51(3), 270–75. doi:
to J.A.G.), as well as supplemental NIH 10.1016/S0278–2626(03)00019–8
funding through the American Recovery Bensafi, M., Sobel, N., & Khan, R. M. (2007).
Hedonic-specific activity in piriform cor-
and Reinvestment Act of 2009 (ARRA)
tex during odor imagery mimics that dur-
(3R01DC010014–01S1).
ing odor perception. Journal of Neuro-
physiology, 98(6), 3254–62. doi: 00349.2007
[pii]10.1152/jn.00349.2007
References Berglund, B., Berglund, U., Engen, T., & Ekman,
G. (1973). Multidimensional analysis of 21
Abdi, H. (2002). What can cognitive psy- odors. Scandinavian Journal of Psychology,
chology and sensory evaluation learn from 14(2), 131–37.
each other? Food Quality and Preference, 13, Berglund, H., Lindstrom, P., & Savic, I. (2006).
445–51. Brain response to putative pheromones in
Anderson, A. K., Christoff, K., Stappen, I., lesbian women. Proceedings of the National
Panitz, D., Ghahremani, D. G., Glover, G., Academy of Sciences 103(21), 8269–74. doi:
. . . Sobel, N. (2003). Dissociated neural rep- 0600331103 [pii]10.1073/pnas.0600331103
resentations of intensity and valence in human Cain, W. S., & Johnson, F. (1978). Lability of
olfaction. Nature Neuroscience, 6(2), 196–202. odor pleasantness – Influence of mere expo-
doi: 10.1038/nn1001nn1001 sure. Perception, 7(4), 459–65.
Ayabe-Kanamura, S., Kikuchi, T., & Saito, S. Carmichael, S. T., Clugnet, M. C., & Price,
(1997). Effect of verbal cues on recogni- J. L. (1994). Central olfactory connections in
tion memory and pleasantness evaluation of the macaque monkey. Journal of Comparative
unfamiliar odors. [Empirical Study]. Percep- Neurology, 346(3), 403–34.
tual and Motor Skills, 85(1), 275–85. doi: Chen, D., & Dalton, P. (2005). The effect of
10.2466/pms.85.5.275–285 emotion and personality on olfactory per-
Ayabe-Kanamura, S., Saito, S., Distel, H., ception. Chemical Senses, 30(4), 345–51. doi:
Martinez-Gomez, M., & Hudson, R. (1998). 10.1093/chemse/bji029
Differences and similarities in the perception Chen, D., & Haviland-Jones, J. (2000). Human
of everyday odors – A Japanese-German cross- olfactory communication of emotion. Percep-
cultural study. Olfaction and Taste XII, 855, tual and Motor Skills, 91(3), 771–81.
694–700. Chrea, C., Grandjean, D., Delplanque, S.,
Bensafi, M., Brown, W. M., Khan, R., Leven- Cayeux, I., Le Calve, B., Aymard, L., &
son, B., & Sobel, N. (2004). Sniffing human Scherer, K. R. (2009). Mapping the semantic
sex-steroid derived compounds modulates space for the subjective experience of emo-
mood, memory and autonomic nervous sys- tional responses to odors. Chemical Senses,
tem function in specific behavioral contexts. 34(1), 49–62. doi: 10.1093/chemse/bjn052
Behavioural Brain Research, 152(1), 11–22. doi: Clark, C. C., & Lawless, H. T. (1994). Limiting
DOI 10.1016/j.bbr.2003.09.009 response alternatives in time-intensity scaling:
Bensafi, M., Rouby, C., Farget, V., Bertrand, B., An examination of the halo-dumping effect.
Vigouroux, M., & Holley, A. (2002a). Auto- Chemical Senses, 19(6), 583–94.
EXAMINING EMOTION PERCEPTION AND ELICITATION VIA OLFACTION 261
Critchley, H. D., & Rolls, E. T. (1996a). Hunger Neuroscience, 11(9), 628–41. doi: nrn2883
and satiety modify the responses of olfac- [pii]10.1038/nrn2883
tory and visual neurons in the primate Gottfried, J. A., Deichmann, R., Winston, J. S., &
orbitofrontal cortex. Journal of Neurophysiol- Dolan, R. J. (2002). Functional heterogeneity
ogy, 75(4), 1673–86. in human olfactory cortex: An event-related
Critchley, H. D., & Rolls, E. T. (1996b). functional magnetic resonance imaging study.
Olfactory neuronal responses in the primate Journal of Neuroscience, 22(24), 10819–28.
orbitofrontal cortex: Analysis in an olfactory Gottfried, J. A., & Dolan, R. J. (2003). The nose
discrimination task. Journal of Neurophysiol- smells what the eye sees: Crossmodal visual
ogy, 75(4), 1659–72. facilitation of human olfactory perception.
de Araujo, I. E., Rolls, E. T., Velazco, M. I., Neuron, 39(2), 375–86. doi: S0896627303003921
Margot, C., & Cayeux, I. (2005). Cogni- [pii]
tive modulation of olfactory processing. Neu- Gottfried, J. A., & Dolan, R. J. (2004).
ron, 46(4), 671–79. doi: S0896–6273(05)00357–0 Human orbitofrontal cortex mediates extinc-
[pii]10.1016/j.neuron.2005.04.021 tion learning while accessing conditioned
Delplanque, S., Grandjean, D., Chrea, C., Cop- representations of value. Nature Neuro-
pin, G., Aymard, L., Cayeux, I., . . . Scherer, science, 7(10), 1144–52. doi: 10.1038/nn1314nn1314
K. R. (2009). Sequential unfolding of nov- [pii]
elty and pleasantness appraisals of odors: Evi- Gottfried, J. A., O’Doherty, J., & Dolan, R. J.
dence from facial electromyography and auto- (2002). Appetitive and aversive olfactory
nomic reactions. Emotion, 9(3), 316–28. doi: learning in humans studied using event-
10.1037/A0015369 related functional magnetic resonance imag-
Distel, H., Ayabe-Kanamura, S., Martinez- ing. Journal of Neuroscience, 22(24), 10829–37.
Gomez, M., Schicker, I., Kobayakawa, T., Gottfried, J. A., O’Doherty, J., & Dolan, R.
Saito, S., & Hudson, R. (1999). Perception of J. (2003). Encoding predictive reward value
everyday odors – Correlation between inten- in human amygdala and orbitofrontal cor-
sity, familiarity and strength of hedonic judge- tex. Science, 301(5636), 1104–7. doi: 10.1126/sci-
ment. Chemical Senses, 24(2), 191–99. ence.1087919301/5636/1104 [pii]
Doty, R. L. (1975). Examination of relationships Gottfried, J. A., Smith, A. P., Rugg, M. D., &
between pleasantness, intensity, and concen- Dolan, R. J. (2004). Remembrance of odors
tration of 10 odorous stimuli. Perception & Psy- past: Human olfactory cortex in cross-modal
chophysics, 17(5), 492–96. recognition memory. Neuron, 42(4), 687–95.
Doty, R. L. (2010). The great pheromone myth. Bal- doi: S0896627304002703 [pii]
timore: Johns Hopkins University Press. Gottfried, J. A., & Zald, D. H. (2005).
Dravnieks, A. (1982). Odor quality: Semantically On the scent of human olfactory
generated multidimensional profiles are sta- orbitofrontal cortex: Meta-analysis and
ble. Science, 218(4574), 799–801. comparison to non-human primates.
Ekman, P. (1992). An argument for basic emo- Brain Research: Brain Research Reviews,
tions. Cognition and Emotion, 6(3/4), 169–200. 50(2), 287–304. doi: S0165–0173(05)00118–9
Ekman, P., & Friesen, W. (1976). Pictures of facial [pii]10.1016/j.brainresrev.2005.08.004
affect. Palo Alto, CA Consulting Psychologists Grabenhorst, F., & Rolls, E. T. (2009).
Press. Different representations of relative and
Faber, T., Joerges, J., & Menzel, R. (1999). Asso- absolute subjective value in the human
ciative learning modifies neural representa- brain. Neuroimage, 48(1), 258–68. doi: DOI
tions of odors in the insect brain. Nature Neu- 10.1016/j.neuroimage.2009.06.045
roscience, 2(1), 74–78. doi: 10.1038/4576 Hattie. J. A. (1985). Methodology review: Asses-
Firestein, S. (2001). How the olfactory system sing unidimensionality of tests and items. [Lit-
makes sense of scents. Nature, 413(6852), 211– erature Review]. Applied Psychological Mea-
18. surement, 9(2), 139–64. doi: 10.1177/01466216-
Gottfried, J. A. (2006). Smell: Central ner- 8500900204
vous processing. Advances in Otorhinolaryn- Henion, K. E. (1971). Odor pleasantness and
gology, 63, 44–69. doi: 10.1159/000093750 intensity: A single dimension? Journal of
[pii]10.1159/000093750 Experimental Psychology, 90(2), 275–79.
Gottfried, J. A. (2010). Central mechanisms Herz, R. S. (2003). The effect of verbal con-
of odour object perception. Nature Reviews text on olfactory perception. Journal of
262 APRAJITA MOHANTY & JAY A. GOTTFRIED
Journal of Magnetic Resonance Imaging, 16(2), lular, and Evolutionary Biology, 281(1), 1212–25.
128–36. doi: 10.1002/jmri.10155 doi: 10.1002/ar.a.20126
Mohanty, A., Howard, J. D., Phillips, K. M., Rolls, E. T., Kringelbach, M. L., & de Araujo,
Wu, K. N., Zelano, C., & Gottfried, J. A. I. E. T. (2003). Different representations of
(2010). Contextual modulation of odor valence pleasant and unpleasant odours in the human
coding. Paper presented at the Association for brain. European Journal of Neuroscience,
Chemoreception Sciences, St. Pete Beach, FL. 18(3), 695–703. doi: 10.1046/j.1460–9568.2003.
Morrot, G., Brochet, F., & Dubourdieu, D. 02779.x
(2001). The color of odors. Brain and Lan- Rolls, E. T., & Rolls, J. H. (1997). Olfactory
guage, 79(2), 309–20. sensory-specific satiety in humans. Physiology
Moskowitz, H. R., Dravnieks, A., & Klarman, L. & Behavior, 61(3), 461–73.
A. (1976). Odor intensity and pleasantness for Rouby, C., Pouliot, S., & Bensafi, M. (2009).
a diverse set of odorants. Perception & Psy- Odor hedonics and their modulators. Food
chophysics, 19(2), 122–28. Quality and Preference, 20(8), 545–49. doi:
Moss, M., Hewitt, S., Moss, L., & Wesnes, 10.1016/j.foodqual.2009.05.004
K. (2008). Modulation of cognitive perfor- Royet, J. P., Plailly, J., Delon-Martin, C.,
mance and mood by aromas of pepper- Kareken, D. A., & Segebarth, C. (2003). fMRI
mint and ylang-ylang. International Journal of emotional responses to odors: Influence of
of Neuroscience, 118(1), 59–77. doi: 787459816 hedonic valence and judgment, handedness,
[pii]10.1080/00207450601042094 and gender. Neuroimage, 20(2), 713–28. doi:
Mueller, C. G. (1951). Frequency of seeing func- 10.1016/S1053–8119(03)00388–4
tions for intensity discrimination at various Rozin, P. (1982). Taste-smell confusions and the
levels of adapting intensity. Journal of General duality of the olfactory sense. Perception & Psy-
Physiology, 34, 463–74. doi: 10.1085/jgp.34.4.463 chophysics, 31(4), 397–401.
Mujica-Parodi, L. R., Strey, H. H., Frederick, B., Rushdie, S. (1981). Midnight’s children [a novel]
Savoy, R., Cox, D., Botanov, Y., . . . Weber, (1st American ed.). New York: Knopf.
J. (2009). Chemosensory cues to conspecific Russell, J. A. (1980). A circumplex model of
emotional stress activate amygdala in humans. affect. Journal of Personality and Social Psy-
PloS One, 4(7). doi: Artn E6415oi 10.1371/Jour- chology, 39(6), 1161–78. doi: 10.1037/h0077714
nal.Pone.0006415 Sacco, T., & Sacchetti, B. (2010). Role of
Pitkänen, A. (2000). Connectivity of the rat secondary sensory cortices in emotional
amygdaloid complex. In J. P. Aggleton (Ed.), memory storage and retrieval in rats. Sci-
The amygdala: A functional analysis (2nd ed., ence, 329(5992), 649–56. doi: 329/5992/649
pp. 31–116). Oxford: Oxford University Press. [pii]10.1126/science.1183165
Porter, J., Craven, B., Khan, R. M., Chang, Schiffman, S. S. (1974). Physicochemical corre-
S. J., Kang, I., Judkewicz, B., . . . Sobel, lates of olfactory quality. Science, 185(4146),
N. (2007). Mechanisms of scent-tracking in 112–17.
humans. Nature Neuroscience, 10(1), 27–29. Schiffman, S. S., & Miller, E. A. S. (1995).
doi: 10.1038/Nn1819 The effect of environmental odors emanat-
Prescott, J., Kim, H., & Kim, K. O. (2008). Cog- ing from commercial swine operations on the
nitive mediation of hedonic changes to odors mood of nearby residents. [Empirical Study].
following exposure. Chemosensory Perception, Brain Research Bulletin, 37(4), 369–75. doi:
1(1), 2–8. doi: 10.1007/s12078–007-9004-y 10.1016/0361–9230(95)00015–1
Proust, M. (1919). Du cote de chez Swann (3rd Shepherd, G. M. (2004). The human sense of
ed.). Paris: Gallimard, Editions de la Nouvelle smell: Are we better than we think? PLoS Biol,
Revue Francaise. 2(5), E146. doi: 10.1371/journal.pbio.0020146
Robin, O., Alaoui-Ismaili, O., Dittmar, A., & Shepherd, G. M. (2006). Smell images and the
Vernet-Maury, E. (1998). Emotional responses flavour system in the human brain. Nature,
evoked by dental odors: An evaluation from 444(7117), 316–21. doi: 10.1038/Nature05405
autonomic parameters. Journal of Dental Sobel, N., Prabhakaran, V., Desmond, J. E.,
Research, 77(8), 1638–46. Glover, G. H., Goode, R. L., Sullivan, E.
Rolls, E. T. (2004). Convergence of sensory sys- V., & Gabrieli, J. D. (1998). Sniffing and
tems in the orbitofrontal cortex in primates smelling: Separate subsystems in the human
and brain design for emotion. Anatomical olfactory cortex. Nature, 392(6673), 282–86.
Record: Part A. Discoveries in Mollecular, Cel- doi: 10.1038/32654
264 APRAJITA MOHANTY & JAY A. GOTTFRIED
Stettler, D. D., & Axel, R. (2009). Represen- sional odors to unidimensional odor objects.
tations of odor in the piriform cortex. Neu- Annual Review of Psychology, 61, 219–241, C211–
ron, 63(6), 854–64. doi: S0896–6273(09)00684– 15. doi: 10.1146/annurev.psych.60.110707.163639
9 [pii]10.1016/j.neuron.2009.09.005 Zajonc, R. B. (1968). Attitudinal effects of mere
Stevenson, R. J. (2010). An initial evaluation of exposure. Journal of Personality and Social Psy-
the functions of human olfaction. Chemical chology, 9(2 Pt. 2), 1–27.
Senses, 35(1), 3–20. doi: 10.1093/chemse/bjp083 Zald, D. H., & Pardo, J. V. (1997). Emotion,
Stevenson, R. J., & Wilson, D. A. (2007). Odour olfaction, and the human amygdala: Amyg-
perception: An object-recognition approach. dala activation during aversive olfactory stim-
Perception, 36(12), 1821–33. doi: 10.1068/P5563 ulation. Proceedings of the National Academy of
Susskind, J. M., & Anderson, A. K. (2008). Facial Sciences, 94(8), 4119–24.
expression form and function. Communicative Zatorre, R. J., Jones-Gotman, M., Evans, A. C.,
and Integrative Biology, 1(2), 148–149. & Meyer, E. (1992). Functional localization
Tanabe, T., Iino, M., & Takagi, S. F. (1975). and lateralization of human olfactory cortex.
Discrimination of odors in olfactory bulb, Nature, 360(6402), 339–40.
pyriform-amygdaloid areas, and orbitofrontal Zelano, C., Montag, J., Johnson, B., Khan, R.,
cortex of the monkey. Journal of Neurophysi- & Sobel, N. (2007). Dissociated represen-
ology, 38(5), 1284–96. tations of irritation and valence in human
Winston, J. S., Gottfried, J. A., Kilner, J. M., primary olfactory cortex. Journal of Neu-
& Dolan, R. J. (2005). Integrated neu- rophysiology, 97(3), 1969–76. doi: 01122.2006
ral representations of odor intensity and [pii]10.1152/jn.01122.2006
affective valence in human amygdala. Jour- Zellner, D. A., & Kautz, M. A. (1990). Color
nal of Neuroscience, 25(39), 8903–7. doi: affects perceived odor intensity. Journal
10.1523/Jneurosci.1569–05.2005 of Experimental Psychology-Human Perception
Yarita, H., Iino, M., Tanabe, T., Kogure, S., & and Performance, 16(2), 391–97.
Takagi, S. F. (1980). A transthalamic olfactory Zhou, W., & Chen, D. (2009). Fear-related
pathway to orbitofrontal cortex in the mon- chemosignals modulate recognition of fear
key. Journal of Neurophysiology, 43(1), 69–85. in ambiguous facial expressions. Psychological
Yeshurun, Y., & Sobel, N. (2010). An odor is not Science, 20(2), 177–83. doi: DOI 10.1111/j.1467–
worth a thousand words: From multidimen- 9280.2009.02263.
CHAPTER 11
Emotional Voices
The Tone of (True) Feelings
Although frequently associated with speech ent message from both sentences based on
and language, the human voice provides the vocal cues that accompanied the verbal
a wealth of information beyond spoken content.
words. Whether in talking or singing, Yet how do human beings infer emotional
screaming or laughing, the sheer sound of meaning from vocal signals? To address this
a voice may not only divulge information question, this chapter reviews research on
about a person’s age, gender or origin but cerebral processes that contribute to the
may also reveal a person’s current affective decoding of emotions from vocal cues such
state to listeners (Belin, Fecteau, & Bedard, as speech prosody or nonverbal vocalizations
2004). Voice-based cues such as the tone like laughter.
of voice while speaking (i.e., prosody) or
laughter, for instance, offer powerful means
to express and decipher emotional mean- Neurobiological Correlates of Prosody
ing alongside spoken language. In our day- Processing
to-day experiences, it is often such nonver-
bal emotional information carried by the In speech communication, information
human voice that “speaks volumes.” Just pic- transfer does not solely depend on the mean-
ture the following situation: You have asked ing of words we use. In fact, often the man-
two colleagues to review a book chapter that ner in which we say something is much
you have written. One concludes his review more revealing than what is actually said.
with the sentence, “Your chapter is really In particular, information about a speaker’s
great,” spoken in a praising tone of voice. current affective state may predominantly
The other, however, passes his judgment be conveyed by the sound of the speaker’s
using the same sentence spoken in a scorn- voice rather than vocabulary (Mehrabian &
ful, taunting tone of voice. Although both Ferris, 1967; Mehrabian & Wiener, 1967).
colleagues used the exact same words, most Here is a simple example: Even when we
listeners will probably derive a totally differ- cannot understand the semantic content of a
265
266 CAROLIN BRÜCK , BENJAMIN KREIFELTS, THOMAS ETHOFER, & DIRK WILDGRUBER
verbal message presented to us, because the decades of research since have shifted
speaker is using a foreign language or dialect, concepts away from the idea of a single
we are still adept in inferring the emotional processing center toward models regarding
state of our interaction partner just by lis- a more widespread cerebral network as the
tening to the tone of his or her voice (Pell, neurobiological basis of prosody compre-
Monetta, Paulmann, & Kotz, 2009). hension (Ackermann, Hertrich, Grodd, &
Whether happy or sad, enraged or sur- Wildgruber, 2004; Schirmer & Kotz, 2006;
prised, each emotional state expresses itself Wildgruber, Ackermann, Kreifelts, &
in a characteristic prosody of speech – a typ- Ethofer, 2006; Wildgruber, Ethofer, Grand-
ical way of speaking mediated by distinc- jean, & Kreifelts, 2009). Several brain
tive modulations of vocal parameters such structures have been suggested as con-
as voice pitch, voice quality, loudness, or stituents of a “prosody network”: Superior
speech rhythm (Banse & Scherer, 1996). temporal as well inferior frontal and subcor-
When happy, for instance, our voices tend to tical structures such as the amygdala have
rise in pitch; we are inclined to speak louder consistently been related to prosody pro-
and perhaps even faster. When sad, how- cessing (Ackermann et al., 2004; Schirmer &
ever, we speak quietly in a low-pitch voice Kotz, 2006; Wildgruber et al., 2006, 2009).
at a slowed pace (Banse & Scherer, 1996; Beyond the mere identification of a
Juslin & Scherer, 2005). Prosodic markers of “prosody network,” neuroscientific studies
a speaking voice not only “support words but have helped characterize the specific func-
give them life” (Krapf, 2007, p. 33). They are tional properties of several brain regions
not “some postscript added on to the verbal implicated in prosody perception. Task-
message” (Krapf, 2007, p. 58), but rather in dependent as well as stimulus-driven response
themselves a rich source of information. profiles have been described for different
Listeners in turn are able to perceive structures associated with prosody process-
and use prosodic acoustic cues with remark- ing, and building on the respective charac-
able ease and accuracy when inferring the terizations, hypotheses regarding the func-
emotional state of a speaker. Current meta- tional roles of these structures have been
analytical reviews (Juslin & Laukka, 2003) developed in the literature (e.g., reviewed
have noted accuracy rates of decoding in Wildgruber et al., 2009).
well above chance level for various emo-
tions, with the highest decoding accuracies
Stimulus-Driven Activation Associated
observed for prosodic expressions of anger
with the Processing of Affective
and sadness, followed by fear and happiness,
Prosody
and the lowest scores obtained for vocal
expressions of love and tenderness. With the advent of modern brain imag-
But which brain processes contribute to ing techniques, research has achieved sub-
the decoding of prosodic signals? Early neu- stantial progress in delineating the neu-
roanatomical models (Ross, 1981) identify robiological bases of (emotional) voice
prosody processing as a function dominant perception. In particular, functional mag-
to the right cerebral hemisphere whose netic resonance imaging (fMRI) has con-
organization in the brain closely mirrors tributed greatly to our understanding of how
the left-lateralized cerebral representation the brain processes emotional information
of language production and comprehen- in human voices. Over the years numerous
sion. These early models assumed that studies pertaining to the field of emotional
prosody production was mediated by a voice perception have been published. In
region within the right inferior frontal reviewing this body of research, one consis-
cortex corresponding in location to Broca’s tent pattern of results appears to emerge:
area while prosody comprehension relied Across studies, findings suggest a contri-
on right superior temporal structures bution of the mid-superior temporal cor-
homologous to Wernicke’s area. However, tex (m-STC) to the decoding of emotional
EMOTIONAL VOICES 267
speech cues (Figure 11.1). Empirical evi- ysis of voices relevant to a rich set of
dence to support this claim can be derived perceptual abilities in a variety of differ-
from a variety of imaging studies indicat- ent contexts or settings (Belin et al., 2000;
ing enhanced responding of the m-STC to Campanella & Belin, 2007). Indeed, the
emotional voices (e.g., Ethofer, Kreifelts, involvement of temporal voice areas has
et al., 2009; Ethofer et al., 2007; Grand- been suggested for several voice perception
jean et al., 2005; Wiethoff et al., 2008), with skills, such as speaker’s identification (von
increases in m-STC activation further linked Kriegstein, Kleinschmidt, Sterzer, & Giraud,
to increases in the emotional intensity of a 2005), voice imagery (Linden et al., 2010),
speaker’s voice (Ethofer, Anders, Wiethoff, or the extraction of emotional information
et al., 2006) and the behavioral relevance of from laughter or speech prosody (Ethofer,
a given prosodic signal (Ethofer et al., 2007). Van De Ville, Scherer, & Vuilleumier, 2009;
Activation maxima have been pinpointed to Grandjean et al., 2005; Szameitat et al., 2010).
aspects of the m-STC frequently suggested In addition to a modulation of response
to play a crucial role in the processing of amplitudes, different prosodic emotional
human voices regardless of emotional con- categories have been linked to distinc-
notation (Ethofer et al., 2012): Studies con- tive spatial activation patterns within the
trasting activation patterns associated with voice-sensitive m-STC. Emotions such as
the perception of human vocal sounds (e.g., anger, sadness, joy, or relief each elicit a
speech, laughter, coughs, cries) to brain specific response that permits a differen-
responses induced by other natural sound tiation of emotional categories based on
structures, such as animal cries or musical activation data of temporal voice areas
or machine sounds, consistently indicate a (Ethofer et al., 2009). Neuroimaging studies
heightened sensitivity to voices for cortical further underline that responses within the
regions located along the middle parts of m-STC occur irrespective of task demands
the right and left superior temporal cortex (Ethofer, Anders, Wiethoff, et al., 2006)
(e.g., Belin & Zatorre, 2003; Belin, Zatorre, or spatial attention focus (Grandjean et
& Ahad, 2002; Belin, Zatorre, Lafaille, Ahad, al., 2005). Ethofer and colleagues (Ethofer,
& Pike, 2000; Figure 11.1). Anders, Wiethoff, et al., 2006), for instance,
Analogous to the fusiform face area, demonstrate that the presentation of happy
a specialized region for human face or angry speech prosody enhances m-STC
perception (Kanwisher, McDermott, & activation, regardless of whether the sub-
Chun, 1997), regions of heightened voice- ject’s attention was directed away or toward
sensitivity – or temporal voice areas (Belin prosodic cues. Aiming to delineate cerebral
& Grosbras, 2010) – may represent a pro- responses of the associative auditory cortex,
cessing module that subserves the anal- Ethofer et al. recorded brain activations of
268 CAROLIN BRÜCK , BENJAMIN KREIFELTS, THOMAS ETHOFER, & DIRK WILDGRUBER
24 healthy young volunteers while process- a broad range of prosodic emotional cues
ing digital recordings of single adjectives might reflect processing stages related to the
spoken either in a happy, neutral, or angry perceptual analysis of acoustic voice prop-
tone of voice. After each stimulus presenta- erties (e.g., Ethofer, Kreifelts, et al., 2009;
tion, participants were asked either to judge Wildgruber et al., 2009). Moreover, based
the emotional states conveyed by the tone on the observation that m-STC responses
of a speaker’s voice or classify each stimulus could only be explained by a combination
according to its word content (positive, of several acoustic features, a contribution
negative, or neutral meaning of the word). of this structure to the analysis of complex
Data analysis indicated that, both during acoustic patterns related to speech prosody
the evaluation of speech prosody and during has been assumed (Wiethoff et al., 2008).
word content judgment, stimuli spoken
with an angry or happy tone of voice
Task-Dependent Activation Associated
elicited stronger responses of the m-STC
with the Processing of Affective
as compared to neutrally spoken stimuli.
Prosody
In other words, regardless of whether a
participant’s attention was focused on Whereas m-STC activation to emotional
the decoding of speech prosody or word prosody proved to be independent of task
content, the perception of stimuli spoken demands or attention focus, response pat-
with an emotional tone of voice was terns of several other structures implicated
associated with an enhanced activation of in prosody processing have been reported to
temporal voice areas relative to activation be task related and linked to task instruc-
patterns obtained for neutrally spoken tions requiring participants to focus atten-
stimuli. tion on naming or labeling vocally expressed
Findings of task independence, in turn, emotions. Task-dependent response charac-
suggest that activation of the m-STC might teristics, for instance, have been described
not be mediated by specific cognitive for several activation clusters observed
demands or attention focus, but rather be within the frontal and temporal cortex
stimulus-driven in nature and linked to (Ethofer, Anders, Erb, Herbert, et al., 2006;
basic acoustic features that define prosodic Ethofer, Kreifelts, et al., 2009; - Mitchell,
markers irrespective of context. Indeed, Elliott, Barry, Cruttenden, & Woodruff,
strong correlations between hemodynamic 2003; Quadflieg, Mohr, Mentzel, Miltner,
responses within the right m-STC and & Straube, 2008; Wildgruber et al., 2004).
several acoustic features, such as mean For example, Wildgruber and colleagues
intensity, mean fundamental frequency, or (2005) found task-related increases in acti-
stimulus duration, have been reported in vation for brain structures located within
the literature (Wiethoff et al., 2008). How- the right posterior superior temporal cor-
ever, when analyzed for their individual con- tex (p-STC) as well as the right inferior
tribution, neither stimulus intensity, fun- frontal cortex (IFC). Using fMRI, Wildgru-
damental frequency, nor duration alone ber et al. obtained brain activation patterns
sufficed to explain the observed activation of healthy young adults while processing
patterns. Rather only by modeling the com- emotional speech stimuli. Stimulus mate-
bined effect of all acoustic parameters could rial comprised short German sentences spo-
increases in m-STC activation during the ken either in a happy, sad, disgusted, fear-
processing of affective prosody be predicted ful, or an angry tone of voice. Participants
sufficiently (Wiethoff et al., 2008). were asked to perform two different tasks:
Such findings suggesting that m-STC One task required participants to label the
responses are driven by acoustic parame- emotion expressed by the speakers’ tone
ters that index emotions at a signal level, of voice (identification task), whereas the
in turn, have led to the assumption that the other asked them to name the first vowel
activation of the voice-sensitive m-STC to following the first /a/ in each presented
EMOTIONAL VOICES 269
sentence (control task). Systematic compar- bral level, this process of retrieving and
isons computed between brain activation integrating matching facial cues might not
patterns associated with both experimen- only be mediated by the p-STC itself but
tal conditions allowed the delineation of might also rely on modality- specific pro-
cerebral structures implicated in the respec- cessing modules, such as the fusiform face
tive tasks. Results revealed that, relative to area (FFA; Kanwisher et al., 1997), which
vowel identification, the explicit evaluation in addition to being implicated in percep-
of prosodic emotional cues was associated tual processes, has also been associated with
with enhanced activation of the right pos- the mental imagery of faces (O’Craven &
terior temporal cortex and the right inferior Kanwisher, 2000). Proceeding from this
frontal cortex. observation, one might assume that “images”
Building on the observation that similar triggered by listening to emotional voices are
prefrontal areas have consistently been sug- supported by activation of the FFA, and via
gested to subserve working memory (Chein, a complex interplay between this modality-
Ravizza, & Fiez, 2003; D’Esposito et al., specific processing module and integration
1998), prosody processing related responses sites within the p-STC, mental images are
within the frontal cortex might be assumed combined with perceived voice cues, thus
to reflect an engagement of working mem- maximizing the information available. How-
ory processes relevant to the decoding of ever, at this point of time this notion remains
affective speech cues (Mitchell, 2007). hypothetical in nature, and future research
Considering the contribution of the is needed to evaluate the claims.
p-STC, reports of involvement of this struc-
ture in the audiovisual integration of non-
Task-Dependent Deactivation: Implicit
verbal emotional signals (see the section,
vs. Explicit Processing
“Emotional Voices in Context: Audiovi-
sual Integration of Emotional Signals”), have In addition to increases in activation, task
led to the assumption that p-STC activa- instructions requiring participants to focus
tion reflects stages of multimodal binding, attention on the evaluation of prosodic
which in turn may aid the identification of speech cues might also be linked to a
emotional information (Wildgruber et al., deactivation of specific brain areas. Partic-
2009). But how might multimodal integra- ularly, response patterns observed for sub-
tion relate to the evaluation of (unimodal) cortical limbic regions such as the amygdala
prosodic cues? One may assume that, even provide the empirical foundation for claims
in situations where no sensory input from of task-related decreases in hemodynamic
other channels of communication is avail- responding: Considering the contribution of
able, additional cues might be retrieved from the amygdala to prosody decoding, seem-
memory on the basis of established mul- ingly contradictory empirical evidence has
timodal associations, and through recalling been presented in the literature: Reports
and integrating those retrieved cues with of the amydala’s increased (Wiethoff
presented prosodic signals, additional infor- et al., 2009), decreased (Morris, Scott, &
mation becomes available on which to base Dolan, 1999), or no “critical” (Adolphs &
emotional inferences. In other words, hear- Tranel, 1999) involvement during the pro-
ing emotional voices during, for instance, cessing of vocally expressed emotions leave
a phone conversation might cue memories its exact role in emotional voice decod-
of matching facial expressions that one has ing an issue of debate. However, across
encountered before in a similar context. Lis- studies a pattern of results has begun to
tening to a happy voice might thus, for emerge that might allow the resolution
example, almost automatically evoke men- of some inconsistencies in the literature:
tal images of smiling faces, because those Enhanced activation of the amygdala often
particular facial and vocal signals have often appears to be associated with task conditions
been closely linked in the past. At the cere- requiring a more pre-attentive or implicit1
270 CAROLIN BRÜCK , BENJAMIN KREIFELTS, THOMAS ETHOFER, & DIRK WILDGRUBER
processing of prosodic signals (Bach et al., terns also reflect cognitive processes associ-
2008; Ethofer, Anders, Erb, Droll, et al., ated with the task used to distract attention
2006; Wiethoff et al., 2009), whereas stud- away from prosodic speech cues?
ies focusing on the explicit2 evaluation of A second approach to study implicit pro-
vocally expressed emotions rarely report cessing that circumvents the issue of com-
increases in amygdala activation, but rather peting task effects might be provided by pas-
indicate strong frontal responses (Ethofer, sive listening designs that present prosodic
Anders, Erb, Herbert, et al., 2006; Wildgru- cues without any instructions but to listen
ber et al., 2004, 2005). Based on these obser- to each speech sample. Given that attention
vations one might conclude that, whereas is not directed toward the decoding of emo-
processing prosodic signals outside of atten- tional signals, passive listening might be con-
tion focus may result in an increased sidered to “model” implicit processing con-
response from limbic areas, explicit atten- ditions. However, the lack of a behavioral
tional processing might, in comparison, lead control to ascertain the exact focus of the lis-
to a deactivation of limbic structures. The tener’s attention prevents a straightforward
latter assumption is corroborated by find- interpretation of results. Even in the absence
ings suggesting that frontal brain structures of specific instructions subjects might focus
inhibit limbic responses during the perfor- their attention on emotional signals pre-
mance of demanding cognitive tasks (Blair sented by the stimuli. Moreover, if sub-
et al., 2007; Mitchell et al., 2007). jects realize that modulations of emotional
However, research on the implicit pro- valence and intensity mark a prominent fea-
cessing of prosodic signals (needed to sub- ture of the employed stimulus material, they
stantiate the aforementioned hypothesis) might even start to create an explicit emo-
struggles with methodological difficulties tional discrimination task by themselves.
that limit the interpretation of obtained With respect to the outlined difficulties
results. The most important methodolog- associated with both experimental designs,
ical challenge associated with this line of the most convincing approach, when aim-
research may be summarized in a sim- ing to identify neurobiological correlates of
ple question: How can one operationalize implicit processing, might be to evaluate
implicit processing in experiments in a way the convergence of results obtained during
that allows delineating cerebral brain struc- both “distracter tasks” and passive listening
tures selectively responsive to this mode of paradigms.
processing? Despite the methodological limitations,
One approach to the problem is sug- however, empirical evidence and hypothe-
gested by experimental designs that employ ses reviewed in this section suggest the idea
identical sets of prosodic stimuli under dif- of two distinct modes of speech prosody pro-
fering task conditions: one (explicit) task cessing, each implemented differently in the
requiring the classification of prosodically human brain:
expressed emotions, and another (implicit)
control task used to distract attention away 1. Explicit processing associated with
from emotional cues, for instance, by ask- focusing attention on the evaluation
ing the participant to determine a speaker’s of emotional information presented by
gender based on the presented speech sam- means of speech prosody
ples. Brain activity associated with the pro- 2. Implicit processing during which speech
cessing of emotional prosody then can be prosody is decoded without deliberately
compared between tasks to delineate acti- directing attention toward prosodic sig-
vation patterns specific to each processing nals and often even without conscious
conditions. However, do cerebral responses awareness
during the chosen control condition solely
relate to the implicit processing of emotional In sum, as far as the cerebral correlates of
prosody, or might obtained activation pat- both modes of processing are concerned,
EMOTIONAL VOICES 271
experimental data indicate a predominant help regulate limbic responses and above
role of inferior frontal as well as pos- all allow a thorough evaluation of the
terior superior temporal regions during prosodic cues sent by our partners of inter-
the explicit decoding of emotional signals, action.
whereas implicit modes of processing have
been described to rely on limbic path-
Summary: A Cerebral Network Model
ways (Critchley et al., 2000; Hariri, Mattay,
of Prosody Comprehension
Tessitore, Fera, & Weinberger, 2003; Tami-
etto & de Gelder, 2010), including the amyg- When the individual pieces of empirical
dala as well as aspects of the anterior ros- evidence provided in this section are put
tral mediofrontal cortex (arMFC; Bach et together, prosody comprehension emerges
al., 2008; Ethofer, Anders, Erb, Droll, et al., as a complex function tied to several
2006; Sander et al., 2005; Szameitat et al., cortical and subcortical brain structures.
2010; Wiethoff et al., 2009). Research has established a contribution of
Ideas of a “limbic” processing path, on temporal regions such as the p-STC and
the one hand, and a “cortical” pathway, on m-STC, frontal structures including the IFC,
the other hand, are in accordance with a and limbic regions such as the amygdala
classic model of emotional information pro- or aspects of the arMFC. Each of these
cessing proposed by Joseph LeDoux (1998): brain structures, in turn, has been suggested
LeDoux argues that, after basic stages of sen- to be associated with distinct aspects of
sory analysis within the thalamus, emotion prosody decoding from the basic stages of
processing relies on two distinct neural cir- acoustic analysis to higher order evaluative
cuits: (1) the “low road,” a pathway connect- processes.
ing the thalamus directly with the amydgala, The localization of cerebral correlates,
and (2) the “high road” relaying information in addition, is complemented by empirical
from the thalamus to the cortex, which in evidence that further details the complex
turn projects back to the amygdala. The low interplay of brain structures implicated in
road is described as representing a “safety prosody decoding. Investigations modeling
system” bypassing conscious awareness that, interactions among different nodes of the
in a quick and efficient way, triggers emo- prosody network point to a strong coupling
tional responses that alert and prepare the between the activation of frontal and tem-
body to act. In contrast, the second loop – poral sites and suggest a flow of information
the so-called high road – relates to a slower, from the right STC to the right and left IFC
yet conscious and much more elaborate way during the processing of emotional speech
of processing that creates a more precise rep- prosody (Ethofer, Anders, Erb, Herbert,
resentation of the environment and a more et al., 2006).
thorough appraisal of the situation at hand. In sum, current research findings pro-
Moreover, via projections to the amygdala pose the idea that prosody comprehension
this pathway allows the control and fine- is mediated by a sequential multistep pro-
tuning of emotional responses triggered by cess unfolding from the basic stages of acous-
the “low road” to fit the needs of a given tic voice analysis (bound to temporal brain
environment. areas) and proceeding to higher level stages
Applying LeDoux’s ideas to prosody pro- of categorization and recognition (associated
cessing, the following hypotheses can be with frontal aspects of the brain). After the
derived: Whereas limbic activation tied to processing of auditory information within
implicit modes of prosody processing may the ear, brainstem, thalamus, and primary
reflect the automatic induction of emo- acoustic cortex (A1), three successive steps
tional responses, cortical activation patterns of prosody decoding can be identified:
observed during the explicit processing con-
dition, in contrast, may be linked to cog- Step 1: Extraction of acoustic features of
nitive control and appraisal processes that prosodic cues
272 CAROLIN BRÜCK , BENJAMIN KREIFELTS, THOMAS ETHOFER, & DIRK WILDGRUBER
Step 2: Identification of vocally expressed erties such as frequency and intensity affect
emotion by means of multimodal inte- ERP components (i.e., N1) that peak as
gration early as approximately 60–80 ms (Woods,
Step 3: Explicit evaluation and cognitive 1995) after the onset of an auditory event
elaboration of vocally expressed emo- (see ERP evidence reviewed in Schirmer &
tions Kotz, 2006), thus underlining the assertions
that the analysis of the auditory input sig-
Each of these steps, in turn, appears to nal occurs early in the process of speech
be differentially represented in the human prosody decoding. However, experimental
brain. Whereas the extraction of acoustic conditions requiring participants to focus
features have been linked to voice-sensitive attention on identifying emotions based
structures of the m-STC, more posterior on prosodic markers of a speaker’s voice
aspects of the right STC have been rec- have been shown to modulate ERP sig-
ognized for their contribution to the iden- nals around 360 ms post stimulus onset
tification and integration of emotional sig- (Wambacq, Shea-Miller, & Abubakr, 2004),
nals expressed by means of speech prosody suggesting processing steps concerned with
(please refer to the subsections, “Stimulus- the explicit evaluation of emotional infor-
Driven Activation Associated with the Pro- mation follow stages of acoustic analysis in
cessing of Affective Prosody” and “Task- time.
Dependent Activation Associated with the In addition to the explicit decoding tied
Processing of Affective Prosody”). In con- to the three processing steps detailed ear-
trast, subprocesses concerned with the eval- lier, a second implicit mode of prosody
uation and cognitive elaboration of vocally processing has been assumed. In contrast
expressed emotions have been linked to to the explicit evaluation, implicit process-
inferior frontal structures of both cerebral ing is believed to occur without intention-
hemispheres. Within the prosody network, ally devoting attention to the interpretation
information transfer from primary acous- of prosodic signals. Considering cerebral
tic regions (A1) to the m-STC has been correlates associated with this mode of
characterized as primarily stimulus driven processing, studies suggest that a network
in nature, whereas projection to the p-STC including the amygdala and arMFC sub-
and IFC has been described as depend- serves the implicit analysis of prosodic
ing on focusing attention on the explicit speech cues (please refer to the subsection,
evaluation of expressed emotions (please “Task-Dependent Deactivation: Implicit vs.
refer to the subsections, “Stimulus-Driven Explicit Processing”).
Activation Associated with the Processing Recent neuroimaging studies indicate a
of Affective Prosody” and “Task-Dependent complex interaction between the struc-
Activation Associated with the Processing tures of the implicit and explicit processing
of Affective Prosody”). routes: Findings support the idea that frontal
The sequential nature of the proposed cortical areas implicated in the explicit anal-
processing steps is further corroborated by ysis of prosodic cues might also contribute
electrophysiological studies that allow dis- to the inhibition of limbic activation when
cerning the time course of various aspects individuals actively attend to emotional sig-
of prosody decoding. Recordings of event- nals. Suppression of limbic activation, in
related potentials (ERPs), for example, link turn, has been assumed to reflect a recruit-
the acoustic analysis of a given speech cue to ment of emotion regulation processes that
changes in brain activation occurring within attenuate the automatic induction of emo-
the first 100 ms after stimulus onset, whereas tional reactions associated with limbic acti-
processes related to the evaluation of vation in order to avoid emotional interfer-
emotional meaning appear to be reflected ence in goal-directed behavior (Blair et al.,
in variations of brain responses with higher 2007).
latencies: Research findings, for instance, Questions, however, remain as to how
indicate that modulations of acoustic prop- complete is this proposed model of prosody
EMOTIONAL VOICES 273
processing. At this point, the current larly, both lesion (Hornak et al., 2003) and
account can only provide a framework to fMRI data (Ethofer, Kreifelts, et al., 2009;
understand the contributions of frontotem- Quadflieg et al., 2008; Wildgruber et al.,
poral and subcortical brain structures most 2004) also point to an involvement of the
consistently implicated in the process of orbitofrontal cortex (OFC) in the processing
prosody decoding. Yet several reports pub- of vocal prosodic markers. However, despite
lished in the literature suggest an involve- the growing body of evidence, the exact role
ment of other brain regions as well: An of the basal ganglia and OFC in prosody
increasing amount of clinical data, for processing remains elusive. Initial evidence
instance, indicate marked impairments in links both structures to the later cognitive-
prosody decoding after traumatic or degen- evaluative stage of prosody processing (Paul-
erative lesions of the basal ganglia (Breit- mann, Ott, & Kotz, 2011; Schirmer & Kotz,
enstein, Van Lancker, Daum, & Waters, 2006), yet further specifications appear to be
2001; Cancelliere & Kertesz, 1990; Pell & needed.
Leonard, 2003). Such clinical observations
are further supported by imaging studies Nonverbal Vocalizations
reporting a recruitment of the basal gan-
glia during the decoding of prosodic cues Even though speech and language no doubt
(Bach et al., 2008; Kotz et al., 2003). Simi- constitutes humankind’s most sophisticated
274 CAROLIN BRÜCK , BENJAMIN KREIFELTS, THOMAS ETHOFER, & DIRK WILDGRUBER
means of information transfer, the vocal Cramon, & Alter, 2005; Sander & Scheich,
communication of emotions extends well 2005; Scott et al., 2009) and of subcortical
beyond speech-related phenomena. In fact, structures such as the amygdala (Fecteau,
human beings quite frequently rely on a Belin, Joanette, & Armony, 2007) or basal
variety of nonverbal vocalizations,3 such ganglia (Morris et al., 1999) have been sug-
as sighs, sobs, screams, groans, moans, or gested for the processing of nonverbal emo-
laughter, to express how they feel. And tional vocalization.
although nonverbal vocalizations in a sense However, compared to the sizable body
may resemble animal calls (Scott, Sauter, & of research on the decoding of emotional
McGettigan, 2009) rather than more elabo- prosody, knowledge concerning the process-
rate human vocal behaviors such as speech, ing of nonverbal vocalizations remains lim-
nonverbal tokens nonetheless prove to be ited. Nonetheless, in recent years, partic-
effective carries of emotional information. ularly vocalizations of laughter – or more
Not only are listeners able to decode non- precisely the (cerebral) processes that allow
verbal vocalization with remarkably high us to perceive laughter signals – have
accuracy (Schröder, 2003; 81% correct clas- received growing attention.
sifications on average across 10 different
emotions) but research findings also sug-
The Neurobiological Basis of Laughter
gest that in vocal communication some
Perception
emotions might typically be expressed by
short vocalizations rather than by speech- Asked for a simple definition of the word
related cues (Schröder, 2003). Disgust, for “laughter,” most people would probably
example, serves as a prime example to resort to descriptions characterizing it as a
illustrate the latter idea: Although recog- vocal expression of joy and happiness. How-
nition rates tend to be comparatively low ever, challenged to recall different situa-
when expressed by means of speech prosody tions in which we have experienced laugh-
(Banse & Scherer, 1996), vocalizations of dis- ter, we most certainly will arrive at the
gust (e.g. Yuck!, Eww!) rank among the conclusion that, in addition to joy and hap-
most reliably decoded types of “affective piness, laughter may encode various other
bursts” (Schröder, 2003; average decoding emotional states and may serve as a valuable
accuracy: 93 %). communicational signal in a variety of social
Despite differences in recognizability, situations and emotional contexts.
however, similarities emerge at the level Different emotions appear to be asso-
of acoustic characteristics: A recent study ciated with the characteristic sound of
conducted by Sauter and colleagues (2010), laughter defined by distinctive modulations
for instance, found that different emotions of voice pitch, loudness, or duration of
expressed by means of nonverbal vocaliza- laughter segments (Szameitat et al., 2009;
tions map onto distinctive acoustic profiles Szameitat, Darwin, Szameitat, Wildgruber,
that resemble to a certain degree acous- & Alter, 2011). Just picture the nervous gig-
tic patterns obtained for vocal-prosodic gles of a speaker who has lost his train of
markers of the respective emotional state thought or the sound of laughter meant
(for more details see Banse & Scherer, to taunt and ridicule its addressee. Thus,
1996; Juslin & Laukka, 2003; Sauter, Eisner, whether a person, for instance, feels joy
Calder, & Scott, 2010). or comtempt towards his or her commu-
Commonalities further translate to cere- nication partner might be discerned based
bral correlates associated with the process- solely on the acoustic signals that consti-
ing of nonverbal vocalizations and prosodic tute human laughter. Indeed, even if no
vocal emotional cues: Similar to speech information but the sheer sound of laugh-
prosody, contributions of the superior tem- ter is available, human beings are still
poral cortex (Meyer, Baumann, Wildgru- able to infer and classify the emotional
ber, & Alter, 2007; Meyer, Zysset, Von states of their partners of interaction with
EMOTIONAL VOICES 275
further information as to what others might thetis, 2009), as well as “convergence zones”
think or feel. Combining or integrating such (Damasio, 1989) – brain areas that receive
visual and acoustic emotional signals, on the afferent input from several senses, such as
one hand, may facilitate emotion judgments; the superior temporal (Seltzer & Pandya,
as for example, behavioral data (Dolan, 1978) and orbitofrontal cortex (Chavis &
Morris, & de Gelder, 2001; de Gelder & Pandya, 1976), the insula (Mesulam & Muf-
Vroomen, 2000; Kreifelts, Ethofer, Grodd, son, 1982), the superior colliculi (Fries, 1984),
Erb, & Wildgruber, 2007; Massaro & Egan, the claustrum (Pearson, Brodal, Gatter, &
1996), indicating shortened response laten- Powell, 1982), thalamus (Mufson & Mesu-
cies and higher classification accuracy for lam, 1984), or amygdala (McDonald, 1998) –
stimuli providing matching facial and vocal have been suggested as potential sites of
information (e.g., a happy facial expression audiovisual integration.
paired with a happy tone of voice while Knowledge derived from anatomical and
speaking) evidences. On the other hand, electrophysiological studies is further com-
information gathered from one channel of plemented by a growing body of fMRI
communication may also modulate or alter research. Even though approaches taken to
the interpretation of emotional signals con- explore audiovisual integration tend to dif-
veyed by the other. Pictures of facial expres- fer considerably among studies, a pattern has
sions, for instance, may be evaluated rather begun to emerge that suggests a crucial role
differently when paired with vocal cues of of the superior temporal cortex in the inte-
different emotional connotation (de Gelder gration of facial and vocal emotional cues.
& Vroomen, 2000; Ethofer, Anders, Erb, Compelling evidence for an involvement of
Droll, et al., 2006; Massaro & Egan, 1996; the STC is presented, for instance, in a series
Müller et al., 2010). Neutral faces, for exam- of fMRI experiments conducted by Kreifelts
ple, are regarded as more fearful when com- and collaborators (Kreifelts et al., 2007;
bined with a fearful voice – even when par- Kreifelts, Ethofer, Huberle, Grodd, & Wild-
ticipants are asked to ignore co-occurring gruber, 2010; Kreifelts, Ethofer, Shiozawa,
vocal information (de Gelder & Vroomen, Grodd, & Wildgruber, 2009). Aiming to
2000). delineate audiovisual integration sites in the
In sum, seeing an affective facial expres- human brain, Kreifelts and colleagues chose
sion while at the same time hearing an to contrast brain activation to audiovisual
emotional voice appears to trigger an emotional stimulation with brain responses
involuntary process of audiovisual binding evoked by unimodal (visual or auditory)
(Pourtois, de Gelder, Vroomen, Rossion, stimuli. Participants were asked to classify
& Crommelinck, 2000) that shapes affec- a range of audiovisual (video clips), visual
tive judgments. In recent years, an increas- (mute videos), and auditory (sound clips)
ing number of neuroscientific studies have stimuli according to the conveyed emo-
aimed at defining the cerebral mechanisms tions. Each video or sound clip depicted
that contribute to the integration of audio men and women expressing different emo-
and visual affective information. Ground- tional states by means of verbal, vocal,
breaking insights into how the brain pro- and facial signals, and each participant was
cesses multisensory stimulation have been instructed to label the type of emotion
gathered from anatomical and electrophys- based on the nonverbal cues (facial expres-
iological studies both in humans and ani- sions, tone of voice) presented. Contrasts
mals (reviewed in Campanella & Belin, between the audiovisual and unimodal pro-
2007; Ethofer, Pourtois, & Wildgruber, cessing conditions computed on behavioral
2006). Building on the results obtained in data (i.e., reaction times, accuracy of decod-
these studies, numerous brain regions have ing) and brain activation data allowed detail-
been related to multisensory processing: ing effects of audiovisual integration both
“Sensory-specific” brain areas such as the at the behavioral and cerebral level. Com-
auditory cortex (Kayser, Petkov, & Logo- pared to unimodal processing conditions,
EMOTIONAL VOICES 277
the audiovisual presentation of nonverbal in both sensory modalities such as the fol-
emotional signals was associated with a lowing:
significant perceptual gain that resulted in
markedly higher accuracy rates. At the cere- a. The reliance on modality- specific “pro-
bral level, observed increases in decoding cessing modules” (i.e., FFA; Kanwisher,
accuracy were associated with increasing et al., 1997, and temporal voice areas;
activation of the right and left p-STC dur- Belin, et al., 2000) related to basic stages
ing the processing of audiovisual nonverbal of perceptual analysis
cues – suggesting the p-STC as a site b. An involvement of orbito- as well as
of audiovisual integration (Kreifelts et al., inferior frontal brain structures during
2007). Considerations of connectivity fur- the explicit evaluation of both visually
ther support this idea: Not only do pro- and vocally expressed emotions (evi-
jections from both primary auditory and dence reviewed in Posamentier & Abdi,
visual cortices converge within the p-STC 2003; Wildgruber et al., 2009)
but rather analyses employed to estimate c. Two routs of processing: explicit “cor-
the functional coupling of different brain tical” and implicit “limbic” processing
regions also reveal an enhanced synchro- (evidence reviewed in Posamentier &
nization of activation within the bilateral p- Abdi, 2003; Wildgruber et al., 2009)
STC and temporal voice areas within the reported for both emotional face pro-
m-STC, as well as face-sensitive aspects of cessing and emotional voice processing
the fusiform gyrus during the processing
of audiovisual nonverbal signals (Kreifelts One might then propose three (general)
et al., 2007). Building on the observed processing steps associated with the percep-
enhanced functional coupling between uni- tion of audiovisual nonverbal emotional sig-
modal associative cortices and the p-STC nals (Wildgruber et al., 2009):
associated with audiovisual emotional stim-
ulation, audiovisual integration has been 1. Extraction of visual and vocal commu-
proposed to be a process during which nicative signals within the respective
information from spatially distinct modality- modality-specific primary cortices and
specific processing sites (e.g., the fusiform specialized processing modules
face area; Kanwisher et al., 1997; and tem- 2. Integration of audio and visual informa-
poral voice areas; Belin et al., 2000) is trans- tion into a single percept within the pos-
mitted to integration areas within the p-STC terior superior temporal cortex
where the information is bound into a sin- 3. Cognitive elaboration and explicit eval-
gle percept (Kreifelts et al., 2007). In sum, uation of emotional information related
current empirical evidence suggests that the to the activation of inferior and
integration of vocal emotional information orbitofrontal brain structures
with co-occurring facial signals might rely
on a common “supramodal” processing step Moreover, in analogy to concepts reviewed
tied to brain structures implicated in audio- for speech prosody processing (see the ear-
visual integration in general (e.g., the p- lier discussion) and in accordance with the
STC), as well as modality-specific process- published literature on the decoding of emo-
ing steps associated with the recruitment of tional facial expression (reviewed elsewhere,
“unimodal” cortices (Campanella & Belin, e.g., Posamentier & Abdi, 2003), our working
2007). model assumes that the processing of audio-
In uniting this idea with current data visual emotional information can occur in
regarding the processing of emotional facial two ways, each of which is tied to differ-
expressions and of emotional vocal cues, a ent neural circuits and functional meaning:
working model of affective face-voice inte- Audiovisual emotional signals may, on the
gration can be conceived. It builds on simi- one hand, be decoded in an explicit, cogni-
larities between the processing of emotions tive controlled way represented by the three
278 CAROLIN BRÜCK , BENJAMIN KREIFELTS, THOMAS ETHOFER, & DIRK WILDGRUBER
Figure 11.3. A working model of the integration and perception of audiovisual emotional signals:
Processing steps and corresponding neural substrates are depicted for explicit and implicit modes of
processing. Hypothetical interactions between cerebral structures are marked with dashed arrows. A1
= primary auditory cortex; TVA = temporal voice areas, V1 = primary visual cortex; FFA = fusiform
face area, p-STS = posterior aspects of the superior temporal cortex, IFC = inferior frontal cortex;
OFC = orbitofrontal cortex.
steps described earlier. On the other hand, At the cerebral level, the human ability to
nonverbal emotional cues may be processed derive emotional meaning from vocal signals
in a rather implicit or un-intentional man- draws on the complex interplay of several
ner implemented mainly via limbic path- cortical regions located in the middle and
ways (see Figure 11.3). posterior superior temporal lobe and frontal
lobe, as well as subcortical structures such as
Overall Summary: Current Knowledge the amygdala or thalamus. Each contribut-
and Future Research ing structure, in turn, has been tied to dis-
tinct aspects of decoding emotional voices
Similar to visual emotional signals such as from the basic stages of acoustic analysis
facial expressions or gestures, vocal sounds to higher order evaluative processes: Supe-
provide a powerful means of affective com- rior temporal regions have been suggested to
munication. Each and every day of our lives, subserve stages of acoustic analysis and mul-
we are surrounded by emotional voices and timodal binding, whereas frontal structures
we use voice-based acoustic cues such as implicated in emotional voice perception
laughter or speech prosody to infer and deci- have been related to subprocesses including
pher the emotional messages sent by our the explicit evaluation of vocally expressed
partners of interaction. emotions or social cognition in general.
EMOTIONAL VOICES 279
Although research of the past decades the ways in which human beings perceive
has greatly broadened our insight into the and process their emotional environment
neurobiology of emotional voice perception, nonetheless often appear to differ tremen-
current evidence and models mark only the dously among individuals. Considering the
beginning, not the end, of the endeavor to cerebral processing of emotional signals,
understand how the brain processes emo- individual differences have been shown
tional vocal cues. Rather than providing an to modulate emotional brain responses to
exhaustive and coherent picture, current facial expressions or emotional pictures (see
knowledge forms a solid ground on which Chapters 24 and 25; Hamann & Canli, 2004).
to base future research. In fact, despite the With respect to individual differences in
wealth of data available on the topic, numer- the processing of emotional voices, how-
ous questions remain as of yet unanswered. ever, empirical evidence remains sparse.
Here are just a few examples: Nontheless, particularly sex differences have
been discussed in the literature (see Chap-
r Can we further specify and extend mod- ter 26). Neuroimaging studies, for instance,
els of emotional voice processing? indicate sex-related modulations of brain
responses associated with the decoding of
Various brain structures have been iden- emotional voices (Schirmer, Zysset, Kotz,
tified as contributing to the processing of & Yves von Cramon, 2004), as well as dif-
emotional voice, yet the functional role of ferences in the time course of emotional
some structures implicated in the process voice processing between men and women
warrants further elaboration. Approaches (Schirmer, Kotz, & Friederici, 2002). Beyond
that could lead the way to a better under- sex differences, research evidence also sug-
standing of how the brain deciphers vocal gests a strong influence of personality on
markers of emotions may be guided by the brain activation related to the processing
following subset of questions: What are the of vocal emotional cues (Brück, Kreifelts,
similarities and differences in the cerebral Kaza, Lotze, & Wildgruber, 2011). To date,
processes mediating the decoding of affec- however, research regarding the impact of
tive prosody or vocalizations such as laugh- interindividual differences appears to focus
ter? Can we delineate differences among almost exclusively on sex or personality.
various emotional categories? For instance, Thus, future studies should aim not only to
are happy voices processed in a different reevaluate current findings but also to inves-
way than fearful ones? How does the brain tigate effects of neglected parameters such
respond to vocal patterns of “mixed” emo- as age or personal experience on the cere-
tions, which we might frequently encounter bral processing of emotional voice cues.
in everyday life (e.g., leaving a job to take
a new position at another firm may render r Can we distinguish persons skilled in the
an individual excited about the new chal- decoding of vocally expressed emotions
lenge while at the same time sad to leave from those individuals with difficulties in
old friends and colleagues)? interpreting such social signals based on
typical patterns of brain activation?
r How do potential determinants of
interindividual differences such as age, Although most individuals would proba-
gender, past experience, or personality bly consider the decoding of vocal emo-
affect the cerebral processing of emo- tional signals a task easily solved, difficul-
tional voice cues? ties in emotional voice processing have been
reported for several patient groups, includ-
Although in some respects each and every ing individuals diagnosed with schizophre-
member of the human races shares funda- nia (Hoekert, Kahn, Pijnenborg, & Ale-
mental mechanisms of information process- man, 2007), depression (Kan, Mimura,
ing with his or her fellow human beings, Kamijima, & Kawamura, 2004; Uekermann,
280 CAROLIN BRÜCK , BENJAMIN KREIFELTS, THOMAS ETHOFER, & DIRK WILDGRUBER
authors for Physics of Life Reviews, 8(4), 383– of emotional prosody in meaningless speech.
403. Neuroimage, 42(2), 919–27.
Bach, D. R., Herdener, M., Grandjean, D.,
Sander, D., Seifritz, E., & Strik, W. K. (2009).
Abbreviations
Altered lateralisation of emotional prosody
processing in schizophrenia. Schizophrenia
arMFC anterior rostral medial frontal cor- Research, 110(1–3), 180–87.
tex Banse, R., & Scherer, K. R. (1996). Acoustic pro-
ERPs event-related potentials files in vocal emotion expression. Journal of
fMRI functional magnetic resonance Personality and Social Psychology, 70(3), 614–
imaging 36.
FFA fusiform face area Belin, P., Fecteau, S., & Bedard, C. (2004). Think-
IFC inferior frontal cortex ing the voice: Neural correlates of voice per-
OFC orbitofrontal cortex ception. Trends in Cognitive Science, 8(3), 129–
m-STC middle aspects of the superior 35.
Belin, P., & Grosbras, M. H. (2010). Before
temporal cortex
speech: Cerebral voice processing in infants.
p-STC posterior aspects of the superior Neuron, 65(6), 733–35.
temporal cortex Belin, P., & Zatorre, R. J. (2003). Adaptation to
speaker’s voice in right anterior temporal lobe.
Neuroreport, 14(16), 2105–9.
Belin, P., Zatorre, R. J., & Ahad, P. (2002).
Notes
Human temporal-lobe response to vocal
sounds. Brain Research: Cognitive Brain
1 The term “implicit“ is uses to described
Research, 13(1), 17–26.
conditions during which the processing of
Belin, P., Zatorre, R. J., Lafaille, P., Ahad, P., &
prosodic signals is task- irrelevant and occurs
Pike, B. (2000). Voice-selective areas in human
rather incidentally or unintentionally (Tami-
auditory cortex. Nature, 403(6767), 309–12.
etto & de Gelder, 2010).
Blair, K. S., Smith, B. W., Mitchell, D. G., Mor-
2 “Explicit” refers to processing conditions dur-
ton, J., Vythilingam, M., Pessoa, L., et al.
ing which the evaluation of prosodic signals
(2007). Modulation of emotion by cognition
is task-relevant and attention is devoted to
and cognition by emotion. Neuroimage, 35(1),
interpreting emotions expressed by speech
430–40.
prosody.
Breitenstein, C., Van Lancker, D., Daum, I., &
3 In some publications nonverbal vocalizations
Waters, C. H. (2001). Impaired perception of
are also referred to as vocal ,,affective bursts”
vocal emotions in Parkinson’s disease: Influ-
(Scherer, 1994).
ence of speech time processing and executive
functioning. Brain Cognition, 45(2), 277–314.
References Brück, C., Kreifelts, B., Kaza, E., Lotze, M.,
& Wildgruber, D. (2011). Impact of person-
Ackermann, H., Hertrich, I., Grodd, W., & Wild- ality on the cerebral processing of emotional
gruber, D. (2004). Das Hören von Gefühlen: prosody. Neuroimage, 58(1), 259–68.
Funktionell-neuroanatomische Grundlage der Campanella, S., & Belin, P. (2007). Integrating
Verarbeitung affektiver Prosodie. Aktuelle face and voice in person perception. Trends in
Neurologie, 31, 449–60. Cognitive Science, 11(12), 535–43.
Adolphs, R., & Tranel, D. (1999). Intact Cancelliere, A. E., & Kertesz, A. (1990). Lesion
recognition of emotional prosody following localization in acquired deficits of emotional
amygdala damage. Neuropsychologia, 37(11), expression and comprehension. Brain Cogni-
1285–92. tion, 13(2), 133–47.
Amodio, D. M., & Frith, C. D. (2006). Meeting Chavis, D. A., & Pandya, D. N. (1976). Further
of minds: the medial frontal cortex and social observations on corticofrontal connections in
cognition. Nature Reviews Neuroscience, 7(4), the rhesus monkey. Brain Research, 117(3),
268–77. 369–86.
Bach, D. R., Grandjean, D., Sander, D., Her- Chein, J. M., Ravizza, S. M., & Fiez, J. A. (2003).
dener, M., Strik, W. K., & Seifritz, E. (2008). Using neuroimaging to evaluate models of
The effect of appraisal level on processing working memory and their implications for
282 CAROLIN BRÜCK , BENJAMIN KREIFELTS, THOMAS ETHOFER, & DIRK WILDGRUBER
language processing. Journal of Neurolinguis- elty on brain regions underlying the processing
tics, 16, 315–39. of speech melody. Journal of Cognitive Neuro-
Chevallier, C., Noveck, I., Happe, F., & Wilson, science, 21(7), 1255–68.
D. (2011). What’s in a voice? Prosody as a test Ethofer, T., Pourtois, G., & Wildgruber, D.
case for the Theory of Mind account of autism. (2006). Investigating audiovisual integration of
Neuropsychologia, 49(3), 507–17. emotional signals in the human brain. Progress
Critchley, H., Daly, E., Phillips, M., Brammer, in Brain Research, 156, 345–61.
M., Bullmore, E., Williams, S., et al. (2000). Ethofer, T., Van De Ville, D., Scherer, K., &
Explicit and implicit neural mechanisms for Vuilleumier, P. (2009). Decoding of emotional
processing of social information from facial information in voice-sensitive cortices. Cur-
expressions: A functional magnetic resonance rent Biology, 19(12), 1028–33.
imaging study. Hum Brain Mapping, 9(2), 93– Ethofer, T., Wiethoff, S., Anders, S., Kreifelts,
105. B., Grodd, W., & Wildgruber, D. (2007).
D’Esposito, M., Aguirre, G. K., Zarahn, E., Bal- The voices of seduction: Cross-gender effects
lard, D., Shin, R. K., & Lease, J. (1998). Func- in processing of erotic prosody. Social
tional MRI studies of spatial and nonspatial Cognition and Affective Neuroscience, 2(4),
working memory. Brain Research: Cognitive 334–37.
Brain Research, 7(1), 1–13. Fecteau, S., Belin, P., Joanette, Y., & Armony,
Damasio, A. R. (1989). Time-locked mul- J. L. (2007). Amygdala responses to nonlin-
tiregional retroactivation: A systems-level guistic emotional vocalizations. Neuroimage,
proposal for the neural substrates of 36(2), 480–87.
recall and recognition. Cognition, 33(1–2), Fries, W. (1984). Cortical projections to the supe-
25–62. rior colliculus in the macaque monkey: A ret-
de Gelder, B., & Vroomen, J. (2000). The percep- rograde study using horseradish peroxidase.
tion of emotions by ear and by eye. Cognition Journal of Comparative Neurology, 230(1), 55–
and Emotion, 14(3), 289–311. 76.
Dolan, R. J., Morris, J. S. & de Gelder, B. (2001). Gervais, H., Belin, P., Boddaert, N., Leboyer,
Crossmodal binding of fear in voice and face. M., Coez, A., Sfaello, I., et al. (2004). Abnor-
Proceedings of the National Academy of Sci- mal cortical voice processing in autism. Nature
ences of the United States of America, 98(17), Neuroscience, 7(8), 801–2.
10006–10010. Grandjean, D., Sander, D., Pourtois, G.,
Ethofer, T., Anders, S., Erb, M., Droll, C., Royen, Schwartz, S., Seghier, M. L., Scherer, K.
L., Saur, R., et al. (2006). Impact of voice R., et al. (2005). The voices of wrath: Brain
on emotional judgment of faces: An event- responses to angry prosody in meaningless
related fMRI study. Human Brain Mapping, speech. Nature Neuroscience, 8(2), 145–46.
27(9), 707–14. Hamann, S., & Canli, T. (2004). Individual differ-
Ethofer, T., Anders, S., Erb, M., Herbert, C., ences in emotion processing. Current Opinion
Wiethoff, S., Kissler, J., et al. (2006). Cerebral in Neurobiology, 14(2), 233–38.
pathways in processing of affective prosody: A Hariri, A. R., Mattay, V. S., Tessitore, A., Fera,
dynamic causal modeling study. Neuroimage, F., & Weinberger, D. R. (2003). Neocortical
30(2), 580–87. modulation of the amygdala response to fear-
Ethofer, T., Anders, S., Wiethoff, S., Erb, M., ful stimuli. Biological Psychiatry, 53(6), 494–
Herbert, C., Saur, R., et al. (2006). Effects of 501.
prosodic emotional intensity on activation of Hoekert, M., Kahn, R. S., Pijnenborg, M.,
associative auditory cortex. Neuroreport, 17(3), & Aleman, A. (2007). Impaired recogni-
249–53. tion and expression of emotional prosody
Ethofer, T., Bretscher, J., Gschwind, M., in schizophrenia: Review and meta-analysis.
Kreifelts, B., Wildgruber, D., & Vuilleumier, Schizophrenia Research, 96(1–3), 135–45.
P. (2012). Emotional voice areas: Anatomic Hornak, J., Bramham, J., Rolls, E. T., Morris,
location, functional properties, and structural R. G., O’Doherty, J., Bullock, P. R., et al.
connections revealed by combined fMRI/DTI. (2003). Changes in emotion after circum-
Cerebral Cortex, 22(1), 191–200. scribed surgical lesions of the orbitofrontal and
Ethofer, T., Kreifelts, B., Wiethoff, S., Wolf, J., cingulate cortices. Brain, 126(Pt. 7), 1691–1712.
Grodd, W., Vuilleumier, P., et al. (2009). Dif- Juslin, P. N., & Laukka, P. (2003). Emotional
ferential influences of emotion, task, and nov- expression in speech and music: evidence of
EMOTIONAL VOICES 283
cross-modal similarities. Annals of the New Linden, D. E., Thornton, K., Kuswanto, C. N.,
York Academy of Sciences, 1000, 279–82. Johnston, S. J., van de Ven, V., & Jackson, M.
Juslin, P. N., & Scherer, K. R. (2005). Vocal C. (2010). The brain’s voices: Comparing non-
expression of affect. In J. Harrigan, R. Rosen- clinical auditory hallucinations and imagery.
thal, & K. R. Scherer (Eds.), The new hand- Cerebral Cortex, 21(2), 330–37.
book of methods in nonverbal behavior research Massaro, D. W., & Egan, P. B. (1996). Perceiv-
(pp. 65–135). Oxford: Oxford University Press. ing affect from the voice and the face. Psycho-
Kan, Y., Mimura, M., Kamijima, K., & Kawa- nomic Bulletin & Review, 3(2), 215–21.
mura, M. (2004). Recognition of emotion McDonald, A. J. (1998). Cortical pathways to the
from moving facial and prosodic stimuli in mammalian amygdala. Progress in Neurobiol-
depressed patients. Journal of Neurology, Neu- ogy, 55(3), 257–332.
rosurgery and Psychiatry, 75, 1667–71. Mehrabian, A., & Ferris, S. R. (1967). Inference
Kanwisher, N., McDermott, J., & Chun, of attitudes from nonverbal communication in
M. M. (1997). The fusiform face area: A mod- two channels. Journal of Consulting Psychology,
ule in human extrastriate cortex specialized 31(3), 248–52.
for face perception. Journal of Neuroscience, Mehrabian, A., & Wiener, M. (1967). Decoding of
17(11), 4302–11. inconsistent communications. Journal of Per-
Kayser, C., Petkov, C. I., & Logothetis, N. K. sonality and Social Psychology, 6(1), 109–14.
(2009). Multisensory interactions in primate Mesulam, M. M., & Mufson, E. J. (1982). Insula
auditory cortex: fMRI and electrophysiology. of the old world monkey. III: Efferent cortical
Hearing Research, 258(1–2), 80–88. output and comments on function. Journal of
Kotz, S. A., Meyer, M., Alter, K., Besson, M., Comparative Neurology, 212(1), 38–52.
von Cramon, D. Y., & Friederici, A. D. (2003). Meyer, M., Baumann, S., Wildgruber, D., &
On the lateralization of emotional prosody: Alter, K. (2007). How the brain laughs.
An event-related functional MR investigation. Comparative evidence from behavioral, elec-
Brain and Language, 86(3), 366–76. trophysiological and neuroimaging studies
Krapf, A. (2007). The human voice: The story of a in human and monkey. Behavioural Brain
remarkable talent. London: Bloomsbury. Research, 182(2), 245–60.
Kreifelts, B., Ethofer, T., Grodd, W., Erb, M., & Meyer, M., Zysset, S., Von Cramon, D. Y., &
Wildgruber, D. (2007). Audiovisual integra- Alter, K. (2005). Distinct fMRI responses to
tion of emotional signals in voice and face: An laughter, speech, and sounds along the human
event-related fMRI study. Neuroimage, 37(4), peri-sylvian cortex. Cognitive Brain Research,
1445–56. 24(2), 291–306.
Kreifelts, B., Ethofer, T., Huberle, E., Grodd, W., Mitchell, D. G., Nakic, M., Fridberg, D., Kamel,
& Wildgruber, D. (2010). Association of trait N., Pine, D. S., & Blair, R. J. (2007). The
emotional intelligence and individual fMRI- impact of processing load on emotion. Neu-
activation patterns during the perception of roimage, 34(3), 1299–1309.
social signals from voice and face. Human Mitchell, R. L. (2007). fMRI delineation of
Brain Mapping, 31(7), 979–91. working memory for emotional prosody in
Kreifelts, B., Ethofer, T., Shiozawa, T., Grodd, the brain: Commonalities with the lexico-
W., & Wildgruber, D. (2009). Cerebral rep- semantic emotion network. Neuroimage,
resentation of non-verbal emotional percep- 36(3), 1015–25.
tion: fMRI reveals audiovisual integration area Mitchell, R. L., Elliott, R., Barry, M., Crutten-
between voice- and face-sensitive regions in den, A., & Woodruff, P. W. (2003). The neural
the superior temporal sulcus. Neuropsycholo- response to emotional prosody, as revealed by
gia, 47(14), 3059–66. functional magnetic resonance imaging. Neu-
LeDoux, J. (1998). The emotional brain: The mys- ropsychologia, 41(10), 1410–21.
terious underpinnings of emotional life. London: Morris, J. S., Scott, S. K., & Dolan, R. J.
Phoenix. (1999). Saying it with feeling: Neural responses
Leitman, D. I., Hoptman, M. J., Foxe, J. J., to emotional vocalizations. Neuropsychologia,
Saccente, E., Wylie, G. R., Nierenberg, J., 37(10), 1155–63.
et al. (2007). The neural substrates of impaired Mufson, E. J., & Mesulam, M. M. (1984). Tha-
prosodic detection in schizophrenia and its lamic connections of the insula in the rhe-
sensorial antecedents. American Journal of Psy- sus monkey and comments on the paralimbic
chiatry, 164(3), 474–82. connectivity of the medial pulvinar nucleus.
284 CAROLIN BRÜCK , BENJAMIN KREIFELTS, THOMAS ETHOFER, & DIRK WILDGRUBER
different laughter types. Neuroimage, 53(4), of the amygdala during processing of emo-
1264–71. tional prosody. Neuroreport, 20(15), 1356–60.
Tamietto, M., & de Gelder, B. (2010). Neural Wiethoff, S., Wildgruber, D., Kreifelts, B.,
bases of the non-conscious perception of emo- Becker, H., Herbert, C., Grodd, W.,
tional signals. Nature Reviews Neuroscience, et al. (2008). Cerebral processing of emotional
11(10), 697–709. prosody – influence of acoustic parameters
Uekermann, J., Abdel-Hamid, M., Lehmkamper, and arousal. Neuroimage, 39(2), 885–93.
C., Vollmoeller, W., & Daum, I. (2008). Per- Wildgruber, D., Ackermann, H., Kreifelts, B., &
ception of affective prosody in major depres- Ethofer, T. (2006). Cerebral processing of lin-
sion: A link to executive functions? Journal guistic and emotional prosody: fMRI studies.
of the International Neuropsychology Society, Progress in Brain Research, 156, 249–68.
14(4), 552–61. Wildgruber, D., Ethofer, T., Grandjean, D., &
Van Lancker, D., Cornelius, C., & Kreiman, Kreifelts, B. (2009). A cerebral network model
J. (1989). Recognition of emotional-prosodic of speech prosody comprehension. Interna-
meaning in speech by autistic, schizophrenic, tional Journal of Speech-Language Pathology,
and normal children. Developmental Neuropsy- 11(4), 277–81.
cholgy, 5(2), 207–26. Wildgruber, D., Hertrich, I., Riecker, A., Erb, M.,
von Kriegstein, K., Kleinschmidt, A., Sterzer, Anders, S., Grodd, W., et al. (2004). Distinct
P., & Giraud, A. L. (2005). Interaction of frontal regions subserve evaluation of linguis-
face and voice areas during speaker recogni- tic and emotional aspects of speech intona-
tion. Journal of Cognitive Neuroscience, 17(3), tion. Cerebral Cortex, 14(12), 1384–89.
367–76. Wildgruber, D., Riecker, A., Hertrich, I., Erb, M.,
Wambacq, I. J., Shea-Miller, K. J., & Abubakr, A. Grodd, W., Ethofer, T., et al. (2005). Identi-
(2004). Non-voluntary and voluntary process- fication of emotional intonation evaluated by
ing of emotional prosody: An event-related fMRI. Neuroimage, 24(4), 1233–41.
potentials study. Neuroreport, 15(3), 555– Woods, D. L. (1995). The component structure
59. of the N1 wave of the human auditory evoked
Wiethoff, S., Wildgruber, D., Grodd, W., & potential. Electroencephalography and Clinical
Ethofer, T. (2009). Response and habituation Neurophysiology. Supplement, 44, 102–9.
CHAPTER 12
Stefan Koelsch
286
EMOTION AND MUSIC 287
(d) With regard to human evolution, odors and images. Moreover, music is pre-
music is originally a social activity. There- sumably not optimal to investigate emo-
fore, music is well suited to study interac- tional phenomena that involve a high load of
tions between emotion and social factors. cognitive processing and cognitive appraisal
(e) Music can be used to study the time such as jealousy, shame, and regret; how-
course of emotional processes: both short- ever, it is nevertheless likely that music can
term emotional phenomena (in the range evoke the affective turmoil of such emo-
of seconds) and longer term emotional phe- tions.
nomena (in the range of minutes). Notably, it is a common misconception
(f) Music can be used to investigate mixed that music-evoked emotions only involve
emotions such as “pleasant sadness.” aesthetic experiences, lacking motivational
(g) Studying the neural correlates of emo- components and goal relevance (for a
tions with music also has direct relevance for detailed discussion see Juslin & Västfjäll,
music-therapeutic applications. 2008; Koelsch, Siebel, & Fritz, 2010). That
It is for such reasons that functional neu- view implies that music is not capable of
roimaging studies with music can extend our evoking “everyday emotions” and is there-
views on the neural correlates of emotion in fore not well suited to investigate the neu-
general. ral basis of “real emotions.” The following
However, using music in the study of points speak against that view.
emotion also bears several difficulties: Simply listening to music can evoke
(a) Musical preferences often differ sub- changes in the major reaction components
stantially between individuals (a death- of an emotion: It elicits changes in (a)
metal enthusiast can utterly despise thrash physiological arousal (i.e., in autonomic and
metal), and the necessary control over the endocrine activity); (b) subjective feeling
stimulus material might result in quite (e.g., when music evokes feelings of plea-
different emotional responses in different sure, happiness, sadness, and so on); (c)
subjects. motor expression (e.g., smiling, crying); (d)
(b) Participants have to be equally action tendencies (dancing, foot tapping,
familiar with different musical pieces, or clapping, or even just premotor activity);
styles, used in different experimental con- and (e) possibly (but not necessarily) cogni-
ditions (so that differences in neural activ- tive appraisal. In addition, as reviewed later,
ity between different emotion conditions music is capable of modulating activity in
are not simply due to differences in all limbic/paralimbic brain structures, ren-
familiarity). dering it unlikely that music cannot evoke
(c) It is often quite difficult to control real emotions.
for the musical and acoustical parameters With regard to the so-called basic emo-
that differ between pieces used for differ- tions, many individuals experience the evo-
ent experimental conditions. For example, cation of joy through listening to music (for
when comparing the effects of happy and many individuals, this is actually a frequent
sad music, differences between conditions motivation for listening to music; Sloboda,
observed in functional neuroimaging data 1992). Music can evoke sadness (Zentner
might simply be caused by differences in et al., 2008) and surprise (Koelsch, Kilches,
the tempo of the pieces, which might result et al., 2008; Meyer, 1956), and some subjects
in different cardiovascular responses, which in an earlier study (Koelsch, Fritz, Cramon,
probably interact with activity in some lim- Müller, & Friederici, 2006) reported that the
bic/paralimbic brain structures. permanently, highly dissonant stimuli used
(d) Some emotions are better investi- in that study evoked feelings of disgust and
gated with other stimuli than music. For vertigo. Most people get quite angry when
example, although individuals might feel they have to listen to music that they utterly
disgusted by certain music, disgust is likely dislike, and music is sometimes even used to
to be investigated more appropriately by stimulate anger and aggression in listeners
288 STEFAN KOELSCH
(e.g., the “hate music” of neo-Nazis, and Mechanisms through which Music
partly also military music). Evokes Emotions
Some people seek negative emotions in
music, for example sadness (for the reward- Many emotion theorists are puzzled by the
ing effects of sad music see Levinson, 1990, fact that music is capable of evoking emo-
and Chapter 10). However, one mecha- tions, because music is not a stimulus that
nism (among many others; Levinson, 1990) would obviously interfere with, or sup-
accounting for this phenomenon is that port, the survival of an individual or the
music evokes real sadness (which is indis- species. How is it then that music can nev-
tinguishable from “everyday” sadness with ertheless “hale souls out of men’s bodies”
regard to its underlying neural activity) and (W. Shakespeare, Much Ado about Nothing,
that the psychological realization that in Act II, Scene 3)? Juslin and Västfjäll (2008)
fact nothing bad happened leads to reward- suggested several mechanisms underlying
related feelings of pleasure. Obviously, there the evocation of emotions with music. These
might be different patterns of motor expres- mechanisms include brainstem reflexes (due
sion, cognition, and neurophysiology while to basic acoustic properties of music such
listening to sad music compared with when as timbre, attack time, intensity, and conso-
being sad because of a loss. Importantly, that nance/dissonance); evaluative conditioning
difference between the subjective experi- (the process of evoking emotion with music
ence of sadness during music listening and that has been paired repeatedly with other
the subjective experience of “day-to-day” positive or negative stimuli); emotional con-
sadness by no means rules out that neu- tagion (where a listener perceives an emo-
ral correlates of sadness are active in both tionally relevant feature or expression of the
cases. Instead it argues for the notion that music and then copies this feature or expres-
sadness can in fact be evoked by listening sion internally; see also Juslin & Laukka,
to music. Likewise, even if music does not 2003); visual imagery (where music evokes
evoke a basic emotion in an individual with images with emotional qualities); episodic
the same intensity as a particular real-life memory (where music evokes a memory
situation, the underlying brain circuits are of a particular event, also referred to as
nevertheless presumably the same (whether the “Darling, they are playing our tune”
I am strongly or moderately happy – the phenomenon; Davies, 1978), and musical
emotion is still happiness). expectancy (where a specific musical feature
In addition, social functions of music violates, delays, or confirms the expectations
such as communication, action coordina- of listeners, leading to feelings of tension and
tion, cooperation, and group cohesion (sum- suspense).
marized in Koelsch, Offermanns, & Franzke, Additional factors suggested by other
2010), are functions that were critical for researchers include the repeated mere expo-
the survival of the human species and that sure that can also contribute to, and mod-
are vital for the well-being of an individual. ify the liking of music (Moors & Kuppens,
Therefore, the pleasure in practicing these 2008); semantic associations with emotional
social functions is indeed related to survival valence (Fritz & Koelsch, 2008); movement
functions. (movement elicited by music, even when
Finally, regenerative effects of music it is not directly expressive of emotion,
(e.g., beneficial hormonal and immunolog- can modify emotional states, such as the
ical effects; see Koelsch & Siebel, 2005) impetus to move in a depressed individual;
are material effects for the individual (a Bharucha & Curtis, 2008); and engagement
polypeptide, for example, consists of mat- in social activities (summarized in Koelsch
ter), thus serving the criterion of a “utilitar- et al., 2010). However, the mechanisms
ian emotion” (as opposed to an “aesthetic through which music can evoke emotions
emotion;” see Scherer, 2004, for this distinc- are still debated, and only few functional
tion). imaging studies have so far investigated the
EMOTION AND MUSIC 289
neural correlates of different mechanisms such as goose bumps or shivers down the
underlying the evocation of emotion with spine). The participants listened to a piece of
music (Ball et al., 2007; Koelsch et al., their own favorite music to which they usu-
2006; Koelsch, Fritz, & Schlaug, 2008). The ally had a chill experience. Increasing chill
next sections provide a review of functional intensity correlated with rCBF decrease in
neuroimaging studies on music and emo- the amygdala and in the anterior hippocam-
tion to illustrate the potential of this field pal formation. An increase in rCBF cor-
for the advancement of our understanding relating with increasing chill intensity was
of human emotion; they relate the find- observed in the ventral striatum, midbrain,
ings where possible to the emotion-evoking anterior insula, anterior cingulate cortex,
mechanisms described earlier. and orbitofrontal cortex (for patient stud-
ies on music-evoked pleasure see Griffiths,
Warren, Dean, & Howard, 2004; Matthews
Limbic and Paralimbic Correlates of et al., 2009; Stewart, von Kriegstein, Warren,
Music-Evoked Emotions & Griffiths, 2006).
Even if individuals do not have intense
Although not well defined, “limbic” and “chill” experiences, music can evoke activ-
“paralimbic” structures are considered as ity changes in the amygdala, ventral stria-
core structures of emotional processing, tum, and hippocampus. Investigating the
because their lesion or dysfunction is asso- emotional valence dimension with music,
ciated with emotional impairment (for an Koelsch et al. (2006) compared brain res-
overview see, e.g., Dalgleish, 2004). How ponses to joyful instrumental tunes (played
limbic/paralimbic structures interact and by professional musicians) to those evoked
which functional networks they form are by electronically manipulated, permanently
still not well understood. dissonant counterparts of these tunes (for
A central structure within the lim- other studies using consonant and disso-
bic/paralimbic neural circuitry is the amyg- nant music see Ball et al., 2007; Blood,
dala, which has been implicated in the Zatorre, Bermudez, & Evans, 1999; Gosselin
initiation, generation, detection, mainte- et al., 2006; Khalfa et al., 2008; Sammler,
nance, and termination of emotions that are Grigutsch, Fritz, & Koelsch, 2007; Mueller et
assumed to be important for the survival of al., 2011). During the presentation of pleas-
the individual and the species (Price, 2005). ant music, increases in blood-oxygen-level
A number of functional neuroimaging (Ball dependent (BOLD) signals were observed
et al., 2007; Baumgartner, Lutz, Schmidt, in the ventral striatum (presumably the
& Jäncke, 2006; Blood & Zatorre, 2001; nucleus accumbens; NAc) and the ante-
Eldar, Ganor, Admon, Bleich, & Hendler, rior insula (among other structures). Disso-
2007; Koelsch et al., 2006; Koelsch et al., nant music, in contrast, elicited increases in
2008; Lerner, Papo, Zhdanov, Belozersky, BOLD signals in the amygdala, hippocam-
& Hendler, 2009) and lesion studies (Del- pus, parahippocampal gyrus, and temporal
lacherie, Ehrlé, & Samson, 2008; Gosselin poles (and decreases of BOLD signals were
et al., 2005; Gosselin, Peretz, Johnsen, & observed in these structures in response to
Adolphs, 2007) have shown involvement the pleasant music). Notably, patients with
of the amygdala in emotional responses to unilateral resection of the medial tempo-
music (see Figure 12.1 for illustration). The ral lobe including the parahippocampal cor-
first neuroimaging study showing activity tex show diminished emotional sensitivity
changes in the amygdala was a positron to dissonant music (Gosselin et al., 2006;
emission topography (PET) study by Blood Khalfa et al., 2008), consistent with activity
and Zatorre (2001), in which changes in changes within the parahippocampal gyrus
regional cerebral blood flow (rCBF) were observed in functional neuroimaging studies
measured during “chills” (i.e., extensive using stimuli with varying degrees of dis-
emotional experiences involving sensations sonance (Blood et al., 1999; Koelsch et al.,
290 STEFAN KOELSCH
2006). The results of these studies (Blood joyful and sad music, see Gosselin et al.,
et al., 1999; Gosselin et al., 2006; Khalfa 2006; Khalfa et al., 2008; Mitterschiffthaler,
et al., 2008; Koelsch et al., 2006) suggest Fu, Dalton, Andrew, & Williams, 2007):
a specific role of the mid-portion of the Activity changes in these structures were
parahippocampal gyrus for the processing stronger during the combined presentation
of acoustic roughness, which is perhaps also of fearful or sad photographs with fearful
relevant for decoding the affective content or sad music, compared to when only visual
of vocal signals. information was presented. The combined
In an attempt to investigate the neural presentation of music and photographs also
correlates of sadness, fear, and joy, Baum- elicited stronger activation in the parahip-
gartner, Lutz, et al. (2006) observed that pocampal gyrus and temporal poles. Activ-
auditory information interacts with visual ity changes in the amygdala, hippocam-
information in several limbic- and paralim- pal formation, parahippocampal gyrus, and
bic structures, including the amygdala and temporal poles was also found in another
the hippocampus (for other studies using fMRI study (Koelsch et al., 2006), suggesting
EMOTION AND MUSIC 291
that these structures form a network that A music study by Ball et al. (2007) pro-
plays a prominent role in emotional process- vided the first insight into different func-
ing (see also Koelsch, 2010). tional properties of subregions of the human
The findings from Baumgartner, Lutz, amygdala in response to auditory stimu-
et al. (2006) received support from a study lation. That study used original (mainly
by Eldar et al. (2007), which showed that consonant) piano pieces as pleasant stim-
when either positive (joyful) or negative uli, and permanently dissonant versions of
(fearful) music was played simultaneously these stimuli as unpleasant stimuli (simi-
with an emotionally neutral film clip, it lar to other studies investigating the valence
evoked stronger signal changes in the amyg- dimension with music; Blood et al., 1999;
dala and in areas of the ventrolateral frontal Koelsch et al., 2006; Sammler et al., 2007).
cortex, compared to when only music or The authors investigated signal changes in
only film clips were presented. Moreover, the amygdala in response to both conso-
the combination of negative (but not pos- nant and dissonant music. An increase of
itive) music and neutral film clips evoked BOLD signal was observed in the basolateral
stronger signal changes in the anterior hip- amygdala (to both types of music) and sig-
pocampal formation compared to when only nal decrease in superficial and centromedial
music or only film clips were presented. amygdala (again, to both types of music).
Subjective ratings showed that the music Also using consonant and dissonant
plus film conditions were not perceived as music, Fritz and Koelsch (2005) reported
significantly more positive or negative than BOLD signal decreases with increasing emo-
when music was presented alone. Therefore, tional valence in a central aspect of the
the functional significance of the increase amygdala (presumably lateral and/or basal
in signal change (in the amygdala and hip- nuclei), whereas BOLD signals increased
pocampal formation) remains to be speci- with increasing valence in a superior aspect
fied. However, the findings that the visual of the amygdala (superficial amygdala,
system modulates signal changes in the extending into the substantia innominata).
amygdala are corroborated by data showing Importantly, the central aspect of the amyg-
that simply closing the eyes during listen- dala was found to be functionally connected
ing to fearful music also leads to increased to the temporal pole, hippocampus, and
amygdalar activity (Lerner et al., 2009). parahippocampal gyrus, whereas the supe-
Another important finding of the study rior aspect of the amygdala (presumably
by Eldar et al. (2007) was that activity the superficial amygdala) was functionally
changes in the amygdala were observed connected with the ventral striatum and
in response to both positive and negative orbitofrontal cortex. This suggests that dif-
stimulus combinations. This supports the ferent nuclei of the amygdala are involved
view that the amygdala is involved not only in modulating activity of different emotion
in negative but also in positive emotions networks.
(e.g., Murray, 2007), clearly challenging the As mentioned earlier, the amygdala and
rather simplistic view that the amygdala related limbic structures play a critical role
is primarily a “fear center” in the human for emotions that some assume to have sur-
brain. Notably, the amygdala is not an vival value for the individual and for the
anatomical unity: It comprises several dis- species (Dalgleish, 2004). The studies men-
tinct nuclei (the lateral, basal, accessory tioned in this section provide evidence that
basal, central, medial, and cortical nuclei), music can evoke activity changes in these
and although the amygdala has become one brain structures, suggesting that at least
of the most heavily studied brain struc- some music-evoked emotions involve the
tures, the functional significance of these very core of evolutionarily adaptive neuroaf-
nuclei, as well as their interaction with other fective mechanisms. This evidence supports
structures, is not well understood (LeDoux, the view that music can evoke “real emo-
2007). tions,” a view that is further supported in the
292 STEFAN KOELSCH
next section, which reviews studies inves- Brown et al., 2004; Janata, 2009; Koelsch
tigating neural correlates of music-evoked et al., 2006; Menon & Levitin, 2005) involved
pleasure. dopaminergic neural activity stems from a
recent PET study by Salimpoor, Benovoy,
Larcher, Dagher, and Zatorre (2011) showing
Music Affects Dopaminergic Neural that strong music-evoked pleasure (includ-
Activity ing “musical frissons”) is associated with
increased dopamine binding in the NAc.
Several studies have shown that listening Importantly, activity in the NAc (as well
to pleasant music activates brain structures as activity in the ventral pallidum; Berridge
implicated in reward and experiences of et al., 2009) correlates with motivation- and
pleasure. Blood and Zatorre (2001) reported reward-related experiences of pleasure; for
that the ventral striatum (presumably the instance during the process of obtaining a
NAc; see Figure 12.1 for an illustration) goal, when encountering an unexpected but
is involved in intensely pleasurable “chill” reachable incentive, or when presented with
responses to music. Similarly, another PET a reward cue (reviewed in Berridge et al.,
study by Brown, Martinez, and Parsons 2009; Nicola, 2007). In humans, NAc activ-
(2004) reported activation of the ventral ity has been reported; for example, during
striatum (in addition to the subcallosal cin- sexual activity, intake of drugs, eating of
gulate cortex, anterior insula, and poste- chocolate, and drinking water when dehy-
rior part of the hippocampus) during lis- drated (Berridge et al., 2009; Nicola, 2007).
tening to two unfamiliar, pleasant pieces It has, therefore, previously been suggested
contrasted with a resting condition. Activa- that NAc activity correlates with the sub-
tion of the ventral striatum in response to jective experience of fun (Koelsch, Siebel, &
pleasant music was also observed in three Fritz, 2010), but more detailed information
studies using fMRI: One investigated the about its functional significance is needed to
valence dimension (Koelsch et al., 2006), determine the role that the NAc possibly
another one examined differences in pleas- plays for other emotions as well.
antness due to the predictability of music The NAc also appears to play a role in
(Menon & Levitin, 2005), and the third invigorating, and perhaps even selecting and
investigated music-evoked memories Janata directing, behavior in response to stimuli
(2009). The study by Menon & Levitin, 2005 with incentive value, as well as in motivat-
reported that activation of the ventral stria- ing and rewarding such behavior (Nicola,
tum was connected to activity in the ventral 2007). The NAc is considered to be a “limbic
tegmental area (VTA) and the hypothala- motor interface” (Nieuwenhuys, Voogd, &
mus. This finding suggests that the hemody- Huijzen, 2008), because (1) it receives input
namic changes observed in the ventral stria- from limbic structures such as amygdala and
tum reflect dopaminergic activity: The NAc hippocampus, (2) injecting dopamine in the
is innervated in part by dopaminergic brain- NAc causes an increase in locomotion, and
stem neurons (located mainly in the VTA (3) the NAc projects to other compartments
as well as in the substantia nigra), and is of the basal ganglia, which play an important
part of the so-called reward circuit (e.g., role in the learning, selection, and execu-
Berridge, Robinson, & Aldridge, 2009). This tion of actions. This motor-related function
circuit includes projections from the lat- of the NAc puts it in a key position for the
eral hypothalamus via the medial forebrain generation of a drive to move to, join in, and
bundle to the mesolimbic dopamine path- dance to pleasant music, although the neural
way involving the VTA with projections to basis for this drive needs to be specified.
the NAc (Björklund & Dunnett, 2007). Fur- It is important to note that in three of
ther support for the assumption that the the earlier mentioned studies (Brown et al.,
hemodynamic changes in the ventral stria- 2004; Koelsch et al., 2006; Menon & Levitin,
tum reported earlier (Blood & Zatorre, 2001; 2005) participants did not report “chill”
EMOTION AND MUSIC 293
responses during music listening, suggesting ing studies that used music to investigate
that dopaminergic pathways including the emotion, it is unlikely that the hippocam-
NAc can be activated by music as soon as pal activations were simply due to such pro-
it is perceived as pleasant (i.e., even in the cesses. With regard to memory, for exam-
absence of extreme emotional experiences ple, participants were probably comparably
involving chills). Results from the reviewed familiar with neutral and sad pieces in the
studies indicate that music can easily evoke study by Mitterschiffthaler et al. (2007), in
experiences of pleasure or fun, associated which sad (as compared to neutral) music
with the activity of a reward pathway involv- elicited changes in the anterior hippocampal
ing the hypothalamus, the VTA, and the formation (e.g., sad stimuli included pieces
NAc. This emotional power of music needs such as the “Concerto de Aranjuez” by
to be explored further to provide more sys- Rodrigo and a suite for violin and orchestra
tematic knowledge that could be used in by Sinding; neutral pieces featured “L’oiseau
support of the therapy of affective disor- prophete: by Schumann or the second violin
ders that are related to dysfunctions involv- romance by Beethoven). Similarly, partic-
ing the mesolimbic reward pathway (such ipants were presumably equally unfamiliar
as depressive disorders or Parkinson’s dis- with the happy and fearful musical pieces
ease; see also Koelsch, 2010). It has previ- used in the study by Eldar et al. (2007).
ously been argued that music can evoke not Even more importantly, in the study by
only subjective experiences of fun (involving Blood and Zatorre (2001), rCBF changes in
the NAc) but also experiences of joy and the anterior hippocampal formation were
happiness (Koelsch, Siebel, & Fritz, 2010). observed even when analyzing responses
The next section puts forward the hypothe- only to stimuli that participants brought
sis that experiences of joy involve different themselves into the experiment (supporting
neural systems than those involved in expe- Figure 5 of Blood & Zatorre, 2001); thus every
riences of fun. subject was highly familiar with the music
included in that analysis.
Therefore, studies on music and emo-
Music and the Hippocampus tion remind us of the view of James Papez
(1937) and Paul MacLean (1990) that the hip-
Compared to studies investigating emotion pocampus also plays an important role in
with stimuli such as emotional faces, affec- emotional processes. It has dense reciprocal
tive pictures, pain stimuli, or reward stim- connections with structures involved in the
uli, the review of functional neuroimag- regulation of behaviors essential for survival
ing studies on music and emotion reveals (such as ingestive, reproductive, and defen-
a particularly noticeable feature: The pro- sive behaviors) and with those involved
portion of studies reporting activity changes in the regulation of autonomic, hormonal,
within the (anterior) hippocampal forma- and immune system activity (Nieuwenhuys
tion in response to music is remarkably high et al., 2008). Such structures include the
(such activity changes have been reported amygdala, hypothalamus, thalamic nuclei,
in Baumgartner, Lutz, et al., 2006; Blood & septal-diagonal band complex, cingulate
Zatorre, 2001; Brown et al., 2004; Eldar et al., gyrus, insula, and autonomic brainstem
2007; Fritz & Koelsch, 2005; Koelsch et al., nuclei. Efferent connections project to the
2006, 2007; Mitterschiffthaler et al., 2007; see NAc, other parts of the striatum, and to
also Figure 12.1). It is well established that numerous other limbic, paralimbic, and
the hippocampus plays an important role nonlimbic structures (Nieuwenhuys et al.,
for learning and memory, spatial orientation, 2008). The functional significance of these
novelty, and expectedness (for reviews see, connections places the hippocampus (along
e.g., Moscovitch, Nadel, Winocur, Gilboa, & with the amygdala and the orbitofrontal cor-
Rosenbaum, 2006; Nadel, 2008). However, tex) in a pivotal position for emotional pro-
at least in some of the functional neuroimag- cessing. Thus, it has previously been noted
294 STEFAN KOELSCH
that the key to understanding the function of the studies mentioned here motivate the
of the hippocampus lies in the fact that it has hypothesis that the hippocampus is a crit-
major projections not only to cortical asso- ical structure for the generation of joy and
ciation areas but also to subcortical limbic happiness, and therefore for emotions that
structures (Nieuwenhuys et al., 2008). play a particular role in social attachments.
The notion that the hippocampus is We (Koelsch et al., 2007) have referred
involved in emotional processes (in addition to such emotions as tender emotions, a
to more cognitive functions such as memory term derived from Charles Darwin’s The
and spatial representation) is supported by Expression of Emotions in Man and Animals
a wealth of empirical evidence. First, lesion (Darwin, 1872/1998), in which Darwin wrote
of the hippocampus leads to impairment of that “tender feelings . . . seem to be com-
maternal behavior in rats (Kimble, Rogers, pounded of affection, joy, and especially
& Hendrickson, 1967), as indexed by less of sympathy” (p. 247; note that Darwin
frequent and less efficient nursing, poorer means sympathy in the sense with which
nest building, increased maternal cannibal- the word “empathy” is often used today;
ism, poorer retrieving, and fewer pups sur- that is, either pity for the grief of some-
viving to weaning. Second, individuals with one else or feeling the other’s happiness or
depression show both structural and func- good fortune). These feelings are “of a plea-
tional abnormalities of the hippocampus surable nature” (p. 247), and it is interest-
(reviewed in Videbech & Ravnkilde, 2004; ing to note that in this chapter about love,
Warner-Schmidt & Duman, 2006). Third, joy, and devotion Darwin also writes about
the hippocampus is unique in its vulner- “the wonderful power of music” (p. 250), an
ability to chronic emotional stressors: In idea that he elaborated on in The Descent
animals, chronic stress related to helpless- of Man. The experience of social attach-
ness and despair leads to death of hip- ments is related to positive tender emo-
pocampal neurons and related hippocam- tions (such as joy and happiness), whereas
pal atrophy (Warner-Schmidt & Duman, social loss is related to negative tender emo-
2006), consistent with studies on humans tions (such as sadness). Attachment-related
that show reduced hippocampal volume in behavior includes licking, grooming, nest
individuals suffering from childhood sexual building, and pup retrieval, and, particu-
abuse (Stein, Koverola, Hanna, Torchia, & larly in humans, hugging, kissing, caressing,
McClarty,1997) and posttraumatic stress dis- stroking, softly touching, and softly vocal-
order (PTSD; Bremner, 1999). The loss of izing. At least in humans, an important
hippocampal volume during and after emo- attachment-related emotion is love. Accord-
tional traumatization or during depression ing to our experience from experiments
is assumed to be due to both a downreg- using music as stimuli to evoke emotions,
ulation of neurogenesis in the hippocam- hippocampal activity appears to be related
pal formation and the death of hippocam- to emotional experiences described by the
pal neurons (Warner-Schmidt & Duman, participants as “being moved.”
2006). Fourth, activity changes in the ante- Negative feelings such as anxiety and
rior hippocampal formation (as well as in depression appear to be related to the inhi-
the amygdala) in response to pleasant and bition of hippocampal activity. Notably,
unpleasant music are reduced in individuals because of its particular sensitivity to emo-
with a reduced capability of producing ten- tional stressors, inhibition of neural path-
der positive feelings (i.e., feelings that can be ways projecting to the hippocampus during
described as soft, loving, warm, and happy) the perception of unpleasant stimuli might
compared to individuals in a normal control represent a sensitive neural mechanism that
group (Koelsch et al., 2007). Although only serves to prevent potential damage of hip-
little specific information about the involve- pocampal neurons. Therefore, it is impor-
ment of the hippocampus in the process- tant that researchers are more cautious in
ing of emotions is yet available, the results attributing activity changes observed in the
EMOTION AND MUSIC 295
amygdala and the hippocampus during the providing an interesting link between emo-
presentation of unpleasant (or threatening) tion, social cohesion, and the evolutionarily
stimuli simply to the generation of fear (or adaptive value of music.
other unpleasant emotions); instead they Whether the present conception of the
should also consider the possibility that quality of hippocampus-centered emotions
these activity changes reflect inhibitory pro- is already sufficient or needs to be expanded,
cesses activated automatically to prevent the it is important to recognize the importance
hippocampus (a particularly sensitive brain of the hippocampus for emotional process-
structure) from traumatization during the ing in affective neuroscience. Future neu-
exposure to potentially harmful stimuli. roimaging studies on emotion should care-
It is also important to differentiate fully control for familiarity, novelty, and
the feelings related to the activation of memory processes elicited by different stim-
the reward circuit (including the lateral ulus categories to rule out the possibility that
hypothalamus, as well as the mesolimbic hippocampal activations are caused by such
dopamine pathway involving the VTA with factors. Notably, because of the capability
projections to the NAc; see the earlier dis- of music to evoke activity changes in the
cussion) from the tender positive emotions hippocampus, it is conceivable that music
that involve activity of the hippocampus therapy with depressed patients and with
(although both are not mutually exclusive). PTSD patients has positive effects on the up-
We (Koelsch, Siebel, & Fritz, 2010) have regulation of neurogenesis in the hippocam-
previously noted that feelings arising from pus, but this is still an open question.
activity of the former circuit (involving the
NAc) might perhaps best be referred to
as fun, whereas attachment-related (tender Effects of Music on Insular and
positive) emotions such as joy, love, and hap- Anterior Cingulate Cortex Activity
piness appear to involve hippocampal activ-
ity (see also Siebel, 2009) Another impor- Current theories of emotion emphasize the
tant difference between reward-related and association between emotion and changes
attachment-related emotions is that the for- in physiological arousal (mainly involving
mer ones satiate: Once an organism has sat- changes in autonomic and endocrine activ-
isfied bodily needs and achieved homeosta- ity). Changes in autonomic activity have
sis, it is satiated, and stimuli that served as been reported to be associated with activ-
incentives before can become even aversive ity changes in the anterior cingulate cortex
(because too much of a chemical compound, (ACC) and the insular cortex (Craig, 2009;
for example, can be harmful for an organ- Critchley, 2005; Critchley, Corfield, Chan-
ism). In contrast, the hippocampus-centered dler, Mathias, & Dolan, 2000), and music
emotions do not become satiated. Note that studies using PET or fMRI have observed
a brain system for attachment-related affect activity changes in both of these struc-
that does not satiate is evolutionary adap- tures during music-evoked chills (Blood
tive, because, for example, feeling attached & Zatorre, 2001), as well as during expe-
to a child, loving a child, and feeling the joy riences of fear and sadness (Baumgart-
of being together with the child are emo- ner, Lutz, et al., 2006). Note, however,
tions that serve the continuous protection that activity changes in the ACC or insu-
and nurturing of the offspring. Similarly, the lar cortex are not necessarily related to
need to belong to a social group and the feel- emotional processing. For example, the
ing of social inclusion (both of which do not ACC is also involved in performance mon-
appear to satiate) serve the formation and itoring, movement-related functions, and
maintenance of social bonds, thus strength- the perception of speech and music (e.g.,
ening social cohesion. It has previously Cole, Yeong, Freiwalkd, & Botvinick, 2009;
been argued that music can promote social Koelsch, Siebel, & Fritz, 2010; Mutschler
cohesion (for a review see Koelsch, 2010), et al., 2007). It has recently been suggested
296 STEFAN KOELSCH
(Koelsch, Siebel, & Fritz, 2010) that the ACC listener. In accordance with this proposal,
is involved in the synchronization of biologi- Sloboda (1991) found that specific musi-
cal subsystems (a term coined by Scherer, cal structures were associated with specific
2000). These “subsystems” comprise physi- psychophysiological reactions (shivers, for
ological arousal, motor expression, motiva- example, were often evoked by new or
tional processes, monitoring processes, and unexpected harmonies).
cognitive appraisal. The synchronization of A study by Steinbeis, Koelsch, and
activity of these subsystems is likely to occur Sloboda (2006) tested the hypothesis that
as an effect of every emotional instance, emotional responses can be evoked by unex-
and it may even be indispensable for subjec- pected chord functions. In that study, phys-
tive emotional experiences (usually referred iological measures including EEG, electro-
to as feelings). The ACC is in a unique dermal activity (EDA), and heart rate were
position to accomplish such synchroniza- recorded while subjects listened to three
tion because of its involvement in cognition, versions of Bach chorales. One version was
autonomic nervous system activity, motor the original version composed by Bach with
activity, motivation, and monitoring. Emo- a harmonic sequence that ended on an unex-
tions are usually accompanied not only by pected chord function (also referred to as
autonomic but also by endocrine effects, “deceptive cadence”). The same chord was
which, in turn, have effects on immune sys- also rendered as expected (using a tonic
tem function (Dantzer, O’Connor, Freund, chord) and as very unexpected (a Neapoli-
Johnson, & Kelley, 2008; Koelsch & Siebel, tan sixth chord). The EDA to these three
2005). With regard to music, such effects are different chord types showed clear differ-
particularly relevant when they are related ences between the expected and the unex-
to a reduction of stress or an amelioration pected (as well as between expected and
of depression and anxiety (Koelsch, 2009; very unexpected) chords. Because the EDA
Koelsch et al., 2011). This relevance also reflects activity of the sympathetic nervous
encourages efforts to gather more system- system and because this system is intimately
atic empirical evidence supporting the use linked to emotional experiences, these data
of music therapy in the treatment of diseases corroborate the assumption that unexpected
related to endocrine, autonomic, or immune harmonies elicit emotional responses. The
system dysfunction (such as autoimmune- findings from this study were later repli-
diseases). cated in another study (Koelsch, Kilches,
et al., 2008) that also obtained behavioral
data showing that irregular chords were
Musical Expectancies and Emotional perceived by listeners as more surprising,
Responses more arousing, and less pleasant than reg-
ular chords.
The studies reviewed so far used exper- Functional neuroimaging experiments
imental paradigms employing “pleasant,” using chord sequences with unexpected har-
“unpleasant,” “scary,” “happy,” or “peace- monies supported these findings, showing
ful” tunes. However, as mentioned in the activations of the amygdala (Koelsch, Fritz,
beginning of this chapter, an important & Schlaug, 2008), orbitofrontal cortex (Till-
mechanism capable of evoking emotional mann et al., 2006), and orbito-lateral cortex
responses is musical expectancy. Leonard (Koelsch, Fritz, Schulze, Alsop, & Schlaug,
Meyer, one of the most influential music 2005) in response to unexpected chord func-
psychologists of the last century, proposed tions. The combined findings show that
a theory of musical emotions on the basis unexpected musical events elicit responses
of fulfilled or suspended musical expectan- not only related to the processing of the
cies (Meyer, 1956). He proposed that the structure (e.g., the syntax) of the music but
confirmation or violation of such musi- also emotional responses; notably, this pre-
cal expectancies produces emotions in the sumably also holds for unexpected words in
EMOTION AND MUSIC 297
sentences and for any other stimulus that is et al., 2005; Mitterschiffthaler et al., 2007).
perceived as more or less expected. Thus, future studies are needed to provide
more information about the neural corre-
lates of happiness and sadness, how major
Major-Minor and Happy-Sad Music and minor tonal features might contribute
to emotional effects related to happiness and
Several functional neuroimaging studies sadness, and how such effects are related to
have used major and minor music to investi- musical preference and cultural experience.
gate “happiness and sadness” (Khalfa, Schon,
Anton, & Liégeois-Chauvel, 2005; Mitter-
schiffthaler et al., 2007), “musical beauty” Electrophysiological Effects of
(Suzuki et al., 2008), or “liking” (Green Music-Evoked Emotions
et al., 2008). However, these studies have
not yet yielded a consistent picture, except This section briefly reviews electrophysio-
perhaps activation of the anterior frontome- logical studies on music and emotion. Only a
dian cortex (BA 10 m/9 m) for minor con- few EEG studies (Altenmüller, Schürmann,
trasted to major music in two studies (Green Lim & Parlitz, 2002; Baumgartner, Esslen, &
et al., 2008; Khalfa et al., 2005). Problems Jäncke, 2006; Sammler et al., 2007; Schmidt
in comparing these studies include (1) dif- & Trainor, 2001) have so far investigated this
ferent participant populations; for example, issue (and up to now, there is a lack of stud-
only males in one study (Suzuki et al., 2008) ies on music and emotion using magneto-
compared to eight males and five females encephalography). All of these studies inves-
in another (Khalfa et al., 2005); (2) inter- tigated the valence dimension: contrasting
pretation of unsystematic effects (such as the effects of pleasant music with the effects
an rCBF decrease in a striatal region during of unpleasant music. Schmidt and Trainor
“beautiful major,” an increase during “beau- (2001) and Altenmüller et al. (2002) reported
tiful minor,” an increase during “ugly major,” more pronounced neural activity in the
and a decrease during “ugly minor” musicl left compared to the right frontal lobes
Suzuki et al., 2008); (3) the use of “true in response to music with positive valence
performances” (Khalfa et al., 2005; Mitter- (and the opposite hemispheric weighting for
schiffthaler et al., 2007) on the one hand, music with negative valence). One of these
and the use of melodies (Green et al., 2008) studies (Altenmüller et al., 2002) measured
or chords (Mizuno & Sugishita, 2007; Suzuki direct current EEG, whereas the other one
et al., 2008) played without musical expres- (Schmidt & Trainor, 2001) measured oscilla-
sion on the other; and (4) the use of differ- tory neural activity in the alpha band.
ent tasks: Participants were asked “how well However, this effect was not observed
they liked it” (Green et al., 2008), to “rate the in the studies by Baumgartner, Esslen, and
beauty of the chord sequence” (Suzuki et Jäncke (2006) and Sammler et al. (2007).
al., 2008), to rate “their moodstate . . . from Instead, the study by Baumgartner and col-
sad . . . to happy” (Mitterschiffthaler et al., leagues (2006) reported a bilateral increase
2007), or to “judge the emotion represented of alpha power for happy music (com-
in the music . . . from sad to happy” (Khalfa bined with happy pictures) compared to sad
et al., 2005). and scary music (combined with sad and
Moreover, whereas some studies aimed scary pictures), and the study by Samm-
to match major and minor stimuli in tempo ler et al. (2007) did not find any differ-
and timbre (Green et al., 2008; Mizuno & ences in the alpha band (or in sub-bands of
Sugishita, 2007; Suzuki et al., 2008), happy the alpha frequency range) between pleas-
and sad stimuli differed considerably in their ant and unpleasant music. However, the lat-
acoustic and musical properties in other ter study (Sammler et al., 2007) reported an
studies (e.g., “happy” excerpts having a increase in fronto-midline theta power in
faster tempo than “sad” excerpts; Khalfa response to pleasant music. This increased
298 STEFAN KOELSCH
to music are reflected in cortical lateralisation Critchley, H. (2005). Neural mechanisms of auto-
patterns. Neuropsychologia, 40(13), 2242–56. nomic, affective, and cognitive integration.
Ball, T., Rahm, B., Eickhoff, S., Schulze- Journal of Comparative Neurology, 493(1), 154–
Bonhage, A., Speck, O., & Mutschler, I. 66.
(2007). Response properties of human amyg- Critchley, H., Corfield, D., Chandler, M., Math-
dala subregions: Evidence based on functional ias, C., & Dolan, R. (2000). Cerebral correlates
MRI combined with probabilistic anatomical of autonomic cardiovascular arousal: A func-
maps. PLoS One, 2(3). tional neuroimaging investigation in humans.
Baumgartner, T., Esslen, M., & Jäncke, L. (2006). Journal of Physiology, 523(1), 259–70.
From emotion perception to emotion experi- Dalgleish, T. (2004). The emotional brain. Nature
ence: Emotions evoked by pictures and classi- Reviews Neuroscience, 5(7), 583–89.
cal music. International Journal of Psychophys- Dantzer, R., O’Connor, J., Freund, G., Johnson,
iology, 60(1), 34–43. R., & Kelley, K. (2008). From inflammation to
Baumgartner, T., Lutz, K., Schmidt, C., & sickness and depression: When the immune
Jäncke, L. (2006). The emotional power of system subjugates the brain. Nature Reviews
music: How music enhances the feeling of Neuroscience, 9(1), 46–56.
affective pictures. Brain Research, 1075(1), 151– Darwin, C. (1998). The expression of emotion in
164. man and animals. London: Murray. (Original
Berridge, K., Robinson, T., & Aldridge, J. (2009). work published 1872)
Dissecting components of reward: Liking, Davies, J. (1978). The psychology of music. Stan-
wanting, and learning. Current Opinions in ford: Stanford University Press.
Pharmacology, 9(1), 65–73. Dellacherie, D., Ehrlé, N., & Samson, S. (2008).
Bharucha, J., & Curtis, M. (2008). Affective spec- Is the neutral condition relevant to study
tra, synchronization, and motion: Aspects of musical emotion in patients? Music Perception,
the emotional response to music. Behavioral 25(4), 285–94.
& Brain Sciences, 31, 579. Eldar, E., Ganor, O., Admon, R., Bleich, A., &
Björklund, A., & Dunnett, S. (2007). Dopamine Hendler, T. (2007). Feeling the real world:
neuron systems in the brain: an update. Trends Limbic response to music depends on related
in Neurosciences, 30(5), 194–202. content. Cerebral Cortex, 7(12), 2828–40.
Blood, A., & Zatorre, R. (2001). Intensely pleasur- Fritz, T., Jentschke, S., Gosselin, N., Sammler,
able responses to music correlate with activ- D., Peretz, I., Turner, R., et al. (2009). Uni-
ity in brain regions implicated in reward and versal recognition of three basic emotions in
emotion. Proceedings of the National Academy music. Current Biology, 19(7), 573–76.
of Sciences, 98(20), 11818. Fritz, T., & Koelsch, S. (2005). Initial response
Blood, A. J., Zatorre, R., Bermudez, P., & Evans, to pleasant and unpleasant music: An fMRI
A. C. (1999). Emotional responses to pleasant study. Neuroimage, 26.
and unpleasant music correlate with activity in Fritz, T., & Koelsch, S. (2008). The role of seman-
paralimbic brain regions. Nature Neuroscience, tic association and emotional contagion for the
2(4), 382–87. induction of emotion with music. Behavioral
Bremner, J. (1999). Does stress damage the brain? & Brain Sciences, 31, 579–80.
Biological Psychiatry, 45(7), 797–805. Gosselin, N., Peretz, I., Johnsen, E., & Adolphs,
Brown, S., Martinez, M., & Parsons, L. (2004). R. (2007). Amygdala damage impairs emo-
Passive music listening spontaneously engages tion recognition from music. Neuropsycholo-
limbic and paralimbic systems. NeuroReport, gia, 45(2), 236–44.
15(13), 2033–37. Gosselin, N., Peretz, I., Noulhiane, M., Hasboun,
Cole, M., Yeung, N., Freiwald, W., & Botvinick, D., Beckett, C., Baulac, M., et al. (2005).
M. (2009). Cingulate cortex: Diverging data Impaired recognition of scary music follow-
from humans and monkeys. Trends in Neuro- ing unilateral temporal lobe excision. Brain,
sciences, 32(11), 566–74. 128(3), 628–40.
Conard, N., Malina, M., & Münzel, S. (2009). Gosselin, N., Samson, S., Adolphs, R., Noulhi-
New flutes document the earliest musical ane, M., Roy, M., Hasboun, D., et al. (2006).
tradition in southwestern Germany. Nature, Emotional responses to unpleasant music cor-
460(7256), 737–40. relates with damage to the parahippocampal
Craig, A. (2009). How do you feel – now? The cortex. Brain, 129(10), 2585.
anterior insula and human awareness. Nature Green, A., Bærentsen, K., Stødkilde-Jørgensen,
Reviews Neuroscience, 10, 59–70. H., Wallentin, M., Roepstorff, A., & Vuust, P.
EMOTION AND MUSIC 301
(2008). Music in minor activates limbic struc- pected chord functions during music listening.
tures: A relationship with dissonance? Neu- Neuroreport, 19(18), 1815–19.
roreport, 19(7), 711–15. Koelsch, S., Fritz, T., Schulze, K., Alsop, D., &
Grewe, O., Nagel, F., Kopiez, R., & Altenmüller, Schlaug, G. (2005). Adults and children pro-
E. (2007a). Emotions over time: Synchronicity cessing music: An fMRI study. Neuroimage,
and development of subjective, physiological, 25(4), 1068–76.
and facial affective reactions of music. Emo- Koelsch, S., Fuermetz, J., Sack, U., Bauer, K.,
tion, 7(4), 774–88. Hohenadel, M., Wiegel, M., et al. (2011).
Grewe, O., Nagel, F., Kopiez, R., & Altenmüller, Effects of music listening on cortisol levels and
E. (2007b). Listening to music as a re-creative propofol consumption during spinal anesthe-
process: Physiological, psychological, and psy- sia. Frontiers in Psychology, 2, 210.
choacoustical correlates of chills and strong Koelsch, S., Kilches, S., Steinbeis, N., & Schelin-
emotions. Music Perception, 24(3),297–314. ski, S. (2008). Effects of unexpected chords
Griffiths, T., Warren, J., Dean, J., & Howard, D. and of performer’s expression on brain
(2004). “When the feeling’s gone”: A selective responses and electrodermal activity. PLoS
loss of musical emotion. British Medical Jour- One, 3(7).
nal, 75(2), 344. Koelsch, S., Offermanns, K., & Franzke, P. (2010).
Janata, P. (2009). The neural architecture Music in the treatment of affective disorders:
of music-evoked autobiographical memories. An exploratory investigation of a new method
Cerebral Cortex, 19(11), 2579. for music-therapeutic research. Music Percep-
Juslin, P., & Laukka, P. (2003). Communication tion, 27(4), 307–16.
of emotions in vocal expression and music per- Koelsch, S., Remppis, A., Sammler, D.,
formance: Different channels, same code? Psy- Jentschke, S., Mietchen, D., Fritz, T., et al.
chological Bulletin, 129(5), 770–814. (2007). A cardiac signature of emotionality.
Juslin, P., & Västfjäll, D. (2008). Emotional European Journal of Neuroscience, 26(11), 3328–
responses to music: The need to consider 38.
underlying mechanisms. Behavioral and Brain Koelsch, S., & Siebel, W. (2005). Towards a neu-
Sciences, 31(05), 559–75. ral basis of music perception. Trends in Cogni-
Khalfa, S., Guye, M., Peretz, I., Chapon, F., tive Sciences, 9(12), 578–84.
Girard, N., Chauvel, P., et al. (2008). Evidence Koelsch, S., Siebel, W. A., & Fritz, T. (2010).
of lateralized anteromedial temporal struc- Functional neuroimaging. In P. Juslin & J. Slo-
tures involvement in musical emotion pro- boda (Eds.), Handbook of music and emotion:
cessing. Neuropsychologia, 46(10), 2485–93. Theory, research, applications (2nd ed., pp. 313–
Khalfa, S., Schon, D., Anton, J., & Liégeois- 46). Oxford: Oxford University Press Oxford.
Chauvel, C. (2005). Brain regions involved in Krumhansl, C. (1997). An exploratory study
the recognition of happiness and sadness in of musical emotions and psychophysiology.
music. Neuroreport, 16(18), 1981–84. Canadian Journal of Experimental Psychology,
Kimble, D., Rogers, L., & Hendrickson, C. (1967). 51(4), 336–53.
Hippocampal lesions disrupt maternal, not LeDoux, J. (2007). The amygdala. Current Biol-
sexual, behavior in the albino rat. Journal ogy, 17(20), R868.
of Comparative and Physiological Psychology, Lerner, Y., Papo, D., Zhdanov, A., Belozer-
63(3), 401–7. sky, L., & Hendler, T. (2009). Eyes wide
Koelsch, S. (2009). A neuroscientific perspective shut: Amygdala mediates eyes-closed effect on
on music therapy. Annals of the New York emotional experience with music. PLoS One,
Academy of Sciences, 1169 (The Neurosciences 4(7), e6230.
and Music III Disorders and Plasticity), 374– Levinson, J. (1990). Music and negative emotion.
84. Ithaca, NY: Cornell University Press.
Koelsch, S. (2010). Towards a neural basis of Lundqvist, L., Carlsson, F., Hilmersson, P., &
music-evoked emotions. Trends in Cognitive Juslin, P. (2009). Emotional responses to
Sciences, 14(3), 131–37. music: Experience, expression, and physiol-
Koelsch, S., Fritz, T., Cramon, D., Müller, K., ogy. Psychology of Music, 37(1), 61.
& Friederici, A. (2006). Investigating emotion MacLean, P. (1990). The triune brain in evolu-
with music: An fMRI study. Human Brain tion: Role in paleocerebral functions. New York:
Mapping, 27(3), 239–50. Plenum Press.
Koelsch, S., Fritz, T., & Schlaug, G. (2008). Matthews, B., Chang, C., De May, M., Engstrom,
Amygdala activity can be modulated by unex- J., & Miller, B. (2009). Pleasurable emotional
302 STEFAN KOELSCH
response to music: A case of neurodegenera- HRV analysis. Medical and Biological Engineer-
tive generalized auditory agnosia. Neurocase, ing and Computing, 48(5), 423–33.
15(3), 248–59. Papez, J. (1937). A proposed mechanism of emo-
Menon, V., & Levitin, D. (2005). The rewards tion. Archives of Neurology and Psychiatry,
of music listening: Response and physiological 38(4), 725–43.
connectivity of the mesolimbic system. Neu- Papousek, M. (1996). Intuitive parenting: A hid-
roimage, 28(1), 175–84. den source of musical stimulation in infancy.
Meyer, L. (1956). Emotion and meaning in music. Musical Beginnings: Origins and Development
Chicago: University of Chicago Press. of Musical Competence, 88–112.
Mitterschiffthaler, M. T., Fu, C. H., Dalton, J. A., Perani, D., Saccuman, M., Scifo, P., Spada,
Andrew, C. M., & Williams, S. C. (2007). A D., Andreolli, G., Rovelli, R., et al. (2010).
functional MRI study of happy and sad affec- Functional specializations for music process-
tive states evoked by classical music. Human ing in the human newborn brain. Proceedings
Brain Mapping, 28, 1150–62. of the National Academy of Sciences, 107(10),
Mizuno, T., & Sugishita, M. (2007). Neural corre- 4758.
lates underlying perception of tonality-related Price, J. (2005). Free will versus survival: Brain
emotional contents. Neuroreport, 18(16), 1651– systems that underlie intrinsic constraints on
55. behavior. Journal of Comparative Neurology,
Moors, A., & Kuppens, P. (2008). Distinguishing 493(1), 132–39.
between two types of musical emotions and Salimpoor, V., Benovoy, M., Larcher, K.,
reconsidering the role of appraisal. Behavioral Dagher, A., & Zatorre, R. (2011). Anatomi-
& Brain Sciences, 31, 588–89. cally distinct dopamine release during antic-
Moscovitch, M., Nadel, L., Winocur, G., Gilboa, ipation and experience of peak emotion to
A., & Rosenbaum, R. (2006). The cognitive music. Nature Neuroscience, 14(2), 257–62.
neuroscience of remote episodic, semantic Sammler, D., Grigutsch, M., Fritz, T., & Koelsch,
and spatial memory. Current Opinions in Neu- S. (2007). Music and emotion: Electrophysio-
robiology, 16(2), 179–90. logical correlates of the processing of pleasant
Mueller, K., Mildner, T., Fritz, T., Lepsien, J., and unpleasant music. Psychophysiology, 44(2),
Schwarzbauer, C., Schroeter, M., & Möller, 293–304.
H. (2011). Investigating brain response to Scherer, K. R. (2000). Emotions as episodes of
music: A comparison of different fmri acqui- subsystem synchronization driven by non-
sition schemes. Neuroimage, 54, 337–43. linear appraisal processes. In M. Lewis &
Murray, E. (2007). The amygdala, reward and I. Granic (Eds.), Emotion, development, and
emotion. Trends in Cognitive Sciences, 11(11), self-organization: Dynamic systems approaches
489–97. to emotional development (pp. 70–99). Cam-
Mutschler, I., Schulze-Bonhage, A., Glauche, bridge: Cambridge University Press.
V., Demandt, E., Speck, O., & Ball, T. Scherer, K. (2004). Which emotions can be
(2007). A rapid sound-action association effect induced by music? What are the underlying
in human insular cortex. PLoS One, 2(2), mechanisms? and how can we measure them?
e259. Journal of New Music Research, 33(3), 239–51.
Nadel, L. (2008). Hippocampus and context Schmidt, L., & Trainor, L. (2001). Frontal brain
revisited. In S. Mizumori (Ed.), Hippocampal electrical activity (EEG) distinguishes valence
place fields: Relevance to learning and memory and intensity of musical emotions. Cognition
(pp. 3–15). New York: Oxford University & Emotion, 15(4), 487–500.
Press. Siebel, W. A. (2009). Thalamic balance can be
Nicola, S. (2007). The nucleus accumbens as part misunderstood as happiness. Journal for Inter-
of a basal ganglia action selection circuit. Psy- disciplinary Research, 3, 48–50.
chopharmacology, 191(3), 521–50. Sloboda, J. A. (1991). Music structure and emo-
Nieuwenhuys, R., Voogd, J., & Huijzen, C. V. tional response: Some empirical findings. Psy-
(2008). The human central nervous system. chology of Music, 19, 110–20.
Berlin: Springer. Sloboda, J. A. (1992). Empirical studies of emo-
Orini, M., Bailón, R., Enk, R., Koelsch, S., tional response to music. In M. Jones &
Mainardi, L., & Laguna, P. (2010). A method S. Holleran (Eds.), Cognitive bases of musi-
for continuously assessing the autonomic cal communication (pp. 33–46). Washington:
response to music-induced emotions through American Psychological Association.
EMOTION AND MUSIC 303
Stein, M., Koverola, C., Hanna, C., Torchia, Tillmann, B., Koelsch, S., Escoffier, N., Bigand,
M., & McClarty, B. (1997). Hippocampal vol- E., Lalitte, P., Friederici, A., et al. (2006). Cog-
ume in women victimized by childhood sex- nitive priming in sung and instrumental music:
ual abuse. Psychological Medicine, 27(4), 951– Activation of inferior frontal cortex. Neuroim-
59. age, 31(4), 1771–82.
Steinbeis, N., Koelsch, S., & Sloboda, J. (2006). Trehub, S., Unyk, A., & Trainor, L. (1993). Adults
The role of harmonic expectancy violations identify infant-directed music across cultures.
in musical emotions: Evidence from sub- Infant Behavior and Development, 16(2), 193–
jective, physiological, and neural responses. 211.
Journal of Cognitive Neuroscience, 18(8), Videbech, P., & Ravnkilde, B. (2004). Hippocam-
1380–93. pal volume and depression: A meta-analysis of
Stewart, L., Von Kriegstein, K., Warren, J., & MRI studies. American Journal of Psychiatry,
Griffiths, T. (2006). Music and the brain: 161(11), 1957.
Disorders of musical listening. Brain, 129(10), Warner-Schmidt, J., & Duman, R. (2006). Hip-
2533–53. pocampal neurogenesis: Opposing effects of
Suzuki, M., Okamura, N., Kawachi, Y., Tashiro, stress and antidepressant treatment. Hip-
M., Arao, H., Hoshishiba, T., et al. (2008). pocampus, 16(3), 239–49.
Discrete cortical regions associated with the Zentner, M., Grandjean, D., & Scherer, K. (2008).
musical beauty of major and minor chords. Emotions evoked by the sound of music:
Cognitive, Affective & Behavioral Neuroscience, Characterization, classification, and measure-
8(2), 126–31. ment. Emotion, 8(4), 494–521.
CHAPTER 13
Johanna Kissler
Language As a Symbol System for meaning is arbitrary, and any symbol or set
Emotions of letters may represent any object, action,
or descriptor. As such, both the lexical
Humans are a symbolic species. They use meaning and the emotional meaning of
language to communicate in a way that words and sentences are entirely acquired
is apparently not available to any other through learning. This characteristic has
species. Although we share much of our been used as an argument that language is
nonlinguistic communicatory repertoire less biologically prepared than other signals
with other animals, and evolutionary of emotion and that, in the absence of a
continua can be laid out, human lan- clear referential context (as is often the case
guages in their generative and referential in laboratory experiments), emotional lan-
properties seem to be unique. Human guage may be a less evocative stimulus class
language is about the world, but unlike than other means of emotion elicitation and
the pictorial renderings in photography and communication.
the visual arts, the mapping between the Nevertheless, human language provides
linguistic symbol and the denoted referent an amazingly versatile and potent means
is arbitrary. There is no clear physical to induce emotions in real life. Affective
relationship between the symbol and its neuroscience has only recently begun to
meaning, perhaps with the exception of systematically explore the processing of
onomatopoeia in spoken language – the emotional content in language, although
formation of words whose sound is imita- neurological case reports indicating a special
tive of the noise or the sound of the action status for emotional content in language
designated, such as hiss, buzz, and bang, or date back to the 19th century (Hughlings
plink. Onomatopoeic processing therefore Jackson, 1866). This chapter reviews current
may be related to prosodic processing. empirical evidence on the processing of
With this notable exception, in human emotional content in human language, pro-
language the mapping between symbol and vides an integrative summary of the extant
304
LOVE LETTERS AND HATE MAIL 305
findings, and identifies new perspectives for (Bradley & Lang, 1994) and drop the domi-
future research. nance dimension. This approach has gener-
Emotions are commonly viewed as ated a large body of research in affective neu-
culturally universal, largely innate, evo- roscience, part of which is reviewed later.
lutionary “old” signaling and activation A more recent investigation into the
systems residing in the “old,” subcortical dimensionality of emotional language iden-
parts of the brain. They are phylogenetically tified four basic dimensions of emotion:
designed to promote survival in critical valence, potency, arousal, and predictabil-
situations (i.e., to signal and activate fight, ity (Fontaine, Scherer, Roesch, & Ellsworth,
flight, or feeding, attachment, and sexual 2007). Neural correlates of these four dimen-
behavior). In contrast, language, in general, sions, however, have not yet been explored.
and reading and writing, in particular, repre- Likewise, the otherwise popular categori-
sent comparatively recent developments in cal emotion theories have not yet inspired
the history of humankind, and in individual much language-related research, although
development they appear later than basic emerging experimental evidence is compat-
emotional expressions that appear within ible with the view that language pertaining
weeks after birth (Meltzoff & Moore, 1983; to discrete emotion categories elicits distinct
see also Chapter 27 for a review of emotional brain activities (see the later discussion).
development). Language and emotion share The study of hemispheric asymmetries in
a communicative function, but linguistic emotional processing has a long tradition in
communicative functions are obviously not neuropsychology. Studies of brain-damaged
restricted to the communication of affect. patients, in particular, often have been
Still language is rich in affective con- conducted to test two competing theories
tent, and influential approaches to the study regarding hemispheric lateralization of emo-
of emotion (Russell, 2003; see Chapter 1 tional processing. The “right brain” hypoth-
for an overview) have derived their basic esis posits that the right hemisphere is domi-
dimensions from analyses of written lan- nant in processing emotional stimuli (Borod
guage. Osgood and collaborators, using the et al., 1998). Alternatively, the “valence”
“semantic differential” technique, were the hypothesis suggests hemispheric specializa-
first to demonstrate empirically that affec- tion, with the left hemisphere (particularly
tive connotations of words are determined frontal regions) subserving the processing of
by three principal dimensions: evaluation or positive stimuli and the right hemisphere
valence (positive – negative), activation or (again putatively frontal regions) predomi-
arousal (calm – active/aroused), and potency nantly engaging in the processing of negative
(weak – strong). The semantic differen- stimuli (Davidson & Irwin, 1999). Emotional
tial technique determines a word’s evalu- language may be a particularly interesting
ative connotation by ratings on a multi- test case of these hemispheric lateraliza-
tude of seven-point scales, spanned by pairs tion theories, because in right-handed peo-
of antonyms such as hot-cold, soft-hard, ple core language regions are predominantly
happy-sad, and so forth. Factor analyses located in the left hemisphere. Lateraliza-
of the judgments of many words on such tion of emotional language processing has
scales, given by large subject samples, reveal been investigated in some detail for the case
a three-dimensional evaluative space, the of affective prosody (see also Chapter 11),
structure of which has been replicated many with a considerable body of evidence imply-
times and across different cultures (Osgood, ing that the right hemisphere predominantly
Miron, & May, 1975). Osgood’s principal extracts the suprasegmental, low-frequency
dimensions are at the core of circumplex aspects of spoken language that code for
theories of affect (Russell, 2003) that, how- its affective modulation. Regarding emo-
ever, often refer to only two principal tional semantics, the question of whether
dimensions – namely, arousal and valence – the right hemisphere plays a special role is
as accounting for most of the variance of considerable theoretical interest because
306 JOHANNA KISSLER
Figure 13.1. A multilayer associative network representation of a complex emotional scene (exam
situation) illustrates how, in dynamic emotional processing, perceptual, semantic, and response
systems are interactively linked. Activation on any level of this system can spread to other subsystems.
Reprinted from Kissler et al. (2006).
significance of language (see also Cato- information both symbolically and percep-
Jackson & Crosson, 2006, for a related tually. In terms of the symbolic language
suggestion). Specifically limbic and frontal system it is debated to what extent percep-
brain structures that together encompass the tual information associated with a concept is
“emotional brain” would be expected to be an integral part of a concept, is supplemen-
involved. The involvement of these regions tary imagination, or even interferes with the
should become apparent using functional concept itself. In this regard, hemispheric
neuroimaging studies, and lesion of those asymmetries have been proposed. The left
regions should result in specific deficits. If hemisphere is traditionally seen as the home
the limbic system is activated during the of the symbolic processing system and the
processing of emotional language, one ques- right hemisphere as housing the perceptual
tion would regard the status of such acti- processing system. In favor of dual coding,
vations: Are they an integral part of the highly imaginable words have sometimes
meaning of an expression, or are they due to been found to activate the right hemisphere
secondary processes – “coactivations” rather more than abstract words (Gazzaniga & Hill-
than activations? yard, 1971; Holcomb, Kounios, Anderson,
A similar discussion surrounds the the- & West, 1999). Similarly, “embodiment”
oretical status of motor cortex activations or “grounded cognition” theories pro-
during the processing of action words (e.g., pose that abstract linguistic concepts are
Hauk, Davis, Kerif, & Pulvermuller, 2008) grounded in embodied and situated knowl-
and can be traced to the dual-coding (Paivio, edge. People possess extensive knowledge
1991) and embodiment (Barsalou, 2008) about their bodies and experience bodily
debates. Dual-coding theory proposes that physiological changes during both move-
humans have the capacity to store and access ment and emotion, and abstract linguistic
308 JOHANNA KISSLER
Emotional Words
Movement-Related Words
Visual Cortex
Motor Cortex IFC
Auditory Cortex
Cingulate
Basal Ganglia
Peri-silvian Insula
Language Regions
Amygdala
Figure 13.2. The distributed representation of language meaning in neural networks. Processing a
word’s meaning activates both left-hemispheric peri-sylvian language areas and associated perceptual
and motor systems in a modal fashion. On the left, the idea is illustrated for the neural representation
of movement-related words (after Pulvermüller, 1999, and Pulvermüller and Fadiga, 2010). On the
right, this concept is extended to the processing of emotion-related language that has been shown to
activate secondary sensory cortices, prefrontal areas, and, notably, limbic brain structures in addition
to classic language regions. Dark nodes indicate classic language regions and light nodes the
modality-dependent extensions of this network that code for specific aspects of movement- (left) or
emotion-related (right) language (see also text for more details).
stream. Is a word’s or an utterance’s emo- clinically valuable in mapping out the range
tional “connotation” activated only after full of deficits to be expected after a certain type
lexico-semantic analysis, with emotional sig- of brain damage.
nificance being subordinate to other lexical Historically, in affective neuropsychol-
attributes? Is emotional meaning processed ogy, one popular approach focused on
concurrently with other lexico-semantic the relative effects of left- and right-
attributes, with emotional meaning being hemispheric lesions on processing differ-
one particular aspect of semantics? Or does ent types of emotional stimuli, mostly aim-
the response to emotional aspects of a lan- ing to test the right-hemisphere hypothesis
guage interact with or sometimes even pre- against the valence hypothesis. As early as
cede the analysis of other linguistic features? the 1860s, John Hughlings Jackson observed
Such questions regarding the timing of neu- that emotional (swear) words could be selec-
ral events can best be answered using elec- tively spared in patients with severe global
tromagnetic measures of brain activity that aphasia with large left-hemisphere lesions
offer temporal resolution on a millisecond (Hughlings Jackson, 1866). This observa-
level (see Chapter 4). tion was later confirmed in a case of
The following is a summary of the left hemispherectomy (Smith, 1966). As in
empirical evidence on neural mechanisms Hughlings Jackson’s report, the patient’s
of emotional language processing, against first successful speech attempts after the
this brief theoretical background and operation consisted of curses and emo-
with reference to the questions posed tional utterances. Conversely, Bloom and
earlier. It integrates current knowledge colleagues (1992) found right-hemisphere
on the anatomical and functional cerebral brain-damaged patients to be impaired
correlates, as well as on the temporal in producing emotional content (Bloom,
dynamics of emotion language processing. Borod, Obler, & Gerstman, 1992).
Contributions from different neuroscience Investigating aphasic patients with left-
methods such as lesion studies, functional sided lesions, Landis, Graves, and Good-
magnetic resonance imaging (fMRI), and glass (1982) also found improved perfor-
electro- and magneto-encephalography mance when patients had to read aloud
(EEG/MEG) are distinguished, and the or write to dictation emotional rather than
theoretical implications of the results are neutral words. Using hemifield presenta-
highlighted. Finally, I identify outstanding tion in healthy controls, Graves, Landis, and
questions in need of future research. Goodglass (1981) confirmed a greater ben-
efit for emotional words presented to the
right hemisphere (left visual field), and this
Lesion Studies on the Processing of pattern held in aphasics (Graves et al., 1981;
Emotional Language Landis et al., 1982). Although in healthy peo-
ple the best performance in absolute terms
Studies of brain-damaged patients reveal the is for emotional words presented to the left
brain regions critically involved in affective hemisphere (right visual field), the data indi-
or cognitive functions, particularly when cate a specific role of the right hemisphere
lesions interfere with or abolish a cer- in processing emotional language: Across
tain function. However, this evidence does studies, the relative advantage for emotional
not indicate that those same regions are words is often largest in the right hemi-
the only ones or even the ones typically sphere.
involved in normal functioning. Moreover, Comparing the extent of left- versus
lesions often vary in size, precluding a close right-hemisphere involvement, J. C. Borod
structure–function mapping. Nevertheless, and colleagues conducted several stud-
lesion studies are helpful in delineating, on a ies on unilaterally brain-damaged patients’
large scale, areas of the brain that are critical auditory identification and discrimination,
for a particular function and are, of course, as well as oral production of emotional
310 JOHANNA KISSLER
language. Together, these studies confirm a of either hemisphere for the processing
greater relative decrement following right- of emotional semantics. On theoretical
than left-hemispheric lesions in the per- grounds, one would expect frontal and
ception and production of emotional lan- limbic lesions to have pronounced effects
guage, both on a single word and sentence on the processing of emotional language,
level and regardless of valence (e.g., Borod, and on the basis of the earlier evidence,
Bloom, Brickman, Nakhutina, & Curko, lateralization effects might be suspected.
2002). Again, after left-hemisphere lesions, Indeed, Adolphs and colleagues (2000), in
the impairment is about the same for emo- a large lesion mapping study (N = 108) of
tional and nonemotional semantics; in con- different subcomponents of facial emotion
trast, after right-hemispheric lesions the recognition, reported that specific emotion-
decrement is disproportionately larger for naming impairments, involving the retrieval
emotional content, implying that processing of appropriate lexical labels for facial
of emotional semantics is one of the right expressions, rather than their perceptual
hemisphere’s specific language functions. discrimination, were associated with bilat-
Thus, the right hemisphere may not be eral lesions in the frontal operculum or the
generally superior for processing emotional supramarginal gyrus, and with right tempo-
language. Rather, it seems specialized in ral lesions. Right temporal lobe involvement
the sense that it retains language capacities in the ability to name facial expressions –
for emotional language and is dispropor- that is, to successfully retrieve their lexical
tionately poor at processing nonemotional labels – in the absence of a perceptual deficit
language. Moreover, its processing capac- had previously been reported in two case
ities even for emotional language may be studies (Rapcsak, Comer, & Rubens, 1993),
restricted to concrete and relatively short although in these reports there was the pos-
words (Eviatar & Zaidel, 1991), perhaps sug- sibility of atypical language lateralization.
gesting that some of the right hemisphere’s An exceptionally large number of stud-
emotion advantage in language processing ies document amygdala involvement in
capitalizes on imagery and dual coding. emotion perception. Patients with unilat-
Blonder et al. (1991) suggest a further eral temporal lobectomy, including the
thematic specificity in right- hemisphere amygdala fail to exhibit an otherwise
patients’ impairment for emotional content, observed identification advantage for emo-
in that patients were most impaired for ver- tional words presented in the attentional
bal descriptions of emotional prosody or blink (Anderson & Phelps, 2001) – interest-
facial expressions. Prosody and facial affect ingly, left lateralized lesions drive this effect.
are stimuli for which right-hemisphere- The left amygdala also appears to play a spe-
damaged patients have primary processing cific role in enhanced memory for emotional
deficits (Kucharska-Pietura, Phillips, Ger- words, but not scenes (Buchanan, Denburg,
nand, & David, 2003). Thus, this observation Tranel, & Adolphs, 2001). A particular role of
could have interesting implications for the the left amygdala in the perception of emo-
general organization of semantic processing tional language is also supported by func-
and would be in line with other recent obser- tional neuroimaging studies and intracranial
vations for domain-specific language deficits recordings (see the later discussion).
outside the classic language areas (Neininger Amygdala activation has been shown to
& Pulvermuller, 2003). More fine-grained be modulated by dopamine input (Taka-
analyses or inferences are, however, compli- hashi et al., 2010); this neurotransmitter is
cated by the small number of patients gener- also otherwise implicated in the process-
ally included in the lesion studies (typically ing of affect and language. In particular,
10 per group) and the heterogeneity of the patients with Parkinson’s disease charac-
lesions. terized by dopamine depletion in the basal
Thus, the laterality studies preclude ganglia show “blunted” ratings of valence
inference on the role of specific brain regions and arousal of emotional words (Hillier,
LOVE LETTERS AND HATE MAIL 311
Beversdorf, Raymer, Williamson, & Heil- least in the visual processing of emotional
man, 2007). Reduced accuracy of emotion language, and here evidence suggests a
recognition in Parkinson’s disease has been particular importance of the left amygdala.
documented across different input chan- Furthermore, dopamine availability affects
nels, including lexico-semantics (Paulmann the processing of emotional language,
& Pell, 2010). The basal ganglia are generally and basal ganglia lesions and dopamine
involved in integrating dynamic perceptual depletion appear to impair particularly
input, with a specific role in syntactic negative emotions, perhaps disgust and fear
(Kotz, Frisch, von Cramon, & Friederici, specifically, although these effects might be
2003), semantic (Lieberman, 2001), and modality dependent and restricted to the
prosodic (Van Lancker Sidtis, Pachana, auditory channel. Future studies with larger
Cummings, & Sidtis, 2006) language pro- samples of patients differentiated according
cesses. Testing basal ganglia involvement in to more circumscribed lesions may reveal
emotional speech processing, Paulmann and more fine-grained, perhaps even categorical
colleagues (Paulmann, Pell, & Kotz, 2009) subdivisions in the semantic organization of
found patients with a left-sided basal ganglia emotional language both within and across
lesion impaired in integrating semantic and hemispheres.
prosodic emotional information, in partic- Functional neuroimaging studies com-
ular relating to disgust and fear. Although plement and extend lesion studies and
these data leave open the question whether have made tremendous contributions to our
these results are specific to dynamic displays knowledge of the functional anatomy of
of emotion as provided by the auditory emotion and language in the brain. I turn
channel or by video clips, they clearly to these studies next.
point to the possibility that selective lesions
may result in categorical impairments in
recognizing and understanding emotional Hemodynamic Studies of Emotional
semantics. Along these lines, one may, Language Processing
for instance, suspect that individuals with
lesions to the insula, which processes Functional neuroimaging can provide much
proprioceptive bodily signals and appears more detailed insights into the organization
particularly involved in the emotional expe- of cerebral structures involved in the pro-
rience of disgust (Phillips et al., 1997), could cessing of emotional language than lesion
be selectively impaired in the processing studies, with the additional advantage of
of disgust-related words. Indeed individual imaging healthy brain function. In recent
differences in disgust sensitivity affect lexi- years, functional magnetic resonance imag-
cal decision reaction times to disgust related ing (fMRI) has been used most widely and
words (Silva, Montant, Ponz, & Ziegler, has surpassed positron emission tomography
2012). (PET) in popularity. For details on these
In sum, lesion studies indicate a consid- methods, see Chapter 5). Only one study
erable capacity of the right hemisphere to on emotional language has used PET (Beau-
process specifically emotional semantics. regard et al., 1997).
This capacity may not represent an absolute Unlike lesion studies, functional neu-
advantage in comparison to the left hemi- roimaging studies generally do not indicate
sphere, but rather a relative propensity on a pronounced role of the right hemisphere
which patients with left-hemispheric lesions in the processing of emotional semantics.
may capitalize. Extant studies therefore are Although some right-hemispheric activity is
partly in line with the right-hemisphere found, a consistent observation across stud-
dominance theory of emotion, but do not ies is a preponderance of left-hemispheric
provide tangible evidence for a valence activity. The comparison of emotional with
asymmetry. Studies of individuals with content-neutral language shows that activ-
lesions indicate amygdala involvement at ity enhancement for emotional language
312 JOHANNA KISSLER
is generally found in lateral temporal and neuroimaging studies represent distinct acti-
frontal brain regions, portions of the cingu- vations in a separate semantic subsystem
late cortex, and sometimes in the amygdala. for emotion or rather the transient modula-
Regions most likely involved in perceptual tory influence of distant emotion-processing
and linguistic stimulus processing are centers of the brain on a unitary seman-
occipitotemporal structures such as the tic store. Allison Cato-Jackson and Bruce
fusiform gyrus or the occipital gyri (visual Crosson (2006) interpreted the outcome
stimuli) or extended auditory cortices of their three imaging studies of emotion
(acoustic stimuli). Regions more likely to word processing as indicative of a separate
be implicated in semantic processing itself retrieval mechanism, a distinct “output lex-
include the angular gyrus, the mid and icon” for emotional semantics that consis-
inferior temporal gyri, as well as the medial tently activates the predominantly left supe-
prefrontal cortex and the inferior frontal rior frontal cortex. Conceptually some of the
gyri. These regions do not show modality- earlier described lesion studies on emotion
dependent activity modulation. The more naming deficits support this view.
posterior areas are hypothesized to store the In contrast, enhanced activity in extrastri-
brain’s semantic knowledge, whereas the ate visual areas elicited by emotional words,
frontal ones are hypothesized to be involved in parallel or correlated with enhanced
in semantic retrieval and lexical selection amygdala activity, has been taken to indicate
(for a review, see Binder, Desai, Graves, & amygdala-mediated amplification processes,
Conant, 2009). Together, regions implicated so-called reentrant processing (Herbert et
in the perceptual and semantic analysis of al., 2009; Isenberg et al., 1999). Extrastri-
language often show enhanced activity in ate regions located in the ventral and lateral
response to emotional language input. parts of the inferior temporal and occipi-
Retrosplenial posterior cingulate cortex tal lobes are sensitive to a word`s lexical
also has been found active during the pro- and semantic aspects (Cohen et al., 2002).
cessing of emotional language (Cato et al., Therefore, these brain areas could repre-
2004), as well as in semantic processing in sent target sites for amygdala-driven reen-
general (see Binder et al., 2009). However, trant processing. Figure 13.3 illustrates cor-
its genuine role across these studies is prob- relative evidence for this process from a
ably to act as an interface between seman- recent study from our group (Herbert et al.,
tic and episodic systems and to aid episodic 2009). The finding is consistent with bidi-
encoding of the stimuli in the scanning ses- rectional modulatory connections between
sion. Its enhanced activity during the pro- the amygdala and extrastriate cortex in non-
cessing of emotional language has therefore human primates and supported by human
been linked to better subsequent recall of lesion data on the processing of emotional
words with emotional content (Maddock, words (Anderson & Phelps, 2001) and faces
Garrett, & Buonocore, 2003). (Vuilleumier, Richardson, Armony, Driver,
Brain structures devoted to the process- & Dolan, 2004). Many authors have favored
ing of emotional stimuli, including emo- reentrant processing as a model to explain
tional language, are the orbitofrontal and facilitated sensory processing of faces and
medial frontal regions, anterior and subgen- pictures in the visual cortex (see Chap-
ual parts of the cingulate cortex, the insula, ter 14 for an in-depth review and discus-
and the amygdala. Thus, an extended net- sion), and evidence suggests that it applies
work of frontal, temporal, and mesial struc- also to visual processing of emotional lan-
tures is active in concert during the pro- guage. Facilitated visual processing thus
cessing of emotional compared to neutral occurs for content with evolutionary pre-
language. pared as well as learned emotional signif-
One important issue regarding the orga- icance. Although reentrant processing in
nization of emotional semantics in the brain emotional perception has been associated
is to what extent the activations seen in most widely with the amygdala, regional
LOVE LETTERS AND HATE MAIL 313
0.08
0.16
0.06 0.12
0.04 0.08
0.02 0.04
0
0 −0.1 0.0 0.1 0.2 0.3 0.4 0.5
Pleasant Unpleasant Neutral Left Amygdala
Figure 13.3. Activation of the left amygdala (top left) and extrastriate visual cortex (top right) during
reading of pleasant emotional words as revealed by fMRI. Activity in the amygdala and visual cortex
was found to be correlated, supporting the idea that the amygdala, via back-projections, can amplify
visual processing of a wide range of stimuli, including symbolic language representations of emotion.
Reproduced from Herbert et al. (2009).
interactions modulating amplification are gyri are active primarily during seman-
likely to operate more widely in the brain. tic retrieval. Additionally, many of these
For instance, for emotional picture pro- studies also report posterior cingulate acti-
cessing reentrant amplification mechanisms vation, which is supposed to mediate
have been recently demonstrated in parieto- emotion-related episodic memory enhance-
occipital networks (Keil et al., 2009). ment (Maddock et al., 2003).
Both transient amplification mechanisms Several neuroimaging studies on emo-
and temporally stable distinct semantic tional word processing report enhanced
organization may contribute to some aspects activation in dorsolateral and medial pre-
of the differential processing of emotional frontal and middle temporal brain regions
language in the brain. Naturally, the experi- during processing of emotional in con-
mental task plays an important role in deter- trast to neutral words, but fail to find
mining the mechanism and may account for amygdala activation (e.g., Kuchinke et al.,
some activation differences across studies. 2005). In these studies “active tasks” have
Studies that require the active retrieval often been used in which subjects were
of emotional words or their explicit eval- explicitly asked to categorize or retrieve
uation find, particularly left frontal activa- words according to emotional, lexical, or
tions, often in orbital and rostral frontal semantic aspects. Although attention often
regions (for an overview see Cato-Jackson facilitates emotional perception, cognitively
& Crosson, 2006). A similar pattern is also demanding experimental tasks involving
reported by Kuchinke et al. (2005) who higher order controlled processing can miti-
found lexical decisions on emotional words gate stimulus-driven perceptual processing,
to elicit specific activities in orbitofrontal attenuating amygdala activation (Pessoa,
and inferior frontal gyri. Orbitofrontal activ- McKenna, Gutierrez, & Ungerleider, 2002).
ities are supposed to primarily indicate stim- Task demands and emotional stimulus con-
ulus evaluation, whereas inferior frontal tent recently have been found to interact
314 JOHANNA KISSLER
when subjects performed either gender dis- nonarousing positive and negative stimuli
criminations on fearful or neutral faces of equally high valence (see Chapter 20 for
or judgments on words overlaid on these an in-depth discussion of emotion effects in
faces. Although during gender discrimina- episodic memory). Lewis et al. investigated
tion the task-irrelevant fearful expression the separate and combined effects of word
led to enhanced amygdala activation, it valence and arousal on brain activity (Lewis,
was greatly attenuated when subjects per- Critchley, Rotshtein, & Dolan, 2007).
formed the linguistic task. At the same time They had participants decide on a word’s
medial prefrontal and sensory cortex activ- self-reference and reported valence sensi-
ity decreased and dorsolateral prefrontal and tivity in lateral orbitofrontal cortex. Addi-
parietal cortex activity increased with task tional valence-specific effects were identi-
difficulty (Mitchell et al., 2007). This study fied in the insula and the anterior cingulate.
provides evidence for an impact of a task- Unlike in the studies of Kensinger and
irrelevant emotional stimulus property on Schacter (2006) or Kuchinke et al. (2005),
cognitive processes and for cognitive load these valence effects were predominantly
modulating brain responses to an emotional right lateralized, however, without appar-
stimulus. ent positive-negative asymmetries. Arousal-
Again, the existence of valence-specific induced activities were identified in the left
activations or activations reflecting even amygdala, insula, and basal ganglia, with
more fine-grained, possibly categorical sub- the distribution of activity within these
divisions of emotional space is debated. Sev- structures differing somewhat depending on
eral studies have specifically investigated whether arousal increased in positive or neg-
valence versus arousal-related brain activa- ative words.
tion during processing of emotional words: Thus, as a rule of thumb regarding
Kensinger and Schacter using nonevalua- valence- or arousal-specific activity, dorsal
tive tasks, found the left amygdala, dor- and orbitofrontal structures tend to respond
somedial prefrontal cortex, and ventrome- to a word’s valence, whereas the amygdala,
dial prefrontal cortex to respond to stimulus insula, and basal ganglia are more sensitive
arousal (Kensinger & Schacter, 2006). In to arousal. In functional imaging studies,
the frontal cortex, valence-specific effects more left than right hemispheric activity is
were also identified, with left ventrolateral found during word processing in general.
(inferior frontal) prefrontal cortex respond- However, this simple rendering is compli-
ing to negative stimuli in particular. Other cated by the fact that, within these struc-
arousal effects in this study were iden- tures, hitherto not clearly delineated sub-
tified near the temporal pole and the regions may code for different dimensions.
temporoparietal junction. Further valence For instance, the amygdala has been shown
effects were found in the superior tem- to sometimes respond to valence regardless
poral gyrus, with more activity for nega- of arousal (Kensinger & Corkin, 2004), pos-
tive than positive words, whereas the mid- sibly because dorsal regions of the amyg-
dle temporal gyrus and fusiform gyrus were dala respond to arousal whereas ventral ones
activated more strongly by positive words. might respond to valence (Kim, Somerville,
The inferior parietal cortex exhibited a Johnstone, Alexander, & Whalen, 2003).
preference for positive words. Examining Similarly, although the evidence reviewed
the mnemonic consequences of emotional earlier implicates the basal ganglia in arousal
word content in an abstract-concrete deci- and/or negative affect, the dopaminergic
sion task, Kensinger and Corkin (2004) also nucleus accumbens of the basal ganglia is
found valence-specific effects in left inferior consistently found active during reward-
and dorso-lateral prefrontal cortex, whereas ing and pleasurable stimulation (Sabatinelli,
the amygdala responded to both arous- Versace, Costa, Bradley, & Lang, 2006).
ing and strongly valent words, the stim- Similarly, Lewis et al. (2007) identified the
uli having been divided into arousing and insula as particularly responsive to arousal
LOVE LETTERS AND HATE MAIL 315
in words, although anterior parts coded for cessing depends on a cascade of stimulus-
arousal in positive words, whereas poste- evaluation checks: Situational and individ-
rior parts responded to arousal in nega- ual relevance checks are proposed to deter-
tive words. At the same time, the insula mine whether stimuli will elicit emotional
has also been specifically implicated in the responses and what kind of response will
experience of disgust and the processing of result. In the absence of more specific evi-
nonverbal disgusting stimuli (Wicker et al., dence, this approach is able to account
2003). for the empirical variability in responses to
In this regard, it is important to keep emotional stimuli that is becoming increas-
in mind that the emotion word categories ingly evident in the literature.
used in most experiments are very broad, In sum, functional neuroimaging stud-
possibly obscuring more fine-grained dif- ies of emotion word processing delineate a
ferences, as suggested, for instance, by network of activations in visual and audi-
studies on movement- versus vision-related tory perceptual regions, perisylvian language
verbs (Hauk et al., 2008). Indeed, a recent regions, and limbic and frontal brain struc-
study identified specific brain activities dur- tures. Activities are predominantly left lat-
ing processing of pain-related compared eralized, although some right-hemispheric
with arousal- and valence-matched nega- activity is found. The activities partly reflect
tive words in general (Richter, Eck, Straube, emotion-related processing enhancement,
Miltner, & Weiss, 2010). Interestingly, this some of which appears to be driven by
study also showed huge regional differ- the amygdala. Valence-dependent activa-
ences depending on whether the words were tions are mostly found in prefrontal struc-
explicitly evaluated or implicitly processed. tures, but have also been observed in the
Notably, frontal activities were consider- insula and the cingulate. Regarding the
ably attenuated during implicit processing. processing of emotional language, there
This is somewhat reminiscent of findings of is so far practically no functional neu-
emotion word studies that used “passive” roimaging evidence for the “valence hypoth-
tasks such as reading and reported mainly esis” in the sense of valence-dependent
effects of arousal-driven perceptual amplifi- hemispheric asymmetries. As yet, there
cation, but little evidence of valence-based are virtually no examinations of category-
evaluation. specific activations during emotion word
Across studies, there is also some ambi- processing, although this would appear to
guity regarding the relative impact of provide a promising direction for future
negative versus positive words on brain research.
activity. Many studies found relatively sym- Identifying category-specific activations
metric activation for both valences when in emotion word processing may be some-
arousal is controlled for, but some stud- what complicated by the spatial resolution
ies reported a greater impact of nega- of fMRI or PET. Although much more pre-
tive and others of positive information. cise than lesion studies, these methods are
Such asymmetries have been explained in still extremely coarse compared to cellular
terms of general (Fredrickson, 2001), self- neuroscience methods. Moreover, the mea-
referential (Fossati et al., 2003), or mood- sured activities are integrated over several
congruent (Herbert et al., 2009) processing seconds, possibly obscuring more transient
biases or differences in the internal structure or weaker local activations. Finally, neither
of positive versus negative material in the lesion nor functional imaging studies give
semantic network (Ashby, Isen, & Turken, information as to the temporal dynamics of
1999; Kuchinke et al., 2005). Alternatively, emotional language processing, and indeed,
appraisal theory suggests a dynamic tun- different processes may take place at dis-
ing of emotional networks (Sander, Grand- tinct temporal processing stages. The tem-
jean, & Scherer, 2005; see also Chapter 1). poral dynamics of emotional language pro-
According to this view, emotional pro- cessing is discussed in the next section.
316 JOHANNA KISSLER
Unpleasant
Pleasant
Neutral
2 2
[ms] [ms]
100 200 300 100 200 300
−1 1 µV
Unpleasant
Pleasant
Neutral
2 2
[ms] [ms]
100 200 300 100 200 300
0 200 nA/cm2
Figure 13.4. Left panel: During the reading of emotionally arousing nouns, event-related cortical
potentials are found to be amplified in comparison to neutral words. This amplification can be found
for a variety of stimulation rates (top left, 3 Hz; bottom left, 1 Hz) and takes the form of an early
posterior negativity (EPN) occurring around 250 ms after word onset. It shows a broad
occipitotemporal scalp distribution with a left-hemisphere preponderance (see arrow, top right). The
sources of this activity are localized in predominantly left posterior-temporal cortical regions (see
arrow, bottom right). Adapted from Kissler et al. (2007). See color plate 13.4.
stimuli whose emotional significance has conscious stimulus evaluation, which has
been acquired by learning (see also Chapter been linked to ERPs occurring later than 300
14 for a related account on emotional face ms after stimulus onset (Del Cul, Baillet,
processing). Figure 13.4 illustrates this effect & Dehaene, 2007). However, enhanced
in a silent reading task. ERPs to emotional words around 200 ms
ERP modulations driven by emotional after onset are consistent with the view
content and arising before 300 ms after stim- that emotion can enhance early lexico-
ulus onset are confirmed by many stud- semantic analysis during word processing
ies and have also been found on the P2 (Kissler et al., 2006), because early effects
and sometimes even N1 or P1 components. of semantics in the ERP have been reported
They have been observed in both healthy around 250 ms (Hinojosa, Martin-Loeches,
subjects (Begleiter & Platz, 1969; Herbert, Munoz, Casado, & Pozo, 2004). A recent
Junghofer, & Kissler, 2008; Ortigue et al., study directly compared early perceptual
2004) and clinical groups (Pauli, Amrhein, enhancements as evident in steady-state
Muhlberger, Dengler, & Wiedemann, 2005). potentials (SSVEPs) with lexico-semantic
Such early responses are unlikely to reflect processing effects reflected in the EPN/P2
318 JOHANNA KISSLER
effect and found emotional words to modu- words with monetary gain in an operant
late the later, but not the former, support- conditioning procedure induced an emotion
ing a lexico-sematic locus of such effects effect around 150 ms in non-Chinese readers.
(Trauer, Andersen, Kotz, & Müller, 2012). Regardless of the underlying mechanism, a
Still, a number of studies report emotion- number of ERP studies demonstrate that
dependent ERP differences in word process- even very early brain potentials, indicative of
ing even before 150 ms (Hofmann et al., preconscious and sometimes also prelexical
2009; Scott et al., 2009; Skrandies, 1998). This processing, can distinguish between affective
suggests that the processing of emotional and neutral words across several different ex-
content sometimes can precede or bypass perimental designs, tasks and subject groups.
full semantic analysis. Some data also sug- A considerable number of studies have
gest very early ERP effects of “unseen” emo- found effects of emotional content on elec-
tional words, which were presented sub- trophysiological cortical activity later than
liminally (Bernat, Bunce, & Shevrin, 2001). 300 ms after word onset. For these, as for the
Other results indicate increased skin con- early effects, task dependence and covari-
ductance to unseen unpleasant words (Sil- ation with arousal or valence of the pre-
vert, Delplanque, Bouwalerh, Verpoort, & sented stimuli are not quite clear. Still,
Sequeira, 2004). Naccache and colleagues in contrast to the very early effects, they
(2005) add further evidence for the sub- occur in a time range where the modula-
cortical processing of invisible emotional tion of cortical responses by word mean-
words by showing that subliminally pre- ing is not unusual in itself. By 300–400 ms
sented threat words evoked stronger field after stimulus onset, ERP tracings reflect
potentials in the amygdala than neutral conscious processing stages and clearly vary
words, although in a later time window with semantic expectancy (Kutas & Feder-
(around 800 ms after word onset). Also meier, 2000), task relevance (Sutton, Tuet-
supporting subliminal effects of emotional ing, Zubin, & John, 1967), or depth of men-
content on word processing, although not tal engagement (Dien, Spencer, & Donchin,
addressing their neural origin, are results of 2004). Thus, it is not surprising that ERPs in
lowered detection thresholds for emotional this time range can reflect processing differ-
than for neutral words following their sub- ences between words of different emotional
liminal presentation (Gaillard et al., 2006). content. Considering emotional content as
ERP effects of emotional content within a semantic aspect of words would suggest
the first 150 ms after word onset have that N400 might represent a likely “classi-
been reported most frequently, although not cal” ERP candidate component to assess for
exclusively, in patient populations (Pauli et emotion effects. Indeed, some studies have
al., 2005) who are thought to have an atten- found modulations of the N400 response
tional bias toward disorder-related stimuli. to emotion words, usually with smaller
These earliest responses to disorder-relevant N400 activation to emotional than to neutral
material may, at least partly, reflect con- words, indicating facilitated semantic inte-
ditioned responses to these stimuli. Effects gration (Herbert et al., 2008; Kiehl, Hare,
of aversive conditioning on word process- McDonald, & Brink, 1999).
ing have been observed at the N100 compo- Still, there is a comparative paucity of
nent (Montoya, Larbig, Pulvermuller, Flor, reports on N400 modulation by emotional
& Birbaumer, 1996), and aversive condi- words, which may partly reflect a surprising
tioning of simple geometric patterns can investigator bias, considering that the N400
affect even the C1 ERP, the earliest primary is often regarded as “the” electrophysiolog-
visual cortex response (Stolarova, Keil, & ical indicator of semantic processes in the
Moratti, 2006). Recent data support a role brain. However, the N400 response does not
of conditioning processes in the acquisition index lexical access or semantic processing
of early emotion word ERPs, but not speci- per se, but reflects semantic integration
ficity to negative associations. Schacht et al. within a larger context, either created
(2012) demonstrate that associating Chinese by expectations on sentence content or
LOVE LETTERS AND HATE MAIL 319
other contextual constraints within exper- been reported repeatedly, but not invariably
iments (Kutas & Federmeier, 2000). Thus, (see, for instance, Vanderploeg, Brown, &
single-word studies may only result in Marsh, 1987). Fischler and Bradley (2006),
N400 modulations if strong expectations in a series of studies, found the LPP to
of emotional word content are established. be sensitive to the arousal and not to the
Priming studies or experiments establishing valence dimension of words, being larger for
an emotional expectation within a sentence both pleasant and unpleasant compared to
context therefore may provide a better test- neutral words. Moreover, a larger LPP to
ing ground for N400 modulations by emo- emotional content arose only when the task
tional language content than single-word required semantic processing. An attenua-
studies. Some studies found N400 modula- tion or even absence of emotional-neutral
tions to targets semantically (mis-)matching differences on the LPP during structural
the prosody of the preceding sentence processing of the stimuli was confirmed by
(Schirmer, Kotz, & Friederici, 2002; for a Schacht and Sommer (2009).
review, see Chapter 26) or distinct effects of As in many sub-areas of affective neuro-
semantic and prosodic (mis-)match within science, the extent to which emotion effects
a sentence (Paulmann & Kotz, 2008). Mood in word processing are resource and task
may also mediate N400 effects and influence dependent or occur (relatively) automati-
the ease of semantic integration, as well cally is debated (see Chapters 14 and 15 for
as the subjective expectation of mood- in-depth discussions). Several studies have
congruent versus incongruent information. found larger ERPs to emotional compared
Federmeier, Kirson, Moreno, and Kutas to neutral words in the absence of overt
(2001) found that when participants were instructions or even to subliminally pre-
in a mildly positive mood, their semantic sented stimuli, supporting a “spontaneous”
processing was facilitated, as reflected by processing enhancement. Still, this finding
a smaller N400 potential to more distant alone does not speak in favor of strictly
members of given categories. Similar mood automatic processing and complete non-
effects were recently observed by Chwilla interference of competing tasks. A mini-
and colleagues (2011). Interestingly, a study mum of lexico-semantic processing seems
from our group reported an N400 reduction required for EPN and LPP emotion effects
specifically to positive words in a silent to occur, because none are found when
reading task. This finding suggests that the a task can be performed on the basis
positive words were easier to semantically of discrimination of low-level perceptual
integrate than neutral or negative words features rather than word reading (Hino-
(Herbert et al., 2008), perhaps because josa, Mendez-Bertolo, & Pozo, 2010; Rel-
healthy students tend to be mostly in mildly lecke, Palazova, Sommer, & Schacht, 2011).
positive moods (Diener & Diener, 1996). In recent lexical decision studies that, by
Thus, although not very abundant, a num- definition, require lexical access, emotion
ber of studies suggest that both a word’s effects have been found quite consistent-
emotional content and a participant’s lyin the EPN/P2 time range (from 200 ms
emotional state may affect the N400 ERP. onwards), with emotion effects either coin-
Many studies of the impact of emotional ciding or following lexicality effects (Pala-
content on language processing have focused zova, Mantwill, Sommer, & Schacht, 2011;
on centroparietal late positive potentials Scott et al., 2009). Indeed, very recent data
(LPPs), arising around 500 ms after stim- suggest that the EPN effect arises as a conse-
ulus onset. These late parietal positivities quence of faster lexical access to emotional
are indicative of later stages of attention than to neutral words (Kissler & Herbert,
allocation and evaluation and have been 2012).
shown to predict subsequent episodic recall Regarding the automaticity of emotion
(Dolcos & Cabeza, 2002), implying a func- effects in word processing, a grammati-
tional role in encoding. During the process- cal decision task does not interfere with
ing of emotional words, enhanced LPPs have the emotion EPN effect (Kissler, Herbert,
320 JOHANNA KISSLER
Winkler, & Junghofer, 2009), indicating con- ulating the participants’ mood affects their
siderable robustness even in the face of a responses to syntactic violations as reflected
competing task, provided a minimum of lex- in the amplitude of the P600 brain potential
ical processing occurs. Where exactly this (Vissers et al., 2010) and findings of rapid and
minimum lies is open to future research. pronounced responses to disorder-related
Also, non-interference of one particular words in patient groups support an impor-
competing task may not generalize across tance of contextual factors and individual
other tasks, providing further challenges for differences. In healthy people a larger LPP
future studies. response to pleasant adjectives may be an
Enhanced LPPs to emotional words are accidental finding. Yet, adjectives with emo-
seen most clearly during semantic stimu- tional connotation often describing emo-
lus processing and in particular during emo- tional states or traits, such as “happy” or
tion evaluation. Emotional-neutral differ- “fearful,” may induce more self-referential
ences were attenuated or even absent during processing, engaging more resources over a
structural processing of the stimuli and were longer period of time. Depending on person-
attenuated for lexical compared with eval- ality factors such self-referential processing
uative decisions (Fischler & Bradley, 2006; may favor positive or negative stimuli, and
Schacht & Sommer, 2009). A recent gram- perhaps in healthy young students it may
matical decision task found no significant induce a bias toward positive stimuli.
interference between the primary task and As mentioned earlier, not all studies
emotion processing on the LPP (Kissler of emotion language processing have sepa-
et al., 2009), but the emotion effect was rately assessed or reported stimulus arousal
considerably smaller than the task effect. and valence, let alone other potential dimen-
During silent reading, the LPP response sions or categories of emotion. Even from
pattern diverges from that of the preced- those that have, it is not easy to discern a
ing EPN, and the effect is also smaller clear pattern of arousal or valence effects
(Herbert et al., 2008), indicating a disso- on ERPs. Whereas Fischler and Bradley
ciation of underlying mechanisms. So far, (2006) reported all of their effects to be
results suggest a higher degree of automatic- driven by arousal, other studies present
ity in the processes that drive the EPN more variable patterns. In fact, behavioral
response to emotional words (Schacht & and neural processing advantages appear to
Sommer, 2009). This is also supported in be reported more frequently for positive
two recent studies on emotional picture language (Herbert, Kissler, Junghofer, Peyk,
processing: Codispoti and colleagues found & Rockstroh, 2006; Hofmann et al., 2009;
habituation of the LPP, but not the EPN Schapkin, Gusev, & Kuhl, 2000). Schap-
response to emotional content (Codispoti, kin and colleagues (2000) reported enhanced
Mazzetti, & Bradley, 2009), and Schupp LPPs to pleasant compared to both neu-
and colleagues reported that the LPP both tral and unpleasant words during evaluative
benefits more from directed attention to decision tasks (Schapkin et al., 2000). Also
emotional content and suffers more from using an evaluation task, Herbert and col-
interference from a primary feature-based leagues (2006) found that earlier positivities
attention task (Schupp et al., 2006). (P2, P3a) responded to the arousal dimen-
Although recent studies have investigated sion of the words, whereas the later LPP was
the extent to which directed attention can selectively enhanced during the processing
interfere with different stages of emotional of pleasant adjectives (Herbert et al., 2006).
face (Eimer & Holmes, 2007; see also Chap- This effect is shown in Figure 13.5.
ter 14), picture (Schupp et al., 2006), and to A similar pattern of responses driven ini-
some extent word processing (Kissler et al., tially by arousal and later by “pleasure” was
2009), there is little work on other contex- also found when subjects read emotional
tual factors, such as mood, self-reference, adjectives without receiving an explicit
or personality variables. However, manip- processing instruction (Herbert et al.,
LOVE LETTERS AND HATE MAIL 321
Emotion - Neutral
O9 O10
left right
−0.3 µV 0 µV −0.3 µV
200 − 300 ms
O9 O10
[µV] [µV]
−3 Pleasant −3
−2 Unpleasant −2
Neutral
−1 −1
1 1
2 2
3 [ms] 3 [ms]
100 200 300 400 500 100 200 300 400 500
Pleasant - Neutral
N400 Potential (360 – 470 ms)
[µV] Pz
Unpleasant - Pleasant −1
[µV] CPz LPP
−1 N400
CPz Pz
1
1
Figure 13.5. Event-related brain potential responses in different time windows during the evaluation of
emotional and neutral adjectives, indicating enhanced initial attention to emotionally arousing words.
The P2 around 230 ms after word onset and the P3 between 300 and 400 ms respond to emotionally
arousing words, regardless of valence (top: scalp topography of the difference waveforms and ERP
tracings from electrode Pz). By contrast, the late positive component (LPC) of the brain is larger for
positive than for neutral or negative words, suggesting more pronounced elaborative processing of
positive material (bottom left: scalp topography of the difference waveforms and ERP tracing from
one parietal electrode). Data indicate that emotional words are preferentially processed at different
stages, although following the initial attention capture by both positive and negative arousing words
the pattern can differ, selectively favoring one valence. Adapted from Herbert et al. (2006).
2008). This sequence underscores the later potentials such as the N400 or LPP in
earlier suggested functional differentiation healthy students mostly show an enhanced
between early and late ERPs in affective response to positively arousing over neutral
processing. Several studies from my group stimuli, with negative stimuli not differ-
indicate that early ERPs like the EPN ing from neutral. Figure 13.5 illustrates
respond to the arousal dimension of stimuli, this sequence of early arousal-driven and
being enhanced for both pleasantly and late valence-driven responses. Because
unpleasantly arousing words. By contrast, this pattern is seen most clearly when
322 JOHANNA KISSLER
adjectives, which often describe states or ments may, for instance, reflect fast condi-
traits, are used as stimuli (Herbert et al., tioned responses to a few highly individ-
2006, 2008), the degree of self-referential ually relevant words. Indeed, evidence for
processing induced by the words may a role of operant conditioning in generat-
determine processing in the LPP window. ing very early emotion effects in words.
At least for student subjects, this latter As in many areas of affective neuroscience
thesis is supported by recent rating data the amygdala has been shown to be active
(Herbert, et al., 2008, 2009). in emotion word processing, and given its
Variation of the linguistic reference (e.g., established prominent role in affective con-
“your success” or “my success” versus “your ditioning and in emotional vision in gen-
fear” or “my fear”) has been found to mod- eral it is likely that it will contribute to
ulate early and late emotion-related ERPs some of these ERP enhancements. How-
during reading (Herbert, Herbert, Ethofer, ever, its precise role in the processing of
& Pauli, 2011). It also provides an impor- emotional language is much less well under-
tant step toward more linguistically and stood than its role in the processing of
socially complex designs. Furthermore, the emotional faces. Several of the previously
internal semantic structure of positive ver- described effects may therefore not rely
sus negative language may also contribute to on online amygdala activity (or the online
valence-specific processing biases (Hofmann activity of any other back-projecting struc-
et al., 2009; Kuchinke et al., 2005). Positive ture), but result from established proper-
information may be better integrated within ties of an individual’s mental lexicon. Con-
associative networks and be represented in a ceivably, structures like the amygdala are
more cohesive manner than negative infor- involved only during the acquisition of a
mation, thereby facilitating the processing word’s or phrase’s emotional significance,
of positive words. permanently enhancing an item’s accessibil-
Taken together, extant studies demon- ity. Thus, studying the processes by which
strate reliable amplification of visual ERPs linguistic symbols acquire emotional sig-
in response to emotional compared to neu- nificance will be highly relevant to the
tral words from about 200 ms after word functional characterization of ERP enhance-
presentation. Although enhanced responses ments to emotional words at different pro-
are frequently found for both positively and cessing stages.
negatively arousing words alike, particularly
for the later processing stages a larger impact
of pleasant content is often reported. Clar- Summary and Outlook
ification of the relative role of valence and
arousal on ERPs elicited by emotional lan- From the perspective of different method-
guage at different temporal processing stages ological approaches, this chapter outlined
awaits further research. Similarly, ERP find- how the processing of emotional language
ings are somewhat inconclusive regarding content differs from the processing of
the dominance of either hemisphere in the semantically neutral language. Emotional
processing of emotional language. However, language is unique in that it can be partly
current findings are in line with the view that preserved in otherwise aphasic patients with
emotional content amplifies lexico-semantic left-hemisphere strokes. Data from the dam-
word processing after initial stimulus iden- aged brain indicate a special capacity of
tification. This enhancement may occur by the right hemisphere for processing emo-
means of amplificatory reentrant process- tional language, but functional neuroimag-
ing via bidirectional connections between ing data from healthy people indicate that
extrastriate visual cortex and the amygdala. such processing normally occurs predomi-
However, given that even earlier, pre-lexical nantly in the left hemisphere. Regions more
responses to emotional words have been strongly activated by emotional than by
identified, other mechanisms may also play content-neutral language include the sec-
a role. Such pre-semantic response enhance- ondary visual and auditory cortices and
LOVE LETTERS AND HATE MAIL 323
mid-temporal and inferior frontal regions, tional significance of language often arises
which are typically activated during seman- from its significance in communicatory
tic processing; medial and orbitofrontal context, practically no research has been
regions that are commonly involved in emo- conducted using more realistic communi-
tional evaluation; and the amygdala, insula, catory settings, establishing clear sender–
and basal ganglia. Regarding the tempo- receiver relationships. Thus, as in much
ral evolution of emotion effects in word of cognitive and affective neuroscience, a
processing, effects have been found from knowledge base has been built, but many
the earliest stages (N100, EPN), appar- of the interesting issues are still unresolved.
ently rapidly alerting the individual to a
word’s or phrase’s emotional significance,
Outstanding Questions
but also affecting contextual integration
(N400), evaluation, and memory encoding r Spatiotemporal dynamics of emotional
(LPP) at later processing stages. Together,
language processing: How and when do
the current findings indicate activity of a
different brain regions interact to decode
widespread network of brain regions, includ-
the emotional meaning of language?
ing, but also exceeding, regions traditionally r Cerebral differentiation between emo-
viewed as part of the semantic processing
tional subcategories: How does the brain
system. Thus, from a neurolinguistic view-
differentiate between emotional cate-
point, the data appear to support the view of
gories and subtle emotional nuances that
a distributed, “modal” semantic system and
a listener or reader can interpret?
outline the elements of a processing system r Processing of complex emotional lan-
for emotional semantics.
guage: How does the brain integrate
Nevertheless, our knowledge on how the
the significance of emotional language
brain processes emotional language content
beyond individual words and very sim-
is incomplete in many respects: Although
ple phrases, laying the groundwork for
lesion and imaging studies have succeeded
the emotional experience of poetry and
in identifying the elements of a processing
literature?
system for emotional semantics, the interac- r Emotional language in realistic communi-
tion between these elements and its tempo-
catory settings: How are communicatory
ral course are virtually unknown. Moreover,
context and social sender–receiver rela-
we do not yet know how the brain differ-
tionships integrated in the processing of
entiates between emotional subcategories.
emotional language?
So far, most data reveal differences between
“the emotional” and “the mundane,” but it
is not even quite clear how the brain dif- References
ferentiates between the meaning of “good”
and “bad” (i.e., differentiates between pos- Adolphs, R., Damasio, H., Tranel, D., Cooper,
itive and negative language), even though G., & Damasio, A. R. (2000). A role for
a difference would appear salient. Clearly, somatosensory cortices in the visual recog-
our linguistic capabilities go far beyond dif- nition of emotion as revealed by three-
ferentiating the meanings of good, neutral, dimensional lesion mapping. Journal of Neu-
and bad, and obviously so should those roscience, 20(7), 2683–90.
of our brains. Furthermore, most research Anderson, A. K., & Phelps, E. A. (2001). Lesions
has been on the processing of single words of the human amygdala impair enhanced per-
ception of emotionally salient events. Nature,
or very short phrases. Yet, linguistic dis-
411(6835), 305–9.
course is normally much more complex, and
Ashby, F. G., Isen, A. M., & Turken, A. U.
neuroscience should begin to capture this (1999). A neuropsychological theory of posi-
complexity without losing its ambition to tive affect and its influence on cognition. Psy-
strive for experimental control. Finally, lan- chology Review, 106(3), 529–50.
guage is fundamentally about communicat- Barsalou, L. W. (2008). Grounded cognition.
ing with other people. Although the emo- Annual Review of Psychology, 59, 617–45.
324 JOHANNA KISSLER
Beauregard, M., Chertkow, H., Bub, D., Murtha, Cato-Jackson, M. A., & Crosson, B. (2006). Emo-
S., Dixon, R., & Evans, A. (1997). The neural tional connotation of words: Role of emotion
substrate for concrete, abstract, and emotional in distributed semantic systems. Progress in
word lexica: A positron emission tomogra- Brain Research, 156, 205–16.
phy study. Journal of Cognitive Neuroscience, Chwilla, D.J., Virgillito, D., Vissers, C. T. (2011).
9(4), 20. The relationship of language and emotion:
Begleiter, H., & Platz, A. (1969). Cortical evoked N400 support for an embodied view of lan-
potentials to semantic stimuli. Psychophysiol- guage comprehension. Journal of Cognitive
ogy, 6(1), 91–100. Neuroscience, 23(9), 2400–14.
Bernat, E., Bunce, S., & Shevrin, H. (2001). Event- Codispoti, M., Mazzetti, M., & Bradley, M. M.
related brain potentials differentiate positive (2009). Unmasking emotion: Exposure dura-
and negative mood adjectives during both tion and emotional engagement. Psychophysi-
supraliminal and subliminal visual processing. ology, 46(4), 731–38.
International Journal of Psychophysiology, 42(1), Cohen, L., Lehericy, S., Chochon, F., Lemer, C.,
11–34. Rivaud, S., & Dehaene, S. (2002). Language-
Binder, J. R., Desai, R. H., Graves, W. W., & specific tuning of visual cortex? Functional
Conant, L. L. (2009). Where is the semantic properties of the Visual Word Form Area.
system? A critical review and meta-analysis of Brain, 125(Pt. 5), 1054–69.
120 functional neuroimaging studies. Cerebral Davidson, R. J., & Irwin, W. (1999). The func-
Cortex, 19(12), 2767–96. tional neuroanatomy of emotion and affec-
Blonder L. X., Bowers D., & Heilman K. M. tive style. Trends in Cognitive Sciences, 3(1),
(1991). The role of the right hemisphere 11–21.
in emotional communication. Brain, 114(3), Del Cul, A., Baillet, S., & Dehaene, S. (2007).
1115–27. Brain dynamics underlying the nonlinear
Bloom, R. L., Borod, J. C., Obler, R. K., & Ger- threshold for access to consciousness. PLoS
stman, L. J. (1992). Impact of emotional con- Biol, 5(10), e260.
tent on discourse production in patients with Dien, J., Spencer, K. M., & Donchin, E. (2004).
unilateral brain damage. Brain and Language, Parsing the late positive complex: Mental
42(2), 153–64. chronometry and the ERP components that
Borod, J. C., Bloom, R. L., Brickman, A. M., inhabit the neighborhood of the P300. Psy-
Nakhutina, L., & Curko, E. A. (2002). Emo- chophysiology, 41(5), 665–78.
tional processing deficits in individuals with Diener, E., & Diener, C. (1996). Most people are
unilateral brain damage. Applied Neuropsy- happy. Psychological Science, 7(3), 181–85.
chology, 9(1), 23–36. Dolcos, F., & Cabeza, R. (2002). Event-related
Borod, J. C., Cicero, B. A., Obler, L. K., potentials of emotional memory: Encoding
Welkowitz, J., Erhan, H. M., Santschi, C., pleasant, unpleasant, and neutral pictures.
et al. (1998). Right hemisphere emotional per- Cognitive, Affective & Behavioral Neuroscience,
ception: Evidence across multiple channels. 2(3), 252–63.
Neuropsychology, 12(3), 446–58. Eimer, M., & Holmes, A. (2007). Event-related
Bradley, M. M., & Lang, P. J. (1994). Measur- brain potential correlates of emotional face
ing emotion: The self-assessment manikin and processing. Neuropsychologia, 45(1), 15–31.
the semantic differential. Journal of Behavioral Eviatar, Z., & Zaidel, E. (1991). The effects of
Therapy and Experimental Psychiatry, 25(1), word length and emotionality on hemispheric
49–59. contribution to lexical decision. Neuropsy-
Buchanan, T. W., Denburg, N. L., Tranel, D., chologia, 29(5), 415–28.
& Adolphs, R. (2001). Verbal and nonver- Federmeier, K. D., Kirson, D. A., Moreno, E.
bal emotional memory following unilateral M., & Kutas, M. (2001). Effects of transient,
amygdala damage. Learning and Memory, 8(6), mild mood states on semantic memory orga-
326–35. nization and use: An event-related potential
Cato, M. A., Crosson, B., Gokcay, D., Soltysik, investigation in humans. Neuroscience Letters,
D., Wierenga, C., Gopinath, K., et al. (2004). 305(3), 149–52.
Processing words with emotional connotation: Fischler, I., & Bradley, M. (2006). Event-related
an FMRI study of time course and laterality in potential studies of language and emotion:
rostral frontal and retrosplenial cortices. Jour- words, phrases, and task effects. Progress in
nal of Cognitive Neuroscience, 16(2), 167–77. Brain Research, 156, 185–203.
LOVE LETTERS AND HATE MAIL 325
Fontaine, J. R., Scherer, K. R., Roesch, E. B., & Hinojosa, J. A., Martin-Loeches, M., Munoz, F.,
Ellsworth, P. C. (2007). The world of emotions Casado, P., & Pozo, M. A. (2004). Electrophys-
is not two-dimensional. Psychological Science, iological evidence of automatic early semantic
18(12), 1050–57. processing. Brain and Language, 88(1), 39–46.
Fossati, P., Hevenor, S. J., Graham, S. J., Grady, Hinojosa, J. A., Mendez-Bertolo, C., & Pozo, M.
C., Keightley, M. L., Craik, F., et al. (2003). A. (2010). Looking at emotional words is not
In search of the emotional self: An fMRI the same as reading emotional words: Behav-
study using positive and negative emotional ioral and neural correlates. Psychophysiology,
words. American Journal of Psychiatry, 160(11), 47(4), 748–57.
1938–45. Hofmann, M. J., Kuchinke, L., Tamm, S., Vo, M.
Fredrickson, B. L. (2001). The role of posi- L., & Jacobs, A. M. (2009). Affective process-
tive emotions in positive psychology. The ing within 1/10th of a second: High arousal
broaden-and-build theory of positive emo- is necessary for early facilitative processing
tions. American Psychologist, 56(3), 218–26. of negative but not positive words. Cognitive,
Gaillard, R., Del Cul, A., Naccache, L., Vinckier, Affective & Behavioral Neuroscience, 9(4), 389–
F., Cohen, L., & Dehaene, S. (2006). Non- 97.
conscious semantic processing of emotional Holcomb, P. J., Kounios, J., Anderson, J. E., &
words modulates conscious access. Proceedings West, W. C. (1999). Dual-coding, context-
of the National Academy of Sciences, 103(19), availability, and concreteness effects in sen-
7524–29. tence comprehension: an electrophysiological
Gazzaniga, M. S., & Hillyard, S. A. (1971). Lan- investigation. Journal of Experimental Psychol-
guage and speech capacity of the right hemi- ogy: Learning, Memory & Cognition, 25(3), 721–
sphere. Neuropsychologia, 9(3), 273–80. 42.
Graves, R., Landis, T., & Goodglass, H. (1981). Hughlings Jackson, J (1866). Clinical remarks on
Laterality and sex differences for visual recog- emotional and intellectual language in some
nition of emotional and non-emotional words. cases of disease of the nervous system. Lancet,
Neuropsychologia, 19(1), 95–102. i, 5.
Hauk, O., Davis, M. H., Kherif, F., & Pul- Isenberg, N., Silbersweig, D., Engelien, A.,
vermuller, F. (2008). Imagery or meaning? Emmerich, S., Malavade, K., Beattie, B., et al.
Evidence for a semantic origin of category- (1999). Linguistic threat activates the human
specific brain activity in metabolic imag- amygdala. Proceedings of the National Academy
ing. European Journal of Neuroscience, 27(7), of Sciences, 96(18), 10456–59.
1856–66. Junghofer, M., Bradley, M. M., Elbert, T. R., &
Herbert, C., Ethofer, T., Anders, S., Junghofer, Lang, P. J. (2001). Fleeting images: A new look
M., Wildgruber, D., Grodd, W., et al. (2009). at early emotion discrimination. Psychophysi-
Amygdala activation during reading of emo- ology, 38(2), 175–78.
tional adjectives – an advantage for pleasant Kanske, P., & Kotz, S. A. (2007). Concreteness in
content. Social Cognition and Affective Neuro- emotional words: ERP evidence from a hemi-
science, 4(1), 35–49. field study. Brain Research, 1148, 138–48.
Herbert, C., Herbert, B. M., Ethofer, T., & Pauli, Keil, A., Sabatinelli, D., Ding, M., Lang, P. J.,
P. (2011). His or mine? The time course of Ihssen, N., & Heim, S. (2009). Re-entrant pro-
self-other discrimination in emotion process- jections modulate visual cortex in affective
ing. Social Neuroscience, 6(3), 277–88. perception: Evidence from Granger causality
Herbert, C., Junghofer, M., & Kissler, J. (2008). analysis. Human Brain Mapping, 30(2), 532–40.
Event related potentials to emotional adjec- Kensinger, E. A., & Corkin, S. (2004). Two routes
tives during reading. Psychophysiology, 45(3), to emotional memory: Distinct neural pro-
487–98. cesses for valence and arousal. Proceedings of
Herbert, C., Kissler, J., Junghofer, M., Peyk, P., the National Academy of Sciences,, 101(9), 3310–
& Rockstroh, B. (2006). Processing of emo- 15.
tional adjectives: Evidence from startle EMG Kensinger, E. A., & Schacter, D. L. (2006). Pro-
and ERPs. Psychophysiology, 43(2), 197–206. cessing emotional pictures and words: Effects
Hillier, A., Beversdorf, D. Q., Raymer, A. M., of valence and arousal. Cognitive, Affective &
Williamson, D. J., & Heilman, K. M. (2007). Behavioral Neuroscience, 6(2), 110–26.
Abnormal emotional word ratings in Parkin- Kiehl, K. A., Hare, R. D., McDonald, J. J.,
son”s disease. Neurocase, 13(2), 81–85. & Brink, J. (1999). Semantic and affective
326 JOHANNA KISSLER
Palazova, M., Mantwill, K., Sommer, W., & related brain potentials evidence from a super-
Schacht, A. (2011). Are effects of emotion ficial task. Brain Cognition.
in single words non-lexical? Evidence from Richter, M., Eck, J., Straube, T., Miltner, W. H.,
event-related brain potentials. Neuropsycholo- & Weiss, T. (2010). Do words hurt? Brain acti-
gia, 49(9), 2766–75. vation during the processing of pain-related
Pauli, P., Amrhein, C., Muhlberger, A., Dengler, words. Pain, 148(2), 198–205.
W., & Wiedemann, G. (2005). Electrocortical Russell, J. A. (2003). Core affect and the psycho-
evidence for an early abnormal processing of logical construction of emotion. Psychological
panic-related words in panic disorder patients. Review, 110(1), 145–72.
International Journal of Psychophysiology, 57(1), Sabatinelli, D., Versace, F., Costa, V. D., Bradley,
33–41. M. M., & Lang, P. J. (2006). Selective nucleus
Paulmann, S., & Kotz, S. A. (2008). An ERP inves- accumbens and medial frontal cortex acti-
tigation on the temporal dynamics of emo- vation in appetitive picture processing. Psy-
tional prosody and emotional semantics in chophysiology, 43, S84.
pseudo- and lexical-sentence context. Brain Sander, D., Grandjean, D., & Scherer, K. R.
and Language, 105(1), 59–69. (2005). A systems approach to appraisal mech-
Paulmann, S., & Pell, M. D. (2010). Dynamic anisms in emotion. Neural Networks, 18(4),
emotion processing in Parkinson’s disease as 317–52.
a function of channel availability. Journal Schacht, A., & Sommer, W. (2009). Time course
of Clinical and Experimental Neuropsychology, and task dependence of emotion effects in
1–14. word processing. Cognitive, Affectove & Behav-
Paulmann, S., Pell, M. D., & Kotz, S. A. (2009). ioral Neuroscience, 9(1), 28–43.
Comparative processing of emotional prosody Schacht, A, Adler, N, & Chen, P, Guo, T, Som-
and semantics following basal ganglia infarcts: mer, W. (2012). Association with positive out-
ERP evidence of selective impairments for dis- come induces early effects in event-related
gust and fear. Brain Research, 1295, 159–69. brain potentials. Biological Psychology, 89(1),
Perfetti, C. A., & Sandak, R. (2000). Reading opti- 130–6.
mally builds on spoken language: implications Schapkin, S. A., Gusev, A. N., & Kuhl, J. (2000).
for deaf readers. Journal of Deaf Studies and Categorization of unilaterally presented emo-
Deaf Education, 5(1), 32–50. tional words: an ERP analysis. Acta Neu-
Pessoa, L., McKenna, M., Gutierrez, E., & Unger- robiologiae Experimentalis (Warsaw), 60(1),
leider, L. G. (2002). Neural processing of 17–28.
emotional faces requires attention. Proceedings Schirmer, A., Kotz, S. A., & Friederici, A. D.
of the National Academy of Sciences, 99(17), (2002). Sex differentiates the role of emo-
11458–63. tional prosody during word processing. Brain
Phillips, M. L., Young, A. W., Senior, C., Bram- Research: Cognitive Brain Research, 14(2), 228–
mer, M., Andrew, C., Calder, A. J., et al. 33.
(1997). A specific neural substrate for per- Schupp, H. T., Flaisch, T., Stockburger, J., &
ceiving facial expressions of disgust. Nature, Junghofer, M. (2006). Emotion and attention:
389(6650), 495–98. Event-related brain potential studies. Progress
Pulvermüller, F. (1999). Words in the brain’s lan- in Brain Research, 156, 31–51.
guage. Behavioral Brain Science, 22(2), 253–79; Scott, G. G., O’Donnell, P. J., Leuthold, H., &
discussion 280–336. Sereno, S. C. (2009). Early emotion word pro-
Pulvermuller, F., & Fadiga, L. (2010). Active cessing: Evidence from event-related poten-
perception: Sensorimotor circuits as a corti- tials. Biological Psychology, 80(1), 95–104.
cal basis for language. Nature Reviews Neuro- Silva C., Montant M., Ponz A., & Ziegler J.C.
science, 11(5), 351–60. (2012). Emotions in reading: Disgust, empathy
Rapcsak, S. Z., Comer, J. F., & Rubens, A. B. and the contextual learning hypothesis. Cogni-
(1993). Anomia for facial expressions: Neu- tion. 2012 Aug 9. [Epub ahead of print] PMID:
ropsychological mechanisms and anatomical 22884243.
correlates. Brain and Language, 45(2), 233–52. Silvert, L., Delplanque, S., Bouwalerh, H.,
Rellecke, J., Palazova, M., Sommer, W., & Verpoort, C., & Sequeira, H. (2004).
Schacht, A. (2011). On the automaticity of Autonomic responding to aversive words
emotion processing in words and faces: Event- without conscious valence discrimination.
328 JOHANNA KISSLER
International Journal of Psychophysiology, 53(2), state visual evoked potential study. Neuroim-
135–45. age, 60(1),130–8.
Skrandies, W. (1998). Evoked potential correlates Van Lancker Sidtis, D., Pachana, N., Cummings,
of semantic meaning – A brain mapping study. J. L., & Sidtis, J. J. (2006). Dysprosodic speech
Brain Research: Cognitive Brain Research, 6(3), following basal ganglia insult: toward a con-
173–83. ceptual framework for the study of the cere-
Smith, A. (1966). Speech and other functions bral representation of prosody. Brain and Lan-
after left (dominant) hemispherectomy. Jour- guage, 97(2), 135–53.
nal of Neurology, Neurosurgery and Psychiatry, Vanderploeg, R. D., Brown, W. S., & Marsh, J.
29(5), 467–71. T. (1987). Judgments of emotion in words and
Stolarova, M., Keil, A., & Moratti, S. (2006). faces – ERP correlates. International Journal of
Modulation of the C1 visual event-related Psychophysiology, 5(3), 193–205.
component by conditioned stimuli: Evi- Vissers, C.T., Virgillito, D., Fitzgerald, D.A,
dence for sensory plasticity in early affec- Speckens, A.E, Tendolkar, I., van Oostrom,
tive perception. Cerebral Cortex, 16(6), 876– I., & Chwilla, D.J. (2010). The influence of
87. mood on the processing of syntactic anoma-
Sutton, S., Tueting, P., Zubin, J., & John, lies: evidence from P600. Neuropsychologia,
E. R. (1967). Information delivery and the 48(12), 3521–31.
sensory evoked potential. Science, 155(768), Vuilleumier, P., Richardson, M. P., Armony,
1436–39. J. L., Driver, J., & Dolan, R. J. (2004).
Takahashi, H., Takano, H., Kodaka, F., Arakawa, Distant influences of amygdala lesion on
R., Yamada, M., Otsuka, T., et al. (2010). Con- visual cortical activation during emotional
tribution of dopamine D1 and D2 receptors to face processing. Nature Neuroscience, 7(11),
amygdala activity in human. Journal of Neuro- 1271–78.
science, 30(8), 3043–47. Wicker, B., Keysers, C., Plailly, J., Royet, J. P.,
Trauer, S.M., Andersen, S.K., Kotz, S.A., & Gallese, V., & Rizzolatti, G. (2003). Both of
Müller, M.M. (2012). Capture of lexical but us disgusted in My insula: The common neu-
not visual resources by task-irrelevant emo- ral basis of seeing and feeling disgust. Neuron,
tional words: a combined ERP and steady- 40(3), 655–64.
Section IV
COGNITIVE-EMOTION
INTERACTIONS
CHAPTER 14
In real life, sensory behaviorally relevant defining component of many, if not most,
stimuli must compete for conscious repre- emotions (Scherer & Peper, 2001). Among
sentation among distracters to elicit appro- these effects, compelling evidence indicates
priate responses. Selective attention refers to that some emotions may act to facilitate
various brain processes that serve to resolve attention to significant stimuli, thereby reg-
this competition by controlling the alloca- ulating sensory processing. Such emotional
tion of processing resources as a function of biases in perception and attention may favor
the salience of external events, as well as the adaptive behavior and survival in some con-
internal goals of the observer. Affective pro- ditions, but also may potentially contribute
cesses interact closely with these attentional to pathological situations, such as in anxiety
mechanisms. Just like attention, emotion or phobia (see Chapter 24). The exact
may guide perception and behavior based on mechanisms involved in these effects, as
the biological or motivational significance of well as their specificities and commonalities
sensory information, as well as on internal with respect to other attentional processes,
states (Pourtois, Schettino, & Vuilleumier, are incompletely known. However, recent
2012; Vuilleumier, 2005). In this chapter we research has provided many novel insights
present a general overview of these effects by using a variety of experimental tech-
and of current knowledge about their neu- niques, including behavioral measures,
ral mechanisms. We focus on the impact of functional neuroimaging in humans, and
emotion on attention, whereas the reverse single-unit recordings in animals. A large
interactions are reviewed by Pessoa and col- part of this research focuses on negative emo-
leagues in Chapter 15. tional events (i.e., threat) and often on the
Emotion is crucial for orchestrating a visual modality only, but attention-biasing
wide range of physiological reactions that effects of emotion extend to other domains
allow the organism to adjust to the environ- than fear, as indicated by a growing number
ment. A modulation of action tendencies of studies on a range of affective dimen-
and of the current focus of thoughts is a sions, including positive (happiness, erotica,
331
332 JUDITH DOMÍNGUEZ-BORRÀS & PATRIK VUILLEUMIER
food) or nonthreatening negative cues, such they may slow down target detection (e.g.,
as sadness or disgust. Nevertheless, there Fenske & Eastwood, 2003), suggesting that
are also some reasons to think that fear some degree of involuntary emotional pro-
may represent a particular kind of affect cessing may operate even when it is irrele-
that has, by essence, a stronger impact on vant (or counterproductive) to the task per-
attention focus. In addition, many studies formance.
on the neural substrates of emotional biases However, enhanced detection of emo-
in attention have focused on the role of the tional targets (or interference by emotional
amygdala (Vuilleumier, 2005); however, distracters) does not necessarily imply that
it is likely that other brain regions make emotional stimuli are unaffected by current
important contributions too, particularly attention focus or that they simply pop out
the (so-called limbic) prefrontal areas (e.g., from a scene (e.g., as a red target does among
orbitofrontal cortex [OFC] and anterior blue shapes). Thus, the search for emotional
cingulate cortex [ACC]) and subcortical stimuli is still serial and its facilitation only
regions in the basal ganglia, thalamus, and relative, reflecting a preferential guidance of
brainstem. However, their exact role still attention to their location, rather than an
remains to be determined. obligatory breaking through into attention.
In this chapter we review behavioral and In visual search tasks with targets differing
neuroimaging data illustrating the impact of in low-level features, detection latencies do
threat and other emotional signals on atten- not increase substantially with an increasing
tion and perception. We present both cur- number of distracters, whereas latencies for
rent models and remaining issues concern- emotional targets increase with the number
ing the brain mechanisms subserving these of distracters but with a shallower slope than
effects. neutral targets (Gerritsen et al., 2008; Lucas
& Vuilleumier, 2008). This indicates that
emotional processing may bias and speed up
Behavioral Evidence for Emotional detection, but does not constitute a short-
Biases in Attention cut to conscious perception. However, this
emotional advantage is observed only when
A great deal of research indicates that, when all items are presented together on the array,
concurrent stimuli compete for process- but disappears when items are revealed one
ing resources and awareness, information by one at locations that participants con-
with emotionally relevant meaning is often secutively explore with a mouse pointer
detected more readily than neutral infor- (Smilek, Frischen, Reynolds, Gerritsen, &
mation or is more likely to interfere with Eastwood, 2007). These findings suggest that
another concomitant task. Classic exam- the facilitation of detection depends on a
ples come from visual search experiments coarse perceptual analysis operating outside
showing that the detection of a visual tar- or before attentive fixation, rather than on
get among an array of distracters is quicker later processes, such as quicker recognition
for emotional than for neutral targets. This or quicker response selection, after attention
effect is often reported for negative or has been focused. Furthermore, this preat-
threat-related stimuli, such as faces with tentive analysis might be limited to certain
angry and fearful expressions among neu- regions in the visual field or certain stimu-
tral faces, or pictures of snakes and spiders lus characteristics, because involuntary cap-
among fruits or plants (see Figure 14.1a,b; ture by task-irrelevant emotional faces is
Flykt & Caldara, 2006; Gerritsen, Frischen, stronger when they are presented at loca-
Blake, Smilek, & Eastwood, 2008); but it tions where targets are also expected, but
also occurs for positive stimuli (Cunning- weaker for distracters presented at other
ham & Brosch, 2012). Alternatively, when locations (Huang, Chang, & Chen, 2011;
emotional stimuli are distracters embed- see also Notebaert, Crombez, Van Damme,
ded in search or selective attention tasks, De Houwer, & Theeuwes, 2010). Likewise,
AFFECTIVE BIASES IN ATTENTION AND PERCEPTION 333
Mean RT (ms)
1000
Spider target
950 894 Snake target
857 Fear-irrelevant
900
822 target
850 798
772
800 760
750
Spider Snake Nonfearful
fearful fearful
(c)
Figure 14.1. Effects of the emotional meaning of stimuli on attention and brain responses. (A)
Example of a visual search task where an emotional target (spider or snake) must be detected
(present/absent) among neutral images (mushrooms). (B) Search performance (response time, RT) of
spider fearful, snake fearful, and nonfearful participants. Fearful participants were faster at detecting
their feared targets, compared to both nonfeared (but still fear-relevant) and nonemotional targets.
Adapted from Flykt & Caldara (2006). (C) Blood-oxygenation-level-dependent (BOLD) response
observed with fMRI shows a precise overlap between emotional modulation effects and
category-selective areas for face (fusiform face area; FFA), body (extrastriate body area; EBA), and
voice processing, demonstrating that the emotional modulation is category specific. Adapted from
Vuilleumier et al. (2001), Grandjean et al., (2005) and Peelen et al. (2007). See color plate 14.1.
preferential attention to threatening (e.g., Other attentional tasks are also influ-
angry) than neutral faces is observed when enced by emotional cues. Whereas visual
searching for any discrepant facial expres- search highlights competition for process-
sion among an homogeneous crowd of ing resources between concurrent stimuli
faces, but not when specifically searching close in space, a different type of compe-
for another specific (e.g., happy) expression tition between stimuli appearing close in
(Hahn & Gronlund, 2007); this finding sug- time occurs during the attentional blink phe-
gests that current top-down goals can mod- nomenon. When different items are shown
ulate the impact of more involuntary biases one after the other at fixation in a rapid
guiding attention to emotionally salient serial visual presentation stream (RSVP),
stimuli. Taken together, these findings indi- the detection of a given target is typi-
cate that emotional influences on visual cally impaired when it occurs shortly (200–
search and perception may be unintentional 500 ms) after another target, as if atten-
and preattentive but not fully automatic, tion capacity transiently blinked. However,
partly depending on the attentional goals, this impairment is reduced when the sec-
resources, or internal states of the observer. ond target is emotional, rather than neutral
334 JUDITH DOMÍNGUEZ-BORRÀS & PATRIK VUILLEUMIER
(Anderson and Phelps, 2001; De Martino, Target detection is typically faster when the
Kalisch, Rees, & Dolan, 2009; Schwabe emotional cue is shown at the same loca-
et al., 2011). This effect has also been tion (valid), but slower when it is presented
observed with words, with either negative at a different location (invalid), suggesting
or positive meaning (Anderson & Phelps, that attention is initially oriented to the
2001). Conversely, attentional blink for the emotional stimulus and facilitates respond-
second target may increase when it is neu- ing to the subsequent target. This facilita-
tral but is preceded by a first emotional tar- tion involves reflexive (exogenous) orienting
get (Schwabe et al., 2011). These emotional mechanisms, because it occurs after short
effects are observed in tasks requiring dual- time intervals between the cue and tar-
target detection, where access to processing get (< 300 ms) and even when the cue is
resources is taxed, but not in conditions with masked (Mogg & Bradley, 2002). Likewise,
single targets where no competition occurs a brief emotional stimulus (e.g., a fearful
(Anderson & Phelps, 2001). Again, this face presented for 75 ms) can enhance con-
finding suggests that emotional processes trast sensitivity and potentiate the effect of
operate at perceptual stages in which pro- spatial attention on detection accuracy for
cessing resources are limited, thereby con- a subsequent visual target (gabor pattern),
trolling access to awareness, rather than suggesting a modulation of early percep-
stimulus recognition per se. Interestingly, in tual processing in the primary visual cortex
attentional blink conditions, a better detec- (Phelps, Ling, & Carrasco, 2006). In keep-
tion of emotional words, relative to neutral, ing with reflexive orienting, eye movement
may occur only when semantic processing studies also show faster saccades and ear-
is required and not during a purely percep- lier fixation on emotional than neutral faces,
tual or phonological task (Huang, Baddeley, but with some differences across expression
& Young, 2008). Task goals and expecta- categories (Nummenmaa, Hyona, & Calvo,
tions may thus also interact with involuntary 2009).
emotional biases in these conditions. Taken together, these behavioral findings
An emotional advantage is also observed converge to indicate that affective biases
in change blindness paradigms where a stim- may guide attention and enhance perception
ulus in a scene is unexpectedly changed for emotionally significant stimuli across var-
into another one (Peelen, Lucas, Mayer, & ious conditions. These biases are generally
Vuilleumier, 2009). Other examples come unintentional, independent of explicit rel-
from studies using binocular rivalry (in evance, and triggered without overt atten-
which each eye is presented with a differ- tion, although they may be modulated
ent stimulus; Alpers & Gerdes, 2007) or con- by expectations, task characteristics, and
tinuous flash suppression (where a stimulus available resources for covert attention. It
presented to one eye is rendered invisible by remains to be seen, however, whether all
dynamic noise presented to the other eye; emotional biases in attention observed with
Yang, Zald, & Blake, 2007). In these con- this large variety of stimuli, emotions, and
ditions, fearful faces emerge into awareness paradigms are subserved by similar or partly
more frequently and more quickly than neu- distinct brain mechanisms.
tral faces, whereas disgust or happy faces
may not show the same advantage (Amting,
Greening, & Mitchell, 2010). Nature of Emotionally Salient Cues
Finally, in the dot-probe task, a neutral
target (e.g., dot) is preceded by a nonpre- A major question that still remains unre-
dictive cue with emotional meaning (e.g., solved is what makes an emotional stimulus
a face or picture; see Figure 14.2a) that is attention capturing. It is generally assumed
briefly presented at the same or at a different that the emotional meaning per se is
location (Armony & Dolan, 2002; Pourtois, crucial, but it is also possible that part of
Grandjean, Sander, & Vuilleumier, 2004). these effects are driven by physical features,
AFFECTIVE BIASES IN ATTENTION AND PERCEPTION 335
A B
L R
+
+ +
Left-sided Right-sided
C-6 Emotional valid
Emotional invalid targets targets
-4
fMRI response
2
6
P1
3 * 5
4
100 200 ms 4
2
3
D 50-100 ms 100-150 ms
1 2
1
0 Left Right
0 Left Right
Visual Visual Visual Visual
Field Field Field Field
target target
Emotional Emotional
Valid Invalid Emotional valid Emotional invalid
Figure 14.2. Typical pattern of neural responses elicited by emotional cues in a spatial dot-probe task.
(A) In this task, targets (vertical or horizontal bars) are always preceded by pairs of faces, one neutral
and one emotional, which do not predict the side of target presentation. To avoid eye movements,
participants are instructed to press a response button when the orientation of the bar presented in the
left or right upper visual field matches that of the thicker segment in the fixation cross. Adapted from
Pourtois, Grandjean, Sander, & Vuilleumier (2004). (B) Targets preceded by neutral cues at the same
location (i.e., with fearful cues presented contralaterally; that is, emotional invalid cues) lead to
reduced fMRI responses in the intraparietal sulcus (IPS) ipsilateral to the target, relative to when
targets are preceded by emotional cues on the same side (emotional valid). This suggests a capture of
attention by the contralateral emotional cue and a reduced ability to reorient to the target. Adapted
from Pourtois, Schwartz, Seghier, Lazeyras, & Vuilleumier (2006). (C) During EEG recording, targets
preceded by emotional cues (emotional valid) elicit a larger visual P1 component, relative to those
preceded by neutral cues (emotional invalid). This is consistent with enhanced visual processing of
targets when cued by emotion. Adapted from Pourtois et al. (2004). (D) Neural source estimation for
EEG modulations at different latencies after target onset. When the target is preceded by an emotional
valid cue, activity to the target is first increased in parietal areas (50–100 ms), presumably reflecting
top-down attentional signals induced by the emotional cue and responsible for faster spatial orienting
to the target location and enhanced processing in extrastriate visual cortex at the P1 latency (100–150
ms). When targets are preceded by an invalid neutral cue, increased activity is observed in the anterior
cingulate cortex instead, reflecting some interference and conflict in attentional orienting due to initial
capture by the fearful face on the opposite side. Adapted from Pourtois, Thut, Grave de Peralta,
Michel, & Vuilleumier (2005). See color plate 14.2.
such as contrast and spatial frequency for the capture by schematic emotional faces during
visual modality, loudness and spectral fre- visual search also occurs when using stimuli
quency for audition, or perceptual distinc- made of oblique lines, physically compara-
tiveness more generally. In fact, attention ble to negative (frown) and positive (smile)
336 JUDITH DOMÍNGUEZ-BORRÀS & PATRIK VUILLEUMIER
and be more sensitive to cognitive fac- However, the exact nature of the effective
tors than simpler (and more natural) stim- emotional signals remains largely unknown.
ulus categories. For example, emotional On the one hand, many results show that
words are better detected than neutral the arousal value of stimuli, rather than
words when presented in rapid succession their valence, may play a crucial role. On
in the attentional blink paradigm (Anderson the other hand, stronger biasing effects are
& Phelps, 2001). Masked negative words sometimes observed with negative, threat-
are also better detected than masked pos- ening stimuli (i.e., fear or anger) relative
itive words (Nasrallah, Carmel, & Lavie, to positive stimuli despite equal arousal.
2009), whereas a reduction of attentional Stronger impact of negative cues on atten-
blink has been reported for both negative tion may reflect a preferential tuning of
and positive words (Anderson & Phelps, the human brain to threat signals acquired
2001; Keil & Ihssen, 2004). These effects by experience or shaped through evolution.
are not explained by differences in visual Young infants, for instance, already turn
appearance between word categories, per- more quickly to images of snakes than flow-
sist when controlling for lexical frequency ers or other animals (Blue, 2010). Yet, differ-
or verbal distinctiveness, and, as noted ear- ent kinds of negative emotion might affect
lier, increase when participants must pro- attention and perception performance dif-
cess word meaning rather than just phonol- ferently. For example, compared to fear,
ogy (Huang et al., 2008). Altogether, these disgust expression appears to slow down
findings imply that written words may still rather than speed up the visual search for
undergo semantic processing under condi- faces (Krusemark & Li, 2011), and atten-
tions where attentional resources are lim- tional blink for a second target is reduced
ited (Naccache et al., 2005), and that their after disgusted faces but increased after fear-
semantic meaning may lower their thresh- ful faces (Vermeulen, Godefroid, & Mermil-
old to access consciousness (Anderson & lod, 2009). These differences accord with the
Phelps, 2001; Huang et al., 2008). Yet, these view that disgust cues may lead to sensory
results do not eliminate the possibility that inhibition rather than facilitation (see Chap-
emotion and semantic processing may be ter 2). Finally, positive or appetitive stim-
reduced when attentional resources are fur- uli, such as pictures of happy faces, babies,
ther depleted (see Chapter 13 and 15). In food, or, erotic scenes, can also influence
addition, Stroop interference effects are also attention (see Cunningham & Brosch, 2012).
observed when emotional words are pre- Some attentional effects may also be evoked
sented during a nonemotional judgment by nonemotional information when it is per-
task (e.g., counting words) or during an sonally relevant to the observer, such as food
emotional judgment task involving another items for people with eating disorders or
stimulus dimension (e.g., categorizing the even pictures from familiar TV programs
expressions of superimposed faces). These (see Cunningham & Brosch, 2012). These
data indicate that emotional word meaning findings give support to appraisal theories
can produce involuntary effects on selective suggesting that a central mechanism of emo-
attention, even when semantic processing tional processing and its impact on attention
is not required. Likewise, facial expressions orienting might reflect an evaluation of stim-
slow down face color judgments, and non- ulus relevance for the goals, needs, and well-
words associated to emotion through condi- being of the individual, regardless of valence
tioning interfere with word-recognition per- (see Chapter 1).
formance. Therefore, more research is needed to
In sum, there is converging evidence that clarify the affective dimensions responsible
emotional biases in attention and perception for the emotional attention bias, as well
are at least partly mediated by emotional as their interaction with individual dif-
processing, rather than just by a discrim- ferences, such as anxiety and personality
ination of distinctive perceptual features. traits (see also Chapter 24). Furthermore,
338 JUDITH DOMÍNGUEZ-BORRÀS & PATRIK VUILLEUMIER
tools) at the voxel-by-voxel level, but not code for face features or configuration more
with the degree of response to faces (versus than emotion information (see Chapter 7).
tools), and vice versa (Peelen et al., 2007). Accordingly, direct intracranial recordings
Some modulatory effects might also involve of fusiform responses to faces have shown an
more global increases, perhaps via indi- emotional enhancement starting only after
rect modulations from frontoparietal atten- the N170 peak (Pourtois, Spinelli, Seeck, &
tional systems or other neuromodulatory Vuilleumier, 2010a; Figure 14.3a). In some
mechanisms. cases, even earlier increases may occur for
As observed in behavioral tasks, emo- the C1 (a component arising ∼80–90 ms
tional effects in category-selective areas are poststimulus and presumably generated in
often determined by the arousal value of the primary visual cortex) when fearful or
stimuli, rather than by valence (e.g., Pee- threat-conditioned faces are presented in
len et al., 2007). However, some studies the peripheral visual field (Pourtois et al.,
did find stronger responses in the FFA to 2004; Stolarova et al., 2006). In the auditory
fearful than to happy faces, even though domain, fearful vocalizations or sounds
both were equally arousing (e.g., Morris conditioned with pleasant and unpleasant
et al., 1998). Emotional enhancement of scenes may also elicit larger ERPs relative
cortical processing is also observed when to neutral stimuli already ∼150 ms after
neutral stimuli become associated with an onset (see Chapter 11). It should be noted,
aversive value through classical condition- however, that some emotional effects may
ing (Armony & Dolan, 2002), again demon- not always reflect prioritized processing, but
strating a key role for the emotional signifi- instead emerge due to reduced habituation
cance rather than the physical features of the over time for emotionally laden stimuli (see
stimuli. It remains possible that arousal and Chapter 11).
valence produce interactive effects depend- In addition to these modulations of early
ing on task demands or concomitant endoge- perceptual responses, emotional stimuli also
nous attention (Monroe et al., 2011), two produce distinctive ERPs at longer laten-
factors that have not systematically been cies (i.e., 300–400 ms), characterized by
manipulated. modulations of the P3, or sustained late
Studies using time-resolved techniques positive potentials (LPPs; see Chapter 4).
such as event-related brain potentials These effects might be related to more
(ERPs) and magneto-encephalography elaborate affective and cognitive evalua-
(MEG) in humans also support the idea tions, as well as memory (see Chapter 4).
that an emotional modulation of perception Source localization analyses suggest that
may take place at early stages of stimulus these LPPs are generated by a widespread
processing (see Chapter 4). Early com- cortical network including prefrontal, cin-
ponents associated with visual attention gulate and parietal, associated with volun-
∼100–200 ms after stimulus onset show tary attentional control and emotion reg-
increased amplitudes to emotional faces, ulation. One study combining EEG and
emotional scenes, or fear-conditioned stim- fMRI in the same participants found, how-
uli (Dolan, Heinze, Hurlemann, & Hinrichs, ever, that the amplitude of LPPs was
2006; Pourtois et al., 2004). These effects also correlated with increased hemody-
overlap with the P1 and N1 exogenous namic responses in occipitotemporal areas
waveforms generated in early extrastriate (Sabatinelli, Lang, Keil, & Bradley, 2007).
cortex, which are also typically enhanced Moreover, the face-responsive fusiform cor-
by top-down and bottom-up attention. tex may show sustained activation to fear-
Similar effects occur for emotional words ful expressions (> 700–800 s), clearly out-
(see Chapter 13). Less frequent modulations lasting stimulus duration itself (Pourtois
are reported for components associated et al., 2010a).
with object-categorization processes, such As with fMRI, emotional enhancement
as the occipitotemporal N170, which may of sensory responses in EEG occurs for
340 JUDITH DOMÍNGUEZ-BORRÀS & PATRIK VUILLEUMIER
Figure 14.3. Intracranial recordings of emotion and attention effects during face processing in patients
prior to epilepsy surgery. (A) Average local-field potentials recorded from an electrode over the right
fusiform gyrus, showing the average response to neutral and fearful faces during a one-back repetition
task. An increased cortical response to fearful faces, compared to neutral, arose after the face-specific
N200 (the intracranial counterpart of the N170), in the form of a sustained negative component lasting
more than 500 ms. Adapted from Pourtois, Spinelli, Seeck, & Vuilleumier (2010a). (B) Average
local-field potentials recorded in the amygdala during the face-house attention task illustrated in
figure 14.4A, showing the main effect of emotion (subtraction between neutral and fearful faces; dark
line) and the main effect of attention (subtraction between task-irrelevant and task-relevant faces;
light line). The emotional modulation started around 130–140 ms, whereas the attentional modulation
appeared later, around 600–700 ms poststimulus onset. Adapted from Pourtois, Spinelli, Seeck, &
Vuilleumier (2010b).
both negative and positive stimuli. Thus, positive stimuli (Weymar, Low, Öhman,
although abundant research has focused on & Hamm, 2011). Again, however, different
fearful and angry faces, more pleasant and types of negative emotion might produce
arousing stimuli elicit very similar increases different effects. For instance, although dis-
(at both early and late latencies), rela- gust represents a bodily threat and speeds
tive to neutral images, including images of up target detection in behavioral stud-
babies (see Cunningham & Brosch, 2012). ies (Bayle, Schoendorff, Henaff, & Krolak-
Nonetheless, these effects are sometimes Salmon, 2011), it was found to elicit weaker
stronger or even earlier for threatening than sensory responses in EEG in some studies,
AFFECTIVE BIASES IN ATTENTION AND PERCEPTION 341
intraparietal sulcus (IPS) ipsilateral to the 2005). Conditions associated with selective
targets, consistent with a capture of atten- increases in amygdala, such as exaggerated
tion by the emotional cue on the contralat- responses to pictures of snakes in individu-
eral side and a reduced ability to reorient als with snake phobia, relative to nonpho-
to the target on the ipsilateral side (Pour- bics, are associated with similar exaggerated
tois, Schwartz, Seghier, Lazeyras, & Vuilleu- increases in visual areas (e.g., Sabatinelli et
mier, 2006; see Figure 14.2b). In addition, al., 2005), in parallel with enhanced behav-
targets appearing after an emotional cue pro- ioral detection (Flykt & Caldara, 2006). Sec-
duce stronger BOLD responses in the lat- ond, the profile of amygdala reactivity is
eral occipital cortex (Pourtois et al., 2006) generally compatible with biases observed
and a larger P1 component in EEG record- in behavioral performance or in sensory
ings (Pourtois et al., 2004; Figure 14.2c), regions. Indeed, the amygdala seems to
consistent with improved visual process- primarily activate in response to the arousal
ing and better target detection (Phelps et or relevance value of sensory events, rather
al., 2006). A detailed analysis of the time than to negative valence only, although
course of these effects with EEG (Pourtois arousal-valence interactions and stronger
et al., 2005) suggests that the modulation of responses to threat are frequently observed
parietal areas in this paradigm may be trig- (see Chapter 1).
gered by an initial response to the emo- Second, the amygdaloid complex is well
tional cue, and subsequently induce top- designed to exert modulations on corti-
down spatial attentional signals responsible cal pathways involved in perception and
for enhanced processing of the target, but attention, because it entertains bidirectional
only when the target appears at the same connections with all sensory systems. In
location as the emotional cue (Figure 14.2d). the visual modality, tracing studies in the
Further research is needed, however, to clar- macaque show that projections to visual cor-
ify how emotion inputs reach parietal areas tices are highly organized, so that rostral
and to what extent the spatial cueing effects regions of the amygdala project to rostral
depend on expectations about possible tar- (i.e., higher level) visual areas, whereas cau-
get locations (e.g., Huang et al., 2011; Note- dal regions of the amygdala project to cau-
baert et al., 2010). dal (i.e., lower level) visual areas (Freese
& Amaral, 2006). At the microscopic level,
there is evidence that projections from the
Role of the Amygdala in Emotional amygdala reach pyramidal neurons in early
Bias Signals visual areas with synaptic patterns sug-
gestive of excitatory feedback (Freese &
What brain circuits are responsible for boost- Amaral, 2006). Similar feedback projections
ing sensorineural responses to emotional have been observed for the auditory and
stimuli, and are they different from atten- somatosensory modalities. In humans, MRI
tional mechanisms driven by frontoparietal studies using diffusion tensor imaging (DTI)
areas? Converging evidence from anatomy, have also identified topographically orga-
imaging, and neuropsychology suggests that nized fibers in the inferior longitudinal fasci-
the amygdala might be a crucial source for culus, which directly connect the amygdala
affective biases on perception and attention, with early visual areas and might contain
in keeping with its key role in orchestrating such back-projections (Gschwind, Pourtois,
emotional responses (see also Chapters 1 Schwartz, Van De Ville, & Vuilleumier,
and 5). 2012).
First, the strength of emotional modula- More direct evidence for a causal role of
tion of cortical sensory areas is often corre- amygdala feedback has been obtained by
lated with the magnitude of amygdala acti- lesion studies. In particular, fMRI experi-
vation (Morris et al., 1998; Peelen et al., ments revealed that patients with medial
2007; Pessoa et al., 2002; Sabatinelli et al., temporal lobe sclerosis, which selectively
AFFECTIVE BIASES IN ATTENTION AND PERCEPTION 343
C Visual increase
A (fearful > neutral)
+ .2
L fusiform BOLD
.1
0
B Healthy controls
-.1
80 85 90 95 100
L amygdala sclerosis
.2
R fusiform BOLD
Hippocampus damage alone
.1
-.1
Amygdala + Hippocampus
damage 80 84 88 92 96
R amygdala sclerosis
Faces relevant
Faces irrelevant
Figure 14.4. Deficit in emotional attention after amygdala damage in patients with medial temporal
lobe sclerosis. (A) Visual matching task used to test for brain responses to fearful expressions when
face stimuli are task relevant (faces attended) or task irrelevant (faces ignored). Two faces and two
houses appear on each trial, orthogonally aligned to each other (right). Participants are instructed to
respond either to the horizontal or to the vertical pair, according to a preceding cue (left). (B)
Contrast between fearful vs. neutral faces regardless of task relevance. Healthy controls and patients
with damage to the hippocampus (sparing the amygdala) show intact modulation of fusiform gyrus to
fearful faces, relative to neutral faces. However, patients with both amygdala and hippocampus
damage show no emotional enhancement. (C) This loss of modulation in visual areas correlates with
the degree of sclerosis measured in the amygdala ipsilateral to the face-responsive fusiform area,
suggesting that each amygdala predominantly projects to the ipsilateral visual cortex, and causally
mediates the emotional enhancement observed in visual cortex (L: left; R: right). Adapted from
Vuilleumier, Richardson, Armony, Driver, & Dolan (2004).
destroys the amygdala and hippocampus, effects in the right fusiform, whereas left
do not show a modulation of the fusiform amygdala sclerosis predicted a loss of effects
gyrus when visualizing fearful faces, rela- in the left fusiform. These findings imply
tive to neutral, unlike the increase typically that each amygdala predominantly projects
observed in healthy controls (Vuilleumier, to the ipsilateral hemisphere, in keeping
Richardson, Armony, Driver, & Dolan, with anatomical tracing data from monkeys
2004; Figure 14.4). A normal modulation, (Freese & Amaral, 2006). A similar loss of
however, was observed in patients with emotional enhancement of visual areas in
temporal lobe sclerosis affecting only the response to faces was observed in another
hippocampus but sparing the amygdala. group of patients with right medial tempo-
Furthermore, the loss of modulation in ral lobe epilepsy, an effect that correlated
visual areas was correlated with the degree with disease duration (Benuzzi et al., 2004).
of sclerosis in ipsilateral amygdala. That is, Importantly, patients with amygdala scle-
more severe sclerosis in the right amyg- rosis still showed an enhancement of
dala predicted a greater loss of emotional fusiform activity when directing attention
344 JUDITH DOMÍNGUEZ-BORRÀS & PATRIK VUILLEUMIER
(T1; Piech et al., 2011) and targets (T2; Bach, tions and combining their influence to mod-
Talmi, Hurlemann, Patin, & Dolan, 2011), or ulate perceptual pathways. Thus, just like
during visual search tasks (Piech et al., exogenous and object-based attention are
2010; Tsuchiya, Moradi, Felsen, Yamazaki, known to interact with endogenous mecha-
& Adolphs, 2009). Enhanced detection in nisms (Driver, 2001), emotion-based biasing
these conditions could still operate based systems may operate in parallel, but be con-
on distinctive features that have been over- strained (Pessoa et al., 2002) or facilitated
learned and might be processed in other (Phelps et al., 2006) by other concomitant
brain regions, including the ACC and OFC, attentional influences.
because activity in these regions also cor- Evidence for independent and additive
relates with attention capture by emotion effects of emotion and endogenous atten-
(De Martino et al., 2009; Lucas & Vuilleu- tion comes from various studies where both
mier, 2008; Schwabe et al., 2011). Thus, the factors were manipulated or measured sep-
neural sources of emotional biases in per- arately. For instance, in an fMRI experi-
ception and attention might not rely exclu- ment described earlier, pictures of faces and
sively on the amygdala, but might also impli- houses were presented at different locations,
cate alternative structures when the former while participants had to focus on either the
is dysfunctional. Likewise, it has been pro- faces or the houses (see an example in Figure
posed that the pulvinar nucleus of the thala- 14.4a). In addition, faces could be fearful or
mus might contribute, in concert or in par- neutral. Selectively attending to faces rather
allel with the amygdala, to integrate sensory than houses enhanced the fusiform cortex,
information coded in different cortical areas but fearful faces produced an additional
and amplify sensory processing in order to increase in the same area whether faces
guide attention to stimuli of potential rele- were attended or ignored. This pattern was
vance (Pessoa & Adolphs, 2010). This would replicated in several studies using the same
be in agreement with an important role paradigm on different participants (Bent-
of this subcortical structure in other atten- ley, Vuilleumier, Thiel, Driver, & Dolan,
tional processes and frequent coactivation 2003; Vuilleumier et al., 2001, 2004), which
of pulvinar and amygdala (Tamietto & de suggests that emotional signals can still act
Gelder, 2010; Vuilleumier, Armony, Driver, when attention is directed to other stimuli
& Dolan, 2003). A better understanding of (i.e., houses). In parallel, the amygdala was
the role of these different circuits for pro- also activated by fearful faces, both when
cessing stimulus saliency and of their speci- attended and when ignored, consistent with
ficity for emotional information relative its presumed role in modulating fusiform
to other characteristics (such as novelty, activity. In another study using steady-state
deviance, and ambiguity) is clearly needed. EEG (Keil et al., 2005), emotional or neutral
scenes were flashed to the right or left visual
field, while attention was directed to one or
Distinct Sources for Emotional and the other side in different trials. Again, visual
Attentional Biases responses in contralateral visual areas were
enhanced both for attended pictures and
A key issue in the framework described for emotional pictures irrespective of atten-
here is that perception can be modulated by tion. By contrast, in a different paradigm
multiple sources simultaneously, including with emotional faces presented centrally and
not only endogenous, exogenous, or object- peripheral bar targets presented in the upper
based attention but also emotional feedback peripheral field (Pessoa et al., 2002), fusiform
signals from the amygdala, together with responses were totally abolished (despite
other emotion-processing regions (Pourtois the presence of central faces), and no emo-
et al., 2012). As mentioned earlier, processing tional modulation was found when partici-
biases due to different sources may operate pants attended to the bar. These results sug-
relatively independently and additively to gest that emotion processing and subsequent
each other, interacting under certain condi- feedback cannot arise when task demands
346 JUDITH DOMÍNGUEZ-BORRÀS & PATRIK VUILLEUMIER
are harder and lead to a total suppression faces (Vuilleumier et al., 2002) and in the
of cortical processing for unattended stim- parahippocampal cortex for negative scenes
uli (see Chapter 15). However, the blocked (Grabowska et al., 2011), relative to neu-
design of the attentional conditions and the tral stimuli. These effects arose even for
absence of fusiform activity in the fMRI data stimuli presented in the left visual field,
could have prevented a reliable measure of despite losses in top-down attention due to
emotion effects for the unattended stimuli. parietal damage, and may explain the rel-
Similar dissociations between attention ative improvement of detection for emo-
and emotion effects are observed in behav- tional stimuli in neglected space. Further-
ioral studies. As noted earlier, facilitated more, in a single case study of a parietal
detection of emotional targets in visual patient with left neglect (Domı́nguez-Borràs
search is accompanied by a general slowing et al., 2012), a brief session of aversive con-
when increasing the number of distractors ditioning through frequent pairing of red
(Gerritsen et al., 2008). Likewise, patients shapes with a loud unpleasant noise led to
with left spatial neglect or extinction after a a selective reduction in left visual misses
right parietal lesion often show better detec- for shapes with red but not other col-
tion of emotional than neutral stimuli in ors, which was accompanied by a marked
contralesional space, although they fail to increase in visual cortex responses to these
voluntarily orient attention to this contrale- stimuli, as well as significant amygdala acti-
sional side and miss more emotional stim- vation. These findings converge with results
uli on the contralesional than the ipsile- in healthy subjects showing that affective
sional side (see Domı́nguez-Borràs et al., conditioning of simple visual stimuli can
2012). Again, this pattern indicates that emo- boost their representation in early visual
tional biases can still operate and boost cortex and improve detection (Stolarova
attention (or partly counteract attentional et al., 2006). More generally, these find-
deficits) in these patients, despite damage to ings are consistent with distinct sources for
frontoparietal networks controlling endoge- emotional and attentional effects on visual
nous and/or exogenous spatial attention. perception.
Interestingly, in two studies investigating Finally, one study systematically manipu-
left neglect for faces (Lucas & Vuilleumier, lated three types of attentional biases within
2008) and voices (Grandjean, Sander, Lucas, a single task using the dot-probe paradigm
Scherer, & Vuilleumier, 2008), a paramet- (Brosch et al., 2011) and showed addi-
ric anatomical lesion analysis demonstrated tive effects of bottom-up (exogenous), top-
that the relative advantage for emotional down (endogenous), and emotional cues on
stimuli was stronger in patients with larger spatial orienting. Targets (small rectangles)
damage in parietal areas, whereas emotional were presented on either side of the screen,
effects were reduced in patients with dam- while attention could be manipulated in
age extending to the orbitofrontal cortex advance by an instruction arrow (engag-
and subcortical basal ganglia circuits. These ing endogenous attention), by a bright flash
findings add further support to the idea (exogenous), or by fearful faces. Each of
of independent neural substrates mediating these cues could be valid or invalid, orthog-
emotional and spatial biases in perception, onally to each other. Behavioral results
and they also accord with the possibility that demonstrated that the facilitation of target
the orbitofrontal cortex might provide an detection latencies added up in a linear fash-
important interface between emotion pro- ion with the combined validity of differ-
cessing and attention control (see the earlier ent cues (such that a valid fearful face or
discussion). exogenous flash preceding a target speeded
Moreover, fMRI studies in patients with up reaction times even when attention was
left neglect have shown preserved emo- endogenously oriented to the other side,
tional effects in visual cortex, with increased but less than when all cues directed atten-
responses in the fusiform cortex for fearful tion to the correct side). A subsequent EEG
AFFECTIVE BIASES IN ATTENTION AND PERCEPTION 347
experiment used a similar approach to con- However, amygdala responses and its pro-
trast the two involuntary effects triggered jections to sensory areas can be regulated by
by exogenous and emotional cues (Brosch et signals from distinct brain areas, such as the
al., 2011) and revealed that these two factors prefrontal cortex (see Figure 14.5a), produc-
operated during two distinct time windows: ing different biasing effects according to the
ERPs time-locked to the exogenous cue context (Vuilleumier, 2008). Thus, amyg-
showed a specific enhancement of the N2pc dala reactivity might be modulated by task-
component, consistent with a rapid shift in related demands, including voluntary atten-
attention to the cued side, whereas the emo- tion or cognitive load (see Chapter 15), as
tional cue enhanced the P1 time-locked to well as by current emotional states and per-
the target, consistent with enhanced visual sonality dispositions (see Chapter 24). Thus,
perception. Thus, ERPs clearly differenti- acute stress can not only increase amygdala
ated between processes mediating atten- responses and counteract the effect of task
tional biases induced by emotional meaning load but also diminish threat selectivity (van
or by exogenous physical properties of the Marle, Hermans, Qin, & Fernandez, 2009).
stimuli. The neural pathways allowing preat-
Nevertheless, despite this evidence for tentive stimulus processing still remain
independent sources of emotional biases, unknown. In the visual modality, one hy-
some data also point to interactive effects pothesis suggests that rudimentary visual
in some conditions. For instance, both signals might be extracted through subcorti-
fMRI (Pessoa et al., 2002; Vuilleumier cal visual (perhaps magnocellular) pathways
et al., 2001) and psychophysiological findings including the superior colliculus, and pul-
(Phelps et al., 2006) suggest that emotion vinar nucleus of the thalamus, then pro-
cues might potentiate activity in early visual jecting directly to the amygdala (Tami-
cortex (e.g., V1) for attended but not for etto & de Gelder, 2010). This subcortical
unattended stimuli. It remains to be seen route (see Figure 14.5a) was postulated based
whether this may reflect a crucial role of on residual emotional processing in patients
V1 in integrating different top-down signals with blindsight after destruction of the occip-
to build a visual saliency map (Li, 2002), ital cortex (Tamietto & de Gelder, 2010) and
which serves, in turn, to guide attention or to on preserved fear conditioning in animals
promote conscious vision. By contrast, emo- without the sensory cortex (Romanski &
tion effects on other regions (such as the LeDoux, 1992). Direct connections between
FFA) seem more often additive to attention visual pulvinar and amygdala remain ques-
and sometimes unrelated to conscious per- tionable in humans (Pessoa & Adolphs,
ception of the stimuli (Vuilleumier et al., 2010), but exist in lower primates (Day-
2001, 2002). Brown, Wei, Chomsung, Petry, & Bickford,
2010). Other subcortical visual pathways
from the brainstem to the amygdala were
Preattentive and Unconscious also reported in rodents (although unknown
Processing of Emotional Cues in humans; Usunoff, Itzev, Rolfs, Schmitt, &
Wree, 2006). Alternatively, blindsight may
Another important aspect of emotional depend on direct inputs from the lateral
biases in perception is that they should geniculate nucleus to extrastriate temporal
take place even in parallel or prior to full cortex (Schmid et al., 2010) or from the pul-
attention to the relevant stimuli. Although vinar to fusiform in humans (Clarke, Riahi-
there is considerable controversy around this Arya, Tardif, Eskenasy, & Probst, 1999), pro-
question (see Chapter 15; Pourtois et al., viding another subcortical route bypassing
2012), the amygdala may activate to stim- early occipital cortex and potentially pro-
uli without explicit attention or awareness jecting to the amygdala (and other brain
in many (though not all) situations (Tami- areas) prior to awareness. However, uncon-
etto & de Gelder, 2010; Whalen et al., 2004). scious or preattentive emotional processing
348 JUDITH DOMÍNGUEZ-BORRÀS & PATRIK VUILLEUMIER
presented at attended or unattended loca- cortical areas through the activation of mod-
tions (see the task in Figure 14.4a), but a later ulatory neurotransmitters in the cholin-
and prolonged response after 600 ms that is ergic and adrenergic systems. Cholinergic
enhanced by overt attention to faces (Figure nuclei in the basal forebrain receive dense
14.3b). Scalp recordings using the same task inputs from the amygdala and modulate
also found an early emotional effect around vigilance through projections to frontal,
100 ms over frontal sites, but at top-down parietal, and sensory areas (Holland &
attentional effects starting around 200 ms Gallagher, 2004). Relative to placebo, the
(Holmes, Vuilleumier, & Eimer, 2003). pro-cholinergic drug physostigmine does not
Therefore, it is plausible that both the modify the emotional enhancement of the
amygdala and other limbic regions involved fusiform cortex to fearful faces (Bentley
in affective appraisal, such as the OFC, et al., 2003). However, when fearful faces are
might activate at early latencies through an unattended, physostigmine leads to greater
initial volley of feedforward inputs (Vuilleu- responses in the lateral OFC and ACC,
mier, 2005), before or in parallel with the but reduced responses in the intraparietal
recruitment of exogenous or endogenous sulcus. These findings suggest that acetyl-
attentional systems. A rapid emotional cat- choline is not responsible for enhanced acti-
egorization in the amygdala might oper- vation of sensory areas, but might promote
ate on coarse perceptual features associated “distraction” by unattended emotional infor-
with emotional meaning, such as low spa- mation through frontoparietal interactions
tial frequencies conveyed by visual mag- (Bentley et al., 2003).
nocellular channels (e.g., Alorda, Serrano- The central amygdala also has strong
Pedraza, Campos-Bueno, Sierra-Vázquez, & outputs to the sympathetic pathways and
Montoya, 2007; Pourtois, Dan, et al., 2005; locus coeruleus in the brainstem. The locus
Vuilleumier et al., 2003; but see Morawetz, coeruleus sends noradrenaline inputs to
Baudewig, Treue, & Dechent, 2011), or other widespread regions throughout the brain,
simple shape attributes such as sharp con- which regulate arousal and autonomic func-
tours (Bar & Neta, 2007). This idea would tions. These projections operate in a pha-
accord with the counterintuitive finding that sic or tonic manner promoting endoge-
facial expressions of fear and disgust in nous attention or flexibility, respectively,
faces are better detected when presented at and modulate the receptive field of sensory
higher eccentricities in the visual field, even neurons (e.g., Aston-Jones, Rajkowski, &
though visual resolution decreases (Bayle Cohen, 1999). Moreover, the locus coeruleus
et al., 2011). This early activation of the responds to fearful faces despite a lack of
amygdala (or other area such as the OFC conscious awareness (Liddell et al., 2005).
or ACC) would then modulate sensory cor- These effects are likely to contribute to emo-
tices via direct feedback (Amaral et al., 2003; tional attention, but have not been system-
Vuilleumier, 2005) or indirect projections to atically investigated in humans in relation to
the dorsal attention system in prefrontal and this specific function.
parietal areas (Vuilleumier, 2005; see Figure
14.5a). Clearly, more work is needed to clar-
ify the exact timing of emotion and attention Reward and Positive Emotions
effects in different brain regions and to test
the causality of interactions between them. Most results and models concerning the neu-
ral mechanisms of emotional enhancement
in perception and attention refer to stimuli
Neuromodulation by Cholinergic and with negative affective load. However, sim-
Adrenergic Pathways ilar patterns of brain activation have been
observed for positive stimuli, particularly
Another pathway for emotional attention when they are highly arousing. Thus, appet-
involves indirect influences of amygdala on itive visual scenes, erotica, smiling faces,
350 JUDITH DOMÍNGUEZ-BORRÀS & PATRIK VUILLEUMIER
joyful voices, and even humor may activate down attention. Reward associations may
the amygdala (Sabatinelli et al., 2005; see also enhance cortical representation of val-
Chapter 11) and sometimes produce atten- ued stimuli in sensory areas, including the
tional biases similar to threat (see the earlier primary visual cortex, at notably early laten-
discussion). In contrast, positive stimuli or cies (Shuler & Bear, 2006). As observed
positive mood has also been reported to pro- for threat-related effects with other stim-
duce distinct effects, opposite to negative uli, such effects seem to arise without
emotion. For instance, the size of the atten- attention and without voluntary control.
tional span in spatial orienting or flanker fil- Neuropsychological findings show that they
tering tasks is larger when associated with still arise in patients with parietal lesions
positive events, but smaller with negative and spatial neglect, improving attention
events (Fenske & Eastwood, 2003). Posi- to the neglected side, suggesting mecha-
tive emotions may also enlarge the breadth nisms partly independent from frontopari-
of attention, increasing, for instance, corti- etal attentional systems (see Domı́nguez-
cal visual response to peripheral distracters Borràs et al., 2012). However, the exact
(Schmitz, De Rosa, & Anderson, 2009). mechanisms subserving these reward biases
More research is needed to better disentan- and their relation to the dopaminergic sig-
gle these different influences of valence on naling are not clear. The possible resem-
attention – in particular to clarify the crit- blance or difference in neural circuits with
ical affective dimensions that are responsi- respect to those associated with amygdala
ble for such effects and to identify which function remains to be fully explored.
mechanisms are common or distinct across
different emotion categories or tasks.
A special case of positive or appeti- Conclusions
tive cues might concern reward. In addi-
tion to the amygdala, reward signals activate In accord with the view that emotions
dopaminergic projections from the ventral involve adaptive responses of the organ-
tegmental area to the striatum and pre- ism to potential challenges in the environ-
frontal cortex. Dopamine might also play a ment, both behavioral and neuroimaging
role in the enhanced processing and detec- data demonstrate tight links between emo-
tion of emotionally significant events. For tional and attentional processes that jointly
example, mesolimbic and striatal dopamin- act to bias perception and awareness. Several
ergic neurons seem to respond to salient aspects of emotion appear to be shared with
changes in environmental conditions, both functions classically associated with atten-
appetitive and aversive (see Chapter 19). tional mechanisms, as both may influence
Furthermore, several studies found that the selection of sensory inputs (and per-
visual search is facilitated for targets or haps other representations, such as actions
features previously paired with rewards and memories) through feedback modula-
(Hickey, Chelazzi, & Theeuwes, 2010), sug- tory signals (Figure 14.5b). There is consid-
gesting an involuntary capture of attention erable evidence from numerous tasks, rang-
by stimuli of high value. These effects are ing from simple detection and orienting to
stronger in participants with reward-seeking search and binocular rivalry, that attention
personalities (Hickey et al., 2010). is preferentially guided to targets or cap-
In keeping with these behavioral effects, tured by distracters with emotional signif-
neurophysiology data in animals show that icance. Although threat cues have often
neuronal activity is enhanced by reward in been investigated and found to produce the
numerous brain regions controlling atten- strongest effects, other emotions, including
tion and/or eye movements, such as the pleasant cues or reward, might also trigger
intraparietal cortex, frontal eye field, or similar advantages in attention performance.
superior colliculus (see Vuilleumier, 2005), Nonetheless, some differences between neg-
overlapping with networks involved in top- ative and positive affect are likely to exist.
AFFECTIVE BIASES IN ATTENTION AND PERCEPTION 351
From a phenomenological point of view, might act together to guide the selection
fear might be more specifically linked to of relevant information among competing
changes in attention focus, compared to stimuli, yet through partly distinct pathways
other emotions. It is also possible that dis- and partly distinct time courses. Hence,
tinct types pf aversive emotional stimuli attention selection works not only by mod-
produce different effects, such as when ulating the gain of neuronal responses based
threat or disgust cues attract or divert atten- on spatial location and feature cues but also
tion, respectively. However, more research based on the potential affective value of sen-
is needed to support this idea. sory events.
At the neural level, there is also consid- In sum, just as attention can be influenced
erable evidence that emotional influences by feature- or object-based effects, reflect-
on perception and attention are mediated ing the readiness of our perceptual systems
both by direct enhancement of stimulus rep- to preferentially encode certain aspects of
resentations in sensory cortices and, more sensory, it is also influenced by emotion-
indirectly, by modulations of frontoparietal based or value-based representations. These
attentional systems. The amygdala seems to effects may reflect the existence of spe-
play a key role in triggering these modu- cific brain circuits for emotional attention or
lations through direct feedback projections motivated attention, which complement the
to sensory areas and indirect projections to dynamic interplay between bottom-up and
frontoparietal areas (possibly via the OFC top-down processes underlying conscious
and ACC), as well as through cholinergic perception.
and noradrenergic subcortical systems. It is
likely that other systems than the amyg- Outstanding Questions and Future
dala also contribute to bias sensory process- Directions
ing based on affective meaning, notably in
relation to reward values where mesolim- r Are mechanisms of emotion-based atten-
bic dopaminergic pathways might be impli- tion partly dependent on other neural cir-
cated, or in patients with amygdala damage cuits than the amygdala, and if so which
who still show residual affective biases in ones? What is the role of these different
some attention tasks. We still need to better systems? Are they differently recruited in
understand the neural substrates recruited different conditions?
by different emotions, their role in different r What are the common and specific effects
tasks, and their reactivity to different cate- of emotional attention observed for dif-
gories of stimuli. ferent emotional cues, particularly when
Finally, there is evidence that modulatory related to negative (e.g., threat) or pos-
effects exerted by affective systems might itive (e.g., reward) information? Which
be partly independent and complementary are the critical emotional dimensions to
(additive or competitive) to those controlled produce these effects?
by voluntary attentional mechanisms. How- r Are emotion-based attention effects sim-
ever, although these effects may be reflex- ilar for different stimuli categories (faces,
ive or even unconscious in many situations, scenes, sounds, voices) and for different
their efficacy might be amplified or mini- tasks? What is the exact role of specific
mized by various factors, just as other reflex physical features, as opposed to affective
loops in the nervous system. Such factors meaning of the stimuli, in triggering these
might include task demands, but also expec- effects?
tations, situational context, past experience, r What are the conditions (or stimuli)
or internal states such as anxiety or stress. allowing emotional influences on percep-
Overall, this neural architecture suggests a tion to be induced without overt atten-
“multiple attention gain control” model of per- tion or without conscious awareness?
ceptual processing (Pourtois et al., 2012), r What is the role of contextual or individ-
in which different sources of neural biases ual factors in modulating emotion-based
352 JUDITH DOMÍNGUEZ-BORRÀS & PATRIK VUILLEUMIER
attention effects; for instance, in relation and behavioral flexibility. Biological Psychia-
to current affect states or moods, expec- try, 46(9), 1309–20.
tations, prior experience, habituation, or Bach, D. R., Talmi, D., Hurlemann, R., Patin,
task goals? A., & Dolan, R. J. (2011). Automatic relevance
r Can direct neuronal recordings in humans detection in the absence of a functional amyg-
dala. Neuropsychologia, 49(5), 1302–5.
(e.g., intracranial EEG) or animal models
Bar, M., & Neta, M. (2007). Visual elements
with causal interventions (e.g., micros-
of subjective preference modulate amyg-
timulation, optogenetics) help deter- dala activation. Neuropsychologia, 45(10), 2191–
mine the functional interactions and 2200.
time course of activity in different brain Bayle, D. J., Schoendorff, B., Henaff, M. A.,
regions during emotional and attentional & Krolak-Salmon, P. (2011). Emotional facial
processing? expression detection in the peripheral visual
field. PLoS ONE, 6(6), e21584.
Bentley, P., Vuilleumier, P., Thiel, C. M., Driver,
J., & Dolan, R. J. (2003). Cholinergic enhance-
ment modulates neural correlates of selective
References attention and emotional processing. Neuroim-
age, 20(1), 58–70.
Alorda, C., Serrano-Pedraza, I., Campos-Bueno, Benuzzi, F., Meletti, S., Zamboni, G., Calandra-
J. J., Sierra-Vazquez, V., & Montoya, P. Buonaura, G., Serafini, M., Lui, F., Nichelli,
(2007). Low spatial frequency filtering mod- P. (2004). Impaired fear processing in right
ulates early brain processing of affective com- mesial temporal sclerosis: A fMRI study. Brain
plex pictures. Neuropsychologia, 45(14), 3223– Research Bulletin, 63(4), 269–81.
33. Blue, V. L. (2010). And along came a spider: An
Alpers, G. W., & Gerdes, A. B. (2007). Here attentional bias for the detection of spiders in
is looking at you: Emotional faces predomi- young children and adults. Journal of Experi-
nate in binocular rivalry. Emotion, 7(3), 495– mental Child Psychology, 107, 8.
506. Brosch, T., Pourtois, G., Sander, D., & Vuilleu-
Amaral, D. G., Bauman, M. D., Capitanio, J. mier, P. (2011). Additive effects of emotional,
P., Lavenex, P., Mason, W. A., Mauldin- endogenous, and exogenous attention: Behav-
Jourdain, M. L., & Mendoza, S. P. (2003). The ioral and electrophysiological evidence. Neu-
amygdala: is it an essential component of the ropsychologia, 49(7), 1779–87.
neural network for social cognition? Neuropsy- Calvo, M. G., & Nummenmaa, L. (2008). Detec-
chologia, 41(4), 517–22. tion of emotional faces: Salient physical fea-
Amting, J. M., Greening, S. G., & Mitchell, D. G. tures guide effective visual search. Journal of
(2010). Multiple mechanisms of consciousness: Experimental Psychology: General, 137(3), 471–
The neural correlates of emotional awareness. 94.
Journal of Neuroscience, 30(30), 10039–47. Clarke, S., Riahi-Arya, S., Tardif, E., Eskenasy, A.
Anderson, A. K., & Phelps, E. A. (2001). Lesions C., & Probst, A. (1999). Thalamic projections
of the human amygdala impair enhanced per- of the fusiform gyrus in man. European Journal
ception of emotionally salient events. Nature, of Neuroscience, 11(5), 1835–38.
411(6835), 305–9 Coelho, C. M., Cloete, S., & Wallis, G. (2010).
Armony, J. L., & Dolan, R. J. (2002). Modulation The face-in-the-crowd effect: When angry
of spatial attention by fear-conditioned stim- faces are just cross(es). Journal of Vision, 10(1),
uli: an event-related fMRI study. Neuropsy- 7, 1–14.
chologia, 40(7), 817–26. Cunningham, W. A., Brosch, T. (2012). Motiva-
Armony, J. L., Quirk, G. J., & LeDoux, J. E. tional salience: Amygdala tuning from traits,
(1998). Differential effects of amygdala lesions needs, values, and goals. Current Directions in
on early and late plastic components of audi- Psychological Science, 21(1), 54–59.
tory cortex spike trains during fear condi- Day-Brown, J. D., Wei, H., Chomsung, R. D.,
tioning. Journal of Neuroscience, 18(7), 2592– Petry, H. M., & Bickford, M. E. (2010). Pulv-
2601. inar projections to the striatum and amygdala
Aston-Jones, G., Rajkowski, J., & Cohen, J. in the tree shrew. Frontiers in Neuroanatomy,
(1999). Role of locus coeruleus in attention 4, 143.
AFFECTIVE BIASES IN ATTENTION AND PERCEPTION 353
De Martino, B., Kalisch, R., Rees, G., & Dolan, R. ally negative stimuli can overcome attentional
J. (2009). Enhanced processing of threat stim- deficits in patients with visuo-spatial hemine-
uli under limited attentional resources. Cere- glect. Neuropsychologia, 49(12), 3327–37.
bral Cortex, 19(1), 127–33. Grandjean, D., Sander, D., Lucas, N., Scherer,
Dehaene, S., Changeux, J. P., Naccache, L., K. R., & Vuilleumier, P. (2008). Effects of
Sackur, J., & Sergent, C. (2006). Conscious, emotional prosody on auditory extinction for
preconscious, and subliminal processing: A voices in patients with spatial neglect. Neu-
testable taxonomy. Trends in Cognitive Science, ropsychologia, 46(2), 487–96.
10(5), 204–11. Grandjean, D., Sander, D., Pourtois, G.,
Dolan, R. J., Heinze, H. J., Hurlemann, R., & Schwartz, S., Seghier, M. L., Scherer, K. R.,
Hinrichs, H. (2006). Magnetoencephalogra- & Vuilleumier, P. (2005). The voices of wrath:
phy (MEG) determined temporal modulation Brain responses to angry prosody in mean-
of visual and auditory sensory processing in ingless speech. Nature Neuroscience, 8(2), 145–
the context of classical conditioning to faces. 46.
Neuroimage, 32(2), 778–89. Gschwind, M., Pourtois, G., Schwartz, S., Van
Domı́nguez-Borràs, J., Saj, A., Armony, J. L., De Ville, D., & Vuilleumier, P. (2012). White-
& Vuilleumier, P. (2012). Emotional process- matter connectivity between face-responsive
ing and its impact on unilateral neglect and regions in the human brain. Cerebral Cortex,
extinction. Neuropsychologia, 50(6), 1054–71. 22(7), 1564–76.
Driver, J. (2001). A selective review of selec- Hahn, S., & Gronlund, S. D. (2007). Top-down
tive attention research from the past century. guidance in visual search for facial expressions.
British Journal of Psychology, 92, 53–78. Psychonomic Bulletin and Review, 14(1), 159–65.
Ethofer, T., Bretscher, J., Gschwind, M., Hickey, C., Chelazzi, L., & Theeuwes, J. (2010).
Kreifelts, B., Wildgruber, D., & Vuilleumier, Reward guides vision when it’s your thing:
P. (2012). Emotional voice areas: Anatomic Trait reward-seeking in reward-mediated
location, functional properties, and structural visual priming. PLoS ONE, 5(11), e14087.
connections revealed by combined fMRI/DTI. Holland, P. C., & Gallagher, M. (2004).
Cerebral Cortex, 22(1), 191–200 Amygdala-frontal interactions and reward
Fenske, M. J., & Eastwood, J. D. (2003). Modu- expectancy Current Opinion in Neurobiology,
lation of focused attention by faces expressing 14(2), 148–55.
emotion: Evidence from flanker tasks. Emo- Holmes, A., Vuilleumier, P., & Eimer, M. (2003).
tion, 3(4), 327–43. The processing of emotional facial expression
Flykt, A. (2005). Visual search with biological is gated by spatial attention: Evidence from
threat stimuli: Accuracy, reaction times, and event-related brain potentials. Brain Research:
heart rate changes. Emotion, 5(3), 349–53. Cognitive Brain Research, 16(2), 174–84.
Flykt, A., & Caldara, R. (2006). Tracking fear in Huang, S. L., Chang, Y. C., & Chen, Y. J. (2011).
snake and spider fearful participants during Task-irrelevant angry faces capture attention
visual search: A multi-response domain study. in visual search while modulated by resources.
Cognition and Emotion, 20(8), 16. Emotion, 11(3), 544–52.
Forbes, S. J., Purkis, H. M., & Lipp, O. V. (2011). Huang, Y. M., Baddeley, A., & Young, A.
Better safe than sorry: Simplistic fear-relevant W. (2008). Attentional capture by emotional
stimuli capture attention. Cognition and Emo- stimuli is modulated by semantic processing.
tion, 25(5), 794–804. Journal of Experimental Psychology: Human
Freese, J. L., & Amaral, D. G. (2006). Synaptic Perception Performance, 4(2), 328–39.
organization of projections from the amyg- Kawasaki, H., Kaufman, O., Damasio, H.,
dala to visual cortical areas TE and V1 in Damasio, A. R., Granner, M., Bakken,
the macaque monkey. Journal of Comparative H., . . . Adolphs, R. (2001). Single-neuron
Neurology, 496(5), 655–67. responses to emotional visual stimuli recorded
Gerritsen, C., Frischen, A., Blake, A., Smilek, in human ventral prefrontal cortex. Nature
D., & Eastwood, J. D. (2008). Visual search Neuroscience, 4(1), 15–16.
is not blind to emotion. Perception and Psy- Keil, A., Costa, V., Smith, J. C., Sabatinelli, D.,
chophysics, 70(6), 1047–59 McGinnis, E. M., Bradley, M. M., & Lang,
Grabowska, A., Marchewka, A., Seniow, J., P. J. (2011). Tagging cortical networks in emo-
Polanowska, K., Jednorog, K., Krolicki, L., tion: A topographical analysis. Human Brain
Kossut, M., Czlonkowska, A. (2011). Emotion- Mapping. doi: 10.1002/hbm.21413
354 JUDITH DOMÍNGUEZ-BORRÀS & PATRIK VUILLEUMIER
Kiel, A., & Ihssen, N. (2004). Identification facil- versus implicit processing of emotion. Human
itation for emotionally arousing verbs during Brain Mapping. doi: 10.1002/hbm.21406
the attentional blink. Emotion, 4(1), 23–35. Morawetz, C., Baudewig, J., Treue, S., &
Keil, A., Moratti, S., Sabatinelli, D., Bradley, M. Dechent, P. (2011). Effects of spatial frequency
M., & Lang, P. J. (2005). Additive effects of and location of fearful faces on human amyg-
emotional content and spatial selective atten- dala activity. Brain Research, 1371, 87–99.
tion on electrocortical facilitation. Cerebral Morris, J. S., Friston, K. J., Buchel, C., Frith, C.
Cortex, 15(8), 1187–97. D., Young, A. W., Calder, A. J., & Dolan,
Kennedy, D. P., & Adolphs, R. (2010). Impaired R. J. (1998). A neuromodulatory role for
fixation to eyes following amygdala damage the human amygdala in processing emotional
arises from abnormal bottom-up attention. facial expressions. Brain, 121(Pt. 1), 47–57.
Neuropsychologia, 48(12), 3392–98. Naccache, L., Gaillard, R., Adam, C., Hasboun,
Krolak-Salmon, P., Henaff, M. A., Vighetto, A., D., Clemenceau, S., Baulac, M., Cohen, L.
Bertrand, O., & Mauguiere, F. (2004). Early (2005). A direct intracranial record of emo-
amygdala reaction to fear spreading in occip- tions evoked by subliminal words. Proceedings
ital, temporal, and frontal cortex: A depth of the National Academy of Sciences, 102(21),
electrode ERP study in human. Neuron, 42(4), 7713–17.
665–76. Nasrallah, M., Carmel, D., & Lavie, N. (2009).
Krusemark, E. A., & Li, W. (2011). Do all Murder, she wrote: Enhanced sensitivity
threats work the same way? Divergent effects to negative word valence. Emotion, 9(5),
of fear and disgust on sensory perception 609–18.
and attention. Journal of Neuroscience, 31(9), Notebaert, L., Crombez, G., Van Damme,
3429–34. S., De Houwer, J., & Theeuwes, J. (2010).
Lang, P. J., Bradley, M. M., Fitzsimmons, J. R., Looking out for danger: An attentional
Cuthbert, B. N., Scott, J. D., Moulder, B., & bias towards spatially predictable threatening
Nangia, V. (1998). Emotional arousal and acti- stimuli. Behavior Research and Therapy, 48(11),
vation of the visual cortex: An fMRI analysis. 1150–54.
Psychophysiology, 35(2), 199–210. Notebaert, L., Crombez, G., Van Damme, S.,
Leppanen, J. M. (2006). Emotional information De Houwer, J., & Theeuwes, J. (2011). Signals
processing in mood disorders: A review of of threat do not capture, but prioritize, atten-
behavioral and neuroimaging findings. Current tion: A conditioning approach. Emotion, 11(1),
Opinion in Psychiatry, 19(1), 34–39. 81–89.
Li, Z. (2002). A saliency map in primary visual Nummenmaa, L., Hyona, J., & Calvo, M. G.
cortex. Trends in Cognitive Science, 6(1), 9–16. (2009). Emotional scene content drives the
Liddell, B. J., Brown, K. J., Kemp, A. H., Barton, saccade generation system reflexively. Journal
M. J., Das, P., Peduto, A., Williams, L. M. of Experimental Psychology: Human Perception
(2005). A direct brainstem-amygdala-cortical and Performance, 35(2), 305–23.
“alarm” system for subliminal signals of fear. Peelen, M. V., Atkinson, A. P., Andersson, F., &
Neuroimage, 24(1), 235–43. Vuilleumier, P. (2007). Emotional modulation
Lucas, N., & Vuilleumier, P. (2008). Effects of of body-selective visual areas. SCAN – Social
emotional and non-emotional cues on visual Cognitive and Affective Neuroscience, 2, 274–83.
search in neglect patients: Evidence for dis- Peelen, M. V., Lucas, N., Mayer, E., & Vuilleu-
tinct sources of attentional guidance. Neu- mier, P. (2009). Emotional attention in
ropsychologia, 46(5), 1401–14. acquired prosopagnosia. Social Cognitive and
Luo, Q., Holroyd, T., Majestic, C., Cheng, X., Affective Neuroscience, 4(3), 268–77.
Schechter, J., & Blair, R. J. (2010). Emotional Pessoa, L., & Adolphs, R. (2010). Emotion pro-
automaticity is a matter of timing. Journal of cessing and the amygdala: From a “low road”
Neuroscience, 30(17), 5825–29. to “many roads” of evaluating biological sig-
Mogg, K., & Bradley, B. P. (2002). Selective ori- nificance. Nature Reviews. Neuroscience, 11(11),
enting of attention to masked threat faces in 773–83.
social anxiety. Behavior Research and Therapy, Pessoa, L., McKenna, M., Gutierrez, E., & Unger-
40(12), 1403–14. leider, L. G. (2002). Neural processing of
Monroe, J. F., Griffin, M., Pinkham, A., Loug- emotional faces requires attention. Proceedings
head, J., Gur, R. C., Roberts, T. P., & Edgar, J. of the National Academy of Sciences, 99(17),
(2011). The fusiform response to faces: Explicit 11458–63.
AFFECTIVE BIASES IN ATTENTION AND PERCEPTION 355
Phelps, E. A., Ling, S., & Carrasco, M. (2006). electrophysiological stages of spatial orienting
Emotion facilitates perception and potentiates towards fearful faces: Early temporo-parietal
the perceptual benefits of attention. Psycholog- activation preceding gain control in extras-
ical Science, 17(4), 292–99. triate visual cortex. Neuroimage, 26(1), 149–
Piech, R. M., McHugo, M., Smith, S. D., Dukic, 63.
M. S., Van Der Meer, J., Abou-Khalil, B., Purkis, H. M., & Lipp, O. V. (2007). Automatic
Zald, D. H. (2011). Attentional capture by attention does not equal automatic fear: Pref-
emotional stimuli is preserved in patients with erential attention without implicit valence.
amygdala lesions. Neuropsychologia, 49(12), Emotion, 7(2), 314–23.
3314–19. Romanski, L. M., & LeDoux, J. E. (1992). Equipo-
Piech, R. M., McHugo, M., Smith, S. D., Dukic, tentiality of thalamo-amygdala and thalamo-
M. S., Van Der Meer, J., Abou-Khalil, B., cortico-amygdala circuits in auditory fear
& Zald, D. H. (2010). Fear-enhanced visual conditioning. Journal of Neuroscience, 12(11),
search persists after amygdala lesions. Neu- 4501–9.
ropsychologia, 48(12), 3430–35. Rotshtein, P., Richardson, M. P., Winston,
Pinkham, A. E., Griffin, M., Baron, R., Sasson, N. J. S., Kiebel, S. J., Vuilleumier, P., Eimer, M.,
J., & Gur, R. C. (2010). The face in the crowd Dolan, R. J. (2010). Amygdala damage affects
effect: Anger superiority when using real event-related potentials for fearful faces at
faces and multiple identities. Emotion, 10(1), specific time windows. Human Brain Map-
141–46. ping, 31(7), 1089–1105.
Pourtois, G., Dan, E. S., Grandjean, D., Sander, Sabatinelli, D., Bradley, M. M., Fitzsimmons,
D., & Vuilleumier, P. (2005). Enhanced extras- J. R., & Lang, P. J. (2005). Parallel amygdala
triate visual response to bandpass spatial fre- and inferotemporal activation reflect emo-
quency filtered fearful faces: Time course tional intensity and fear relevance. Neuroim-
and topographic evoked-potentials mapping. age, 24(4), 1265–70.
Human Brain Mapping, 26(1), 65–79. Sabatinelli, D., Fortune, E. E., Li, Q., Siddiqui,
Pourtois, G., Grandjean, D., Sander, D., & A., Krafft, C., Oliver, W. T., Jeffries, J. (2011).
Vuilleumier, P. (2004). Electrophysiological Emotional perception: Meta-analyses of face
correlates of rapid spatial orienting towards and natural scene processing. Neuroimage,
fearful faces. Cerebral Cortex, 14(6), 619–33. 54(3), 2524–33.
Pourtois, G., Schettino, & Vuilleumier, P. Sabatinelli, D., Lang, P. J., Keil, A., & Bradley, M.
(2012). Brain mechanisms for emotional influ- M. (2007). Emotional perception: Correlation
ences on perception and attention: what is of functional MRI and event-related poten-
magic and what is not. Biological Psychology. tials. Cerebral Cortex, 17(5), 1085–91
doi:10.1016/j.biopsycho.2012.02.007 Scherer, K. R., & Peper, M. (2001). Psychologi-
Pourtois, G., Schwartz, S., Seghier, M. L., cal theories of emotion and neuropsychologi-
Lazeyras, F., & Vuilleumier, P. (2006). Neural cal research. In G. Gainotti (Ed.), Handbook
systems for orienting attention to the location of neuropsychology. Vol. 5: Emotional behav-
of threat signals: An event-related fMRI study. ior and its disorders (pp. 17–48). New York:
Neuroimage, 31(2), 920–33. Elsevier.
Pourtois, G., Spinelli, L., Seeck, M., & Vuilleu- Schmid, M. C., Mrowka, S. W., Turchi,
mier, P. (2010a). Modulation of face process- J., Saunders, R. C., Wilke, M., Peters,
ing by emotional expression and gaze direc- A. J., . . . Leopold, D. A. (2010). Blindsight
tion during intracranial recordings in right depends on the lateral geniculate nucleus.
fusiform cortex. Journal of Cognitive Neuro- Nature, 466(7304), 373–77.
science, 22(9), 2086–2107. Schmitz, T. W., De Rosa, E., & Anderson, A. K.
Pourtois, G., Spinelli, L., Seeck, M., & Vuilleu- (2009). Opposing influences of affective state
mier, P. (2010b). Temporal precedence of valence on visual cortical encoding. Journal of
emotion over attention modulations in the lat- Neuroscience, 29(22), 7199–7207.
eral amygdala: Intracranial ERP evidence from Schwabe, L., Merz, C. J., Walter, B., Vaitl,
a patient with temporal lobe epilepsy. Cogni- D., Wolf, O. T., & Stark, R. (2011). Emo-
tive, Affective & Behavioral Neuroscience, 10(1), tional modulation of the attentional blink: The
83–93. neural structures involved in capturing and
Pourtois, G., Thut, G., Grave de Peralta, R., holding attention. Neuropsychologia, 49(3),
Michel, C., & Vuilleumier, P. (2005). Two 416–25.
356 JUDITH DOMÍNGUEZ-BORRÀS & PATRIK VUILLEUMIER
Serences, J. T., & Yantis, S. (2006). Selective Vuilleumier, P. (2005). How brains beware: Neu-
visual attention and perceptual coherence. ral mechanisms of emotional attention. Trends
Trends in Cognitive Science, 10(1), 38–45. in Cognitive Science, 9(12), 585–94.
Shuler, M. G., & Bear, M. F. (2006). Reward Vuilleumier, P. (2008). The role of human amyg-
timing in the primary visual cortex. Science, dala in perception and attention. In P. J.
311(5767), 1606–9. Whalen & E. A. Phelps (Eds.), The human
Smilek, D., Frischen, A., Reynolds, M. G., Ger- amygdala (pp. 220–49). New York: Guilford
ritsen, C., & Eastwood, J. D. (2007). What Press.
influences visual search efficiency? Disen- Vuilleumier, P., Armony, J. L., Clarke, K.,
tangling contributions of preattentive and Husain, M., Driver, J., & Dolan, R. J. (2002).
postattentive processes. Perception and Psy- Neural response to emotional faces with and
chophysics, 69(7), 1105–16. without awareness: Event-related fMRI in a
Stolarova, M., Keil, A., & Moratti, S. (2006). parietal patient with visual extinction and spa-
Modulation of the C1 visual event-related tial neglect. Neuropsychologia, 40(12), 2156–66.
component by conditioned stimuli: Evidence Vuilleumier, P., Armony, J. L., Driver, J., &
for sensory plasticity in early affective percep- Dolan, R. J. (2001). Effects of attention and
tion. Cerebral Cortex, 16(6), 876–87. emotion on face processing in the human
Tamietto, M., & de Gelder, B. (2010). Neural brain: An event-related fMRI study. Neuron,
bases of the non-conscious perception of emo- 30(3), 829–41.
tional signals. Nature Reviews. Neuroscience, Vuilleumier, P., Armony, J. L., Driver, J., &
11(10), 697–709. Dolan, R. J. (2003). Distinct spatial frequency
Tsuchiya, N., Moradi, F., Felsen, C., Yamazaki, sensitivities for processing faces and emotional
M., & Adolphs, R. (2009). Intact rapid expressions. Nature Neuroscience, 6(6), 624–31.
detection of fearful faces in the absence of Vuilleumier, P., Richardson, M. P., Armony,
the amygdala. Nature Neuroscience, 12(10), J. L., Driver, J., & Dolan, R. J. (2004). Dis-
1224–25. tant influences of amygdala lesion on visual
Usunoff, K. G., Itzev, D. E., Rolfs, A., Schmitt, cortical activation during emotional face pro-
O., & Wree, A. (2006). Brain stem afferent cessing. Nature Neuroscience, 7(11), 1271–78.
connections of the amygdala in the rat with Weymar, M., Low, A., Ohman, A., & Hamm,
special references to a projection from the A. O. (2011). The face is more than its parts –
parabigeminal nucleus: A fluorescent retro- Brain dynamics of enhanced spatial attention
grade tracing study. Anatomy and Embryology, to schematic threat. Neuroimage, 58(3), 946–
211(5), 475–96. 54.
van Marle, H. J., Hermans, E. J., Qin, S., & Fer- Whalen, P. J., Kagan, J., Cook, R. G., Davis,
nandez, G. (2009). From specificity to sen- F. C., Kim, H., Polis, S., Johnstone, T.
sitivity: How acute stress affects amygdala (2004). Human amygdala responsivity to
processing of biologically salient stimuli. Bio- masked fearful eye whites. Science, 306(5704),
logical Psychiatry, 66(7), 649–55. 2061.
Vermeulen, N., Godefroid, J., & Mermillod, M. Yang, E., Zald, D. H., & Blake, R. (2007). Fear-
(2009). Emotional modulation of attention: ful expressions gain preferential access to
Fear increases but disgust reduces the atten- awareness during continuous flash suppres-
tional blink. PLoS ONE, 4(11), e7924. sion. Emotion, 7(4), 882–86.
CHAPTER 15
357
358 LUIZ PESSOA, LETICIA OLIVEIRA, & MIRTES PEREIRA
automaticity is not only in terms of counter stimulus that was interspersed with picture
evidence, but with the term itself, which has presentation (Schupp, Junghofer, Weike, &
been used in a variety of contexts (Logan, Hamm, 2003). Together, these studies are
1988). Third, in the context of awareness, often interpreted as supporting the notion
historically, several authors have suggested that the processing of emotional stimuli
that unaware effects are automatic. How- is obligatory and that increased processing
ever, the relationship between automatic- resources are allocated toward their process-
ity and unawareness is far from simple, ing even when they are irrelevant to the task
as both conceptual and experimental work at hand.
have recently suggested (Koch & Tsuchiya, A stronger argument can be advanced
2007; Lamme, 2003; Most, Scholl, Clifford, that emotional processing is obligatory based
& Simons, 2005). Thus, terms such as preat- on studies in which the spatial focus of atten-
tentive, automatic, and unaware, which are tion is explicitly manipulated – note that
routinely used when describing the effects in the experiments described the previous
of emotional information, have many unin- paragraph, emotional stimuli, although task
tended meanings and relationships. Whereas irrelevant, were fully attended. In a well-
in some cases they should be simply avoided, known study (Vuilleumier, Armony, Driver,
as in the case of preattentive, in others, for & Dolan, 2001), the attentional focus was
progress to be made, the use of automatic manipulated by having subjects maintain
and unaware should be made as precise central fixation while they were asked to
as possible. In the subsequent sections, we compare either two faces or two houses pre-
review some of the specific evidence against sented eccentrically. On each trial, subjects
and for the idea that attention is required for either compared the faces to each other
emotional processing. or the houses to each other (Figure 15.1a).
Thus, the focus of attention was varied by
having subjects attend to the left and the
Attention Is Not Required for right of fixation (while ignoring top/bottom
Emotional Perception stimuli) or above and below fixation (while
ignoring left/right stimuli). In each case,
Emotional stimuli effectively divert pro- participants indicated whether the attended
cessing resources and interfere with perfor- stimuli were the same or not. When condi-
mance even when they are task irrelevant tions involving fearful faces were contrasted
(Pessoa & Ungerleider, 2004; Vuilleumier, to those involving neutral ones, differential
2005). For instance, reaction times when responses in the amygdala – which are often
subjects performed auditory tasks (e.g., considered as a “signature” of emotional
word discrimination) were slower when processing – were not modulated by the
they viewed distracter pictures that were focus of attention, consistent with the view
unpleasant relative to neutral ones (Bradley, that the processing of emotional items does
Cuthbert, & Lang, 1996; Buodo, Sarlo, & not require attention (Figure 15.1b). Related
Palomba, 2002). Strikingly, such type of findings were observed when manipulat-
interference has been observed even when ing object-based attention while maintain-
the primary task is very basic, such as the ing constant the spatial locus of attention
detection of a simple visual stimulus (Pereira (Anderson, Christoff, Panitz, De Rosa E., &
et al., 2006). Interference effects are not Gabrieli, 2003). Amygdala responses evoked
only evident in terms of behavioral perfor- by fearful faces were equivalent whether
mance but are also manifested physiolog- or not the faces were attended. Interest-
ically. For example, specific event-related ingly, however, during unattended condi-
potential (ERP) components activated by tions, responses evoked by fear and dis-
picture viewing were modulated by emo- gust faces were comparable to each other,
tional content even when the main task consistent with the idea that attention is
involved simply detecting a checkerboard needed to discriminate emotional content
360 LUIZ PESSOA, LETICIA OLIVEIRA, & MIRTES PEREIRA
(both expressions evoked greater responses attentional demand of a task – namely, the
than neutral faces). We now turn to review- extent to which it uses up resources.
ing evidence that suggests that emotional Several fMRI studies have attempted to
perception requires attention. follow the strategy just outlined. For exam-
ple, they evaluated the responses evoked by
centrally presented emotional faces when
Attention Is Required for Emotional a very demanding peripheral task was per-
Perception formed. Under these conditions, differen-
tial responses to fearful vs. neutral faces
Based on the findings summarized in the were eliminated in both the amygdala and
previous section, emotional stimuli com- visual cortex (Pessoa, McKenna, Gutier-
prise a privileged stimulus category that is rez, & Ungerleider, 2002). Consistent with
not only prioritized but whose processing the notion that task demand was impor-
also takes place in an obligatory fashion that tant in determining the extent of processing
is independent of attention. However, it of the face stimuli, when the difficulty
is also known that, in general, visual pro- of the peripheral task was parametrically
cessing capacity is limited. Because of this manipulated, a valence effect (i.e., fear-
finite capacity, competition among visual ful > neutral) was observed during low
items is proposed to “select” the most impor- task demand conditions, but not during
tant information at any given time (Des- medium or high-demand conditions (Pes-
imone & Duncan, 1995; Grossberg, 1980). soa, Padmala, & Morland, 2005). The depen-
When resources are not fully consumed, it dence of emotional perception on atten-
has been suggested that spare processing tion was also observed in studies that used
capacity is used to process unattended centrally presented, overlapping compet-
items (Lavie, 1995). This line of reason- ing stimuli (i.e., paradigms that manipu-
ing, which has been successfully applied to late object-based attention; Mitchell et al.,
regular, nonemotional stimuli, suggests that 2007), including emotional stimuli of higher
the automaticity of affective processing can affective significance that were paired with
be tested by attentional manipulations that shock (Lim, Padmala, & Pessoa, 2008) – or
more fully consume processing resources. by using highly aversive, mutilation pictures
Thus, a critical variable in understanding (Erthal et al., 2005). Posner-type manipula-
the extent of unattended processing is the tions of attention also indicate that amygdala
TOP-DOWN ATTENTION AND THE PROCESSING OF EMOTIONAL STIMULI 361
activation depends on the focus of atten- Wilson, Muroi, & MacLeod, 2011). In one
tion (Brassen, Gamer, Rose, & Buchel, study, subjects performed a difficult target-
2010). Furthermore, attentional modulation detection task while task-irrelevant though
of the emotional valence effect has also been emotionally arousing pictures were shown
observed for peripherally presented faces in the background (Muller, Andersen, &
(Silvert et al., 2007). Keil, 2008). Despite the difficulty of the
ERP studies, which unlike fMRI studies task, emotional pictures interfered behav-
offer temporal information on the order of iorally with the main task. Parallel findings
milliseconds (see Chapters 4 and 5), have were registered in steady-state visual evoked
also investigated how emotional percep- potentials, which were reduced during the
tion depends on attentional factors. In one presentation of emotional relative to neu-
study (Schupp et al., 2007), the process- tral background images (such reduction was
ing of emotional pictures from the Inter- suggested to reflect the withdrawal of pro-
national Affective Picture System (IAPS) cessing resources from the main task by the
was strongly attenuated (as measured by an emotional distracters).
ERP component labeled the “early posterior Another recent MEG study provided evi-
negativity”) when participants performed dence for mandatory processing of fearful
demanding attention tasks. On the con- faces (Fenker et al., 2010). Both low- and
trary, passively viewing the same emotional high-demand conditions were investigated
images generated increased responses rela- in separate experiments. During the high-
tive to those evoked by neutral stimuli. Like- demand condition, the target was defined
wise, differential ERP responses to periph- by a conjunction of features, such as a red-
eral, emotional IAPS pictures also relied on green (vs. blue-yellow) vertical bar. In the
the availability of processing resources (De low-demand condition, subjects determined
Cesarei, Codispoti, & Schupp, 2009). Emo- the orientation of an oriented bar (vertical
tional pictures in the near periphery mod- versus horizontal) presented in a given color
ulated brain activity only when they were (indicated at the beginning of the block).
attended (though passively viewed), but not During the low-demand condition, task-
when participants were engaged in a dis- irrelevant fearful faces slowed down reaction
tracter task (determining whether a rectan- time when they were presented in the same
gular outline contained a gap or not). visual field as the bar target (relative to when
faces were presented in the opposite visual
field). To investigate the neural impact of
Attention Is Not Required for the task-irrelevant faces, the authors probed
Emotional Perception, Again the so-called N2pc component, which is
believed to reflect attentional focusing in
The results suggesting that emotional per- visual search. They observed that lateralized
ception is automatic or that it depends fearful faces elicited an N2pc approximately
on attention can be reconciled by mak- 240–400 ms in the contralateral visual cortex.
ing use of the concept of attentional Importantly, the N2pc was observed dur-
demand (Lavie, 1995). When demand is ing conditions of high demand, although no
low, “spillover” capacity is available for behavioral effects were detected.
the processing of task-irrelevant emotional
stimuli. As demand is increased, however,
fewer resources will be available, and in Interim Summary
the limit, emotional perception will be
eliminated. Whereas this framework can Taken together, results from behavioral
be used to explain a broad set of results, and neuroimaging methods suggest that
some findings appear to resist this explana- although emotional processing is prioritized,
tion (and for recent twists concerning the in many contexts it depends on processing
notion of “load,” see Tsal & Benoni, 2010; resources. These findings come from diverse
362 LUIZ PESSOA, LETICIA OLIVEIRA, & MIRTES PEREIRA
significant for both CS+ and CS- trials, indi- significant (a direct paired comparison was
cating that trial-by-trial fluctuations in fMRI also significant).
signals reliably predicted perceptual T2 deci- Taken together with other findings, our
sions (Figure 15.3a,b). Importantly, a direct results suggest that affective significance
comparison of the CS+ and CS- conditions potentially determines the fate of a visual
revealed that the predictive power of the item during competitive interactions by
logistic regression fit was stronger during enhancing sensory processing. By helping
CS+ relative to CS- scenes. An analogous establish affective significance, the amygdala
trial-by-trial analysis was performed for the helps separate the significant from the mun-
amygdala (Figure 15.3c,d). The mean logis- dane. One way to interpret these results is
tic regression slope was significant for CS+ in terms of an attentional function of the
trials, but not for CS- trials, indicating that amygdala (Pessoa, 2010b). For example, in
variability in fMRI signals in the amygdala studies of attention and visual cortical func-
contributed to perceptual T2 decisions more tion, fluctuation of responses in the visual
robustly when these stimuli were affectively cortex is often conceptualized as dependent
364 LUIZ PESSOA, LETICIA OLIVEIRA, & MIRTES PEREIRA
1.5
logistic slope
0.8
0.6
1.0
0.4 CS+
0.5
0.2 CS–
0.0
–0.5 –0.25 0 0.25 0.5 0.75
CS+ CS–
(c) (d)
0.75
R AMYG P < .05
1
probability of HIT
0.6 CS+
0.25
CS–
0.4
0.00
–0.5 –0.25 0 0.25 0.5 0.75 1 CS+ CS–
% signal change
Figure 15.3. Trial-by-trial analysis. (A) Logistic regression analysis of evoked responses in the right
parahippocampal gyrus (PHG) as a function of affective significance (CS+ and CS-) for a sample
individual (dichotomous variable: hits vs. misses). The slope of the logistic fit indicates the strength of
the predictive effect. For clarity, only binned data for the CS+ condition are shown (red dots). (B)
Mean logistic slopes across individuals for the parahippocampal gyrus. (C) The same analysis as in
(A) but for the right amygdala (AMYG). (D) Mean logistic slopes across individuals for the amygdala.
Adapted from Lim et al. (2009) with permission.
on “source” regions in parietal and frontal of affectively significant stimuli, see Padmala
cortices (Corbetta & Shulman, 2002; Kast- et al., (2010) and Pessoa and Adolphs, (2010).
ner & Ungerleider, 2000), and these mecha- In the context of this chapter, an espe-
nisms are typically viewed as linked to how cially important finding of the study was
the processing of attended objects is priori- that the contrast of affectively significant
tized. In our study of the attentional blink, and neutral Miss trials did not reveal signif-
something quite similar was observed inso- icant differential responses in the amygdala.
far as fluctuations in the amygdala were pre- In contrast, the comparison of affectively
dictive of the strength of the link between significant and neutral Hit trials exhibited
visual cortex and behavior. In this case, the differential responses. In other words, differ-
amygdala was found to behave much like an ential responses were not produced unless a
“attentional device” would be expected to – T2 scene was detected. More generally, this
namely, it helped prioritize the processing finding suggests that affective perception
of certain stimuli over others (Pessoa, 2010b; is indeed under the control of attentional
see also Vuilleumier, 2005 and Chapter 14). mechanisms during temporal “bottleneck”
For closely related findings suggesting a role conditions, in addition to during spatial
of the pulvinar in prioritizing the processing competition conditions. These findings are
TOP-DOWN ATTENTION AND THE PROCESSING OF EMOTIONAL STIMULI 365
system. For instance, the earliest responses Intracranial studies in humans generally find
in the LGN, which receives direct reti- the earliest single-unit responses around 200
nal input, are observed at approximately ms (Mormann et al., 2008; Oya, Kawasaki,
30 ms, and on average they occur around Howard, & Adolphs, 2002). In addition, in
33 ms (for the magno system) and 50 ms a lesion case study, affective modulation
(for the parvo system; Lamme & Roelfsema, of amygdala responses was also observed
2000). When considering neuronal response starting at 200 ms (Krolak-Salmon, Henaff,
latencies, other issues are also important. In Vighetto, Bertrand, & Mauguiere, 2004); see
addition to the latency itself, one needs to also (Oya et al., 2002). For more extensive
consider “computation time.” It has been discussion, see Pessoa and Adolphs (2010).
estimated (Tovee & Rolls, 1995) that most Recently, we have advocated that bypass
of the information encoded by visual neu- systems involving the cortex may rapidly
rons may be available in segments of activ- convey affective information throughout the
ity 100 ms long, and that a fair amount brain (Pessoa & Adolphs, 2010) – see the
of information is available in 50-ms seg- related notion of a two-stage mechanism
ments, and even some is found in 20- to underlying emotion and attention prioriti-
30-ms segments (note that these segments zation (Rudrauf et al., 2008; Vuilleumier,
take into account response latency; namely, 2005). This “parallel processing” architecture
they consider neuronal spikes after a cer- would allow fast affective responses, cer-
tain delay). Although these figures demon- tainly around 100–150 ms poststimulus onset.
strate the remarkable speed of neuronal Interestingly, this potential time course
computation (at least under some condi- matches the one observed in the intracranial
tions), they add precious milliseconds to the study by Pourtois and colleagues, in which
time required to, for instance, discriminate affective influences were observed starting
between stimuli (e.g., hypothetical differen- at 140 ms. A potential concern with that
tial responses in the LGN itself would be study, however, is that the task employed
expected no earlier than 60 ms poststim- was not challenging. Specifically, the patient
ulus onset). An additional consideration is was correct 95% of the time during face tri-
that responses in humans are possibly slower als, and 97% of the time during house tri-
than in monkeys, further adding time. For als. Whereas this made for balanced task
instance, in one human study (Yoshor, Bosk- performance for faces and houses (and was
ing, Ghose, & Maunsell, 2007), the fastest probably determined by the testing of a
recording sites had latencies just under 60 patient in the context of neurosurgery),
ms and were probably located in V1 (or pos- in all likelihood, the task was not suffi-
sibly V2). In monkey, the fastest responses ciently demanding. As discussed, process-
in V1 can be observed under 40 ms (Lamme ing resources “spill over” when the central
& Roelfsema, 2000). task is not taxing (Lavie, 1995), and therefore
What are the response latencies of neu- effects of valence under these conditions are
rons in the amygdala? In the monkey, amyg- not entirely surprising. And although the
dala responses typically range from 100 to 200 effect of valence can be referred to as “auto-
ms (Gothard, Battaglia, Erickson, Spitler, & matic” in the sense of implicit processing
Amaral, 2007; Kuraoka & Nakamura, 2007; of task-irrelevant information, the task does
Leonard, Rolls, Wilson, & Baylis, 1985; Naka- not allow for a stricter test of automaticity
mura, Mikami, & Kubota, 1992) – although in terms of obligatory processing.
shorter response latencies to unspecific stim- In light of these remarks on capacity
uli (e.g., fixation spots) have at times been limitations, it is worth considering that,
reported (Gothard et al., 2007). Differences even in the MEG study, the attentional
in evoked responses between threat and neu- manipulation may not have been suffi-
tral or appeasing facial expressions in the ciently strong. Observers performed at 83%
monkey amygdala have been reported in the correct in the high-demand condition. In
range of 120–250 ms (Gothard et al., 2007). contrast, in a similar bar-orientation task,
368 LUIZ PESSOA, LETICIA OLIVEIRA, & MIRTES PEREIRA
34
0.0
ms
–5 0 5 10 15
TIME (SECONDS)
Figure 15.5. fMRI responses to brief stimuli. Original results by
Savoy and colleagues, illustrating the fact that a clear signal
change is observed for very brief events. Data from Savoy
et al. (1995). Reproduced from Rosen, Buckner, and Dale (1998)
with permission.
performance was at 64% correct during the not be sensitive to brief events. Whereas
most demanding condition (Pessoa et al., this is a possible concern given the low-
2002); notably, in another bar-orientation pass characteristics of the BOLD signal,
study, we did observe valence effects on several examples show that this concern
reaction time when performance was 79% does not necessarily hold. Perhaps surpris-
correct, but they disappeared when per- ingly at first, the BOLD signal is indeed
formance was at approximately 60% cor- sensitive to transient events, as originally
rect (Erthal et al., 2005). In general, atten- demonstrated by Savoy and colleagues (1995;
tional manipulations that more completely Figure 15.5). Furthermore, fMRI responses
exhaust processing capacity must be sought, have been consistently reported for stim-
as in similar questions of the need for atten- uli that are presented very briefly (∼30 ms)
tion during scene perception in which a and even masked (Morris, Ohman, & Dolan,
clearer demonstration of the impact of the 1998; Whalen et al., 1998), a result repli-
attentional manipulation has been provided cated many times with affective stimuli, for
(Li, VanRullen, Koch, & Perona, 2002). instance. Thus, whereas it is clearly desir-
A second important issue with the able to obtain millisecond-level data, such as
intracranial study refers to the timing of provided by techniques like MEG, fMRI is
the attention effect, which was reported certainly not blind to brief, transient events,
to start at around 700 ms. As noted as illustrated in Figure 15.6. Indeed, even
by the authors, this timing is considerably sub-millisecond stimulation has been shown
later than effects of attention on sensory to evoke detectable hemodynamic responses
processing, which can be observed as early (Hirano, Stefanovic, & Silva, 2011).
as 60–100 ms (Luck, Woodman, & Vogel, In summary, both studies described here
2000). As the authors suggested, this late provide important insights about the tem-
effect may be due to other task-related dif- poral unfolding of affective responses in the
ferences associated with processing the emo- brain. Both argued that the effects of emo-
tional significance of the faces. In any case, tional content temporally precede those of
the effect would seem to have a different ori- attention and that prior discrepancies in the
gin than the modulation of visual processing literature may stem from the temporal char-
by attentional mechanisms. acteristics of the fMRI signal. The points
A final issue merits discussion. A com- raised earlier suggest, however, that it may
mon objection to fMRI is that it might not be yet time to accept those conclusions:
TOP-DOWN ATTENTION AND THE PROCESSING OF EMOTIONAL STIMULI 369
and vocal emotions. Journal of Neurophysiol- Marcel, A. J. (1983). Conscious and unconscious
ogy, 97(2), 1379–87. perception: Experiments on visual masking
Lamme, V. A. (2003). Why visual attention and and word recognition. Cognitive Psychology,
awareness are different. Trends in Cognitive 15(2), 197–237.
Sciences, 7(1), 12–18. Milders, M., Sahraie, A., Logan, S., & Donnellon,
Lamme, V. A., & Roelfsema, P. R. (2000). The N. (2006). Awareness of faces is modulated
distinct modes of vision offered by feedfor- by their emotional meaning. Emotion, 6(1),
ward and recurrent processing. Trends in Neu- 10–17.
rosciences, 23(11), 571–79. Mitchell, D. G., Nakic, M., Fridberg, D., Kamel,
Lang, P. J., & Davis, M. (2006). Emotion, moti- N., Pine, D. S., & Blair, R. J. (2007). The
vation, and the brain: Reflex foundations in impact of processing load on emotion. Neu-
animal and human research. Progress in Brain roimage, 34(3), 1299–1309.
Research, 156, 3–29. Mormann, F., Kornblith, S., Quiroga, R. Q.,
Lavie, N. (1995). Perceptual load as a neces- Kraskov, A., Cerf, M., Fried, I., & Koch, C.
sary condition for selective attention. Journal (2008). Latency and selectivity of single neu-
of Experimental Psychology: Human Perception rons indicate hierarchical processing in the
and Performance, 21(3), 451–68. human medial temporal lobe. Journal of Neu-
Lavie, N., & de Fockert, J. W. (2003). Contrast- roscience, 28(36), 8865–72.
ing effects of sensory limits and capacity limits Morris, J. S., Ohman, A., & Dolan, R. J.
in visual selective attention. Attention, Percep- (1998). Conscious and unconscious emotional
tion, & Psychophysics, 65(2), 202–12. learning in the human amygdala. Nature,
Leonard, C. M., Rolls, E. T., Wilson, F. A., & 393(6684), 467–470.
Baylis, G. C. (1985). Neurons in the amygdala Most, S. B., Chun, M. M., Widders, D. M.,
of the monkey with responses selective for & Zald, D. H. (2005). Attentional rubber-
faces. Behavioural Brain Research, 15(2), 159– necking: Cognitive control and personality in
76. emotion-induced blindness. Psychonomic Bul-
Li, F. F., VanRullen, R., Koch, C., & Perona, P. letin & Review, 12(4), 654–61.
(2002). Rapid natural scene categorization in Most, S. B., Scholl, B. J., Clifford, E. R., &
the near absence of attention. Procedings of Simons, D. J. (2005). What you see is what you
the National Academy of Sciences, 99(14), 9596– set: Sustained inattentional blindness and the
9601. capture of awareness. Psychological Review,
Lim, S. L., Padmala, S., & Pessoa, L. (2008). 112(1), 217–42.
Affective learning modulates spatial compe- Mothes-Lasch, M., Mentzel, H. J., Miltner, W.
tition during low-load attentional conditions. H., & Straube, T. (2011). Visual attention mod-
Neuropsychologia, 46(5), 1267–78. ulates brain activation to angry voices. Journal
Lim, S. L., Padmala, S., & Pessoa, L. (2009). Seg- of Neuroscience, 31(26), 9594–98.
regating the significant from the mundane on a Muller, M. M., Andersen, S. K., & Keil, A. (2008).
moment-to-moment basis via direct and indi- Time course of competition for visual process-
rect amygdala contributions. Procedings of the ing resources between emotional pictures and
National Academy of Sciences, 106(39), 16841– foreground task. Cerebral Cortex, 18(8), 1892–
46. 99.
Logan, G. D. (1988). Automaticity, resources, Murphy, K. R., & Myors, B. (2004). Statistical
and memory: Theoretical controversies and power analysis: A simple and general model
practical implications. Human Factors, 30(5), for traditional and modern hypothesis tests (2nd
583–98. ed.). Mahwah, NJ: Erlbaum.
Luck, S. J., Woodman, G. F., & Vogel, E. K. Nakamura, K., Mikami, A., & Kubota, K. (1992).
(2000). Event-related potential studies of att- Activity of single neurons in the monkey
ention. Trends in Cognitive Sciences, 4, 432–40. amygdala during performance of a visual dis-
Luo, Q., Holroyd, T., Majestic, C., Cheng, X., crimination task. Journal of Neurophysiology,
Schechter, J., & Blair, R. J. (2010). Emotional 67(6), 1447–63.
automaticity is a matter of timing. Journal of Nakayama, K., & Joseph, J. S. (1998). Atten-
Neuroscience, 30(17), 5825–29. tion, pattern recognition, and pop-out in
MacLeod, C., Mathews, A., & Tata, P. (1986). visual search. In R. Parasuraman (Ed.), The
Attentional bias in emotional disorders. Jour- attentive brain (pp. 279–98). Cambridge: MIT
nal of Abnormal Psychology, 95(1), 15–20. Press.
TOP-DOWN ATTENTION AND THE PROCESSING OF EMOTIONAL STIMULI 373
Oya, H., Kawasaki, H., Howard, M. A., III, capture by emotional stimuli is preserved
& Adolphs, R. (2002). Electrophysiological in patients with amygdala lesions. Neuropsy-
responses in the human amygdala discriminate chologia, 49(12), 3314–19.
emotion categories of complex visual stimuli. Posner, M. I., & Snyder, C. R. R. (1975). Atten-
Journal of Neuroscience, 22(21), 9502–12. tion and cognitive control. In R. L. Solso (Ed.),
Padmala, S., Lim, S.-L., & Pessoa, L. (2010). Information processing and cognition: The Loy-
Pulvinar and affective significance: Responses ola symposium (pp. 55–85). Hillsdale, NJ: Erl-
track moment-to-moment visibility. Frontiers baum.
in Human Neuroscience, 4, 1–9. Pourtois, G., Spinelli, L., Seeck, M., & Vuilleu-
Padmala, S., & Pessoa, L. (2008). Affective learn- mier, P. (2010). Temporal precedence of emo-
ing enhances visual detection and responses in tion over attention modulations in the lateral
primary visual cortex. Journal of Neuroscience, amygdala: Intracranial ERP evidence from a
28(24), 6202–10. patient with temporal lobe epilepsy. Cogni-
Pereira, M. G., Volchan, E., de Souza, G. G., tive, Affective, & Behavioral Neuroscience, 10(1),
Oliveira, L., Campagnoli, R. R., Pinheiro, W. 83–93.
M., & Pessoa, L. (2006). Sustained and tran- Raymond, J. E., Shapiro, K. L., & Arnell, K. M.
sient modulation of performance induced by (1992). Temporary suppression of visual pro-
emotional picture viewing. Emotion, 6(4), 622– cessing in an RSVP task: An attentional blink?
634. Journal of Experimental Psychology: Human
Pessoa, L. (2005). To what extent are emotional Perception and Performance, 18(3), 849–60.
visual stimuli processed without attention and Rosen, B. R., Buckner, R. L., & Dale, A. M.
awareness? Current Opinions in Neurobiology, (1998). Event-related functional MRI: Past,
15(2), 188–96. present, and future. Proceedings of the National
Pessoa, L. (2010a). Emergent processes in Academy of Sciences, 95(3), 773–80.
cognitive-emotional interactions. Dialogues in Rudrauf, D., David, O., Lachaux, J. P., Kovach,
Clinical Neuroscience, 12(4), 433–48. C. K., Martinerie, J., Renault, B., & Dama-
Pessoa, L. (2010b). Emotion and cognition and sio, A. (2008). Rapid interactions between
the amygdala: From “what is it?” to “what’s the ventral visual stream and emotion-related
to be done?” Neuropsychologia, 48(12), 3416– structures rely on a two-pathway architec-
29. ture. Journal of Neuroscience, 28(11), 2793–
Pessoa L. (2010c). Emotion and attention effects: 2803.
is it all a matter of timing? Not yet. Frontiers Savoy, R. L., Bandettini, P. A., O’Craven,
in Human Neuroscience, 4, 172. K. M., Kwong, K. K., Davis, T. L., Baker,
Pessoa, L., & Adolphs, R. (2010). Emotion pro- J. R., . . . Rosen, B. R. (1995). Pushing the
cessing and the amygdala: From a ‘low road’ temporal resolution of fMRI: Studies of very
to ‘many roads’ of evaluating biological sig- brief visual stimuli, onset variability and asyn-
nificance. Nature Reviews Neuroscience, 11(11), chrony, and stimulus-correlated changes in
773–83. noise. Presented at the Annual Meeting of
Pessoa, L., McKenna, M., Gutierrez, E., & Unger- the Society of Magnetic Resonance in Nice,
leider, L. G. (2002). Neural processing of France, August.
emotional faces requires attention. Proceedings Schupp,
of the National Academy of Sciences, 99(17), H. T., Junghofer, M., Weike, A. I., &
11458–63. Hamm, A. O. (2003). Attention and emotion:
Pessoa, L., Padmala, S., & Morland, T. (2005). an ERP analysis of facilitated emotional
Fate of unattended fearful faces in the stimulus processing. Neuroreport, 14(8),
amygdala is determined by both attentional 1107–10.
resources and cognitive modulation. NeuroIm- Schupp, H. T., Stockburger, J., Bublatzky, F.,
age, 28(1), 249–55. Junghofer, M., Weike, A. I., & Hamm, A. O.
Pessoa, L., & Ungerleider, L. G. (2004). Neu- (2007). Explicit attention interferes with selec-
roimaging studies of attention and the pro- tive emotion processing in human extrastriate
cessing of emotion-laden stimuli. Progress in cortex. BMC Neuroscience, 8, 16.
Brain Research, 144, 171–82. Silvert, L., Lepsien, J., Fragopanagos, N.,
Piech, R. M., McHugo, M., Smith, S. D., Goolsby, B., Kiss, M., Taylor, J. G., . . . Nobre,
Dukic, M. S., van der Meer, E., Abou- A. C. (2007). Influence of attentional
Khali, B., . . . Zald, D. H. (2011). Attentional demands on the processing of emotional facial
374 LUIZ PESSOA, LETICIA OLIVEIRA, & MIRTES PEREIRA
expressions in the amygdala. Neuroimage, Vuilleumier, P. (2005). How brains beware: Neu-
38(2), 357–66. ral mechanisms of emotional attention. Trends
Stein, T., Peelen, M. V., Funk, J., & Seidl, in Cognitive Sciences, 9(12), 585–94.
K. N. (2010). The fearful-face advantage is Vuilleumier, P., Armony, J. L., Driver, J., &
modulated by task demands: Evidence from Dolan, R. J. (2001). Effects of attention and
the attentional blink. Emotion, 10(1), 136–40. emotion on face processing in the human
Streit, M., Dammers, J., Simsek-Kraues, S., brain: An event-related fMRI study. Neuron,
Brinkmeyer, J., Wolwer, W., & Ioannides, 30(3), 829–41.
A. (2003). Time course of regional brain Vuilleumier, P., Richardson, M. P., Armony, J.
activations during facial emotion recogni- L., Driver, J., & Dolan, R. J. (2004). Distant
tion in humans. Neuroscience Letters, 342(1–2), influences of amygdala lesion on visual cortical
101–4. activation during emotional face processing.
Tovee, M. J., & Rolls, E. T. (1995). Information Nature Neuroscience, 7(11), 1271–78.
encoding in short firing rate epochs by single Whalen, P. J., Rauch, S. L., Etcoff, N. L., McIn-
neurons in the primate temporal visual cortex. erney, S. C., Lee, M. B., & Jenike, M. A.
Visual Cognition, 2(1), 35–58. (1998). Masked presentations of emotional
Treisman, A. M., & Gelade, G. (1980). A feature- facial expressions modulate amygdala activity
integration theory of attention. Cognitive Psy- without explicit knowledge. Journal of Neuro-
chology, 12(1), 97–136. science, 18(1), 411–18.
Tsal, Y., & Benoni, H. (2010). Diluting the Wilson, D. E., Muroi, M., & MacLeod, C.
burden of load: Perceptual load effects are M. (2011). Dilution, not load, affects distrac-
simply dilution effects. [Research Support, tor processing. [Research Support, Non-U.S.
Non-U.S. Gov’t]. Journal of Experimental Psy- Gov’t]. Journal of Experimental Psychology:
chology: Human Perception and Performance, Human Perception and Performance 37(2), 319–
36(6), 1645–56. 35.
Tzelgov, J. (1997). Specifying the relations Yoshor, D., Bosking, W. H., Ghose, G. M., &
between automaticity and consciousness: A Maunsell, J. H. (2007). Receptive fields in
theoretical note. Consciousness and Cognition, human visual cortex mapped with surface
6(2–3), 441–51. electrodes. Cerebral Cortex, 17(10), 2293–2302.
CHAPTER 16
Emotion Regulation
Emotions are ever present in our day-to- situational context in order to respond more
day lives. We experience emotions when we effectively to situations and pursue their
watch horror movies, listen to amusing sto- long-term goals (Gross, 1999).
ries, witness the birth of our child, ruminate The past two decades have seen a flood of
over the death of a loved one, and experi- interest in affective neuroscience in investi-
ence perceived disrespect. Indeed it is hard gating the psychological, social, and biologi-
to imagine a person going through these cal basis of the production and experience
events without experiencing profound emo- of emotions. Mostly in parallel, the field
tions, illustrating the importance of emo- of cognitive neuroscience has been clarify-
tions in imbuing structure and meaning to ing the brain basis of higher order cognitive
life events. Emotions attach salience to goal- processes such as attention, reasoning, mem-
relevant aspects of the environment and, ory, and problem solving. The study of emo-
once triggered, bias cognition and action in tion regulation invites a synthesis of theories
characteristic ways that have proven to be and findings from these two perspectives.
adaptive across evolutionary time. Emotions In the study of the cognitive regulation of
are thus often helpful and indeed essen- emotions, “hot” emotional processes (e.g.,
tial for survival. However, emotions are fear, pleasure, anger) and “cold” cognitive
not always useful or adaptive; they can be processes (e.g., decision making, memory,
problematic when they are exaggerated in attention) are inextricably linked (also see
intensity, persist for long periods, emerge Chapters 14, 15, and 17). This chapter draws
unpredictably, or are evoked out of con- on existing frameworks for understanding
text. When emotional reactions are inappro- emotion regulation to critically review a
priate, humans are uniquely able to deploy large and growing body of work in affec-
an array of strategies to flexibly modify the tive and cognitive neuroscience on the cog-
experience of emotion. Humans can alter nitive regulation of emotion. It focuses on
the onset, duration, content, and quality of studies employing neuroimaging, which is
an emotion experience based on the current one powerful and widely used method to
375
376 K. LUAN PHAN, & CHANDRA SEKHAR SRIPADA
delineate the neural mechanisms underlying nated effects across response systems can
the capacity to flexibly alter and refine the be seen as evolutionary-conserved “response
experience of emotions in humans. profiles” that have proven to be effective
in recurrent adaptive challenges (Nesse,
1990). For example, the emotion of fear
What Is Emotion Regulation? arising from an impromptu encounter with
a snake promotes a coordinated sequence
To understand emotion regulation, it is of adaptive responses including enhanced
critical to gain a clearer understanding of sympathetic arousal, freezing, recoil, and
emotion. According to current integrative withdrawal.
models (Levenson, 1994), emotions consist Although emotions specify certain pro-
of transient, valenced responses that are totypical ways of appraising and respond-
elicited by prototypical environmental sit- ing to recurrent situations in the environ-
uations (see Chapter 1). Once triggered, ment, many aspects of the unfolding of an
they produce a coordinated suite of changes emotion are nonetheless importantly sub-
across multiple cognitive and physiological ject to modification. For example, whereas
systems, and they generate specific tenden- some appraisals are often thought to be
cies to act. Additionally, they are typically fast, automatic, and largely outside con-
accompanied by distinctive subjective feel- scious awareness (Ortony et al., 1990), others
ings and facial expressions. It is useful to may be subsequently modified thorough
divide the unfolding of an emotion into conscious, volitional strategies. Addition-
a perception/appraisal stage and response ally, autonomic responses and action ten-
phase, though doing so does not imply that dencies associated with an emotion can
the stages are temporally non-overlapping or be modulated or suppressed (Gross, 1998;
that the causal links between the stages are Gross & Thompson, 2007). Indeed, the fore-
unidirectional. In the perception/appraisal going description of emotions raises the pos-
stage, the individual confronts a situation sibility that there are multiple points during
and allocates attention to its key perceptual the unfolding of an emotion where a per-
features. Additionally, the person engages son can intervene to modulate or alter the
in cognitive appraisals, which can be fast emotion in fundamental respects, and in the
and automatic or slower, deliberate evalu- third section we discuss specific strategies
ations of how the situation relates to his or for emotion regulation that have been devel-
her goals (Ortony, Clore, & Collins, 1990). oped and elaborated by Gross (Gross, 1998,
These goals can be momentary or endur- 1999, 2002; Gross & Thompson, 2007).
ing; they can be held consciously with voli- Two terminological caveats are in order.
tional intent or reflexively without aware- First, it can be argued that emotion regu-
ness (Gross & Thompson, 2007). Specific lation could also encompass the processes
emotions are then triggered depending on by which emotions regulate thoughts and
how the situation is appraised relative to the actions (i.e., regulation by emotions; Cam-
person’s goals (e.g., fear is triggered if a core pos, Frankel, & Camras, 2004). Indeed, a
survival-related goal is threatened, anger is parallel literature has emerged regarding the
triggered if a goal is frustrated, and so on) processes by which emotions can affect sen-
(Ortony et al., 1990). sory perception and cognitive functions such
Turning now to the response stage, emo- as attention, working memory, and decision
tions are expressed across multiple response making and their associated brain substrates;
systems including (1) cognitive systems a detailed discussion of this literature can
(attention, memory; see Chapters 15 and 20); be found in Chapters 14 and 16. For exam-
(2) physiological systems (parasympathetic/ ple, emotional information can facilitate or
sympathetic tone, peripheral vasculature; interfere with cognitive processes such as
see Chapter 3); and (3) motivation/action threat-mediated avoidance and appetitive-
systems (see Chapter 19). These coordi- mediated approach, which are known to
EMOTION REGULATION 377
facilitate behavioral responding and activa- dissipates). This chapter focuses on the neu-
tion in the amgydala and ventral striatum, ral mechanisms that underlie emotion regu-
respectively (see Chapter 19). However, lation: What is the brain basis by which peo-
Gross has noted that if “emotion regula- ple regulate their experience of emotions?
tion” also refers to ways by which emo-
tions coordinate and regulate subsequent
responding (and cognition) then the term Why Is Emotion Regulation Important?
itself is redundant with that of “emotion” –
as such, all situations that involve emo- The importance of emotion regulation can
tion would also involve emotion regulation be appreciated by recognizing its roots in
(Gross, 1999, 2002), suggesting that using the a number of influential psychological tra-
term “emotion regulation” to refer to the ditions, including work on psychological
cognitive regulation of emotion rather than defenses against impulses and inhibition of
vice versa has certain advantages. anxious states (Freud, 1946), coping under
Second, the concept of emotion could stress (Lazarus, 1966), parent-child attach-
be interpreted broadly to encompass several ment (Bowlby, 1969), cognitive-behavioral
other related psychological states – affect, strategies to treat depression (Beck, 1963),
mood, feeling, motivation. In the literature, and emotional growth and development
these terms are at times used interchange- (Thompson, 1994). Moreover, dysregulated
ably, and in human imaging neuroscience emotion is a prominent part of the clin-
research they are all often studied using ical phenotype in more than half of the
identical or closely related paradigms. It has classified mental disorders, and contem-
been argued that emotion differs from mood porary process models of psychopathology
in that the former involves a discrete mental centrally implicate faulty, insufficient, or
state change and is associated with a speci- absent emotion regulation. Even in psy-
fied duration and/or with an identifiable trig- chiatric illnesses that are not conceptual-
ger. Nonetheless, it is not yet clear whether ized as disorders of emotion, emotion reg-
the difference between emotion and mood ulation may nonetheless be important. For
is critical for emotion regulation research or example, in substance use disorders, diffi-
whether, for the purposes of studying reg- culty with coping with day-to-day stress is
ulation, we could subsume emotion, mood, linked to worsened control of drug-directed
and related concepts such as stress response impulses, undermining attempts at absti-
and motivational impulse under the broader nence and treatment success, and increas-
concept of “affect” (Gross & Thompson, ing vulnerability to relapse back to addic-
2007). In what follows, we focus on the reg- tion (Fox, Hong, & Sinha, 2008). Moreover,
ulation of “emotion” and allow that future the benefits of an effective capacity to reg-
work may demonstrate that a broader con- ulate emotion extends beyond “psychiatric”
cept of affect regulation may be more justi- illness to a broader sense of well-being; for
fied or meaningful. example, greater use of emotion regulation
In sum, emotion regulation is defined strategies by hemodialysis patients improves
here as the processes by which emotional subjective well-being and self-management
responses are interrupted and/or altered of their kidney disease (Gillanders, Wild,
such that the experience and expression of Deighan, & Gillanders, 2008), and inter-
emotional states are different from the case ventions aimed at improving emotion reg-
in which they are left unregulated. Regula- ulation lead to physical and psychosocial
tion can potentially alter features along mul- improvement for patients with medical or
tiple dimensions of the emotion episode, psychiatric illness (Smyth & Arigo, 2009).
including latency (when it starts), time to In short, emotion regulation has a far-
peak (how fast it emerges), magnitude (how reaching impact on multiple aspects of
intensely it manifests), duration (how long health, disease, and interpersonal function-
it lasts), and pace of offset (how slowly it ing and is now recognized as a critical
378 K. LUAN PHAN, & CHANDRA SEKHAR SRIPADA
Emoon-related
Cognive
g
mental processes
responses
Movaonal
responses
area of study in psychology, psychiatry, and gies can be deployed: (1) “attentional deploy-
neuroscience. ment” or directing attention toward/away
from the inciting stimulus within the situa-
tion to change its emotional impression and
What Strategies Are Deployed during (2) “cognitive change” or modifying the eval-
Emotion Regulation? uation of the situation to change its meaning
or emotional significance. Lastly, “response
Earlier, we described a model of emotions modulation” refers to changing the reaction
dividing the unfolding of emotions into a after the emotional/behavioral response has
perception/appraisal stage and a response been generated and represents a response-
stage. The schematic illustrated in Figure 16.1 focused emotion regulation strategy; here,
provides the backdrop for the studies dis- response modulation is conceptualized as an
cussed in this chapter and is derived from the alteration that can occur in the internal (e.g.,
contemporary and influential model of emo- heart rate change) or external (e.g., facial
tion regulation developed by Gross and col- expression change) milieu.
leagues, who propose a number of strategies As with all frameworks used to describe a
that intervene during the unfolding emo- relatively new, rapidly growing, and chang-
tional episode at relatively specific places ing domain of research, the model con-
(Gross, 1998, 1999, 2002; Gross & Thompson, ceptualized by Gross and colleagues does
2007). First, at least two antecedent-focused not encompass all elements that may be
(or prior to behavioral responding) strategies involved, as they note and acknowledge
occur at the stage of perception/appraisal (see, for example, Gross & Thompson,
before the emotional response is evoked: 2007). First, emotion is a dynamic process,
(1) “situation selection” or taking actions and the path from stimulus/situation to
to change the likelihood that the situation response may not always proceed in a step-
will lead to the un/desired emotional out- wise sequential fashion, continuous feed-
come and (2) “situation modification” or back and feedforward could occur at each
changing the situation to alter its emotional step, and steps could be skipped as an emo-
impact. Second, during the appraisal phase, tion unfolds. Second, emotion regulation is
two additional antecedent-focused strate- an interactive process, by which one strategy
EMOTION REGULATION 379
can influence the emergence or impact of Successes of the existing literature thus
another strategy. Third, attempts to regulate prompt extending our investigation to hith-
emotion are iterative, additive, and/or syn- erto less tractable processes related to how
ergistic. We humans likely use trial and error we regulate our emotions.
to test which among many candidates is the As with all studies that couple func-
best strategy to use at the moment given the tional brain imaging methods and experi-
situation; we likely adopt and try more than mental tasks and that make brain-behavior
one strategy at any given time; and we are inferences from observations of changes in
likely to order and reorder strategies based maps of brain “activation,” it is worthwhile
on their immediate success at modifying the to inquire about additional evidence (other
emotional response according to our goals. than patterns of differences in brain activa-
Despite these observations, the model pro- tion) that supports the interpretation that
posed by Gross is nonetheless a straightfor- regulation was successful and change in
ward and highly useful heuristic based on a emotion has indeed occurred. Such corrob-
synthesis of convergent ideas over the past orating data should come from more than
decades about how we generate and regulate just the brain readout of emotions. First, an
emotions (Gross, 1999). A number of human index of “regulation success” can be derived
neuroscience studies using functional brain from subjective awareness of change in the
imaging techniques (mostly functional mag- intensity of affect or arousal, as measured
netic resonance imaging [fMRI], but also by asking subjects to rate their emotional
positron emission tomography [PET] and state during the course of the imaging exper-
event-related potential electroencephalog- iment. Second, changes in peripheral phys-
raphy [ERP EEG]) have experimentally iology can also mark the change as mea-
probed these component processes and their sured by skin conductance, startle reflexes,
neural correlates. The following section and/or cardiac-respiratory reactivity. Third,
reviews, organizes, and critically evaluates emotion modification can also be indexed
these findings. by changes in central brain responses. There
As a prelude, it is worth noting that are rich, growing, and converging data from
contemporary research in human imaging animal and human lesion studies and human
neuroscience has focused primarily on voli- functional imaging studies that implicate a
tional, effortful, cognitive strategies – atten- set of brain regions during the appraisal,
tional deployment and cognitive change – processing, and generation of emotions; see
and less on those that involve antecedent Chapters 6–13 as well as previous reviews
attempts to select and modify the exter- and meta-analyses (Adolphs, 2002; Fusar-
nal environment/situation or on those that Poli et al., 2009; Phan et al., 2003).
occur automatically or without conscious Chief among these brain regions is the
awareness. This focus is partly driven by amygdala, thought of as the gateway to emo-
the limiting confines of the functional brain tional expression and perception (Adolphs,
imaging environment, which make it more 2002). Most relevant to the topic of emo-
difficult to approximate real-life scenarios tion regulation in humans, several studies
and behaviors, and by the lack of val- have observed that the extent to which
idated techniques to objectively measure amygdala reacts is in part dependent on
and ascribe brain-function interpretations of (1) the amount of perceived threat, (2) the
unconscious phenomena. Nonetheless, the magnitude of subjective and physiological
growing literature on the functional neu- arousal, and/or (3) the salience of the stim-
roanatomy of the cognitive control of emo- ulus/environmental change (Adolphs, 2002;
tion has richly informed us about the brain Phan et al., 2003). In addition, a set of paral-
mechanisms by which cognitive strategies imbic structures are increasingly recognized
can modify subjective and neural correlates as part of an emotion sensing and generat-
of emotions, and it is producing conver- ing circuit in humans including, though not
gent findings across studies and laboratories. limited to, the insula (Craig, 2009), ventral
380 K. LUAN PHAN, & CHANDRA SEKHAR SRIPADA
Frankenstein et al 2001 Beauregard et al 2001 Ochsner et al 2002 Ohira et al 2006 Phelps et al 2004
Mitchell et al 2003 Levesque et al 2003 Ochsner et al 2004 Goldin et al 2008 Kalisch et al 2006
Anderson et al 2004 Ochsner et al 2004 Phan et al 2005 Milad et al 2007
Erk et al 2006 Kalisch et al 2005 Urryet al 2006 Delgado et al 2008
Kalisch et al 2006 Goldin et al 2008 Banks et al 2007
Blair et al 2007 Goldin et al 2009a Eippert et al 2007
Delgado et al 2008 Koenigsberg et al 2010 Kim et al 2007
Kompus et al 2009 Erk et al 2010 Wager et al 2008
Goldin et al 2009b
Figure 16.2. Representative citations of functional neuroimaging studies investigating brain correlates
of the various emotion regulation strategies.
striatum including the nucleus accumbens, est represent the current conceptualization
and orbitofrontal cortex (OFC; Rolls, 2000). that different brain regions are segregated
Moreover, negative emotional experience in function or modules. However, as our
and attempts to maintain negative affect understanding of brain function increases
are also associated with increases in amyg- and we advance more sophisticated analy-
dala and insula activity. Compared against ses of brain activity, we will likely deepen
a control condition involving passive view- our understanding that an integrated (rather
ing of negatively valenced pictures, actively than segregated) neural network subserves
maintaining a negative emotional state leads the multiple strategies for regulation of
to greater self-reported negative affect and emotion.
greater amygdala reactivity (Schaefer et al.,
2002). Moreover, volitional self-induced sad-
ness has been shown to lead to greater What Are the Functional Brain
negative affect, which was correlated with Mechanisms that Implement Emotion
increased amygdala activation (Posse et al., Regulation?
2003). Interestingly, greater self-reported
Perception Change: Attentional
regular/day-to-day use of reappraisal is asso-
Control and Distraction
ciated with decreased amygdala activity
and increased PFC and parietal engage- Perception change refers to the emotion reg-
ment when processing negative information ulatory process by which we can alter what
during fMRI scanning (Drabant, McRae, we perceive or become aware of directly
Manuck, Hariri, & Gross, 2009). Thus, a through our senses, especially vision. It
number of functional neuroimaging stud- includes how we control our attentional
ies of emotion regulation (summarized in stance (i.e., “attentional deployment”) by
Figure 16.2 and discussed later) have focused looking toward or away from emotion-
on these structures broadly, and on the ally salient (aversive or pleasant) cues in
amygdala in particular, as neural markers the environment. For example, to down-
for the efficacy of regulation in altering regulate affect, we can use distraction to
emotion. Here, the neural representations refocus our attention to a different aspect of
of emotion regulation, classified as cogni- the stimulus to lessen its emotional impact;
tive control over emotion, can be subsumed we may direct our gaze from the eyes to
under a broader construct of bidirectional the nose when looking at someone who
emotion-cognition interactions, as noted in expresses anger, or we can focus our atten-
Figure 16.3 and earlier reviews (Ochsner & tion on how wide the nose is rather than
Gross, 2005, 2007). To date, regions of inter- how angry that person is. Similarly, we can
Figure 3.4. Co-localization of insula activations
to subjective feelings of disgust and associated
increase in tachygastric responses. The figure
shows brain regions activated to the experience
of disgust (both core and body-boundary-
violation) in yellow and the associated
tachygastric response in blue. Commonly
activated regions illustrated in green show that
insula regions underpinning subjective feelings
of disgust also provide a representation of
peripheral gastric responses. Activations
illustrated at p < 0.005. Reproduced with
permission from Harrison et al., Journal of
Neuroscience (2010), 30(38):12878–84.
(a) (b)
Figure 10.1. Anatomy of the human olfactory brain. (a) A ventral view of the human brain in which
the right anterior temporal lobe has been resected in the coronal plane to expose the limbic olfactory
areas depicted in panel (b). Afferent output from the olfactory bulb (OB) passes through the lateral
olfactory tract (LOT) and projects monosynaptically to numerous regions, including the anterior
olfactory nucleus (AON), olfactory tubercle (OTUB), anterior piriform cortex (APC), posterior
piriform cortex (PPC), amygdala (AM), and entorhinal cortex (EC). Downstream relays include the
hippocampus (HP) and the putative olfactory projection site in the human orbitofrontal cortex
(OFColf ). As noted in the inset, information is not transferred serially through this circuit.
Monosynaptic projections from the lateral olfactory tract reach numerous downstream regions in
parallel, and these regions are then reciprocally interconnected (not shown). Adapted from Gottfried
(2010).
(a) (b) (c)
1.0 tgCS+ chCS+ CS− chCS−
0.8 0.9
Pre
(tirangular test)
Pre
0.6
Post 0.8
0.4
Post
0.7
0.2
0.0 0.6
CS+ pair CS– pair CS+ pair CS– pair
Figure 10.2. Role of olfactory cortical areas in aversive learning. In an fMRI version of olfactory fear
learning, subjects smelled pairs of odor enantiomers that were perceptually indistinguishable. One of
these enantiomers, designated the target CS+ (tgCS+), was subsequently paired with mild footshock.
Neither its chiral counterpart (chCS+) nor a control pair (CS- and chCS-) was paired with shock.
From preconditioning to postconditioning, perceptual discrimination between tgCS+ and chCS+
selectively increased (a), in parallel with fMRI ensemble decorrelation (less pattern overlap) in PPC
(b). Ensemble activation maps across PPC voxels in one subject (c) show increasing pattern
divergence between tgCS+ and chCS+ after learning. Adapted from Li et al. (2008).
Figure 12.1. Illustration of limbic (amygdala, nucleus accumbens,
anterior cingulate cortex [AAC], and hippocampus) and paralimbic
structures (orbitofrontal cortex, parahippocampal gyrus). The
diamonds represent music-evoked activity changes in these
structures (see figure legend for references and the main text for
details). Note the repeatedly reported activations of amygdala,
nucleus accumbens, and hippocampus, reflecting that music is
capable of modulating activity in core structures of emotion (see the
text for details). Top left: view of the right hemisphere; top right:
medial view; bottom left: anterior view; bottom right: bottom view.
3 Hz presentation speed Scalp potential
09 010
[µV] [µV]
−2 −2
Unpleasant
Pleasant
Neutral
2 2
[ms] [ms]
100 200 300 100 200 300
−1 1 µV
Unpleasant
Pleasant
Neutral
2 2
[ms] [ms]
100 200 300 100 200 300
0 200 nA/cm2
Figure 13.4. Left panel: During the reading of emotionally arousing nouns, event-related cortical
potentials are found to be amplified in comparison to neutral words. This amplification can be found
for a variety of stimulation rates (top left, 3 Hz; bottom left, 1 Hz) and takes the form of an early
posterior negativity (EPN) occurring around 250 ms after word onset. It shows a broad
occipitotemporal scalp distribution with a left-hemisphere preponderance. The sources of this activity
are localized in predominantly left posterior-temporal cortical regions. Adapted from Kissler et al.
(2007).
(a) (b) 1150
1058 1057
1100
1004
1050
Mean RT (ms)
1000
Spider target
950 894 Snake target
857 Fear-irrelevant
900
822 target
850 798
772
800 760
750
Spider Snake Nonfearful
fearful fearful
(c)
Figure 14.1. Effects of the emotional meaning of stimuli on attention and brain responses. (A)
Example of a visual search task where an emotional target (spider or snake) must be detected
(present/absent) among neutral images (mushrooms). (B) Search performance (response time, RT) of
spider fearful, snake fearful, and nonfearful participants. Fearful participants were faster at detecting
their feared targets, compared to both nonfeared (but still fear-relevant) and nonemotional targets.
Adapted from Flykt & Caldara (2006). (C) Blood-oxygenation-level-dependent (BOLD) response
observed with fMRI shows a precise overlap between emotional modulation effects and
category-selective areas for face (fusiform face area; FFA), body (extrastriate body area; EBA), and
voice processing, demonstrating that the emotional modulation is category specific. Adapted from
Vuilleumier et al. (2001), Grandjean et al., (2005) and Peelen et al. (2007).
A B
L R
+
+ +
Left-sided Right-sided
C-6 Emotional valid
Emotional invalid targets targets
-4
fMRI response
2
3 6
P1 * 5
4
100 200 ms 4
2
3
D 50-100 ms 100-150 ms
1 2
1
0 Left Right
0 Left Right
Visual Visual Visual Visual
Field Field Field Field
target target
Emotional Emotional
Valid Invalid Emotional valid Emotional invalid
Figure 14.2. Typical pattern of neural responses elicited by emotional cues in a spatial dot-probe task.
(A) In this task, targets (vertical or horizontal bars) are always preceded by pairs of faces, one neutral
and one emotional, which do not predict the side of target presentation. To avoid eye movements,
participants are instructed to press a response button when the orientation of the bar presented in the
left or right upper visual field matches that of the thicker segment in the fixation cross. Adapted from
Pourtois, Grandjean, Sander, & Vuilleumier (2004). (B) Targets preceded by neutral cues at the same
location (i.e., with fearful cues presented contralaterally; that is, emotional invalid cues) lead to
reduced fMRI responses in the intraparietal sulcus (IPS) ipsilateral to the target, relative to when
targets are preceded by emotional cues on the same side (emotional valid). This suggests a capture of
attention by the contralateral emotional cue and a reduced ability to reorient to the target. Adapted
from Pourtois, Schwartz, Seghier, Lazeyras, & Vuilleumier (2006). (C) During EEG recording, targets
preceded by emotional cues (emotional valid) elicit a larger visual P1 component, relative to those
preceded by neutral cues (emotional invalid). This is consistent with enhanced visual processing of
targets when cued by emotion. Adapted from Pourtois et al. (2004). (D) Neural source estimation for
EEG modulations at different latencies after target onset. When the target is preceded by an emotional
valid cue, activity to the target is first increased in parietal areas (50–100 ms), presumably reflecting
top-down attentional signals induced by the emotional cue and responsible for faster spatial orienting
to the target location and enhanced processing in extrastriate visual cortex at the P1 latency (100–150
ms). When targets are preceded by an invalid neutral cue, increased activity is observed in the anterior
cingulate cortex instead, reflecting some interference and conflict in attentional orienting due to initial
capture by the fearful face on the opposite side. Adapted from Pourtois, Thut, Grave de Peralta,
Michel, & Vuilleumier (2005).
DLPFC
A)
VLPFC
B))
Evoked Response
6 0.1 med 0.1
low
4 0 0
−0.1 −0.1
2
−0.2 −0.2
0 1 3 5 7 9 1113 15 1 3 5 7 9 1113 15
Peri-Stimulus Time Peri-Stimulus Time
(b)
(b) 6
preSEF SMA
Contrast Estimate (a.u.)
−2
Ve Vh Ve Vh
Figure 17.4. Action values in supplementary motor cortex. (A)
Region of supplementary motor area and pre-supplementary
eye fields correlating with action values for hand movements
(green; Vh), and eye movements (red, Ve), respectively. (B)
Plot of parameter estimates depicting correlations with action
values for eye and hand movements in these areas.
Figure 20.1. The basic mechanisms through which the emotional content of an experience can influence the way in which it is remembered.
Figure 21.2. Using fMRI in healthy participants this study investigated the neuroanatomical basis of
abstract moral and social values (figure adapted from Zahn, Moll, Paiva. et al., 2009). Participants had
to imagine actions in accordance with or counter to a value described by a written sentence and to
decide whether they would feel pleasantly or unpleasantly about the action. After the scan they rated
the unpleasantness/pleasantness on a scale and chose labels that best described their feelings (the
analysis compares each moral sentiment versus visual fixation and versus two other moral sentiments;
only selective effects were reported). There were four experimental conditions: (1) positive
self-agency: “Tom (first name of participant) acts generously toward Sam (first name of best friend)”–
pride in this condition was associated with ventral tegmental, septal, and ventral medial FPC
activation (not depicted); 2) positive other-agency: “Sam acts generously toward Tom” – gratitude in
this condition was associated with hypothalamic activation; (3) negative self-agency: “Tom acts
stingily toward Sam” – guilt in this condition was associated with subgenual cingulate cortex as well as
ventral medial FPC activation (not depicted and only when modeling individual frequency of guilt
trials); and (4) negative other-agency: “Sam acts stingily toward Tom” – indignation/anger in this
condition was associated with lateral orbitofrontal/insular activation. In the center, one can see the
right superior aTL region showing equally strong activation during all moral sentiment and agency
contexts; this region showed increased activity with increasing richness of conceptual details
describing social behavior and is identical to the activation found in a semantic judgment task (Zahn
et al., 2007). These results confirmed the right superior anterior temporal lobe as a
context-independent store of social conceptual knowledge that allows us to understand the core
meaning of social and moral values regardless of what exact feelings or actions we tie to the value.
Figure 21.3. Brain regions activated when participants donated or were opposed to donate to
charitable organizations during fMRI (Moll et al., 2006); figure adapted from Moll & Schulkin (2009).
(a) Both pure monetary rewards and decisions to donate (with or without personal financial costs)
activated the mesolimbic reward system, including the ventral tegmental area (VTA) and the ventral
and dorsal striatum. (b) The septal-subgenual region (SG), however, was selectively activated by
decisions to donate, as compared with pure monetary rewards (both by costly and noncostly
decisions, conjunction analysis). The lateral orbitofrontal cortex (latOFC) was activated by decisions
to oppose charities. This activation extended to the anterior insula and to the inferior dorsolateral
PFC and was present both for costly and noncostly decisions (conjunction analysis). The FPC and
ventral medial PFC (mPFC) were activated for costly decisions (when voluntarily sacrificing one’s
own money either to donate to a charity or to oppose it (conjunction analysis).
Figure 23.2. Shared and distinct neural networks for
self-experienced pain and empathy for pain. Depicted functional
neural activations are the result of a meta-analysis based on nine
fMRI studies investigating empathy for pain (Lamm et al., 2011).
Activations related to self-experienced pain (green) encompass a
large portion of the insula, including the middle and posterior
insular cortex, whereas activations related to empathy for pain (red)
are restricted to the most anterior parts of AI, where they overlap
with activations related to self-experienced pain. Functional
activation maps are overlaid on a high-resolution structural MRI
scan in standard stereotactic space (MNI space). White labels
indicate slice number in stereotactic space, L = left hemisphere, AI
= anterior insula, ACC = anterior cingulated cortex, PI = posterior
insula, S1 = primary somatosensory cortex. With permission from
Springer Science+Business Media: Brain Structure and Function,
The role of anterior insular cortex in social emotions, 214, 2010,
p. 581, Lamm, C., & Singer, T., Figure 1.
Figure 27.1. Recent studies suggest that development of the functional neural bases of complex
affective processing continues between adolescence and adulthood. This figure shows a region of the
ventromedial prefrontal cortex (peak voxel: −10 46 8) that responded to a greater extent in adolescent
males (mean age 14.1 years; n = 15) than in matched adult controls during the presentation of cartoon
scenarios requiring affective Theory of Mind (understanding emotions in a social context) relative to
those requiring cause-and-effect reasoning (physical causality). Group differences in this region were
not seen in response to scenarios requiring cognitive Theory of Mind (mentalizing), suggesting that
integrating affective and social cognition may represent a unique challenge for the developing
adolescent brain. Figure adapted from Sebastian et al. (2011).
EMOTION REGULATION 381
authors observed negative connectivity be less effective than other cognitive change
between the VLPFC and amygdala, and strategies to regulate anxiety and emotional
positive connectivity between VLPFC and reactivity (Kalisch, Wiech, Herrmann, &
dorsal ACC and superior frontal gyrus, Dolan, 2006). This finding is in contrast
including DLPFC, consistent with current with another study that employed a distract-
models of emotion regulation circuitry ing verbal attention task in the context of
(Ochsner, 2004, Ochsner & Gross, 2005). cold pressor pain induction; it found ros-
However, two points should be noted. tral ACC, not LPFC, activation during cog-
First, the observation that directing atten- nitive distraction from pain (Frankenstein,
tion away from emotional inputs can effec- Richter, McIntyre, & Remy, 2001).
tively dampen amygdala reactivity is in con- Self-generation of autobiographical
trast to the prevailing view that amygdalar memories, particularly those that are
emotional processing, particularly fear pro- incongruent with the current evoked or
cessing, is carried out in an automatic man- to-be-evoked negative emotional state, may
ner, with little dependence on attentional be another way to direct attention away
resources (Anderson, Christoff, Panitz, De from the current situation. For example,
Rosa, & Gabrieli, 2003; Whalen et al., 1998). Cooney and colleagues instructed partici-
Second, the findings that attention mod- pants to retrieve positive memories while
ification is an effective regulator of emo- watching sadness-inducing film clips; during
tional reactivity in the brain have been mood-incongruent recall, they observed
inconsistent. For example, several stud- activation of the VLPFC and OFC, but not
ies have observed that amygdala responses activation of more dorsal PFC networks
increase when participants attend to percep- (Cooney, Joormann, Atlas, Eugene, &
tual (rather than emotional) features (e.g., Gotlib, 2007). The authors speculated that
Critchley, et al., 2000; Hariri, Bookheimer, interruption of a negative affective state
& Mazziotta, 2000). A recent meta-analysis with positive memories may occur via
did confirm this variability in findings but a more ventral regulatory circuit, unlike
reported that, on the whole across stud- other strategies such as reappraisal and/or
ies, implicit processing of emotional faces distancing (discussed later).
(e.g., attention directed to nonemotional Physical pain can be thought of as an
features) is associated with less amgydala emotion-like state, which is highly arous-
reactivity compared to explicit processing of ing, is negatively valenced (and thus affec-
emotional faces (e.g., attention to emotional tively colored), and tends to activate regions
expressions; Fusar-Poli et al., 2009), associated with emotional responding (see
As an alternative strategy to modify atten- Chapter 9 for a review). Active distraction
tional allocation, some studies have investi- strategies that limit attention to a pain stim-
gated the neurocircuitry subserving thought ulus, such as engaging in a parallel ver-
distraction, whereby subjects are instructed bal fluency task (Frankenstein et al., 2001)
to use an alternative focus of attention (e.g., or a Stroop task (Bantick et al., 2002) or
self-generated focus or focus on inciden- simply thinking of something else (Tracey
tal external/internal stimuli) rather than the et al., 2002), are associated with reduced
emotional content of the stimulus. Kalisch pain-related responding in mid-cingulate
and colleagues asked participants to self- cortex, insula, and periacqueductal gray;
distract away from anticipatory anxiety for enhanced activation of orbitofrontal cortex
pain (“suppress any thought or feeling of (OFC), ACC, DLPFC, VLPFC; and low-
anxiety or about the shock”). Thought dis- ered subjective perception of pain intensity.
traction was inconclusively associated with Unlike other emotion evocation paradigms
reduction of behavioral measures of anxiety using low-intensity stimuli (faces, pictures),
and was associated only with tonic activa- studies using physical elicitation of pain
tion of left LPFC (nearby DLPFC), suggest- have produced more consistent and direct
ing that self-distraction by this method may evidence of prefrontal engagement during
EMOTION REGULATION 383
affect (Ochsner, Bunge, Gross, & Gabrieli, regulation, thought to represent behavioral
2002). The authors followed up these ini- inhibition, interference resolution, and/or
tial findings with another study to fur- reversal learning. The authors also noted
ther dissect the neural mechanisms under- that down-regulation was associated with
lying different component processes of more widespread activation across both left
cognitive change (Ochsner et al., 2004). and right PFC, whereas up-regulation local-
First, subjects were taught to use reap- ized primarily in the left PFC.
praisal strategies using a self-focused (i.e., A number of studies have confirmed and
distancing, decreasing self-relevance of the elaborated on these initial findings on the
situation) or a situation-focused (i.e., pos- neural bases of reappraisal (see also Harenski
itive reframing of the situation) approach & Hamann, 2006; van Reekum et al., 2007).
to decrease negative emotion. Second, Phan and colleagues compared volitional
they were instructed to down- and up- attempts at reappraisal to maintaining a neg-
regulate their negative emotion. The authors ative emotional state (e.g., responding nat-
found that both self- and situation-focused urally) evoked by highly arousing, aversive
strategies similarly reduced subjective nega- images (similar to those used by Ochsner
tive affect, attenuated amygdala responses, and colleagues) and observed greater activity
and engaged overlapping regions in pre- in the dmPFC, DLPFC, lateral OFC, right
frontal and cingulate cortices. When down- VLPFC, and dorsal ACC and less activity in
regulating (but not up-regulating) emo- the extended amygdala, nucleus accumbens,
tion, distancing recruited greater rostral and lateral PFC during reappraisal (Phan
MPFC and ACC activity, areas involved et al., 2005; Figure 16.4a). Moreover, the
in self-referential processing (Kelley et al., authors reported that greater activations of
2002), whereas situation-based reinterpreta- the dorsal ACC and anterior insula were
tion engaged more lateral PFC areas asso- associated with decreasing negative affect,
ciated with maintenance and manipulation whereas greater activation of the amygdala
of alternative meanings of a given event and visual association occipital cortex was
based on current and past experience. Simi- correlated with increasing negative affect.
larly, both up- and down- regulation evoked These findings were followed up with a
activity in the lateral PFC thought to relate functional connectivity analysis that showed
to the maintenance of these strategies in (1) activity in the DLPFC, dmPFC, ACC,
the context of immediate goals, as well as OFC, and inferior parietal cortex (IPC)
dorsal ACC activity that may involve per- covaried with activity in the amygdala in
formance monitoring of the task at hand a task-dependent manner (that is, coupling
(Botvinick, Braver, Barch, Carter, & Cohen, was greater in the reappraisal task than the
2001), while modifying amygdala activity maintenance task; Figure 16.4); and (2) the
and negative affect in concert with the reg- strength of amygdala-OFC and amygdala-
ulatory goal (up-regulation was associated dmPFC connectivity predicted the extent
with greater amygdala reactivity and greater of attenuation of negative affect following
negative affect, whereas down-regulation reappraisal (Banks, Eddy, Angstadt, Nathan,
was associated with less amygdala reactivity & Phan, 2007).
and less negative affect); in general, up- and Using several sources of emotional read-
down-regulation goals engaged very similar, outs as indices of effective emotion regula-
overlapping areas of the prefrontal cortex tion, Eippert and colleagues used fMRI along
thought to reflect the common processes with peripheral psychophysiological record-
shared by both strategies. More engagement ings (startle eyeblink responses [SERs] and
of the left rostral MPFC was related to up- skin conductance responses [SCRs]) while
regulation, interpreted by the authors as subjects reappraised threat-related images in
relating to the retrieval of emotional knowl- order to down- and up-regulate their emo-
edge, and greater activation of right lateral tional responses. Down-regulation was asso-
PFC and OFC was associated with down- ciated with dampened amygdala response,
EMOTION REGULATION 385
DLPFC
A)
VLPFC
B))
along with greater ACC, DLPFC, and OFC (dmPFC, left LPFC, ACC, left OFC) and
activation, whereas up-regulation was asso- attenuated amgydala reactivity when engag-
ciated with greater SER, SCR, and amgydala ing in both up- and down-regulation of
reactivity along with widespread PFC acti- emotional responding of both negative and
vation; the authors noted a significant over- positive pictures. This result further sup-
lap in activation of the PFC associated with ports the notion that, during reappraisal,
both strategies (Eippert et al., 2007). Along a core set of cognitive processes (work-
with new evidence on effects of appraisal ing memory, action monitoring, response
on peripheral physiology, this report also inhibition, reversal learning, and so on)
showed that trial-by-trial indices of reap- are engaged, regardless of the regulatory
praisal success (subjective ratings of affect) goal (i.e., up- or down-regulation) or the
correlated with amgydala reactivity during valence of the reappraisal target (negative
up-regulation and with OFC activity during or positive). Yet in addition to these com-
down-regulation. mon core processes, the authors also point
Until recently, it was unknown whether out some salient differences in brain acti-
the same neural circuitry as that used to reg- vations based on emotion and task. For
ulate negative emotions would be engaged in negative emotions, although brain activa-
reappraising positive emotions. To answer tion in the PFC recruited for up-regulation
this question, Kim and Hamann exposed was not greater than that recruited for
participants to images of negative, posi- down-regulation (negative, up > down), the
tive, and neutral valence and instructed reverse contrast engaged greater bilateral
them to increase or decrease this evoked OFC and other right PFC and right parietal
affect using reappraisal, or to just watch areas (negative, down > up). In contrast,
(Kim & Hamann, 2007). Consistent with the for positive emotions, activity for down-
findings of Ochsner and colleagues (2004), regulation was not greater than that for
this study observed broadly shared activa- up-regulation (positive, down > up). How-
tions in prefrontal and cingulate cortices ever, up-regulation selectively engaged the
386 K. LUAN PHAN, & CHANDRA SEKHAR SRIPADA
ventral medial (orbitofrontal) PFC, addi- between the PFC and subcortical/limbic
tional medial PFC, and left frontal thalamus, regions underlie the mechanism for regu-
and caudate (positive, up > down). lation, such that the PFC reduces negative
These findings were partly replicated in a emotion by influencing the reactivity
subsequent study by Mak, Hu, Zhang, Xiao, of these emotional appraisal/responding
and Lee, (2009) using a design with nega- systems, and (2) a direct path hypothesis
tive and positive pictures, in which subjects that success in negative affect reduction is
were allowed to use a regulation strategy directly due to PFC activation with little
of their own choosing (see Delgado, Gillis, influence on appraisal/responding systems
& Phelps, 2008; Staudinger, Erk, Abler, & (Wager, Davidson, Hughes, Lindquist, &
Walter, 2009; Staudinger, Erk, & Walter, 2011 Ochsner, 2008). First, consistent with prior
for studies of regulation of positive stim- studies, greater activation of the cogni-
uli consisting of monetary rewards). Down- tive change network, including bilateral
regulation of negative and positive emotions DLPFC, VLPFC (encompassing inferior
was associated with increased activation of frontal gyrus), MPFC, anterior PFC, and
the DLPFC and dmPFC/ACC, and the OFC dorsal ACC, and inferior parietal lobe (IPL),
was selectively recruited to regulate nega- was observed during reappraisal. Success of
tive affect. Finally, online (collected while the reappraisal strategy (drop in negative
scanning) ratings of arousal were correlated affect) was associated with activation of
with insula and ventral and dorsal MPFC the bilateral VLPFC, dmPFC, and IPL.
activity specifically during negative emotion Selecting the right VLPFC region as the
regulation, and OFC activity tracked both predictor region and reappraisal success as
positive and negative emotion regulation. the outcome, the authors observed that
As observed previously, amgydala reactiv- reappraisal-induced activation in this region
ity to emotional stimuli was modulated in was associated with activation of two emo-
accord with the regulatory goal. Interest- tion appraising and/or generating regions:
ingly, because the study also employed posi- the amygdala and ventral striatum/nucleus
tive images, the authors were able to demon- accumbens. Moreover, these regions medi-
strate that regulation of positive emotion ated the link between VLPFC activation
was related to changes in ventral striatal and negative affect reduction, although
responses, a region associated with reward in opposite directions – VLPFC increased
and pleasure (see Chapter 19). nucleus accumbens activation, which led
Despite the convergence of evidence to increased reappraisal success, whereas
supporting an apparent reciprocal rela- the VLPFC increased amygdala activation,
tionship between emotion reappraisal which led to reduced reappraisal success.
systems (dmPFC, DLPFC, ACC, OFC, The two paths via the nucleus accumbens
VLPC) and emotion appraisal/generation (less negative emotion) and via the amygdala
regions (amygdala, extended amygdala, (more negative emotion) explained as much
ventral striatum), little is known about as 50% of the variance in subjective negative
how these systems might interact and how affect.
these circuits might mediate the observed In a subsequent analysis in which the
changes in subjective emotional experience authors specified the amygdala and nucleus
(e.g., negative affect). It had been shown accumbens as mediators and performed a
that amygdala-OFC and amygdala-dmPFC whole-brain search for predictors, it was
reappraisal-dependent coupling predicts noted that the amygdala and nucleus accum-
the extent of attenuation of negative affect bens also mediate the relationship between
(Banks, et al., 2007). Using a novel medi- other PFC regions and reappraisal success. In
ation effect parametric mapping (MEPM) particular, the amygdala also mediated the
analysis, Wager and colleagues formally OFC, whereas the nucleus accumbens medi-
tested two competing hypotheses: (1) a ated rostral MPFC, anterior PFC, DLPFC,
mediation hypothesis that interactions and dmPFC. This study, using a novel path
EMOTION REGULATION 387
analytic method, suggests that there are Most studies of reappraisal using fMRI
two pathways (via the nucleus accumbens and PET have used emotionally evoca-
and via the amygdala) by which prefrontal tive images, typically from the validated
activation influences emotion during reap- stimulus set, the International Affective
praisal. Picture System (IAPS; Lang, Bradley, &
It should be noted that not all stud- Cuthbert, 1997); relatively few laboratories
ies on the reappraisal of negative images have employed other affective probes.
have produced the expected pattern of One such study comes from Goldin and
robust PFC activation and amygdala atten- colleagues who instructed participants to
uation. For example, Urry and colleagues reappraise the emotional interpretation of
showed that older (62- to 64-year old) vol- “harsh” (angry, contemptuous) faces (using
unteers using reappraisal strategies engaged linguistic-cognitive self-talk such as “This
the VLPFC, DLPFC, dmPFC and amygdala does not involve me,” “This does not influ-
when asked to enhance (vs. attend) negative ence me,” or “This does not impact me”), in
affect evoked by unpleasant images, con- the context of comparing healthy controls
sistent with previous observations (Ochsner (HC) and patients with social anxiety
et al., 2004). However, the authors observed disorder (SAD), also known as social phobia
few prefrontal or amygdala activation differ- (Goldin, Manber, Hakimi, Canli, & Gross,
ences in comparing decrease versus attend 2009). Although not the primary focus of
conditions (Urry et al., 2006). Instead, the study, the authors did report subjective
they showed that individuals who exhib- ratings and brain activation data from
ited greater attenuation of the amgydala within-group analyses of the healthy par-
by reappraisal (attend > decrease) showed ticipants. Looking at harsh faces produced
greater vmPFC/medial OFC activation and higher levels of subjective negative emotion
a “steeper, more normative” diurnal decline and activation of the bilateral dorsal amyg-
in the stress hormone cortisol. Interest- dala, relative to looking at neutral scenes.
ingly, this pattern was selective for down- When viewing harsh faces, compared
regulation attempts and not observed dur- to the unregulated “look” condition, the
ing enhancement of negative emotion. The reappraise task resulted in reduced negative
study also showed a similar reciprocal rela- affect and greater activation of the bilateral
tionship between the amgydala and dmPFC VLPFC/inferior frontal gyrus and dmPFC
during reappraisal and that vmPFC medi- and less activation of the bilateral insula.
ated this dmPFC-amygdala relationship. A separate study (Goldin, Manber-Ball,
Additional corroborating evidence of Werner, Heimberg, & Gross, 2009) of SAD
emotion regulation comes from measure- vs. HC by the same group used linguistic
ments of pupil reactivity, such that both probes of negative self-beliefs (NSBs; e.g.,
increasing and decreasing negative emotion “No one likes me,” “Others think I’m a fool”,
evoked greater proportional change in pupil- “I am weird”). Participants were trained to
lary diameter. Using this index, the authors reframe these NSBs in a way that reinter-
observed a positive relationship between prets and thus diminishes their negative
the VLPFC/inferior frontal gyrus and pupil content, using linguistic-cognitive self-talk
diameter, whereas its relationship with the thoughts (e.g., “That is not always true”). In
amygdala was negative, during attempts the HC group alone, relative to the unregu-
to decrease negative emotion. Collectively, lated “react” condition, reappraisals of NSBs
these findings support the notion that down- produced less negative emotion, attenuated
regulation of negative affect is associated amgydala reactivity, and recruited activation
with meaningful interindividual differences of the dmFC, DLPFC, and VLPFC. More-
in brain activity, particularly that between over, follow-up analyses of functional con-
the vmPFC and amygdala, and correspond- nectivity during the reappraisal task using
ing change in diurnal variation in cortisol and the amgydala as the seed region showed
pupillary reactivity. the amygdala has an inverse/reciprocal
388 K. LUAN PHAN, & CHANDRA SEKHAR SRIPADA
relationship with the DLPFC and VLPFC, gard, Levesque, & Bourgouin, 2001). Using
but lacks positive connectivity with any erotic films, they evoked in participants sex-
regions in the PFC. Findings from these two ual arousal, arguably a transient emotional
studies complement those using aversive state. Subjects were instructed to “distance
pictures as emotion probes and extend the themselves from these stimuli, that is, to
evidence for VLPFC and dmPFC engage- become a detached observer” in order to vol-
ment during reappraisal of aversive social untarily decrease the intensity of the sex-
signals. ual arousal felt in reaction to the erotic
Very few studies have directly compared films (of note, the authors refer to this emo-
reappraisal against another emotion regu- tion regulatory strategy as “suppression,” but
lation strategy to investigate shared and it would be classified as a form of reap-
nonshared neural substrates. One study praisal according to the typology adopted
by McRae and colleagues contrasted reap- in this chapter). As intended, the partic-
praisal and distraction (McRae et al., 2010), ipants experienced less subjective sexual
and another by Goldin and colleagues arousal during the inhibition task (versus
(discussed later) compared reappraisal and passive view/ “react normally” task). When
suppression (Goldin, McRae, Ramel, & allowing sexual arousal to emerge, greater
Gross, 2008). Relative to looking at aversive responses were observed in the amygdala,
images, reappraisal and cognitive distrac- temporal pole, and hypothalamus, whereas
tion (using a working memory distracter) the distancing task reduced amygdala (as
similarly reduced negative affect and amyg- well as that in the temporal pole and
dala reactivity and similarly increased acti- hypothalamus) responses and engaged acti-
vation in the middle and inferior LPFC, vation of the right anterior MPFC (supe-
dmPFC, dorsal ACC. Interestingly, distrac- rior frontal gyrus) and right ACC, as well
tion was shown to be more effective than as left VLPFC (inferior frontal gyrus) and
reappraisal in reducing amgydala response bilateral superior parietal lobule. A follow-
and selectively engaged LPFC, inferolateral up study from this laboratory also involved
PFC, and superior parietal cortex, whereas distancing, this time in response to sad
reappraisal engaged dmPFC, DLPFC, and film clips, which were confirmed to selec-
VLPFC. Reductions in negative affect were tively evoke sadness and emotion-related
associated with activation of dmPFC and activation of VLPFC, amygdala, insula, and
DLPFC via reappraisal-based strategies but temporal pole (Levesque, et al., 2003).
only with activation of the superior pari- Along with reductions in subjective sad-
etal cortex for distraction-based strategies. ness, distancing was associated with activa-
These findings preliminarily suggest that tion of right-sided OFC, DLPFC, and to
these two approaches taken to regulate emo- a lesser extent, VLPFC. Positive correla-
tions recruit different brain areas and/or tions between self-report ratings of sadness
recruit the same PFC areas to different and OFC and DLPFC activation were also
extents, which is likely explained by the observed, which the authors interpret in
different cognitive processes (i.e., attention terms of the hypothesis that residual sad-
deployment in distraction and reframing or ness that remained during the reappraisal
interpretation change in reappraisal) under- task led to greater activation/work in OFC
lying the strategies. and DLPFC regulatory circuits. Together,
As noted earlier, a distinct and alternative these findings are in agreement with those
reappraisal-related strategy involves distanc- reported by Ochsner and colleagues (2004)
ing, a process of taking the perspective of and suggest that the same neural machinery
a detached and distant observer while pro- employed during distancing is used to down-
cessing emotional content. Beauregard and regulate general negative affect evoked by
colleagues were among the first to study con- aversive pictures and more specific emo-
scious, effortful, voluntary emotion regula- tional states (e.g., sexual arousal, sadness)
tion with functional neuroimaging (Beaure- evoked by films.
EMOTION REGULATION 389
Kalisch and colleagues also trained sub- in the amygdala, dmPFC, and sensory cor-
jects on a distancing strategy, but in this tices responding to the visual salience (e.g.,
case to suppress state anxiety evoked during fusiform gyrus, thalamus). Compared to
anticipation of receiving painful shock stim- looking, regulation by distancing from neg-
uli (Kalisch, Wiech, Herrmann, & Dolan, ative (>neutral) images was associated with
2006). The authors instructed the subjects less negative emotion scores, greater activity
to distance by imagining themselves to be in dorsal ACC/dmPFC, VLPFC including
in a “Special Place” of their choosing, and inferior frontal gyrus, anterior vmPFC, and
in particular to think to themselves “I am superior temporal gyrus, as well as less activ-
here in my [“Special Place”], feeling safe and ity in the amygdala. These findings corrob-
comfortable. Those emotions out there can- orate those previously observed by Ochsner
not reach me here. Nothing bothers me.” and colleagues (2004), who also trained sub-
In the control condition, subjects were told jects to use distancing to regulate negative
to imagine themselves in the actual labora- affect evoked by both social and nonsocial
tory surroundings at Queens Square while content; the findings also suggest that dis-
thinking “I am now in this experiment at tancing from aversive social cues engages a
Queens Square. I can clearly feel my emo- sociocognitive network of regions associated
tions. They affect my body and my mind.” with social perception, perspective-taking,
In a behavioral study, the authors confirmed and attentional control.
that the distancing task reduced subjective In sum, distancing appears to engage
anxiety, heart rate, and SCRs. In a follow- a PFC network (ACC, dmPFC, VLPFC,
up fMRI experiment, subjects who engaged DLPFC, OFC) similar to that recruited
in distancing during pain anticipation also by situation-focused reinterpretation (e.g.,
reported reduced anxiety, reduced pain- reframing negative content to neutral or
related heart rate acceleration response and positive terms). Recently, Erk and col-
pain anticipation-related MPFC/ACC acti- leagues showed that distancing has long-
vation, and displayed enhanced activation term effects on episodic memory and subse-
in anterolateral PFC (located at the junc- quent brain activation patterns when tested
tion between DLPFC and VLPFC). This one year later (Erk, von Kalckreuth, &
study extended prior findings by demon- Walter, 2010). Here the authors showed that
strating that detachment produced an anxi- successful recognition of negative images
olytic effect using subjective, physiological, that were previously subjected to distancing
and neural indices of pain, and also identi- was positively associated with DLPFC acti-
fied a discrete source in anterior lateral PFC vation, whereas the opposite relationship
that is associated with these effects. was observed with the amygdala – greater
In another study, Koenigsberg and col- activation of amygdala during distancing-
leagues were particularly interested in the mediated down-regulation of negative affect
neural correlates of distancing from nega- was associated with less memory for these
tive emotion elicited by pictures with social items.
content (e.g., pictures of situations involv- Turning now to a strategy that is closely
ing loss or grief, abuse, or physical threat) related to distancing and reinterpretation,
rather than nonsocial content (e.g., fearsome Kross and colleagues first evoked negative
animals, weapons; Koenigsberg, et al., 2010). affect by asking participants to recall a series
Here, subjects were trained to distance of highly arousing negative autobiograph-
themselves from the emotional material by ical memories (Kross, Davidson, Weber,
relating to the emotionally aversive images & Ochsner, 2009); they then trained sub-
as though they themselves were not person- jects to engage in emotion regulatory strate-
ally connected in any way to the pictured gies such as accept (“to recognize that the
individuals in the depicted unpleasant situa- feelings they experienced during recollec-
tion. As expected, relative to neutral images, tion were passing mental events that were
socially aversive pictures elicited activation psychologically distant from the self and
390 K. LUAN PHAN, & CHANDRA SEKHAR SRIPADA
did not control them”) and analyze (“to scores from the emotion regulation ques-
objectively analyze the causes and reasons tionnaire (ERQ; Gross & John, 2003).
underlying their feelings”). The authors did The preceding studies of antecedent-
not observe that these strategies activated focused emotion regulation differed widely
PFC areas more so than the control condi- in the duration of regulation that subjects
tion of feel (“to focus on the specific feel- engaged in during each regulation trial,
ings that naturally flowed through their with the duration ranging 5 to 25 sec-
mind as they thought about their recalled onds. Using activation likelihood estima-
experiences”). Instead, the feel condition, tion (ALE) meta-analytic methods, Rafael
previously associated with pathological Kalisch investigated the role of duration
rumination and dysfunctional coping, was of regulation in explaining variability in
associated with the greatest activity in the brain activations in studies of regulation of
rostral, anterior MPFC, VLPFC, and sub- negative stimuli (Kalisch, 2009). He found
genual ACC, and activity in this “rumi- that, as the duration of regulation increased,
nation” network was positively correlated frontal cortical activation reliably shifted
with the participants‘ self-reported negative both from left to right and from posterior
affect. Here, unlike many of the other stud- to anterior. Kalisch proposed that this shift
ies of cognitive strategies, the analyze condi- reflects two dissociable stages of emotion
tion may have been less effective in reduc- regulation, an early implementation stage
ing negative affect, as indicated by greater during which the specific regulation strategy
engagement of regions in the “rumination” is selected, and a later maintenance phase
network in the analyze condition compared during which the selected strategy is con-
to the accept condition. These findings tinued and monitored for success. Kalisch’s
provide a neural model for understanding quantitative review sheds important light on
the maladaptive recall of negative memo- the temporal dynamics of emotion regula-
ries and are thus potentially highly relevant tion and underscores the need for further
to understanding certain psychopathologies investigation into this understudied topic.
(e.g., major depression, generalized anx-
iety). However, the model has not yet
Response Change: Suppression and
been formally tested in clinical popula-
Extinction
tions.
A relatively unique antecedent-focused Suppression is a “response modulation”
strategy was investigated by Herwig and col- strategy, and thus it takes place after
leagues. They instructed subjects to perform the perception and appraisal stages are
“reality checking” while anticipating images completed and the emotional response
of negative valence in order to reduce their has already begun to unfold (Figure 16,1;
negative affect; subjects were told to repeat- Gross, 1998; Gross & Levenson, 1993; Gross
edly reconsider the context of their actual & Thompson, 2007). Suppression involves
situation by, e.g., thinking: “I am lying in fairly direct attempts to influence cognitive,
a scanner,” “They will show me a picture, physiological, or behavioral manifesta-
this is part of the study” (Herwig, et al., tions of emotion responses; for example,
2007). Compared to a control group of sub- attempts to inhibit the facial expressions
jects who did not engage in this reality- and other physical displays associated with
checking strategy, subjects who took on a emotions. Unlike cognitive reappraisal,
realistic stance to the experimental context suppression results in little or no change
showed higher MPFC and DLPFC activa- in subjective negative emotion, and it
tion while attenuating activity in extended is associated with increased sympathetic
amygdala and sensory areas (e.g., fusiform activation and reduced emotional memory
gyrus). Moreover, the extent to which the (Gross, 1998, 2002; Gross & Levenson, 1997).
amgydala activated during reality checking In addition to suppression, extinction is
was correlated negatively with reappraisal another process that reduces the expression
EMOTION REGULATION 391
of negative emotional responses, but it dicted from prior studies, reappraisal atten-
specifically applies to responses that have uated reactivity in the insula and amgydala
previously been acquired through stimulus- and engaged MPFC, DLPFC, VLPFC, lat-
reinforced/conditioned associations. Al- eral OFC, and inferior frontal gyrus. Inter-
though extinction is not directly considered estingly, analyses of the temporal dynam-
in contemporary models of “response mod- ics of these PFC regions showed that
ulation” as proposed by Gross (Gross, 2002; their activation emerged early (within 5 s
Gross & Thompson, 2007), because extinc- of the film clip starting). Suppression did
tion is increasingly recognized as involving not modify amygdala reactivity, but in fact,
active learning and inhibitory processing increased insula reactivity. It also increased
(Milad, Rauch, Pitman, & Quirk, 2006), it activity in VLPFC, dmPFC, and DLPFC, but
is potentially usefully thought of as a more in a later phase of emotional responding (last
automatic, less effortful kind of emotion 5 s of the film clip). Although both strate-
regulation strategy (Ochsner & Gross, 2005). gies reduced subjective negative affect and
extent of facial disgust expressions, reap-
SUPPRESSION praisal was more effective in reducing neg-
Ohira and colleagues examined the neu- ative affect, whereas suppression was more
ral correlates of suppression using PET – effective in reducing facial expressions. At
instructing participants “to voluntarily the brain level, reappraisal was more effec-
suppress any emotional responses” during tive than suppression in reducing amygdala
viewing of emotionally negative, positive, and insula responses to negative emotion,
and neutral images; it should be noted perhaps because reappraisal engaged PFC at
that the authors did not expand on this an earlier phase of responding than suppres-
instruction (suppress expression or experi- sion. Resonant with prior findings on sub-
ence), nor did they provide specifics about jective and autonomic psychophysiological
which strategies were used by subjects to change associated with suppression (Gross,
implement suppression, except that all sub- 1998), these results demonstrate that differ-
jects denied using cognitive distraction or ent strategies for regulation, reappraisal vs.
verbal processing of stimuli (Ohira et al., suppression, are differentially effective in
2006). Emotion suppression, regardless of their ability to affect emotional experience,
the valence of the emotional response, did expressive behavior, and brain responses.
not result in a change in negative affect, Moreover, the different temporal patterns of
heart rate, or the stress-related adrenocorti- prefrontal engagement associated with sup-
cotropic hormone (ACTH), but did increase pression versus reappraisal (i.e., significantly
SCRs and activity in bilateral OFC, rostral- later engagement with suppression), provide
ventral ACC, superior parietal gyrus, and a potential brain-based mechanism for the
lateral PFC. Moreover, activation in the reduced efficacy of suppression.
medial OFC was positively correlated with
SCR amplitude during suppression. The FEAR LEARNING AND EXTINCTION
unregulated “attend” condition, in contrast, We now turn to the emerging imaging neu-
elicited greater amygdala activity. roscience of fear learning and extinction (see
In addition to the study by McRae et al. Chapter 18). Negative emotions, particularly
discussed earlier, only one other study has fear, can be acquired by Pavlovian learn-
directly compared two emotion regulation ing, in which a previously neutral stimulus
strategies. Goldin and colleagues exam- such as a tone (the conditioned stimulus,
ined commonalities and differences in the CS) acquires emotional importance when
use of reappraisal and suppression (sub- paired with an aversive stimulus (Quirk,
jects were instructed to “keep their face Garcia, & Gonzalez-Lima, 2006) such as
still” [expression suppression]) to regulate a footshock (the unconditioned stimulus,
emotional responses to negative/aversive US). After a few such pairings, the presenta-
film clips (Goldin, et al., 2008). As pre- tion of the CS alone is capable of eliciting the
392 K. LUAN PHAN, & CHANDRA SEKHAR SRIPADA
conditioned fear response (CR). In extinc- the neural mechanisms by which we voli-
tion, the CR degrades over time when the tionally and cognitively regulate and more
CS is presented repeatedly without being effortlessly inhibit fear.
paired with the US. The success of extinc-
tion learning can be later tested and con-
firmed (typically the next day) by once again Other Strategies: Mental State
presenting the CS and observing a dimin- Change, Meditation
ished conditioned response.
Current models propose that during A number of other strategies exist that may
extinction, the original CS-US association not easily fit into the emotion regulation
is not erased or overwritten. Rather, extinc- classification scheme (perception change,
tion involves active learning of a new CS-US interpretation change, response change)
relationship, along with inhibition of the old developed in this chapter. These strategies
CS-US relationship that remains encoded, may span multiple phases of emotion (per-
but its manifestation is suppressed (Bouton, ception, experience, and response) across
2004). Extinction can thus usefully be con- self- versus situation-focused stances. Alter-
ceptualized as a form of emotion regulation natively, they may be difficult to define
in that conditioned fear responses are sup- or relate to a particular cognitive construct
pressed with this newly acquired learning. (e.g., attention, appraisal, and so on).
Of the various emotion strategies deployed One prime example is the family of
by animals (e.g., active coping, reconsoli- strategies that evoke enhanced self-awareness:
dation), extinction learning has been most so-called mindfulness or other meditation
often translated to human studies. strategies. To understand how mindfulness
In a elegant study that examines inter- might work as an emotion regulation strat-
connections between traditional strategies egy, its useful to revisit previous neuroimag-
for cognitive emotion regulation and fear ing research that showed that the seem-
extinction, Delgado and colleagues used a ingly simple act of labeling the emotional
partial reinforcement procedure to instill impact of visually salient stimuli changes
conditioned fear in subjects (Delgado, Near- their effects on subjective and brain (e.g.,
ing, Ledoux, & Phelps, 2008). Subjects then amgydala and paralimbic) responses. For
used volitional emotion regulation tech- example, when looking at an aversive stim-
niques (relaxation and distraction [“try to ulus, we can view, process, and experience
think of something calming in nature”]) in the unpleasantness of its content passively,
response to fear-conditioned stimuli, and or we can direct our attention toward lin-
results were compared with this group’s guistically labeling our subjective emotional
prior neuroimaging study of fear extinc- response by asking “how unpleasant does
tion. Cognitive regulation of fear responses this picture make me feel?” This procedure
engaged lateral PFC regions, which modu- of cognitive labeling of evocative stimuli and
lated amygdala activity via vmPFC. Acti- shifting attention to one’s own mental state
vations in vmPFC and the amygdala asso- (self-awareness) represents a mental process
ciated with volitional regulation exhibited (mixed with cognitive, experiential, linguis-
substantial overlap with similar regions iden- tic components) that could allow subjects to
tified in their prior study of extinction learn- regulate their experience of the emotionally
ing and retrieval. The authors propose that salient stimuli. It is noteworthy that rating
volitional regulation strategies involve lat- the intensity of an anxiety-producing situa-
eral PFC regions that interact with and har- tion is a mainstay of behavioral therapy that
ness a phylogenetically older emotion regu- permits patients to gain a feeling of control
lation circuit linking vmPFC and amygdala over their anxiety (Marks, 1985).
(see also Schiller & Delgado, 2010). Given In one of the first imaging studies of
the paucity of data in this area, more studies this phenomenon, Taylor and colleagues
are needed to directly compare and contrast showed that switching from passively
EMOTION REGULATION 393
MPFC, mid cingulate cortex; Herwig, et al., the existing studies add to a growing liter-
2010). ature regarding the role of enhancing self-
Most recently, Modinos and colleagues awareness, meditation, and mindfulness as
directly examined the relationship between a distinct form of emotion regulation from
dispositional mindfulness and brain activa- interpretation change (e.g., cognitive reap-
tion during the reappraisal of negative emo- praisal), although whether they recruit dif-
tion. As with prior studies on reappraisal, ferential specialized brain circuitry remains
the authors observed that reinterpretation an unanswered question.
(versus viewing) negative images engaged
DLPFC, dmPFC, IFG including the VLPFC,
and dorsal ACC and attenuated amygdala Problems and Future Directions
reactivity; moreover they reported that acti-
Temporal Dynamics – Use of
vation of bilateral dorsal PFC, particularly
ERP/EEG
the dmPFC, predicted reappraisal success
(Modinos, Ormel, & Aleman, 2010). Cor- Although emotion and its regulation are
relation analyses of several PFC regions of thought of as processes that unfold dynam-
interest (ROIs) showed that dmPFC activity ically over time, most measures of central
during reappraisal was positively correlated physiology such as functional MRI are lim-
with individual differences in mindfulness ited by their temporal resolution (over sev-
traits as measured by the Kentucky Inven- eral seconds) and thus have low sensitivity
tory of Mindfulness Skills (KIMS) and nega- to detect neural processes that occur within
tively correlated with amygdala response to a 2- to 3-second window (see Chapter 5).
negative emotion. In support, Farb and col- Methodologies such as event-related poten-
leagues showed that individuals with mod- tials (ERPs) using electroencephalography
erate levels of depression and anxiety who (EEG) have much better temporal resolu-
underwent 8 weeks of mindfulness train- tion (on the order of milliseconds; see Chap-
ing using mindfulness-based stress reduc- ter 4). Using these methods, researchers
tion had reduced symptoms posttraining and have identified several components related
exhibited more activation of insula, lateral to the automatic and controlled process-
PFC, and subgenual ACC while watching ing of emotional information (e.g., positive
sadness-inducing film clips (Farb et al., 2010). potential at 300 ms [P300], and late posi-
These findings lend support to the prior find- tive potential [LPP] between 300–1500 ms)
ings from this group on the neural effects of that could be used to provide more detailed
mindfulness meditation training (Farb et al., information on the temporal dynamics of
2007). emotion regulation (Hajcak, MacNamara, &
How does meditation training work? Olvet, 2010). However, there are no pub-
One possibility is that meditation training lished reports that link ERP and fMRI data
enhances control over focused and sustained in the same subjects or that combine simul-
attention, enhancing the individual’s abil- taneous acquisition of ERP and fMRI data
ity to engage frontal networks to reduce during emotion regulation.
distraction and better focus (Lutz et al.,
2009). Compared to untrained individuals
Emotion Regulation in
or novices, expert meditation practitioners
Psychopathologies
appear to have a greater ability to detect
emotional sounds and, in doing so, are able The dysregulation of emotion is a hallmark
to tap into neural circuitry involved in empa- of psychiatric disorders, including mood dis-
thy and theory of mind (Lutz, Brefczynski- orders (e.g., major depression and bipo-
Lewis, Johnstone, & Davidson, 2008). That lar disorder), anxiety disorders, and person-
said, the exact psychological mechanisms by ality disorders (e.g., borderline personality
which mindfullness exerts its emotion regu- disorder), suggesting that deficits in emo-
latory effects on negative affect and on brain tion regulation play a prominent role in the
activation require further study. Together pathophysiology of these disorders (Aldao,
EMOTION REGULATION 395
Nolen-Hoeksema, & Schweizer, 2010). Neu- tioning, and psychiatric disease. Research
roimaging studies are increasingly uncover- over the last decade has elucidated the func-
ing the brain basis of abnormalities in emo- tional neuroanatomy of the regulation of
tion regulation in psychiatric conditions. emotions, focusing on cognitive strategies
Major depression has been studied most aimed at attention modification and inter-
extensively in relation to emotion regula- pretation change. It has implicated a broad
tion. In an early study, Beauregard and col- network involving the prefrontal cortex
leagues showed that depressed individuals (including DLPFC, VLPFC, dmPFC, ACC,
(compared to controls) report greater sub- OFC) in the successful regulation of emo-
jective difficulty distancing themselves from tions. Moreover, recent studies have started
sad films, and the degree of reported diffi- to inform us about complex interrelation-
culty correlated with activation in a medial ships among emotion regulation, personal-
PFC region that this group had previously ity, genetics, and psychopathology. Much
linked to the regulation of emotion (Beau- has been learned, but many more questions
regard, Paquette, & Levesque, 2006). Using remained unanswered. Moreover, existing
pupillary dilation as an index of effort, research has yet to be connected with trans-
Johnstone and colleagues confirmed that lational approaches that link evolving theory
depressed individuals expend more effort and findings with fMRI-to-bedside strate-
in regulating emotion. They also showed gies to identify individuals vulnerable to
that a regulation circuit present in con- psychopathology and to refine existing and
trol subjects that links VLPFC and vmPFC innovate new clinical interventions. Ongo-
in the suppression of amygdala activity ing and new approaches taken by the human
is not present in depressed patients, who affective neuroscience of emotion regula-
instead show a positive relationship between tion promise to be one of the most inter-
the amygdala and vmPFC (Johnstone, van esting and impactful avenues of inquiry
Reekum, Urry, Kalin, & Davidson, 2007). in basic and translational neuroscience for
A very similar pattern of hyperactivation in many years to come.
emotion generation regions (e.g., amygdala,
insula), and hypoactivation in emotion regu-
lation regions (DLPFC, dACC, dmPFC) has Outstanding Questions
been reported in other psychiatric disorders,
including social anxiety disorder (Goldin, r A growing number of studies point to a
Manber, et al., 2009; Goldin, Manber-Ball, broad prefrontal regulatory circuit that is
et al., 2009), borderline personality dis- engaged across emotion regulation strate-
order (Koenigsberg et al., 2009; Schulze gies, though very few have directly exam-
et al., 2011), and posttraumatic stress dis- ined overlapping and discrete activations
order (New et al., 2009). Further work is between different strategies head to head
required to clarify whether this pattern of (Goldin et al., 2008; McRae, et al., 2010;
results across disorders represents a unitary Ochsner, et al., 2004). Do these find-
deficit in emotion regulation that spans mul- ings imply that a common brain mecha-
tiple disorders or whether there are more nism is shared among the various emotion
subtle emotion regulation deficits that are regulation strategies, or might there be
relatively specific to each of these disorders more subtle differences that distinguish
that await further clarification. one from strategy from another at the
neural level?
r Growing evidence suggests that phar-
Conclusion macological strategies, particularly those
using empirically validated anxiolytic,
Emotion regulation is a distinctively human antidepressant, and/or mood-altering
capacity that is increasingly recognized compounds, can regulate amgydala reac-
as critical to understanding psychological tivity to negative emotional stimuli (e.g.,
health and well-being, interpersonal func- Paulus, Feinstein, Castillo, Simmons, &
396 K. LUAN PHAN, & CHANDRA SEKHAR SRIPADA
Stein, 2005; Phan et al., 2008), much like OFC: orbitofrontal cortex
cognitive strategies. Do pharmacological R: Right
and cognitive strategies for emotion regu-
lation share common prefrontal circuitry?
r Emotion regulation is a core aspect References
of empirically validated cognitive and
behavioral therapies for psychiatric dis- Adolphs, R. (2002). Neural systems for recogniz-
ing emotion. Current Opinions in Neurobiol-
orders. Moreover, neuroimaging is being
ogy, 12(2), 169–77.
used to study brain mechanisms of the
Aldao, A., Nolen-Hoeksema, S., & Schweizer,
effects and response to treatments for S. (2010). Emotion-regulation strategies across
psychiatric disorders, including psychoso- psychopathology: A meta-analytic review.
cial interventions. However, no study Clinical Psychology Review, 30(2), 217–37.
to date has directly examined emotion Anderson, A. K., Christoff, K., Panitz, D., De
regulation strategies in the context of Rosa, E., & Gabrieli, J. D. (2003). Neural cor-
psychotherapy clinical trials using func- relates of the automatic processing of threat
tional brain imaging. Does cognitive- facial signals. Journal of Neuroscience, 23(13),
behavioral therapy directly modify the 5627–33.
neural substrates subserving emotion reg- Banks, S. J., Eddy, K. T., Angstadt, M., Nathan,
P. J., & Phan, K. L. (2007). Amygdala–
ulation? Can we use what we know
frontal connectivity during emotion regula-
about the brain mechanisms that under-
tion. Social, Cognitive, & Affective Neuro-
lie different cognitive (attention modifi- science, 2, 303–12.
cation, reappraisal) and behavioral (sup- Bantick, S. J., Wise, R. G., Ploghaus, A., Clare,
pression) strategies to refine interventions S., Smith, S. M., & Tracey, I. (2002). Imag-
and improve the likelihood of their suc- ing how attention modulates pain in humans
cess? using functional MRI. Brain, 125(Pt. 2), 310–19.
r Exciting methodological advances in Beauregard, M., Levesque, J., & Bourgouin, P.
fMRI have allowed for the opportunity (2001). Neural correlates of conscious self-
to measure brain activation in real time regulation of emotion. Journal of Neuroscience,
(e.g., Phan et al., 2004; Posse, et al., 2003), 21(18), RC165.
Beauregard, M., Paquette, V., & Levesque, J.
which opens the possibility that brain sig-
(2006). Dysfunction in the neural circuitry of
nals can be used as a source of “neuro-
emotional self-regulation in major depressive
feedback” to guide the training or retrain- disorder. Neuroreport, 17(8), 843–46.
ing of emotion regulation circuits. Given Beck, A. T. (1963). Thinking and depression.
evidence of substantial individual differ- I. Idiosyncratic content and cognitive distor-
ences in emotion regulation, especially in tions. Archives of General Psychiatry, 9, 324–33.
relation to maladaptive personality styles Blair, K. S., Smith, B. W., Mitchell, D. G., Mor-
and psychiatric disease, fMRI-based real- ton, J., Vythilingam, M., Pessoa, L., et al.
time neurofeedback provides a promis- (2007). Modulation of emotion by cognition
ing avenue for helping people change and cognition by emotion. Neuroimage, 35(1),
their characteristic styles of emotion reg- 430–40.
Botvinick, M. M., Braver, T. S., Barch, D. M.,
ulation. Can real-time fMRI reveal the
Carter, C. S., & Cohen, J. D. (2001). Conflict
dynamic mechanisms by which we reg-
monitoring and cognitive control. Psychologi-
ulate emotion regulation? cal Review, 108(3), 624–52.
Bouton, M. E. (2004). Context and behavioral
processes in extinction. Learning & Memory,
Abbreviations 11(5), 485–494.
Bowlby, J. (1969). Attachment and loss: Attach-
ACC: anterior cingulate cortex ment. New York: Basic Books.
DLPFC: dorsolateral prefrontal cortex Campos, J. J., Frankel, C. B., & Camras, L. (2004).
dmPFC: dorsomedial prefrontal cortex On the nature of emotion regulation. Child
VLPFC: ventrolateral prefrontal cortex Development, 75(2), 377–94.
EMOTION REGULATION 397
Carter, C. S., Botvinick, M. M., & Cohen, J. D. reference. Social, Cognitive, & Affective Neuro-
(1999). The contribution of the anterior cingu- science, 2(4), 313–22.
late cortex to executive processes in cognition. Fox, H. C., Hong, K. A., & Sinha, R. (2008).
Reviews in the Neurosciences, 10(1), 49–57. Difficulties in emotion regulation and impulse
Cooney, R. E., Joormann, J., Atlas, L. Y., Eugene, control in recently abstinent alcoholics com-
F., & Gotlib, I. H. (2007). Remembering the pared with social drinkers. Addictive Behav-
good times: Neural correlates of affect regula- iors, 33(2), 388–94.
tion. Neuroreport, 18(17), 1771–74. Frankenstein, U. N., Richter, W., McIntyre, M.
Craig, A. D. (2009). How do you feel – now? The C., & Remy, F. (2001). Distraction modulates
anterior insula and human awareness. Nature anterior cingulate gyrus activations during the
Reviews Neuroscience, 10(1), 59–70. cold pressor test. Neuroimage, 14(4), 827–36.
Creswell, J. D., Way, B. M., Eisenberger, N. I., & Freud, S. (1946). The ego and the mechanisms of
Lieberman, M. D. (2007). Neural correlates of defense. New York: International Universities
dispositional mindfulness during affect label- Press.
ing. Psychosomatic Medicine, 69(6), 560–65. Fusar-Poli, P., Placentino, A., Carletti, F., Landi,
Critchley, H., Daly, E., Phillips, M., Brammer, P., Allen, P., Surguladze, S., et al. (2009).
M., Bullmore, E., Williams, S., et al. (2000). Functional atlas of emotional faces process-
Explicit and implicit neural mechanisms for ing: A voxel-based meta-analysis of 105 func-
processing of social information from facial tional magnetic resonance imaging studies.
expressions: A functional magnetic resonance Journal of Psychiatry and Neuroscience, 34(6),
imaging study. Human Brain Mapping, 9(2), 418–32.
93–105. Gillanders, S., Wild, M., Deighan, C., & Gillan-
Delgado, M. R., Gillis, M. M., & Phelps, E. A. ders, D. (2008). Emotion regulation, affect,
(2008). Regulating the expectation of reward psychosocial functioning, and well-being in
via cognitive strategies. Nature Neuroscience, hemodialysis patients. American Journal of
11(8), 880–81. Kidney Disease, 51(4), 651–62.
Delgado, M. R., Nearing, K. I., Ledoux, J. E., & Goldin, P. R., Manber, T., Hakimi, S., Canli,
Phelps, E. A. (2008). Neural circuitry underly- T., & Gross, J. J. (2009). Neural bases of
ing the regulation of conditioned fear and its social anxiety disorder: Emotional reactivity
relation to extinction. Neuron, 59(5), 829–38. and cognitive regulation during social and
Drabant, E. M., McRae, K., Manuck, S. B., Hariri, physical threat. Archives of General Psychia-
A. R., & Gross, J. J. (2009). Individual differ- try, 66(2), 170–80.
ences in typical reappraisal use predict amyg- Goldin, P. R., Manber-Ball, T., Werner, K.,
dala and prefrontal responses. Biological Psy- Heimberg, R., & Gross, J. J. (2009). Neural
chiatry, 65(5), 367–73. mechanisms of cognitive reappraisal of nega-
Eippert, F., Veit, R., Weiskopf, N., Erb, M., Bir- tive self-beliefs in social anxiety disorder. Bio-
baumer, N., & Anders, S. (2007). Regulation of logical Psychiatry, 66(12), 1091–99.
emotional responses elicited by threat-related Goldin, P. R., McRae, K., Ramel, W., & Gross,
stimuli. Human Brain Mapping, 28(5), 409–23. J. J. (2008). The neural bases of emotion reg-
Erk, S., Abler, B., & Walter, H. (2006). Cognitive ulation: Reappraisal and suppression of nega-
modulation of emotion anticipation. European tive emotion. Biological Psychiatry, 63(6), 577–
Journal of Neuroscience, 24(4), 1227–36. 86.
Erk, S., von Kalckreuth, A., & Walter, H. (2010). Gross, J. J. (1998). Antecedent- and response-
Neural long-term effects of emotion regula- focused emotion regulation: Divergent con-
tion on episodic memory processes. Neuropsy- sequences for experience, expression, and
chologia, 48(4), 989–96. physiology. Journal of Personality and Social
Farb, N. A., Anderson, A. K., Mayberg, H., Psychology, 74(1), 224–37.
Bean, J., McKeon, D., & Segal, Z. V. (2010). Gross, J. J. (1999). Emotion regulation: Past,
Minding one’s emotions: Mindfulness training present, future. Cognition and Emotion, 13(5),
alters the neural expression of sadness. Emo- 551–73.
tion, 10(1), 25–33. Gross, J. J. (2002). Emotion regulation: Affec-
Farb, N. A., Segal, Z. V., Mayberg, H., Bean, tive, cognitive, and social consequences. Psy-
J., McKeon, D., Fatima, Z., et al. (2007). chophysiology, 39(3), 281–91.
Attending to the present: mindfulness med- Gross, J. J., & John, O. P. (2003). Individ-
itation reveals distinct neural modes of self- ual differences in two emotion regulation
398 K. LUAN PHAN, & CHANDRA SEKHAR SRIPADA
processes: Implications for affect, relation- Kalisch, R., Wiech, K., Critchley, H. D., & Dolan,
ships, and well-being Journal of Personality and R. J. (2006). Levels of appraisal: A medial
Social Psychology, 85(2), 348–62. prefrontal role in high-level appraisal of emo-
Gross, J. J., & Levenson, R. W. (1993). Emo- tional material. Neuroimage, 30(4), 1458–66.
tional suppression: Physiology, self-report, Kalisch, R., Wiech, K., Herrmann, K., &
and expressive behavior. Journal of Personal- Dolan, R. J. (2006). Neural correlates of self-
ity and Social Psychology, 64(6), 970–86. distraction from anxiety and a process model
Gross, J. J., & Levenson, R. W. (1997). Hiding of cognitive emotion regulation. Journal of
feelings: The acute effects of inhibiting nega- Cognitive Neuroscience, 18(8), 1266–76.
tive and positive emotion. Journal of Abnormal Kelley, W. M., Macrae, C. N., Wyland, C. L.,
Psychology, 106(1), 95–103. Caglar, S., Inati, S., & Heatherton, T. F. (2002).
Gross, J. J., & Thompson, R. A. (2007). Emotion Finding the self? An event-related fMRI study.
regulation: Conceptual foundations. In J. J. Journal of Cognitive Neuroscience, 14(5), 785–
Gross (Ed.), Handbook of emotion regulation 94.
(pp. 3–24). New York: Guilford Press. Kim, S. H., & Hamann, S. (2007). Neural corre-
Gur, R. C., Schroeder, L., Turner, T., McGrath, lates of positive and negative emotion regula-
C., Chan, R. M., Turetsky, B. I., et al. (2002). tion. Journal of Cognitive Neuroscience,, 19(5),
Brain activation during facial emotion process- 776–98.
ing. Neuroimage, 16(3 Pt. 1), 651–62. Koenigsberg, H. W., Fan, J., Ochsner, K. N., Liu,
Hajcak, G., MacNamara, A., & Olvet, D. M. X., Guise, K., Pizzarello, S., et al. (2010). Neu-
(2010). Event-related potentials, emotion, ral correlates of using distancing to regulate
and emotion regulation: An integrative emotional responses to social situations. Neu-
review. Developmental Neuropsychology, 35(2), ropsychologia, 48(6), 1813–22.
129–55. Koenigsberg, H. W., Fan, J., Ochsner, K. N., Liu,
Harenski, C. L., & Hamann, S. (2006). Neu- X., Guise, K. G., Pizzarello, S., et al. (2009).
ral correlates of regulating negative emotions Neural correlates of the use of psychological
related to moral violations. Neuroimage, 30(1), distancing to regulate responses to negative
313–24. social cues: A study of patients with border-
Hariri, A. R., Bookheimer, S. Y., & Mazziotta, line personality disorder. Biological Psychiatry,
J. C. (2000). Modulating emotional responses: 66(9), 854–63.
Effects of a neocortical network on the limbic Kompus, K., Hugdahl, K., Ohman, A., Mark-
system. Neuroreport, 11(1), 43–48. lund, P., & Nyberg, L. (2009). Distinct con-
Herwig, U., Baumgartner, T., Kaffenberger, T., trol networks for cognition and emotion in the
Bruhl, A., Kottlow, M., Schreiter-Gasser, U., prefrontal cortex. Neuroscience Letters, 467(2),
et al. (2007). Modulation of anticipatory emo- 76–80.
tion and perception processing by cognitive Kross, E., Davidson, M., Weber, J., & Ochsner, K.
control. Neuroimage, 37(2), 652–62. (2009). Coping with emotions past: The neu-
Herwig, U., Kaffenberger, T., Jancke, L., & Bruhl, ral bases of regulating affect associated with
A. B. (2010). Self-related awareness and emo- negative autobiographical memories. Biologi-
tion regulation. Neuroimage, 50(2), 734–41. cal Psychiatry, 65(5), 361–66.
Hutcherson, C. A., Goldin, P. R., Ochsner, Lang, P. J., Bradley, M. M., & Cuthbert, B. N.
K. N., Gabrieli, J. D., Barrett, L. F., & (1997). International Affective Picture System
Gross, J. J. (2005). Attention and emotion: (IAPS): Technical manual and affective rat-
Does rating emotion alter neural responses ings. Gainesville, FL: NIMH Center for the
to amusing and sad films? Neuroimage, 27(3), Study of Emotion and Attention, University
656–68. of Florida.
Johnstone, T., van Reekum, C. M., Urry, H. L., Lazarus, R. S. (1966). Psychological stress and the
Kalin, N. H., & Davidson, R. J. (2007). Failure coping process. New York: McGraw Hill.
to regulate: Counterproductive recruitment Levenson, R. W. (1994). Human emotions: A
of top-down prefrontal-subcortical circuitry functional view. In P. Ekman & R. J. David-
in major depression. Journal of Neuroscience, son (Eds.), The nature of emotion. (pp. 123–126).
27(33), 8877–84. New York: Oxford University Press.
Kalisch, R. (2009). The functional neuroanatomy Levesque, J., Eugene, F., Joanette, Y., Paque-
of reappraisal: Time matters. Neuroscience and tte, V., Mensour, B., Beaudoin, G., et al.
Biobehavioral Review, 33(8), 1215–26. (2003). Neural circuitry underlying voluntary
EMOTION REGULATION 399
John P. O’Doherty
Value-based decision making can be defined the contribution of specific neural circuits
as the process of selecting actions from to these functions. I focus on the situa-
among several alternatives to maximize tion where the relevant information to make
future possible rewards and minimize future decisions must be learned through trial-and-
possible punishers. Underlying this capacity error experience, in the presence of con-
are a number of distinct neural signals. The siderable uncertainty about the underlying
first signal is the “experienced utility” or out- decision variables, rather than the situation
come value. It is essentially a neural repre- where complete information about all the
sentation of the subjective affective response decision variables is provided using explicit
engendered by a stimulus as it is consumed description. The former scenario is perhaps
or experienced by an individual. Another the most realistic framework for studying
key signal is the chosen value, which codes decision making in the real world where
for the expected utility of a particular deci- all the relevant information about partic-
sion option after that option has been cho- ular decision options must be inferred on
sen. Another key signal is the decision value the basis of prior experience, as opposed to
and/or action value – the value assigned to being provided up front. Knowledge about
particular decision options or possible paths decision-making mechanisms and reward
of action based on the expected utility that outcome processing has major implications
would occur if and only if those decision for several domains beyond neuroscience,
options are selected. Such decision value including the economy, addiction, and social
signals are used as inputs into the decision behavior (see Chapter 19).
process. Finally, these decision value signals
need to be compared to yield a decision
about which option ultimately should be Outcome Values
chosen.
In this chapter, I consider each of these Outcome values correspond to the over-
signals in turn and evaluate evidence about all subjective utility or benefit that a given
401
402 JOHN P. O’DOHERTY
et al., 2008; Kirk et al., 2009; O’Doherty these claims, although a recent study did
et al., 2001; O’Doherty, Winston, et al., 2003; report that activity in response to mone-
Ursu & Carter, 2005). However, several stud- tary outcomes was located more anteriorly
ies have also reported activity in the lateral within the OFC than activity in response
OFC in response to reward outcomes (Bre- to sexual rewards (Sescousse, Redoute, &
iter, Aharon, Kahneman, Dale, & Shizgal, Dreher, 2010).
2001; Elliott, Newman, Longe, & Deakin,
2003; O’Doherty, Critchley, Deichmann, &
Dolan, 2003), and this region has also been Chosen Values
implicated in possible functions other than
outcome values, such as inhibiting previ- Another key signal present during the deci-
ously learned responses following a change sion process is the chosen value, which cor-
in contingencies (Cools, Clark, Owen, & responds to the anticipatory value for the
Robbins, 2002), encoding aversive prediction expected outcome associated with the deci-
errors (Seymour et al., 2005), and detecting sion option that is ultimately selected. Like
changes in contingency (O’Doherty, Critch- outcome values, chosen values are a post-
ley, et al., 2003). decision signal, in that they are a conse-
Because aversive outcome values are quence as opposed to a precursor or input
often confounded with these other variables, into the decision process. A number of stud-
ascertaining the precise role of human OFC ies have revealed chosen value signals to be
in encoding aversive outcomes is still an present in the medial OFC extending dor-
ongoing issue in the literature. Neverthe- sally up the medial wall of the prefrontal
less, it is known that at the single-neuron cortex. For example, Hampton, Bossaerts,
level, spatially intermingled populations of and O’Doherty (2006) scanned volunteers
neurons in lateral and central parts of the while they participated in a probabilistic
OFC encode both rewarding and punish- choice task in which they could choose
ing outcomes (Morrison & Salzman, 2009). between pairs of stimuli yielding reward-
It is possible therefore that at the resolution ing or punishing outcomes with varying fre-
available for fMRI measurements, a standard quencies. Furthermore, the outcome prob-
univariate contrast between appetitive and abilities assigned to the different stimuli
aversive outcome values would fail to iden- switched or reversed over time. Hampton
tify some regions encoding outcome values, et al. used a computational model that took
even if neurons at the single-neuron level are into account the pattern of each subjects’
coding for both types of outcome signal. The choices and the outcomes obtained in the
deployment of multivariate pattern analysis past for selecting those options to derive
approaches may help provide new insight trial-by-trial estimates about the expected
into this question (Kahnt, Heinzle, Park, & value of particular decision options as sub-
Haynes, 2010) and clarify the contribution jects proceeded through the experiment.
of the medial and lateral OFC in encoding Those model-based predictions were then
positive and negative outcome value signals. correlated against the fMRI data in a proce-
It has also been proposed that outcome dure that has come to be known as “model-
values within the OFC can be differenti- based fMRI” (Glascher, Daw, Dayan, &
ated with respect to the “complexity” of O’Doherty, 2010). Using this approach,
the reinforcer. A meta-analysis revealed a Hampton and colleagues looked for brain
trend for abstract reinforcers such as mone- areas exhibiting activity at the time of choice
tary gain or loss to be represented more ante- but before the outcome was received, which
riorly, whereas more basic reinforcers such correlated with the value of the chosen stim-
as odor and taste were represented more ulus. This analysis revealed activity in the
posteriorly (Kringelbach & Rolls, 2004). To medial OFC and adjacent medial prefrontal
date little direct evidence beyond the meta- cortex (Figure 17.1A), implicating these areas
analysis has been forthcoming to support in encoding the expected value of the chosen
404 JOHN P. O’DOHERTY
Figure 17.1. Chosen value signals in vmPFC. (A) Left panel shows areas of vmPFC exhibiting
significant correlations with the expected value of the chosen action at the time of choice while
human volunteers are performing a simple reward-based choice task (data from Hampton et al.,
2006). Right panel shows the BOLD activity in this region plotted against a measure of expected
reward (prior correct) generated by the computational model. (B) Similar findings from a study by
Daw et al., (2006).
option. The more reward that was expected when activity at the time of choice was cor-
for the option that was chosen, the greater related against the model-estimated predic-
the activity in this area; in contrast, if the tions about the expected value correspond-
volunteer chose an option that was pre- ing to the option that was ultimately chosen,
dicted by the model to lead to less reward, significant effects were found in the medial
activity was decreased. OFC and adjacent medial prefrontal cortex
Similarly, Daw, O’Doherty, Dayan, Sey- (Figure 17.1B). In a recording study from
mour, and Dolan (2006) presented subjects a region of the central OFC in nonhuman
with a four-armed bandit task in which on primates, Padoa-Schioppa and Assad (2006)
each trial they were invited to choose one also found single neurons that coded for the
of four different colored slot machines. The value of the chosen option. When taken
magnitude of reward available on each slot together these studies indicate that chosen
machine drifted over time, although volun- value signals are encoded within the ven-
teers were not made aware of the specific tromedial prefrontal cortex (vmPFC) during
underlying reward contingencies, but rather decision making.
had to infer the value of each machine at dif-
ferent points in the course of the experiment
Chosen Values: Associated with
through sampling. Daw et al. also used a
Actions or Stimuli?
computational model to derive trial-by-trial
estimates for the expected value associated The finding of chosen value signals in the
with each of the available slots. Once again, vmPFC raises the question as to the nature
NEURAL MECHANISMS UNDERLYING VALUE- BASED DECISION MAKING 405
vmPFC [6 30 −9]
(a) 8 Action-based Reversal Stimulus-based Reversal
0.2 0.2
high
Evoked Response
6 0.1 med 0.1
low
4 0 0
−0.1 −0.1
2
vmpfc
−0.2 −0.2
0 x=6 1 3 5 7 9 1113 15 1 3 5 7 9 1113 15
Peri-Stimulus Time Peri-Stimulus Time
the action chosen was an eye or a hand based and action-based chosen values. To
movement, a mid-region of the vmPFC was address this question, Wunderlich, Rangel,
correlated with choice values only when and O’Doherty (2010) used a task in which
a hand movement was selected; a more the presentation of stimuli depicting the
posterior region again correlated only with available decision options was temporally
choice values only when an eye movement separated from the time at which specific
was selected (Figure 17.2B). Such modality- actions could be selected to make a choice.
specific signals could not easily be explained At the beginning of a trial, the subject was
in a stimulus-based account and likely reflect presented with a choice between two of
encoding of action-specific chosen value sig- three possible stimuli, each of which was
nals in this region. associated with distinct drifting probabil-
These findings raise the question of ities of obtaining reward. After a delay,
whether the vmPFC encodes only action- subjects were subsequently presented with
based chosen values or whether this region additional symbols informing them whether
is involved in encoding both stimulus- a given stimulus could be selected by
NEURAL MECHANISMS UNDERLYING VALUE- BASED DECISION MAKING 407
Let us first consider values assigned to & Glimcher, 1999; Sohn & Lee, 2007; Sug-
individual actions. These are referred to as rue, Corrado, & Newsome, 2004). However,
“action values” in the computational rein- with respect to the LIP, these signals appear
forcement learning literature. Consider an not to be pure action value signals, because
action a that is available in a state of the they are ultimately modulated as a function
world s; then the action value Q(s,a), cor- of what action is ultimately chosen. In the
responds to the average future reward that supplementary motor cortex, although the
would be expected to be obtained if action a value signals reported may be action values,
were selected. Assuming actions b and c the monkey experimental paradigms to date
are also available to be selected in the same have not generally permitted action values
state, then it follows that a value signal to be fully disambiguated from other kinds
will exist for each of these other actions as of value signal (such as action-dependent
well: Q(s,b) and Q(s,c). At its simplest, a chosen values).
decision-making strategy for the course of In humans, the Wunderlich et al. (2009)
action to pursue in state s could involve study referred to earlier attempted to iden-
opting to choose the action associated with tify the existence of action values in the
the highest action value. For such a deci- brain above and beyond chosen values.
sion to be rendered in the brain, it is neces- Recall that the paradigm required subjects
sary to encode separately the value of each to make a choice between an eye move-
action simultaneously: Q(s,a), Q(s,b), and ment and a hand movement, whereby the
Q(s,c). Importantly these signals should not probability of obtaining a reward following
be modulated by or dependent on choice in the choice of these actions changed over the
the way that chosen values are, but instead course of the experiment. That experiment
should remain the same regardless of the required choices to be made between effec-
choice made on a given trial (at least before tor modalities rather than between individ-
the outcome is experienced). ual motor movements within a modality
Relatively few studies have looked for because, although it might be expected that
or provided direct evidence for pure action neurons coding for action values within a
value signals in humans. In monkeys, (Same- modality would be located within the same
jima, Ueda, Doya, and Kimura (2005), overlapping area of the cortex and/or stria-
recorded from the striatum while monkeys tum and thus would be difficult to sepa-
performed a simple choice task in which rate with fMRI, action values for choices
they could choose between one of four made between modality effectors might be
possible actions associated with different expected to be represented in spatially dis-
amounts of reinforcement in different stim- tinct brain areas, thus enabling their identi-
ulus conditions. By correlating neural signals fication with fMRI. To identify action values
against action values generated from a rein- for eye movements, the researchers looked
forcement learning model, the researchers for areas correlating with the reinforcement-
reported that some neurons appeared to learning-derived value for an eye movement,
uniquely code for the value of individual with the proviso that such value signals
actions (as opposed to the value of the are invariant regardless of whether the eye
action ultimately chosen). Lau and Glim- movement was chosen or is not chosen.
cher (2007) also reported action value sig- Similarly, to identify action values for hand
nals in the striatum using a similar monkey movements, it was possible to look for areas
choice paradigm. Many other neurophysiol- showing correlations with the value of hand
ogy studies have reported action-dependent movement regardless of whether that move-
value signals in striatum and in areas of the ment was chosen on a given trial. Activ-
cortex such as the lateral-intraparietal sulcus ity in a region of the presupplementary eye
or supplementary motor cortex (Lee, Con- fields in the medial frontal cortex was found
roy, McGreevy, & Barraclough, 2004; Platt to be correlated with action values for eye
NEURAL MECHANISMS UNDERLYING VALUE- BASED DECISION MAKING 409
(a)
(b) 6
preSEF SMA
Contrast Estimate (a.u.)
−2
Ve Vh Ve Vh
Figure 17.4. Action values in supplementary motor cortex.
(A) Region of supplementary motor area and
pre-supplementary eye fields correlating with action values
for hand movements (green; Vh), and eye movements (red,
Ve), respectively. (B) Plot of parameter estimates depicting
correlations with action values for eye and hand movements
in these areas. See color plate 17.2.
movements, whereas a nearby region of sup- ues. Such value signals are tied not to spe-
plementary motor cortex was found to be cific actions, but instead to particular stimuli
correlated with action values for hand move- that denote particular available options. Evi-
ments (Figure 17.4). These findings suggest dence that such signals likely exist in mon-
that it is indeed possible to detect pre-choice key orbitofrontal cortex came from Padoa-
action value signals in the human brain and Schioppa and Assad (2006), who recorded
that these signals can be found in supple- from the OFC while monkeys made choices
mentary motor areas. Given the single-unit between stimuli denoting different amounts
neurophysiology findings described earlier, of one of two types of liquid juices. Single
it is also likely that such signals are present neurons were found that correlated with the
elsewhere in the human brain, such as the subjective value of the particular magnitude
dorsal striatum; however, if such signals in of juice denoted by each stimulus, with sep-
those other areas are intermixed at the neu- arate neurons coding for the value of each
ronal population level, conventional univari- juice type, independently of the particular
ate fMRI analyses may not permit their iden- action the monkey needed to perform to
tification. obtain it. Padoa-Schioppa and Assad (2006)
Moving on from action values, another proposed that these signals corresponded to
type of pre-decision signal corresponds to the value of “goods” and that these were
the values of particular decision options deployed as inputs into the decision process.
regardless of the action needed to select In a human fMRI study (Plassmann,
them, which are often called decision val- O’Doherty, & Rangel, 2007), hungry human
410 JOHN P. O’DOHERTY
subjects were presented with a variety of (Glimcher, Dorris, & Bayer, 2005; Shadlen,
food stuffs while eliciting their “willingness Britten, Newsome, & Movshon, 1996),
to pay” for each of the food items (from an whereas others argue in favor of the goods-
initial endowment of $4), yielding a trial-by- based framework (Padoa-Schioppa, 2007;
trial representation of their underlying sub- Padoa-Schioppa & Assad, 2006).
jective value for each item. After the experi- Although this remains an area of active
ment was over, one of the trials was selected research, one possible “hybrid” proposal on
at random, and if the reported willingness to the basis of the findings from the Glaescher
pay exceeded a random draw from a lottery et al. and Wunderlich et al. studies described
then subjects were provided with the food earlier is that both mechanisms might exist
item and invited to consume it (and their for computing choice at the same time, and
endowment was drawn on); otherwise they the extent to which a particular mechanism
kept the endowment and did not receive the is engaged for a particular decision problem
good. This procedure is designed to ensure (whether an action-based or a goods-based
that subjects give their true underlying val- mechanism) may depend substantively on
uation for each item. Activity in a region how the decision problem is framed, such as
of the vmPFC was found to be correlated the extent to which relevant outcomes and
with trial-by-trial variations in the willing- relevant actions are made salient (Wunder-
ness to pay, suggesting a role for this region lich et al., 2010). It appears that the vmPFC
in encoding the value of potential outcomes. likely contributes to both kinds of decision
This signal was suggested to correspond to making, and one possibility is that, in both
a representation of “decision values” that types, decisions are ultimately rendered by
were used as an input to the decision pro- retrieving goal values by virtue of their asso-
cess, which in this case corresponded to how ciation with either individual actions or with
much money to pay for a given item. individual stimulus cues. This proposal is
also compatible with another set of findings
that implicate the vmPFC in goal-directed
Decision Values versus Action Values:
learning (involving learning derived from
Which Are Necessary for Choice?
stimuli, actions and the current incentive
Action values and decision values are not value of outcomes; Valentin, Dickinson, &
only distinct types of pre-choice value sig- O’Doherty, 2007).
nals but are also the core signals under-
lying what seem at first glance to be two
very different ways to compute choices. In Comparing Decision Values or Action
the action value case, decisions are made Values: Locating the Comparator
in “action space” by considering each of Process
the possible actions available for a given
choice, attaching values to those actions, To actually generate a choice, it is neces-
and then implementing those actions yield- sary to compare the different values assigned
ing the highest expected reward. Alterna- to particular decision options to establish
tively, in the decision value case, decisions which option yields the highest utility. This
are computed in “goods space,” and they is the case regardless of whether decisions
constitute a more abstract decision whether are computed over actions or over stim-
to select a particular “good” or “goal,” inde- uli linked to outcomes, although the input
pendent of the actions needed to implement signals into a comparator and the neural
that selection, which are argued to occur system implementing the comparison pro-
at a later time. There is an ongoing debate cess might potentially differ depending on
in the decision neuroscience literature as whether decisions are taken over actions or
to which of these mechanisms accounts for stimuli. Putting aside the action versus goods
how choices are computed: Some argue debate, what would such a comparison pro-
strongly in favor of the action-based account cess look like on a computational level?
NEURAL MECHANISMS UNDERLYING VALUE- BASED DECISION MAKING 411
(a)
initialization race convergence
(b)
X=0
Figure 17.5. Decision comparator signals in dmPFC. (A) Illustration of a simple model of a putative
decision process. Processing units for eye (circle) and hand movements (square) are activated in
proportion to the magnitude of the predicted value for the corresponding actions. The average activity
corresponds to the sum of the constituent action values. These activated populations compete with
each other through mutual inhibition. Ultimately only some units remain, and the average activity
remaining corresponds to the value of the action chosen compared to the action not chosen. (B) Area
of dmPFC corresponding to the difference between the action not chosen and the action chosen (the
inverse of the outcome of the decision signal described earlier). Data from Wunderlich et al., (2009).
human reward learning (McClure, Berns, & cortex and the dorsomedial prefrontal cor-
Montague, 2003; O’Doherty, Dayan, Fris- tex, respectively. Furthermore, other post-
ton, Critchley, & Dolan, 2003; O’Doherty decision signals such as chosen values and
et al., 2004). As it happens, chosen values outcome values appear to be located in the
and outcome values are precisely the kinds ventromedial prefrontal cortex. These post-
of value signal that could be provided as decision signals are likely to serve a key role
input into the generation of a prediction in facilitating updating of action and/or deci-
error signal that could be used to update sion values through trial-and-error learn-
the pre-choice value signals. The prediction ing. Taken together, these findings appear
error signal would correspond to the differ- to implicate two key brain areas in value-
ence between outcome values (the value of based decision making: the ventromedial
what is experienced) and chosen values (the prefrontal cortex, and the dorsomedial pre-
value of what is predicted to occur given a frontal cortex and adjacent supplementary
choice made). Thus, a very plausible func- motor cortex. Although the vmPFC appears
tional interpretation of these post-decision to participate in the decision process regard-
signals is that they serve as an input for gen- less of whether decisions are computed over
erating prediction error signals that are then stimuli or actions, dorsomedial frontal areas
subsequently used to update the value sig- may be involved more selectively in deci-
nals needed to compute future choice. sions over actions, as opposed to stimuli.
Understanding the neural mechanisms
underlying value-based decision making is
Conclusions still in its early stages. Many open questions
remain. Perhaps the most fundamental of
In the present chapter we have reviewed these questions concerns the nature of the
several types of value signals that are likely mechanism by which the values assigned to
to play an important role in the decision- different options are compared in the brain.
making process, either as an input into or As yet we do not know where this compar-
a consequence of that process. Action val- ison process takes place, nor do we know
ues and decision values enable decisions to how this comparison is implemented on a
be computed in the first place, because they computational level. Another major issue
are the signals that need to be compared and alluded to here is that, although mecha-
contrasted to generate a decision. We have nisms exist to enable decisions to be com-
reviewed evidence to suggest that decision puted over stimuli as well as actions, it is
and action values may be located in distinct unclear what factors influence or control
parts of the cortex. Whereas decision values which of these mechanisms is to be deployed
are present in the ventromedial prefrontal for a given decision. Although it is possible
cortex, action values are present in the sup- to set up experimental paradigms in which
plementary motor cortex and are likely (as the nature of the task is such as to make
shown in monkey neurophysiology studies actions more salient than stimuli and vice
but not fMRI) in parts of the human stria- versa, and thus differentially bias individu-
tum. The role that such signals play in the als toward action-based and stimulus-based
decision process may depend on whether decision making, the role that these differ-
decisions are computed over stimuli that ent decision mechanisms might play in more
elicit outcomes or over actions. We also con- complex everyday choices is unclear. The
sidered evidence for the existence of signals role of individual differences related to gen-
that may reflect the output of the decision- der (see Chapter 26) and genetics (see Chap-
making process, most prominently the dif- ter 25) needs also to be clarified. Further-
ference between the value of the option that more, it is likely that decisions emerge as a
is ultimately chosen and the option that is function of the interaction among a num-
not chosen and its converse; these signals ber of distinct brain regions – not only the
were found in the ventromedial prefrontal two areas focused on in this chapter but
414 JOHN P. O’DOHERTY
also additional regions such as the amyg- responses to facial expressions of sadness and
dala, striatum, and intra-parietal cortex. A anger. Brain, 122(Pt. 5), 883–93.
complete understanding of the neural com- Blood, A. J., Zatorre, R. J., Bermudez, P., &
putations underlying value-based decision Evans, A. C. (1999). Emotional responses to
making is likely to be achievable only if pleasant and unpleasant music correlate with
activity in paralimbic brain regions. Nature
the nature of the causal interactions among
Neuroscience, 2(4), 382–87.
distinct brain regions during this process is
Boorman, E. D., Behrens, T. E., Woolrich,
fully elucidated. Finally, significant progress M. W., & Rushworth, M. F. (2009). How green
in the neurobiology of human decision mak- is the grass on the other side? Frontopolar cor-
ing is unlikely to be achieved through the tex and the evidence in favor of alternative
pursuit of hemodynamic imaging methods courses of action. Neuron, 62(5), 733–43.
alone. To answer some of the key ques- Breiter, H. C., Aharon, I., Kahneman, D., Dale,
tions raised here, it will be necessary to A., & Shizgal, P. (2001). Functional imaging
combine hemodynamic and electrodynamic of neural responses to expectancy and expe-
methods to obtain sufficient temporal res- rience of monetary gains and losses. Neuron,
olution to characterize the type of rapid 30(2), 619–39.
Busemeyer, J. R., & Townsend, J. T. (1993).
dynamic changes underlying the value com-
Decision field theory: A dynamic-cognitive
parison process itself.
approach to decision making in an uncer-
tain environment. Psychological Review, 100(3),
432–59.
Outstanding Questions and Future Cloutier, J., Heatherton, T. F., Whalen, P. J., &
Directions Kelley, W. M. (2008). Are attractive people
rewarding? Sex differences in the neural sub-
r How is the value comparison imple- strates of facial attractiveness. Journal of Cog-
mented on a computational level, and nitive Neuroscience, 20(6), 941–51.
where in the brain does this process Cools, R., Clark, L., Owen, A. M., & Robbins,
happen? T. W. (2002). Defining the neural mecha-
r Do neural systems involved in assigning nisms of probabilistic reversal learning using
event-related functional magnetic resonance
value to actions and those involved in imaging. Journal of Neuroscience, 22(11), 4563–
assigning value to stimuli interact, com- 67.
pete, or cooperate to facilitate value- Critchley, H. D., & Rolls, E. T. (1996). Hunger
based decision making, and how is this and satiety modify the responses of olfac-
implemented? tory and visual neurons in the primate
r What properties of the decision-making orbitofrontal cortex. Journal of Neurophysiol-
network give rise to individual differences ogy, 75(4), 1673–86.
in decision-making behavior, and what Daw, N. D., O’Doherty, J. P., Dayan, P., Sey-
role do genetic factors play in this pro- mour, B., & Dolan, R. J. (2006). Cortical sub-
cess? strates for exploratory decisions in humans.
r What is the nature of the causal interac- Nature, 441(7095), 876–79.
de Araujo, I. E., Kringelbach, M. L., Rolls,
tions among different brain regions dur- E. T., & McGlone, F. (2003). Human corti-
ing value-based decision making? cal responses to water in the mouth, and the
r What are the temporal dynamics under-
effects of thirst. Journal of Neurophysiology,
pinning the contribution of different 90(3), 1865–76.
regions to the decision-making process? de Araujo, I. E., Rolls, E. T., Velazco, M. I., Mar-
got, C., & Cayeux, I. (2005). Cognitive mod-
ulation of olfactory processing. Neuron, 46(4),
671–79.
References Elliott, R., Frith, C. D., & Dolan, R. J. (1997). Dif-
ferential neural response to positive and neg-
Blair, R. J., Morris, J. S., Frith, C. D., Perrett, D. ative feedback in planning and guessing tasks.
I., & Dolan, R. J. (1999). Dissociable neural Neuropsychologia, 35(10), 1395–1404.
NEURAL MECHANISMS UNDERLYING VALUE- BASED DECISION MAKING 415
Elliott, R., Newman, J. L., Longe, O. A., & ing and decision making in monkeys during a
Deakin, J. F. (2003). Differential response pat- competitive game. Brain Research: Cognitive
terns in the striatum and orbitofrontal cortex Brain Research, 22(1), 45–58.
to financial reward in humans: A parametric McClure, S. M., Berns, G. S., & Montague, P. R.
functional magnetic resonance imaging study. (2003). Temporal prediction errors in a passive
Journal of Neuroscience, 23(1), 303–7. learning task activate human striatum. Neu-
Glascher, J., Daw, N., Dayan, P., & O’Doherty, ron, 38(2), 339–46.
J. P. (2010). States versus rewards: Dissocia- Morrison, S. E., & Salzman, C. D. (2009). The
ble neural prediction error signals underlying convergence of information about rewarding
model-based and model-free reinforcement and aversive stimuli in single neurons. Journal
learning. Neuron, 66(4), 585–95. of Neuroscience, 29(37), 11471–83.
Glascher, J., Hampton, A. N., & O’Doherty, J. P. O’Doherty, J., Critchley, H., Deichmann, R., &
(2009). Determining a role for ventromedial Dolan, R. J. (2003). Dissociating valence of
prefrontal cortex in encoding action-based outcome from behavioral control in human
value signals during reward-related decision orbital and ventral prefrontal cortices. Journal
making. Cerebral Cortex, 19(2), 483–95. of Neuroscience, 23(21), 7931–39.
Glimcher, P. W., Dorris, M. C., & Bayer, O’Doherty, J., Dayan, P., Friston, K., Critchley,
H. M. (2005). Physiological utility theory and H., & Dolan, R. J. (2003). Temporal differ-
the neuroeconomics of choice. Games and ence models and reward-related learning in
Economic Behavior, 52(2), 213–56. the human brain. Neuron, 38(2), 329–37.
Hampton, A. N., Bossaerts, P., & O’Doherty, O’Doherty, J., Dayan, P., Schultz, J., Deich-
J. P. (2006). The role of the ventromedial pre- mann, R., Friston, K., & Dolan, R. J. (2004).
frontal cortex in abstract state-based inference Dissociable roles of ventral and dorsal stria-
during decision making in humans. Journal of tum in instrumental conditioning. Science,
Neuroscience, 26(32), 8360–67. 304(5669), 452–54.
Hollerman, J. R., & Schultz, W. (1998). O’Doherty, J., Kringelbach, M. L., Rolls, E. T.,
Dopamine neurons report an error in the tem- Hornak, J., & Andrews, C. (2001). Abstract
poral prediction of reward during learning. reward and punishment representations in the
Nature Neuroscience, 1(4), 304–9. human orbitofrontal cortex. Nature Neuro-
Kahnt, T., Heinzle, J., Park, S. Q., & Haynes, science, 4(1), 95–102.
J. D. (2010). The neural code of reward antic- O’Doherty, J., Rolls, E. T., Francis, S., Bowtell,
ipation in human orbitofrontal cortex. Pro- R., McGlone, F., Kobal, G., et al. (2000).
ceedings of the National Academy of Sciences, Sensory-specific satiety-related olfactory acti-
107(13), 6010–15. vation of the human orbitofrontal cortex.
Kawabata, H., & Zeki, S. (2004). Neural corre- Neuroreport, 11(4), 893–97.
lates of beauty. Journal of Neurophysiology, O’Doherty, J., Winston, J., Critchley, H., Perrett,
91(4), 1699–1705. D., Burt, D. M., & Dolan, R. J. (2003). Beauty
Kirk, U., Skov, M., Hulme, O., Christensen, in a smile: The role of medial orbitofrontal
M. S., & Zeki, S. (2009). Modulation of aes- cortex in facial attractiveness. Neuropsycholo-
thetic value by semantic context: an fMRI gia, 41(2), 147–55.
study. Neuroimage, 44(3), 1125–32. Padoa-Schioppa, C. (2007). Orbitofrontal cor-
Kranz, F., & Ishai, A. (2006). Face perception is tex and the computation of economic value.
modulated by sexual preference. Current Biol- Annals of the New York Academy of Sciences,
ogy, 16(1), 63–68. 1121, 232–53.
Kringelbach, M. L., & Rolls, E. T. (2004). Padoa-Schioppa, C., & Assad, J. A. (2006). Neu-
The functional neuroanatomy of the human rons in the orbitofrontal cortex encode eco-
orbitofrontal cortex: Evidence from neu- nomic value. Nature, 441(7090), 223–26.
roimaging and neuropsychology. Progress in Plassmann, H., O’Doherty, J., & Rangel, A.
Neurobiology, 72(5), 341–72. (2007). Orbitofrontal cortex encodes willing-
Lau, B., & Glimcher, P. W. (2007). Action and ness to pay in everyday economic transac-
outcome encoding in the primate caudate tions. Journal of Neuroscience, 27(37), 9984–
nucleus. Journal of Neuroscience, 27(52), 14502– 88.
14. Plassmann, H., O’Doherty, J., Shiv, B., & Rangel,
Lee, D., Conroy, M. L., McGreevy, B. P., & Bar- A. (2008). Marketing actions can modu-
raclough, D. J. (2004). Reinforcement learn- late neural representations of experienced
416 JOHN P. O’DOHERTY
pleasantness. Proceedings of the National Smith, P. L., & Ratcliff, R. (2004). Psychology
Academy of Sciences, 105(3), 1050–54. and neurobiology of simple decisions. Trends
Platt, M. L., & Glimcher, P. W. (1999). Neural in Neurosciences, 27(3), 161–68.
correlates of decision variables in parietal cor- Sohn, J. W., & Lee, D. (2007). Order-dependent
tex. Nature, 400(6741), 233–38. modulation of directional signals in the
Ratcliff, R., & McKoon, G. (2008). The diffu- supplementary and presupplementary motor
sion decision model: Theory and data for two- areas. Journal of Neuroscience, 27(50), 13655–
choice decision tasks. Neural Computation, 66.
20(4), 873–922. Sugrue, L. P., Corrado, G. S., & Newsome, W. T.
Rolls, E. T., Sienkiewicz, Z. J., & Yaxley, S. (2004). Matching behavior and the represen-
(1989). Hunger modulates the responses to tation of value in the parietal cortex. Science,
gustatory stimuli of single neurons in the cau- 304(5678), 1782–87.
dolateral orbitofrontal cortex of the macaque Thorpe, S. J., Rolls, E. T., & Maddison, S. (1983).
monkey. European Journal of Neuroscience, The orbitofrontal cortex: Neuronal activity
1(1), 53–60. in the behaving monkey. Experimental Brain
Samejima, K., Ueda, Y., Doya, K., & Kimura, Research, 49(1), 93–115.
M. (2005). Representation of action-specific Ursu, S., & Carter, C. S. (2005). Outcome rep-
reward values in the striatum. Science, resentations, counterfactual comparisons and
310(5752), 1337–40. the human orbitofrontal cortex: Implications
Schultz, W. (1998). Predictive reward signal of for neuroimaging studies of decision-making.
dopamine neurons. Journal of Neurophysiol- Brain Research: Cognitive Brain Research,
ogy, 80(1), 1–27. 23(1), 51–60.
Sescousse, G., Redoute, J., & Dreher, J. C. (2010). Valentin, V. V., Dickinson, A., & O’Doherty, J.
The architecture of reward value coding in the P. (2007). Determining the neural substrates
human orbitofrontal cortex. Journal of Neuro- of goal-directed learning in the human brain.
science, 30(39), 13095–13104. Journal of Neuroscience, 27(15), 4019–26.
Seymour, B., O’Doherty J, P., Koltzenburg, M., Vrticka, P., Andersson, F., Grandjean, D.,
Wiech, K., Frackowiak, R., Friston, K., et al. Sander, D., & Vuilleumier, P. (2008). Indi-
(2005). Opponent appetitive-aversive neural vidual attachment style modulates human
processes underlie predictive learning of pain amygdala and striatum activation during social
relief. Nature Neuroscience, 8(9), 1234–40. appraisal. PloS One, 3(8), e2868.
Shadlen, M. N., Britten, K. H., Newsome, W. Wunderlich, K., Rangel, A., & O’Doherty,
T., & Movshon, J. A. (1996). A computational J. P. (2009). Neural computations underlying
analysis of the relationship between neuronal action-based decision making in the human
and behavioral responses to visual motion. brain. Proceedings of the National Academy of
Journal of Neuroscience, 16(4), 1486–1510. Sciences, 106(40), 17199–17204.
Small, D. M., Zatorre, R. J., Dagher, A., Evans, Wunderlich, K., Rangel, A., & O’Doherty,
A. C., & Jones-Gotman, M. (2001). Changes J. P. (2010). Economic choices can be made
in brain activity related to eating chocolate: using only stimulus values. Proceedings of the
From pleasure to aversion. Brain, 124(Pt. 9), National Academy of Sciences, 107(34), 15005–
1720–33. 10.
Section V
EMOTIONAL LEARNING
AND MEMORY
CHAPTER 18
An important goal for the study of emotions a behavioral system that reacts quickly (but
is to design research experiments for the not always appropriately) to signals of dan-
laboratory that can help shed light on real- ger, how these signals are perceived and
life behaviors. Fear provides a model emo- interpreted involves a host of other sys-
tional response that transitions well from the tems. Understanding how fear integrates
laboratory to the real world. The range of with other emotional and cognitive systems
behaviors associated with fear can be evoked can be particularly complex when trying to
without much difficulty in a laboratory set- understand disorders of fear.
ting, where these behaviors can be system-
atically observed and measured. In addition,
many of the biological mechanisms involved Fear Pathways in the Brain
in fear learning and expression are shared
across species, which makes it a model Much of what we know about the cogni-
emotion for comparative research. In fact, tive neuroscience of human fear learning has
because a number of different species are been informed by neurophysiological stud-
predisposed to rapidly learn and retain fear ies in nonhuman animals. Decades of animal
memories, several branches of neuroscience research has delineated the neural circuitry
without a direct interest in human emotions important for processing and responding to
use fear learning as a useful instrument. both acquired and innate fears. Although an
Yet, despite the basic nature of fear in-depth review of the neurophysiology of
behaviors, it is not a simple emotion. For animal fear learning is beyond the scope of
instance, although fearful reactions to immi- this chapter, an overview of the essential
nent threats are more or less hard-wired findings from this line of research is impor-
responses, organisms must learn to adapt tant to fully appreciate investigations into
these behaviors to predict and avoid a diver- human fear learning.
sity of potential threats from the environ- To investigate how organisms learn about
ment. And although evolution has favored and respond to threats, researchers often
419
420 JOSEPH E. DUNSMOOR & KEVIN S. LABAR
use classical, or Pavlovian, fear-conditioning the BLA and CE are clusters of intercalated
paradigms. In this procedure, an emotion- cell masses that may serve a role in gat-
ally neutral stimulus (conditioned stimu- ing activity between these subnuclei of the
lus, CS), such as a tone or light, predicts amygdala. In the classic anatomical model
a naturally aversive or threatening stimulus of fear conditioning, the BLA is the prin-
(unconditioned stimulus, US), such as an cipal site for receiving sensory information
electric shock. Just as Pavlov’s experimental and is important for forming the associa-
dogs would salivate to a tone that had been tion between the CS and US. This region
paired with food, animals that undergo fear may also be involved in the consolidation
conditioning will begin to express a num- and storage of conditioned fear memories
ber of autonomic emotional responses (e.g., (Schafe, Nader, Blair, & LeDoux, 2001). The
a change in heart rate, perspiration, or res- CE receives projections from the BLA and
piration rate) to the presentation of the pre- is principally involved in initiating the con-
viously neutral CS. These fear-conditioned ditioned fear response through output pro-
responses (CR) reflect the well-known fight- jections to the hypothalamus and brainstem
or-flight-or-freezing behaviors and indicate structures.
that an association has been formed between Notably, anatomically constrained mod-
the CS and US. els of fear conditioning are continually
Studies of fear conditioning, predomi- evolving. For instance, the roles of the BLA
nantly in rodents, have traced the neural and CE in forming the CS-US association
pathways involved in forming the CS-US and producing the CR, respectively, may not
association and in producing the CR. Neu- be entirely dissociable, and some evidence
rophysiological studies have shown that sen- suggests that the CE may in some cases be
sory information concerning the CS and US involved in forming the CS-US association
converges in the amygdala (see Pape & Pare, and consolidating fear memories (Wilen-
2010, for a review; Figure 18.1). The amyg- sky, Schafe, Kristensen, & LeDoux, 2006).
dala is a structure within the anterior por- In addition, other regions closely associated
tion of the medial temporal lobe (MTL) that with the amygdala may play distinct roles
receives extensive afferent projections from in fear-related behaviors. The bed nucleus
a number of brain systems, including all sen- of the stria terminalis (BNST) in particular
sory systems and higher order association may contribute to behaviors associated with
cortex. A widespread distribution of effer- sustained tonic fear states indicative of anxi-
ent projections from the amygdala is impor- ety, rather than phasic fear responses (Davis,
tant for modulating information processing Walker, Miles, & Grillon, 2010).
broadly across the brain. For instance, pro- A key feature of amygdala neurocir-
jections from the amygdala to the ventral cuitry is that sensory information reaches
visual stream may be important for modu- the amygdala from two partially separated
lating the representation of sensory stimuli neural pathways: (1) a relatively slow cortical
following emotional experiences (Vuilleu- pathway that flows from the thalamus to pri-
mier, 2005). Importantly, the amygdala is not mary sensory cortex and then to higher level
a homogeneous structure, but is instead a association cortex before reaching the amyg-
collection of interconnected subnuclei. The dala and (2) a more rapid subcortical path-
nuclei most often implicated in fear condi- way that sends projections directly from the
tioning include the basal (B), accessory basal thalamus. Whereas the cortical route sup-
(AB), lateral (LA), and central (CE) nuclei. plies detailed information concerning a sen-
The L, AB, and B nuclei are often referred sory stimulus, the subcortical route quickly
to collectively as the basolateral complex detects potentially threatening objects and
(BLA), and the subnuclei within this com- generates immediate fear responses. These
plex can be further reduced into anatom- dual processing routes to the amygdala
ically distinct regions. Interposed between may have implications for understanding
EMOTIONAL LEARNING AND MEMORY 421
Output connections
Hypothalamus: sympathetic nervous activation
Bed nucleus of stria terminalis: stress hormone release, anxiety behaviors
Periaqueductal gray: freezing
Locus coeruleus: mediates arousal through release of norepinephrine
Ventral tegmental area : dopamine release important for associative fear
learning
CE
ITC
BLA
Input
Figure 18.1. Simplified fear-conditioning circuit depicting the inputs and outputs to the amygdala, as
well as the role of the output regions in fear conditioning. The diagram simplifies the layout of the
amygdala subnuclei, combining several distinct nuclei that constitute the basolateral complex. The
intrinsic connections between the amygdala subnuclei and the ITC are not detailed here. BLA =
basolateral complex; CE = central nucleus; ITC = intercalated cells.
complex fear behaviors (LeDoux, 1996). invasive procedures, the role of particular
For instance, because the thalamo-amygdala brain regions in fear conditioning is now
pathway contains fewer synapses, informa- being explored in humans using noninva-
tion about potential threats is sped to the sive functional brain imaging techniques.
amygdala before the cortex has had time In the following sections we highlight find-
to process the stimulus thoroughly. There- ings from human patients with brain dam-
fore, an organism might react rapidly to a age and from functional magnetic reso-
potential threat (e.g., initiating a freezing nance imaging (fMRI) studies, which have
response), only to realize a split second later revealed a crucial role for the amygdala
that the threat has passed or was nonexis- and other regions in the acquisition, expres-
tent. Once a CR is initiated, however, it is sion, and control of fear. Although positron
difficult to quickly shut it off, because the emission tomography (PET) studies on fear
physiological response systems engaged have processes have also been conducted, the
a relatively slow time course compared to blocked designs inherent to the PET tech-
the neural response. nique constrain interpretations. Specifically,
Although discoveries of the neurophysi- because PET signals are integrated over
ological substrates of fear conditioning have many seconds, brain activity uniquely asso-
been made predominantly in rodents using ciated with CS and US presentations cannot
422 JOSEPH E. DUNSMOOR & KEVIN S. LABAR
be readily distinguished, and different trial was widely cast as the explanatory mecha-
types cannot be intermixed in the experi- nism behind the acquisition of pathological
mental design (see Chapter 5). Pre- versus behaviors (Pavlov, 1927; Watson & Rayner,
post-conditioning comparisons can be made 1920).
with PET (e.g., Morris, Öhman, & Dolan, Theories of how mental illness arises
1998), but event-related fMRI designs have through S-R learning fell out of favor in the
proved optimal for examining the neural mid- to late 20th century, due in part to the
systems involved in fear conditioning not shift away from behaviorism toward cogni-
only because of these design issues but also tive models of learning and behavior. One
because of improved spatial resolution (e.g., of the chief criticisms leveled at behaviorism
for distinguishing activation in the amygdala was that it was too simplistic to explain the
and adjacent hippocampus). etiology of fear and anxiety disorders. For
instance, behaviorism made little account
for differences between humans and other
Acquiring Fear animals in their ability to condition, and it
considered all classes of conditioned stim-
From Historical to Contemporary
uli more or less equally (the equipotential-
Views of Conditioned Fear Learning
ity fallacy; Seligman, 1970). Moreover, S-R
The earliest behavioral studies of fear con- models could not readily account for why
ditioning in humans demonstrated that some individuals are more susceptible to
laboratory-based fear conditioning might be acquire a fear disorder following a condi-
a useful analog to investigate psychophysi- tioning experience than others. The direct
ological reactions to emotional experiences. correspondence between presentations of a
Early behaviorist researchers, such as James CS and the production of the behavioral
B. Watson, used these conditioning princi- response, characterized by S-R theory, was
ples as a basis to refute theories of anxi- not sufficient to explain how the CS can
ety grounded in Freudian psychology, which evoke variable displays of behavior. Finally,
placed the origin of neuroses in various com- individuals frequently develop fears to stim-
plexes and developmental issues while min- uli or situations to which they have never
imizing the role of direct stimulus learning. been directly exposed, seemingly circum-
Perhaps the most infamous early study venting S-R learning altogether.
of human fear conditioning was the “Little Models of classical conditioning that
Albert” experiment conducted by Watson evolved in the 20th century led to new
and Raynor (1920). In this study, an 11- insights into fear learning. One of the
month-old infant was conditioned to fear monumental shifts away from behaviorism
a white rat that was not feared prior toward cognitive models was the conception
to conditioning. The experimenters fear- that conditioned learning involves form-
conditioned Little Albert by presenting a ing mental representations of conditioned
white rat (CS) with the loud noise of a ham- and unconditioned stimuli (S-S learning).
mer knocking a steel bar (US). After sev- In contrast to the black-box approach of
eral rat-noise pairings, Albert began to cry S-R theories, these more cognitively ori-
when the rat appeared and would attempt ented learning models described how con-
to avoid it by crawling away (CR). This fear tingencies surrounding CS-US pairing influ-
of the rat then generalized to other neu- ence the acquisition of learned behaviors.
tral but perceptually related stimuli, such For instance, conditioning will only occur to
as a white rabbit or white beard. Because stimuli that provide predictive value for the
these responses resembled those observed US, and learning is a result of experiencing
in phobias, early Pavlovian fear condition- the difference between what a CS predicts
ing research supported the notion that anx- and what actually occurs (e.g., Rescorla &
iety was a reflection of stimulus-response Wagner, 1972). Importantly, because con-
(S-R) learning. The process of S-R learning ditioning can involve representations of
EMOTIONAL LEARNING AND MEMORY 423
CSs and USs, learning can occur even in ciation between fear-relevant stimuli and
the absence of direct CS-US pairing. For aversive USs (for a review see Öhman &
instance, increasing or decreasing the aver- Mineka, 2001). Öhman and Mineka (2001)
sive intensity of the US after conditioning have incorporated the idea of selective asso-
leads to an increase or decrease in fear to the ciations into an encompassing “fear mod-
CS when it is later encountered – an effect ule,” which supports rapid and automatic
known as US deflation and inflation, respec- fear learning between fear-relevant and bio-
tively (Davey, 1992). Advances in cognitive logically significant stimuli. The putative
models of associative learning have had ben- anatomical substrate of this fear module is
eficial applications to understanding fear dis- the amygdala, which they regard as oper-
orders from the perspective of behavior the- ating independently of conscious control
ory (Mineka & Zinbarg, 2006). during fear learning and expression. How-
An important development in theories ever, as discussed later, neuroimaging stud-
of fear acquisition concerns the qualita- ies have shown that the amygdala’s response
tive nature of the CS. Animal research has is modulated by regulatory processes dur-
shown that learning occurs best to partic- ing fear learning. The amygdala’s emotional
ular classes of stimuli that, in many cases, response profile is not exclusively concerned
are species-specific. For instance, a rat can with fear, and it contributes to other func-
readily learn to associate a taste (CS) with tions, such as social cognition and motiva-
illness (US), or a noise (CS) with shock tion (see Chapter 19).
(US), but cannot easily form the crossed
associations (taste with shock, or noise with
Human Lesion Studies of Fear
illness; Garcia & Koelling, 1966). Seligman
Acquisition
(1971) proposed that humans are also pre-
disposed to form associations between par- Early nonhuman primate studies revealed
ticular types of CSs and USs. As evidence that large lesions of the medial tempo-
for the selectiveness of stimuli to which ral lobe (MTL) led to a marked loss of
people readily condition, Seligman noted fear responses and abnormal social behav-
that phobias generally fall under a select ior (Kluver & Bucy, 1939). Although initial
number of “fear-relevant” objects or situ- accounts attributed these results to the hip-
ations that have served as evolutionarily pocampal damage sustained in the animals,
significant stimuli throughout mammalian the behavioral effects were later replicated
development. These include certain ani- when lesions were restricted to the amyg-
mals (e.g., snakes and spiders), environmen- dala alone (Weiskrantz, 1956). Experiments
tal phenomenon (e.g., thunder and light- in human patients with MTL damage due to
ning), physical locations (e.g., being in an epilepsy, viral encephalitis, or a congenital
enclosed space or high off the ground), and disease known as Urbach-Wiethe syndrome
social interactions (e.g., emotionally charged demonstrated a necessary link between the
exchanges or evaluations). Fear learning integrity of the MTL and fear learning. In
between conditional stimuli from these pre- these studies, fear conditioning was assessed
pared classes of stimuli and an aversive by measuring changes in the skin conduc-
US may occur more readily than between tance response (SCR), which measures the
stimuli that are not related through fear change in electrodermal conductance on the
relevance. palmar surface of the hands due to sympa-
In support of Seligman’s theory, fear- thetic arousal (see Chapter 3). Patients with
relevant stimuli (e.g., images of snakes and MTL damage, including the amygdala, are
spiders) sometimes lead to superior con- impaired at acquiring and expressing condi-
ditioning and they delay extinction learn- tioned SCRs, but are similar to controls in
ing relative to fear-irrelevant environmental responding to the aversive US. This pattern
stimuli (e.g., images of flowers and mush- of results suggests that the amygdala and
rooms), suggesting a strong selective asso- surrounding cortex are crucial for learning
424 JOSEPH E. DUNSMOOR & KEVIN S. LABAR
(a) (b)
0.3
0.2
–0.1
–0.2
Control
Temporal Lobectomy
–0.3
H1 H2 A1 A2 A3 A4 E1 E2 E3 E4
HABITUATION ACQUISITION EXTINCTION
R L
(d) R L
(c)
ACg
Thal 2.5
Amy 2.0 *
1.5
% AUC
1.0
0.5
0.0
–0.5
–1.0
CS+ CS–
RESPONSE NONRESPONSE
Figure 18.2. (A, B) Patients with unilateral amygdala lesions show impaired fear conditioning, as
assessed by the skin conductance response (SCR). (C) Neuroimaging studies of healthy adults shows
that several areas, including the amygdala, thalamus, and prefrontal cortex, respond to a
fear-conditioned stimulus. (D) Activity in the amygdala tracks conditioned fear responses to a
fear-conditioned stimulus (CS+), whereas fear responses evoked by an unpaired control stimulus
(CS−) are not correlated with amygdala activations. Panels A and B reproduced with permission
from LaBar et al. (1995). Panel C from LaBar & Cabeza (2006). Panel D from Cheng et al. (2006).
and initiating conditioned responses (Figure (Bechara et al., 1995; LaBar & Phelps, 2005).
18.2A,B; LaBar, LeDoux, Spencer, & Phelps, This double dissociation between the phys-
1995). Importantly, amygdala damage does iological expression of learning and explicit
not interfere with declarative knowledge awareness suggests that fear conditioning
of the fear-learning episode – non-amnesic can operate at an implicit level of processing.
patients with amygdala lesions are able to In another study, individuals with amyg-
explicitly state that the CS was paired with dala damage were shown to have impaired
the US (contingency awareness). In contrast, fear-potentiated startle responses (Weike
patients with damage to the hippocampus, et al., 2005), another measure of conditioned
but not the amygdala, retain the capac- fear that relies extensively on the amygdala
ity to fear condition, but lack declarative (Davis, 1992). The startle response, which is
knowledge of the stimulus contingencies often measured using electromyography on
EMOTIONAL LEARNING AND MEMORY 425
(a)
Direct Pavlovian fear conditioning Observational fear conditioning Instructed fear conditioning
CS US
CS US
CS US
(b)
1.4
∗ CS+ CS−
Mean Skin Conductance Response
1.2
∗
1.0
∗
0.8
n.s.
0.6 †
0.4 ∗
0.2
0.0
Unmasked Masked Unmasked Masked Unmasked Masked
PAVLOVIAN OBSERVATION INSTRUCTION
Figure 18.3. Direct and indirect fear-learning paradigms. (A) In standard Pavlovian fear conditioning,
the individual encounters the CS and US directly. In observational fear conditioning, the individual
acquires fear of the CS vicariously by watching another individual undergo Pavlovian conditioning. In
instructed fear conditioning, the individual acquires fear of the CS without ever experiencing the CS
and US together, often through language. (B) Results from a behavioral study that directly compared
these three forms of fear conditioning across individuals using both supraliminal (unmasked) and
subliminal (masked) presentations of the fear-conditioned stimulus (CS+) and an unpaired control
stimulus (CS−). Although fear was acquired to the CS+ using all three procedures, a masked CS+
only evoked a fear response if fear was acquired through Pavlovian or observational fear conditioning.
CS = conditioned stimulus. Panel B reprinted with permission from Olsson & Phelps (2004).
fearfully with stimuli that are affectively 2010). Inactivation of the medial pain sys-
neutral (Cook & Mineka, 1989). Observa- tem, including the anterior cingulate cortex,
tional fear learning has also been reported in reduced observational fear learning in the
mice, and the effect of vicarious fear learning observer mouse, suggesting that this form
was more pronounced when the observer of fear learning necessitates an intact pain
mouse was a sibling or a long-term mate of pathway to vicariously acquire a learned fear
the mouse receiving electric shocks (Jeon, response.
428 JOSEPH E. DUNSMOOR & KEVIN S. LABAR
Olsson and Phelps (2004) showed that across brain systems involved in learning
human observers acquire fear of a CS after and memory (for a review see Rodrigues,
watching someone else react as if they them- LeDoux, & Sapolsky, 2009). For instance,
selves had received a shock paired with during avoidance learning the amygdala is
a CS. Observationally acquired CRs were involved in initiating the peripheral release
also elicited when the CS was later pre- of glucocorticoids from the adrenal cor-
sented subliminally. Neuroimaging studies tex. Stress hormones released from the
have revealed that the amygdala, a region periphery can then circulate to the brain,
implicated in direct fear learning, is acti- where they bind to the amygdala and hip-
vated during observational fear learning as pocampus (among other areas) and mod-
well. For instance, when subjects watch a ulate activity in these regions (McGaugh,
video of an individual undergo fear condi- Cahill, & Roozendaal, 1996). In humans,
tioning, exposure to the observationally con- there are gender differences in the effects of
ditioned CS during fMRI evokes activity in stress hormones on fear learning (Jackson,
the amygdala (Olsson & Phelps, 2007). Payne, Nadel, & Jacobs, 2006). For instance,
Another indirect fear-learning pathway introducing a psychosocial stressor (giving
involves the verbal communication of a public speech) prior to fear condition-
threat. Several animal species communicate ing enhances fear acquisition in males, but
threat through vocalizations. For instance, diminishes fear acquisition in females (Jack-
rats produce an ultrasonic vocalization when son et al., 2006). Similarly, endogenous
they are in aversive situations, and this may cortisol levels measured after fear acquisi-
send an alarm to conspecifics about nearby tion correlate positively with CRs in males
threats (Blanchard, Blanchard, Agullana, & but not females (Zorawski, Cook, Kuhn,
Weiss, 1991). The ability to communicate & LaBar, 2005). These sex differences are
through symbolic language has greatly facil- hypothesized to relate to a neuroprotective
itated this ability in humans. In laboratory effect of estrogen on the stress response in
studies of instructed fear conditioning, sub- females. Subjects with high endogenous cor-
jects are merely told that a CS will be tisol levels during fear learning also have
paired with a US, but they never actually a higher magnitude of conditioned SCRs
receive the US in combination with the expressed in a 24-hour retrieval test, sug-
CS. Nonetheless, the CS can still evoke gesting that stress hormones released during
a CR when presented alone, suggesting conditioning strengthen consolidation of the
that similar fear-conditioning circuits oper- fear memory (Zorawski, Blanding, Kuhn, &
ate under conditions of direct and instructed LaBar, 2006).
fear learning (Olsson & Phelps, 2007). Unlike
an observationally fear-conditioned stimu-
Learning to Fear the Context
lus, an instructed CS does not evoke a CR
Surrounding an Aversive Event
when it is presented subliminally, suggest-
ing that conscious awareness is necessary The discrete sensory cues present during fear
to detect a threat learned through language learning rarely occur in isolation, but rather
(Figure 18.3; Olsson & Phelps, 2004). The left exist among a wide set of environmental
amygdala is active in response to a CS that stimuli. These background features provide
subjects have been told will be followed by the context for a fear-learning event. In con-
a US, even when no direct pairing occurs text conditioning the organism learns that
(Phelps et al., 2001). the CS is not the only predictor for the
US, but instead the context itself should be
regarded as threatening. Importantly, a con-
Involvement of Stress Hormones in
text can be made up of numerous sensory,
Human Fear Learning
spatial, and temporal features, and thus the
Stress hormones that are released during features that constitute a context can differ
fearful situations have widespread effects markedly from a more easily defined sensory
EMOTIONAL LEARNING AND MEMORY 429
CS. Because contextual cues remain rel- and the manipulations are not readily trans-
atively constant in the environment, an ferrable to neuroimaging applications. A
organism may be in a prolonged state of fear second approach has been to simply change
in a context that has been associated with background environments on a computer
fearful experiences, even though cued fears screen on which the CS is presented. This
are only expressed in the presence of the approach has the advantages of portability
CS (Davis, Walker, Miles, & Grillon, 2009). and a wide selection of background images,
Learning to fear a context and learning to but is limited to unimodal context shifts.
fear a cue may therefore involve partially Using this latter approach during fMRI,
distinct neural systems and cognitive pro- Alvarez, Biggs, Chen, Pine, and Grillon
cesses. (2008) showed that activity in the hippocam-
Animal models of context conditioning pus and amygdala was elevated when par-
have focused on the role of the hippocam- ticipants were presented with a background
pus in encoding and representing the feared visual context associated with an unpre-
environment. The hippocampus is thought dictable aversive US relative to another one
to provide a substrate for binding the numer- in which the US was never presented. In
ous features of the context into a uni- this study, however, no explicit CS was pre-
tary representation; it may be specialized sented, so the neural substrates differentiat-
for calling to mind a full representation of ing cued and contextual fear could not be
a feared context from partial information ascertained.
through the process of pattern completion
(O’Reilly & Rudy, 2001). For instance, ani-
Is Fear Conditioning a Purely Implicit
mals that undergo cued fear conditioning in
Form of Learning?
a particular environment will later express
fear to that context, but damage to the In Squire’s taxonomy of memory (Squire,
dorsal hippocampus following conditioning 1986), simple conditioning is described as a
reduces that fear (e.g., Anagnostaras, Maren, nondeclarative form of learning. In support
& Fanselow, 1999). Although portions of the of this view, MTL damage, including the
amygdala are required for both cued and hippocampus, impairs declarative memory
context conditioning, the dorsal hippocam- while leaving cued fear conditioning intact
pus is primarily involved in context condi- (Bechara et al., 1995; LaBar & Phelps, 2005).
tioning (Phillips & LeDoux, 1992). Interestingly, some patients can express an
The vast majority of context condition- intact CR to an explicit CS physiologically,
ing investigations have been conducted in but fail to declaratively state that the CS pre-
animals, because manipulating the context dicts the reinforcer. Yet, the extent to which
for animals often only requires a change conscious awareness plays a role in Pavlovian
from one testing cage to another one with conditioning is a long-standing issue that
distinctive features. Manipulating the con- is not easily resolved (LaBar & Disterhoft,
text in human studies is more difficult, 1998; Lovibond & Shanks, 2002).
because humans might not construe a sub- Some behavioral studies have shown that
tle change in a traditional laboratory room as conscious awareness in healthy participants
an altogether new context. One approach to may be necessary for fear conditioning. For
conducting human behavioral studies with instance, Hamm and Vaitl (1996) showed
a context manipulation has been to con- that subjects who cannot accurately report
duct different phases of the experiment in the CS-US association fail to show condi-
entirely different rooms with distinct envi- tioned SCRs. Lovibond and Shanks (2002)
ronmental features (LaBar & Phelps, 2005). have argued that organisms produce a CR in
This approach has the advantage of being preparation for the US, and therefore aware-
multimodal and immersive, although the ness of the CS-US relationship is essen-
number of rooms and kinds of manipula- tial to the expression of learned fear. They
tions are limited in most research settings argue that studies that show conditioning
430 JOSEPH E. DUNSMOOR & KEVIN S. LABAR
without awareness use imprecise measures lap between these systems is in the amyg-
of awareness; for instance, making retro- dala, a region that receives extensive pro-
spective ratings of awareness using post- jections from both higher order cortical
experimental questionnaires. The issue of and lower level subcortical regions. In this
when awareness is assessed might be partic- way, the amygdala is uniquely situated to
ularly important for assessing contingency rapidly detect and respond to low-level sen-
awareness in patients with MTL damage, sory information that may initially bypass
because these individuals would likely have the higher level sensory systems in the cor-
trouble recalling any details from the event tex. More detailed information concerning
after a delay. Moreover, making retrospec- the stimulus (e.g., sensory information and
tive ratings of awareness activates long-term related memories) is then supplied to the
memory processes, and this system is likely amygdala via sensory and association cortex,
different from that involved during initial which helps inform the individual about the
fear acquisition. Some studies have there- threat relevance of the stimulus and modify
fore opted to use online measures of aware- emotional reactions.
ness (i.e., making a rating for US expectancy Behavioral studies manipulating con-
on every trial) to resolve issues related to scious awareness of CS-US contingencies
retrospective reports. However, these pro- in healthy adults often employ backward
cedures alter task demands and draw atten- masking procedures, wherein the CS is
tion to the stimulus contingences (LaBar & rapidly presented (on the order of 10 to 30
Disterhoft, 1998), putting more emphasis on ms) and then quickly “masked” by another
declarative systems during fear learning. stimulus. In this case, the subject is made
Numerous studies have endeavored to unaware of the presentation of the CS, but
resolve the relationship between awareness might still produce a CR when the CS
and conditioning. For the most part, these is later presented supraliminally (Esteves,
studies have relied on the notion that sep- Parra, Dimberg, & Öhman, 1994). A poten-
arate memory systems are used during fear tial issue with backward masking proce-
learning (LeDoux, 1996). On the one hand, dures, however, is that some subjects may
the declarative memory system is important be able to consciously perceive the CS or
for the conscious aspects of a fear-learning to perceptually discriminate a masked CS+
experience. This system might be important and CS−, but are just unable to precisely
for processing the contextual details of the describe the details of the masked stimu-
fear-learning episode and relating it to pre- lus (Lovibond & Shanks, 2002). Avoiding
vious experiences or acquired knowledge. this problem of backward masking, Knight,
For example, an individual who regularly Waters, and Bandettini (2009) used a novel
travels on airplanes might disregard turbu- approach to investigate subjective levels of
lence based on past experiences that turbu- awareness of fear conditioning by manip-
lence is common. On the other hand, the ulating the threshold of auditory CSs on
nondeclarative system is concerned with the a trial-by-trial basis. During fMRI, subjects
automatic processes involved in acquiring pressed a button to indicate that they heard
and producing learned behavioral responses one of two auditory CSs, one of which
(Squire & Zola, 1996). This system may be (the CS+) was paired with the US. Unbe-
less concerned with interpreting a poten- knownst to the subjects, the volume for
tial threat with regard to its prior history the CS was lowered by 5 dB on the next
and more concerned with reacting in the trial if they reported hearing the tone, or
moment. For example, an abrupt change raised by 5 dB if they did not press a but-
in altitude might jolt even the most reg- ton to indicate hearing the tone. Ratings of
ular plane traveler, despite the fact that US expectancy were measured continuously
he or she is consciously aware that tur- throughout the experiment, and these rat-
bulence is usually not dangerous. LeDoux ings confirmed that subjects only expected
(1996, 2000) proposed that the neural over- the US on CS+ trials that were perceived,
EMOTIONAL LEARNING AND MEMORY 431
and not when the CS+ was unperceived. In sum, evidence from behavioral and
Activity in the hippocampus increased dur- fMRI experiments supports the view that
ing perceived, relative to unperceived, CS+ conditioning is in some ways an automatic
trials as well, which suggests that contin- form of implicit learning that can occur out-
gency awareness evoked differential activ- side a high degree of cognitive awareness.
ity in this region. In contrast, conditioned Of course, the distinct role of deliberative
SCRs and amygdala activity were observed and automatic systems in mediating condi-
on CS+ trials that were both perceived and tioning begs the question of whether there
unperceived. This finding demonstrates that are in fact dissociable fear-learning systems
individuals can acquire conditioned fears to (Shanks, 2010). It may be the case that dif-
stimuli that are presented below the thresh- ferent systems interact and overlap in the
old of awareness and that this learning is course of learning about affective stimuli
likely mediated by the amygdala. One issue and that these systems may not be as easily
with measuring awareness in conditioning is dissociated as suggested by Squire’s taxon-
the conflation of CS awareness with CS-US omy.
contingency awareness (Lovibond & Shanks,
2002), highlighting the complexities of how
Learning to Avoid a Feared Stimulus
awareness as a construct is defined (see
Chapter 15). Once an organism has learned through clas-
Another way to test the role of aware- sical conditioning that the CS portends
ness in conditioning is by using trace con- delivery of an aversive US, it can begin
ditioning procedures. In trace conditioning, to take measures to avoid the US alto-
a temporal gap is introduced between the gether. The latter process involves instru-
offset of the CS and the onset of the US, mental behaviors, in which the organism
during which time organisms must maintain learns that a particular course of action (i.e.,
a representation of the CS to successfully response) will result in a particular outcome.
learn the CS-US relationship. This form of In the case of active avoidance, the desired
associative learning involves the hippocam- outcome is to steer clear of the US. This
pus and may entail a higher degree of con- outcome can be achieved in some cases by
scious awareness than standard delay con- learning to escape to another location or per-
ditioning, in which the CS co-terminates forming an action that terminates the CS.
with the US. For instance, it has been shown Extensive research by McGaugh and col-
that taxing working memory processes dur- leagues has shown that the amygdala plays
ing the course of acquisition differentially a critical role in acquiring and expressing
impairs trace conditioning relative to delay avoidance learning (reviewed in McGaugh,
conditioning (Carter, Hofstotter, Tsuchiya, 2004). The basal nucleus of the amyg-
& Koch, 2003). Trace conditioning has also dala provides the output connections to
been conducted in individuals with minimal the striatum that may be important for
conscious awareness (i.e., vegetative states; initiating the motor movements associated
Bekinschtein et al., 2009). Using a trace eye- with escape behaviors, or “active coping”
blink conditioning task with an aversive air- (LeDoux & Gorman, 2001). This pathway
puff to the eye as the US, individuals with between the amygdala and striatum may
minimal consciousness were capable of pro- compete with the pathway between the CE
ducing a conditioned eyeblink response to a and the brainstem for behavioral expres-
tone that predicted the delivery of the US. sion. For instance, projections from the CE
Performance in the trace eyeblink condition- to the brainstem lead to freezing behav-
ing task was positively related to patient iors that would inhibit motor movements
recovery. In contrast, subjects under gen- needed to avoid the impending US. If the
eral anesthesia (i.e., near total loss of aware- CE is blocked, however, then the animal can
ness) did not show a conditioned eyeblink express avoidance behaviors, such as flee-
response. ing to another chamber to avoid a footshock
432 JOSEPH E. DUNSMOOR & KEVIN S. LABAR
(Choi, Cain, & LeDoux, 2010). Human neu- taneously maintaining the previous mem-
roscience research on active avoidance is ory of the CS as a danger signal. The
sparse, but one fMRI study found that active latter view is supported by the fact that
avoidance of the US (achieved by pressing a extinguished fear responses can return. For
particular button during CS+ trials) resulted instance, numerous human and animal stud-
in increased activity in the striatum that was ies have shown that an extinguished CR
positively correlated with amygdala activ- can return and be expressed when some
ity (Delgado, Nearing, LeDoux, & Phelps, amount of time has passed since extinction
2008). training (spontaneous recovery), when the CS
is encountered in a different context from
where it was originally extinguished (fear
Controlling Fear renewal), or when the US is presented alone
(reinstatement; for a review see Bouton,
Although acquiring and expressing learned Westbrook, Corcoran, & Maren, 2006).
fear are evolutionarily adaptive abilities, it is These phenomena (collectively referred to
also important to control fears when they as the return of fear) indicate that the old
no longer serve an adaptive purpose. For CS-US association is far from forgotten and
instance, once a stimulus no longer predicts may compete with the new memory of the
a threat, reacting as if it was still a reli- CS formed after extinction (Bouton, 2004).
able danger signal would be a waste of time The neural substrates of fear extinction
and energy resources. In humans and other have been explored in depth using ani-
species, the ability to control fear responses mal models. Neurophysiological research
relies extensively on the prefrontal cortex. has established a critical role of the medial
PFC for extinction learning. In particular,
the ventromedial PFC (vmPFC) appears to
Extinction
be important for exerting inhibitory con-
Through conditioned learning, an organism trol over the amygdala in order to reduce
forms the association between the CS and fear expression (Milad & Quirk, 2002). The
aversive US and learns to produce a CR. vmPFC may be important for consolidat-
If the CS is repeatedly presented without ing memories of extinction learning for later
the US, then the acquired CR will begin to recall, because lesions to this region prior to
decrease over time. The process that leads to fear extinction do not delay extinction over
a reduction in CR expression (by eliminating the course of training, but impair extinction
the predictive relationship between the CS when tested at a later date (Quirk, Russo,
and US) is called extinction and represents an Barron, & Lebron, 2000). The amygdala is
important form of learning originally identi- also critical for extinction learning; block-
fied in Pavlov’s laboratory. Repeated expo- ing activity in the basolateral amygdala has
sure to a feared stimulus forms the basis for been shown to block extinction (Kim et al.,
several cognitive behavioral therapies aimed 2007). The hippocampus also serves a role in
to treat clinical anxiety disorders, such as the context specificity of extinction learning
phobias and obsessive-compulsive disorder. (Bouton et al., 2006). For instance, presen-
Exposure therapies based on the principles tation of the US or a reminder of it can
of Pavlovian conditioning have proved to be reinstate fear responses to a CS that has
highly effective treatments for certain anxi- previously been extinguished. This effect is
ety disorders (e.g., Foa et al., 2005). context dependent, such that reinstatement
Fear extinction implies that the organism only leads to a return of fear if the CS is
no longer conceives of the CS as threatening. encountered in the same context as the US
The question arises whether the organism reminder. Amnesic patients with hippocam-
has simply forgotten that the CS was once pal damage do not show context-dependent
threatening or, alternatively, has formed a reinstatement, however, indicating that the
new memory for the CS as safe while simul- hippocampus is important for regulating
EMOTIONAL LEARNING AND MEMORY 433
0.6
*
0.5
Normalized SCR
0.4
0.3
0.2
0.1
0
Acquisition Extinction Recovery
WS
the expression of fear following extinction extinction, Phelps et al. (2004) showed that
learning (LaBar & Phelps, 2005; Figure 18.4). the recall of extinction over a 24-hour delay
Together, the medial PFC, amygdala, and involved activity in the vmPFC, replicat-
hippocampus provide an important neural ing findings from animal research (Milad &
circuit for fear extinction, although their rel- Quirk, 2002). Using a context manipulation
ative contributions and interactions remain in extinction learning, Milad et al. (2009)
to be specified. found that the recall of extinction memory
Findings from animal studies of extinc- in an extinguished context involves activity
tion have been extended to humans using in the hippocampus and vmPFC, replicat-
fMRI. In the first demonstration of extinc- ing findings from nonhuman animal studies
tion in humans using fMRI, LaBar et al. (Milad & Quirk, 2002).
(1998) showed that responses in the amyg- Widespread clinical observations that
dala initially increased in early extinction tri- fears can return following extinction sug-
als and then decreased over the course of gest that extinction memory may be weaker
extinction. In a 2-day fMRI study of fear in kind than the competing fear memory.
434 JOSEPH E. DUNSMOOR & KEVIN S. LABAR
Reactivation of consolidated
Acquisition of conditioned fear fear memory
CS US CS
LTM 1
Manipulation
Time
Memory
Time
e.g., pharmacological
consolidation Reconsolidation blockade of protein synthesis,
extinction learning
LTM 1 LTM 2
stable long-term memory requires the mem- ings have been extended to humans using
ory to be reconsolidated. Reconsolidation the nonselective beta-blocker propranolol.
temporarily makes the long-term memory For instance, Kindt and colleagues (Kindt,
labile before it once again returns to a sta- Soeter, & Vervliet, 2009) showed that reac-
ble state. Relevant to emotional memories, tivation of a fear memory led to a sig-
the reconsolidation hypothesis suggests that nificant decrease in the expression of fear
a labile memory can be modified such that 24 hours later – but only if the individual
new information can alter an existing mem- had been administered propranolol prior to
ory before it once again becomes stable reactivation. A group-administered placebo
(Figure 18.5; Nader & Hardt, 2009). prior to reactivation did not show the same
The notion that existing fear memories effect in the test of fear expression 24 hours
can be altered via reconsolidation processes later. Importantly, the subjects’ declara-
has received some empirical support in both tive knowledge of the CS-US contingencies
human and animal experiments. For exam- was unaffected by propranolol, suggesting
ple, Nader, Schafe, and LeDoux (2000) fear- that pharmacological blockade of the beta-
conditioned rodents to a tone paired with an adrenergic system differentially affects the
aversive US. Later, the tone was replayed expression of fear while leaving the explicit
without the US to reactive the memory. memory intact. The impact of reactivating
Soon after the memory was reactivated, fear memories under the effect of propra-
the protein synthesis inhibitor anisomycin nolol has been shown to extend to a one-
was injected into the amygdala. Blocking month follow-up of fear expression (Kindt
protein synthesis during the reconsolida- & Soeter, 2010).
tion time window led to a decrease in fear The ability to behaviorally modify fear
responses to the CS, suggesting that the memories without the use of protein block-
fearful memory was modified. These find- ing drugs or pharmacological agents has also
436 JOSEPH E. DUNSMOOR & KEVIN S. LABAR
been shown both in rodents and humans ments. In addition, there are still many ques-
(Monfils, Cowansage, Klann, & LeDoux, tions concerning the biobehavioral systems
2009; Schiller et al., 2010). These studies take involved in different aspects of human fear
advantage of the reactivation window pro- learning.
vided by reconsolidation to introduce new
information regarding the feared CS. In a
human behavioral study, a decrease of fear Generalization of Learned Fears
expression was observed when extinction
learning was conducted within the recon- An important question that has received
solidation time window (10 minutes after sparse attention in human fear-learning
reactivation), and this effect persisted up research concerns how fears acquired to a
to a year after fear conditioning (Schiller et particular conditioned stimulus generalize
al., 2010). Conversely, a group that received to other stimuli that have never directly
extinction learning outside the reconsolida- predicted an aversive outcome. The phe-
tion time window (6 hours after reactiva- nomenon of stimulus generalization was
tion) did show recovery of fear 24 hours uncovered in early classical conditioning
after reactivation. These results indicate that experiments in Pavlov’s laboratory (Pavlov,
new learning that occurs during a period 1927) where he found that animals con-
of memory reconsolidation can control the ditioned to salivate to a specific stimulus
expression of fear. The clinical implica- (e.g., a tone of 1,000 Hz) would salivate to
tions for this procedure are particularly rel- other stimuli that were perceptually simi-
evant for treating anxiety disorders, espe- lar to the CS (e.g., tones above and below
cially given that fear memories are often 1,000 Hz). Remarkably, the amount of sali-
persistent and return over time. One caution vation was in proportion to the similar-
about reconsolidation, however, is that sub- ity between the nonconditioned stimulus
tle changes in the experimental parameters and the CS. Because tones do not natu-
can greatly affect the outcome and, conse- rally cause an animal to salivate, Pavlov
quently, the interpretation of the behavioral deduced that learning acquired to one
data (Soeter & Kindt, 2011). Therefore, fur- stimulus transferred to other, perceptually
ther investigations using both pharmacolog- similar stimuli. Other studies of stimulus
ical and nonpharmacological interventions generalization, using instrumental condi-
will be needed to better understand recon- tioning procedures, showed that generaliza-
solidation effects in humans and assess how tion gradients could be quantified on the
well these controlled laboratory studies gen- basis of individual subjects’ response pat-
eralize to the treatment of anxiety disorders terns (Guttman & Kalish, 1956). These gen-
(Schiller & Phelps, 2011). eralization gradients were found to track
similarity along the sensory dimension itself
and were not influenced by the organism’s
ability to perceptually discriminate between
Future Issues the CS and another stimulus. This find-
ing formed the basis for the contempo-
Cellular and systems neuroscience research rary view that stimulus generalization is
that is based on animal models has steadily in large measure a cognitive process, not
advanced for several decades since Pavlov’s simply a failure to discriminate between
original discoveries. The cognitive neuro- conditioned and nonconditioned stimuli
science of human fear learning has advanced (Shepard, 1987).
more recently, facilitated by functional brain Fear generalization occurs when the fear-
imaging techniques. Many important chal- related CR is evoked by stimuli other
lenges still exist in extending findings from than the CS. In humans, fear generaliza-
animal research to human populations, tion is perhaps best demonstrated in anxi-
especially for developing novel clinical treat- ety disorders characterized by excessive fear
EMOTIONAL LEARNING AND MEMORY 437
responses to innocuous cues, such as in post- related with individual differences in trait
traumatic stress disorder (PTSD). In PTSD, anxiety.
a number of stimuli can act as reminders of a The limited number of human behav-
traumatic event and can lead to a state of fear ioral and neuroimaging studies examining
similar to that experienced during the initial fear generalization show that stimuli that
trauma. Behavioral studies have reported never enter into direct association with a
generalization gradients of the CR follow- US, but are perceptually similar to a CS,
ing fear conditioning when participants are evoke a generalized CR. However, there are
presented with stimuli that resemble the CS still many questions regarding the biobehav-
to varying degrees (Dunsmoor, Mitroff, & ioral mechanisms governing generalization
LaBar, 2009; Lissek et al., 2008). Generaliza- in humans and their role in anxiety disor-
tion gradients are broader in patients with ders.
panic disorder, indicating that fears are read-
ily elicited to a wider range of stimuli in this
population (Lissek et al., 2010). Genetics of Human Fear Learning
Dunsmoor et al. (2009) demonstrated
a unique form of fear generalization in The genetic basis of fear conditioning has
humans that was found to be influenced received increasing interest in human behav-
by the amount of fear intensity in non- ioral and imaging research. One advan-
conditioned stimuli (Figure 18.6). During tage to using this model system is that the
fear conditioning, subjects received pairings effects of pharmacological agents and neu-
of a face expressing a moderate amount rotransmitters are well documented from
of fear intensity (CS+) with an electrical analogous studies in nonhuman animals, so
shock US. A control stimulus (CS−) was the putative actions of the genetic mark-
also presented during fear conditioning and ers can be inferred more readily. Early find-
was alternated between groups as either the ings have demonstrated a link between the
most intense or least intense fear face. Before expression of conditioned fear in humans
fear conditioning, stimuli evoked undiffer- and genes involved in emotional reactivity
entiated SCRs; after fear conditioning, how- (i.e., serotonin transporter gene, 5-HTTLPR
ever, a gradient was observed that peaked polymorphism) and neuroplasticity (i.e.,
at stimuli of greater intensity than the CS+. brain-derived neurotrophic factor, BDNF;
Interestingly, this gradient was reduced for Garpenstrand, Annas, Ekblom, Oreland, &
the group that received discriminative fear Fredrikson, 2001; Mahan & Ressler, 2012). For
conditioning between the same CS+ and example, Garpenstrand et al. (2001) found
a CS− that expressed the most fear, sug- that strong fear conditioning, as assessed
gesting that associative learning processes by the SCR, was related to the short sero-
acting against a natural gradient of fear tonin transporter promoter allele, whereas
intensity can influence the extent to which delayed extinction (that is, sustained fear
individuals generalize conditioned fear. responses over the course of extinction
These behavioral findings were extended trials) was related to a long dopamine
to fMRI, wherein brain activity related to D4 receptor allele. Findings related to the
fear generalization was observed in areas genetic basis of fear conditioning, in combi-
involved in initial fear learning, including nation with evidence on the role of specific
the striatum, insula, thalamus, and periac- brain regions, are helping inform compre-
queductal gray (Dunsmoor, Prince, Murty, hensive models of fear learning that may be
Kragel, & LaBar, 2011). Moreover, the behav- important for profiling certain anxiety dis-
ioral expression of fear generalization (i.e., orders. However, it is important to note
SCRs) was correlated with activity in the that there are only a limited number of
amygdala and insula, and functional connec- genetic studies of human fear learning at this
tivity between the amygdala and fusiform time, and further research is needed in this
gyrus during fear generalization was cor- area, especially when extrapolating results
438 JOSEPH E. DUNSMOOR & KEVIN S. LABAR
0.6
0.6
√μS)
√μS)
0.4
SCR (√μ
SCR (√μ
0.4
0.2
0.2 Pre-Conditioning
0
Generalization Pre-Conditioning
Generalization
Difference Difference
0 –0.2
to understand the molecular basis of anxiety ders involves the use of virtual reality envi-
disorders. ronments. The advantage of this technol-
ogy is that it simulates real-world settings,
including immersive 3-D applications that
Laboratory and Real-World Fear
can be tailored to the fearful stimuli or con-
Learning Using Virtual Reality
texts experienced by a particular patient
The study of human fear conditioning has or groups of patients. This technology is
typically involved laboratory or neuroimag- being employed not only to simulate out-
ing examinations that make use of two- comes of extinction-based therapies in anxi-
dimensional objects as CSs presented to the ety disorders but also as a basic research tool
subject over a computer monitor. Although to investigate context specificity of condi-
these studies have provided a model of real- tioned fears (Huff et al., 2011). As reviewed
world encounters with feared stimuli, they earlier, previous animal research has shown
do not approximate some aspects of com- that extinction learning is specific to the
plex fearful experiences in the real world. context in which it was acquired (Bou-
For instance, environmental CSs are often ton, 2004). This effect has proved problem-
encountered during a dynamic exchange atic for the treatment of anxiety disorders,
rather than passively presented, and real- because anxieties often relapse outside the
world contexts are far richer than a typical confines of the therapist’s office. Human
laboratory setting. A frontier in the study research on the context specificity of extinc-
of fear learning that has direct implications tion learning is limited by the ability to
for understanding and treating anxiety disor- manipulate the environment in a behavioral
EMOTIONAL LEARNING AND MEMORY 439
testing room. Immersive virtual reality, fear responses? What environmental in-
however, promises to shed light on how fluences mitigate against genetic risk fac-
fears return when CSs are encountered in a tors?
variety of lifelike contexts, which allows for r Can fear memories be selectively erased
an in-depth examination of different param- using knowledge of the neuroscience of
eters that mediate the contextual recall of memory? Can these practices be success-
extinction memories. fully modified for use as therapeutic treat-
ment of clinical anxiety disorders? What
ethical issues arise when applying fear-
Concluding Remarks erasure techniques to treat anxiety disor-
ders?
In this chapter, we highlighted behavioral
and neuroimaging research on human fear
learning that has been informed and sup- Acknowledgments
ported by advancements in nonhuman ani-
mal research. The human brain mechanisms This work was supported in part by NSF
involved in the acquisition, expression, and grant 0745919 and NIH grants 2 P01 NS041328
extinction of fear are becoming increasingly and R01 DA027802. J. E. D. was supported
well delineated. This knowledge is being by National Research Service Award no.
used to develop advanced neurobehavioral F31MH090682,
models of anxiety disorders that have led
to innovative treatments, including the use
of pharmacological agents that putatively References
enhance learning systems to strengthen the
effect of exposure therapy. Nevertheless, Alvarez, R. P., Biggs, A., Chen, G., Pine, D. S.,
future research will be needed to answer & Grillon, C. (2008). Contextual fear con-
many remaining questions in the realm of ditioning in humans: Cortical-hippocampal
human and nonhuman fear learning. and amygdala contributions. Journal of Neu-
roscience, 28(24), 6211–19.
Anagnostaras, S. G., Maren, S., & Fanselow, M. S.
Outstanding Questions and Future (1999). Temporally graded retrograde amnesia
Directions of contextual fear after hippocampal damage
in rats: Within-subjects examination. Journal
r Under what circumstances do humans of Neuroscience, 19(3), 1106–14.
Bechara, A., Tranel, D., Damasio, H., Adolphs,
generalize fear from a single learning R., Rockland, C., & Damasio, A. R. (1995).
episode? Is fear generalization influ- Double dissociation of conditioning and
enced solely by the perceptual similar- declarative knowledge relative to the amyg-
ity between the CS and a noncondi- dala and hippocampus in humans. Science,
tioned stimulus, or is generalization also 269(5227), 1115–18.
influenced by the conceptual or categor- Bekinschtein, T. A., Shalom, D. E., Forcato, C.,
ical relationship between stimuli or situ- Herrera, M., Coleman, M. R., Manes, F. F., &
ations? What neural systems mediate the Sigman, M. (2009). Classical conditioning in
generalization of learned fear, and what the vegetative and minimally conscious state.
Nature Neuroscience, 12(10), 1343–49.
procedures (e.g., extinction, cognitively
Blanchard, R. J., Blanchard, D. C., Agullana, R.,
based emotion regulation) best limit fear & Weiss, S. M. (1991). 22 khz alarm cries to
generalization? presentation of a predator, by laboratory rats
r What is the genetic basis of fear learning?
living in visible burrow systems. Physiology &
Are some individuals genetically predis- Behavior, 50(5), 967–72.
posed to form associations between stim- Bouton, M. E. (2004). Context and behavioral
uli and aversive outcomes, and are some processes in extinction. Learning & Memory,
individuals better equipped to inhibit 11(5), 485–94.
440 JOSEPH E. DUNSMOOR & KEVIN S. LABAR
Hamm, A. O., & Vaitl, D. (1996). Affective learn- LaBar, K. S., & Disterhoft, J. F. (1998). Con-
ing: Awareness and aversion. Psychophysiol- ditioning, awareness, and the hippocampus.
ogy, 33(6), 698–710. Hippocampus, 8(6), 620–26.
Hofmann, S. G., Meuret, A. E., Smits, J. A. LaBar, K. S., Gatenby, J. C., Gore, J. C., LeDoux,
J., Simon, N. M., Pollack, M. H., Eisen- J. E., & Phelps, E. A. (1998). Human amygdala
menger, K., . . . Otto, M. W. (2006). Augmen- activation during conditioned fear acquisition
tation of exposure therapy with D-cycloserine and extinction: A mixed-trial fMRI study.
for social anxiety disorder. Archives of General Neuron, 20(5), 937–45.
Psychiatry, 63(3), 298–304. LaBar, K. S., LeDoux, J. E., Spencer, D. D., &
Huff, N. C., Hernandez, J. A., Fecteau, M. E., Phelps, E. A. (1995). Impaired fear condition-
Zielinski, D. J., Brady, R., & Labar, K. S. (2011). ing following unilateral temporal lobectomy
Revealing context-specific conditioned fear in humans. Journal of Neuroscience, 15(10),
memories with full immersion virtual reality. 6846–55.
Frontiers in Behavioral Neuroscience, 5, 75. LaBar, K. S., & Phelps, E. A. (2005). Reinstate-
Jackson, E. D., Payne, J. D., Nadel, L., & Jacobs, ment of conditioned fear in humans is context
W. J. (2006). Stress differentially modulates dependent and impaired in amnesia. Behav-
fear conditioning in healthy men and women. ioral Neuroscience, 119(3), 677–86.
Biological Psychiatry, 59(6), 516–22. Ledgerwood, L., Richardson, R., & Cranney, J.
Jeon, D. E. A. (2010). Observational fear learn- (2003). Effects of D-cycloserine on extinc-
ing involves affective pain system and Cav1.2 tion of conditioned freezing. Behavioral Neu-
Ca2 +channels in ACC. Nature Neuroscience, roscience, 117(2), 341–49.
13(4), 482–88. LeDoux, J. E. (1996). The emotional brain. New
Kim, J., Lee, S., Park, H., Song, B., Hong, I., York: Simon and Schuster.
Geum, D., . . . Choi, S. (2007). Blockade of LeDoux, J. E. (2000). Emotion circuits in the
amygdala metabotropic glutamate receptor brain. Annual Review of Neuroscience, 23, 155–
subtype 1 impairs fear extinction. Biochemi- 84.
cal and Biophysical Research Communications, LeDoux, J. E., & Gorman, J. M. (2001). A call
355(1), 188–93. to action: Overcoming anxiety through active
Kimble, G. A., & Ost, J. W. P. (1961). Condi- coping. American Journal of Psychiatry, 158(12),
tioned inhibitory process in eyelid condition- 1953–55.
ing. Journal of Experimental Psychology, 61(2), Lissek, S., Biggs, A. L., Rabin, S. J., Cornwell,
150–56. B. R., Alvarez, R. P., Pine, D. S., & Grillon,
Kindt, M., & Soeter, M. (2010). Dissociating C. (2008). Generalization of conditioned fear-
response systems: Erasing fear from memory. potentiated startle in humans: Experimental
Neurobiology of Learning and Memory, 94(1), validation and clinical relevance. Behaviour
30–41. Research and Therapy, 46(5), 678–87.
Kindt, M., Soeter, M., & Vervliet, B. (2009). Lissek, S., Rabin, S., Heller, R. E., Luken-
Beyond extinction: Erasing human fear baugh, D., Geraci, M., Pine, D. S., & Grillon,
responses and preventing the return of fear. C. (2010). Overgeneralization of conditioned
Nature Neuroscience, 12(3), 256–58. fear as a pathogenic marker of panic disor-
Kluver, H., & Bucy, P. C. (1939). Preliminary der. American Journal of Psychiatry, 167(1),
analysis of functions of the temporal lobes in 47–55.
monkeys. Archives of Neurology and Psychia- Lovibond, P. F., & Shanks, D. R. (2002). The
try, 42(6), 979–1000. role of awareness in Pavlovian conditioning:
Knight, D. C., Nguyen, H. T., & Bandettini, Empirical evidence and theoretical implica-
P. A. (2005). The role of the human amyg- tions. Journal of Experimental Psychology – Ani-
dala in the production of conditioned fear mal Behavior Processes, 28(1), 3–26.
responses. Neuroimage, 26(4), 1193–1200. Mahan, A. L., & Ressler, K. J. (2012). Fear condi-
Knight, D. C., Waters, N. S., & Bandettini, tioning, synaptic plasticity and the amygdala:
P. A. (2009). Neural substrates of explicit and Implications for posttraumatic stress disorder.
implicit fear memory. Neuroimage, 45(1), 208– Trends in Neurosciences, 35(1), 24–35.
14. McGaugh, J. L. (2004). The amygdala modulates
LaBar, K. S., & Cabeza, R. (2006). Cognitive the consolidation of memories of emotionally
neuroscience of emotional memory. Nature arousing experiences. Annual Review of Neu-
Reviews Neuroscience, 7(1), 54–64. roscience, 27, 1–28.
442 JOSEPH E. DUNSMOOR & KEVIN S. LABAR
McGaugh, J. L., Cahill, L., & Roozendaal, B. Olsson, A., & Phelps, E. A. (2007). Social learning
(1996). Involvement of the amygdala in mem- of fear. Nature Neuroscience, 10(9), 1095–1102.
ory storage: Interaction with other brain sys- O’Reilly, R. C., & Rudy, J. W. (2001). Conjunc-
tems. Proceedings of the National Academy of tive representations in learning and memory:
Sciences, 93(24), 13508–14. Principles of cortical and hippocampal func-
McNally, G. P., Johansen, J. P., & Blair, H. T. tion. Psychological Review, 108(2), 311–45.
(2011). Placing prediction into the fear circuit. Pape, H. C., & Pare, D. (2010). Plastic synaptic
Trends in Neurosciences, 34(6), 283–92. networks of the amygdala for the acquisition,
Milad, M. R., Pitman, R. K., Ellis, C. B., Gold, expression, and extinction of conditioned fear.
A. L., Shin, L. M., Lasko, N. B., . . . Rauch, Physiological Reviews, 90(2), 419–63.
S. L. (2009). Neurobiological basis of failure Pavlov, I. P. (1927). Conditioned reflexes. London:
to recall extinction memory in posttraumatic Oxford University Press.
stress disorder. Biological Psychiatry, 66(12), Phelps, E. A., Delgado, M. R., Nearing, K. I., &
1075–82. LeDoux, J. E. (2004). Extinction learning in
Milad, M. R., & Quirk, G. J. (2002). Neurons humans: Role of the amygdala and vmPFC.
in medial prefrontal cortex signal memory for Neuron, 43(6), 897–905.
fear extinction. Nature, 420(6911), 70–74. Phelps, E. A., O’Connor, K. J., Gatenby, J. C.,
Mineka, S., Davidson, M., Cook, M., & Keir, R. Gore, J. C., Grillon, C., & Davis, M. (2001).
(1984). Observational conditioning of snake Activation of the left amygdala to a cognitive
fear in rhesus-monkeys. Journal of Abnormal representation of fear. Nature Neuroscience,
Psychology, 93(4), 355–72. 4(4), 437–41.
Mineka, S., & Zinbarg, R. (2006). A contempo- Phillips, R. G., & LeDoux, J. E. (1992). Differen-
rary learning theory perspective on the etiol- tial contribution of amygdala and hippocam-
ogy of anxiety disorders – It’s not what you pus to cued and contextual fear conditioning.
thought it was. American Psychologist, 61(1), Behavioral Neuroscience, 106(2), 274–85.
10–26. Quirk, G. J., Russo, G. K., Barron, J. L., &
Monfils, M. H., Cowansage, K. K., Klann, E., & Lebron, K. (2000). The role of ventromedial
LeDoux, J. E. (2009). Extinction-recon- prefrontal cortex in the recovery of extin-
solidation boundaries: Key to persistent atten- guished fear. Journal of Neuroscience, 20(16),
uation of fear memories. Science, 324(5929), 6225–31.
951–55. Rachman, S. (1977). Conditioning theory of fear-
Morris, J. S., Öhman, A., & Dolan, R. J. acquisition – critical-examination. Behaviour
(1998). Conscious and unconscious emotional Research and Therapy, 15(5), 375–87.
learning in the human amygdala. Nature, Rescorla, R. A., & Wagner, A. R. (1972). A the-
393(6684), 467–70. ory of Pavlovian conditioning: Variations in the
Nader, K., & Hardt, O. (2009). A single stan- effectiveness of reinforcement and nonreinforce-
dard for memory: The case for reconsolida- ment. New York: Appleton-Century-Crofts.
tion. Nature Reviews Neuroscience, 10(3), 224– Ressler, K. J., Rothbaum, B. O., Tannen-
34. baum, L., Anderson, P., Graap, K., Zimand,
Nader, K., Schafe, G. E., & Le Doux, J. E. (2000). E., . . . Davis, M. (2004). Cognitive enhancers
Fear memories require protein synthesis in the as adjuncts to psychotherapy – Use of D-
amygdala for reconsolidation after retrieval. cycloserine in phobic individuals to facilitate
Nature, 406(6797), 722–26. extinction of fear. Archives of General Psychi-
Ochsner, K. N., & Gross, J. J. (2008). Cognitive atry, 61(11), 1136–44.
emotion regulation: Insights from social cogni- Rodrigues, S. M., LeDoux, J. E., & Sapolsky, R.
tive and affective neuroscience. Current Direc- M. (2009). The Influence of stress hormones
tions in Psychological Science, 17(2), 153–58. on fear circuitry. Annual Review of Neuro-
Öhman, A., & Mineka, S. (2001). Fears, phobias, science, 32, 289–313.
and preparedness: Toward an evolved module Schafe, G. E., Nader, K., Blair, H. T., & LeDoux,
of fear and fear learning. Psychological Review, J. E. (2001). Memory consolidation of Pavlo-
108(3), 483–522. vian fear conditioning: A cellular and molecu-
Olsson, A., & Phelps, E. A. (2004). Learned fear lar perspective. Trends in Neurosciences, 24(9),
of “unseen” faces after Pavlovian, observa- 540–46.
tional, and instructed fear. Psychological Sci- Schiller, D., Monfils, M. H., Raio, C. M., John-
ence, 15(12), 822–28. son, D. C., LeDoux, J. E., & Phelps, E.
EMOTIONAL LEARNING AND MEMORY 443
A. (2010). Preventing the return of fear in Walker, D. L., Ressler, K. J., Lu, K. T., &
humans using reconsolidation update mech- Davis, M. (2002). Facilitation of conditioned
anisms. Nature, 463(7277), 49–51. fear extinction by systemic administration or
Schiller, D., & Phelps, E. A. (2011). Does reconsol- intra-amygdala infusions of D-cycloserine as
idation occur in humans? Frontiers in Behav- assessed with fear-potentiated startle in rats.
ioral Neuroscience, 5, 24. Journal of Neuroscience, 22(6), 2343–51.
Sehlmeyer, C., Schoning, S., Zwitserlood, P., Watson, J. B., & Rayner, R. (1920). Conditioned
Pfleiderer, B., Kircher, T., Arolt, V., & Kon- emotional reactions. Journal of Experimental
rad, C. (2009). Human fear conditioning Psychology, 3, 1–14.
and extinction in neuroimaging: A systematic Weike, A. I., Hamm, A. O., Schupp, H. T.,
review. PloS One, 4(6), 16. Runge, U., Schroeder, H. W. S., & Kessler,
Seligman, M. E. (1970). On the generality of the C. (2005). Fear conditioning following unilat-
laws of learning. Psychological Review, 77(5), eral temporal lobectomy: Dissociation of con-
406–18. ditioned startle potentiation and autonomic
Seligman, M. E. (1971). Phobia and preparede- learning. Journal of Neuroscience, 25(48), 11117–
ness. Behavior Therapy, 2(3), 307–20. 24.
Shanks, D. R. (2010). Learning: From association Weiskrantz, L. (1956). Behavioral changes associ-
to cognition. Annual Review of Psychology, 61, ated with ablation of the amygdaloid complex
273–301. in monkeys. Journal of Comparative and Phys-
Shepard, R. N. (1987). Toward a universal law of iological Psychology, 49(4), 381–91.
generalization for psychological science. Sci- Wilensky, A. E., Schafe, G. E., Kristensen, M. P.,
ence, 237(4820), 1317–23. & LeDoux, J. E. (2006). Rethinking the fear
Soeter, M., & Kindt, M. (2011). Disrupting recon- circuit: The central nucleus of the amygdala
solidation: Pharmacological and behavioral is required for the acquisition, consolidation,
manipulations. Learning & Memory, 18(6), and expression of Pavlovian fear conditioning.
357–66. Journal of Neuroscience, 26(48), 12387–96.
Squire, L. R. (1986). Mechanisms of memory. Sci- Zorawski, M., Blanding, N. Q., Kuhn, C. M., &
ence, 232(4758), 1612–19. LaBar, K. S. (2006). Effects of stress and sex on
Squire, L. R., & Zola, S. M. (1996). Struc- acquisition and consolidation of human fear
ture and function of declarative and non- conditioning. Learning & Memory, 13(4), 441–
declarative memory systems. Proceedings of the 50.
National Academy of Sciences, 93(24), 13515– Zorawski, M., Cook, C. A., Kuhn, C. M., &
22. LaBar, K. S. (2005). Sex, stress, and fear:
Vuilleumier, P. (2005). How brains beware: Neu- Individual differences in conditioned learning.
ral mechanisms of emotional attention. Trends Cognitive Affective & Behavioral Neuroscience,
in Cognitive Sciences, 9(12), 585–94. 5(2), 191–201.
CHAPTER 19
Reward Learning
Contributions of Corticobasal Ganglia Circuits
to Reward Value Signals
Throughout our lives, we learn to value mary rewards, whether they rely on a com-
behaviors and stimuli in our environment mon neural reward circuitry, and whether
that are likely to bring satisfaction or learning socially relevant information occurs
achievement. We might learn to value cer- via similar reward-learning mechanisms.
tain foods, for example, because they are The goal of this chapter is to provide
healthier or satisfy our hunger more so than an overview of neural mechanisms involved
others, or we may learn to value something in reward learning, concentrating largely
such as a school bell in the middle of the day on corticobasal ganglia circuits. Detailing
that signals that it is time to eat. As humans, findings primarily from human neuroimag-
we also come to value more secondary, less ing research, a specific focus is on how
immediate reinforcers such as money and these neural circuits contribute to comput-
the actions that allow the accumulation of ing value signals for both natural and more
wealth (e.g., working overtime for bonus abstract social rewards and how these value
pay). Research across species has implicated signals contribute to learning.
a specific neural circuitry subserving these
goal-directed behaviors, centering largely on
corticobasal ganglia circuits. Valued social The Neuroanatomy of Reward
outcomes and considerations play additional Processing
motivating roles in our environment, as we
often strive for social approval or praise and A large body of nonhuman animal research
are motivated during social interactions by has sought to delineate neural correlates
preferences for valued actions, such as reci- underlying reward processing at both the
procity and fairness. These social values aid cellular and more circuit-based level. A
in learning about others and inform our common finding across many of these inves-
social behavior. Recent investigations have tigations is the involvement of corticobasal
begun to examine whether social rewards ganglia loops as a key component of a re-
are valued and experienced similarly to pri- ward-processing circuit (Haber & Knutson,
444
REWARD LEARNING 445
2010; Sesack & Grace, 2010). These func- originate via the SN/GPi and mediodorsal
tional loops are formed based on both com- and posterior nuclei of the thalamus (for a
munication between structures within the review see Sesack & Grace, 2010).
basal ganglia and prefrontal cortex and on The ventral striatum shares connections
modulatory influence by dopaminergic neu- with ventral cortical and subcortical regions.
rons in midbrain regions; for example, the It comprises ventral portions of the cau-
substantia nigra (SN) and ventral tegmen- date nucleus and putamen, as well as the
tal area (VTA). The integrity of connections nucleus accumbens (NAcc), which can be
and of interactions between these areas sub- divided into core (medial) and shell (lat-
serves both motor and cognitive behavior eral) subcomponents (for a review see
(for a review see Middleton & Strick, 2000). Sesack & Grace, 2010). Findings from non-
Given its heterogeneity in terms of connec- human primates indicate that the ventral
tivity and functionality, the basal ganglia and caudate and ventral putamen receive affer-
associated projections are a key component ents from the orbitofrontal cortex (OFC)
of a putative reward circuit and are the focus and dorsal anterior cingulate cortex (dACC;
of the research described in this chapter. Haber & Knutson, 2010). The NAcc, in
The basal ganglia comprise several sub- contrast, receives excitatory afferents from
regions including the striatum, globus the prefrontal cortex (e.g., ventromedial
pallidus (internal [GPi] and external [GPe] prefrontal cortex, vmPFC) and subcorti-
components), and subthalamic nucleus (for cal structures including the midline and
a review see Middleton & Strick, 2000), with intralaminar nuclei of the thalamus, baso-
the striatum receiving vital projections from lateral amygdala, and ventral subiculum of
both prefrontal and midbrain dopaminergic the hippocampus (Haber & Knutson, 2010;
regions. The striatum serves as the input Sesack & Grace, 2010). The ventral stria-
unit of the basal ganglia, in turn playing tum, particularly the NAcc, has been shown
an integral role in processing and integrat- to be integral for both appetitive and con-
ing reward-related information (Haber & summatory aspects of reward processing, as
Knutson, 2010). The striatum can be further well for learning associations between stim-
subdivided into dorsal and ventral compo- uli and rewards (Belin, Jonkman, Dickinson,
nents (Figure 19.1), each with important Robbins, & Everitt, 2009). Further, the struc-
afferent and efferent connections (Haber tures connecting with the ventral striatum
& Knutson, 2010). The caudate nucleus and have been implicated in different aspects
putamen comprise the dorsal striatum and of affective learning and reward evaluation
share connections with higher cognitive, and representation (for reviews, see Haber &
motor, and sensory regions. The dorsal Knutson, 2010; Kringelbach, 2005; Sesack &
striatum receives afferents from the dorso- Grace, 2010). The NAcc also has reciprocal
lateral prefrontal cortex (DLPFC; Haber & inhibitory connections with the VTA, from
Knutson, 2010), frontal eyefields, and motor which dopamine modulates NAcc activity
cortex (Alexander, Crutcher, & DeLong, (Sesack & Grace, 2010). These dopaminer-
1990; Middleton & Strick, 2000), connections gic projections from the VTA have been
that may facilitate reward processing. Pro- suggested to be critical for the learning of
jections from the anterior medial prefrontal reward-related information (Niv, 2009).
cortex to medial portions of the dorsal Other functional and structural neural
striatum, for instance, have been postulated models exist that complement and extend
to be important for learning reward-related the current knowledge of reward processing
actions (e.g., Ostlund & Balleine, 2005), gained from the ideas of corticobasal gan-
whereas projections from the sensorimotor glia functional loops and the dorsal-ventral
cortex to the lateral portions of dorsal divide of the striatum. One such model
striatum are important for habit learning focuses on information processing and inte-
(e.g., Barnes, Kubota, Hu, Jin, & Graybiel, gration within striatal subregions, proposing
2005). Both of these pathways project a striato-nigral-striatal loop that facilitates
back to regions of cortex from which they communication within the striatum via
446 DOMINIC S. FARERI & MAURICIO R. DELGADO
Figure 19.1. Striatum subdivisions. Dorsal striatum includes the caudate nucleus and putamen.
Ventral striatum includes the nucleus accumbens (NAcc), which is subdivided into core and shell
components, as well as ventral portions of the caudate nucleus and putamen. The NAcc is critically
connected with the dopaminergic midbrain, particularly the substantia nigra (SN) and ventral
tegmental area (VTA); the NAcc shell projects to the substantia nigra pars compacta (SNc), whereas
the NAcc core sends projections to the SNc and substantia nigra pars reticulata (SNr). Both
subdivisions of the NAcc share reciprocal connections with the VTA.
receptors) showed decreased responding for value of actions and stimuli in their environ-
food over time in comparison to controls, ment that will best allow the achievement
advocating a role for dopamine in the rein- of positive outcomes. This notion is related
forcing effects of rewarding stimuli (e.g., to computational ideas put forth by rein-
food; for a review, see Wise, 2004). This forcement learning theory, which holds that
theory has been revised over the years to agents attempt to learn which actions to take
propose that dopamine is important for when navigating an environment to achieve
motivation (e.g., lever pressing for food) optimal reward (for a review, see Niv,
and that elevated levels aid in reinforcing 2009).
action-outcome associations (Wise, 2004). A Evidence of this learning signal in non-
third hypothesis is that midbrain dopamine human primates spurred the search for
signals aid in the learning or reinforce- similarities within the human brain. How-
ment of novel actions (see Redgrave & ever, much of this work has been indirect,
Gurney, 2006). This hypothesis oper- employing neuroimaging techniques such
ates on the premise that dopaminer- as positron emission tomography (PET)
gic signaling actually responds to preat- and functional magnetic resonance imag-
tentive sensory processing, because the ing (fMRI). Taking advantage of the abil-
time course of phasic dopamine signals ity of radioactive tracers (e.g., raclopride) to
does not allow (i.e., they occur too bind to dopamine receptors in the striatum,
quickly) coding for the value of unexpected PET allows for the measurement of changes
rewards. in striatal dopaminergic release; decreased
A functional model of dopaminergic tracer binding potential is thought to indi-
function that has been highly influen- cate dopamine release into the striatum.
tial comes from elegant electrophysiolog- Because cocaine functions by blocking the
ical recordings from midbrain dopamine reuptake of striatal dopamine, PET studies
neurons in nonhuman primates (Schultz, have revealed impaired dopaminergic func-
Dayan, & Montague, 1997). In these record- tion in cocaine addicts as compared to con-
ings, it was observed that dopamine neurons trol subjects: Addicts show decreased stri-
responded to the delivery of unexpected atal binding potential for raclopride both
rewards (i.e., when unaware of contingen- when given methylphenidate (similar effects
cies leading to juice delivery) and the earliest to cocaine) and a placebo (Volkow et al.,
predictor of a reward (i.e., a conditioned cue 1997).
that signaled an expected reward). These Measuring the blood-oxygen-level-
neurons further showed a depressed firing dependent (BOLD) response, fMRI
rate when an expected reward was omit- provides an alternative, indirect measure
ted. Taken together, these findings sug- of neural activity that is thought to reflect
gest that dopamine neurons code a learn- synaptic input to a region (Logothetis,
ing signal known as a prediction error (e.g., Pauls, Augath, Trinath, & Oeltermann,
the difference between the expected and 2001; see Chapter 5). High-resolution fMRI
received reward; Niv & Schoenbaum, 2008). scanning demonstrates that increases in
This theory continues to be developed and VTA BOLD activation correspond to a
linked to other specific processes, such as positive reward prediction error signal
temporal prediction learning signals (for a to unexpected primary (liquid) and sec-
review see Niv, 2009) while being quanti- ondary (money) rewards (D’Ardenne,
fied even further by suggestions that the McClure, Nystrom, & Cohen, 2008).
error signal is computed by comparing the Thus, although there is some evidence
average previous reward history and cur- of reward-related dopamine function in
rent experienced reward, with a specificity humans, most neuroimaging investigations
for positive prediction errors (i.e., unex- of the human brain have focused primarily
pected rewards; Bayer & Glimcher, 2005). on the targets of midbrain dopaminergic
In sum, reward-related midbrain dopamin- regions, such as the striatum and prefrontal
ergic activity serves to inform animals of the cortex.
448 DOMINIC S. FARERI & MAURICIO R. DELGADO
Figure 19.2. Expectation of high-magnitude monetary rewards elicits significant increases in ventral
striatum (NAcc) BOLD activity. Adapted from “Anticipation of monetary reward selectively recruits
nucleus accumbens,” by B. Knutson, C.M. Adams, G.W. Fong, and D. Hommer, 2001, Journal of
Neuroscience, 21, RC159. Adapted with permission.
Recent work also has shown that the striatal a sensitivity to the specific context in which
BOLD response nonlinearly increases with value is represented (De Martino, Kumaran,
expected reward probability during choices Holt, & Dolan, 2009).
between risky options (Hsu, Krajbich, Zhao, Advancing this idea, reward-outcome-
& Camerer, 2009), supporting a potential related activity in the striatum has been
role for the striatum in coding subjective shown to be modulated by the manner in
value – which can be thought of as a series which a reward is obtained. For example,
of considerations based on one’s percep- increases in both dorsal and ventral stria-
tion of the objective properties of available tum activity (caudate nucleus and NAcc)
choices and outcomes (for extensive review have been reported when reward deliv-
of valuation and decision making, see Doya, ery was contingent on a successful button
2008). press (Zink, Pagnoni, Martin-Skurski, Chap-
Reward value signals are also generated pelow, & Berns, 2004), although in that
on the receipt or consumption of poten- study, the authors interpreted the activ-
tially rewarding outcomes, which can facil- ity as due to the salience of the condi-
itate subsequent behavior and formation tion. An alternative interpretation is that
of expectations. Initial fMRI experiments the dorsal striatum, specifically the caudate
used monetary reinforcers to demonstrate nucleus, is critical for establishing action-
the involvement of cortical regions such as outcome contingencies. Evidence indicates
the OFC (O’Doherty, Kringelbach, Rolls, that the caudate nucleus comes online
Hornak, & Andrews, 2001) and medial pre- when an action is required during the pre-
frontal cortex (Knutson et al., 2003) in the sentation of a reward-predicting cue, and
affective valuation of rewards at the time specifically when participants believed the
of outcome. The role of the dorsal and ven- valence of the outcome (e.g., monetary
tral striatum in outcome processing was sug- reward/punishment) was contingent on this
gested to involve differentiating between response (Tricomi, Delgado, & Fiez, 2004).
positive (gains) and negative (losses) out- This finding is not limited to the use of mon-
comes (Delgado, Nystrom, Fissell, Noll, & etary reinforcers. Outcomes reflecting per-
Fiez, 2000). The dorsal striatal response to formance and goal achievement on declar-
outcome-related reward value signals is par- ative memory tasks elicit caudate nucleus
ticularly interesting, given that it is mod- activity similarly to monetary reinforcers
ulated by magnitude (Nieuwenhuis et al., (Tricomi & Fiez, 2008); this activity may
2005), probability (Haruno et al., 2004), also be modulated specifically by one’s goals
and motivational properties of the outcome and motivations (Han, Huettel, Raposo,
(Delgado, Stenger, & Fiez, 2004), suggesting Adcock, & Dobbins, 2010). These studies
450 DOMINIC S. FARERI & MAURICIO R. DELGADO
suggest that the human striatum value signal mates suggesting that dopamine neurons
at the time of reward outcome may not be code a temporal prediction error (see Niv,
coding a reward per se, but instead a rein- 2009, for a review).
forcement value signal that facilitates learn- After these first attempts, an explo-
ing (O’Doherty et al., 2004; Tricomi et al., sion of modeling-based fMRI studies has
2004). come about that aimed to fit reinforcement-
learning models to the BOLD response dur-
ing reward-learning paradigms. One influ-
Affective Learning and the Human
ential model is a temporal difference (TD)
Striatum
learning model (for a review, see Niv, 2009)
The notion that expected reward value that, as alluded to, attempts to predict the
signals and reward-outcome value signals outcomes of future events based on incor-
might contribute to an overall neural learn- porating information regarding the tempo-
ing mechanism corresponds with associa- ral relationship between stimuli and rewards
tive learning models such as that proposed that may occur at any time point (see Niv &
by Rescorla and Wagner (1972). This model Schoenbaum, 2008 for a review). Increases
suggests that learning the value of stim- in BOLD signals in the ventral striatum
uli occurs most effectively when there is a have been observed in response to reward-
mismatch between one’s expectations and predicting cues and to both positive and neg-
experience, because it allows for an updating ative temporal prediction errors in Pavlo-
of one’s environmental model. Although the vian learning tasks (O’Doherty, Dayan, Fris-
Rescorla-Wagner model was initially con- ton, Critchley, & Dolan, 2003). Further-
ceived for Pavlovian learning situations, its more, activity in the left ventral putamen
basic principles have since been extended shifts from outcome to cue over time, sug-
to develop learning models that can explain gesting a neural representation for learning
behavior in more varied (e.g., instrumental) (O’Doherty, Dayan, et al., 2003).
and complex situations. A shared underly- Differential contributions of striatal sub-
ing tenet across these more detailed mod- regions during instrumental and Pavlovian
els is that learning occurs when there is an reward-learning tasks have also been pro-
error between predicted and experienced posed (O’Doherty et al., 2004) akin to
outcomes (Niv & Schoenbaum, 2008). a model of reinforcement learning known
Because these models of value learn- as the actor-critic model (for a review,
ing appear to conform to the function- see Niv, 2009). Briefly, this model sug-
ing of the dopamine reward prediction gests that a critic uses a TD learning sig-
error model put forth by Schultz and col- nal to generally evaluate situations, whereas
leagues (1997), recent efforts have exam- an actor keeps track of past outcome his-
ined whether mechanisms of human reward tory in order to choose an effective plan of
learning might function in a similar way. Ini- action (see Niv & Schoenbaum, 2008, for a
tial human work searching for confirmations review). Prediction-error-related activations
of a prediction error learning signal focused were seen in the ventral striatum, particu-
on responses of midbrain dopaminergic tar- larly in the ventral putamen and NAcc in
gets, primarily the striatum, to predicted and both Pavlovian and instrumental learning
unpredicted rewards. The human ventral conditions, supporting a role for the ventral
striatum and the medial OFC region were striatum as critic (O’Doherty et al., 2004).
found to be particularly sensitive to pas- Interestingly, prediction error activity was
sively delivered unpredicted rewards (Berns, also observed in the caudate nucleus, but
McClure, Pagnoni, & Montague, 2001), with only during instrumental learning, which is
the striatum also being sensitive to the tem- congruent with the notion of the dorsal stria-
poral delay of expected rewards (Pagnoni, tum as the actor that puts into action the
Zink, Montague, & Berns, 2002); these find- evaluative information coded by the critic
ings echo evidence from nonhuman pri- (O’Doherty et al., 2004; Tricomi et al., 2004);
REWARD LEARNING 451
this finding correlates with improved behav- in nonhuman primates have long suggested
ioral learning indices (Haruno et al., 2004). that the OFC is involved in reward represen-
Prediction error learning signals have since tation and affective learning, demonstrat-
been reported in the striatum in multiple ing diverse functionality among neuronal
investigations (Daw, Gershman, Seymour, populations within this region (Kringelbach,
Dayan, & Dolan, 2011; Hare, O’Doherty, 2005; Ostlund & Balleine, 2007). Lesions to
Camerer, Schultz, & Rangel, 2008), as well the OFC lead to deficits in learning the
as in the midbrain (D’Ardenne et al., 2008); reward value of a stimulus and adapting
recent findings suggest that these signals play to changing reward contingencies (Clarke,
a role not just in trial-and-error learning situ- Robbins, & Roberts, 2008). Other groups
ations but also during other forms of learning of OFC neurons code for relative prefer-
(Burke, Tobler, Baddeley, & Schultz, 2010) ence or value when choosing among out-
and can be modulated by explicit knowledge come predictive cues, and OFC neurons
about a situation (Li, Delgado, & Phelps, track changing reward expectations and
2011). preferences over time (e.g., before and
Findings of prediction error signals in after learning action-outcome contingen-
humans thus enable a more detailed char- cies; for a review, see Ostlund & Balleine,
acterization of the striatum’s role in reward 2007).
learning and allow for parallels to be made Neuroimaging investigations show sim-
to electrophysiological studies of reward ilar functional dissociations within human
learning and prediction error in nonhuman OFC; it is involved in the hedonic response
primates. Finally, the combination of phar- to rewards – monitoring reward value and
macological agents and neuroimaging (e.g., coding positive and negative outcomes (for
Pessiglione, Seymour, Flandin, Dolan, & review see Kringelbach, 2005). The medial
Frith, 2006) as well as investigations of OFC (mOFC), for example, is more sen-
learning models in populations with defi- sitive to rewards than punishments, with
cient dopaminergic systems such as Parkin- BOLD activity there scaling with increas-
son’s disease (e.g., Rutledge et al., 2009) ing magnitude, whereas the lateral OFC
have helped strengthen the hypothesis that and ventral PFC are more sensitive to pun-
dopaminergic learning signals influence stri- ishments (O’Doherty et al., 2001). Human
atal function during reward-related process- mOFC is also implicated in more com-
ing. plex representation processes, showing sim-
ilar BOLD activation when representing
experienced and imagined rewards (Bray,
Prefrontal Cortex and Reward
Shimojo, & O’Doherty, 2010), as well as
Valuation
contributing to complex subjective value
Prefrontal cortical regions are another key signals, such as how much one is willing to
component of corticobasal ganglia circuits pay for an item – a goal value (Hare et al.,
that aid in the computation of reward value 2008). Such goal value representations in
signals that mediate goal-directed behavior. the mOFC can be susceptible to top-down
The most commonly identified regions in control from more cognitive dorsal regions
the prefrontal cortex involved in reward val- (e.g., DLPFC) when control over behavior
uation include the ventral portions of PFC, is exerted (e.g., when choosing healthy over
medial PFC, dorsolateral PFC, and ante- unhealthy, but favored, food items; Hare,
rior cingulate. Although the specific con- Camerer, & Rangel, 2009).
tributions of these regions to reward pro- Prefrontal cortical regions also serve
cessing are reviewed elsewhere (e.g., Doya, important integrative functions that aid
2008), the role of the orbitofrontal cortex in guiding behavior. Caudal portions of
(OFC) in reward valuation has been a sub- the OFC are important for tracking out-
ject of widespread investigation and is worth come history, thereby facilitating sub-
noting here. Electrophysiological recordings sequent behavioral choices (O’Doherty,
452 DOMINIC S. FARERI & MAURICIO R. DELGADO
Critchley, Deichmann, & Dolan, 2003). Evi- increased response rates during the anticipa-
dence from recordings in nonhuman pri- tory period, whereas punishment-preferring
mates suggests that a proscribed region of neurons showed decreased response rates,
the ACC (ACC sulcus) act in a similar lending credence to a role for the amygdala
manner (Kennerley, Walton, Behrens, Buck- in encoding or representing an organism’s
ley, & Rushworth, 2006), and parallel neu- state values.
roimaging work in humans points to a spe- Neuroimaging investigations have cor-
cific region in the ACC as critical for relying roborated these animal findings, showing
on an overall reward history, particularly decreased BOLD activity in the amygdala
under rapidly changing (volatile) circum- after devaluation of a food item after satia-
stances (Behrens, Woolrich, Walton, & tion (Gottfried, O’Doherty, & Dolan, 2003).
Rushworth, 2007). Prefrontal regions are Patients with bilateral amygdala lesions also
thus critical for representing and comput- showed impairments in the use of posi-
ing various reward-related value signals that tive feedback to guide subsequent behav-
guide understanding of one’s environment ioral choices in comparison to control par-
and behavior. ticipants (Hampton, Adolphs, Tyszka, &
O’Doherty, 2007). In that study, amygdala-
lesioned patients also exhibited irregular
Additional Regions Involved in
expected reward-related BOLD activity in
Reward Processing
the mPFC: BOLD activity increased lin-
Learning about rewards in a dynamic envi- early with increases in expected reward only
ronment is critically dependent on the in control participants, indicating that con-
integrity of corticobasal ganglia circuitry. nectivity between the amygdala and pre-
It is worth noting, however, that other frontal regions contributes to the appropri-
regions that generally support affective pro- ate appraisal of reward values.
cessing and learning also play direct or In addition to the amygdala, the insular
indirect functions in the computation of cortex has been linked with reward process-
reward value signals. Although the amyg- ing at various stages. Although many studies
dala is typically thought of as involved implicate this region in the general affec-
in aversive processing and fear learning – tive processing and experience of pain (for
associating stimuli with aversive outcomes a review, see Hein & Singer, 2008), func-
such as electric shock (for review, see tional activity in the insula has also been
Phelps & LeDoux, 2005) – it has also been linked to predictive learning. Reports impli-
implicated in reward processing and gen- cate the insula in coding a temporal dif-
erating reward value signals. Recordings ference learning signal in aversive learning
from amygdala neurons in nonhuman pri- studies of pain (Seymour et al., 2004). Fur-
mates during an affective reversal learning ther, BOLD activity in the insula has been
task (Paton, Belova, Morrison, & Salzman, reported to correlate with an aversive pre-
2006) show that some amygdala neurons diction error signal in an instrumental learn-
respond after the presentation of predictive ing task for cues predicting losses only (Pes-
cues, whereas other populations respond to siglione et al., 2006). Finally, the insula has
stimuli predicting both positive and nega- been linked with risky behaviors and coding
tive outcomes. Interestingly, amygdala neu- for a risk prediction error (e.g., Preuschoff,
ronal response patterns change with chang- Quartz, & Bossaerts, 2008). These results
ing contingencies. A related study focused highlight the fact that, although reward-
on an anticipatory period before presenta- related processing and learning are highly
tion of reward- or punishment-predicting dependent on corticobasal ganglia loops,
cues (Belova, Paton, & Salzman, 2008), particularly the striatum and prefrontal cor-
revealing that populations of amygdala neu- tex, other regions can also play key roles
rons demonstrate differential response pat- in the evaluation of potential choices and
terns. Reward-preferring neurons showed outcomes.
REWARD LEARNING 453
to a forced transfer to charity (Harbaugh, acquire social value signals to facilitate learn-
Mayr, & Burghart, 2007). Higher tenden- ing and decision making. Employing eco-
cies to donate to charities (as opposed to nomic game paradigms aimed to study inter-
keeping money for oneself) and associated active behavior, recent studies in the bur-
ventral striatal responses are modulated by geoning field of neuroeconomics have begun
the presence of others and expectations of to explore the neural underpinnings of how
social (dis)approval based on one’s decision social information may modulate neural
(Izuma, Saito, & Sadato, 2010). These stud- reward circuitry during social interactions.
ies suggest that attitudes toward others may Economic theory generally holds that typical
play a significant role in goal-directed behav- human behavior should be self-interested;
iors and modulating the processing of social that is, one should always aim to max-
outcomes. imize one’s outcome no matter the cir-
In a similar vein, certain competitive cumstances (Camerer & Fehr, 2006). How-
behaviors or attitudes may bring about ever, behavior in social interactions may
satisfaction or pleasure, particularly in the be informed by other-regarding preferences
context of social comparisons. Outcome- (e.g., empathic concerns), perhaps because
related responses in the ventral striatum are these preferences confer future benefits or
modulated by relative social comparison, are viewed as inherently rewarding (Fehr
showing increases when participants earn & Camerer, 2007). Two-person interactive
more monetary reward than a partner for the economic games have been instrumental
same correct performance, but decreases in in showing that social preferences such as
the reverse situation (Fliessbach et al., 2007). cooperation, reciprocity, trust, and fairness
Similarly, receiving information that mis- motivate behavior during interactions with
fortune has come to envied others recruits others, perhaps in part because these con-
the OFC and dorsal and ventral striatum siderations may be evolutionarily adaptive
(Takahashi et al., 2009), suggesting that this (Axelrod & Hamilton, 1970).
information may be valued as a positive One cooperative economic game known
outcome. A potential motivating explana- as the prisoner’s dilemma game (Rapoport
tion for responses observed during social & Chammah, 1965), for instance, allows two
comparisons is a dislike for social inequity participants to choose to either cooperate or
(Fehr & Camerer, 2007; Tricomi, Rangel, defect for monetary outcomes or some kind
Camerer, & O’Doherty, 2010). When of “good.” Mutual decisions to cooperate
monetary inequity is established between result in slightly lower than maximal pay-
two people, the one with a higher initial outs for both participants. This is the most
endowment exhibits both blunted subjec- commonly occurring outcome, even though
tive feelings of pleasure and responses in the unilateral defection results in the maximum
ventral striatum and vmPFC when receiving possible amount for one person at the com-
subsequent positive financial outcomes in plete expense of the other, who is left with
comparison to his or her partner, whereas nothing (see Fehr & Camerer, 2007, for a
the opposite pattern is observed for those review).
with a lower initial endowment (Tricomi In another interactive economic para-
et al., 2010). Results from these studies thus digm known as the trust game, which
provide evidence that social considerations has been used to examine reciprocity and
and preferences play important modulatory cooperation, one participant (investor) is
roles in the computation of reward value endowed with a sum of money, any amount
signals. of which can be sent to a partner (trustee).
Whatever amount is sent to the trustee is
multiplied, typically by a factor of three or
Social Interactions
four, and the trustee can then choose how
Our daily experiences are frequently marked much to return. Although standard self-
by social interactions from which we can interested behavior suggests that the optimal
REWARD LEARNING 455
decision would be for the investor to send no ACC (sgACC) exhibited increases in BOLD
money, so as to ensure an optimal outcome, activity, and decisions to cooperate follow-
participants often send nonzero amounts, ing reciprocation elicited caudate nucleus
likely motivated by concern for their rep- and ACC activity. Dissociable patterns of
utation and the hope of reciprocity (Berg, outcome-related BOLD activity in the ven-
Dickhaut, & McCabe, 1995). Finally, in a tral striatum and sgACC also occur when
bargaining paradigm (ultimatum game), one contrasting mutual cooperation (BOLD
participant (proposer) proposes a manner increases) and unreciprocated cooperation
of splitting a sum of money with another (e.g., unilateral defection; BOLD decreases;
(responder), who can accept the offer or Rilling, Sanfey, Aronson, Nystrom, &
reject it so both participants get nothing. Cohen, 2004). This latter result might be
Responders typically reject offers they per- explained by a social prediction error sig-
ceive as unfair, although any nonzero offer nal, whereby social expectations may mod-
should be accepted according to economic ulate reward-related circuitry (Rilling et al.,
theory (Güth, Schmittberger, & Schwarze, 2004).
1982). In certain social interactions, agents
Neuroimaging investigations have built do not always act in a trustworthy or
on the groundwork laid by behavioral eco- fair manner (e.g., cooperation may not
nomics to investigate how these social be reciprocated). Such occurrences may
preferences that influence behavior during be viewed as a violation of social norms,
interactions may influence reward expec- which evidence suggests is processed in the
tations and neural reward circuitry. Early anterior insula (for a review, see Rilling &
work began looking at cooperative behav- Sanfey, 2011). Opportunities may arise in
ior in social interactions, using the trust interactions in which we may be able to
and prisoner’s dilemma paradigms. Dur- respond to perceived norm violations, but at
ing a trust game when participants were costs to ourselves. An interesting PET study
awaiting a response after having cooper- examined decisions to punish defections by
ated with a human opponent (McCabe, an opponent in a trust game (de Quervain
Houser, Ryan, Smith, & Trouard, 2001), et al., 2004). Punishments could be costly to
regions of the medial PFC demonstrated the punishers (e.g., they would lose some of
increased BOLD activity as compared to their earnings), free (e.g., it would not cost
when playing with a computer, suggesting them anything to punish), or symbolic (e.g.,
that the social component of an interac- they could send a message of punishment,
tion creates a heightened sense of expec- but with no monetary penalty). Caudate
tation. The medial PFC has additionally nucleus activity was strongest preceding
been implicated in processing social infor- all decisions to punish intentional norm
mation about self and others (see Amodio & violations, whereas the vmPFC and OFC
Frith, 2006, for review) and reward expec- showed increased sensitivity preceding a
tation in certain social contexts (Hampton, costly decision to punish, as compared to
Bossaerts, & O’Doherty, 2008). Confirm- a free punishment. Although these authors
ing findings from behavioral economic stud- suggest that altruistic punishment (e.g.,
ies, mutual cooperation was the behavior punishing despite a cost to oneself) is
of choice when participants played a pris- valued and rewarding, their results might
oner’s dilemma game with real freely acting also be explained by inequity aversion (Fehr
human partners (Rilling et al., 2002), again & Camerer, 2007; Tricomi et al., 2010).
demonstrating a general preference against Behavior in social interactions is therefore
self-interested behavior that is not observed motivated by considerations for others,
when playing with human confederates or along with values acting to confer social
computer partners. Further, on viewing out- success that are coded in neural structures
comes from mutual cooperation, the ventral that also code for primary and secondary
striatum, vmPFC/OFC, and the subgenual reward value.
456 DOMINIC S. FARERI & MAURICIO R. DELGADO
Figure 19.3. a) Depiction of trust game task from Delgado et al. (2005). Participants would choose
whether to keep or share money with one of three fictional characters on any given trial. If
participants chose to share, they would receive either positive (partner shared) or negative (partner
did not share) feedback. If participants chose to keep the money for themselves, they would receive
feedback saying “You have decided to keep the money.” b) Increases in caudate nucleus BOLD
activity were observed for positive versus negative outcomes. c) Time course of caudate nucleus
region-of-interest (ROI) activation during the outcome phase with the neutral partner shows
differential responses to positive (share) and negative (keep) outcomes. d) Time course of caudate
nucleus ROI activation during the outcome phase with the good partner reveals no significant
differences in responses to positive and negative outcomes. Adapted from “Perceptions of moral
character modulate the neural systems of reward during the trust game,” by M.R. Delgado, R.H.
Frank, and E.A. Phelps, 2005, Nature Neuroscience, 8 (11), 1611–1618. Adapted with permission.
signals in the striatum differentiated positive expectation violations elicits increased activ-
and negative outcomes (e.g., partner shar- ity in portions of corticobasal ganglia cir-
ing /defecting) most strongly for the neutral cuitry including the dorsal and ventral stria-
character, consistent with previous findings tum and anterior cingulate, corroborating
of differential signals with monetary rewards the idea that previous social value consider-
(see Delgado, 2007, for a review) and sugges- ations can modulate reward learning (Chang
tive of a learning signal within the striatum & Sanfey, 2009).
(Haruno et al., 2004; O’Doherty et al., 2004; Neuroendocrine influences may underlie
Tricomi et al., 2004). However, no differen- the propensity to exhibit other-regarding
tial responses between positive and negative behavior in goal-directed situations. The
outcomes were observed when interacting hormone oxytocin is implicated in affiliative
with the good partner, suggesting that the social behavior in animals (for a review,
previously acquired social information mod- see Insel & Young, 2001); administering
ulated the ability of the caudate to effec- oxytocin to investors in a trust game led to
tively respond to and learn from feedback significantly more generous amounts sent to
(Figure 19.3). Similarly, memory of social trustees as compared to a placebo group of
458 DOMINIC S. FARERI & MAURICIO R. DELGADO
One potential explanation for these Axelrod, R., & Hamilton, W. (1970). The evolu-
behavioral patterns is that prefrontal cor- tion of cooperation. Science, 211, 1390–96.
tical gray matter development is pro- Barnes, T. D., Kubota, Y., Hu, D., Jin, D. Z.,
tracted and continues into early adult- & Graybiel, A. M. (2005). Activity of stri-
hood in comparison to subcortical gray atal neurons reflects dynamic encoding and
recoding of procedural memories. Nature,
matter (Somerville & Casey, 2010). What
437(7062), 1158–61.
are the implications for these develop-
Baumgartner, T., Heinrichs, M., Vonlanthen,
mental patterns in reward-related learn- A., Fischbacher, U., & Fehr, E. (2008). Oxy-
ing and decision making in a social tocin shapes the neural circuitry of trust and
context? That is, do adolescents place dif- trust adaptation in humans. Neuron, 58(4),
ferential value on social as compared to 639–50.
nonsocial rewards, and how might this Bayer, H. M., & Glimcher, P. W. (2005). Mid-
affect decision making? brain dopamine neurons encode a quantitative
r How can we apply our understand- reward prediction error signal. Neuron, 47(1),
ing of social and nonsocial reward val- 129–41.
ues to a clinical setting? For instance, Behrens, T. E., Hunt, L. T., Woolrich, M. W., &
Rushworth, M. F. (2008). Associative learning
there is evidence to suggest that disso-
of social value. Nature, 456(7219), 245–49.
ciable portions of the cingulate cortex Behrens, T. E., Woolrich, M. W., Walton, M. E.,
are sensitive to processing social informa- & Rushworth, M. F. (2007). Learning the value
tion, particularly within the context of of information in an uncertain world. Nature
social interactions with others (Tomlin et Neuroscience, 10(9), 1214–21.
al., 2006). Interestingly, individuals with Belin, D., Jonkman, S., Dickinson, A., Robbins,
autism spectrum disorders demonstrate a T. W., & Everitt, B. J. (2009). Parallel and
decreased ability to effectively recruit this interactive learning processes within the basal
region during social interactions. Consid- ganglia: Relevance for the understanding of
ering that the neural mechanisms facil- addiction. Behavioral Brain Research, 199(1),
itating the learning of social informa- 89–102.
Belova, M. A., Paton, J. J., & Salzman, C. D.
tion have begun to be delineated, do
(2008). Moment-to-moment tracking of state
individuals with autism spectrum disor- value in the amygdala. Journal of Neuroscience,
ders demonstrate abnormal social learn- 28(40), 10023–30.
ing patterns that underlie their social Berg, J., Dickhaut, J., & McCabe, K. (1995). Trust,
deficits? reciprocity, and social history. Games and Eco-
nomic Behavior, 10, 122–42.
Berns, G. S., McClure, S. M., Pagnoni, G., &
Acknowledgments Montague, P. R. (2001). Predictability modu-
lates human brain response to reward. Journal
This work was supported by a National Insti- of Neuroscience, 21(8), 2793–98.
Bray, S., Shimojo, S., & O’Doherty, J. P.
tute of Mental Health grant (MH084081).
(2010). Human medial orbitofrontal cortex
is recruited during experience of imagined
References and real rewards. Journal of Neurophysiology,
103(5), 2506–12.
Alexander, G. E., Crutcher, M. D., & DeLong, Breiter, H. C., Gollub, R. L., Weisskoff, R.
M. R. (1990). Basal ganglia-thalamocortical M., Kennedy, D. N., Makris, N., Berke, J.
circuits: Parallel substrates for motor, ocu- D., . . . Hyman, S. E. (1997). Acute effects of
lomotor, “prefrontal” and “limbic” func- cocaine on human brain activity and emotion.
tions. Progress in Brain Research, 85, 119– Neuron, 19(3), 591–611.
46. Burke, C. J., Tobler, P. N., Baddeley, M.,
Amodio, D. M., & Frith, C. D. (2006). Meeting & Schultz, W. (2010). Neural mechanisms
of minds: The medial frontal cortex and social of observational learning. Proceedings of the
cognition. Nature Reviews Neuroscience, 7(4), National Academy of Sciences, 107(32), 14431–
268–77. 36.
REWARD LEARNING 461
Camerer, C. F., & Fehr, E. (2006). When does De Martino, B., Kumaran, D., Holt, B., & Dolan,
“economic man” dominate social behavior? R. J. (2009). The neurobiology of reference-
Science, 311(5757), 47–52. dependent value computation. Journal of Neu-
Chang, L. J., Doll, B. B., van ‘t Wout, M., Frank, roscience, 29(12), 3833–42.
M. J., & Sanfey, A. G. (2010). Seeing is believ- de Quervain, D. J., Fischbacher, U., Treyer, V.,
ing: Trustworthiness as a dynamic belief. Cog- Schellhammer, M., Schnyder, U., Buck, A., &
nitive Psychology, 61(2), 87–105. Fehr, E. (2004). The neural basis of altruistic
Chang, L. J., & Sanfey, A. G. (2009). Unfor- punishment. Science, 305(5688), 1254–58.
gettable ultimatums? Expectation violations Doya, K. (2008). Modulators of decision making.
promote enhanced social memory following Nature Neuroscience, 11(4), 410–16.
economic bargaining. Frontiers in Behavioral Fareri, D. S., Chang, L. J., & Delgado, M. R.
Neuroscience, 3, 36. (2012). Effects of direct social experience on
Clarke, H. F., Robbins, T. W., & Roberts, A. C. trust decisions and neural reward circuitry.
(2008). Lesions of the medial striatum in mon- Frontiers in Decision Neuroscience, 6(148).
keys produce perseverative impairments dur- Fareri, D. S., Niznikiewicz, M. A., Lee, V. K. &
ing reversal learning similar to those produced Delgado, M. R. (2012). Social network modula-
by lesions of the orbitofrontal cortex. Journal tion of reward-related signals. Journal of Neu-
of Neuroscience, 28(43), 10972–82. roscience, 32(26), 9045–52.
D’Ardenne, K., McClure, S. M., Nystrom, Fehr, E., & Camerer, C. F. (2007). Social neuroe-
L. E., & Cohen, J. D. (2008). BOLD responses conomics: The neural circuitry of social pref-
reflecting dopaminergic signals in the human erences. Trends in Cognitive Sciences, 11(10),
ventral tegmental area. Science, 319(5867), 419–27.
1264–67. Fliessbach, K., Weber, B., Trautner, P., Dohmen,
Daw, N. D., Gershman, S. J., Seymour, B., T., Sunde, U., Elger, C. E., & Falk, A.
Dayan, P., & Dolan, R. J. (2011). Model-based (2007). Social comparison affects reward-
influences on humans’ choices and striatal pre- related brain activity in the human ventral
diction errors. Neuron, 69(6), 1204–15. striatum. Science, 318(5854), 1305–8.
Delgado, M. R. (2007). Reward-related responses Gottfried, J. A., O’Doherty, J., & Dolan, R. J.
in the human striatum. Annals of the New York (2003). Encoding predictive reward value in
Academy of Sciences, 1104, 70–88. human amygdala and orbitofrontal cortex. Sci-
Delgado, M. R., Frank, R. H., & Phelps, ence, 301(5636), 1104–7.
E. A. (2005). Perceptions of moral character Güth, W., Schmittberger, R., & Schwarze, B.
modulate the neural systems of reward dur- (1982). An experimental analysis of ultima-
ing the trust game. Nature Neuroscience, 8, tum bargaining. Journal of Economic Behavior
1611–18. & Organization, 3, 367–88.
Delgado, M. R., Li, J., Schiller, D., & Phelps, E. Haber, S. N., & Knutson, B. (2010). The reward
A. (2008). The role of the striatum in aversive circuit: Linking primate anatomy and human
learning and aversive prediction errors. Philo- imaging. Neuropsychopharmacology, 35(1), 4–
sophical Transactions of the Royal Society of 26.
London. Series B: Biological Sciences, 363(1511), Hampton, A. N., Adolphs, R., Tyszka, M. J., &
3787–00. O’Doherty, J. P. (2007). Contributions of the
Delgado, M. R., Nystrom, L. E., Fissell, C., Noll, amygdala to reward expectancy and choice
D. C., & Fiez, J. A. (2000). Tracking the hemo- signals in human prefrontal cortex. Neuron,
dynamic responses to reward and punishment 55(4), 545–55.
in the striatum. Journal of Neurophysiology, Hampton, A. N., Bossaerts, P., & O’Doherty,
84(6), 3072–77. J. P. (2008). Neural correlates of mentalizing-
Delgado, M. R., Schotter, A., Ozbay, E. Y., & related computations during strategic inter-
Phelps, E. A. (2008). Understanding overbid- actions in humans. Proceedings of the
ding: Using the neural circuitry of reward to National Academy of Sciences, 105(18), 6741–
design economic auctions. Science, 321(5897), 46.
1849–52. Han, S., Huettel, S. A., Raposo, A., Adcock,
Delgado, M. R., Stenger, V. A., & Fiez, R. A., & Dobbins, I. G. (2010). Functional sig-
J. A. (2004). Motivation-dependent responses nificance of striatal responses during episodic
in the human caudate nucleus. Cerebral Cor- decisions: Recovery or goal attainment? Jour-
tex, 14(9), 1022–30. nal of Neuroscience, 30(13), 4767–75.
462 DOMINIC S. FARERI & MAURICIO R. DELGADO
Harbaugh, W. T., Mayr, U., & Burghart, D. R. rapid event-related fMRI. Neuroimage, 18(2),
(2007). Neural responses to taxation and vol- 263–72.
untary giving reveal motives for charitable Kosfeld, M., Heinrichs, M., Zak, P. J., Fis-
donations. Science, 316(5831), 1622–25. chbacher, U., & Fehr, E. (2005). Oxytocin
Hare, T. A., Camerer, C. F., & Rangel, A. (2009). increases trust in humans. Nature, 435(7042),
Self-control in decision-making involves mod- 673–76.
ulation of the vmPFC valuation system. Sci- Kringelbach, M. L. (2005). The human orbi-
ence, 324(5927), 646–48. tofrontal cortex: Linking reward to hedo-
Hare, T. A., O’Doherty, J., Camerer, C. F., nic experience. Nature Reviews Neuroscience,
Schultz, W., & Rangel, A. (2008). Dissociat- 6(9), 691–702.
ing the role of the orbitofrontal cortex and Li, J., Delgado, M. R., & Phelps, E. A. (2011). How
the striatum in the computation of goal values instructed knowledge modulates the neural
and prediction errors. Journal of Neuroscience, systems of reward learning. Proceedings of the
28(22), 5623–30. National Academy of Sciences, 108(1), 55–60.
Haruno, M., Kuroda, T., Doya, K., Toyama, Lin, A., Adolphs, R., & Rangel, A. (2012). Social
K., Kimura, M., Samejima, K., . . . Kawato, M. and monetary reward learning engage over-
(2004). A neural correlate of reward-based lapping neural substrates. Social Cognitive &
behavioral learning in caudate nucleus: A Affective Neuroscience, 7(3), 274–81.
functional magnetic resonance imaging study Logothetis, N. K., Pauls, J., Augath, M., Trinath,
of a stochastic decision task. Journal of Neuro- T., & Oeltermann, A. (2001). Neurophysio-
science, 24(7), 1660–5. logical investigation of the basis of the fMRI
Hein, G., & Singer, T. (2008). I feel how you feel signal. Nature, 412(6843), 150–57.
but not always: The empathic brain and its McCabe, K., Houser, D., Ryan, L., Smith, V.,
modulation. Current Opinion in Neurobiology, & Trouard, T. (2001). A functional imaging
18(2), 153–58. study of cooperation in two-person reciprocal
Hsu, M., Krajbich, I., Zhao, C., & Camerer, C. F. exchange. Proceedings of the National Academy
(2009). Neural response to reward anticipation of Sciences, 98(20), 11832–35.
under risk is nonlinear in probabilities. Journal Middleton, F. A., & Strick, P. L. (2000). Basal
of Neuroscience, 29(7), 2231–37. ganglia and cerebellar loops: Motor and cog-
Insel, T. R., & Young, L. J. (2001). The neurobi- nitive circuits. Brain Research Review, 31(2–3),
ology of attachment. Nature Reviews Neuro- 236–50.
science, 2(2), 129–36. Mobbs, D., Yu, R., Meyer, M., Passamonti, L.,
Izuma, K., Saito, D. N., & Sadato, N. (2008). Seymour, B., Calder, A. J., . . . Dalgleish, T.
Processing social and monetary rewards in (2009). A key role for similarity in vicarious
the human striatum. Neuron, 58(2), 284– reward. Science, 324(5929), 900.
294. Nieuwenhuis, S., Heslenfeld, D. J., von Geusau,
Izuma, K., Saito, D. N., & Sadato, N. (2010). Pro- N. J., Mars, R. B., Holroyd, C. B., & Yeung,
cessing of the incentive for social approval in N. (2005). Activity in human reward-sensitive
the ventral striatum during charitable dona- brain areas is strongly context dependent.
tion. Journal of Cognitive Neuroscience, 22(4), Neuroimage, 25(4), 1302–9.
621–31. Niv, Y. (2009). Reinforcement learning in the
Kennerley, S. W., Walton, M. E., Behrens, T. E., brain. Journal of Mathematical Psychology,
Buckley, M. J., & Rushworth, M. F. (2006). 53(3), 139–54.
Optimal decision making and the anterior cin- Niv, Y., & Schoenbaum, G. (2008). Dialogues on
gulate cortex. Nature Neuroscience, 9(7), 940– prediction errors. Trends in Cognitive Sciences,
47. 12(7), 265–72.
King-Casas, B., Tomlin, D., Anen, C., Camerer, O’Doherty, J., Critchley, H., Deichmann, R., &
C. F., Quartz, S. R., & Montague, P. R. (2005). Dolan, R. J. (2003). Dissociating valence of
Getting to know you: Reputation and trust outcome from behavioral control in human
in a two-person economic exchange. Science, orbital and ventral prefrontal cortices. Journal
308(5718), 78–83. of Neuroscience, 23(21), 7931–39.
Knutson, B., Fong, G. W., Bennett, S. M., Adams, O’Doherty, J. P., Dayan, P., Friston, K., Critch-
C. M., & Hommer, D. (2003). A region of ley, H., & Dolan, R. J. (2003). Temporal dif-
mesial prefrontal cortex tracks monetarily ference models and reward-related learning in
rewarding outcomes: Characterization with the human brain. Neuron, 38(2), 329–37.
REWARD LEARNING 463
O’Doherty, J., Dayan, P., Schultz, J., Deich- novel actions? Nature Reviews Neuroscience,
mann, R., Friston, K., & Dolan, R. J. (2004). 7(12), 967–75.
Dissociable roles of ventral and dorsal stria- Rescorla, R. A., & Wagner, A. R. (1972). A
tum in instrumental conditioning. Science, theory of Pavolovian conditioning: Varia-
304(5669), 452–54. tions in the effectiveness of reinforcement
O’Doherty, J. P., Deichmann, R., Critchley, H. and nonreinforcement. In A. H. Black & W.
D., & Dolan, R. J. (2002). Neural responses F. Prokasy (Eds.), Classical conditioning II
during anticipation of a primary taste reward. (pp. 64–99). New York: Appleton-Century-
Neuron, 33(5), 815–26. Crofts.
O’Doherty, J., Kringelbach, M. L., Rolls, E. T., Rilling, J., Gutman, D., Zeh, T., Pagnoni, G.,
Hornak, J., & Andrews, C. (2001). Abstract Berns, G., & Kilts, C. (2002). A neural basis
reward and punishment representations in the for social cooperation. Neuron, 35(2), 395–405.
human orbitofrontal cortex. Nature Neuro- Rilling, J. K., & Sanfey, A. G. (2011). The neu-
science, 4(1), 95–102. roscience of social decision-making. Annual
Ostlund, S. B., & Balleine, B. W. (2005). Lesions Review of Psychology, 62, 23–48.
of medial prefrontal cortex disrupt the acqui- Rilling, J. K., Sanfey, A. G., Aronson, J. A., Nys-
sition but not the expression of goal-directed trom, L. E., & Cohen, J. D. (2004). Opposing
learning. Journal of Neuroscience, 25(34), 7763– BOLD responses to reciprocated and unrecip-
70. rocated altruism in putative reward pathways.
Ostlund, S. B., & Balleine, B. W. (2007). The Neuroreport, 15(16), 2539–43.
contribution of orbitofrontal cortex to action Risinger, R. C., Salmeron, B. J., Ross, T. J.,
selection. Annals of the New York Academy of Amen, S. L., Sanfilipo, M., Hoffmann, R.
Sciences, 1121, 174–92. G., . . . Stein, E. A. (2005). Neural correlates
Pagnoni, G., Zink, C. F., Montague, P. R., & of high and craving during cocaine self-
Berns, G. S. (2002). Activity in human ventral administration using BOLD fMRI. Neuroim-
striatum locked to errors of reward prediction. age, 26(4), 1097–1108.
Nature Neuroscience, 5(2), 97–98. Rutledge, R. B., Lazzaro, S. C., Lau, B., Myers, C.
Paton, J. J., Belova, M. A., Morrison, S. E., & E., Gluck, M. A., & Glimcher, P. W. (2009).
Salzman, C. D. (2006). The primate amyg- Dopaminergic drugs modulate learning rates
dala represents the positive and negative value and perseveration in Parkinson’s patients in a
of visual stimuli during learning. Nature, dynamic foraging task. Journal of Neuroscience,
439(7078), 865–70. 29(48), 15104–14.
Pessiglione, M., Seymour, B., Flandin, G., Dolan, Said, C. P., Haxby, J. V., & Todorov, A. (2011).
R. J., & Frith, C. D. (2006). Dopamine- Brain systems for assessing the affective value
dependent prediction errors underpin reward- of faces. Philosophical Transactions of the Royal
seeking behaviour in humans. Nature, Society of London. Series B: Biological Sciences,
442(7106), 1042–45. 366(1571), 1660–70.
Phan, K. L., Sripada, C. S., Angstadt, M., & Schott, B. H., Minuzzi, L., Krebs, R. M., Elmen-
McCabe, K. (2010). Reputation for reciprocity horst, D., Lang, M., Winz, O. H., . . . Bauer,
engages the brain reward center. Proceedings A. (2008). Mesolimbic functional magnetic
of the National Academy of Sciences, 107(29), resonance imaging activations during reward
13099–13104. anticipation correlate with reward-related
Phelps, E. A., & LeDoux, J. E. (2005). Contribu- ventral striatal dopamine release. Journal of
tions of the amygdala to emotion processing: Neuroscience, 28(52), 14311–19.
from animal models to human behavior. Neu- Schultz, W., Dayan, P., & Montague, P. R. (1997).
ron, 48(2), 175–87. A neural substrate of prediction and reward.
Preuschoff, K., Quartz, S. R., & Bossaerts, P. Science, 275(5306), 1593–99.
(2008). Human insula activation reflects risk Sesack, S. R., & Grace, A. A. (2010). Cortico-
prediction errors as well as risk. Journal of Neu- basal ganglia reward network: Microcircuitry.
roscience, 28(11), 2745–52. Neuropsychopharmacology, 35(1), 27–47.
Rapoport, A., & Chammah, A. M. (1965). Pris- Seymour, B., O’Doherty, J. P., Dayan, P.,
oner’s dilemma. Ann Arbor: University of Koltzenburg, M., Jones, A. K., Dolan, R.
Michigan Press. J., . . . Frackowiak, R. S. (2004). Temporal dif-
Redgrave, P., & Gurney, K. (2006). The short- ference models describe higher-order learning
latency dopamine signal: a role in discovering in humans. Nature, 429(6992), 664–67.
464 DOMINIC S. FARERI & MAURICIO R. DELGADO
Somerville, L. H., & Casey, B. J. (2010). Devel- lates of reciprocity in the Trust Game. Social,
opmental neurobiology of cognitive control Cognitive, & Affective Neuroscience, 4(3),
and motivational systems. Current Opinions 294–304.
in Neurobiology, 20(2), 236–41. Volkow, N. D., Wang, G. J., Fowler, J. S., Logan,
Somerville, L. H., Heatherton, T. F., & Kel- J., Gatley, S. J., Hitzemann, R., . . . Pappas,
ley, W. M. (2006). Anterior cingulate cortex N. (1997). Decreased striatal dopaminer-
responds differentially to expectancy violation gic responsiveness in detoxified cocaine-
and social rejection. Nature Neuroscience, 9(8), dependent subjects. Nature, 386(6627), 830–
1007–8. 33.
Takahashi, H., Kato, M., Matsuura, M., Mobbs, Volkow, N. D., Wang, G. J., Fowler, J. S., Logan,
D., Suhara, T., & Okubo, Y. (2009). When J., Jayne, M., Franceschi, D., . . . Pappas, N.
your gain is my pain and your pain is my gain: (2002). “Nonhedonic” food motivation in
Neural correlates of envy and schadenfreude. humans involves dopamine in the dorsal
Science, 323(5916), 937–39. striatum and methylphenidate amplifies this
Tomlin, D., Kayali, M. A., King-Casas, B., Anen, effect. Synapse, 44(3), 175–80.
C., Camerer, C. F., Quartz, S. R., & Mon- Voorn, P., Vanderschuren, L. J., Groenewegen,
tague, P. R. (2006). Agent-specific responses H. J., Robbins, T. W., & Pennartz, C. M.
in the cingulate cortex during economic (2004). Putting a spin on the dorsal-ventral
exchanges. Science, 312(5776), 1047–50. divide of the striatum. Trends in Neurosciences,
Tricomi, E. M., Delgado, M. R., & Fiez, J. 27(8), 468–74.
A. (2004). Modulation of caudate activity by Wise, R. A. (2004). Dopamine, learning and moti-
action contingency. Neuron, 41(2), 281–92. vation. Nature Reviews Neuroscience, 5(6),
Tricomi, E., & Fiez, J. A. (2008). Feedback signals 483–94.
in the caudate reflect goal achievement on a Wyvell, C. L., & Berridge, K. C. (2000). Intra-
declarative memory task. Neuroimage, 41(3), accumbens amphetamine increases the condi-
1154–67. tioned incentive salience of sucrose reward:
Tricomi, E., Rangel, A., Camerer, C. F., & Enhancement of reward “wanting” without
O’Doherty, J. P. (2010). Neural evidence for enhanced “liking” or response reinforcement.
inequality-averse social preferences. Nature, Journal of Neuroscience, 20(21), 8122–30.
463(7284), 1089–91. Zink, C. F., Pagnoni, G., Martin-Skurski, M.
van den Bos, W., van Dijk, E., Westenberg, M., E., Chappelow, J. C., & Berns, G. S. (2004).
Rombouts, S. A., & Crone, E. A. (2009). Human striatal responses to monetary reward
What motivates repayment? Neural corre- depend on saliency. Neuron, 42(3), 509–17.
CHAPTER 20
Memory can take many forms. It can reflect episodic memory stores. We may be able
the conscious access of factual knowledge to remember most of yesterday’s events,
or past experiences (explicit or declarative but it is likely that we remember only a
memory), or it can be revealed as a change minority of the events from one month
in our behavior that results from the influ- ago. Although many factors influence the
ence of past experiences (implicit or non- likelihood that an experience becomes part
declarative memory). This chapter focuses of our memory stores, one important fac-
on episodic memory, a form of consciously tor is the experience of emotion. Many of
accessible memory (see Chapter 18 for a dis- our episodic memories pertain to experi-
cussion of the effects of emotion on implicit ences that elicited an emotional reaction; we
learning). More specifically, episodic mem- experienced altered physiological or somatic
ory refers to memory for a unique event. reactions, or changes in our subjective feel-
Episodic memories generally include the ings, as the event unfolded (see Chapter 1
content of the event itself, as well as infor- for discussion of the best way to define emo-
mation on the spatial and temporal context tion). The term “emotional memory” refers
in which the event occurred (Tulving, 1972). to our ability to remember these episodic
For an event to be remembered in an events.
episodic fashion, three phases of process- Extensive research on emotional memory
ing must occur. First, the information from demonstrates that emotion can interact with
the initial experience must be encoded into episodic memory processes in a number of
a format that can be stored in memory. ways that influence each of the encoding,
Second, that information must be consoli- consolidation, and retrieval phases of pro-
dated or stabilized into a lasting represen- cessing. When an event elicits an emotion,
tation. Third, that representation must be it can influence which event details we ini-
retrieved and consciously ascribed to our own tially encode and how cognitive demanding
personal past. Not all experiences progress it is for us to encode them. The emotion
through these phases to become a part of our experienced during or shortly after an event
465
466 ALISHA C. HOLLAND & ELIZABETH A. KENSINGER
can also influence the likelihood that the processes (see Figure 20.1). Relatively auto-
encoded information becomes consolidated matic influences can include the capture of
in memory. The emotional content of the attention by emotion-relevant information,
event we are retrieving can influence how the preattentive or prioritized processing of
subjectively rich our memories for the expe- such stimuli, and the enhanced fluidity of
rience seem, or how easily the details of the processing of such stimuli (see Chapters
experience come to mind when we are pre- 14 and 15; see also Whittlesea, 1993). Con-
sented with a retrieval cue. Not only can the trolled processes encompass the enhanced
emotion experienced during the initial event elaboration and rehearsal given to emotional
affect memory processes, so too can the information, as well as the sustained atten-
emotions experienced during later memory tion focused on it (Talmi, Luk, McGarry, &
phases. For instance, the emotions we expe- Moscovitch, 2007).
rience while retrieving an event can influ- A few lines of evidence suggest that
ence which events come to mind most easily the distribution of automatic and controlled
and can also influence the way in which dif- processes may differ for emotional infor-
ferent aspects of a past experience are recon- mation with different affective character-
structed. This chapter discusses the ways istics. Emotions are often described in a
in which emotion interacts with encoding, two-dimensional space consisting of valence
consolidation, and retrieval processes, focus- (pleasantness or unpleasantness) and arousal
ing on how both the emotional content of an (level of excitation; Russell, 1980). As we
event and the emotional state of the individ- describe next, where an emotional reaction
ual can influence memory. falls along each of these axes seems to influ-
ence the types of processing that contribute
most readily to the information’s encoding.
The Encoding of Emotional
Experiences
The Effects of High-Arousal Emotion
on Encoding Processes
Encoding refers to the set of processes that
transform an initial experience into a format For information that is highly arousing,
that can be stored in memory. Just as the many of the effects of emotion on encod-
signals from the keys we press on a com- ing processes may occur relatively automat-
puter must be transformed into a format ically. Information that is high in arousal is
that can be recognized and stored within a noticed more quickly and more often than
document, so must the sights, sounds, and lower arousal information, and attention is
other details of an event be converted into directed selectively toward that informa-
a format that can be stored in memory. The tion (e.g., Leclerc & Kensinger, 2008). High-
way we initially process an event can there- arousal information may also require fewer
fore have large implications for the types of attentional resources to process than low-
information that become stored in memory. arousal information (see Chapter 14; see also
If our attention is captured by some part Matthews & Margetts, 1991), and it may be
of the event or if we contemplate a par- prioritized for processing such that it will
ticular feature of an experience, then those beat out the competition when attentional
aspects are more likely to become encoded. resources are taxed (see Chapter 15).
In fact, encoding may be best thought of as These alterations in processing for high-
a byproduct of the way that we initially pro- arousal information can have downstream
cess an experience (Paller & Wagner, 2002). effects on the likelihood that people can
In a broad sense, the factors that influ- remember the information. For example,
ence the encoding of emotional events can people show enhanced detection of high-
be broken down into those that involve rel- arousal stimuli on the emotional Stroop task,
atively automatic influences and those that and they also are more likely to remem-
arise via the engagement of more controlled ber the high-arousal items (e.g., MacKay
Figure 20.1. The basic mechanisms through which the emotional content of an experience can influence the way in which it is remembered. See color
plate 20.1.
468 ALISHA C. HOLLAND & ELIZABETH A. KENSINGER
et al., 2004). Similarly, when people’s atten- salient information, it does not exert these
tion is divided across two tasks as they influences on its own. In fact, it seems
attempt to encode information (e.g., when that some of the largest contributions that
they are asked to encode words while simul- the amygdala makes to successful encod-
taneously monitoring sound patterns), they ing reflect its modulation of other sen-
appear better able to process the high- sory and mnemonic processes. Once active,
arousal information as compared to the low- the amygdala sends efferent connections to
arousal information, and their memory suf- many regions of the brain (e.g., Amaral,
fers less for the high-arousal information Price, Pitkanen, & Carmichael, 1992), inner-
(Kensinger & Corkin, 2004; Kern, Libku- vating regions important for sensory and
man, Otani, & Holmes, 2005). These find- mnemonic processing. Neuroimaging anal-
ings are consistent with the proposal that yses have suggested that the amygdala
high-arousal information is privy to auto- has strong functional connections with the
matic processing advantages, which enhance fusiform gyrus and extrastriate visual cortex
the likelihood that the information will be (e.g., Tabert et al., 2001), as well as with the
encoded into memory. hippocampus (reviewed by LaBar & Cabeza,
At a neural level, many of these rela- 2006). The amygdala may also evoke activ-
tively automatic influences seem to stem ity in a broader affective-attentional net-
from amygdala activation. Research con- work (see meta-analysis by Murty, Ritchey,
ducted in patients with amygdala damage Adcock, & LaBar, 2010), including regions
has revealed the necessity of this region for of the orbitofrontal cortex, anterior cin-
enhanced detection of high-arousal informa- gulate gyrus, and caudate nucleus (dis-
tion: Individuals with amygdala damage do cussed in Kensinger, 2009). This broader
not show the same automatic detection and network of regions is brought online when
rapid processing of high-arousal informa- a task requires attention to affective stim-
tion as do those with intact amygdalae (e.g., uli (e.g., Robbins & Everitt, 1996), and it
LaBar & Phelps, 1998). More recent neu- may help focus attention selectively on the
roimaging studies have confirmed that the high-arousing information, increasing the
activation patterns of the amygdala make it likelihood of remembering it (discussed in
a likely candidate for such automatic influ- Kensinger, 2009).
ences. In particular, the amygdala shows It seems, therefore, that the success-
increases in activity when salient stimuli – ful encoding of high-arousal items results
such as a fearful face or a snake – are pre- from a combination of domain-general fac-
sented, even when those stimuli are afforded tors, which could enhance the encoding of
only minimal attention (e.g., Whalen et al., any type of information, and of domain-
2004). There has been debate over whether specific factors that are uniquely associated
the amygdala can become activated in the with the encoding of emotional informa-
absence of attention (see Chapters 14 and 15), tion. The engagement of sensory cortices
raising doubt as to whether high-arousal (e.g., Talmi, Anderson, Riggs, Caplan, &
stimuli could be encoded without the exer- Moscovitch, 2008) or of medial tempo-
tion of any cognitively demanding resources ral lobe regions (e.g., Dolcos, LaBar, &
(see also Pottage & Schaefer, 2012). But evi- Cabeza, 2004) can be considered domain
dence does suggest that the amygdala can general, because although activity in these
process even those high-arousal stimuli that regions is increased when stimuli are high
have not been afforded extensive attention, in arousal, their activity also occurs for
suggesting that high-arousal stimuli can be nonarousing information and their engage-
encoded using fewer of these cognitively ment enables the encoding of both neu-
demanding resources than are required for tral information and emotional informa-
the encoding of lower arousing stimuli. tion. In fact, in some circumstances these
Although the amygdala seems to play regions may be activated independently of
a critical role in the rapid encoding of the amygdala (e.g., Talmi et al., 2008),
EMOTION IN EPISODIC MEMORY 469
suggesting that the encoding of emotional cess of encoding (reviewed by Symons &
information can sometimes be explained by Johnson, 1997), and a few lines of evidence
domain-general processes. However, in the suggest that these types of processes may
majority of studies to date, the modulation be particularly important for enhancing the
of this sensory and mnemonic activation encoding of low-arousal information. For
seems to be enhanced by activation of the one, neuroimaging studies have confirmed
amygdala and of other affective-processing that regions implicated in elaborative pro-
regions such as the orbitofrontal cortex. cessing, such as the lateral prefrontal cor-
These regions are not brought online unless tex, are disproportionately recruited dur-
information is interpreted to have particu- ing the successful encoding of emotional
lar saliency (e.g., Sander, Grafman, & Zalla, information (e.g., Dolcos et al., 2004), par-
2003; Chapter 1), and therefore their engage- ticularly if that emotional information is
ment can be considered to be domain spe- low in arousal (e.g., Kensinger & Corkin,
cific. Thus, although the end result may be 2004). Second, when participants’ atten-
domain-general activity, the route through tion is divided as they encode information,
which that activity becomes enhanced often memory for low-arousal items is dispropor-
appears to be specific to the processing of tionately impaired. Because elaboration is an
emotionally salient information. In other attention-demanding process, divided atten-
words, engagement of emotion-specific pro- tion tasks disrupt the ability to engage those
cesses can influence memory by modulating processes; the fact that the encoding of low-
domain-general sensory and mnemonic pro- arousal information is most affected by this
cesses. disruption suggests that memory for those
items was particularly reliant on elaboration
(e.g., Kensinger & Corkin, 2004; Kern et al.,
The Effects of Low-Arousal Emotion
2005).
on Encoding Processes
Emotion can also benefit memory by pro-
For emotional items that are lower in viding semantic coherence and an organizing
arousal, automatic processes seem to play principle (e.g., Talmi et al., 2007). “Emo-
less of a role in guiding encoding. Low- tion” can be thought of as a category, and so
arousal items are less likely to be detected stimuli that elicit any emotion (or particu-
than high-arousal items (see Chapters 14 lar emotions) may be clustered together in
and 15), and their encoding is impeded by memory. It is well known that this type of
the presence of a secondary task (Kensinger organization assists in the encoding of infor-
& Corkin, 2004). In contrast to encoding of mation and can later be used as a power-
high-arousal items, encoding of items that ful retrieval cue, and so the ability for emo-
are low in arousal seems to benefit from tional information to be clustered together
more controlled processes. As we discuss in may affect the way it is retained in mem-
this section, some of the key processes that ory. Evidence to support this view has come
may affect the encoding of low-arousal items from studies that have revealed elevated
are the elaboration it evokes and the seman- false recognition of emotional information
tic clustering and organization from which (e.g., Brainerd, Stein, Silveira, Rohenkohl,
it benefits. & Reyna, 2008). These data have been inter-
Elaboration refers to the processes by preted as suggesting that individuals use the
which new information becomes linked to thematic coherence of emotional informa-
previously stored information. This linkage tion to encode the “gist” or general theme of
can occur by extracting meaning from new the associated information and then endorse
information, by forming associations with any information that is consistent with that
other semantic knowledge or episodic expe- encoded theme. Interestingly, many of the
riences, or by integrating the information studies that have revealed these effects of
into a person’s self-view. It is well known emotion on false memory have used words
that these elaborations enhance the suc- and other stimuli that are fairly low in
470 ALISHA C. HOLLAND & ELIZABETH A. KENSINGER
arousal (e.g., Brainerd et al., 2008; Kapucu, the valence of information influences the
Rotello, Ready, &Seidl, 2009). Although the engagement of these processes. In other
effects of emotion on false memory are not words, even if positive and negative experi-
limited to low-arousal stimuli (e.g., Gallo, ences elicit an equal amount of arousal, the
Foster, & Johnson, 2009), and more research two types of information may be encoded
is needed to clarify whether high-arousal differently because of their valence. There
stimuli are less prone to false memory effects is not a large body of research that has
than low-arousal stimuli, it is possible that explored this topic, but two lines of evi-
the tendency to encode gist-based infor- dence suggest that negative valence may
mation is greater when information is low enhance the automatic processing of emo-
arousal than when it is high arousal because tional information. First, manipulations of
the reliance on semantic clustering is greater divided attention seem to have less of an
for low-arousal items (see discussion by impact on memory for negative information
Kapucu et al., 2009). It is also possible that than positive information (Kern et al., 2005;
high- and low-arousal stimuli are equally Talmi et al., 2007), suggesting that negative
likely to be encoded in a way that takes information may be encoded more auto-
advantage of semantic clustering, but that matically than positive information. Second,
because high-arousal stimuli benefit from neuroimaging studies have revealed that the
additional, automatic processes, it is easier successful encoding of negative high-arousal
for people to encode item-specific informa- information is more likely to be tied to sen-
tion, as well as gist-based, categorical infor- sory activity than is the successful encod-
mation, about high-arousal items. This item- ing of positive information (e.g., Mickley
specific information may be used to combat Steinmetz & Kensinger, 2009). Positive high-
category-consistent false memories in some arousal information, by contrast, may be
instances. more associated with thematic or gist-based
In summary, there has been extensive processing, such that its encoding is more
evidence to suggest that memory for high- likely to be disrupted by divided attention
arousal items may benefit from automatic (Kern et al., 2005; Talmi et al., 2007) and
processes, whereas memory for low-arousal to be associated with activity in lateral pre-
items is more likely to be enhanced because frontal regions associated with elaborative
of controlled processes. This dissociation encoding and semantic organization (e.g.,
does not require that controlled processing Mickley Steinmetz & Kensinger, 2009).
is unique to low-arousal information; it is
likely that high-arousal items also are elabo-
The Effects of Emotional State and
rated on and semantically organized. How-
Emotional Goals on Encoding
ever, because memory for the high-arousal
information is so strongly influenced by rela- So far, we have discussed how the emo-
tively automatic processes, these more con- tions elicited by the content of an event can
trolled processes are less likely to exert an influence the way that event is encoded. But
impact. By contrast, because low-arousal the way we process an event is also shaped
information does not seem to be privy to by our current affective state and by our
the same automatic processing advantages goals. Our emotional states can modulate
as high-arousal information, the encoding of the type of event details we attend to and
low-arousing information is dominated by therefore encode (see Figure 20.2). Gener-
the changes in these controlled processes. ally, we are more likely to attend to informa-
tion that is congruent in valence to our cur-
rent emotional state; for example, being in a
The Effects of Valence on Encoding
negative mood (either transiently or chron-
Although so far we have discussed the ically, as in depression) enhances our atten-
effects of arousal on automatic and con- tion to negative information and makes it
trolled processes, it is also possible that more likely that this negative information
Event
occurrence
A depressed state is
associated with greater
lateral orbitofrontal Encoding Retrieval
cortex activity−thought
to reflect deployment of
attentional resources− in
response to negative
stimuli (from Elliott et al., Encoding and retrieval of Encoding and retrieval of
•Emotional state modulates attention positive information negative information following
2002). State to and facilitates encoding of mood-
•Emotional state facilitates retrieval of
mood-congruent information. following a positive mood a negative mood induction
congruent information. induction (i.e., mood commonly activate left
congruent memory) posteriolateral orbitofrontal
commonly activate right cortex (from Lewis et al., 2005).
subgenual cingulate.
•Activated emotion regulation goals •Activated emotion regulation goals
Reappraise
Goals and motivation modulate attention to and motivation facilitate the selection
0.6 goal-congruent information of events or event-specific details
LIFG Dm
0.4
0.2
depending on strategy. that are goal-congruent.
0
O Mood-incongruent memory Recalling positive
−0.2 O Cognitive reappraisal
−0.4 r = 0.63
may engage a ventral autobiographical memories
−0.6
enhances memory, likely via
P < 0.001 following a negative mood
−0.8 prefrontal and medial prefrontal and anterior
induction is associated with
−1 −0.5 0 0.5 1 temporal lobe interactions. cingulate cortex network. greater activity in left
R. Hippocampus Dm
O Cognitive reappraisal subgenual cingulate cortex
O Expressive suppression
(from Cooney et al., 2007).
The successful encoding of negative information is impairs memory, perhaps increases lateral prefrontal
associated with stronger correlations between left inferior because this strategy draws cortex activity and decreases
frontal gyrus and right hippocampus when that information attention away from the to-be- medial prefrontal cortex and
is being cognitively reappraised compared to viewed or encoded information. amygdala activity.
suppressed (from Heyes et al., 2010).
Figure 20.2. The basic mechanisms through which a person’s emotional state and goals can influence the way in which an event is encoded or retrieved.
472 ALISHA C. HOLLAND & ELIZABETH A. KENSINGER
becomes part of our memory stores (e.g., increased prefrontal activity might reflect
Mathews & MacLeod, 2005). increased emotion regulation and/or an
Motivational goals also can affect encod- increase in self-referential processing in
ing in a comparable way to mood. For response to positive information (see Chap-
instance, approach motivation during a task ter 28; Kensinger & Leclerc, 2009).
leads to an overestimation of the remem- Although these studies of age-related
bered happiness during that task, whereas changes in emotional memory provide one
avoidance motivation during encoding leads window into exploring the effects of emo-
to an overestimation of remembered anxi- tional goals on memory, there is a need for
ety (Lench & Levine, 2010). Our emotional further clarification of the ways in which
goals – in other words, how we want to affect-relevant goals influence the memories
feel – can similarly affect the information of younger adults. Studies in young adults
that we encode. Perhaps one of the clearest will be important because it is currently dif-
demonstrations of the influence of emotion ficult to disentangle the age-related changes
goals on memory is age-related changes in due specifically to alterations in goal states
the types of experiences that are remem- from those that are due to more general age-
bered. As adults progress from middle age related changes in cognition or in brain func-
into older adulthood, they appear to place tion (see discussion by Nashiro, Sakaki, &
additional emphasis on emotional fulfill- Mather, 2012).
ment and to have emotion regulation goals Most of the research examining the
chronically activated; this shift has been pro- effects of emotion goals on young adults’
posed to occur because older adults perceive memory has focused on distinguishing the
their time as limited and want to maxi- influences of different types of strategies
mize their emotional well-being (reviewed that can be used to regulate emotions.
by Mather, 2006). In line with these emo- Many studies concerning emotion regulation
tion goals, older adults are more likely than contrast two dissociable emotion regulation
young adults to sustain their attention on, strategies. One is cognitive reappraisal, a
and therefore to encode, positive informa- specific type of emotion regulation strat-
tion (Mather, 2006). egy that involves changing the way that
The changes in emotion goals may yield one reacts to or thinks about an emotional
a positivity bias by affecting the con- stimulus (Gross, 1998). The second type of
trolled processing of emotional information strategy involves suppressing one’s outward
(Mather, 2006): Older adults may be more emotional reaction to a stimulus (Gross,
likely than young adults to process positive 1998; see also Chapter 16).
information in a controlled fashion. Consis- The type of strategy employed at encod-
tent with this interpretation, older adults ing can have important implications for
are most likely to show a positivity bias what is later remembered. Several studies
if they have good cognitive control ability have demonstrated that reappraising nega-
and can deploy their full resources toward tive emotions associated with information
task performance (Mather, 2006) and if improves free recall of that information
they are encoding low-arousal items that, as after a short delay, but that emotion-ex-
noted earlier, may be a type of information pressive suppression impairs recall (e.g.,
whose retention is particularly dependent Dillon, Ritchey, Johnson, & LaBar, 2007;
on controlled processing. The conclusion Richards & Gross, 1999). As with the age-
that emotion goals affect controlled pro- related changes in emotional memory, the
cessing of emotional information is further different mnemonic implications of these
supported by neuroimaging evidence. Aging emotion regulation strategies may have to do
does not seem to have a large effect on amyg- with the way in which controlled processes
dala activity, but instead seems to increase are engaged during the encoding of an expe-
prefrontal cortex activity (particularly me- rience. Cognitive reappraisal engages cogni-
dial) during encoding (see Chapter 28). The tive control regions in the prefrontal cortex
EMOTION IN EPISODIC MEMORY 473
that seem to dampen amygdala activity as comparable effect of emotion across all pos-
emotional stimuli are being attended to; sible delay intervals. In actuality, however,
thus, reappraisal results in increased pre- the effects of emotion often become exag-
frontal engagement but reduced amygdala gerated after a long delay. If we think back
activity (e.g., Ochsner. Bunge, Gross, & to this morning, we probably can remem-
Gabrieli, 2002). By contrast, suppression ber the mundane moments (what we ate
is associated with a rebound in amygdala for breakfast, what traffic was like on our
activity (Goldin, McRae, Ramel, & Gross, morning commute), but if we think back
2008). The fact that cognitive reappraisal to last week, we are unlikely to remember
leads to maximal encoding suggests that the these types of details unless they were par-
mnemonic benefit stems from the engage- ticularly emotional (we might remember if
ment of the controlled processes imple- the milk had spoiled or we hit a traffic jam,
mented by the prefrontal cortex, rather than but we probably will not remember those
from the engagement of the amygdala. parts of our day if they proceeded in a usual
It is also possible that it is the inter- fashion). Indeed, although emotion conveys
actions between the more controlled (pre- a benefit to memory even after a short delay
frontal) and more automatic (amygdala) (e.g., Talmi et al., 2007), this benefit tends to
processes that lead to such a robust mem- become exaggerated across longer intervals
ory trace. Indeed, the successful encoding of (e.g., Sharot & Yonelinas, 2008). This pat-
negative images during cognitive reappraisal tern cannot easily be explained by the effects
(versus suppression and passive viewing) of emotion on encoding processes alone.
more strongly coactivates the left inferior Rather, this pattern suggests that emotion
frontal gyrus, hippocampus, and amygdala exerts its influence not only during encoding
(Hayes et al., 2010). Further, a separate study but also during the subsequent consolidation
revealed that the fronto-amygdalar neural phase of episodic memory.
circuitry involved with reappraisal (versus Consolidation refers to the set of pro-
passive viewing) of emotional pictures dur- cesses that stabilize a memory trace, making
ing encoding was reactivated up to a year it less prone to disruption (see McGaugh,
later during retrieval (Erk, von Kalckreuth, 2004). It is generally agreed that the hip-
& Walter, 2010), suggesting that there may pocampus is required for consolidation to
be a permanent change in the connectiv- occur, although there are debates over the
ity within these regions that helps solid- time course of hippocampal involvement
ify memory. It will be important for future (see Nadel & Moscovitch, 1997; Squire &
research to clarify why the neural activ- Zola, 1998). As noted earlier, the amyg-
ity recruited for cognitive reappraisal leads dala is well connected with a number of
to such strong memories and to examine regions, including the hippocampus, and
why emotion suppression may have differ- amygdala activation is believed to modulate
ent long-term neural effects with regard to hippocampal function, increasing the likeli-
the permanency of a memory. hood that an emotional experience is con-
solidated. Indeed, patients with damage to
the amygdala show normal enhancements
The Consolidation and Retention in emotional memory over short delays, but
of Emotional Experiences do not retain those enhancements over long
delays (e.g., LaBar & Phelps, 1998), consis-
As the previous section has emphasized, tent with the proposal that the amygdala
emotion exerts large influences on the way exerts many of its effects via its influences
we initially attend and encode information. on hippocampal consolidation.
If emotion only affected encoding processes, Many neuroimaging studies have con-
then its effects should be consistent regard- firmed that the amygdala and hippocampus
less of whether memory was tested after a interact to create durable episodic memo-
short or after a long delay: There should be a ries (reviewed by LaBar & Cabeza, 2006).
474 ALISHA C. HOLLAND & ELIZABETH A. KENSINGER
Figure 20.3. The key regions involved in emotion’s modulation of memory encoding and
consolidation. PFC = prefrontal cortex (shaded portions highlight the ventrolateral and orbitofrontal
regions that are most often implicated in studies of emotional memory), TEMP = inferior temporal
lobe, HIPP = hippocampus, A = amygdala.
of design examines autobiographical mem- its activation during retrieval may enable the
ories to discriminate among the neural pro- recall of the event’s importance.
cesses engaged over the course of this type Interestingly, despite this overlap in the
of memory’s protracted retrieval time (e.g., neural structures implicated in the encod-
Daselaar et al., 2008; Greenberg et al., ing and retrieval of emotional informa-
2005). tion, there may also be laterality differences
Evidence from laboratory and autobi- between these two phases of emotional
ographical memory studies implicates an memory. For instance, Sergerie, LePage, and
overlap in the neural network engaged dur- Armony (2006) found that right amygdala
ing both the encoding and retrieval of activity predicted successful encoding of
emotional information (Dolcos et al., 2005; emotional information, whereas left amyg-
Maratos, Dolan, Morris, Henson, & Rugg, dala activity predicted successful retrieval.
2001). In particular, the amygdala, the hip- Although other studies have found the
pocampus, and regions of the prefrontal cor- opposite laterality effects (see Cabeza & St.
tex are engaged not only during the encoding Jacques, 2007), the extant data do not pro-
of an experience but also when emo- vide strong evidence for engagement of the
tional information is recalled. Thus, to some identical regions within the amygdala during
extent, retrieval of emotional experiences both encoding and retrieval.
may involve the recapitulation of activity Although the connectivity between the
that was present during the instantiation of amygdala and hippocampus during emo-
emotion during the initial experience. As is tional memory retrieval is fairly well estab-
true during encoding, it appears that emo- lished, one remaining question is how this
tional memory retrieval engages many of the connectivity leads to the qualitative and
same regions as neutral memory retrieval – quantitative differences between emotional
with the prefrontal cortex guiding mem- and neutral memory retrieval. One possi-
ory selection and monitoring and the hip- bility is that emotion facilitates the mem-
pocampus implicated in the reconstruction ory search process. It has been proposed
of past event features – but that the activity that early activation of the amygdala and
in these regions is modulated by limbic areas medial prefrontal cortex, through some par-
key for emotional processing (Kensinger & tial cue of an emotional event, produces an
Schacter, 2007; see Figure 20.3). Functional affective state. In turn, the presence of an
connectivity analyses have revealed recipro- affective state serves as a further retrieval
cal connections between the amygdala and cue for emotional information; the engage-
hippocampus as well as between the pre- ment of prefrontal regions and of the amyg-
frontal cortex and these medial temporal dala subsequently enhances the hippocam-
lobe regions during the retrieval of infor- pally mediated retrieval of event details
mation encoded in an emotional context (Buchanan, 2007). The retrieval of additional
(Smith, Stephan, Rugg, & Dolan, 2006). affective details by the hippocampus leads
These results suggest that the retrieval of to a recapitulation of the regions active dur-
emotional information might be orches- ing the initial perception of those details
trated by top-down processing originating in (Buchanan, 2007). Evidence from studies
the prefrontal cortex, which then influences using event-related potentials (ERPs), which
activity in regions known to be critical for provide excellent temporal resolution, sug-
emotion (the amygdala) and episodic recol- gests that the retrieval of emotional infor-
lection (the hippocampus; Buchanan, 2007; mation recruits additional early and late
Greenberg et al., 2005). It is possible that the processing relative to the retrieval of neu-
amygdala’s role in modulating retrieval may tral information, possibly reflecting the early
be to index the past behavioral significance facilitation of emotional memory search
of an event (Dolan, Lane, Chua, & Fletcher, and the later recapitulation of emotion dur-
2000); if the amygdala is a “salience detector” ing recollection (reviewed by Buchanan,
during encoding (Sander et al., 2003), then 2007). Recent research using fMRI has further
EMOTION IN EPISODIC MEMORY 477
suggested that emotional retrieval cues may (Daselaar et al., 2008). Increased amygdala
trigger stronger retrieval-related activity in activity is also positively correlated with a
early sensory areas than in areas associ- sense of recollection (Sharot et al., 2004).
ated with higher-level cognition (Hofstetter, There has been debate as to whether amyg-
Achaibou, & Vuilleumier, 2012), suggesting dala engagement during retrieval corre-
that emotion may enhance sensory recapit- sponds with the accuracy or amount of
ulation during retrieval. detail recovered about an experience or
The findings of another fMRI study lend whether amygdala activity during retrieval
further evidence to the hypothesis that emo- serves only to inflate the subjective vividness
tion might facilitate the search for mem- or the confidence associated with a memory
ory representations during retrieval. The (Sharot et al., 2004). We return to this point
authors took advantage of the relatively later, when we discuss the effects of emotion
lengthy process required for autobiograph- on memory confidence.
ical memory retrieval and of the emotional
nature of most autobiographical memories
Mood-Congruent Memory
to examine the time course over which emo-
tion modulated memory retrieval (Daselaar Evidence reviewed to this point in the chap-
et al., 2008). They measured neural activity ter demonstrates that the emotions we expe-
during both the initial construction phase rience when an event is occurring can influ-
of autobiographical memories and the later ence the way we encode, consolidate, and
elaboration and maintenance phase of those retrieve that event. It is also the case, how-
memories. Analyses revealed that regions ever, that our emotional experience at the
critical for memory retrieval, including the time of memory retrieval can affect mem-
hippocampus, retrosplenial cortex, and pre- ory. Our emotional state during retrieval can
frontal cortex, were active in the early, ini- influence both which events are most acces-
tial access phase of the retrieval. Critically, sible for recall and which details from those
participants’ ratings of emotional intensity events are most likely to be remembered.
were related to the amount of amygdala A robust example of mood’s influence on
and hippocampal activity during this initial memory is mood-congruent recall, or the
period, but not during the later elaboration tendency to recall events that are congru-
period. The early activation of the amyg- ent in valence to one’s current mood (e.g.,
dala – before the memories were even fully to recall negative information when in a neg-
constructed – supports the hypotheses that ative mood; see Rusting, 1988, for review). In
emotion can act as an early “warning” that a addition to influencing which memories are
personally significant event is being retrieved recalled, the mood-congruency effect can
and that amygdala activity may help guide also affect the speed with which memories
the memory search process (Daselaar et al., are recalled and how positively or negatively
2008). Amygdala activity was not increased memories are rated. Mood-congruent mem-
during the elaboration phase, suggesting that ory appears to occur under both induced
the amygdala is not simply responding to the and naturally occurring moods like depres-
emotional appraisal of an event after it is sion and is evident on both laboratory and
already retrieved (Daselaar et al., 2008). autobiographical memory tasks (see Rust-
The amygdala and hippocampal activity ing, 1998, for review). This effect is most
that seems to be critical for the retrieval of often explained in terms of Bower’s (1981)
emotional memories might also be related network theory of affect, which purports
to the phenomenological characteristics of that information associated with particular
emotional memory mentioned at the begin- emotion “nodes” becomes activated when
ning of this section. For example, activity of those nodes are engaged, as in the case of a
both the hippocampus and amygdala is asso- mood state (see Rusting, 1998, for a review of
ciated with the reported emotional inten- alternative views). Thus, being in a negative
sity of a retrieved autobiographical memory mood would activate the negative node and
478 ALISHA C. HOLLAND & ELIZABETH A. KENSINGER
the information associated with that node, mation that are attended and encoded.
facilitating the retrieval of memories con- Indeed, neuroimaging evidence suggests that
gruent with a negative mood. mood-congruency effects occur during both
Neuroimaging work conducted in the the encoding phase and the retrieval phase.
past several years has begun to examine Encoding emotional information that is
the underlying neural correlates of mood- mood congruent appears to be less effortful
congruent memory (Lewis & Critchley, in terms of semantic processing, with mood-
2003). Perhaps unsurprisingly, some of the congruent information leading to a lesser
same limbic and paralimbic regions that neural signature of semantic processing
have been implicated in the experience of (the N400) than mood-incongruent infor-
emotional states and in the encoding of emo- mation (Kiefer, Schuch, Schenck, & Fiedler,
tional experiences are implicated in remem- 2007). ERP and fMRI investigations also
bering mood-congruent information (see have demonstrated that depressed individu-
Buchanan, 2007, for a review). For exam- als allocate a greater proportion of working
ple, the subgenual cingulate gyrus is active memory (associated with increased dorso-
during both the successful encoding and lateral prefrontal cortex activity) and atten-
retrieval of positive information following a tional (associated with lateral orbitofrontal
happy mood induction, whereas the posteri- activity) resources to negative informa-
olateral orbitofrontal cortex is linked to the tion (e.g., Elliott, Rubinsztein, Sahakian, &
successful encoding and retrieval of negative Dolan, 2002). If mood-congruent informa-
information during a sad mood induction tion is privy to easier processing and a greater
(Lewis, Critchley, Smith, & Dolan, 2005). share of cognitive resources, then this might
The overlap of the regions recruited during help explain the later mnemonic benefit evi-
the encoding and mood-congruent retrieval dent for the retrieval of mood-congruent
of negative versus positive information may information.
provide support for Bower’s (1981) network
theory, with the subgenual cingulate gyrus
Episodic Memory and Emotion
serving as the positive emotion “node” and
Regulation
the posteriolateral orbitofrontal cortex serv-
ing as the negative emotion “node” (Lewis Although mood congruency has been an
et al., 2005). often replicated effect, mood is sometimes
The amygdala may also play a role in associated with just the opposite effect:
mood-congruent memory: Increased amyg- Mood-incongruent memory occurs when
dala activity in formerly depressed patients memories opposite in valence to mood
at high risk for relapse is associated with are retrieved under induced (e.g., Joseph-
mood-congruent retrieval of negative self- son, Singer, & Salovey, 1996) or natural
referential information following a sad mood mood conditions (Parrott & Sabini, 1990).
induction (Ramel et al., 2007). Although the Mood-incongruent memory sometimes
reason for this association is not known, occurs after an initial period of mood
one possibility is that the amygdala guides congruency (e.g., Josephson et al., 1996)
mood-congruent retrieval because, as dis- and therefore is often ascribed the function
cussed earlier, it facilitates the search for of mood regulation or repair (e.g., Isen,
a personally significant memory (Daselaar 1984). Indeed, emotion or mood regulation
et al., 2008) and activates to salient informa- is a reported function of autobiographical
tion (Sander et al., 2003). It is likely that indi- remembering in everyday life (Bluck, Ale,
viduals at highest risk for relapse are those Habermas, & Rubin, 2005). Such goals can
for whom negative memories are perceived influence episodic retrieval by at least two
to be most self-relevant and salient. routes. First, emotion regulation goals can
Although mood congruency is often dis- influence which memories are most likely to
cussed as a retrieval effect, as noted earlier, be recalled in a given context. For example,
mood can also influence the types of infor- when individuals in one study were told
EMOTION IN EPISODIC MEMORY 479
that they would be playing a game in which on memory encoding. There has also been
aggressive behavior would be rewarded, a limited amount of work examining the
they preferred to recall angry memories relationship between regulation and emo-
prior to playing that game (Tamir, Mitchell, tional autobiographical memory retrieval at
& Gross, 2008). There is also some evidence a neural level. One study found that, in
to suggest that regulation goals can influ- contrast to the more dorsal regions impli-
ence which specific emotional details we cated in top-down emotion regulation, being
remember about particular events (Holland, instructed to recall a positive memory fol-
Tamir, & Kensinger, 2010). lowing induction into a negative mood (i.e.,
The second way in which emotion regu- mood-incongruent recall) led to engagement
lation goals can influence memory retrieval of ventral regions including ventromedial
is by modulating the level of specificity with prefrontal and orbitofrontal cortices, as well
which a memory is recalled. The construc- as the subgenual cingulate gyrus (Cooney,
tion of autobiographical memories can occur Joormann, Atlas, Eugène, & Gotlib, 2007).
at either a temporally and contextually spe- In contrast to mood-incongruent recall,
cific level (e.g., my first day of classes in regulating the emotions associated with a
college) or at a general level that incorpo- specific past experience by changing the
rates similar, repeated events (e.g., every way that event is interpreted is presum-
time I attended class on Tuesdays; reviewed ably a more effortful task and perhaps
by Conway & Pleydell-Pearce, 2000). A as such engages different neural circuitry.
large body of literature has demonstrated Kross, Davidson, Weber, and Ochsner
that individuals with a range of affective (2009) found that subjective emotion rat-
disorders are more likely to recall these ings and neural activity depended on the
latter general autobiographical events that type of regulation strategy employed dur-
are repeated over time than those that ing the recall of negative autobiographical
are specific in time and place (reviewed memories. Both emotion ratings and activ-
by Williams et al., 2007). One possible ity in subgenual anterior cingulate gyrus and
explanation for this so-called overgeneral medial prefrontal cortex activity were great-
memory phenomenon is affect regulation est when participants focused on their nega-
(see Williams et al., 2007, for a review). tive feelings (similar to rumination), inter-
Williams and colleagues (2007) have pro- mediate when they performed a strategy
posed that overgeneral recall allows for the analogous to cognitive reappraisal, and low-
functional avoidance of specific, negative est when they mentally distanced them-
event-related details (though note that this selves from the event. This study illus-
avoidance may actually be dysfunctional; trates that the successful regulation of a
Phillipot, Baeyens, Douilliez, & Francart, negative memory differs from the regula-
2004). tion of other stimuli in its dampening down
Several studies examining the neu- of the medial prefrontal cortex (associated
ral correlates of emotion regulation in with self-referential processing). Interest-
response to emotional images have estab- ingly, recent work further suggests that the
lished that the down-regulation of neg- direction of regulation may influence when
ative emotion engages dorsolateral pre- regulation-related neural activity is engaged.
frontal, orbitofrontal, and cingulate cortices, Down-regulation via cognitive reappraisal
which appear to suppress ventral and limbic appears to engage medial and lateral PFC
regions associated with emotional respond- and MTL activity as memories are ini-
ing, including the amygdala and insula (see tially being constructed and presumably
Ochsner & Gross, 2008, for a review; see being appraised as negative, whereas up-
Chapter 16 for discussion of the key regions regulation engages similar regions as indi-
implicated in emotion regulation). As we viduals prepare to recall events (i.e., prior
discussed earlier, a small amount of work to the presentation of a memory cue) and
has focused on the effects of regulation later as they elaborate upon the details
480 ALISHA C. HOLLAND & ELIZABETH A. KENSINGER
of those memories (Holland & Kensinger, viduals who reappraised their initial nega-
2012). Taken together, this work suggests tive responses to the outcome of an elec-
that the circuitry underlying emotion reg- tion remembered feeling less negative about
ulation during memory retrieval, and the the election than those who had not reap-
time course over which it is engaged, can be praised their response (Levine, 1997). As we
dependent on the type of strategy invoked noted previously, older adults, a group that
(see Cooney et al., 2007, for a similar discus- demonstrates enhanced emotion regulation
sion). and a positivity bias in attention and mem-
ory, also tend to remember the past as more
positive than they experienced it (reviewed
The Constructive Nature of Memory
by Mather, 2006).
for Emotions
From a behavioral standpoint, the recon-
Up to this point, we have focused on the structive nature of memory for emotions
way that emotion and emotional goals can might stem from the same mechanisms that
influence which details are encoded or later underlie the reconstructive nature of mem-
brought to mind. The implicit assumption ory for other details. It would be inefficient
throughout this discussion has been that to have a memory system that recorded and
these memories are fairly accurate represen- stored every detail that we encounter on a
tations. As we reviewed earlier, however, moment-by-moment basis. Perhaps for this
the constructive nature of memory leaves reason, specific perceptual and conceptual
it susceptible to biases and distortions; these details are forgotten relatively quickly after
errors can be influenced by our emotional an episode ends (reviewed by Robinson &
state or goals at encoding or retrieval. For Clore, 2002) and are reconstructed based on
example, although it often feels as though heuristics. This heuristic information might
we will never forget the emotions associated include the peak and end moments of affec-
with episodic events, a plethora of research tive intensity during an episode (e.g., Fred-
has demonstrated that emotional details are erickson, 2000). Alternatively, or in addition
subject to the same types of constructive to the peak and end heuristic, our recon-
processes (and therefore errors) as other structions can be guided by contextual infor-
episodic details (reviewed by Levine, Safer, mation (e.g., remembering what we were
& Lench, 2006). In particular, the emotions thinking as an event occurred) or seman-
we remember feeling are often related more tic information (e.g., knowing that vaca-
strongly to how we predicted we would tions are usually positive; Robinson & Clore,
feel before an event (Mitchell, Thompson, 2002).
Peterson, & Cronk, 1997) and how we feel Just as with other types of details, the
at the time of recall (Levine, 1997) than construction of emotional details might be
to how we reported feeling as the event quite functional in that the process allows
was occurring. For example, the “rosy view” for the updating of memory (Levine et al.,
phenomenon has been used to describe the 2006). Given that a key function of episodic
observation that individuals who expect an memory construction is to create future
event like a vacation to be positive recall simulations that will guide future behav-
it that way, even if they actually experi- ior (Schacter & Addis, 2008), it is criti-
enced it as a mix of both positive and neg- cal to understand the neural mechanisms
ative emotions (Mitchell et al., 1997). The that underlie the reconstruction of past
same is true for events that we expect to emotions. Much of the evidence in this
be negative, such as Mondays: Even though domain comes from neuropsychological
individuals tend to experience these events patients and points to a distinction between
as fairly neutral, they remember them as memory systems for event details versus
being negative (Areni & Burger, 2008). Cog- emotional details (reviewed by Conway
nitive reappraisal has also been associated & Pleydell-Pearce, 2000). Amnesics with
with biases in memory for emotions. Indi- medial temporal lobe damage demonstrate
EMOTION IN EPISODIC MEMORY 481
impaired memory for narrative details but assassination. The authors reported that the
spared memory for affective impressions of personally significant and surprising nature
a person (Johnson, Kim, & Risse, 1985). of the events led to memories that were
The reverse can also be true for individ- vivid and highly detailed. The nature of
uals with temporary or permanent cortical the details was such that both event-related
right-hemisphere lesions: These individuals and seemingly unimportant personal con-
report normal event-related details, but fail text details (e.g., who I was with when I
to report emotional details (see Conway & learned the news) were reported. Brown and
Pleydell-Pearce, 2000). Kulik (1977) hypothesized that the surpris-
More recent research has honed in on ing and personally relevant nature of flash-
the hippocampus, amygdala, and nearby bulb events signals biological relevance to
cortical areas as potential supporters of individuals, setting off a cascade of neu-
the retrieval of negative emotional autobi- ral activation that results in a vivid snap-
ographical memories (Buchanan, Tranel, & shot of the details – even those idiosyn-
Adolphs, 2006). When damage occurs to cratic in nature – on first learning of the
the right-lateralized anteromedial temporal event.
lobe (but not to the hippocampus alone), Follow-up research confirmed that indi-
there is a decrease in the number of neg- viduals can in fact form very vivid and
ative details and the reported emotional detailed memories about flashbulb events
intensity associated with negative memories (reviewed by Talarico & Rubin, 2003). How-
(Buchanan et al., 2006). These patient find- ever, investigations into the accuracy and
ings are in line with the evidence reviewed consistency of these memories over time
earlier that the amygdala is recruited during have revealed that flashbulb memories are
emotional memory retrieval and is associ- subject to the same types of reconstructive
ated with intensity during retrieval (Dase- errors as other neutral and emotional events.
laar et al., 2008; Greenberg et al., 2005). One of the first studies to demonstrate
Given that the retrieval of event-related flashbulb memory inconsistencies over time
details can occur without affective details examined individuals’ memories for the per-
(Conway & Pleydell-Pearce, 2000), it seems sonal details they experienced when they
that the neural correlates of the reconstruc- learned the news of the Challenger space
tion of event-related and emotional details shuttle explosion in 1986 (Neisser & Harsch,
may also be dissociable. 1992). Participants’ memories were exam-
ined shortly after the explosion and again
30 months later; even though participants
Flashbulb Memory
believed their memories were vivid and
To this point, we have reviewed evidence accurate, they actually contained a num-
that emotional events and emotional details ber of distortions. Similar inconsistencies
are reconstructed at retrieval and there- have been shown for people’s memories
fore are susceptible to the same types of for the outcome of the O. J. Simpson trial
errors and biases as neutral information. (Schmolck, Buffalo, & Squire, 2000) and
There exists a special category of highly the September 11 terrorist attacks (Pezdek,
emotional, personally significant, and sur- 2003).
prising events, termed “flashbulb memo- Several lines of research may shed light
ries” by Brown and Kulik (1977). Initially, on why there are inaccuracies and incon-
it was believed that flashbulb memories sistencies in even the most emotional of
were remembered more accurately (i.e., experiences. One line of research concerns
with fewer reconstructive errors) than other what has been termed the emotional mem-
events. Brown and Kulik’s (1977) seminal ory “tradeoff.” Emotion does not enhance
paper examined individuals’ memories for a memory equally for all types of details.
number of negative public events, the most Instead, the details that are central to the
cited being President John F. Kennedy’s emotional aspect of an event often receive a
482 ALISHA C. HOLLAND & ELIZABETH A. KENSINGER
mnemonic benefit that comes at the cost of such as a sporting event (Kensinger & Schac-
background or peripheral details (see Levine ter, 2006) or a political election (Holland &
& Edelstein, 2009, for a review). The mem- Kensinger, 2012; Levine, 1997). These stud-
ory tradeoff has been illustrated in terms of ies generally converge on the finding that
the types of details – either those directly negative emotion appears to confer a par-
tied to the event or those tied to personal ticular mnemonic benefit for the memory
context – that are remembered most consis- of that event’s details above and beyond
tently or accurately. Pezdek (2003) queried the mnemonic advantage evident when an
individuals about the details of the Septem- event is viewed as positive. This valence-
ber 11 terrorist attacks (e.g., What time did dependent difference in memory might be
the first tower fall?) and about the per- due to the mode of information process-
sonal details they experienced during the ing that is induced by negative and posi-
day (e.g., Who were you with when you tive emotions. Negative emotion is associ-
learned about the attacks?). Those indi- ated with detailed, item-specific processing
viduals who were located closest to the that enhances attention to the details that
World Trade Centers on that day demon- might have led to the negative emotion. In
strated the most accurate memory for event- contrast, positive emotion is associated with
related details when compared to partici- a greater reliance on heuristics and relational
pants in California and Hawaii. Despite this processing (reviewed by Kensinger, 2009),
mnemonic benefit for the facts of the event, which may increase the number of recon-
however, the Manhattan sample reported structive errors that are later made during
a relative paucity of personal details from retrieval.
the day compared to the more distant sam-
ples. These findings can be explained in
The Confidence of Emotional
terms of the tradeoff: The central details for
Memories
the Manhattan sample were those tied to
the event, which potentially had important The definition of episodic memory requires
implications for their own physical safety. that an event be ascribed to one’s per-
The personal details of the day were likely sonal past, and the hallmark of an episodic
remembered less well because they con- memory is the feeling of reexperience that
stituted the less critical, background infor- accompanies the reconstruction of some of
mation (Pezdek, 2003). Interestingly, Brown the features of the past event (Tulving, 1972).
and Kulik (1977) alluded to this very phe- A number of studies have suggested that
nomenon in their seminal paper when they this feeling of reexperience may be par-
noted that flashbulb memory is far from ticularly pronounced when information is
complete and misses details just as a cam- emotional. People are more likely to claim
era misses information that falls outside of that they vividly remember emotional expe-
its focus. Therefore, the measured accuracy riences (e.g., Ochsner, 2000; Sharot et al.,
and consistency of flashbulb memories may 2004), and as just discussed in relation to
depend on what information is being re- flashbulb memories, people often feel con-
membered. fident in their recollections of emotional
The second factor that can influence the events.
memory of flashbulb events is the perceived There has been much discussion as to
valence of the event. Most flashbulb mem- whether these subjective reports reflect an
ory studies have followed highly negative accurate interpretation of the high fidelity
public events, like natural disasters, terror- with which an emotional event has been
ist attacks, and the deaths of public fig- recorded or whether emotion may simply
ures. One way to examine how the valence bias a person to believe that a vivid mem-
of an event influences its later retrieval is ory has been maintained (see Kensinger,
to make use of events whose outcome can 2009; Mather, 2007; Sharot et al., 2004).
be perceived as either negative or positive, The findings with regard to the amygdala’s
EMOTION IN EPISODIC MEMORY 483
role in emotional memory have been sim- structures known to be associated with emo-
ilarly mixed; some studies have revealed tional processing.
that amygdala activity during encoding or Now that a solid foundation of neu-
retrieval corresponds with the accuracy of a roimaging research on emotional memory
memory (e.g., Smith et al., 2006), whereas is being built, it appears that a particularly
other studies have shown that its activ- ripe area for future work will be examining
ity does not correspond with the ability to not only how emotion affects each phase of
remember episodic detail (e.g., Sharot et al., memory but also how emotion influences
2004). the interaction among the different phases
A recent neuroimaging study may help of memory and how our attempts to control
shed light on the reason for these conflict- or change those emotions might influence
ing findings. In this study (Kensinger, Addis, what is encoded or retrieved.
& Atapattu, 2011), participants viewed emo-
tional objects and were later asked to report
the vividness of their memory for the object, Outstanding Questions and Future
as well as to recall a number of contex- Directions
tual details associated with the object’s pre-
sentation. The results revealed that amyg- r Through what mechanisms do differ-
dala activity corresponded with a paramet- ent motivational states influence the
ric increase in the subjective vividness of encoding and retrieval of memories?
a memory, but that its engagement only Motivational states appear to influence
enabled individuals to encode the visual attentional resources (e.g., narrowing or
details of the object that had been presented, broadening attention) when information
and not any other episodic details. These is being encoded and may modulate the
findings suggest that amygdala engagement accessibility of particular event details at
corresponds with memory for only a select retrieval. On a neural level, these changes
set of episodic details, but that it may be are likely associated with interactions
those details that people use to gauge the between the prefrontal cortex and medial
vividness of a memory. In other words, an temporal lobe. More work is needed to
emotional memory may be vivid not because delineate how the connections between
lots of different details are remembered, but these regions may be affected by differ-
because a small set of details are remem- ent motivational states, particularly those
bered very well. that are defined beyond the dimension
of valence (e.g., discrete emotions). It
has been suggested that specific, discrete
Conclusion emotions (i.e., sadness, happiness, anger,
fear) are associated with unique motiva-
In this chapter, we reviewed evidence that tional states such as approach and avoid-
the emotional content of information influ- ance (reviewed by Levine & Edelstein,
ences – and selectively enhances – each 2009), and so it is possible that these
phase of episodic memory: encoding, con- states might modulate the memory net-
solidation, and retrieval. Our present emo- work in different ways. One study that
tional states and emotional goals can further contrasted the recall of negative and posi-
modulate which information we are likely tive events demonstrated dissociable con-
to encode or retrieve. Critically, neuroimag- tributions of the prefrontal cortex, tem-
ing evidence has elucidated that memory poral lobe, and retrosplenial cortex to
for emotional information relies on many each (Piefke, Weiss, Ziller, Markowitsch,
of the same medial temporal lobe and pre- & Fink, 2003), but it is unclear whether
frontal cortex structures as memory for neu- effects like these are due to the emotional
tral information, but that this memory net- valence of the events or to the different
work receives additional modulation from motivations associated with each valence
484 ALISHA C. HOLLAND & ELIZABETH A. KENSINGER
(e.g., approach vs. avoidance motivation not match those invoked during retrieval?
for positive and negative valence, respec- Examinations of how motivational goals
tively). Examining events associated with influence memory have so far focused
different discrete emotions might further on goals present at either encoding or
disentangle the contributions of valence retrieval. However, motivational goals
and motivational goals on memory. likely exist at both time points. As yet, it
r How do our attempts to regulate emotion is unclear how goals and their associated
influence what is encoded or retrieved? neural activity at encoding and retrieval
Prior research has demonstrated the might interact. Erk et al. (2010) suggested
effects of emotional valence and inten- that regulating emotional responses at
sity on memory encoding and retrieval, encoding led individuals to “cognitivize”
but investigations into how attempts to emotional images and subsequently rely
change such valence or intensity affect on more prefrontal areas during retrieval
memory are in their infancy. Only a small when compared to the recognition of
subset of emotion regulation strategies unregulated images. It remains to be seen
in the context of memory retrieval has what would happen if additional regula-
been examined so far, and it remains tory goals were invoked at retrieval or if
to be seen which neural structures are incongruent goals were in place at each
engaged by strategies that focus on the time (e.g., down-regulating a negative
down-regulation of positive information, emotion at encoding but up-regulating
up-regulation of negative information, a negative emotion at retrieval). Future
or suppression at retrieval. It is likely research should investigate how the goals
that emotion regulation influences both present at both encoding and retrieval
the content of an experience that is might interact and influence the content
encoded and retrieved and the qual- and quality of the information retrieved,
ity (e.g., vividness, accuracy) of that particularly if those goals are incongruent
information. Understanding the effects with one another.
of emotion regulation on memory may
shed light on affective disorders associ-
ated with emotion dysregulation. Acknowledgments
r How do encoding, consolidation, and
retrieval processes interact to influence Preparation of this chapter was supported
the retention of emotional experiences? by the National Institutes of Health (grant
Although most research has focused on MH080833 to E. A. K.), the Searle Schol-
each of these stages in isolation, there are ars Program (to E. A. K.), and a National
likely to be many interactions among the Defense Science and Engineering Graduate
phases. The details that are attended dur- Fellowship (to A. C. H.). We thank Donna
ing encoding or the mood elicited during Addis, Angela Gutchess, Brendan Murray,
the initial experience of an event is likely Jessica Payne, Christina Leclerc, Katherine
to influence the dimensions of an event Mickley Steinmetz, Maya Tamir, and Jill
that become consolidated in memory or Waring for helpful discussion of the issues
the efficacy with which retrieval pro- reviewed in this chapter.
cesses guide the recovery of information.
Now that we have a basic understand-
ing of the way that emotional content or References
emotional state can modulate each indi-
vidual phase of memory, future research Amaral, D., Price, J., Pitkanen, A., & Carmichael,
will do well to consider each phase of S. (1992). The amygdala: Neurobiological
memory in the context of the others. aspects of emotion, memory, and mental dys-
r How is memory affected when the moti- function. In J. P. Aggleton (Ed.), The amyg-
vational goals present during encoding do dala: Neurobiological aspects of emotion, mem-
EMOTION IN EPISODIC MEMORY 485
ory, and mental dysfunction (pp. 1–66). New Dolcos, F., LaBar, K. S., & Cabeza, R. (2004).
York: Wiley-Liss. Interaction between the amygdala and the
Areni, C., & Burger, M. (2008). Memories of medial temporal lobe memory system predicts
“bad” days are more biased than memories better memory for emotional events. Neuron,
of “good” days: Past Saturdays vary, but past 42(5), 855–63.
Mondays are always blue. Journal of Applied Dolcos, F., LaBar, K. S., & Cabeza, R. (2005).
Social Psychology, 38, 1395–1415. Remembering one year later: Role of the
Bartlett, F. C. (1932). Remembering: A study amygdala and the medial temporal lobe mem-
in experimental and social psychology Cam- ory system in retrieving emotional memories.
bridge: Cambridge University Press. Proceedings of the National Academy of Sci-
Bluck, S., Alea, N., Habermas, T., & D. C. Rubin. ences, 102, 2626–31.
(2005). A tale of three functions: The self- Elliott, R., Rubinsztein, J. S., Sahakian, B. J.,
reported uses of autobiographical memory. & Dolan, R. J. (2002). The neural basis of
Social Cognition, 23, 91–117. mood-congruent processing biases in depres-
Bower, G. H. (1981). Mood and memory, Ameri- sion. Archives of General Psychiatry, 59, 597–
can Psychologist, 36, 129–48. 604.
Brainerd, C.J., Stein, L.M., Silveira, R.A., Erk, S, von Kalckreuth, A., & Walter, H. (2010).
Rohenkohl, G., & Reyna, V. F. (2008). Does Neural long-term effects of emotion regula-
negative emotion cause false memories?. Psy- tion on episodic memory processes. Neuropsy-
chological Science, 19, 919–25. chologia, 48, 989–96.
Brown, R., & Kulik, J. (1977). Flashbulb memo- Fredrickson, B. L. (2000). Extracting meaning
ries. Cognition, 5, 73–99. from past affective experiences: The impor-
Buchanan, T.W. (2007). Retrieval of emotional tance of peaks, ends, and specific emotions.
memories. Psychonomic Bulletin, 133, 761– Cognition and Emotion: Special Issue: Emotion,
79. Cognition, and Decision Making, 14(4), 577–
Buchanan, T. W., Tranel, D., & Adolphs, R. 606.
(2006). Memories for emotional autobio- Gallo, D. A., Foster, K. T. & Johnson, E. L.
graphical events following unilateral damage (2009). Elevated false recollection of emo-
to medial temporal lobe. Brain, 129, 115–27. tional pictures in young and older adults. Psy-
Cabeza, R., & St. Jacques, P. (2007). Functional chology and Aging, 24, 981–88.
neuroimaging of autobiographical memory. Goldin, P. R., McRae, K., Ramel, W., & Gross,
Trends in Cognitive Sciences, 11, 219–27. J. J. (2008). The neural bases of emotion reg-
Conway, M. A., & Pleydell-Pearce, C.W. (2000). ulation: Reappraisal and suppression of nega-
The construction of autobiographical memo- tive emotion. Biological Psychiatry, 63, 577–86.
ries in the self-memory system. Psychological Greenberg, D. L., Rice, H. J., Cooper, J. J.,
Review, 107, 261–88. Cabeza, R., Rubin, D. C., & LaBar, K. S.
Cooney, R. E., Joormann, J., Atlas, L. Y., Eugène, (2005). Coactivation of the amygdala, hip-
F., & Gotlib, I. H. (2007). Remembering the pocampus, and inferior frontal gyrus during
good times: Neural correlates of affect regula- autobiographical memory retrieval. Neuropsy-
tion. Neuroreport, 18, 1771–74. chologia, 43, 659–74.
Daselaar, S. M., Rice, H. J., Greenberg, D. L., Gross, J. (1998). The emerging field of emotion
Cabeza, R., LaBar, K. S., & Rubin, D. C. regulation: An integrative review. Review of
(2008). The spatiotemporal dynamics of auto- General Psychology, 2, 271–99.
biographical memory: Neural correlates of Hayes, J. P., Morey, R. A., Petty, C. M., Seth, S.,
recall, emotional intensity, and reliving. Cere- Smoski, M. J., McCarthy, G., & LaBar, K. S.
bral Cortex, 18(1), 217–29. (2010). Staying cool when things get hot: Emo-
Dillon, D. G., Ritchey, M., Johnson, B. D., & tion regulation modulates neural mechanisms
LaBar, K. S. (2007). Dissociable effects of of memory encoding. Frontiers in Human Neu-
conscious emotion regulation strategies on roscience, 4, 1–10.
explicit and implicit memory. Emotion, 7, 354– Hofstetter, C., Achaibou, A., & Vuilleumier,
65. P. (2012). Reactivation of visual cortex dur-
Dolan, R. J., Lane, R., Chua, P., & Fletcher, P. ing memory retrieval: Content specificity and
(2000). Dissociable temporal lobe activations emotional modulation. Neuroimage, 60, 1734–
during emotional episodic memory retrieval. 45. Holland, A. C., & Kensinger, E. A.
Neuroimage, 11, 203–9. (2012). The neural correlates of cognitive
486 ALISHA C. HOLLAND & ELIZABETH A. KENSINGER
activity associated with episodic memory for Ochsner, K. N., Bunge, S. A., Gross, J. J., &
emotional context. Neuropsychologia, 39, 910– Gabrieli, J. D. (2002). Rethinking feelings: An
20. FMRI study of the cognitive regulation of
Mather, M. (2006). Why memories may become emotion. Journal of Cognitive Neuroscience, 14,
more positive as people age. In B. Uttl, N. 1215–29.
Ohta, & A. L. Siegenthaler (Eds.), Mem- Ochsner, K., & Gross, J. J. (2008). Cognitive emo-
ory and emotion: Interdisciplinary perspectives tion regulation: Insights from social cognitive
(pp. 135–59). Oxford: Blackwell. and affective neuroscience. Current Directions
Mather, M. (2007). Emotional arousal and mem- in Psychological Science, 17, 153–58.
ory binding: An object-based framework. Per- Paller, K. A., & Wagner, A.D. (2002). Observing
spectives on Psychological Science, 2, 33–52. the transformation of experience into mem-
Mathews, A., & MacLeod, C. (2005). Cognitive ory. Trends in Cognitive Sciences, 6(2), 93–
vulnerability to emotional disorders. Annual 102.
Review of Clinical Psychology, 1, 167–95. Parrott, W. G., & Sabini, J. (1990). Mood and
Matthews, G., & Margetts, I. (1991). Self-report memory under natural conditions: Evidence
arousal and divided attention: A study of per- for mood incongruent recall. Journal of Per-
formance operating characteristics. Human sonality and Social Psychology, 59, 321–36.
Performance, 4, 107–25. Payne, J. D., & Kensinger, E. A. (2011). Sleep leads
McGaugh, J. L. (2004). The amygdala modulates to changes in the emotional memory trace:
the consolidation of memories of emotionally Evidence from fMRI. Journal of Cognitive Neu-
arousing experiences. Annual Review of Neu- roscience, 23, 1285–97.
roscience, 27, 1–28. Payne, J. D., Swanberg, K., Stickgold, R., &
Mickley Steinmetz, K. R., & Kensinger, E. A. Kensinger, E. A. (2008). Sleep preferentially
(2009). The effects of valence and arousal enhances memory for emotional components
on the neural activity leading to subsequent of scenes. Psychological Science, 19, 781–88.
memory. Psychophysiology, 46, 1190–99. Pezdek, K. (2003). Event memory and autobio-
Mitchell, T. R., Thompson, L., Peterson, E., & graphical memory for the events of Septem-
Cronk, R. (1997). Temporal adjustments in the ber 11, 2001. Applied Cognitive Psychology, 17,
evaluation of events: The “rosy view.” Journal 1033–45.
of Experimental Social Psychology, 33(4), 421–48. Philippot, P., Baeyens, C., Douilliez, C., & Fran-
Murty, V. P., Ritchey, M., Adcock, R. A., & cart, B. (2004). Cognitive regulation of emo-
LaBar, K. S. (2010). fMRI studies of success- tion: Application to clinical disorders. In P.
ful emotional memory encoding: A quantita- Philippot & R. S. Feldman (Eds.), The regula-
tive meta-analysis. Neuropsychologia, 48, 3459– tion of emotion (pp. 71–98). Mahwah, NJ: Erl-
69. baum.
Nadel, L., & Moscovitch, M. (1997). Memory Piefke, M., Weiss, P. H., Zilles, K., Markow-
consolidation, retrograde amnesia and the hip- itsch, H. J., & Fink, G. R. (2003). Differential
pocampal complex. Current Opinion in Neu- remoteness and emotional tone modulate the
robiology, 7(2), 217–27. neural correlates of autobiographical memory.
Nashiro, K., Sakaki, M., & Mather, M. (2012). Age Brain, 126, 650–68.
differences in brain activity during emotion Pottage, C. L., & Schaefer, A. (2012). Visual atten-
processing: Reflections of age-related decline tion and emotional memory: Recall of aversive
or increased emotion regulation? Gerontology, pictures is partially mediated by concurrent
58, 156–63. task performance. Emotion, 12, 33–38.
Neisser, U., & Harsch, N. (1992). Phantom flash- Ramel, W., Goldin, P. R., Eyler, L. T., Brown,
bulbs: False recollections of hearing the news G. G., Gotlib, I. H., & McQuaid, J. R. (2007).
about Challenger. In E. Winograd & U. Amygdala reactivity and mood-congruent
Neisser (Eds.), Affect and accuracy in recall: memory in individuals at risk for depressive
Studies of ‘flashbulb’ memories (pp. 9–31). New relapse. Biological Psychiatry, 61, 231–39.
York: Cambridge University Press. Richards, J. M., & Gross, J. J. (1999). Compo-
Ochsner, K. N. (2000). Are affective events richly sure at any cost? The cognitive consequences
“remembered” or simply familiar? The experi- of emotion suppression. Personality and Social
ence and process of recognizing feelings past. Psychology Bulletin, 25, 1033–44.
Journal of Experimental Psychology: General, Robbins, T. W., & Everitt, B. J. (1996). Neurobe-
129, 242–61. havioural mechanisms of reward and motiva-
488 ALISHA C. HOLLAND & ELIZABETH A. KENSINGER
tion. Current Opinion in Neurobiology, 6, 228– Sleep promotes the neural reorganization of
36. remote emotional memory. Journal of Neuro-
Robinson, M. D., & Clore, G. L. (2002). Belief and science, 29, 5143–52.
feeling: Evidence for an accessibility model Svoboda, E., McKinnon, M. C., & Levine, B.
of emotional self-report. Psychological Bulletin, (2006). The functional neuroanatomy of auto-
128, 934–60. biographical memory: A meta-analysis. Neu-
Rubin, D. C., & Kozin, M. (1984). Vivid memo- ropsychologia, 44(12), 2189–2208.
ries. Cognition, 16(1), 81–95. Symons, C. S., & Johnson, B. T. (1997). The self-
Rubin, D. C., Schrauf, R. W., & Greenberg, reference effect in memory: A meta-analysis.
D. L. (2003). Belief and recollection of autobi- Psychological Bulletin, 121(3), 371–94.
ographical memories. Memory and Cognition, Tabert, M. H., Borod, J. C., Tang, C. Y., Lange,
31, 887–901. G., Wei, T. C., Johnson, R., et al. (2001).
Russell, J. A. (1980). A circumplex model of Differential amygdala activation during emo-
affect. Journal of Personality and social Psychol- tional decision and recognition memory tasks
ogy, 39, 1161–78. using unpleasant words: An fMRI study. Neu-
Rusting, C. L. (1998). Personality, mood, and cog- ropsychologia, 39, 556–73.
nitive processing of emotional information: Talarico, J. M., & Rubin, D. C. (2003). Con-
Three conceptual frameworks. Psychological fidence, not consistency, characterizes flash-
Bulletin, 124, 165–96. bulb memories. Psychological Science, 14, 455–
Sander, D., Grafman, J., & Zalla, T. (2003). The 61.
human amygdala: An evolved system for rele- Talmi, D., Anderson, A. K., Riggs, L., Caplan,
vance detection. Reviews in the Neurosciences, J. B., & Moscovitch, M. (2008). Immediate
14, 303–16. memory consequences of the effect of emo-
Schacter, D. L., & Addis, D. R. (2008). The tion on attention to pictures. Learning & Mem-
cognitive neuroscience of constructive mem- ory, 15, 172–82.
ory: Remembering the past and imagining the Talmi, D., Luk, T. C. B., McGarry, L., &
future. In J. Driver, P. Haggard, & T. Shal- Moscovitch, M. (2007). Are emotional pic-
lice (Eds.), Mental processes in the human brain tures remembered better just because they are
(pp. 27–47). Oxford: Oxford University Press. semantically related and relatively distinct?
Schmolck, H., Buffalo, E. A., & Squire, L. R. Journal of Memory and Language, 56, 555–74.
(2000). Memory distortions develop over time: Tamir, M., Mitchell, C., & Gross, J. J. (2008).
Recollections of the O.J. Simpson trial verdict Hedonic and instrumental motives in anger
after 15 and 32 months. Psychological Science, 11, regulation. Psychological Science, 19, 324–28.
39–45. Tulving, E. (1972). Episodic and semantic mem-
Sergerie, K., Lepage, M., & Armony, J. L. (2006). ory. In E. Tulving & W. Donaldson (Eds.),
A process-specific functional dissociation of Organization of memory (pp. 381–403). New
the amygdala in emotional memory. Journal York: Academic Press.
of Cognitive Neuroscience, 18, 1359–67. Wagner, U., Hallschmid, M., Rasch, B., & Born,
Sharot, T., Delgado, M. R., & Phelps, E. A. J. (2006). Brief sleep after learning keeps emo-
(2004). How emotion enhances the feeling of tional memories alive for years. Biological Psy-
remembering. Nature Neuroscience, 12, 1376– chiatry, 60, 788–90.
80. Walker, M. P., & Stickgold, R. (2006). Sleep,
Sharot, T., & Yonelinas, A. P. (2008). Differen- memory, and plasticity. Annual Review of Psy-
tial time-dependent effects of emotion on the chology, 57, 139–66.
recollective experience and memory for con- Whalen, P. J., Kagen, J., Cook, R. G., Davis, F.
textual information. Cognition, 106, 538–47. C., Hackjin, K., et al. (2004). Human amyg-
Smith, A. P., Stephan, K. E., Rugg, M. D., & dala responsivity to masked fearful eye whites.
Dolan, R. J. (2006). Task and content mod- Science, 306, 2061.
ulate amygdala-hippocampal connectivity in Whittlesea, B. W. A. (1993). Illusions of familiar-
emotional retrieval. Neuron, 49, 631–38. ity. Journal of Experimental Psychology: Learn-
Squire, L. R., & Zola, S. M. (1998). Episodic mem- ing, Memory, and Cognition, 19, 1235–53.
ory, semantic memory, and amnesia. Hip- Williams, J. M. G., Barnhofer, T., Crane, C.,
pocampus, 8, 205–11. Hermans, D., Raes, F., et al. (2007). Autobi-
Sterpenich, V., Albouy, G., Darsaud, A., ographical memory specificity and emotional
Schmidt, C., Vandewalle, G., et al. (2009). disorder. Psychological Bulletin, 133, 122–48.
Section VI
SOCIAL EMOTIONS
CHAPTER 21
Moral Emotions
Moral feelings motivate humans to act on moral rules (Kant, 1786). This distinction
other people’s needs (e.g. “giving money to was important, because feelings or senti-
a beggar”) or on moral values (e.g. “gen- ments as generated from the senses cannot
erosity,” “honesty”), even in the absence have moral significance according to Kant,
of negative consequences for not doing so, because they are not generated by free will –
such as being punished by law enforce- a prerequisite of moral motivation (pp. 15–
ment systems. The importance of moral feel- 17).
ings has been highlighted by philosophers As recognized by philosophers early on,
of the 18th-century Scottish enlightenment there is a close relationship between causal
(Bishop, 1996): most prominently by Fran- agency (i.e., willful action) and moral eval-
cis Hutcheson, Adam Smith, and David uation (Hume, 1777). If people have con-
Hume. Adam Smith stated that the chief trol over their actions and intentionally vio-
“moral sentiment” is “sympathy” (Lamb, late moral rules or act against other people’s
1974). The German philosopher Immanuel needs, we are more likely to blame them
Kant opposed the idea that moral senti- than when this is not the case. Equally, the
ments could be used as a principle of decid- quality of moral feelings changes depending
ing between actions that are morally right on whether we direct blame toward our-
and wrong. Nevertheless, he stressed the selves (e.g., guilt, shame) or others (e.g.,
importance of moral feelings as motiva- indignation). The same is true of praise for
tional forces for moral behavior. In con- our own (e.g., pride) and other people’s
trast to the Scottish school of philosophy, actions: gratitude when being the recipient
he highlighted “respect for the moral law” and awe when being the observer. Sympa-
[Achtung vor dem Moralgesetz] as the only thy (pity, compassion) is felt when observ-
true “moral” motive (Kant, 1786, p. 17). ing other people’s suffering and is usually
This “respect” was, however, not a feel- stronger for people whom we perceive as
ing generated from sensory experience, but victims rather than causal agents of their
was directly generated by the internalized miserable situation. Some feelings, such as
491
492 ROLAND ZAHN, RICARDO DE OLIVEIRA-SOUZA, & JORGE MOLL
contempt, hate, disgust, and anger, can be feeling always at least implicitly entails social
directed both toward ourselves and others knowledge (Moll, Zahn, de Oliveira-Souza,
in a moral context. Krueger, & Grafman, 2005). One impor-
Before we can decompose these com- tant argument for such an inextricable link
plex moral feelings into cognitive compo- is that moral feelings can be distinguished
nents and investigate their neuroanatomi- most clearly from each other by complex
cal basis, we need a definition of what is attributions of causal agency that imply
a feeling and, more specifically, what is a social knowledge, as detailed later. We then
moral feeling. We suggest a working def- briefly touch on evidence that abnormalities
inition here rather than presenting a defi- in the experience of moral sentiments are
nite answer, because there is too little direct important symptoms of specific neuropsy-
evidence to settle disputes about the valid- chiatric disorders. Subsequently, we review
ity of different definitions of these terms. evidence on the neuroanatomical basis of
Here, we use the term “feelings” for com- moral sentiments and summarize opposing
plex subjective emotional experiences that models of how to explain this evidence. We
people would label as “feelings” themselves. then address the question whether the brain
Thus the term “feelings” may refer to quite has developed specialized systems for moral
variable experiences. We use the term “sen- motivations (e.g., helping others or society)
timent” as synonymous with “feeling” and as opposed to selfish motivations (e.g., seek-
avoid using the term “emotion” as much as ing monetary rewards). This chapter closes
we can, because it often has the connotation with a discussion on future directions for this
of specific psychological or neuroanatomical challenging and promising field of research.
models of feelings. We use the terms “moral
feelings” and “sentiments” for those subjec-
tive experiences that enable us to be moti- Causal Attribution and Moral
vated by other people’s needs (i.e., interper- Sentiments
sonal altruism) or sociocultural norms (Moll,
De Oliveira-Souza, & Zahn, 2008). Feelings are very tightly linked with moti-
Moral motivations are defined here as vations and volition. Motivation psychology
consisting of (1) the motivational/emotional has provided rich evidence that humans are
state/force (e.g., “anxiety”) and (2) the goal motivated not only by simple conditioned
with which this motivational state is asso- learning of associations between actions or
ciated (“soothing a child”). This definition goals with pleasant or unpleasant valence,
highlights an important second ingredient but that human motivation entails striv-
for moral motivation: knowing about the ing for understanding ourselves and the
needs of others and about sociocultural world around us (Weiner, 1992). Human
norms (i.e., social knowledge) that serve as motivation has been shown to critically
the goals of moral actions. Social knowl- depend on attributions of causal agency
edge allows us not only to interpret peo- (i.e., the understanding of who carried out
ple’s actions and infer their needs but also to a social action and why she or he did so).
store more general social information such Different qualities of moral sentiments can
as action rules (e.g., “greeting a colleague be directly linked with different types of
at work”) and social concepts describing causal attributions (Weiner, 1985). Table 21.1
values (e.g., “politeness”). One possibility illustrates how different moral sentiments
is that moral feelings are evoked by social can be distinguished on the basis of causal
knowledge, but are themselves devoid of attributions. In addition to the valence of
social knowledge content. Another possibil- the feeling (positive or negative), one can
ity, which we consider to be more plausi- distinguish moral sentiments on the basis
ble, is that social knowledge is inextricably of different types of agency roles (agent,
linked with moral feelings such that a moral recipient, observer) and causal attributions
MORAL EMOTIONS 493
Table 21.1: Association of Causal Attributions, Agency Role, and Valence with Different
Qualities of Moral Sentiments
Modified from Moll, Oliveira-Souza, Zahn, & Grafman (2007). Please note that not all the categoriza-
tions within this table are evidence-based yet, and that there exist discrepancies within the literature.
(For further reading, we recommend Eisenberg, 2000; Fischer & Roseman, 2007; Haidt, 2003; Higgins,
1987; Tangney, Stuewig, & Mashek, 2007; Tracy & Robins, 2006; Tracy, Shariff, & Cheng, 2010; Weiner,
1985.)
Vignemont & Singer, 2006). Empathic sim- mal forms of guilt seem to be more promi-
ulation may be the prerequisite of pity, nent (Alexander, Brewin, Vearnals, Wolff,
because we cannot feel pity for someone & Leff, 1999). Shame-proneness was found
whose feelings we cannot read. However, to be largely increased in people with emo-
being able to simulate other people’s feel- tionally unstable (“borderline”) personal-
ings does not necessarily result in feelings of ity disorder (Rusch et al., 2007). Clinical
pity; it can also result in withdrawal because descriptions of people with manic episodes
of being distressed (Decety & Jackson, 2004). as part of a bipolar disorder report increases
Other-critical (blaming) feelings such as in pride and anger or indignation. However,
indignation or contempt are important for people with manic episodes may develop
the enforcement of sociocultural norms in quite different types of behaviors, probably
societies, especially when law enforcement depending on their overall personality. For
is absent or insufficient. It has been hypoth- example, we observed a patient who gave
esized that punishment of other members all her money away to people in need dur-
of a social group who violate moral rules ing her manic phase, whereas others would
while risking their own lives was of evolu- spend it on shopping for themselves.
tionary importance in enhancing group sur- There is a great need for further exper-
vival in a competition between rival small imental studies probing proneness to spe-
groups (Gintis, Henrich, Bowles, Boyd, & cific moral sentiments in different psychi-
Fehr, 2008). In the next section, we describe atric disorders. Experimental investigations
the importance of keeping the right balance of disturbances of specific moral sentiments
between self- and other-blaming moral feel- in affective disorders are scarce in compar-
ings for our mental health. ison with those investigating disturbances
of “basic” emotions (Elliott, Zahn, Deakin,
& Anderson, 2010). Impairments of spe-
Psychopathology of Moral Sentiments cific moral sentiments, however, are more
likely to account for pathogenetic mecha-
Psychopathology describes unusual experi- nisms specific to one affective disorder (e.g.,
ences or behavior that cause suffering to major depression; Zahn, 2009) compared
an individual or society. A lack of remorse, with others (e.g., panic disorder). The next
guilt, and compassion has been described section describes abnormalities in moral
early on as a core feature of people with sentiments associated with structural brain
“moral insanity,” as it was then called lesions.
(Augstein, 1996; Cleckley, 1976). More
recently, this disorder has been labeled as
“psychopathy” and can be reliably assessed Neuroanatomical Basis of Moral
using psychometric scales that measure Sentiments
callousness as one important component
Evidence from Patients with Brain
(Hare, 2003). On the contrary, an exag-
Lesions
geration of guilt (overgeneralization of
guilt to inappropriate contexts) and other In 1888, Leonore Welt reviewed a series of
types of self-blaming feelings are frequently her own as well as published cases with
observed in people with major depression brain lesions, including the now famous
(Berrios et al., 1992; O’Connor, Berry, Weiss, one of Phineas Gage (Damasio, Grabowski,
& Gilbert, 2002). Increased shame-proneness Frank, Galaburda, & Damasio, 1994), and
has been associated with depressive symp- concluded that the orbitofrontal cortex,
toms in people with no clinical diagnosis especially its right medial aspect, was the
of depression (Tangney et al., 1992), but most consistently affected region associated
was not associated with the severity of with changes in character while leaving gen-
depression in people with a clinical diag- eral intelligence intact. The descriptions of
nosis of major depression in which abnor- character changes were mostly related to
MORAL EMOTIONS 495
social behavior such as talking about inti- Taken together, evidence from patients
mate matters with strangers, or exhibiting with brain lesions reveals that a network
jocular childish and aggressive behavior. of specific fronto-temporo-mesolimbic areas
In the 1980s, with the advent of comput- is necessary for appropriate social behavior
ed tomography, renewed interest in link- (Figure 21.1; Moll, Zahn, et al., 2005). Sub-
ing specific brain lesions in humans with tle volume reductions in this network are
changes in social behavior was sparked by jointly associated with callousness in peo-
Eslinger and Damasio’s description of a ple with developmental psychopathy (de
patient with a ventral frontal lesion (EVR) Oliveira-Souza et al., 2008). In contrast,
who had severe changes in everyday behav- patients with lesions to the lateral parietal
ior and human relationships despite intact lobes, precuneus, posterior cingulate cortex,
executive functioning (Eslinger & Damasio, and medial temporal lobe as occur in typi-
1985). cal Alzheimer’s disease early on (Herholz,
Another source of evidence came from 2003) do not display marked abnormalities
patients with neurodegeneration of the ven- of social behavior (Bozeat, Gregory, Ralph,
tral and anterior temporal cortex, called & Hodges, 2000).
frontotemporal dementia (Neary et al., The challenge lies in identifying the
1998): They consistently display a lack of different contributions that parts of this
empathic concern and inappropriate social network are making to appropriate social
behavior, which caregivers observe as a behavior. Inappropriate social behavior
clear change from their premorbid character could either result from the loss of or defi-
(e.g., one of our patients, a female 80-year- cient access to knowledge of appropriate
old woman, patted male strangers’ bottoms behavior (i.e., social knowledge), the loss
when they bent over in the supermarket of the motivation to act on this knowl-
while expressing no increased sexual desire edge (i.e., moral motivation), or a com-
and actually being sexually inactive). Inter- bination of both. A third mechanism that
estingly, these patients often show intact is widely used to explain inappropriate
executive functions on standard tests such social behaviour is the inability to con-
as the Wisconsin Card Sorting Test (Hodges, trol one’s inappropriate urges. This “dis-
2001). inhibition” or “urge-suppression” model of
Modern MRI and positron emission inappropriate social behavior rests on the
tomography make use of individual differ- assumption that people have access to the
ences in the pattern of the regional dis- knowledge of appropriate social behavior
tribution of pathology within groups of and have the motivation to act on it. Inter-
patients with frontotemporal dementia to estingly, the idea that the frontal cortex acts
tease apart the contributions of anterior to inhibit subcortical limbic urges probably
temporal and ventral frontal regions to the originated from conditioning experiments
degree of abnormal social behavior. It has with nonhuman animals (Brutkowski, 1965);
been shown that both the right anterior tem- although this idea is widely propagated,
poral and ventromedial frontal cortex make we are unaware of compelling support-
independent contributions to inappropriate ing anatomical evidence; namely, a solely
social behavior in these patients (Liu et al., inhibitory pathway from the frontal cortical
2004). Right superior anterior temporal atro- areas to subcortical structures without addi-
phy has been further linked with a lack of tional evidence for excitatory or reciprocal
cognitive and emotional empathy in fron- connections between the same structures. A
totemporal dementia (Rankin et al., 2006). detailed discussion of frontolimbic suppres-
Acquired lesions of subcortical mesolimbic sion models is beyond the scope of this chap-
structures such as the amygdala, hypotha- ter, but we later discuss top-down frontal-
lamus, septal area, and basal ganglia may subcortical control models that are much
also lead to antisocial behavior (reviewed in more in keeping with anatomical evidence,
Moll, de Oliveira-Souza, & Eslinger, 2003). in that they do not claim frontal-subcortical
MORAL EMOTIONS 497
suppression as the sole mechanism of frontal posed by Wood and Grafman (2003). An
control. influential alternative explanation has been
Evidence that inappropriate social behav- that the ventral frontal lobe only stores link-
ior in patients with brain lesions may at ages between subcortical (hypothalamic)
least in part derive from the loss of social motivational states (“somatic markers”) and
knowledge is the finding that neurodegener- social knowledge that is stored elsewhere
ation of the right superior anterior temporal (Bechara, Damasio, & Damasio, 2000).
lobe, recently shown to selectively represent Evidence directly probing the experience
conceptual knowledge of social behavior of different moral sentiments in patients
(e.g., what it means to act “stingily”; Zahn with brain lesions is scarce, but recently the
et al., 2007), leads to the selective loss loss of guilt, embarrassment, and compas-
of conceptual social knowledge while leav- sion has been specifically linked to frontopo-
ing other types of conceptual knowledge lar neurodegeneration in frontotemporal
relatively intact (Zahn, Moll, Iyengar, et dementia (Moll et al., 2011). In addi-
al., 2009). Moreover, patients with selective tion, impairments of guilt and compassion
impairments of conceptual social knowledge were associated with neurodegeneration of
exhibited higher degrees of inappropriate the septal area. These effects were spe-
social behavior (i.e., “disinhibition”) than cific to these sentiments and contrasted
patients without such deficits. These find- with impaired disgust and anger being
ings may also explain why patients with associated with amygdala and dorsomedial
ventral frontal lesions appear to have intact frontal neurodegeneration. Caregiver ques-
social knowledge when the gist of the sit- tionnaires indicate reduced feelings of guilt
uation is summarized for them (Eslinger in people with ventromedial frontal lesions
& Damasio, 1985; Saver & Damasio, 1991), that also include damage to the frontopolar
because this gist description may allow their cortex (Koenigs et al., 2007). Functional neu-
anterior temporal lobes to enable judging of roimaging data in keeping with this lesion
the social appropriateness of behavior. The evidence are reviewed in the next section.
question remains whether ventral frontal
regions store another type of social knowl-
Evidence from Functional
edge that is related more closely to the
Neuroimaging in Healthy Participants
detailed knowledge of action sequences one
needs for real-life decisions, as well as plan- Moral feelings were initially investigated in
ning and executing social behavior, as pro- functional MRI studies using pictures or
←
Figure 21.1. Brain regions involved in moral functions based on evidence from brain lesions and
functional neuroimaging (adapted from Moll, Zahn, de Oliveira-Souza, Krueger, & Grafman, 2005, &
Moll & Schulkin, 2009). Cortical region: frontopolar cortex (FPC), medial and lateral ventral
prefrontal cortex (PFC), right anterior dorsolateral PFC, anterior temporal lobes (aTL), and posterior
superior temporal sulcus (pSTS). Subcortical structures include the extended amygdala,
hypothalamus, basal forebrain (especially the preoptic and septal regions), basal ganglia, and midbrain
regions. Integration across these corticolimbic structures gives rise to event-feature-emotion
complexes (EFEC) by temporal binding according to the fronto-temporo-mesolimbic integration
model (Moll, Zahn, de Oliveira-Souza, Krueger, & Grafman, 2005). The hypothesized
cognitive-anatomical components are the following: (1) Sequential knowledge of actions/events
represented within PFC subregions. FPC: complex branching of consequences of actions and ventral
PFC regions representing associative knowledge of motivational/emotional states embedded into
sequential event/action contexts; (2) social sensory features stored in pSTS and abstract (i.e.
context-independent) conceptual knowledge of social behavior stored in the anterior temporal cortex,
especially in the superior sectors; (3) central motive or basic emotional states, such as “free-floating”
anger, attachment, sadness, and sexual arousal (represented by the subcortical limbic structures listed
earlier).
498 ROLAND ZAHN, RICARDO DE OLIVEIRA-SOUZA, & JORGE MOLL
short verbal statements of moral or no moral Garrido, & Moll, 2009). Subgenual cingu-
relevance as determined by rating scores late activation was selective for guilt versus
obtained before the experiments. Brain acti- other-critical feelings (indignation) when
vation patterns for morally relevant com- ensuring that both conditions were matched
pared with morally irrelevant conditions on negative valence and conceptual detail
were compared while controlling for emo- (Zahn, Moll, Paiva, et al., 2009).
tional intensity. These studies used explicit Pity (compassion, sympathy): Pity is
moral judgments (morally right or wrong) closely related to emotional empathy, which
or tasks without explicit moral decisions is, however, usually conceived as empathic
(i.e., implicitly moral tasks). Taken together, simulation of other people’s feelings rather
the evidence suggests involvement of fron- than feeling pity (for the neural basis
topolar, ventral frontal, anterior temporal, of emotional and cognitive empathy, see
posterior superior temporal sulcus, and Decety & Jackson, 2004; de Vignemont &
mesolimbic regions in morally relevant tasks Singer, 2006; Eslinger, 1998; Shamay-Tsoory,
independent of task demands (reviewed in Aharon-Peretz, & Perry, 2009; see Chap-
Moll, Zahn, et al., 2005). ter 23). Frontopolar activation comparable
Recently, studies have investigated differ- to studies of guilt was found for compas-
ences in activation patterns associated with sion when compared with neutral condi-
specific moral sentiments. Here, we only tions (Immordino-Yang, McColl, Damasio,
report brain regions that have been also & Damasio, 2009) and with control con-
associated with changes in moral behavior ditions evoking self-directed anger (Kedia
in patient lesion studies (see Figure 21.1) et al., 2008) or indignation toward others
and that were systematically investigated in (Moll et al., 2007). Interestingly, the ventral
more than one study. striatum and ventral tegmental area showed
Guilt: Guilt, shame, and embarrassment higher activity for empathic moral senti-
may be quite different with regard to ments (compassion and guilt) than for other-
entailed causal attributions (Eisenberg, 2000; critical feelings (disgust, indignation; Moll
O’Connor et al., 2002; Tangney, Stuewig, et al., 2007).
& Mashek, 2007; Tracy & Robins, 2006; Other-critical (other-blaming) moral
see Table 21.1). Most neuroimaging stud- sentiments: Moral indignation/anger and
ies have focused on guilt, and activation contempt/disgust toward others led to
of the frontopolar cortex is the most con- overlapping patterns of activation in fMRI
sistent finding across studies. This result is studies in the lateral orbitofrontal and
obtained when using different control con- anterior insular cortex (Zahn, Moll, Paiva,
ditions: other-critical feelings (e.g., indigna- et al., 2009). Bilateral orbitofrontal cortex
tion; Moll, Oliveira-Souza, Zahn, & Graf- activation was stronger for indignation and
man, 2007; Zahn, Moll, Paiva, et al., 2009), moral disgust than for nonmoral disgust
embarrassment (Takahashi et al., 2004), or in one study that reported right amygdala
self-directed anger (Kedia, Berthoz, Wessa, activation to be more pronounced for
Hilton, & Martinot, 2008). The first study nonmoral than moral disgust (Moll, de
that examined the neural correlates of guilt Oliveira-Souza, et al., 2005). Another study
using functional imaging reported anterior compared different forms of moral disgust
cingulate activation dorsally to the genu of and disgust for behaviors related to health
the corpus callosum in comparison with a risks for the agent (pathogen disgust) versus
neutral condition (Shin et al., 2000). The an emotionally and morally neutral con-
subgenual portion of the cingulate gyrus was dition. (Borg, Lieberman, & Kiehl, 2008).
detected as guilt-selective, but only when All disgust conditions shared the following
modeling individual differences in either the activations: frontopolar cortices, anterior
frequency of the guilt experience (Zahn, temporal lobes, left lateral orbitofrontal
Moll, Paiva, et al., 2009) or empathic con- cortex, bilateral amygdala, and basal gan-
cern (Zahn, de Oliveira-Souza, Bramati, glia. There was no direct comparison with
MORAL EMOTIONS 499
prosocial moral sentiments such as guilt to were most invariably associated with lateral
investigate whether frontopolar activation orbitofrontal and anterior insular activations
was specific for disgust. Another study when compared with prosocial moral senti-
also found frontopolar cortex activation ments. Few studies have investigated pride
for anger toward others, compared with and gratitude, but activation of mesolimbic
a condition evoking anger toward oneself and basal forebrain regions was found in one
(Kedia et al., 2008). Further, left lateral study (Figure 21.2).
orbitofrontal and right dorsolateral frontal A recent fMRI study has shown that
activity have been shown in fMRI investi- moral and social values are associated
gations of punishment for violating social with the activation of fronto-temporo-
norms relative to a condition in which mesolimbic networks representing the
participants did not expect punishment for abstract context-independent conceptual
those norm violations (Spitzer, Fischbacher, detail (within the anterior temporal lobe)
Herrnberger, Gron, & Fehr, 2007). These and context-dependent moral sentiments
findings may be explained by shared lateral (within different fronto-mesolimbic subre-
orbitofrontal representations of other peo- gions) tied to the same concept (Figure 21.2;
ple’s indignation toward ourselves and our Zahn, Moll, Paiva, et al., 2009). This neu-
own indignation towards others. ral architecture may enable us to commu-
Pride: Adam Smith saw pride as a self- nicate about moral values such as “honor”
interested feeling, postulating a separate across sociocultural groups, even if the feel-
striving for “dignity” as motivating “eth- ings and actions we associate with the same
ical improvement” (Lamb, 1974). David value vary. For example, an environmental
Hume recognized “good and bad forms of activist may associate the action of block-
pride” (Hume, 1777). Neuroimaging studies ing railways to stop transport of atomic
of pride have aimed at the moral variant of waste with the moral values of “honor,”
pride. One study found pride-evoking stim- “courage,” and “dutifulness” and anticipate
uli to be related to right posterior superior feeling pride, whereas a police officer may
temporal sulcus and left anterior temporal associate ensuring dissolution of the block-
lobe activation relative to a neutral con- ade despite adversity as an act of honor,”
dition (Takahashi et al., 2008). Activations “courage,” and “dutifulness” and anticipate
within the mesolimbic reward system (ven- feeling pride as well. Some people may
tral tegmental area) with its projections to despise the values of honor,” “courage,”
the basal forebrain (posterior septum) and and “dutifulness” altogether and feel con-
within the ventral frontopolar cortex were tempt for actions associated with these val-
reported for pride compared with gratitude ues. All these different individuals are nev-
and guilt (Figure 21.2; Zahn, Moll, Paiva, ertheless able to understand the common
et al., 2009). core (context-independent) meaning of the
Taken together, these studies of specific quality of honorable, courageous, or dutiful
moral sentiments point to certain fronto- behavior. Communication between differ-
mesolimbic subregions within the network ent sociocultural groups may be enhanced
being more preferentially activated for some by finding common value dimensions on a
moral sentiments relative to others. Fron- more abstract level. Recent research shows
topolar activations were probably the most that important dimensions of values are
consistent for moral sentiments in general shared across cultures (Schwartz, 1992).
(Moll, Zahn, et al., 2005). Those studies Whereas this section has focused on possible
that compared prosocial moral sentiments neuroanatomical differences between differ-
(especially guilt, compassion) with other- ent types of moral feelings, some would
critical moral sentiments have demonstrated argue that the type of process or task is more
selective activation for prosocial moral senti- important for understanding moral emo-
ments within the frontopolar cortex. Other- tions and cognition than the content. This
critical (other-blaming) moral sentiments debate is summarized in the next section.
500 ROLAND ZAHN, RICARDO DE OLIVEIRA-SOUZA, & JORGE MOLL
Figure 21.2. Using fMRI in healthy participants this study investigated the neuroanatomical basis of
abstract moral and social values (figure adapted from Zahn, Moll, Paiva. et al., 2009). Participants had
to imagine actions in accordance with or counter to a value described by a written sentence and to
decide whether they would feel pleasantly or unpleasantly about the action. After the scan they rated
the unpleasantness/pleasantness on a scale and chose labels that best described their feelings (the
analysis compares each moral sentiment versus visual fixation and versus two other moral sentiments;
only selective effects were reported). There were four experimental conditions: (1) positive
self-agency: “Tom (first name of participant) acts generously toward Sam (first name of best friend)”–
pride in this condition was associated with ventral tegmental, septal, and ventral medial FPC
activation (not depicted); 2) positive other-agency: “Sam acts generously toward Tom” – gratitude in
this condition was associated with hypothalamic activation; (3) negative self-agency: “Tom acts
stingily toward Sam” – guilt in this condition was associated with subgenual cingulate cortex as well as
ventral medial FPC activation (not depicted and only when modeling individual frequency of guilt
trials); and (4) negative other-agency: “Sam acts stingily toward Tom” – indignation/anger in this
condition was associated with lateral orbitofrontal/insular activation. In the center, one can see the
right superior aTL region showing equally strong activation during all moral sentiment and agency
contexts; this region showed increased activity with increasing richness of conceptual details
describing social behavior and is identical to the activation found in a semantic judgment task (Zahn et
al., 2007). These results confirmed the right superior anterior temporal lobe as a context-independent
store of social conceptual knowledge that allows us to understand the core meaning of social and
moral values regardless of what exact feelings or actions we tie to the value. See color plate 21.2.
runaway trolley to save five other individ- anger. Thus, patients with lesions to the
uals. When facing this decision, the emo- ventral medial PFC appeared to decide
tional (intuitive) system is predicted to more “rationally” in moral dilemmas and
prefer avoiding this choice, whereas the cog- more “emotionally” in economic interac-
nitive system is predicted to favor this deci- tions. Our view has therefore been that
sion, because it is the “utilitarian” (i.e., the their performance can neither be accounted
“rational”) choice that leads to the maxi- for by a single mechanism of overall emo-
mum overall benefit (Greene et al., 2004). tional blunting, as predicted by the somatic
The dual-process model thus predicts that marker model, nor by the dual-process
the decision to push the man on the tracks model, in which dorsal cortical cognition
is the result of cognitive brain areas success- suppresses ventral cortical-subcortical emo-
fully overcoming or suppressing the emo- tion (Greene et al., 2004); for a reply to our
tional bias of refraining from this action – an criticism, please see (Greene, 2007).
extension of the influential cognitive con- An alternative account of these findings
trol model (Miller & Cohen, 2001) of moral can be made using a representational model
cognition. of the frontal cortex (Wood & Grafman,
In one study, patients with focal brain 2003), which we adopted in our extension of
lesions that encompassed the ventral medial this model to moral cognition (Moll, Zahn,
PFC bilaterally were exposed (Ciaramelli, et al., 2005). If the frontal cortex is con-
Muccioli, Ladavas, & di Pellegrino, 2007; ceived of as a store of information such as
Koenigs et al., 2007) to “trolley-type” dilem- any other cortical area, then the prediction
mas. They made “utilitarian” decisions more follows that there should be a topographic
often than healthy controls in high-conflict coding depending on the contents and/or
scenarios (highly emotionally aversive deci- format of represented information, as can
sions that result in a greater overall ben- be found for example in the motor cor-
efit; for example, more lives saved). Sev- tex (Wood & Grafman, 2003). As summa-
eral explanations have been offered for this rized in the previous section, there is some
increased bias toward “utilitarian” (rational) evidence from fMRI studies to support the
decisions in patients with ventral medial notion that the frontopolar cortex and the
PFC lesions. One account is that general subgenual region (parts of the ventral medial
emotional blunting and reduced autonomic PFC) are involved more strongly in proso-
signaling due to ventral medial PFC dam- cial moral sentiments (guilt, compassion)
age may have led to an increased bias than other-critical moral feelings (indigna-
toward “rational” decisions – an interpreta- tion, contempt toward others). This could
tion favored by the somatic-marker hypoth- explain a selective decrease in the ability to
esis (Bechara et al., 2000). This explanation, experience guilt and compassion with pre-
however, is not supported by the results served other-critical feelings in patients with
of another study (Koenigs & Tranel, 2007) ventral medial PFC lesions. It would account
investigating the same group of patients for increased anger in the context of unfair
using the two-person ultimatum game. In offers and decreased guilt/compassion when
the ultimatum game, participants anony- faced with moral dilemmas.
mously interact only once with another Our model of moral cognition has
player, and must decide between accept- emphasized the functional integration of
ing an unfair but financially rewarding pro- information between subcortical mesolim-
posal (the economically “rational” choice) bic and frontotemporal cortical areas as the
or rejecting the offer to punish the unfair correlate of reasoning and emotion (Moll,
player (the “emotional” choice; see Chapter Zahn, et al., 2005). According to this fronto-
19). Patients with ventral medial PFC lesions temporo-mesolimbic integration model of
made “emotional” decisions more often than moral cognition, there is no opposition
controls, rejecting unfair offers more often. between “emotions” and “rational thoughts;”
This response is normally accompanied by however, there is competition among different
502 ROLAND ZAHN, RICARDO DE OLIVEIRA-SOUZA, & JORGE MOLL
Table 21.2. Alternative Models of the Role of Frontal Cortex in Social and Moral Behavior
Top-down frontal cognitive control Frontal cortex does not store linkages between
stimulus and response, but it represents goals that
control the information flow in other cortical and
subcortical areas when an automatic response
needs to be overcome (Miller & Cohen, 2001).
Bottom-up subcortical control Ventral (medial) frontal cortex stores linkages of
subcortical “somatic markers” and action
knowledge in posterior brain areas. Explains
problems with rapid and complex decision
making after ventral (medial) frontal lesions
(Bechara, Damasio, & Damasio, 2000).
Reciprocal fronto-temporo-subcortical Context-dependent knowledge of sequences of
integration social actions/events is stored in ventral frontal
(and frontopolar) cortex necessary to enable
context-appropriate social behavior (Moll, Zahn,
de Oliveira-Souza, Krueger, & Grafman, 2005). In
this model the frontal cortex stores sequential
information linking stimuli and responses in
different contexts.
mother-offspring bonding in human as well ings rely on parts of this network to dif-
as nonhuman animal species (Insel & Young, ferent degrees. The frontopolar cortex may
2001) and may be an evolutionary precursor be more important for prosocial moral feel-
to the motivational states enabling humans ings (e.g., guilt) than for other-critical feel-
to act morally. Attachment has been ings (e.g., indignation toward others). Major
shown to depend on oxytocin (Zak, controversies exist, however, with regard
Kurzban, & Matzner, 2004), opioidergic, to the exact function of subregions of this
and monoaminergic (Depue & Morrone- network (especially regarding the role of
Strupinsky, 2005) neurochemistry and has the frontal cortex) and whether one should
been linked with a network of areas overlap- primarily study different types of moral
ping with the mesolimbic reward and basal processes (cognitive vs. emotional) or dif-
forebrain system (Insel & Young, 2001). One ferent types of moral feelings (e.g., guilt
hypothesis is that human moral motivation versus indignation). In addition, there is no
arises by binding attachment-related moti- agreement on how to subdivide the different
vational states within the basal forebrain types of moral feelings. Future studies will
together with complex interpersonal goals have to test more directly the predictions of
and moral value representations in fron- alternative models of moral emotions (top-
totemporal circuits (Moll & Schulkin, 2009). down frontal control, bottom-up subcortical
control, fronto-temporo-mesolimbic inte-
gration).
Conclusions and Future Directions To address these questions, one will
need experiments in which psycholin-
Taken together, there is considerable agree- guistic and other important differences
ment that a fronto-temporo-mesolimbic between conditions are controlled, some-
network of brain regions is associated with thing that has not been done rigorously in
moral feelings. Further it has been sug- many previous studies. Further, the field
gested that different types of moral feel- needs to work to develop criteria for the
504 ROLAND ZAHN, RICARDO DE OLIVEIRA-SOUZA, & JORGE MOLL
Figure 21.3. Brain regions activated when participants donated or were opposed to donate to
charitable organizations during fMRI (Moll et al., 2006); figure adapted from Moll & Schulkin (2009).
(a) Both pure monetary rewards and decisions to donate (with or without personal financial costs)
activated the mesolimbic reward system, including the ventral tegmental area (VTA) and the ventral
and dorsal striatum. (b) The septal-subgenual region (SG), however, was selectively activated by
decisions to donate, as compared with pure monetary rewards (both by costly and noncostly
decisions, conjunction analysis). The lateral orbitofrontal cortex (latOFC) was activated by decisions
to oppose charities. This activation extended to the anterior insula and to the inferior dorsolateral
PFC and was present both for costly and noncostly decisions (conjunction analysis). The FPC and
ventral medial PFC (mPFC) were activated for costly decisions (when voluntarily sacrificing one’s
own money either to donate to a charity or to oppose it (conjunction analysis). See color plate 21.3.
neurobiological validity of cognitive mod- tively for one content/format over another.
els of moral functions. It is still unclear Reproducibility across different types of
whether defining moral functions accord- methods and populations including patients
ing to philosophical traditions (e.g., “util- with brain lesions will be crucial for the val-
itarian” or “non-utilitarian”) is valid for idation of future models of moral cognitive
neurobiological research or whether it is neuroscience. Taken together, the study of
preferable to break down moral function- moral brain functions will profit from the
ing into cognitive components derived from expertise of scientists who have worked in
other fields of cognitive neuroscience, not all more traditional areas of cognitive neuro-
of which need to be specific to moral func- science to improve the methodological and
tions (e.g., “conceptual-semantic,” “sensory- terminological rigor in the field and con-
semantic,” and so on). Further, it will be tribute to tackling the exciting experimental
important to demonstrate process selectiv- questions about one of the most evolution-
ity by showing common involvement of a arily recent functions of the human brain.
brain region across two different types of We are looking toward an exciting and
contents/formats while being selective for stimulating future that allows investigations
one task over another. Conversely, con- of one of the most sophisticated human
tent/format selectivity of a brain region abilities with ever more refined experimen-
could be concluded from involvement of tal rigor. Importantly, these new insights
that brain region regardless of task, but selec- allow novel approaches to improving the
MORAL EMOTIONS 505
prevention and treatment of neuropsychi- the neural correlates of disgust and morality.
atric disorders (Zahn, 2009), as well as bal- Journal of Cognitive Neuroscience, 20(9), 1529–
ancing moral motivations in complex social 46.
systems. Bozeat, S., Gregory, C. A., Ralph, M. A., &
Hodges, J. R. (2000). Which neuropsychiatric
and behavioral features distinguish frontal and
Outstanding Questions and Future temporal variants of frontotemporal dementia
Directions from Alzheimer’s disease? Journal of Neurol-
ogy, Neurosurgery, & Psychiatry, 69(2), 178–86.
r Are there impairments of specific types Brutkowski, S. (1965). Functions of prefrontal
of moral sentiments and judgments after cortex in animals. Physiological Reviews, 45(4),
dorsolateral and ventrolateral frontal 721–46.
Ciaramelli, E., Muccioli, M., Ladavas, E., & di
lesions as compared with ventromedial
Pellegrino, G. (2007). Selective deficit in per-
frontal lesions?
r Do different sectors of the frontal cor- sonal moral judgment following damage to
ventromedial prefrontal cortex. Social Cogni-
tex store different contents/formats of tive & Affective Neuroscience, 2(2), 84–92.
action/event sequences, or are these Cleckley, H. M. (1976). The mask of sanity (5th
regions necessary for content/format- ed.). St. Louis: Mosby.
independent processing of action knowl- Damasio, H., Grabowski, T., Frank, R., Gal-
edge stored elsewhere in the brain? aburda, A. M., & Damasio, A. R. (1994). The
r Are functional MRI associations of spe- return of Phineas Gage: Clues about the brain
cific moral sentiments with specific brain from the skull of a famous patient. Science,
regions reproducible? 264(5162), 1102–5.
r What is the specific functional contribu- Decety, J., & Grezes, J. (2006). The power of
simulation: Imagining one’s own and other’s
tion of particular brain areas in the moral
behavior. Brain Research, 1079, 4–14.
cognition/emotion network? Decety, J., & Jackson, P. L. (2004). The func-
tional architecture of human empathy. Behav-
ioral Cognitive Neuroscience Review, 3, 71–100.
References de Oliveira-Souza, R., Hare, R. D., Bramati, I. E.,
Garrido, G. J., Azevedo Ignácio, F., Tovar-
Alexander, B., Brewin, C. R., Vearnals, S., Wolff, Moll, F., et al. (2008). Psychopathy as a dis-
G., & Leff, J. (1999). An investigation of shame order of the moral brain: Fronto-temporo-
and guilt in a depressed sample. British Journal limbic grey matter reductions demonstrated
of Medical Psychology, 72, 323–38. by voxel-based morphometry. NeuroImage,
Augstein, H. F. (1996). J. C. Prichard’s concept 40(3), 1202–13.
of moral insanity: A medical theory of the Depue, R. A., & Morrone-Strupinsky, J. V.
corruption of human nature. Medical History, (2005). A neurobehavioral model of affiliative
40(3), 311–43. bonding: Implications for conceptualizing a
Bechara, A., Damasio, H., & Damasio, A. R. human trait of affiliation. Behavioral and Brain
(2000). Emotion, decision making and the Sciences, 28(3), 313–350.
orbitofrontal cortex. Cerebral Cortex, 10(3), de Vignemont, F., & Singer, T. (2006). The
295–307. empathic brain: How, when and why? Trends
Berrios, G. E., Bulbena, A., Bakshi, N., Dening, in Cognitive Sciences, 10(10), 435–41.
T. R., Jenaway, F, Markar, H., et al. (1992). Eisenberg, N. (2000). Emotion, regulation, and
Feelings of guilt in major depression – concep- moral development. Annual Review of Psychol-
tual and psychometric aspects. British Journal ogy, 51, 665–97.
of Psychiatry, 160, 781–87. Elliott, R., Zahn, R., Deakin, J. F., & Anderson,
Bishop, J. D. (1996). Moral motivation and the I. M. (2010). Affective cognition and its disrup-
development of Francis Hutcheson’s philoso- tion in mood disorders. Neuropsychopharma-
phy. Journal of the History of Ideas, 57(2), 277– cology. doi:10.1038/npp.2010.77
95. Eslinger, P. J. (1998). Neurological and neuropsy-
Borg, J. S., Lieberman, D., & Kiehl, K. A. (2008). chological bases of empathy. European Neu-
Infection, incest, and iniquity: Investigating rology, 39(4), 193–99.
506 ROLAND ZAHN, RICARDO DE OLIVEIRA-SOUZA, & JORGE MOLL
Eslinger, P. J., & Damasio, A. R. (1985). Severe Insel, T. R., & Young, L. J. (2001). The neurobi-
disturbance of higher cognition after bilateral ology of attachment. Nature Reviews Neuro-
frontal lobe ablation: patient EVR. Neurology, science, 2(2), 129–36.
35(12), 1731–41. Janoff-Bulman, R. (1979). Characterological ver-
Fischer, A. H., & Roseman, I. J. (2007). Beat them sus behavioral self-blame – inquiries into
or ban them: The characteristics and social depression and rape. Journal of Personality and
functions of anger and contempt. Journal of Social Psychology, 37(10), 1798–1809.
Personality and Social Psychology, 93(1), 103–15. Kant, I. (1786). Grundlegung zur Metaphysik der
Gintis, H., Henrich, J., Bowles, S., Boyd, R., Sitten (2nd ed.). Riga: Johann Friedrich Hart-
& Fehr, E. (2008). Strong reciprocity and knoch.
the roots of human morality. Social Justice Kedia, G., Berthoz, S., Wessa, M., Hilton, D., &
Research, 21(2), 241–253. Martinot, J. L. (2008). An agent harms a vic-
Greene, J. D. (2007). Why are VMPFC patients tim: A functional magnetic resonance imaging
more utilitarian? A dual-process theory of study on specific moral emotions. Journal of
moral judgment explains. Trends in Cognitive Cognitive Neuroscience, 20(10), 1788–98.
Sciences, 11(8), 322–323. Koenigs, M., & Tranel, D. (2007). Irrational
Greene, J. D., Nystrom, L. E., Engell, A. D., Dar- economic decision-making after ventromedial
ley, J. M., & Cohen, J. D. (2004). The neural prefrontal damage: Evidence from the Ulti-
bases of cognitive conflict and control in moral matum Game. Journal of Neuroscience, 27(4),
judgment. Neuron, 44(2), 389–400. 951–56.
Greene, J. D., Sommerville, R. B., Nystrom, Koenigs, M., Young, L., Adolphs, R., Tranel, D.,
L. E., Darley, J. M., & Cohen, J. D. (2001). Cushman, F., Hauser, M., et al. (2007). Dam-
An fMRI investigation of emotional engage- age to the prefrontal cortex increases utilitar-
ment in moral judgment. Science, 293(5537), ian moral judgements. Nature, 446(7138), 908–
2105–8. 11.
Haidt, J. (2003). The moral emotions. In R. J. Kringelbach, M. L., & Rolls, E. T. (2004).
Davidson, K. R. Scherer, & H. H. Goldsmith The functional neuroanatomy of the human
(Eds.), Handbook of affective sciences (pp. 852– orbitofrontal cortex: Evidence from neu-
70). Oxford: Oxford University Press. roimaging and neuropsychology. Progress in
Harbaugh, W. T., Mayr, U., & Burghart, D. R. Neurobiology, 72(5), 341–72.
(2007). Neural responses to taxation and vol- Krueger, F., McCabe, K., Moll, J., Kriegesko-
untary giving reveal motives for charitable rte, N., Zahn, F, Strenziok, M., et al. (2007).
donations. Science, 316(5831), 1622–25. Neural correlates of trust. Proceedings of the
Hare, R. D. (2003). The Hare Psychopathy National Academy of Sciences, 104(50), 20084–
Checklist-Revised (2nd ed.). Toronto: Multi- 89.
Health Systems. Lamb, R. B. (1974). Adam Smith’s system: Sym-
Herholz, K. (2003). PET studies in dementia. pathy not self-interest. Journal of the History
Annals of Nuclear Medicine, 17(2), 79–89. of Ideas, 35(4), 671–82.
Higgins, E. T. (1987). Self-discrepancy – a theory Liu, W., Miller, B. L., Kramer, J. H., Rankin, K.,
relating self and affect. Psychological Review, Wyss-Coray, C., Gearhart, R., et al. (2004).
94(3), 319–40. Behavioral disorders in the frontal and tempo-
Hodges, J. R. (2001). Frontotemporal dementia ral variants of frontotemporal dementia. Neu-
(Pick’s disease): Clinical features and assess- rology, 62(5), 742–48.
ment. Neurology, 56(11 Suppl. 4), S6–10. McClure, S. M., Botvinick, M. M., Yeung, J. D.,
Hsu, M., Anen, C., & Quartz, S. R. (2008). The & Cohen, J. D. (2006). Conflict monitoring in
right and the good: Distributive justice and cognition-emotion competition. In J. J. Gross
neural encoding of equity and efficiency. Sci- (Ed.), Handbook of emotion regulation. New
ence, 320(5879), 1092–95. Yor k Guilford Press.
Hume, D. (1777). An enquiry into the principles of Miller, E. K., & Cohen, J. D. (2001). An inte-
morals, Vol. 2. London: T. Cadell. grative theory of prefrontal cortex function.
Immordino-Yang, M. H., McColl, A., Damasio, Annual Review of Neuroscience, 24, 167–202.
H., & Damasio, A. (2009). Neural correlates of Moll, J., de Oliveira-Souza, R., & Eslinger, P. J.
admiration and compassion. Proceedings of the (2003). Morals and the human brain: A work-
National Academy of Sciences, 106(19), 8021–26. ing model. Neuroreport, 14(3), 299–305.
MORAL EMOTIONS 507
Moll, J., de Oliveira-Souza, R., Garrido, G. J., empathy in depression. Journal of Affective
Bramati, I. E., Caparelli-Daquer, E. M. A., Disorders, 71(1–3), 19–27.
Paiva, M. M. F., et al. (2007). The self as a Rankin, K. P., Gorno-Tempini, M. L., Allison,
moral agent: Linking the neural bases of social S. C., Stanley, C. M., Glenn, S., Weiner,
agency and moral sensitivity. Social Neuro- M. W., et al. (2006). Structural anatomy of
science, 2(3 & 4), 336–52. empathy in neurodegenerative disease. Brain,
Moll, J., de Oliveira-Souza, R., Moll, F. T., Igna- 129(11), 2945–56.
cio, F. A., Bramati, I. E., Caparelli-Daquer, Rusch, N., Lieb, K., Gottler, I., Hermann, C.,
E. M., et al. (2005). The moral affiliations of Schramm, E., Richter, H., et al. (2007). Shame
disgust: A functional MRI study. Cognitive & and implicit self-concept in women with bor-
Behavioral Neurology, 18(1), 68–78. derline personality disorder. American Journal
Moll, J., De Oliveira-Souza, R., & Zahn, R. of Psychiatry, 164(3), 500–8.
(2008). The neural basis of moral cognition: Saver, J. L., & Damasio, A. R. (1991). Preserved
Sentiments, concepts, and values. Annals of access and processing of social knowledge in a
the New York Academy of Sciences, 1124(1), 161– patient with acquired sociopathy due to ven-
80. tromedial frontal damage. Neuropsychologia,
Moll, J., Eslinger, P. J., & Oliveira-Souza, R. 29(12), 1241–49.
(2001). Frontopolar and anterior temporal cor- Schwartz, S. H. (1992). Universals in the content
tex activation in a moral judgment task: Pre- and structure of values – theoretical advances
liminary functional MRI results in normal sub- and empirical tests in 20 countries. Advances
jects. Arquivos de Neuro-Psiuiatria, 59(3-B), in Experimental Social Psychology, 25, 1–65.
657–64. Shamay-Tsoory, S. G., Aharon-Peretz, J., &
Moll, J., Krueger, F., Zahn, R., Pardini, M., Perry, D. (2009). Two systems for empathy:
de Oliveira-Souza, R., & Grafman, J. (2006). A double dissociation between emotional and
Human fronto-mesolimbic networks guide cognitive empathy in inferior frontal gyrus
decisions about charitable donation. Proceed- versus ventromedial prefrontal lesions. Brain,
ings of the National Academy of Sciences, 132, 617–27.
103(42), 15623–28. Shin, L. M., Dougherty, D. D., Orr, S. P., Pit-
Moll, J., Oliveira-Souza, R., Zahn, R., & Graf- man, R. K., Lasko, M., Macklin, M. L., et
man, J. (2007). The cognitive neuroscience al. (2000). Activation of anterior paralimbic
of moral emotions. In W. Sinnott-Armstrong structures during guilt-related script-driven
(Ed.), Moral psychology, Vol. 3: Morals and imagery. Biological Psychiatry, 48(1), 43–50.
the brain. Cambridge, MA: MIT Press. Spitzer, M., Fischbacher, U., Herrnberger, B.,
Moll, J., & Schulkin, J. (2009). Social attach- Gron, G., & Fehr, E. (2007). The neural signa-
ment and aversion in human moral cognition. ture of social norm compliance. Neuron, 56(1),
Neuroscience and Biobehavioral Reviews, 33(3), 185–96.
456–65. Takahashi, H., Matsuura, M., Koeda, M., Yahata,
Moll, J., Zahn, R., de Oliveira-Souza, R., Bra- N., Suhara, T., Kato, M., et al. (2008). Brain
mati, I. E., Krueger, F., Tura, B., et al. (2011). activations during judgments of positive self-
Impairment of prosocial sentiments is associ- conscious emotion and positive basic emo-
ated with frontopolar and septal damage in tion: Pride and joy. Cerebral Cortex, 18(4),
frontotemporal dementia. Neuroimage, 54(2), 898–903.
1735–42. Takahashi, H., Yahata, N., Koeda, M., Matsuda,
Moll, J., Zahn, R., de Oliveira-Souza, R., T., Asai, K., & Okubo, Y. (2004). Brain acti-
Krueger, F., & Grafman, J. (2005). Opinion: vation associated with evaluative processes of
The neural basis of human moral cognition. guilt and embarrassment: an fMRI study. Neu-
Nature Reviews Neuroscience, 6(10), 799–809. roimage, 23(3), 967–74.
Neary, D., Snowden, J. S., Gustafson, L., Passant, Tangney, J. P., Stuewig, J., & Mashek, D. J.
U., Stuss, D., Black, S., et al. (1998). Fron- (2007). Moral emotions and moral behavior.
totemporal lobar degeneration: A consensus Annual Review of Psychology, 58, 345–72.
on clinical diagnostic criteria. Neurology, 51(6), Tangney, J. P., Wagner, P., & Gramzow, R.
1546–54. (1992). Proneness to shame, proneness to guilt,
O’Connor, L. E., Berry, J. W., Weiss, J., & and psychopathology. Journal of Abnormal
Gilbert, P. (2002). Guilt, fear, submission, and Psychology, 101(3), 469–78.
508 ROLAND ZAHN, RICARDO DE OLIVEIRA-SOUZA, & JORGE MOLL
Tononi, G., & Koch, C. (2008). The neural cor- ics in Magnetic Resonance Imaging, 20(5),
relates of consciousness – An update. Year in 279–89.
Cognitive Neuroscience, 1124, 239–61. Zahn, R., de Oliveira-Souza, R., Bramati, I.,
Tracy, J. L., & Robins, R. W. (2006). Appraisal Garrido, G., & Moll, J. (2009). Subgenual
antecedents of shame and guilt: Support for a cingulate activity reflects individual differ-
theoretical model. Personality and Social Psy- ences in empathic concern. Neuroscience Let-
chology Bulletin, 32(10), 1339–51. ters, 457(2), 107–10.
Tracy, J. L., Shariff, A. F., & Cheng, J. T. (2010). Zahn, R., Moll, J., Iyengar, V., Huey, E. D., Tier-
A naturalist’s view of pride. Emotion Review, ney, M., Krueger, F., et al. (2009). Social con-
163–77. ceptual impairments in frontotemporal lobar
Weiner, B. (1985). An attributional theory of degeneration with right anterior temporal
achievement-motivation and emotion. Psy- hypometabolism. Brain, 132(Pt. 3), 604–16.
chological Review, 92(4), 548–73. Zahn, R., Moll, J., Krueger, F., Huey, E. D., Gar-
Weiner, B. (1992). Human motivation: Metaphors, rido, G., & Grafman, J. (2007). Social con-
theories, and research. Beverly Hills, CA: cepts are represented in the superior anterior
Sage. temporal cortex. Proceedings of the National
Welt, L. (1888). Über Charakterveränderungen Academy of Sciences, 104(15), 6430–35.
des Menschen. Dtsch Arch Klin Med, 42, 339– Zahn, R., Moll, J., Paiva, M. M. F., Garrido, G.,
90. Krueger, F., Huey, E. D., et al. (2009). The
Wood, J. N., & Grafman, J. (2003). Human neural basis of human social values: Evidence
prefrontal cortex: processing and represen- from functional MRI. Cerebral Cortex, 19(2),
tational perspectives. Nature Reviews Neuro- 276–83.
science, 4(2), 139–47. Zak, P. J., Kurzban, R., & Matzner, W. T. (2004).
Zahn, R. (2009). The role of neuroimaging The neurobiology of trust. Annals of the New
in translational cognitive neuroscience. Top- York Academy of Sciences, 1032, 224–27.
CHAPTER 22
509
510 MARKUS HEINRICHS, FRANCES S. CHEN, GREGOR DOMES, & ROBERT KUMSTA
Following 5 minutes of free speech, the par- outcome measures increase significantly and
ticipant is asked to perform an arithmetic are often measured. These include adrenalin
task (repeatedly subtracting from a starting and noradrenalin, heart rate, systolic and
number, 2,023, in increments of 17), again diastolic blood pressure, growth hormone,
for 5 minutes. The judges are introduced prolactin, testosterone, and immune param-
as experts in monitoring nonverbal behav- eters.
ior, and subjects are informed that the video
recordings will be used to analyze their per-
TSST for Children and TSST for
formance. The panel is composed of two
Groups
or three confederates of the experimenter
who are unacquainted with the subject. The An adapted version of the TSST has been
judges are trained to communicate with the developed for children in the age range of
subject in an neutral manner and to with- 7–14 years. The TSST-C (Buske-Kirschbaum
hold facial or verbal feedback. et al., 1997) also consists of a preparation
period, a free speech, and mental arithmetic
task. In contrast to a job interview, children
Stress Measures
receive the beginning of a story and are told
To capture the entire response dynamic to finish telling it and to make it as exciting as
of the selected outcome measure(s), mul- possible. The numbers of the mental arith-
tiple samples must be taken. These typi- metic task are adapted to the performance
cally cover prestress levels, the initial stress level of the respective age group.
response, peak level, and recovery. A typical Recently, our laboratory has developed a
response curve for ACTH and salivary corti- group version of the TSST (TSST-G; von
sol is shown in Figure 22.2. ACTH levels peak Dawans, Kirschbaum, & Heinrichs, 2011),
immediately after stress, whereas cortisol in which up to six participants can be
shows slightly delayed responses with peak tested at a time. The procedure is compara-
levels around 10–20 min after cessation of the ble to the single-participant version. During
task. About 70–80% of subjects respond to stress exposure, participants are separated
the TSST with increases of 50–200% in corti- by dividing walls that prevent eye contact
sol. In addition to HPA axis hormones, other and interaction with the other participants.
SOCIAL STRESS AND SOCIAL APPROACH 513
Each participant is called on in random order roimaging environment. The MIST com-
to speak for 2 minutes. In the remaining bines mental arithmetic with an adaptive
8 minutes, each participant is required to algorithm and a built-in social comparison
perform a mental arithmetic task for 80 sec- component. The algorithm adjusts the diffi-
onds. Notably, the TSST-G also consists culty of the arithmetic beyond the subject’s
of a specific control condition containing capabilities, so that the subject obtains only
all factors of the TSST-G (e.g., orthosta- 50% correct answers independent of math
sis, speech task, cognitive load, timeline), aptitude. Furthermore, with feedback, the
except for the psychosocially stressful com- subjects are made to believe that they are
ponents (i.e., social-evaluative threat and performing much poorer than the average
uncontrollability. subject. Between runs, subjects are told that
they should try and improve their perfor-
What Makes the TSST Stressful? mance in subsequent runs.
Compared to laboratory stressors that
A meta-analysis of more than 200 stud-
include face-to-face interaction with an
ies using laboratory stress paradigms has
audience such as the TSST, the MIST has
delineated the situational elements capa-
to be considered a less potent psychological
ble of eliciting HPA axis responses (Dick-
stressor. About 50% of the subjects exposed
erson & Kemeny, 2004). It was shown that
to the MIST show significant increases in
robust cortisol responses are observed in
cortisol (responders), whereas nonrespon-
the context of performance tasks that are
ders show the typical circadian decline of
(1) uncontrollable, (2) create a context of
cortisol levels over the course of the exper-
forced failure, or (3) are characterized by
iment. Furthermore, in the responders, the
social-evaluative threat. In these tasks, sub-
overall increases in cortisol are, on average,
jects are unable to avoid negative conse-
30–50%, compared to 50–200% in the TSST.
quences or cannot succeed despite their best
The subgroups of responders and nonre-
efforts, and task performance can be neg-
sponders also differ with respect to brain
atively judged by others. Tasks containing
activation changes after MIST exposure.
both the components of uncontrollability
Distinct brain activation differences could
and social-evaluative threat were associated
be observed in the relevant contrast (mental
with the largest HPA axis responses and the
arithmetic under social threat versus mental
longest recovery times. The TSST is one of
arithmetic under control conditions). In
the few available protocols that satisfies the
the responders, deactivation of specific
criteria of a motivated performance task that
parts of the limbic system, including the
combines these two critical elements (Dick-
hypothalamus, hippocampus, amygdala,
erson & Kemeny, 2004).
and medio-orbitofrontal cortex, were
observed. Furthermore, the degree of deac-
Stress in the fMRI Environment tivation in the hippocampus was correlated
with the overall cortisol stress response
The Montreal Imaging Stress Test
(Pruessner et al., 2008; see also the earlier
To investigate brain activation involved in discussion).
stress responsitivity and stress regulation,
stress paradigms for the functional neu-
Scanstress
roimaging environment have been devel-
oped. As noted earlier, an important Recently, a new stress paradigm for the
situational element to elicit HPA axis acti- fMRI environment was introduced with
vation is social-evaluative threat (i.e., nega- the goal of increasing the social-evaluative
tive judgment of one’s task performance by threat component during task performance.
others). The Montreal Imaging Stress Task In the ScanStress test (Streit et al., in
(MIST; Dedovic et al., 2005) was developed preparation), participants perform a mental
to create social-evaluative threat in a neu- arithmetic task and a mental rotation task.
514 MARKUS HEINRICHS, FRANCES S. CHEN, GREGOR DOMES, & ROBERT KUMSTA
Subjects are required to respond under time 2009). Here, the focus is on genetic factors,
pressure, which is visualized by a count- including gene-environment interactions
down, while the software adapts task speed and epigenetics, and the influence of early
and difficulty to the individual’s perfor- life experience on HPA axis regulation in
mance. Critically, during the scanning pro- adulthood.
cedure as well between the sequences, an
observer panel is presented to the subject
Genetic Factors
though a live video stream, thereby inducing
social-evaluative threat. Although observers It is not the aim of this section to review
remain passive in the control blocks, they in full all candidate gene association stud-
give disapproving visual and verbal feed- ies regarding HPA axis regulation. Instead,
back during the task blocks and between it provides a few examples of the most
the sequences. In a first evaluation study, relevant candidates – glucocorticoid recep-
ScanStress was shown to induce signifi- tor (GR) and mineralocorticoid receptor
cant ACTH and cortisol responses, and (MR) – and of polymorphisms studied in
on the neural level, a distinct pattern of other contexts of emotion regulation (5-
activation and deactivation became evident HTTLPR, GABA, BDNF).
(unpublished data). Recently, ScanStress Twin studies have shown that HPA axis
and the MIST tests have been used in an responses to acute challenge are signifi-
investigation of the effects of city living cantly influenced by genetic factors (Wüst
and urban upbringing on stress processing. et al., 2004). Sequencing of the human
Urban upbringing affected the anterior cin- genome has made it possible to associate
gulate cortex, a key region for regulation polymorphisms in HPA axis-related genes
of amygdala activity, and current city liv- with differences in HPA axis response pat-
ing was associated with increased amygdala terns. Because the HPA axis is a complexly
activity (Lederbogen et al., 2011). regulated system, multiple genes involved
in different peripheral and central pathways
contribute to interindividual response vari-
Determinants of HPA Axis Reactivity ability. Putative candidates are the genes
coding for hormones and receptors impli-
With the availability of standardized labora- cated in the direct regulation of HPA axis
tory stress protocols, extensive phenotyping activity (e.g., the glucocorticoid receptor)
of HPA axis stress responses has been and those involved in indirect modulation
performed over the past years. Both from (e.g., opioidergic, catecholaminergic, or
a basic research perspective and to obtain GABAergic modulation). Other candi-
a better understanding of the mechanisms dates are the genes coding for proteins or
linking stress and mental health, knowledge enzymes involved in cortisol biosynthesis,
of intervening and moderating variables bioavailability (e.g., 11-beta hydroxysteroid
that affect HPA axis responses is important. dehydrogenases), transport (e.g., CBG), or
Research has shown that a consistent fea- systemic absorption (e.g., P-Glycoprotein)
ture of HPA axis responses to psychosocial or factors playing a role in the cellular
challenge is large intra- and interindividual pathways transducing the cortisol signal
variability. The identified factors include (e.g., heat-shock proteins or cofactors).
gender, age, endogenous and exogenous sex The genes coding for the two receptors
steroid levels, smoking, coffee and alcohol that mediate the effects of cortisol have
consumption, dietary energy supply, lac- been considered prime candidates for HPA
tation and breastfeeding, and personality axis regulation. Cortisol operates through
factors. Not all of these factors can be a dual receptor system, which consists of
discussed here in detail, and the reader is the high-affinity mineralocorticoid receptor
referred to more comprehensive reviews (MR) and the 10-fold lower affinity gluco-
(e.g., Kudielka, Hellhammer, & Wüst, corticoid receptor (GR). The GR modulates
SOCIAL STRESS AND SOCIAL APPROACH 515
adults with a history of childhood abuse. elapsed since exposure to traumatic experi-
These observations raised the question of ence, as well as by the nature and chronic-
how the long-lasting health consequences ity of the stressful experience (Miller,
of early adverse rearing conditions are sus- Chen, & Zhou, 2007). Because control of
tained. Research on the biological mecha- the HPA axis involves multiple regulatory
nisms linking early adversity and poor health mechanisms allowing for counter-regulation
outcomes has shown an important mediat- at different levels, these observations are
ing role of physiological stress response sys- not surprising. The crucial points, how-
tems. ever, are that adverse experiences lead to
A number of studies have demonstrated changes in HPA axis response dynamics and
an association between childhood trauma that both hyper- and hypo-reactivity have
(neglect, sexual and physical abuse) and been associated with physical and mental
dysregulations of the HPA axis. The HPA illness.
axis has been suggested as both a target
for environmental influences and a mediator
Gene-Environment Interactions
of the relationship between early adversity
and mental health in adulthood. The cen- Despite the strong link between early
tral hypothesis is that early trauma leads to trauma and increased risk for psychopathol-
dysregulations of the HPA axis and sensiti- ogy, there is extensive evidence of a huge
zation to the effects of acute stress (Heim heterogeneity in the response of individu-
& Nemeroff, 2001). Childhood trauma has als to all environmental hazard, both physi-
been associated with alterations in HPA axis cal and psychosocial (Rutter, 2006). Accord-
response dynamics. For instance, following ingly, not all individuals with childhood
the TSST, increased HPA axis responses adverse experiences develop mental health
were observed in women with abuse experi- problems. One important part of the mech-
ence, most robustly in subjects with current anism involved in outcome heterogene-
symptoms of depression and anxiety (Heim ity concerns gene-by-environment (GxE)
et al., 2000). In response to different phar- interaction, observed when the effect of
macological provocation tests to assess HPA exposure to an environmental pathogen on
axis dynamics, dysregulations became evi- health outcomes is moderated by geno-
dent on multiple regulatory levels (Heim, type (Moffitt, Caspi, & Rutter, 2005). For
Newport, Mletzko, Miller, & Nemeroff, instance, the common promoter polymor-
2008), and the evidence suggests that child- phism in the serotonin transporter gene
hood abuse leads to an increased drive of (5-HTTLPR) has repeatedly been shown to
the HPA axis and associated disinhibition moderate the likelihood of whether an indi-
of central stress responses. However, Car- vidual develops depression following early
penter et al. (2009) found decreased cortisol life stress (Karg et al., 2011). Such find-
levels and decreased ACTH responses to the ings for GxE are also emerging for HPA
TSST in a sample of men reporting child- axis genes with regard to depression, PTSD,
hood maltreatment. Furthermore, relative and HPA axis regulation in patients with a
hypocortisolism has been observed in PTSD, history of childhood trauma. For instance,
reflected by low basal cortisol concentra- Bradley, et al. (2008) studied 15 SNPs of the
tions, low awakening cortisol response, and CRH1 receptor gene (total sample: n = 621)
increased ACTH and cortisol suppression and identified an interaction between two
following low-dose dexamethasone admin- CRHR1 haplotypes and child abuse predict-
istration (Yehuda, 2002). ing both current depressive symptoms and
Thus, trauma can produce both exagger- lifetime diagnosis of major depression. The
ated or dampened stress system reactivity, most significant association was found for
and the nature of HPA axis dysregulation the haplotype formed of the TCA alleles
(i.e., hyper- versus hypo-reactivity) seems to of SNPs rs7209436, rs4792887, and rs110402
be determined in part by the time that has spanning intron 1, which seems to confer
518 MARKUS HEINRICHS, FRANCES S. CHEN, GREGOR DOMES, & ROBERT KUMSTA
Heinrichs, 2011). Recent research also sug- combined with established behavioral and
gests that impaired functioning of the oxy- neuroimaging paradigms to clarify OXT’s
tocin system contributes to the difficulties actions in the human brain. Intranasal
associated with social approach behavior administration provides a relatively nonin-
experienced by individuals with disorders vasive method for delivering OXT directly
such as autism and social anxiety disorder. to the brain (Born et al., 2002; Heinrichs &
OXT, especially in combination with psy- Domes, 2008).
chotherapy, is emerging as a promising com- Peripheral levels of OXT can be mea-
ponent of novel treatment approaches for sured through the plasma. Although these
these and other mental disorders character- results are intriguing, the relationship
ized by social dysfunction. between peripheral levels and CNS avail-
ability of OXT remains unclear (Anderson,
2006; Carter et al., 2007; Landgraf & Neu-
Neurophysiological Bases
mann, 2004). Therefore, further investiga-
OXT is synthesized in magnocellular neu- tion of this issue will be necessary before
rons in the paraventricular and supraop- conclusions can be drawn about a direct rela-
tic nuclei of the hypothalamus. It is pro- tionship between peripheral OXT levels and
cessed along the axonal projections to the brain function.
posterior lobe of the pituitary, where it is A third approach to studying OXT effects
stored in secretory vesicles and released into involves the measurement of OXT levels
peripheral circulation. It is released from in cerebrospinal fluid (CSF). The level of
both axonal terminals and dendrites into the neuropeptides in CSF has been shown to
extracellular space, resulting in both local reflect their immediate availability in the
action and diffusion to reach distant tar- brain (Born et al., 2002), which is more
gets in the brain (Ludwig & Leng, 2006). In directly relevant for behavioral effects or
addition, smaller OXT-producing parvocel- psychopathology than peripheral neuropep-
lular neurons in the paraventricular nucleus tide levels (Heinrichs & Domes, 2008). How-
project directly to other regions in the brain, ever, obtaining CSF samples involves an
including the amygdala, hippocampus, stria- invasive methodology that cannot be easily
tum, suprachiasmatic nucleus, bed nucleus used in routine human research.
of stria terminalis, and brainstem, where Molecular genetic methods are well
OXT acts as a neuromodulator or neuro- suited for investigating natural interindi-
transmitter. For example, OXT modulates vidual variations in the OXT system and
neural populations in the central amygdala their implications for human social behav-
(Viviani et al., 2011). For an overview of ior (Kumsta & Heinrichs, 2012). In several
studies on endogenous levels of OXT and studies, a single nucleotide polymorphism
human behavior, see Heinrichs et al. (2009). (SNP) located in the third intron of the
For an overview of the neurogenetic mech- oxytocin receptor gene (OXTR), rs53576
anisms of the OXT system including neu- (G/A), has been associated with socioemo-
roimaging studies, see Kumsta & Heinrichs tional functioning. Specifically, the A allele
(2012). of rs53576 has been associated with a larger
startle response (Rodrigues, Saslow, Garcia,
John, & Keltner, 2009) and reduced amyg-
Methodological Issues of Oxytocin
dala activation during emotional face pro-
Research in Humans
cessing (Inoue et al., 2010). It has also been
Researchers have used several methods to linked to increased risk for autism (Lerer
study the functioning of the oxytocin sys- et al., 2008), reduced maternal sensitivity
tem in humans. The remainder of this to child behavior (Bakermans-Kranenburg
chapter focuses on studies that have used & van Ijzendoorn, 2008), lower empathy
the method of intranasal administration, (Rodrigues et al., 2009) and, in men, reduced
which in recent years has been successfully positive affect (Lucht et al., 2009).
520 MARKUS HEINRICHS, FRANCES S. CHEN, GREGOR DOMES, & ROBERT KUMSTA
Figure 22.3. (A) Oxytocin improved performance in the Reading the Mind in the Eyes Test (RMET)
compared to placebo. (B) Performance in the RMET as a function of item difficulty: Oxytocin
improved performance on the difficult items and not on the easy items. Figure modified from Domes
et al. (2007), with permission from
C 2007, Society of Biological Psychiatry.
Visual attention to neutral and emotional during preparation for the Trier Social
faces plays a critical role in the recognition Stress Test (TSST). Subjects receiving
of facial emotions (Adolphs, 2002). To date, social support and OXT showed the
four studies have examined the effects of lowest cortisol response to the TSST,
OXT on patterns of visual attention to faces. whereas subjects who received no social
With one exception (Domes et al., 2010), support and placebo showed the highest
these studies have reported increased gazing response (Figure 22.4; Heinrichs, Baumgart-
time on the eye region compared to other ner, Kirschbaum, & Ehlert, 2003). Notably,
parts of the face (Domes, Steiner, Porges, subjects who received social support and
& Heinrichs, 2012). Although these results OXT also reported the lowest levels of
suggest that improved facial emotion recog- subjective stress (lower anxiety and higher
nition after OXT treatment might be due calmness). The stress-buffering effect of
at least in part to increased eye gaze, this OXT has been replicated in other recent
hypothesis has not yet been explicitly tested. studies (Quirin, Kuhl, & Dusing, 2011).
Because several mental disorders involving Complementary interindividual differences
social deficits have been linked to impaired in stress responses have been linked to the
or biased processing of emotional expres- variation in the oxytocin receptor gene: The
sions as well as reduced or atypical gaze to OXTR rs53576 AA genotype has been asso-
the eye region, a more complete understand- ciated both with a reduced tendency to
ing of the conditions under which oxytocin seek social support (Kim et al., 2010) and
can enhance emotion recognition and eye to reduced physiological and psychological
gaze will likely have translational implica- responsivity to social support (Chen, Kum-
tions for the treatment of these disorders. sta, von Dawans, Monakhov, Ebstein, &
Compared to studies investigating the Heinrichs, 2011).
effects of OXT on more cognitive aspects In a study on couples, OXT administra-
of empathy such as emotion recognition, tion increased positive communication and
research on OXT effects on emotional reduced plasma cortisol levels in both men
empathy (i.e., the vicarious feeling of an and women during a conflict (Ditzen et al.,
emotion) has been relatively rare to date 2009), suggesting that central OXT facili-
(Hurlemann et al., 2010). Recent studies tates human pair bonding in a manner paral-
have begun to address this gap in the lit- lel to that observed in prior animal studies.
erature, with one study reporting positive In men with an insecure attachment pat-
effects of intranasal OXT on emotional tern, OXT enhanced secure interpretations
empathy but not cognitive empathy (Hurle- of ambiguous attachment-related scenarios
mann et al., 2010). (Buchheim et al., 2009). Because secure
attachment in humans is associated with
lower stress reactivity and a better ability to
Oxytocin, Stress, and Social Support
interact socially (Ditzen et al., 2009), under-
A number of studies suggest that OXT standing the role of OXT in attachment may
dampens the typical endocrine response to have clinical implications for several mental
stressful social interaction, which includes and developmental disorders associated with
HPA axis activation and the secretion of stress and impairments in social approach
CRH, ACTH and cortisol (see the ear- behavior.
lier discussion). Breastfeeding, which is These results suggest that OXT enhances
associated with endogenous secretion of the buffering effect of positive social inter-
OXT, has been associated with damp- actions on stress responsivity, although the
ened cortisol responses to psychosocial stres- underlying biological and developmental
sors in women (Heinrichs et al., 2001). mechanisms of this effect remain unclear
In another study, healthy males were ran- (Gamer & Buchel, 2012). It is likely that
domly assigned to receive social support the baseline sensitivity of the CNS to OXT
or no social support and OXT or placebo is influenced by significant events occurring
522 MARKUS HEINRICHS, FRANCES S. CHEN, GREGOR DOMES, & ROBERT KUMSTA
Figure 22.4. Mean salivary free cortisol concentrations (± SEM) during psychosocial stress exposure
(Trier Social Stress Test). Participants were randomly assigned to receive intranasal oxytocin (24 IU)
or placebo and either no social support or social support from their best friend before stress. The
shaded area indicates the period of the stress tasks (public speaking followed by mental arithmetic in
front of a panel of evaluators). The areas under the individual response curves (AUC) represent
cumulative cortisol release (calculated by aggregating data from eight saliva sampling points)
throughout the session. Significant interaction effects on cortisol were observed (social support by
time effect, p < .001; social support by oxytocin by time effect, p < .01). Figure modified from
Heinrichs et al. (2003), with permission from C 2003 Society of Biological Psychiatry.
early in life. Early parental separation stress, improved recognition memory for faces but
for example, has been shown to reduce the not for nonsocial stimuli (Rimmele, Hedi-
suppressive effect of OXT on cortisol lev- ger, Heinrichs, & Klaver, 2009). In one study,
els (Meinlschmidt & Heim, 2007). Because intranasal OXT administered before a learn-
stress increases risk for many psychiatric dis- ing task enhanced memory for happy faces
orders, whereas positive social interactions over angry and neutral faces (Guastella,
decrease it, further research on these issues Carson, Dadds, Mitchell, & Cox, 2009).
may enhance possibilities for treatment and Whether and how OXT administration
early intervention for these disorders. influences memory for specific emotions
remains unclear.
Oxytocin and Social Memory
Oxytocin and Approach Behavior
In one early study, OXT was associated
with an impairment of general semantic In many mammalian species, OXT pro-
memory (Fehm-Wolfsdorf, Born, Voigt, & motes social approach behavior and reduces
Fehm, 1984). However, more recent stud- the tendency to avoid proximity with unfa-
ies suggest that OXT selectively modu- miliar others. In humans, OXT increases
lates social memory. In one study, intranasal trust, which can be considered an indi-
OXT selectively reduced men’s implicit cator of psychological readiness for social
memory of socially relevant (and not neu- approach. In the first study to investigate the
tral) words (Heinrichs, Meinlschmidt, Wip- role of OXT in interpersonal trust (Kosfeld,
pich, Ehlert, & Hellhammer, 2004). In Heinrichs, Zak, Fischbacher, & Fehr, 2005),
another study, intranasal OXT selectively participants who had received OXT showed
SOCIAL STRESS AND SOCIAL APPROACH 523
0.50
0.45
0.40
Relative frequency
0.35
0.30
0.25
0.20
0.15
0.10
0.05
0.00
0 1 2 3 4 5 6 7 8 9 10 11 12
Individual investor's average transfer (MUs)
Trust experiment
B
0.50
0.45
0.40
Relative frequency
0.35
0.30
0.25
0.20
0.15
0.10
0.05
0.00
0 1 2 3 4 5 6 7 8 9 10 11 12
Individual investor's average transfer (MUs)
Risk experiment
Figure 22.5. Transfers in the trust and risk experiments. Each observation represents the average
transfer amount (in monetary units, MU) per investor across four transfer decisions. (A) Relative
frequency of investors’ average transfers in the oxytocin (filled bars) and placebo (open bars) groups
in the trust experiment: Subjects given oxytocin showed significantly higher transfer levels. (B)
Relative frequency of investors’ average transfers in the oxytocin (filled bars) and placebo (open bars)
groups in the risk experiment: Subjects in the oxytocin and the placebo group showed statistically
identical transfer levels. Figure modified from Kosfeld, Heinrichs et al. (2005), with permission from
C 2005, Nature Publishing Group.
greater willingness to take social risks in a then, Fischbacher, & Fehr, 2008). After sev-
trust game relative to the placebo group eral rounds of a trust game, participants
(Figure 22.5). Notably, OXT increased par- were informed that their social partners had
ticipants’ willingness to take risks only when made selfish decisions that were disadvan-
the interaction involved a social component. tageous to the participant (i.e., betrayed
Another study showed that OXT can the participant’s trust). Whereas partici-
sustain trusting behavior after a breach pants who had received placebo subse-
of trust (Baumgartner, Heinrichs, Vonlan- quently made decisions indicating reduced
524 MARKUS HEINRICHS, FRANCES S. CHEN, GREGOR DOMES, & ROBERT KUMSTA
trust of the social partner, participants who on social behavior, links between atypical
had received intranasal OXT continued to levels of endogenous oxytocin and mental
make decisions indicating sustained trust. In disorders (for a review, see Heinrichs et al.,
another study, OXT increased motivation 2009), associations between OXTR poly-
for continued social interaction following an morphisms and social behavior, and associ-
experience of inclusion (in a virtual ball- ations between OXTR polymorphisms and
tossing game, “cyberball”), although it did risk for mental disorders characterized by
not buffer against the negative feelings asso- severe social deficits (Kumsta & Heinrichs,
ciated with blunt social ostracism (Alvares, 2012). Because only a small fraction of intra-
Hickie, & Guastella, 2010). venously administered neuropeptide passes
Evidence of OXT’s effects on social through the blood-brain barrier, the method
approach-related behaviors and cognitions of intravenous administration of OXT has
has been found in other domains. OXT limited applicability in clinical settings. Fur-
has also been shown to enhance percep- thermore, intravenous infusion could poten-
tions of facial trustworthiness and attractive- tially have side effects due to actions on hor-
ness (Theodoridou, Rowe, Penton-Voak, & mone systems. Currently, the most promis-
Rogers, 2009). In one study, OXT posi- ing clinical intervention method is intranasal
tively affected the responsiveness of fathers administration, which affords a direct path-
toward their toddlers, thereby possibly pro- way to the human brain (Born et al., 2002;
moting positive interactions (Naber, van Heinrichs et al., 2009).
Ijzendoorn, Deschamps, van Engeland, & So far, no systematic, randomized control
Bakermans-Kranenburg, 2010). The social trials on the therapeutic effects of intranasal
context appears to be a critical modulator OXT treatment have been completed. To
of OXT’s effects on social interaction: Brief fully assess the therapeutic value of OXT,
prior face-to-face contact with a social part- large-scale neuropharmacological studies on
ner has been found to enhance the effects clinical samples that systematically manip-
of OXT on cooperative or prosocial behav- ulate neuropeptide availability in the CNS
ior (Declerck, Boone, & Kiyonari, 2010), and are necessary. However, preclinical stud-
ingroup versus outgroup membership mod- ies in patients have demonstrated promising
ulates the effects of OXT on cooperation results of a single dose of intranasal OXT
(Chen et al., 2011; De Dreu et al., 2010; De on various mental disorders. The following
Dreu, Greer, Van Kleef, Shalvi, & Hand- section reviews recent advances using OXT
graaf, 2011). as part of a therapeutic program for psy-
In summary, OXT appears to increase chopathological states.
motivation to engage in social interactions, In healthy individuals, intranasal admin-
enhance the ability to decode and recall key istration of OXT improves social cogni-
social cues such as facial expressions of emo- tion, emotion recognition, secure attach-
tion, and promote trusting behavior, coop- ment, and empathy (Buchheim et al., 2009;
eration, and willingness to take social risks. Ditzen et al., 2009; Domes, Heinrichs,
The few studies that have directly investi- Michel, et al., 2007; Guastella, Mitchell, &
gated the specificity of these effects through Dadds, 2008; Heinrichs et al., 2004; Rim-
the inclusion of both social and nonsocial mele et al., 2009), reduces physiological and
stimuli suggest that the effects are more pro- psychological stress responses (Heinrichs et
nounced for social stimuli (Norman et al., al., 2003), mediates stress-protective conse-
2011; Rimmele et al., 2009). quences of social support (“social buffer-
ing;” Heinrichs et al., 2003), and attenu-
ates amygdala reactivity to social stimuli;
Translational Perspectives for Social
Baumgartner et al., 2008; Domes, Heinrichs,
Neuropeptides
Glascher, et al., 2007; Gamer et al., 2010;
OXT’s potential therapeutic value for social Kirsch et al., 2005. Pharmacological inter-
disorders is clearly evidenced by its effects vention in the OXT system represents a
SOCIAL STRESS AND SOCIAL APPROACH 525
+ CENTRAL +
SOCIAL STRESS-BUFFERING OXYTOCINERGIC OXYTOCIN
- SYSTEM STIMULATION
PSYCHO-
HEALTH
PATHOLOGY
(e.g., autism, social
anxiety disorder)
Figure 22.6. Integrative translational model of the interactions of the oxytocin system, social approach
behavior, and social stress in humans. Left: Social stress and social anxiety stimulate the
amygdala–cingulate circuit and the hypothalamic–pituitary–adrenal (HPA) axis. In healthy
individuals, stress and anxiety encourage social approach behavior as a coping strategy. They also
stimulate oxytocin release, which further promotes social approach behavior. Furthermore, positive
social interaction (e.g., physical contact) is itself associated with OXT release and therefore promotes
continued social approach behavior. OXT reduces amygdala and HPA axis reactivity to social
stressors and as such serves as an important mediator of the anxiolytic and stress-protective effects of
positive social interaction (“social buffering”). Right: Patients with mental and developmental
disorders characterized by severe deficits in social interactions (e.g., autism, social anxiety disorder,
borderline personality disorder) may benefit from novel “psychobiological therapy” approaches in
which psychotherapy is combined with the administration of OXT or OXT receptor agonists. Figure
modified from Heinrichs & Domes (2008), with permission from C 2008, Elsevier.
particularly promising new angle for treat- 1994) and thus represents a major public
ment for those with pathologies in these health issue. SAD is characterized by a
domains (Figure 22.6). Especially in com- fear of negative evaluation by others and
bination with interaction-based psychother- extreme anxiety and discomfort before, dur-
apy, administration of OXT or selective and ing, and after exposure to social settings. In
longer acting OXTR agonists such as carbe- one study, patients with SAD were admin-
tocin may represent an effective treatment istered intranasal OXT and participated
option for mental disorders characterized in five weekly sessions of brief exposure
by extreme difficulties in social interactions intervention (Guastella, Howard, Dadds,
and/or disrupted attachment relationships, Mitchell, & Carson, 2009). OXT adminis-
such as social anxiety disorder and autism tration improved speech performance over
spectrum disorder (Meyer-Lindenberg et al., the course of the exposure sessions; how-
2011). Initial results of experimental neu- ever, possibly because of the low frequency
ropeptide administration in these patient of sessions, a more generalized overall
groups have been encouraging, especially improvement in treatment outcome was not
given the fact that these “social disorders” observed. In another study (Labuschagne
are notoriously difficult to treat or as, in the et al., 2010), SAD patients and healthy con-
case of autism, currently cannot be effec- trols were administered intranasal OXT or
tively treated at all. placebo and performed an emotional face
Social anxiety disorder (SAD) is the third matching task involving pictures of fear-
most common mental health disorder after ful, angry, and happy faces. In the placebo
depression and alcoholism (Kessler et al., condition, patients with SAD showed a
526 MARKUS HEINRICHS, FRANCES S. CHEN, GREGOR DOMES, & ROBERT KUMSTA
social brain and to tailor new treatment for the etiology of depression. Psychoneuroen-
strategies sensitive to individual differences. docrinology, 34(9), 1294–1303.
These studies may also clarify how the Alvares, G. A., Hickie, I. B., & Guastella, A. J.
neuroanatomical distribution and sensitiv- (2010). Acute effects of intranasal oxytocin
ity of OXT receptors are influenced by on subjective and behavioral responses to
social rejection. Experimental and Clinical Psy-
variations in the regulatory regions of their
chopharmacology, 18(4), 316–21.
respective genes. Overall, the tremendous
Andari, E., Duhamel, J. R., Zalla, T., Herbrecht,
growth in this research field offers not only E., Leboyer, M., & Sirigu, A. (2010). Promot-
a promising new path for exploring the ing social behavior with oxytocin in high-
neuroendocrinology of the social brain but functioning autism spectrum disorders. Pro-
also a translational perspective for devel- ceedings of the National Academy of Sciences,
oping novel treatment strategies for social 107(9), 4389–94.
disorders. Anderson, G. M. (2006). Report of altered uri-
nary oxytocin and AVP excretion in neglected
orphans should be reconsidered. Journal of
Outstanding Questions and Future Autism and Developmental Disorders, 36(6),
Directions 829–30.
Bakermans-Kranenburg, M. J., & van Ijzendoorn,
r How could stress response measures (hor- M. H. (2008). Oxytocin receptor (OXTR) and
monal and/or neural) in combination serotonin transporter (5-HTT) genes associ-
with genetic information be used to tailor ated with observed parenting. Social, Cognitive
& Affective Neurosciences, 3(2), 128–34.
therapeutic approaches for stress-related
Baron-Cohen, S., Wheelwright, S., Hill, J., Raste,
disorders? Y., & Plumb, I. (2001). The “Reading the Mind
r Epigenetic mechanisms have been shown
in the Eyes” Test revised version: A study with
to influence HPA axis regulation; how- normal adults, and adults with Asperger syn-
ever, the effects seem to be tissue spe- drome or high-functioning autism. Journal of
cific. It will be critical to investigate Child Psychology and Psychiatry, 42(2), 241–51.
whether peripheral, easy-to-obtain mark- Bartz, J. A., & Hollander, E. (2008). Oxytocin and
ers in humans will be useful; that is, experimental therapeutics in autism spectrum
reflect epigenetic changes in central ner- disorders. Progress in Brain Research, 170, 451–
vous tissue. 62.
r Which methods of administration of neu- Bartz, J. A., Zaki, J., Bolger, N., Hollander, E.,
Ludwig, N. N., Kolevzon, A., et al. (2010).
ropeptides that penetrate the brain are
Oxytocin selectively improves empathic accu-
currently available? How could their racy. Psychological Science, 21(10), 1426–28.
effectiveness (e.g., half-life time, speci- Baumgartner, T., Heinrichs, M., Vonlanthen, A.,
ficity) be improved? Fischbacher, U., & Fehr, E. (2008). Oxytocin
r How could a combination of genotyping
shapes the neural circuitry of trust and trust
of neuropeptide receptor gene polymor- adaptation in humans. Neuron, 58(4), 639–50.
phisms and neuropeptide administration Binder, E. B., Bradley, R. G., Liu, W., Epstein,
improve potential clinical effects? M. P., Deveau, T. C., Mercer, K. B., et al.
(2008). Association of FKBP5 polymorphisms
and childhood abuse with risk of posttrau-
References matic stress disorder symptoms in adults.
Journal of the American Medical Association,
Adolphs, R. (2002). Recognizing emotion from 299(11), 1291–1305.
facial expressions: psychological and neuro- Born, J., Lange, T., Kern, W., McGregor, G.
logical mechanisms. Behavioral and Cognitive P., Bickel, U., & Fehm, H. L. (2002). Sniff-
Neuroscience Review, 1(1), 21–62. ing neuropeptides: A transnasal approach to
Alexander, N., Kuepper, Y., Schmitz, A., Osin- the human brain. Nature Neuroscience, 5(6),
sky, R., Kozyra, E., & Hennig, J. (2009). 514–16.
Gene-environment interactions predict corti- Bradley, R. G., Binder, E. B., Epstein, M. P.,
sol responses after acute stress: Implications Tang, Y., Nair, H. P., Liu, W., et al. (2008).
528 MARKUS HEINRICHS, FRANCES S. CHEN, GREGOR DOMES, & ROBERT KUMSTA
Influence of child abuse on adult depression: tional imaging to investigate the effects of per-
Moderation by the corticotropin-releasing ceiving and processing psychosocial stress in
hormone receptor gene. Archives of General the human brain. Journal of Psychiatry & Neu-
Psychiatry, 65(2), 190–200. roscience, 30(5), 319–25.
Bremner, J. D., Southwick, S. M., Johnson, D. R., De Dreu, C. K., Greer, L. L., Handgraaf, M.
Yehuda, R., & Charney, D. S. (1993). Child- J., Shalvi, S., Van Kleef, G. A., Baas, M., et
hood physical abuse and combat-related post- al. (2010). The neuropeptide oxytocin regu-
traumatic stress disorder in Vietnam veterans. lates parochial altruism in intergroup conflict
American Journal of Psychiatry, 150(2), 235–39. among humans. Science, 328(5984), 1408–11.
Buchheim, A., Heinrichs, M., George, C., Poko- De Dreu, C. K., Greer, L. L., Van Kleef, G. A.,
rny, D., Koops, E., Henningsen, P., et al. Shalvi, S., & Handgraaf, M. J. (2011). Oxytocin
(2009). Oxytocin enhances the experience of promotes human ethnocentrism. Proceedings
attachment security. Psychoneuroendocrinol- of the National Academy of Sciences, 108(4),
ogy, 34(9), 1417–22. 1262–66.
Buske-Kirschbaum, A., Jobst, S., Wustmans, A., de Kloet, E. R., Fitzsimons, C. P., Datson, N. A.,
Kirschbaum, C., Rauh, W., & Hellhammer, Meijer, O. C., & Vreugdenhil, E. (2009). Glu-
D. (1997). Attenuated free cortisol response cocorticoid signaling and stress-related limbic
to psychosocial stress in children with atopic susceptibility pathway: About receptors, tran-
dermatitis. Psychosomatic Medicine, 59(4), 419– scription machinery and microRNA. Brain
26. Research, 13(1293),129–41.
Carpenter, L. L., Tyrka, A. R., Ross, N. S., de Kloet, E. R., Joels, M., & Holsboer, F. (2005).
Khoury, L., Anderson, G. M., & Price, L. H. Stress and the brain: From adaptation to dis-
(2009). Effect of childhood emotional abuse ease. Nature Reviews Neuroscience, 6(6), 463–
and age on cortisol responsivity in adulthood. 75.
Biological Psychiatry, 66(1), 69–75. Derijk, R. H., Wüst, S., Meijer, O. C., Zennaro,
Carter, C. S., Pournajafi-Nazarloo, H., Kramer, M. C., Federenko, I. S., Hellhammer, D. H.,
K. M., Ziegler, T. E., White-Traut, R., Bello, et al. (2006). A common polymorphism in the
D., et al. (2007). Oxytocin: behavioral associ- mineralocorticoid receptor modulates stress
ations and potential as a salivary biomarker. responsiveness. Journal of Clinical Endocrinol-
Annals of the New York Academy of Sciences, ogy and Metabolism, 91(12), 5083–89.
1098, 312–22. Di Simplicio, M., Massey-Chase, R., Cowen, P.,
Chen, F. S., Kumsta, R., von Dawans, B., & Harmer, C. (2009). Oxytocin enhances pro-
Monakhov, M., Ebstein, R. P., & Heinrichs, cessing of positive versus negative emotional
M. (2011). Common oxytocin receptor gene information in healthy male volunteers. Jour-
(OXTR) polymorphism and social support nal of Psychopharmacology, 23(3), 241–48.
interact to reduce stress in humans. Proceed- Dickerson, S. S., & Kemeny, M. E. (2004).
ings of the National Academy of Sciences of the Acute stressors and cortisol responses: A the-
United States of America (PNAS), 108, 19937– oretical integration and synthesis of labora-
19942. tory research. Psychological Bulletin, 130(3),
Chen, F. S., Kumsta, R., & Heinrichs, M. (2011). 355–91.
Oxytocin and intergroup relations: Goodwill Ditzen, B., Schaer, M., Gabriel, B., Boden-
is not a fixed pie. Proceedings of the National mann, G., Ehlert, U., & Heinrichs, M. (2009).
Academy of Sciences, 108(13), E45; author reply Intranasal oxytocin increases positive com-
E46. munication and reduces cortisol levels during
Chrousos, G. P. (2009). Stress and disorders of the couple conflict. Biological Psychiatry, 65(9),
stress system. Nature Reviews: Endocrinology, 728–31.
5(7), 374–81. Domes, G., Heinrichs, M., Glascher, J., Buchel,
Declerck, C. H., Boone, C., & Kiyonari, T. (2010). C., Braus, D. F., & Herpertz, S. C. (2007).
Oxytocin and cooperation under conditions of Oxytocin attenuates amygdala responses to
uncertainty: The modulating role of incentives emotional faces regardless of valence. Biologi-
and social information. Hormones and Behav- cal Psychiatry, 62(10), 1187–90.
ior, 57(3), 368–74. Domes, G., Heinrichs, M., Michel, A., Berger,
Dedovic, K., Renwick, R., Mahani, N. K., Engert, C., & Herpertz, S. C. (2007). Oxytocin
V., Lupien, S. J., & Pruessner, J. C. (2005). The improves “mind-reading” in humans. Biologi-
Montreal Imaging Stress Task: Using func- cal Psychiatry, 61(6), 731–33.
SOCIAL STRESS AND SOCIAL APPROACH 529
Domes, G., Lischke, A., Berger, C., Grossmann, disorders. Biological Psychiatry, 67(7), 692–
A., Hauenstein, K., Heinrichs, M., et al. 94.
(2010). Effects of intranasal oxytocin on emo- Guastella, A. J., Howard, A. L., Dadds, M. R.,
tional face processing in women. Psychoneu- Mitchell, P., & Carson, D. S. (2009). A ran-
roendocrinology, 35(1), 83–93. domized controlled trial of intranasal oxy-
Domes, G., Steiner, A., Porges, S. W., tocin as an adjunct to exposure therapy for
& Heinrichs, M. (2012). Oxytocin dif- social anxiety disorder. Psychoneuroendocrinol-
ferentially modulates eye gaze to nat- ogy, 34(6), 917–23.
uralistic social signals of happiness and Guastella, A. J., Mitchell, P. B., & Dadds, M.
anger. Psychoneuroendocrinology, in press (doi: R. (2008). Oxytocin increases gaze to the eye
10.1016/j.psyneuen.2012.10.002). region of human faces. Biological Psychiatry,
Evans, S., Shergill, S. S., & Averbeck, B. B. (2010). 63(1), 3–5.
Oxytocin decreases aversion to angry faces in Hariri, A. R., Goldberg, T. E., Mattay, V. S.,
an associative learning task. Neuropsychophar- Kolachana, B. S., Callicott, J. H., Egan, M.
macology, 35(13), 2502–9. F., et al. (2003). Brain-derived neurotrophic
Fehm-Wolfsdorf, G., Born, J., Voigt, K. H., & factor val66met polymorphism affects human
Fehm, H. L. (1984). Human memory and neu- memory-related hippocampal activity and
rohypophyseal hormones: Opposite effects predicts memory performance. Journal of
of vasopressin and oxytocin. Psychoneuroen- Neuroscience, 23(17), 6690–94.
docrinology, 9(3), 285–92. Heim, C., & Nemeroff, C. B. (2001). The role
Fischer-Shofty, M., Shamay-Tsoory, S. G., of childhood trauma in the neurobiology of
Harari, H., & Levkovitz, Y. (2010). The effect mood and anxiety disorders: Preclinical and
of intranasal administration of oxytocin on clinical studies. Biological Psychiatry, 49(12),
fear recognition. Neuropsychologia, 48(1), 179– 1023–39.
84. Heim, C., Newport, D. J., Heit, S., Graham,
Gamer, M., & Buchel, C. (2012). Oxytocin specif- Y. P., Wilcox, M., Bonsall, R., et al. (2000).
ically enhances valence-dependent parasym- Pituitary-adrenal and autonomic responses to
pathetic responses. Psychoneuroendocrinology, stress in women after sexual and physical
37(1), 87–93. abuse in childhood. Journal of the American
Gamer, M., Zurowski, B., & Buchel, C. Medical Association, 284(5), 592–97.
(2010). Different amygdala subregions mediate Heim, C., Newport, D. J., Mletzko, T., Miller,
valence-related and attentional effects of oxy- A. H., & Nemeroff, C. B. (2008). The
tocin in humans. Proceedings of the National link between childhood trauma and depres-
Academy of Sciences, 107(20), 9400–5. sion: Insights from HPA axis studies in
Givalois, L., Naert, G., Rage, F., Ixart, G., humans. Psychoneuroendocrinology, 33(6), 693–
Arancibia, S., & Tapia-Arancibia, L. (2004). 710.
A single brain-derived neurotrophic fac- Heinrichs, M., Baumgartner, T., Kirschbaum, C.,
tor injection modifies hypothalamo-pituitary- & Ehlert, U. (2003). Social support and oxy-
adrenocortical axis activity in adult male rats. tocin interact to suppress cortisol and subjec-
Molecular and Cellular Neuroscience, 27(3), tive responses to psychosocial stress. Biological
280–95. Psychiatry, 54(12), 1389–98.
Gotlib, I., Joormann, J., Minor, K., & Hall- Heinrichs, M., & Domes, G. (2008). Neuropep-
mayer, J. (2008). HPA axis reactivity: A mech- tides and social behavior: Effects of oxytocin
anism underlying the associations among 5- and vasopressin in humans. Progress in Brain
HTTLPR, stress, and depression. Biological Research, 170, 337–50.
Psychiatry, 63(9), 847–51. Heinrichs, M., Meinlschmidt, G., Neumann,
Guastella, A. J., Carson, D. S., Dadds, M. R., I., Wagner, S., Kirschbaum, C., Ehlert,
Mitchell, P. B., & Cox, R. E. (2009). Does oxy- U., et al. (2001). Effects of suckling on
tocin influence the early detection of angry hypothalamic-pituitary-adrenal axis responses
and happy faces? Psychoneuroendocrinology, to psychosocial stress in postpartum lactating
34(2), 220–25. women. Journal of Clinical Endocrinology and
Guastella, A. J., Einfeld, S. L., Gray, K. M., Rine- Metabolism, 86(10), 4798–4804.
hart, N. J., Tonge, B. J., Lambert, T. J., et al. Heinrichs, M., Meinlschmidt, G., Wippich, W.,
(2010). Intranasal oxytocin improves emotion Ehlert, U., & Hellhammer, D. H. (2004).
recognition for youth with autism spectrum Selective amnesic effects of oxytocin on
530 MARKUS HEINRICHS, FRANCES S. CHEN, GREGOR DOMES, & ROBERT KUMSTA
human memory. Physiology and Behavior, atric disorders in the United States. Results
83(1), 31–38. from the National Comorbidity Survey.
Heinrichs, M., von Dawans, B., & Domes, G. Archives of General Psychiatry, 51(1), 8–
(2009). Oxytocin, vasopressin, and human 19.
social behavior. Frontiers in Neuroendocrinol- Kim, H. S., Sherman, D. K., Sasaki, J. Y., Xu,
ogy, 30(4), 548–57. J., Chu, T. Q., Ryu, C., et al. (2010). Cul-
Holmes, S. E., Slaughter, J. R., & Kashani, J. ture, distress, and oxytocin receptor polymor-
(2001). Risk factors in childhood that lead to phism (OXTR) interact to influence emo-
the development of conduct disorder and anti- tional support seeking. Proceedings of the
social personality disorder. Child Psychiatry National Academy of Sciences, 107(36), 15717–
and Human Development, 31(3), 183–93. 21.
Hurlemann, R., Patin, A., Onur, O. A., Cohen, Kirsch, P., Esslinger, C., Chen, Q., Mier, D., Lis,
M. X., Baumgartner, T., Metzler, S., et al. S., Siddhanti, S., et al. (2005). Oxytocin mod-
(2010). Oxytocin enhances amygdala- ulates neural circuitry for social cognition and
dependent, socially reinforced learning and fear in humans. Journal of Neuroscience, 25(49),
emotional empathy in humans. Journal of 11489–93.
Neuroscience, 30(14), 4999–5007. Kirschbaum, C., Pirke, K. M., & Hellhammer,
Inoue, H., Yamasue, H., Tochigi, M., Abe, O., D. H. (1993). The “Trier Social Stress Test” – a
Liu, X., Kawamura, Y., et al. (2010). Associa- tool for investigating psychobiological stress
tion between the oxytocin receptor gene and responses in a laboratory setting. Neuropsy-
amygdalar volume in healthy adults. Biological chobiology, 28(1–2), 76–81.
Psychiatry, 68(11), 1066–72. Kosfeld, M., Heinrichs, M., Zak, P. J., Fis-
Insel, T. R., & Young, L. J. (2001). The neurobi- chbacher, U., & Fehr, E. (2005). Oxytocin
ology of attachment. Nature Reviews Neuro- increases trust in humans. Nature, 435(7042),
science, 2(2), 129–36. 673–76.
Ising, M., Depping, A., Siebertz, A., Lucae, S., Kudielka, B. M., Hellhammer, D. H., & Wust,
Unschuld, P., Kloiber, S., et al. (2008). Poly- S. (2009). Why do we respond so differently?
morphisms in the FKBP5 gene region mod- Reviewing determinants of human salivary
ulate recovery from psychosocial stress in cortisol responses to challenge. Psychoneuroen-
healthy controls. European Journal of Neuro- docrinology, 34(1), 2–18.
science, 28(2), 389–98. Kudielka, B., Wüst, S., Kirschbaum, C., & Hell-
Jaenisch, R., & Bird, A. (2003). Epigenetic reg- hammer, D. H. (Eds.). (2007). Trier Social
ulation of gene expression: how the genome Stress Test, Vol. 3. Oxford: Academic Press.
integrates intrinsic and environmental signals. Kumsta, R., Entringer, S., Koper, J., Van-
Nature Genetics, 33(Suppl.), 245–54. rossum, E., Hellhammer, D., & Wust, S.
Karg, K., Burmeister, M., Shedden, K., & Sen, (2007). Sex specific associations between
S. (2011). The serotonin transporter promoter common glucocorticoid receptor gene vari-
variant (5-HTTLPR), stress, and depression ants and hypothalamus-pituitary-adrenal axis
meta-analysis revisited: Evidence of genetic responses to psychosocial stress. Biological Psy-
moderation. Archives of General Psychiatry, chiatry, 62(8), 863–69.
68(5), 444–54. Kumsta, R., Moser, D., Streit, F., Koper, J.
Kendler, K. S., Bulik, C. M., Silberg, J., Het- W., Meyer, J., & Wust, S. (2009). Char-
tema, J. M., Myers, J., & Prescott, C. A. (2000). acterization of a glucocorticoid receptor
Childhood sexual abuse and adult psychiatric gene (GR, NR3C1) promoter polymorphism
and substance use disorders in women: An reveals functionality and extends a haplo-
epidemiological and co-twin control analysis. type with putative clinical relevance. Ameri-
Archives of General Psychiatry,57(10), 953–59. can Journal of Medical Genetics B: Neuropsy-
Kendler, K. S., Kuhn, J. W., & Prescott, C. chiatric Genetic s, 150B(4), 476–82.
A. (2004). Childhood sexual abuse, stressful Kumsta, R., & Heinrichs, M. (2012). Oxy-
life events and risk for major depression in tocin, stress and social behavior: neuroge-
women. Psychological Medicine, 34(8), 1475–82. netics of the human oxytocin system. Cur-
Kessler, R. C., McGonagle, K. A., Zhao, rent Opinion in Neurobiology, in press (doi:
S., Nelson, C. B., Hughes, M., Eshle- 10.1016/j.conb.2012.09.004).
man, S., et al. (1994). Lifetime and 12- Labuschagne, I., Phan, K. L., Wood, A.,
month prevalence of DSM-III-R psychi- Angstadt, M., Chua, P., Heinrichs, M., et al.
SOCIAL STRESS AND SOCIAL APPROACH 531
(2010). Oxytocin attenuates amygdala reactiv- down? Chronic stress and the hypothalamic-
ity to fear in generalized social anxiety dis- pituitary-adrenocortical axis in humans. Psy-
order. Neuropsychopharmacology, 35(12), 2403– chological Bulletin, 133(1), 25–45.
13. Moffitt, T. E., Caspi, A., & Rutter, M.
Landgraf, R., & Neumann, I. D. (2004). Vaso- (2005). Strategy for investigating interactions
pressin and oxytocin release within the brain: between measured genes and measured envi-
A dynamic concept of multiple and variable ronments. Archives of General Psychiatry,
modes of neuropeptide communication. Fron- 62(5), 473–81.
tiers in Neuroendocrinology, 25(3–4), 150–76. Naber, F., van Ijzendoorn, M. H., Deschamps, P.,
Lederbogen, F., Kirsch, P., Haddad, L., Streit, van Engeland, H., & Bakermans-Kranenburg,
F., Tost, H., Schuch, P., et al. (2011). City M. J. (2010). Intranasal oxytocin increases
living and urban upbringing affect neural fathers’ observed responsiveness during play
social stress processing in humans. Nature, with their children: A double-blind within-
474(7352), 498–501. subject experiment. Psychoneuroendocrinol-
Lerer, E., Levi, S., Salomon, S., Darvasi, A., Yir- ogy, 35(10), 1583–86.
miya, N., & Ebstein, R. P. (2008). Association Norman, G. J., Cacioppo, J. T., Morris, J. S.,
between the oxytocin receptor (OXTR) gene Karelina, K., Malarkey, W. B., Devries, A. C.,
and autism: Relationship to Vineland Adap- et al. (2011). Selective influences of oxytocin
tive Behavior Scales and cognition. Molecular on the evaluative processing of social stimuli.
Psychiatry, 13(10), 980–88. Journal of Psychopharmacology, 25(10), 1313–19.
Lucht, M. J., Barnow, S., Sonnenfeld, C., Rosen- Pruessner, J. C., Dedovic, K., Khalili-Mahani,
berger, A., Grabe, H. J., Schroeder, W., et al. N., Engert, V., Pruessner, M., Buss, C., et al.
(2009). Associations between the oxytocin (2008). Deactivation of the limbic system dur-
receptor gene (OXTR) and affect, loneliness ing acute psychosocial stress: Evidence from
and intelligence in normal subjects. Progress in positron emission tomography and functional
Neuropsychopharmacology and Biological Psy- magnetic resonance imaging studies. Biological
chiatry, 33(5), 860–66. Psychiatry, 63(2), 234–40.
Ludwig, M., & Leng, G. (2006). Dendritic pep- Quirin, M., Kuhl, J., & Dusing, R. (2011). Oxy-
tide release and peptide-dependent behaviors. tocin buffers cortisol responses to stress in
Nature Reviews Neuroscience, 7(2), 126–36. individuals with impaired emotion regulation
Marsh, A. A., Yu, H. H., Pine, D. S., & Blair, R. abilities. Psychoneuroendocrinology, 36(6), 898–
J. (2010). Oxytocin improves specific recogni- 904.
tion of positive facial expressions. Psychophar- Repetti, R. L., Taylor, S. E., & Seeman, T. E.
macology (Berl), 209(3), 225–32. (2002). Risky families: Family social environ-
Mason, J. W. (1968). A review of psychoen- ments and the mental and physical health of
docrine research on the pituitary-adrenal cor- offspring. Psychological Bulletin, 128(2), 330–66.
tical system. Psychosomatic Medicine, 30(5), Rimmele, U., Hediger, K., Heinrichs, M., &
576–607. Klaver, P. (2009). Oxytocin makes a face
McGowan, P. O., Sasaki, A., D’Alessio, A. C., in memory familiar. Journal of Neuroscience,
Dymov, S., Labonte, B., Szyf, M., et al. (2009). 29(1), 38–42.
Epigenetic regulation of the glucocorticoid Rodrigues, S. M., Saslow, L. R., Garcia, N., John,
receptor in human brain associates with child- O. P., & Keltner, D. (2009). Oxytocin recep-
hood abuse. Nature Neuroscience, 12(3), 342– tor genetic variation relates to empathy and
48. stress reactivity in humans. Proceedings of the
Meinlschmidt, G., & Heim, C. (2007). Sensitiv- National Academy of Sciences, 106(50), 21437–
ity to intranasal oxytocin in adult men with 41.
early parental separation. Biological Psychiatry, Rutter, M. (2006). Implications of resilience
61(9), 1109–11. concepts for scientific understanding. Annals
Meyer-Lindenberg, A., Domes, G., Kirsch, P., of the New York Academy of Sciences, 1094,
& Heinrichs, M. (2011). Oxytocin and vaso- 1–12.
pressin in the human brain: Social neu- Sapolsky, R. M., Romero, L. M., & Munck A.
ropeptides for translational medicine. Nature U. (2000). How do glucocorticoids influence
Reviews Neuroscience, 12(9), 524–38. stress responses? Integrating permissive, sup-
Miller, G. E., Chen, E., & Zhou, E. S. pressive, stimulatory, and preparative actions.
(2007). If it goes up, must it come Endocrine Review, 21(1), 55–89.
532 MARKUS HEINRICHS, FRANCES S. CHEN, GREGOR DOMES, & ROBERT KUMSTA
Scaccianoce, S., Del Bianco, P., Caricasole, A., Viviani, D., Charlet, A., van den Burg, E., Robi-
Nicoletti, F., & Catalani, A. (2003). Relation- net, C., Hurni, N., Abatis, M., et al. (2011).
ship between learning, stress and hippocam- Oxytocin selectively gates fear responses
pal brain-derived neurotrophic factor. Neuro- through distinct outputs from the central
science, 121(4), 825–28. amygdala. Science, 333(6038), 104–7.
Schulze, L., Lischke, A., Greif, J., Herpertz, S. von Dawans, B., Kirschbaum, C., & Heinrichs,
C., Heinrichs, M., & Domes, G. (2011). Oxy- M. (2011). The Trier Social Stress Test for
tocin increases recognition of masked emo- Groups (TSST-G): A new research tool for
tional faces. Psychoneuroendocrinology, 36(9), controlled simultaneous social stress exposure
1378–82. in a group format. Psychoneuroendocrinology,
Shalev, I., Lerer, E., Israel, S., Uzefovsky, F., 36(4), 514–22.
Gritsenko, I., Mankuta, D., et al. (2009). Weaver, I. C., Cervoni, N., Champagne, F. A.,
BDNF Val66Met polymorphism is associated D’Alessio, A. C., Sharma, S., Seckl, J. R., et al.
with HPA axis reactivity to psychological (2004). Epigenetic programming by maternal
stress characterized by genotype and gender behavior. Nature Neuroscience, 7(8), 847–54.
interactions. Psychoneuroendocrinology, 34(3), Wilhelm, I., Born, J., Kudielka, B. M., Schlotz,
382–88. W., & Wust, S. (2007). Is the cortisol awaken-
Theodoridou, A., Rowe, A. C., Penton-Voak, ing rise a response to awakening? Psychoneu-
I. S., & Rogers, P. J. (2009). Oxytocin roendocrinology, 32(4), 358–66.
and social perception: Oxytocin increases Wüst, S., Federenko, I. S., van Rossum, E.
perceived facial trustworthiness and attrac- F., Koper, J. W., Kumsta, R., Entringer, S.,
tiveness. Hormones and Behavior, 56(1), et al. (2004). A psychobiological perspective
128–32. on genetic determinants of hypothalamus-
Uhart, M., McCaul, M. E., Oswald, L. M., pituitary-adrenal axis activity. Annals of the
Choi, L., & Wand, G. S. (2004). GABRA6 New York Academy of Sciences, 1032, 52–62.
gene polymorphism and an attenuated stress Yehuda, R. (2002). Post-traumatic stress disorder.
response. Molecular Psychiatry, 9(11), 998– New England Journal of Medicine, 346(2), 108–
1006. 14.
Ulrich-Lai, Y. M., & Herman, J. P. (2009). Neural Young, E. A., Abelson, J. L., Curtis, G. C., &
regulation of endocrine and autonomic stress Nesse, R. M. (1997). Childhood adversity and
responses. Nature Reviews Neuroscience, 10(6), vulnerability to mood and anxiety disorders.
397–409. Depression and Anxiety, 5(2), 66–72.
CHAPTER 23
Empathy, which can be broadly defined empathy may also help us understand clin-
as the capacity to share and understand ical phenomena related to a lack of affec-
other people’s emotions (for comprehen- tive and social skills such as autism, which is
sive reviews, see Batson, 2009a; Decety characterized by impairments in social inter-
& Jackson, 2006; de Vignemont & Singer, action and communication (American Psy-
2006; Eisenberg, 2000; Hoffman, 2000; Singer chiatric Association, 2000), or alexithymia, a
& Lamm, 2009; Singer & Leiberg, 2009), subclinical phenomenon associated with dif-
has recently become an important focus of ficulties in identifying and describing emo-
attention in the field of social neuroscience. tions (Nemiah, Freyberger, & Sifneos, 1976).
What motivates the quest for the neural The importance of empathy in our every-
substrates underlying our understanding of day lives becomes clear when we try to imag-
emotions in others? ine what it would be like to live in a world
After many years in which neurosci- completely devoid of empathy. Take the fol-
entific research mainly focused on cogni- lowing scenario, for instance: A baby starts
tive and sensory processing, attention has crying while her mother is reading a book.
increasingly turned to understanding how Without empathy, the mother would prob-
the human brain tackles emotions and social ably continue reading and not look after the
interactions, which after all are both phe- baby. Her capacity to empathize, however,
nomena at the core of our existence as enables her to realize the baby’s needs and
social beings. Thus, the field of social neuro- react to them appropriately. As this exam-
science has started to investigate the neural ple illustrates, empathy does not only moti-
mechanisms underlying social cognition and vate other-related prosocial behavior but
emotions, such as our ability to empathize. also enables us to better predict the behav-
In addition to the basic understanding of ior of others and adapt our reactions accord-
the biological mechanisms underlying social ingly. Finally, empathy also plays a crucial
emotions and empathy in healthy individ- role in observational learning – by witnessing
uals, research on the neural substrates of the emotional reaction of others in different
533
534 OLGA KLIMECKI & TANIA SINGER
circumstances, we learn which situations are other words, we empathize with another
good for us and which situations are better human being when we vicariously share
avoided. their affective state, but at the same time
In this chapter, we start out by revisiting are aware that the other person’s emotion is
the definition of empathy and delineating it causing our response.
from other routes to social understanding; In this section, we first point out the con-
namely, theory of mind and action under- ceptual difference among empathy, mentaliz-
standing. We then examine the theoretical ing, and action understanding, which can be
and neural underpinnings of concepts such conceived of as different routes to the under-
as emotion contagion and mimicry, which standing of others. After showing that the
can be thought of as antecedents of empa- psychological distinction among these three
thy, and compassion and empathic distress, concepts is paralleled by differences in the
which are introduced as consequences of underlying neural networks, we take a closer
empathy. Before turning our focus to how look at the “sisters of empathy” – emotion
research in social neuroscience has advanced contagion, mimicry, sympathy and compas-
our understanding of empathy in the human sion – all of which are concepts closely
brain, we review the major contributions of related to empathy.
psychological research to our understand-
ing of empathy and its relation to proso-
Mentalizing and Action
cial behavior. Because the neural underpin-
Understanding as Alternative Routes
nings of empathy have been examined most
for Understanding Others
prominently in the domain of empathy for
pain, we begin by summarizing this line of In addition to empathy, which can be seen
research and discussing the reported results as the emotional route for understanding
in light of the shared network hypothe- others, there are at least two other ways of
sis. In this context, we stress the specific putting oneself into another person’s shoes.
role of the insula as a neural structure that On the one hand, we have the cognitive abil-
processes both, interoception and empa- ity to understand the thoughts, beliefs, and
thy. Subsequently, we describe factors that intentions of others, which is called men-
modulate the experience of empathy for talizing, perspective-taking, or theory of mind
pain along with their neural underpinnings, (ToM; Frith and Frith, 2003; Premack and
before turning to neural correlates of empa- Woodruff, 1978). On the other hand, we
thy in other domains such as touch or smell. have the capacity to understand the motor
Finally, we show initial findings from social intentions of others, which has been asso-
neuroscience research focusing on more pos- ciated with the discovery of mirror neu-
itive aspects of empathy, such as compas- rons (see Rizzolatti & Sinigaglia, 2010, for
sion. We conclude the chapter by outlining a review). Although all three often occur
outstanding questions in the field. simultaneously in everyday social cogni-
tion, the psychological and neural processes
underlying these distinct routes to under-
Defining Empathy and Related standing others can be clearly distinguished
Concepts (for reviews, see de Vignemont & Singer,
2006; Preston & de Waal, 2002; Singer &
Empathy is commonly defined as the human Lamm, 2009). Cognitive processes related to
capacity to understand and share another theory of mind have been associated with
person’s emotion without confusing it with activations in the medial prefrontal cortex
one’s own emotional state (for comprehen- (mPFC), superior temporal sulcus (STS),
sive reviews, see Batson, 2009a; Decety & and the adjacent temporoparietal junction
Jackson, 2006; de Vignemont & Singer, 2006; (TPJ; for reviews, see Amodio & Frith, 2006;
Eisenberg, 2000; Hoffman, 2000; Singer & Frith & Frith, 2006; Mitchell, 2009; Saxe,
Lamm, 2009; Singer & Leiberg, 2009). In 2006; Saxe & Baron-Cohen, 2006), whereas
EMPATHY FROM THE PERSPECTIVE OF SOCIAL NEUROSCIENCE 535
consequences of empathy
Self-related emotion
Empathic or Personal
Negative feelings
Distress
Withdrawal
Figure 23.1. Schematic model showing the precursors and consequences of empathy.
and that are not shared with someone, but psychology established the link between
felt for someone (Batson, 2009a; Singer & empathy and prosocial behavior. Theodor
Lamm, 2009). Empathy or “feeling with” Lipps (1903) first introduced the concept of
denotes a state in which the feelings of some- empathy by proposing that we imitate the
one else are vicariously shared so that the gestures and actions of others in order to
empathizer feels an isomorphic state (de understand their inner states. To measure
Vignemont & Singer, 2006): On witness- empathy, Davis (1980) developed a ques-
ing a sad person, the empathizer becomes tionnaire called the Interpersonal Reactiv-
sad him- or herself. In contrast, “feeling for” ity Index (IRI) that includes four distinct
someone refers to an emotional state that components: perspective-taking, empathic
is not necessarily isomorphic to the target’s concern, personal distress, and fantasy.
affective state, but instead relies on feel- Perspective-taking is very close to the ear-
ings of concern for the other. More pre- lier mentioned notion of theory of mind,
cisely, compassion can be defined as “the because it measures the tendency of peo-
emotion one experiences when feeling con- ple to cognitively adopt the perspective
cern for another’s suffering and desiring to of others. Empathic concern, however, is
enhance that individual’s welfare” (Keltner related more closely to the concept of sym-
& Goetz, 2007). In other words, compas- pathy or compassion as discussed earlier,
sion consists of two main components. First, whereas the personal distress subscale mea-
there is a caring feeling for the suffering sures how prone individuals are to experi-
of others that secondly motivates behav- encing discomfort as a result of witnessing
ior aimed at relieving the other’s suffer- distress in others. Finally, the fantasy scale
ing. As described in detail later, research asks people how well they tend to iden-
in social and developmental psychology tify with fictional characters in books or
has demonstrated that empathic concern movies. The Balanced Emotional Empathy
actually motivates prosociality and help- Scale (BEES; Mehrabian, 1997; Mehrabian
ing behaviors (see Batson, 2009b, or Eisen- & Epstein, 1972), which includes subscales
berg, 2000, for a review). In sum, whereas such as “susceptibility to emotional conta-
empathy is a vicarious emotion isomorphic gion” or “sympathetic tendency,” is another
to the emotional experience of the other, questionnaire that measures the emotional
the experience of compassion, sympathy, or aspects of empathy.
empathic concern denotes an affective state These self-report measures are comple-
experienced with regard to the other that mented by the assessment of empathic accu-
does not encompass the sharing of nega- racy. Levenson and Ruef (1992), for instance,
tive affect, but instead relies on a feeling of presented subjects with videotaped mari-
care and concern that motivates prosocial tal interactions. By comparing the subjects’
behavior. Before turning to empirical evi- ratings of positive and negative emotions
dence for the promotion of prosocial behav- displayed in the video with the self-report
ior by compassion, we provide an overview of the actual target, an empathy accuracy
of measures developed by psychologists to score is obtained that assesses the degree to
assess empathy and its different compo- which subjects correctly identify the emo-
nents. tional state of others. Additionally, physi-
ological measures such as skin conductance
and heart rate can be compared with respect
Empathy Research in Psychology to the level of physiological linkage between
the target and the subject empathizing with
Psychologists from various fields have inves- the target (see also Gottman & Levenson,
tigated empathy and its relation to prosocial 1985). Ickes and colleagues (Ickes, 1993, for a
behavior. In this section, we first describe review) used a similar approach to deter-
self-report, behavioral, and physiological mine to what degree subjects succeed in
measures of empathy before summarizing inferring another person’s thoughts and the
how research in developmental and social content of their emotions.
538 OLGA KLIMECKI & TANIA SINGER
another person experiencing pain. During showed that higher self-reports of empa-
the brain scan, arrows in different colors thy as measured by the BEES and the
that can be seen by the participant through IRI were accompanied by increased neu-
the help of a mirror system indicate who ral activity during empathy for pain in the
is going to be stimulated next and whether left AI and ACC, thereby supporting the
the stimulation is going to be painful or not. external validity of the empathy-for-pain
Under the assumption that people in a very paradigm. Further evidence for the link
close relation feel strong empathy for each between the observed brain responses and
other, Singer and colleagues (2004) started empathic experience stems from studies in
out by examining empathy for pain in cou- which neural activation patterns in spe-
ples. Intriguingly, the results of this study cific empathy-related regions were shown
show that the neural signature of empa- to correlate with self-reported impressions
thy for pain is very similar to the neu- and actual helping behavior (Hein et al.,
ral processes underlying the self-experience 2010), as well as with individual unpleas-
of pain (for brain circuits mediating pain antness ratings (e.g., Jabbi, Swart, & Key-
perception, the so-called pain matrix, see sers, 2007; Lamm, Nusbaum, et al., 2007;
Chapter 9). More specifically, empathy for Saarela et al., 2007; Singer et al., 2008).
pain activated selective parts of the neural Importantly, the existence of a shared neu-
pain matrix including the AI and the ante- ral network for self-experienced pain and
rior cingulate cortex (ACC), which both empathy for pain is supported by a recent
are key regions in processing bodily and meta-analysis of Lamm, Decety, and Singer
feeling states (see also Singer, Critchley, (2011), which showed consistent overlaps in
& Preuschoff, 2009, and Chapter 3). These AI and ACC across nine independent fMRI
findings have been replicated by several studies and is discussed in more detail later.
studies using similar paradigms (Bird et al., Although the studies described here point
2010; Hein, Silani, Preuschoff, Batson, & to a shared neural representation of self-
Singer, 2010; Singer et al., 2006, 2008). Con- experienced and vicariously experienced
verging evidence for the involvement of AI pain, the question still remains whether
and, less consistently, the ACC in empathy activations in the AI and the ACC over-
for pain also has been obtained from sev- lap on the level of neuronal subpopulations
eral other studies (some of which are dis- and single neurons (e.g., Singer & Lamm,
cussed in more detail later) using various 2009).
paradigms ranging from simultaneous pain
administration in the scanner to the presen-
Shared and Distinct Neural Networks
tation of photographs or videos depicting
in Empathy for Pain
painful events (e.g., Botvinick et al., 2005;
Cheng, Chen, Liu, Chou, & Decety, 2010; In addition to confirming an overlap be-
Chen et al., 2007; Danziger, Faillenot, & Pey- tween first-hand and vicarious pain experi-
ron, 2009; Decety, Echols, & Correll, 2010; ence in the AI and ACC, the meta-analysis
Gu & Han, 2007; Jackson, Brunet, Meltzoff, of Lamm and colleagues (2011) suggests that
& Decety, 2006; Lamm, Batson, & Decety, this shared network can be accessed via
2007; Lamm & Decety, 2008; Lamm, Nus- several routes depending on the paradigm
baum, Meltzoff, & Decety, 2007; Moriguchi employed. Whereas the use of picture-based
et al., 2007; Morrison, Lloyd, di Pellegrino, empathy paradigms is linked to additional
& Roberts, 2004; Ogino et al., 2007; Saarela activation increases in the inferior parietal,
et al., 2007; Zaki, Ochsner, Hanelin, Wager, ventral premotor, and dorsomedial cortex
& Mackey, 2007). (a neural circuitry typically observed in
In support of the claim that the observed action understanding), cue-based paradigms
neural activation patterns are closely related induce activation in networks typically
to the concept of empathy developed by linked to theory of mind, such as the mPFC,
psychologists, Singer and colleagues (2004) precuneus, STS, and TPJ.
540 OLGA KLIMECKI & TANIA SINGER
Distinct connectivity patterns of the AI nance with empathy for visually depicted
and ACC with other brain regions dur- pain has been shown by studies using
ing self-experienced pain versus empathy TMS (Avenanti, Bueti, Galati, & Agli-
for pain have been examined by Zaki and oti, 2005; Avenanti, Pauuello, Bufalari, &
colleagues (2007), who reported that self- Aglioti, 2006) and EEG (Bufalari, Aprile,
experienced pain is associated with stronger Avenanti, Di Russo, & Aglioti, 2007; Vale-
connectivity between the AI and regions riani et al., 2008). Taken together, these
involved in the transmission of painful sen- results point to a core shared network for
sations, such as clusters in the midbrain, self-experienced pain and empathy for pain
periaqueductal gray, and mid-insula. Con- in the ACC and AI that varies in its con-
versely, empathy for pain has been shown nectivity with other brain regions. Thus,
to be associated with higher connectivity of the AI and ACC are coactivated with areas
the ACC and AI with brain regions impli- involved in processing self-related compo-
cated in social cognition and affect process- nents of nociceptive experience when expe-
ing, such as the medial prefrontal cortex. riencing pain in oneself. However, when
Converging evidence for the activation empathizing with others, the AI and ACC
of sensory brain structures during self-pain are co-activated with networks involved in
stems from the earlier mentioned meta- social cognition (ToM and action observa-
analysis (Lamm et al., 2011), where the direct tion). This finding suggests that the informa-
experience of pain recruited mid- and pos- tion available in the task and the situational
terior insula, as well as primary sensory cor- demands determine which of the social
tices in addition to the AI and ACC (see cognition networks will be predominantly
Figure 23.2 for illustration). The stronger engaged.
involvement of brain regions processing sen- With regard to the quality of the em-
sory information in self-experienced pain pathic experience, the reported studies sug-
suggests that we share the pain of others by gest that we primarily share painful expe-
accessing the neural structures representing riences by simulating the affective and not
our own affective states, while leaving aside so much the sensory and nociceptive com-
sensory and nociceptive components. ponents of pain. Note, however, that there
Moreover, the meta-analysis on empa- seems to be a graded activation of the neural
thy for pain showed that contralateral pri- pain matrix, such that activations in the pos-
mary somatosensory cortex (SI) activation, terior insula and secondary somatosensory
which most likely encodes somatosensory cortex can be observed, even without the
aspects of painful sensations, is restricted administration of pain, if subjects adopt a
to self-experienced pain in cue-based stud- first-person perspective (Jackson et al., 2006)
ies. Picture-based studies, on the contrary, or simply imagine a painful event (Ogino
evoke bilateral SI activations during both et al., 2007). Because these activation pat-
empathy for painful and nonpainful situ- terns do not include the primary somatosen-
ations, thus pointing to an unspecific role sory cortex, there seems to be a continuum
of the SI in empathy for pain that is between directly and vicariously experi-
presumably related to seeing body parts enced pain that is reflected in the underlying
being touched. Interestingly, activity in neural substrates.
somatosensory cortices can also be increased
when subjects are instructed to evaluate
The Role of the Insula in Empathy and
the sensory consequences of painful stim-
Interoception
uli, suggesting that attention can influence
the quality of the empathic experience The posterior-anterior gradient of the insula
and that this shift in focus is accom- in self-experienced pain and empathy for
panied by corresponding brain activations pain is mirrored by research showing that
(Lamm, Nusbaum, et al., 2007). Comple- primary nociceptive information is first pro-
menting these findings, sensorimotor reso- cessed in dorsal posterior parts of the insula
EMPATHY FROM THE PERSPECTIVE OF SOCIAL NEUROSCIENCE 541
before being remapped and integrated with tation implies that a deficit in understand-
other information in the AI where emotions ing our own feelings should entail difficulties
access consciousness (Craig, 2002, 2009). in empathizing with the feelings of others.
These results fit into the general notion Indeed, this claim was confirmed in studies
of the insula as a key player in process- focusing on people with alexithymia, a sub-
ing interoceptive information from the body clinical phenomenon characterized by dif-
(Craig, 2002; Damasio, 1994; Ostrowsky et ficulties in identifying and describing emo-
al., 2000, 2002; see Chapter 3) and emotions tions (Nemiah et al., 1976). Silani and
more generally (shown by a meta-analysis colleagues (2008) demonstrated that activ-
by Kober et al., 2008). Given these find- ity in the AI during interoception on emo-
ings, Singer et al. (2004, 2009) proposed that tional stimuli diminished with increasing
the insula fulfills a dual role: (1) processing alexithymia scores, whereas activation in
bodily information, such as heart beat or the AI during interoception was positively
temperature-related sensations, which are related to trait empathy. Furthermore, this
then integrated into global feeling states, study showed that a higher degree of alex-
and (2) predicting the affective states of oth- ithymia was accompanied by lower levels of
ers in the process of empathizing. In other trait empathy. Extending these results, Bird
words, we use our own bodily and affec- and colleagues (2010), using the Singer et al.
tive representation to understand the emo- (2004) empathy-for-pain paradigm, found
tional experiences of others. This interpre- that AI activations also decrease when highly
542 OLGA KLIMECKI & TANIA SINGER
alexithymic participants are asked to em- in the men’s neural empathy response was
pathize with others in pain. accompanied by increased activation in the
nucleus accumbens – a region known to
be crucially involved in reward processing
Modulation of Empathy (for recent reviews, see Knutson & Cooper,
2005; Schultz, 2000; Chapter 19). Moreover,
In this section, we discuss findings indicat- the extent of nucleus accumbens activa-
ing that empathic brain responses in the AI tion was positively correlated to the subjec-
are not only modulated by person-specific tively expressed desire of revenge. In other
characteristics, such as alexithymia, but can words, activity in this reward-related brain
also be affected by the relation to the tar- structure was higher when men reported a
get (e.g., liking or disliking), contextual stronger desire for revenge toward the unfair
and attentional factors, and the appraisal player. This activation pattern may imply
of the situation (for recent reviews, see de that men actually experienced “Schaden-
Vignemont & Singer, 2006; Singer & Lamm, freude” when witnessing the unfair player
2009). Several studies have shown that stim- being punished.
uli that under normal circumstances lead to In another study, Hein and colleagues
empathic responses fail to induce empathy (2010) extended these findings by show-
in certain situations and may even evoke the ing a relationship between empathy-related
opposite; namely what has been referred to brain responses in the AI and subsequent
as “Schadenfreude” – the joy of witnessing prosocial behavior. The authors examined
another person’s misfortune. ingroup-outgroup biases in male soccer
fans while they witnessed a fan of their
favorite team (ingroup) or a fan of a rival
Relationship between Empathizer and
team (outgroup) receiving painful electric
Target
shocks. As expected, the observation of
Singer and colleagues (2006) observed such ingroup members receiving pain was linked
a reversal of emotional response to the to greater AI activations. More importantly,
pain of others when they studied how per- the intensity of AI activation actually pre-
ceived fairness affects empathy for pain. In dicted the degree to which subjects would
their experiment, participants first played later help their ingroup member by tak-
an economic game with two other volun- ing the painful shocks themselves. In con-
teers, one of whom played fairly while the trast, nucleus accumbens (NAcc) activation
other played unfairly. When the partici- elicited by witnessing an outgroup mem-
pant was subsequently scanned, the fair and ber suffering predicted a refusal to help and
unfair players (who were actually confed- reflected how negatively the subject evalu-
erates) were sitting next to the scanner, ated the outgroup member. These findings
and all three alternately received painful imply that empathy-related insula activa-
or nonpainful stimulation on the back of tion drives altruistic behavior, whereas an
their hands. In line with previous findings, antagonistic signal in the NAcc reduces the
Singer and colleagues observed that both propensity to help. Other examples of fac-
the first-hand experience of pain and empa- tors that influence the nature of social rela-
thy for pain experienced for the fair per- tionships, and thereby the degree of empa-
son rely on shared neural representations thy and its neural correlates, are ethnicity
in the AI and the ACC. However, when (Xu, Zuo, Wang, & Han, 2009) and close-
empathizing with the pain of the unfair ness to the other person (Cheng et al., 2010).
player, only female subjects showed greater
activations in these regions. Male subjects,
Characteristics of the Empathizer
by contrast, showed a decline in AI activa-
tion while witnessing the unfair, as opposed In a similar line as the earlier described
to the fair, player in pain. This reduction alexithymia research, which established the
EMPATHY FROM THE PERSPECTIVE OF SOCIAL NEUROSCIENCE 543
link between high alexithymia (Silani et al., versus imagining the feelings of the patient
2008) and low empathy, as well as the during the medical treatment. To manipu-
decrease of AI activations during empa- late cognitive appraisal, subjects were told
thy for pain in alexithymic participants either that the medical treatment had been
(Bird et al., 2010), the characteristics of beneficial or unsuccessful. When subjects
the empathizer have also been shown to were told that the treatment had not been
influence empathic experiences in other successful, they provided higher ratings of
domains. Cheng and colleagues (2007), for pain and unpleasantness than in the condi-
instance, showed that, when observing nee- tion in which the treatment had been benefi-
dles being inserted into different body parts, cial. Brain data corroborate these findings by
participants without experience in acupunc- showing stronger activations in the perigen-
ture showed activations in the neural net- ual ACC for the ineffective treatment con-
work involved in empathy for pain, whereas dition. With regard to the different perspec-
physicians who practice acupuncture them- tives adopted, results show that participants
selves did not show such a neural response. reported higher personal distress when they
imagined themselves in the patient’s situa-
tion (self-perspective), whereas participants
Situational Context, Attention, and
reported more empathic concern when
Appraisal
they cognitively differentiated between the
The role of contextual appraisal in relation patient and themselves (other-perspective).
to empathy has been examined in relation The observed increase in empathic dis-
to the attribution of responsibility. Decety tress when adopting a self-perspective fos-
and colleagues (2010), for example, tested ters the claim that empathic distress arises
the degree to which empathic responses to due to an over-identification with the suf-
videos of pain in AIDS patients differed fering of others. On the neural level, adopt-
as a function of the target’s responsibility ing a self-perspective was associated with
(infection through transfusion or drug tak- increased activations of the neural pain
ing). The self-report measures and the neu- matrix. This finding speaks for a stronger
ral activations of the subjects both conveyed sharing of pain in distress and under-
that attributed responsibility influences the scores the important distinction between
extent of empathic response. Participants empathic distress and empathic concern as
reported higher pain and empathy ratings two opposing outcomes of empathy asso-
toward the pain of transfusion targets as ciated with different qualities of emotional
compared to drug targets, and these reports experience.
were accompanied by greater activation of
the neural networks involved in processing
pain (AI and ACC). Empathy for Touch, Smell, and Taste
Another factor that has been shown to
influence empathy is attention. Gu and As reviewed earlier, a majority of neuro-
Han (2007) reported lower ACC activity science studies on empathy have focused on
when participants were asked to count neu- pain. However, an early study focused on
tral stimuli in images displaying painful the examination of shared representations
events compared to rating the intensity for smell and disgust (Wicker et al., 2003).
of the pain. Finally, Lamm, Nusbaum, In their experiment, Wicker and colleagues
et al. (2007) investigated the effects of studied how processing the actual experi-
cognitive appraisal and perspective-taking ence of disgusting olfactory stimuli differs
on empathic responses by presenting video from processing others’ visual display of dis-
clips of painful facial expressions during a gust. The results of this fMRI study show
medical treatment. Perspective-taking was that both self- and other-related disgust are
varied by instructing the participants to accompanied by overlapping activation in
imagine themselves in the depicted situation the AI and ACC. Further support for this
544 OLGA KLIMECKI & TANIA SINGER
finding comes from a study by Jabbi and these brain areas in compassion is also sup-
colleagues (2008), who showed AI activa- ported by cross-sectional studies on compas-
tion regardless of whether subjects tasted sion and love. Kim and colleagues (2009),
an unpleasant substance, viewed disgusted for instance, showed that adopting a com-
facial expressions, or read disgusting scenar- passionate attitude towards pictures of sad
ios. Finally, Keysers and colleagues (2004) faces augmented activations in ventral stria-
reported that, whereas the sensory experi- tum and VTA/SN. Moreover, romantic as
ence of being touched is specifically linked well as maternal love (Bartels and Zeki,
to activation in the contralateral primary 2000; 2004) have been associated to activa-
somatosensory cortex, the neural signatures tions in the middle insula, the dorsal part
of being touched and observing touch over- of the ACC, and the striatum (comprised
lap in the secondary somatosensory cortex. of the putamen, globus pallidus, and cau-
In summary, these studies parallel findings date nucleus). Similar results were reported
on empathy in the domain of pain by provid- by Beauregard and colleagues (2009) who
ing evidence for the involvement of a shared observed increased activations of the middle
neural network underlying empathy in other insula, the dorsal ACC, the globus pallidus,
modalities such as touch, smell, and taste. and the caudate nucleus when their partici-
pants adopted a stance of unconditional love
toward pictures of individuals with intel-
The Compassionate Brain lectual disabilities. Finally, a cross-sectional
study revealed that expert meditators, but
Although social neuroscience has so far not novice meditators had augmented acti-
mostly focused on finding evidence for vations in middle insula when listening
shared networks and their modulation, the to distressing sounds (Lutz, Brefczynski-
field has recently moved forward to the Lewis, Johnstone, & Davidson, 2008).Given
investigation of positive consequences of that the observed regions are linked to
empathy such as empathic concern, sympa- reward processing and show a high den-
thy, or compassion. In a recent intervention sity of oxytocin and vasopressin receptors –
study, Klimecki and colleagues (Klimecki, neuropeptides that play a crucial role in
Leiberg, Lamm, & Singer, 2012) investi- attachment and bonding (for a review,
gated how training compassion over sev- see Depue & Morrone-Strupinsky, 2005;
eral days changes neural function and sub- Zeki, 2007), the described results might
jective emotional experience. To this end, be interpreted as reflecting the rewarding
the researchers specifically developed a new nature of experiencing love and warmth,
paradigm, the Socio-affective Video Task even when faced with the suffering of
(SoVT) in which participants witness the others.
distress of others. At pre-training, this stim- Taken together, these results suggest
ulus material elicited strong empathy and that the previously introduced distinction
strong negative emotions. Consistent with between empathic distress and compassion
findings from a recent empathy for pain as the two consequences of empathy is par-
meta-analysis (Lamm et al., 2011), empa- alleled by the involvement of different neu-
thy ratings in response to other’s distress ral substrates. Whereas distressing empathic
were accompanied by activations in ante- experiences have been shown to be associ-
rior insula and ACC. Training compassion ated with the AI and ACC, compassionate
over several days changed this response or loving experiences seem to involve the
pattern: participants reported feeling more medial orbitofrontal cortex, as well as mid-
positive emotions towards the suffering of insular and striatal regions. Because of the
others and showed increased activations in scarcity of research in this field, many more
medial orbitofrontal cortex, ventral tegmen- studies are needed to refine the delineation
tal area/ substantia nigra (VTA/SN), puta- of the neural networks involved in the pos-
men and pallidum. The involvement of itive emotions of compassion and love and
EMPATHY FROM THE PERSPECTIVE OF SOCIAL NEUROSCIENCE 545
to compare those in sharing negative feelings r What role do neurotransmitters like oxy-
such as pain or unpleasant tastes and odors. tocin and vasopressin play in empathy,
and how do they interact in the circuitry
of emotion processing?
Conclusion r Given that the acquisition of empathic
and compassionate skills probably depe-
The field of social neuroscience is evolv- nds on the maturation of certain cortical
ing quickly and the studies described in structures, which neural changes accom-
this chapter have (among others) greatly pany the development of empathy and
advanced our understanding of the neural related concepts from early childhood to
bases of empathy. Most importantly, the adolescence? How do these findings tie
development and use of ecologically valid in with emotional plasticity over the life-
paradigms could show that self-experienced span?
and vicariously experienced pain rely on
shared neural substrates in the AI and ACC.
Together with the finding of distinct acti- Acknowledgments
vation patterns in self-experienced affective
states, this suggests that we understand the O. K. received funding from the Univer-
feelings of others by simulating the affec- sity of Zurich (Forschungskredit). T. S.
tive component of the observed states. In received grants from the Neuroscience Cen-
this context, the insula plays a specific role, ter Zurich, the Betty & David Koetser Foun-
because it generally serves as an interocep- dation for Brain Research, and the European
tive cortex supporting representations and Research Council (ERC, Grant agreement
predictions of feeling states for oneself and no. 205557).
for others. Despite the major advances in
our understanding of the empathic brain,
many questions remain to be answered Notes
while at the same time new questions arise
1 Because these three terms are often used to
with newly gained insights.
denote the same underlying concept (Batson,
2009a) – with Batson primarily using the term
empathic concern (Batson, 2009b), whereas
Eisenberg instead speaks of sympathy (Eisen-
Outstanding Questions and Future berg, 2000) – we use these terms interchange-
Directions ably throughout the chapter.
Avenanti, A., Paluello, I. M., Bufalari, I., & Agli- the temporal, parietal, and premotor cortices.
oti, S. M. (2006). Stimulus-driven modulation Cerebral Cortex, bhp291.
of motor-evoked potentials during observa- Cheng, Y., Chen, C., Lin, C. P., Chou, K. H., &
tion of others’ pain. Neuroimage, 32, 316–24. Decety, J. (2010). Love hurts: An fMRI study.
Bartels, A., & Zeki, S. (2000). The neural basis of Neuroimage, 51, 923–29.
romantic love. Neuroreport, 11, 3829–34. Cheng, Y., Lin, C. P., Liu, H. L., Hsu, Y. Y.,
Bartels, A., & Zeki, S. (2004). The neural corre- Lim, K. E., Hung, D., et al. (2007). Expertise
lates of maternal and romantic love. Neuroim- modulates the perception of pain in others.
age, 21, 1155–66. Current Biology, 17, 1708–13.
Batson, C. D. (2009a). These things called empa- Craig, A. D. (2002). How do you feel? Interocep-
thy: Eight related but distinct phenomena. In tion: The sense of the physiological condition
J. Decety & W. Ickes (Eds.), The social neuro- of the body. Nature Reviews Neuroscience, 3,
science of empathy (pp. 3–15). Cambridge, MA: 655–66.
MIT Press. Craig, A. D. (2009). How do you feel – now? The
Batson, C. D. (2009b). Empathy-induced altru- anterior insula and human awareness. Nature
istic motivation. In M. Mikulincer & P. R. Reviews Neuroscience, 10, 59–70.
Shaver (Eds.), Prosocial motives, emotions, and Damasio, A. R. (1994). Descartes’ error and the
behavior (pp. 15–34). Washington, DC: Amer- future of human life. Scientific American, 271,
ican Psychological Association. 144.
Batson, C. D., Duncan, B. D., Ackerman, P., Danziger, N., Faillenot, I., & Peyron, R. (2009).
Buckley, T., & Birch, K. (1981). Is empathic Can we share a pain we never felt? Neural cor-
emotion a source of altruistic motivation? relates of empathy in patients with congenital
Journal of Personality and Social Psychology, 40, insensitivity to pain. Neuron, 67, 203–12.
290–302. Davis, M. H. (1980). A multidimensional
Batson, C. D., Fultz, J., & Schoenrade, P. A. approach to individual differences in empa-
(1987). Distress and empathy: Two qualita- thy. JSAS Catalogue of Selected Documents in
tively distinct vicarious emotions with differ- Psychology, 10, 85.
ent motivational consequences. Journal of Per- Decety, J., Echols, S., & Correll, J. (2010). The
sonality, 55, 19. blame game: The effect of responsibility and
Batson, C. D., O’Quin, K., Fultz, J., Vander- social stigma on empathy for pain. Journal of
plas, M., & Isen. A. (1983). Influence of self- Cognitive Neuroscience, 22, 985–97.
reported distress and empathy on egoistic Decety, J., & Jackson, P. L. (2006). A social-
versus altruistic motivation to help. Journal of neuroscience perspective on empathy. Cur-
Personality and Social Psychology, 45, 706–18. rent Directions in Psychological Science, 15, 54–
Beauregard, M., Courtemanche, J., Paquette, V., 58.
& St-Pierre, E. L. (2009). The neural basis of Depue, R. A., & Morrone-Strupinsky, J. V.
unconditional love. Psychiatry Research, 172, (2005). A neurobehavioral model of affiliative
93–98. bonding: Implications for conceptualizing a
Bird, G., Silani, G., Brindley, R., White, S., human trait of affiliation. Behavioral and Brain
Frith, U., & Singer, T. (2010). Empathic brain Sciences, 28, 313–50.
responses in insula are modulated by levels of de Vignemont, F., & Singer, T. (2006). The
alexithymia but not autism. Brain, 133, 1515–25. empathic brain: How, when and why? Trends
Botvinick, M., Jha, A. P., Bylsma, L. M., Fabian, in Cognitive Sciences, 10, 435–41.
S. A., Solomon, P. E., & Prkachin, K. M. Dimberg, U., & Öhman, A. (1996). Behold the
(2005). Viewing facial expressions of pain wrath: Psychophysiological responses to facial
engages cortical areas involved in the direct stimuli. Motivation and Emotion, 20, 149–82.
experience of pain. Neuroimage, 25, 312– Eisenberg, N. (2000). Emotion, regulation, and
19. moral development. Annual Review of Psychol-
Bufalari, I., Aprile, T., Avenanti, A., Di Russo, ogy, 51, 665–97.
F., & Aglioti, S. M. (2007). Empathy for pain Eisenberg, N., Fabes, R. A., Miller, P. A., Fultz, J.,
and touch in the human somatosensory cor- Shell, R., Mathy, R. M., et al. (1989). Relation
tex. Cerebral Cortex, 17, 2553–61. of sympathy and personal distress to proso-
Cattaneo, L., Sandrini, M., & Schwarzbach, J. cial behavior: A multimethod study. Journal
(2010). State-dependent TMS reveals a hier- of Personality and Social Psychology, 57, 55–
archical representation of observed acts in 66.
EMPATHY FROM THE PERSPECTIVE OF SOCIAL NEUROSCIENCE 547
Fadiga, L., Fogassi, L., Pavesi, G., & Rizzolatti, G. G. (2005). Grasping the intentions of others
(1995). Motor facilitation during action obser- with one’s own mirror neuron system. PLoS
vation: A magnetic stimulation study. Journal Biology, 3, e79.
of Neurophysiology, 73, 2608–11. Ickes, W. (1993). Empathic accuracy. Journal of
Frith, C., & Frith, U. (2006). The neural basis of Personality, 61, 587–610.
mentalizing. Neuron, 50, 531–34. Jabbi, M., Bastiaansen, J., & Keysers, C. (2008).
Frith, U., & Frith, C. D. (2003). Development and A common anterior insula representation of
neurophysiology of mentalizing. Philosophical disgust observation, experience and imagina-
Transactions of the Royal Society of London. tion shows divergent functional connectivity
Series B: Biological Sciences, 358, 459–73. pathways. PLoS One, 3, e2939.
Gallese, V., Fadiga, L., Fogassi, L., & Rizzolatti, Jabbi, M., Swart, M., & Keysers, C. (2007). Empa-
G. (1996). Action recognition in the premotor thy for positive and negative emotions in the
cortex. Brain, 119 (Pt. 2), 593–609. gustatory cortex. Neuroimage, 34, 1744–53.
Goetz, J. L., Keltner, D., & Simon-Thomas, E. Jackson, P. L., Brunet, E., Meltzoff, A. N., &
(2010). Compassion: An evolutionary analy- Decety, J. (2006). Empathy examined through
sis and empirical review. Psychological Bulletin, the neural mechanisms involved in imagining
136, 351–74. how I feel versus how you feel pain. Neuropsy-
Gottman, J. M., & Levenson, R. W. (1985). chologia, 44, 752–61.
A valid measure for obtaining self-report of Keltner, D., & Goetz, J. L. (2007). Compassion.
affect. Journal of Consulting and Clinical Psy- In R. F. Baumeister & K. D. Vohs (Eds.),
chology, 53, 151–60. Encyclopedia of social psychology (pp. 159–60).
Grèzes, J., & Decety, J. (2001). Functional Thousand Oaks, CA: Sage.
anatomy of execution, mental simulation, Keysers, C., Wicker, B., Gazzola, V., Anton, J. L.,
observation, and verb generation of actions: Fogassi, L., & Gallese, V. (2004). A touching
A meta-analysis. Human Brain Mapping, 12, sight: SII/PV activation during the observation
1–19. and experience of touch. Neuron, 42, 335–46.
Gu, X., & Han, S. (2007). Attention and reality Klimecki, O.M., Leiberg, S., Lamm, C., & Singer,
constraints on the neural processes of empathy T. (2012). Functional Neural Plasticity and
for pain. Neuroimage, 36, 256–67. Associated Changes in Positive Affect After
Hari, R., Forss, N., Avikainen, S., Kirveskari, E., Compassion Training. Cerebral Cortex. doi:
Salenius, S., & Rizzolatti, G. (1998). Activa- 10.1093/cercor/bhs142
tion of human primary motor cortex during Klimecki, O., & Singer, T. (2012). Empathic dis-
action observation: A neuromagnetic study. tress fatigue rather than compassion fatigue?
Proceedings of the National Academy of Sci- Integrating findings from empathy research
ences, 95, 15061–65. in psychology and social neuroscience. In B.
Harrison, N. A., Singer, T., Rotshtein, P., Dolan, Oakley, A. Knafo, G. Madhavan, & D. S. Wil-
R. J., & Critchley, H. D. (2006). Pupillary con- son (Eds.), Pathological altruism (pp. 368–83).
tagion: Central mechanisms engaged in sad- New York: Oxford University Press.
ness processing. Social Cognitive and Affective Knutson, B., & Cooper, J. C. (2005). Functional
Neuroscience, 1, 5–17. magnetic resonance imaging of reward predic-
Hatfield, E., Rapson, R. L., & Le, Y. L. (2009). tion. Current Opinion in Neurology, 18, 411–17.
Emotional contagion and empathy. In J. Kober, H., Barrett, L. F., Joseph, J., Bliss-Moreau,
Decety & W. Ickes (Eds.), The social neu- E., Lindquist, K., & Wager, T. D. (2008).
roscience of empathy (pp. 19–30) Cambridge, Functional grouping and cortical-subcortical
MA: MIT. interactions in emotion: A meta-analysis of
Hein, G., Silani, G., Preuschoff, K., Batson, C. neuroimaging studies. Neuroimage, 42, 998–
D., & Singer, T. (2010). Neural responses to 1031.
ingroup and outgroup members’ suffering pre- Lamm, C., Batson, C., & Decety, J. (2007). The
dict individual differences in costly helping. neural substrate of human empathy: Effects
Neuron, 68, 149–60. of perspective-taking and cognitive appraisal.
Hoffman, M. L. (2000). Empathy and moral Journal of Cognitive Neuroscience, 19, 42–58.
development. Cambridge: Cambridge Univer- Lamm, C., & Decety, J. (2008). Is the extras-
sity Press. triate body area (EBA) sensitive to the per-
Iacoboni, M., Molnar-Szakacs, I., Gallese, V., ception of pain in others? Cerebral Cortex, 18,
Buccino, G., Mazziotta, J. C., & Rizzolatti, 2369–73.
548 OLGA KLIMECKI & TANIA SINGER
Lamm, C., Decety, J., & Singer, T. (2011). Meta- trends in psychosomatic medicine (pp. 430–39).
analytic evidence for common and distinct London: Butterworths.
neural networks associated with directly expe- Neumann, R., & Strack, F. (2000). “Mood con-
rienced pain and empathy for pain. Neuroim- tagion”: The automatic transfer of mood
age, 54, 2492–2502. between persons. Journal of Personality &
Lamm, C., Nusbaum, H. C., Meltzoff, A. N., Social Psychology, 79, 211–23.
& Decety, J. (2007). What are you feeling? Ogino, Y., Nemoto, H., Inui, K., Saito, S.,
Using functional magnetic resonance imaging Kakigi, R., & Goto, F. (2007). Inner experi-
to assess the modulation of sensory and affec- ence of pain: Imagination of pain while view-
tive responses during empathy for pain. PLoS ing images showing painful events forms sub-
One, 2, e1292. jective pain representation in human brain.
Lamm, C., & Singer, T. (2010). The role of ante- Cerebral Cortex, 17, 1139–46.
rior insular cortex in social emotions. Brain Ostrowsky, K., Isnard, J., Ryvlin, P., Guénot,
Structure and Function, 214, 579–91. M., Fischer, C., & Mauguière, F. (2000). Func-
Leiberg, S., Klimecki, O. & Singer, T. (2011). tional mapping of the insular cortex: Clinical
Short-term compassion training increases implication in temporal lobe epilepsy. Epilep-
prosocial behavior in a newly developed sia, 41, 681–86.
prosocial game. PLoS One, 6, e17798. Ostrowsky, K., Magnin, M., Ryvlin, P., Isnard, J.,
Levenson, R. W., & Ruef, A. M. (1992). Empathy: Guenot, M., & Mauguiere, F. (2002). Repre-
A physiological substrate. Journal of Personal- sentation of pain and somatic sensation in the
ity and Social Psychology, 63, 234–46. human insula: A study of responses to direct
Lipps, T. (1903). Einfühlung, innere Nachah- electrical cortical stimulation. Cerebral Cor-
mung, und Organempfindungen [Empathy, tex, 12, 376–85.
inner imitation, and sense-feelings]. Archiv für Premack, D., & Woodruff, G. (1978). Does the
die gesamte Psychologie, 1, 185–204. chimpanzee have a theory of mind? Behavioral
Lutz, A., Brefczynski-Lewis, J., Johnstone, T., & and Brain Sciences, 1, 515–26.
Davidson, R. J. (2008). Regulation of the neu- Preston, S. D., & de Waal, F. B. M. (2002). Empa-
ral circuitry of emotion by compassion med- thy: Its ultimate and proximate bases. Behav-
itation: Effects of meditative expertise. PLoS ioral and Brain Science, 25, 1–72.
One, 3, e1897. Rizzolatti, G., & Craighero, L. (2004). The mir-
Mehrabian, A. (1997). Relations among person- ror neuron system. Annual Review of Neuro-
ality scales of aggression, violence, and empa- science, 27, 169–92.
thy: Validational evidence bearing on the risk Rizzolatti, G., Fadiga, L., Gallese, V., & Fogassi,
of eruptive violence scale. Aggressive Behavior, L. (1996). Premotor cortex and the recogni-
23, 433–45. tion of motor actions. Brain Research: Cogni-
Mehrabian, A., & Epstein, N. (1972). A measure tive Brain Research, 3, 131–41.
of emotional empathy. Journal of Personality, Rizzolatti, G., & Sinigaglia, C. (2010). The func-
40, 525–43. tional role of the parieto-frontal mirror cir-
Mitchell, J. P. (2009). Inferences about mental cuit: Interpretations and misinterpretations.
states. Philosophical Transactions of the Royal Nature Reviews Neuroscience, 11, 264–74.
Society of London. Series B: Biological Sciences, Saarela, M. V., Hlushchuk, Y., Williams,
364, 1309–16. A. C., Schurmann, M., Kalso, E., &
Moriguchi, Y., Decety, J., Ohnishi, T., Maeda, Hari, R. (2007). The compassionate brain:
M., Mori, T., Nemoto, K., et al. (2007). Humans detect intensity of pain from
Empathy and judging other’s pain: An fMRI another’s face. Cerebral Cortex, 17, 230–
study of alexithymia. Cerebral Cortex, 17, 37.
2223–34. Saxe, R. (2006). Why and how to study Theory of
Morrison, I., Lloyd, D., di Pellegrino, G., Mind with fMRI. Brain Research, 1079, 57–65.
& Roberts, N. (2004). Vicarious responses Saxe, R., & Baron-Cohen, S. (2006). The neuro-
to pain in anterior cingulate cortex: Is science of theory of mind. Social Neuroscience,
empathy a multisensory issue? Cognitive, 1, i–ix.
Affective & Behavioral Neuroscience, 4, Schultz, W. (2000). Multiple reward signals in
270–78. the brain. Nature Reviews Neuroscience, 1,
Nemiah, J. C., Freyberger, H., & Sifneos, P. E. 199–207.
(1976). Alexithymia: A view of the psychoso- Silani, G., Bird, G., Brindley, R., Singer, T., Frith,
matic process. In O. W. Hill (Ed.), Modern C., & Frith, U. (2008). Levels of emotional
EMPATHY FROM THE PERSPECTIVE OF SOCIAL NEUROSCIENCE 549
awareness and autism: An fMRI study. Social of oxytocin and prosocial behavior on brain
Neuroscience, 3, 97–112. responses to direct and vicariously experi-
Singer, T., Critchley, H. D., & Preuschoff, K. enced pain. Emotion, 8, 781–91.
(2009). A common role of insula in feelings, Valeriani, M., Betti, V., Le Pera, D., De Armas,
empathy and uncertainty. Trends in Cognitive L., Miliucci, R., Restuccia, D., et al. (2008).
Sciences, 13, 334–40. Seeing the pain of others while being in pain:
Singer, T., & Lamm, C. (2009). The social neuro- A laser-evoked potentials study. Neuroimage,
science of empathy. Year in Cognitive Neuro- 40, 1419–28.
science 2009: Annals of the New York Academy Wicker, B., Keysers, C., Plailly, J., Royet, J. P.,
of Sciences, 1156, 81–96. Gallese, V., & Rizzolatti, G. (2003). Both of us
Singer, T., & Leiberg, S. (2009). Sharing the emo- disgusted in My insula: The common neural
tions of others: The neural bases of empathy. basis of seeing and feeling disgust. Neuron, 40,
In M. S. Gazzaniga (Ed.), The cognitive neu- 655–64.
rosciences IV (pp. 971–84). Cambridge, MA: Xu, X., Zuo, X., Wang, X., & Han, S.
MIT. (2009). Do you feel my pain? Racial
Singer, T., Seymour, B., O’Doherty, J., Kaube, group membership modulates empathic neu-
H., Dolan, R., & Frith, C. (2004). Empathy ral responses. Journal of Neuroscience, 29, 8525–
for pain involves the affective but not sensory 29.
components of pain. Science, 303, 1157–62. Zaki, J., Ochsner, K. N., Hanelin, J., Wager, T.
Singer, T., Seymour, B., O’Doherty, J. P., D., & Mackey, S. C. (2007). Different circuits
Stephan, K. E., Dolan, R. J., & Frith, C. D. for different pain: Patterns of functional con-
(2006). Empathic neural responses are mod- nectivity reveal distinct networks for process-
ulated by the perceived fairness of others. ing pain in self and others. Social Neuroscience,
Nature, 439, 466–69. 2, 276–91.
Singer, T., Snozzi, R., Bird, G., Petrovic, P., Zeki, S. (2007). The neurobiology of love. FEBS
Silani, G., Heinrichs, M. et al. (2008). Effects Letters, 581, 2575–79.
Section VII
INDIVIDUAL
DIFFERENCES IN
EMOTION
CHAPTER 24
Studies of the brain basis of “norma- pathways through which risk to affective ill-
tive” or “healthy” processing of emotion- ness is conferred.
ally salient stimuli have flourished over Understanding the brain basis of vulner-
the last two decades. An initial focus on ability to affective disorder goes hand in
regions implicated in the detection of emo- hand with a focus on individual variation
tionally salient stimuli (Morris et al., 1996; and, in particular, trait differences in the
Whalen et al., 1998) has broadened to mechanisms supporting the detection and
include discussion of mechanisms support- controlled processing of emotional stimuli.
ing regulatory functions (Bishop, Duncan, How do we study trait differences in vul-
Brett, & Lawrence, 2004; Davidson, 2002; nerability to anxiety and depressive disor-
Ochsner, Bunge, Gross, & Gabrieli, 2002; ders and try to unpack the brain mechanisms
Kim, Somerville, Johnstone, Alexander, & though which these might act? A number of
Whalen, 2003; Phelps, Delgado, Nearing, & approaches have been adopted, with both
LeDoux, 2004). Running in parallel to this shared and unique advantages and limita-
literature, psychiatric imaging studies have tions.
described alterations in brain function across Studies of the brain basis of vulnerabil-
a wide range of anxiety and depressive dis- ity to affective disorders typically rely on
orders (for reviews and meta-analyses see recruiting nonclinical volunteer samples and
Etkin & Wager, 2007; Ressler & Mayberg, then regressing scores on self-report mea-
2007; Shin & Liberzon, 2010; Stein 2009). The sures of trait affect or experience of disorder-
study of the brain mechanisms underlying related symptomatology against indices of
vulnerability to disorder has, for some rea- brain function or structure. It is important
son, fallen outside of the primary spotlight. to remember that this approach is correla-
We argue that work of this nature is criti- tional in nature, and hence no conclusions
cal to bridging studies in healthy volunteer can be drawn about the direction of causal-
and patient groups and to identifying the ity. For example, if we find that, in a student
553
554 SONIA BISHOP & SOPHIE FORSTER
population, elevated scores on a measure category uses measures taken from the per-
of neuroticism are linked to poor frontal sonality literature, such as the Neuroticism
recruitment, this could equally plausibly scale from either the Eysenck Personality
reflect individuals scoring high on neuroti- Questionnaire (EPQ; Eysenck & Eysenck,
cism being less able to cope with envi- 1975) or the NEO Personality Inventory
ronmental stress, resulting in diminished (Costa & MacCrae, 1992). The third cat-
frontal function; individuals with compro- egory focuses on genetic markers that
mised frontal function being more likely differ among individuals (functional poly-
to develop a neurotic personality style; morphisms with two or more common vari-
both elevated neuroticism and compro- ants) and that have been linked to differ-
mised frontal function stemming from a pri- ences in affect-related behaviors in humans
mary disruption to neurotransmitter func- and other species. In this chapter, we focus
tion; or all of these factors in combination. on the first two of these categories, com-
There are also important methodological menting only briefly on the third (for
issues pertaining to good practice in con- further discussion, see Chapter 25). The
ducting correlational analyses of brain activ- majority of studies on the brain basis of vul-
ity, which are discussed briefly later in the nerability to affective disorder falling within
chapter. these categories have used measures of trait
A shared positive feature of studies in this anxiety (category 1) or neuroticism (cate-
area is that constructs such as trait anxiety or gory 2). We use these examples to explore
neuroticism can be examined as continuous the state of the field as it stands and to
factors, facilitating exploration of their lin- address outstanding questions for future
ear and nonlinear relationships with regional research.
brain activity. This allows for a more com-
plex and potentially accurate picture to be
drawn than one that solely uses DSM cat- Trait Anxiety and Neuroticism:
egorical assessments of the binary presence Indexing Vulnerability to Affective
or absence of a given disease state. The lat- Disorders through Self-Report
ter “diagnostic” approach faces limitations
arising from difficulties in applying cate- The trait subscale of the Spielberger State
gorical cutoffs, high comorbidity between Trait Anxiety Inventory (Spielberger et al.,
many anxiety and depressive disorders, and 1983) is a widely used measure of trait
poor diagnostic reliability (Brown & Barlow, propensity to anxiety. It has been shown
2009). to have good concurrent validity, with
Studies of the brain basis of vulnerabil- patients with anxiety disorders (ADs) scor-
ity to affective disorder can be categorized ing higher on the STAI trait subscale than
according to the measure used to assess controls (Bieling, Antony, & Swinson, 1998).
individual differences in trait affective style. Although fewer studies have examined pre-
Arguably there are three main categories. dictive validity, pretrauma STAI trait scores
The first involves measures derived from have been found to predict levels of post-
the clinical literature and normed for use traumatic stress disorder symptomatology
in nonclinical populations to assess individu- after trauma (Weems et al., 2007). How-
als’ tendency to show disorder-related symp- ever, the STAI has been criticized for having
tomatology, affective and cognitive styles. poor discriminative validity, with individu-
A prominent example is the Spielberger als with major depressive disorder (MDD)
State Trait Anxiety Inventory (STAI; Spiel- also showing elevated scores on the trait
berger, Gorsuch, Lushene, Vagg, & Jacobs, subscale (Mathews, Ridgway, & Williamson,
1983), widely used in studies of the cog- 1996). One possibility is that this low dis-
nitive correlates of trait anxiety within criminant validity reflects a poor choice
nonselected student samples. The second of anxiety-specific items for the scale. A
TRAIT ANXIETY, NEUROTICISM, AND THE BRAIN BASIS OF VULNERABILITY TO AFFECTIVE DISORDER 555
or current mood levels and not on trait dif- subject to criticism (Vul, Harris, Winkiel-
ferences between individuals. Indeed, the man, & Pashler, 2009). However, many of
scarcity of trait measures of propensity to the issues raised – pertaining to whole-brain
anxiety and depression poses a major dif- correlational analyses, insufficient correc-
ficulty for researchers aiming to investigate tion for multiple comparisons, and biased
the brain basis of these tendencies. Not only selection of regions of interest – can be
the MASQ but also the major depressive avoided if investigations of differences in
inventories – including both the BDI and the regional brain function associated with per-
Center for Epidemiologic Studies Depres- sonality indices or trait affective style are
sion Scale (CES-D; Radloff 1977) – focus on constrained to ones that specifically test the-
current levels of symptomatology. This may ories derived from the cognitive or social
explain why several imaging studies have psychological literature. An elegant argu-
used neuroticism as a proxy for vulnerability ment for this approach was initially made
to depression, but doing so inevitably hin- by Kosslyn et al. (2002). In this article, Koss-
ders attempts to disentangle the extent to lyn and colleagues take the proposal put
which neuroticism, trait anxiety, and trait forward by Underwood (1975) – that nat-
depression involve disruption to common or urally occurring individual differences can
unique mechanisms at either a cognitive or be used to test psychological theories and
systems (regional brain structure and func- to reveal the structure of psychological pro-
tion) level of analysis. cesses, potentially providing greater insights
than group-based methods – and argue that
the same logic can be applied to the use
Studying Affective Trait-Related of individual differences to investigate the
Differences in Cognitive and Brain biological mechanisms that underpin cogni-
Function: Lessons from the Last tive processes. The authors make the case
Decade that there is natural variation around every
central tendency, that individuals may differ
Over the last 10 years, much progress has in the efficiency and recruitment of mecha-
been made in using neuroimaging tech- nisms (which can be studied at various lev-
niques to study individual differences in els including both regional brain activation
neurocognitive function; recent challenges and cognitive processing), and that pooling
and developments set the stage for equiva- information across individuals may be unin-
lent progress over the next decade. In the formative or misleading. They also point out
early 2000s, neuroimaging studies of person- that the main dangers of unguided correla-
ality led to a conceptual shift in the approach tional studies can be avoided by theoretically
to the investigation of human brain func- grounding the study design and the analy-
tion. These studies argued that individual sis and interpretation of results, with alter-
variation need not simply be treated as a nate theoretical explanations of observed
source of noise in group studies of “nor- correlations being used to generate further
mative” neurocognitive function but that hypotheses than can in turn be tested. In
between-subject differences could be stud- the sections that follow, we illustrate how
ied in their own right by examining associa- the approach proposed by Kosslyn and col-
tions between traits such as extraversion or leagues can be applied, using the example of
neuroticism and regional brain function (for studies that have investigated the brain basis
a review see Canli, 2004). of the association between trait anxiety and
Although these studies were ground- attentional capture by threat. In addition,
breaking in advancing the investigation we explore if we can ascertain whether neu-
of individual differences in neurocognitive roticism shows a similar, potentially com-
function, as with many first steps in a new mon, relationship to the function of these
field, several of these studies have since been mechanisms.
TRAIT ANXIETY, NEUROTICISM, AND THE BRAIN BASIS OF VULNERABILITY TO AFFECTIVE DISORDER 557
Top-down
attentional control
mechanisms
Bottom-up sensory-
driven mechanisms
(a) (b)
**
TUMOR
APPLE TUMOR
Figure 24.3. Two widely used tasks in the attention to threat literature. (A) In the Emotional Stroop
task, participants are asked to name the font color of a word (here green/gray), ignoring its meaning,
which can be either threat-related or neutral in valence. High trait anxious subjects show RT slowing
for threat-related words. (B) In the Probe Detection task, participants are presented with two words,
followed by a “probe” (the two asterisks presented here) in the location of one of the words. They are
typically asked either to indicate when the probe appears or to specify its orientation. On key trials,
one word is threat-related, and the other is neutral. High trait anxious individuals show RT speeding
when the location of the probe was previously occupied by a threat word (as in the example here),
suggesting that attention was allocated to the location of the threat word.
558 SONIA BISHOP & SOPHIE FORSTER
Trait Anxiety, Neuroticism, and models typically propose that anxiety acts
Threat-Related Biases in Selective by amplifying the signal from a bottom-
Attention: Cognitive Models and up preattentive threat detection mecha-
Findings nism that biases attentional competition in
favor of threat-related stimuli. When these
Both AD patients and high trait anxious par- stimuli are distracters (non-task-relevant),
ticipants show elevated attentional capture this is held to interfere with the process-
by threat-related stimuli (Mathews & Mack- ing of target stimuli, as indexed by slowed
intosh, 1998). According to biased compe- reaction times (RTs) and/or elevated error
tition models of selective attention, atten- rates. These models have not, in the most
tional competition is influenced both by part, argued for disrupted top-down or con-
“bottom-up” sensory mechanisms that prior- trolled processing of threat-related stimuli in
itize the processing of salient stimuli and by anxiety.
“top-down” attentional control mechanisms A number of studies have used variants
that support the processing of task-relevant of the Probe Detection task to examine
stimuli (Figure 24.2; Desimone & Duncan, whether attentional biases are also associ-
1995; Kastner & Ungerleider, 2000). Stimu- ated with elevated neuroticism (“N”) scores.
lus valence – the extent to which a given These have produced rather mixed results.
stimulus is threat or reward related – is an Reed and Derryberry (1995) found evidence
important dimension of stimulus salience. for a correlation between N scores and atten-
A number of selective attention tasks have tional bias toward negative trait adjectives
been adapted to examine how stimulus previously rated as self-applicable. How-
valence, especially threat-relatedness, influ- ever, this correlation was only observed at
ences attentional competition. Two notable one of three alternate adjective-probe stim-
examples are the Emotional Stroop and ulus onset asynchronies (500 ms, not 250 ms
Probe Detection tasks (Figure 24.3). In the or 750 ms). In addition, Chan, Goodwin,
Emotional Stroop task, participants name and Harmer (2007) and Rijsdijk et al. (2009)
the ink color of a stimulus word while ignor- failed to find any relationship between neu-
ing its semantic content. On this task, high roticism and performance on probe detec-
trait anxious individuals show slower color tion tasks using social threat words and
naming of threat-related words than emo- subliminally presented threat-related faces,
tionally neutral words; in low trait anxious respectively. Differences among these stud-
individuals this slowing is reduced or absent ies in the choice of stimuli, stimulus onset
(Richards & Millwood, 1989). In the Probe asynchrony, and neuroticism scale (EPQ
Detection task, participants are presented versus NEO) further complicate interpreta-
with two words or pictures (e.g. faces) fol- tion of these findings.
lowed by a single dot or pair of dots in the Interestingly, the limited evidence for an
position previously occupied by one of the association between neuroticism and atten-
two stimuli. Here, high trait anxious indi- tional bias toward negatively valenced stim-
viduals are faster to detect the presence of a uli parallels similarly mixed findings within
single dot or to determine the orientation the subclinical depression literature. Here,
of a pair of dots, when the dot probe is two studies using the Emotional Stroop
presented in the position previously occu- task have reported increased RT slowing
pied by a threat-related stimulus (Macleod for color naming of negatively valenced
& Mathews, 1988). words as a function of scores on the Beck
These findings have informed cognitive Depression Inventory (BDI). These effects
models of anxiety that extend the biased were strongest in participants with elevated
competition model of selective attention to BDI scores at two time points a year apart
specifically deal with attentional capture by (Williams & Nulty, 1986) and were not found
threat-related stimuli (Mathews & Mackin- when a depressed mood was induced in par-
tosh, 1998; Mogg & Bradley, 1998). These ticipants with low BDI scores (Gotlib &
TRAIT ANXIETY, NEUROTICISM, AND THE BRAIN BASIS OF VULNERABILITY TO AFFECTIVE DISORDER 559
McCann, 1984), potentially suggesting the lar pattern of task-specific hyperactivity and
role of an enduring trait conferring vulnera- hypo-recruitment. Studies that have begun
bility to depression, rather than simply state to address this question are reviewed next.
affect. Other studies have found no rela-
tionship between individual differences in
subclinical levels of depression and atten- From Networks to Modules and Back
tional bias toward negative or threat-related Again
stimuli across both the Emotional Stroop
and Probe Detection tasks (Bradley, Mogg, The late 1930s through to the early 1950s
Falla, & Hamilton, 1998; Gotlib, MacLachan, saw the advent of theories that proposed
& Katz, 1988; Hill & Dutton, 1989; Hill & that networks of brain regions including
Knowles, 1991; Macleod & Hagan, 1992). areas such as the hippocampus, cingulate
One possibility is that a partial corre- gyrus, amygdala, and orbital frontal cortex
late of depression scores, such as trait anx- were responsible for emotional processing
iety levels, rather than depression itself, (MacLean 1949; Papez, 1937). Support for
might be responsible for the intermittently the “Papez circuit” and “limbic system” was
reported attentional interference effects. A countered by criticisms that these accounts
similar argument can be made for neuroti- were more descriptive than functional and
cism. The NEO measure of neuroticism lacked a clear rationale for inclusion of cer-
comprises different subfacets, one of which tain regions and the exclusion of others (for
is especially related to anxiety and another to more extensive discussion of these criticism,
depression (Costa & MacCrae, 1992; see also see Chapter 9 in Gazzaniga Ivry, & Mangun,
Figure 24.1). Variability among studies in 2009).
the items that high-N participants endorse Early neuroimaging studies of emotion
might explain the occasional but inconsis- conducted in the late 1990s and early 2000s
tent findings of attentional biases reported took a more modular approach. Much
by studies using this measure. research was focused on the amygdala and
It is also possible that multiple mecha- its role in the detection or evaluation of
nisms contribute to attentional capture by threat (Morris et al., 1996; Vuilleumier,
threat stimuli – with trait anxiety, neu- Armony, Driver, & Dolan, 2001; Whalen
roticism, and depression potentially sharing et al., 1998, 2004) Indeed, a number of stud-
a common relationship with one of these ies from the later part of this era used scan
mechanisms but differing in their associa- parameters that focused data acquisition on
tion with others. The most obvious can- a narrow slab of slices covering the amygdala
didate mechanisms are those involved in but omitting much of the rest of the brain.
bottom-up responsivity to threat versus top- In contrast, within the last 5 to 10 years,
down attentional control. Investigation of there has been an increasing focus on the
the brain basis of these mechanisms opens interplay of regions involved in the evalua-
up a new door for examining how different tion of threat stimuli with those that enable
trait characteristics are linked to the func- the regulation of emotional state and phys-
tion of these component processes. Specif- iological fear responses, the (re)appraisal of
ically, it is possible to build on what is stimuli, and the control of attentional focus
known about the function of different brain (Bishop, Duncan, Brett, & Lawrence, 2004a;
regions to explore whether trait anxiety Kim et al., 2003; Ochsner et al., 2002; Phelps
is associated with increased activation of et al., 2004). This focus has been accom-
brain mechanisms involved in processing panied by a shift from examining brain
stimulus threat value, with impoverished regions in isolation and toward conceptu-
recruitment of brain mechanisms involved alizing regions as nodes within intercon-
in attentional control, or with altered func- nected networks, the activation of which
tion of both processes. We can also explore varies with task engagement and may be
whether neuroticism is linked to a simi- meaningful even at “rest” (Deco, Jirso, &
560 SONIA BISHOP & SOPHIE FORSTER
Rostral anterior
cingulate cortex
(rACC)
Striatum:
Ventrolateral
caudate & putamen
prefrontal cortex
(VLPFC)
Cerebellum
Figure 24.4. Frontal brain regions implicated in regulation of attention to emotionally and
non-emotionally salient stimuli include the dorsolateral prefrontal cortex (DLPFC), ventrolateral
prefrontal cortex, (VLPFC), rostral anterior cingulate cortex (rACC), and dorsal anterior (mid)
cingulate cortex (dACC). Subgenual anterior cingulate cortex is not shown here. Adapted with
permission from Bush, 2010.
Intriguingly, a more recent study has pro- tional distracter and response-competition
duced findings that are discrepant from both paradigms. One hopes that these discrepant
those of Bishop and colleagues (2004) and findings will spark further research that may
those from the earlier response-competition help resolve and unite these results and
literature. Using a “face/word” version of advance our theoretical understanding of
the Stroop task, Etkin, Peraza, Kandel, and the role of the ACC and LPFC in the detec-
Hirsch (2006) asked volunteers to indicate tion and resolution of different types of pro-
whether faces showed fearful or happy cessing competition.
expressions while ignoring the word “happy” A further point worth considering here
or “fearful” superimposed on each face. concerns the role of rostral versus dorsal
They reported that activation of the ACC, subdivisions of the ACC (see Figure 24.4).
rather than the LPFC, increased when pro- Early studies reported dorsal ACC activity
cessing competition was expected (frequent when processing competition arose from
face/word incongruent trials) and that LPFC response competition, and rostral ACC
activity increased when processing compe- activity when processing competition was
tition was infrequent or unexpected. One caused by the presence of emotion-
interesting difference from the study by ally salient but task-irrelevant distracters
Bishop, Duncan, Brett, and Lawrence (2004) (Bishop, Duncan, Brett, & Lawrence, 2004;
is that in Etkin”s task, both the distracters Bush, Luu, & Posner, 2000). This distinc-
and targets were emotionally valenced, with tion is also apparent within the psychiatric
the valence of targets and distracters being imaging literature (Bush et al., 1999; Shin
balanced across “high-conflict” and “low- et al., 2001). However, more recently this
conflict” trials; those conditions differed distinction has been challenged by a range
instead in face/word congruence. It is diffi- of studies suggesting a role for the dorsal
cult to disentangle effects of emotional con- ACC in emotional processing – not only in
gruency from response congruency in this emotional distracter tasks but also in stud-
design. However, this still does not easily ies investigating the anticipation and expe-
explain why the conditions under which rience of pain and the expression of con-
the LPFC and ACC were activated differed ditioned fear (Bishop et al., 2007; Milad,
from those previously observed in both emo- Quirk, et al., 2007; see Shackman et al., 2011,
TRAIT ANXIETY, NEUROTICISM, AND THE BRAIN BASIS OF VULNERABILITY TO AFFECTIVE DISORDER 563
for a comprehensive review). This also calls independent relationship between trait anx-
for further investigation of the precise func- iety and impoverished recruitment of frontal
tion of this region. The difference in nomen- attentional control mechanisms comes from
clature conventions used across groups and the ERP literature. Using an antisaccade
the changes in those terms across time com- task (where volunteers must saccade away
plicate this investigation. In this chapter, we from the position in which a cue is pre-
follow the nomenclature convention intro- sented), Ansari and Derakshan (2011) found
duced by Mayberg and colleagues (1999) that high trait anxious individuals showed
and adopted in our earlier work (Bishop, longer antisaccade latencies together with
Duncan, Brett, & Lawrence, 2004 , 2007), reduced frontocentral activity during anti-
whereby rostral ACC excludes subgenual saccade preparation.
ACC (the region ventral to the corpus callo- The studies discussed here provide some
sum). The subdivision referred to here as the initial evidence that trait anxiety is linked to
dorsal ACC or dACC (see Figure 24.4) has impoverished recruitment of frontal regions
elsewhere been renamed the dorsal anterior important for attentional control both when
mid-cingulate gyrus since this term arguably processing competition is caused by threat-
reflects more accurately the region under related distracters and when it is caused by
consideration (Bush, 2010). response conflict. This raises the question
The second question raised above con- of whether this deficient recruitment can be
cerns the nature of the association between remediated by cognitive interventions such
trait anxiety and the deficient recruitment as attentional training. A number of early tri-
of frontal attentional control mechanisms. Is als provide some initial hope that this might
this association specific to attentional con- indeed be the case (Amir, Weber, Beard,
trol over threat, perhaps being secondary Bomyea, & Taylor, 2008; Hakamata et al.,
to hyper-responsivity of the amygdala to 2010).
threat-related stimuli? Or does it reflect a Although work reviewed here suggests
more general dysregulation of frontal atten- a relationship between trait anxiety and
tional function that is independent of amyg- frontal dysfunction in the absence of task-
dala responsivity to threat? The cognitive lit- related differences in amygdala activity
erature provides some suggestion that the (Bishop, 2009), this leaves open the question
latter might be the case, with findings link- as to whether trait anxiety is also linked to
ing trait anxiety to impoverished or inef- amygdala hyper-responsivity to threat in the
ficient attentional control (Derryberry & absence of differential activation of frontal
Reed, 2002; Eysenck, Derakshan, Santos, mechanisms. To address this question, we
& Calvo, 2007). In a test of the hypoth- need to turn to tasks that require the rel-
esis that trait anxiety is associated with atively passive processing of threat-related
reduced recruitment of frontal attentional stimuli in the absence of demands on atten-
control mechanisms even in the absence tion or other executive processes. Surpris-
of threat-related stimuli, Bishop (2009) ingly few such studies exist that meet the
examined DLPFC recruitment while volun- constraints of having examined amygdala
teers performed a response-competition task function and also having measured indi-
under conditions of low versus high per- vidual differences in trait anxiety. In one
ceptual load. High trait anxious volunteers early study, Etkin et al. (2004) reported that
showed reduced DLPFC recruitment to trait anxiety was associated with elevated
high response-competition trials under con- amygdala responsivity to masked threat-
ditions of low but not high perceptual load, related faces, but not to unmasked threat-
in line with the Lavie model and consis- related faces. In a second study, Stein, Sim-
tent with trait-anxiety-related dysregulation mons, Feinstein, and Paulus (2007) reported
of frontal attentional mechanisms extending that elevated STAI trait scores were linked
beyond the specific case of attentional con- to greater amygdala activity during condi-
trol over threat. Further support for a threat- tions requiring matching of facial emotions
564 SONIA BISHOP & SOPHIE FORSTER
than during conditions requiring matching trait anxiety only partially covary in their
of basic shapes. relationship with the function of discrete
One intriguing possibility is that an asso- brain mechanisms as might, for example, be
ciation between trait anxiety and amyg- predicted by Clark and Watson’s tripartiate
dala responsivity to threat-related stimuli model (see Figure 24.1).
might primarily be seen when the stimuli At the time of writing, three studies
are ambiguous or require some form of res- have examined the correlates of neuroti-
olution of their threat value (Whalen, 2007). cism while participants perform fMRI tasks
This is arguably more the case for masked involving manipulation of selective atten-
than for unmasked threat-related faces and tion and emotional stimuli. Two of the
also potentially the case when different faces three – one using the Emotional Stroop task
with varying emotional expressions need to and the other the Probe Detection task –
be judged for their equivalence in expres- reported no significant association between
sion. This is clearly a tentative hypothesis, neuroticism scores and regional brain activ-
and it remains to be more firmly established ity during conditions of attentional compe-
under precisely which conditions trait anxi- tition from emotional stimuli (Amin, Con-
ety is linked to amygdala hyper-responsivity stable, & Canli, 2004; Canli, Amin, Haas,
to threat. Furthermore, given the possibil- Omura, & Constable, 2004). However, given
ity that frontal mechanisms may be impor- that the sample size in each case was fairly
tant for certain forms of ambiguity resolu- small for a correlational study (12 or fewer
tion (Kim et al., 2003; Nomura et al., 2003), subjects), it is difficult to draw strong con-
it will also be important to confirm that clusions from these null results.
trait-anxiety-related differences in amygdala In a subsequent larger study (n = 36),
responsivity to weak or ambiguous threat Haas, Omura, Constable, and Canli (2007)
stimuli are not secondary to the differential administered a Stroop-like task, similar to
recruitment of frontal mechanisms. that of Etkin et al. (2006), in which tri-
als differed in the emotional congruence
of target words and the expressions of
Neuroticism and the Brain background faces. In this study, the words
Mechanisms Influencing Attentional did not map directly onto the names of
Capture by Threat the facial expressions, thereby reducing the
association between emotional incongru-
As reviewed in the previous section of this ency and response conflict. Individuals with
chapter, in recent years an increasing num- high neuroticism levels were found to show
ber of studies have examined the relation- increased amygdala and subgenual anterior
ship between trait anxiety and recruitment cingulate activity to trials with emotion-
of the frontal and amygdaloid mechanisms ally incongruent face/word pairs (collapsing
implicated in attentional control over threat positive-face/negative-word and negative-
distracters. This makes it possible to start face/positive-word trials). This result is
to ask relatively specific questions about intriguing, but difficult to relate to find-
which parts of the neural circuitry influ- ings from imaging studies examining the
encing attentional capture by threat show influence of trait anxiety on the neural
altered function in high trait anxious indi- mechanisms regulating selective attention
viduals. Although the corresponding litera- to threat, because the task manipulation
ture on neuroticism is more limited, we can used by Haas and colleagues (emotional con-
look at the initial studies available to begin gruency) is not one of distracter threat-
to assess whether neuroticism and trait anx- relatedness, being orthogonal to distracter
iety show similar relationships with regional valence. One possibility is that the height-
brain function – as might be expected if ened amygdala responsivity to emotion-
they both represent the same single underly- ally incongruent stimuli in high-N individ-
ing construct – or whether neuroticism and uals reported here might primarily reflect
TRAIT ANXIETY, NEUROTICISM, AND THE BRAIN BASIS OF VULNERABILITY TO AFFECTIVE DISORDER 565
sensitivity to stimuli that are ambiguous in supporting attentional control, during non-
their emotional significance, in line with the emotional task performance, in a similar
proposals put forward by Whalen and col- manner to that observed for trait anxiety.
leagues (Whalen, 2007). There is little existing work that pertains to
To explore the “ambiguity sensitivity” this issue. Using an oddball detection task,
hypothesis further, we review studies that Eisenberger, Lieberman, and Satpute (2005)
have examined the influence of neuroti- found that neuroticism was associated with
cism on regional brain activity to emotional reduced lateral prefrontal cortical and ros-
stimuli using passive viewing or cognitively tral ACC recruitment but increased dorsal
undemanding tasks. Here, the question of ACC activity. Further detailed investigation
interest is whether neuroticism is particu- of the relationship between neuroticism and
larly linked to heightened amygdala respon- recruitment of lateral frontal and anterior
sivity to emotional stimuli when these stim- cingulate subregions is required to form a
uli are in some form ambiguous in their clearer picture of the commonalities and dif-
valence or threat-relatedness. Canli and col- ferences in the relationship between neu-
leagues reported that although the amygdala roticism and trait anxiety with regional brain
response to happy faces and positive images function. In particular, as noted earlier, the
was predicted by extraversion, there was no precise role of the dorsal ACC in cognitive
relationship between neuroticism and amyg- and emotional processing remains an issue
dala responsivity to either negative emo- under active debate.
tional faces or negative emotional images It is hoped that this chapter has provided
(Canli et al. 2001, 2002). Similarly, Britton, a flavor of how, in line with the case made
Ho, Taylor, and Liberzon (2007) found no by Kosslyn and colleagues, it is possible
relationship between neuroticism and amyg- to conduct neuroimaging studies that may
dala responsivity during the passive view- advance both our understanding of the brain
ing of emotional images, facial expressions, mechanisms supporting attentional capture
and emotional films. Cremers et al. (2010) by threat and the relationship between trait
also found no relationship between neuroti- anxiety and variation in the function of these
cism and the amygdala response to negative mechanisms. Although there are fewer stud-
emotional faces during a gender discrimi- ies pertaining to neuroticism, the available
nation task. The one exception is a study studies serve to generate hypotheses that
by Chan, Norbury, Goodwin, and Harmer could form the basis of future research.
(2009) that examined amygdala responsiv- The work reviewed here also raises ques-
ity to fearful, happy, neutral, and morphed tions as to the extent to which the associ-
facial expressions during a gender discrim- ation among trait predisposition to anxiety,
ination task in individuals high and low amygdala hyperactivity, and frontal hypo-
in neuroticism. Here high N scores were activity is dependent on task domain. Specif-
associated with stronger amygdala activity ically, will the associations reported here
to faces with expressions “morphed” part- also be observed in the context of per-
way between neutral and fear. It could be formance of other tasks? Or even in the
argued that these morphed facial stimuli are absence of any task? Studies pertaining to
milder or more ambiguous in their threat these questions are reviewed in the next two
value than the “fully” negative stimuli used sections.
in the other studies. However, this one find-
ing does not permit any definitive conclu-
sions to be drawn in favor of the “ambi- Trait Anxiety and Hyper-Amygdala
guity” account without further empirical and Hypo-Frontal Function: The Case
investigation. of Fear Conditioning
It also remains to be established whether
neuroticism is associated with impover- The rodent fear conditioning literature pro-
ished recruitment of the frontal mechanisms vides strong evidence for the role of frontal
566 SONIA BISHOP & SOPHIE FORSTER
inhibitory influences on the amygdala in that the medial and lateral regions of ven-
the attenuation of physiological and behav- tral prefrontal cortex reported in this study,
ioral fear responses. Findings indicate that the activation of which was associated with
the amygdala is involved in the acqui- lower cued and contextual fear responses,
sition and expression of cued fear, with overlap with those reported elsewhere to
medial prefrontal cortical inputs inhibit- be activated during deliberate emotion reg-
ing amygdala responsivity to conditioned ulation and affective stimulus reappraisal
fear stimuli (CSs) following extinction train- (see Chapter 16). Delgado and colleagues
ing (Maren & Quirk, 2004; Sotres-Bayon, have further reported that activation of
Bush, & LeDoux, 2004). Human studies similar ventral PFC regions accompanies a
of conditioned fear have implicated simi- reduction in conditioned fear, regardless of
lar circuitry with ventral medial regions of whether it occurs due to extinction or emo-
prefrontal cortex (vmPFC) facilitating tion regulation (Delgado, Nearing, LeDoux,
context-specific recall of “CS – uncondi- & Phelps, 2008).
tioned stimulus (UCS) absent” associations Together with the work reviewed ear-
formed during extinction training (Milad, lier, these findings raise the possibility that
Wright, et al., 2007; Phelps et al., 2004). individual variation in the recruitment of
Disruption to this circuitry has been doc- different subregions of frontal cortex may
umented in adults with posttraumatic stress influence volunteers’ ability to regulate their
disorder (Milad et al., 2009) and has been emotional responses to the anticipation of
proposed to be of potential relevance to aversive stimuli, as well as their locus of
other anxiety disorders. attention when threat-related visual stimuli
Recently, a handful of studies have begun are presented. It is important to note that
to address whether trait vulnerability to characteristics such as trait anxiety may map
anxiety is linked to altered functioning of onto individual differences not only in the
this circuitry. Two initial studies reported ability to regulate responses to aversive stim-
that high trait anxious individuals showed uli but also in the nature of the regulation
elevated amygdala activity during extinc- strategy selected. Initial studies have begun
tion (Barrett & Armony, 2009; Sehlmeyer et to examine the brain regions activated by
al., 2011). Associations of extinction-related different emotion regulation strategies (e.g.,
ACC activity with trait anxiety were also Vrtička, Sander, & Vuilleumier, 2011), but
reported, but the directionality and specific as yet this work has not been integrated
locus of these effects did not replicate across with investigation of individual differences
the two studies. In recent work from our in strategy selection or success in implemen-
own lab, we found that elevated amygdala tation.
activity to a CS that predicted an aversive
stimulus (the UCS) mediated the relation-
ship between trait anxiety and the strength Trait Vulnerability to Affective
of initial acquisition of skin conductance Disorder: What Might We Learn from
responses to the predictive CS (Indovina Resting State Studies?
et al., 2011). We also found that trait anx-
iety was negatively related to recruitment Recently, there has been increasing inter-
of ventral frontal regions linked to context- est in whether the structural and functional
appropriate down-regulation of both cued connectivity between different brain regions
and contextual fear prior to omission of may be informative even in the absence of
the UCS. Hierarchical regression revealed task performance. One high-profile exam-
that the relationship between trait anxiety ple is the recently launched Human Con-
and amygdala responsivity to the predic- nectome Project, which aims to “compre-
tive CS was independent of that between hensively map human brain circuitry . . .
trait anxiety and context-appropriate ven- using cutting-edge methods of noninva-
tral PFC recruitment. It is interesting to note sive neuroimaging . . . yield[ing] invaluable
TRAIT ANXIETY, NEUROTICISM, AND THE BRAIN BASIS OF VULNERABILITY TO AFFECTIVE DISORDER 567
information about brain connectivity, its of this nature. Second, an issue that per-
relationship to behavior, and the con- tains to all resting state studies is that it is
tributions of genetic and environmental not clear how to interpret negative versus
factors to individual differences in brain positive patterns of BOLD connectivity at
circuitry” (http://humanconnectome.org/). rest. Are inhibitory connections at the neu-
This interest has been accompanied by a ronal level likely to be reflected as negative
renewed emphasis on examining the func- BOLD connectivity patterns? This is often
tion of brain regions in the context of assumed but far from established. A third
the networks in which they are embedded problem is that it is unclear as to what crite-
as “nodes.” It has also brought increased ria should be used for including or excluding
recognition of the need to study individual regions in different “resting state” networks,
differences in order to understand norma- raising spectra of the criticisms applied to
tive brain function. It is beyond the scope the circuits of Papez (1937) and MacLean
of the current chapter to provide a com- (1949). Specifically, with both seed-based
prehensive review of this literature (see and component-based approaches, the same
Deco et al., 2011). We limit this section to question occurs as to what threshold to
a consideration of a few findings to date use – whether in terms of significance lev-
that may inform our understanding of the els for whole-brain seed-driven analyses or
brain basis of trait vulnerability to affective the number of components for indepen-
disorder. dent or principal-components-based anal-
Roy et al. (2009) provided a detailed yses. As the field develops, the challenge
report of connectivity between the amyg- will be to find ways to address these
dala and other brain regions at rest. issues.
Positive correlations were found between Building on the work by Roy and col-
resting state blood-oxygen-level-dependent leagues, Kim et al. (2011) examined the
(BOLD) activity in the amygdala and a num- influence of individual differences in self-
ber of brain regions, including the medial reported pre-scan anxiety on resting state
frontal gyrus, rostral ACC, dorsal ACC, functional connectivity. They reported that
insula, thalamus, and striatum. Negative anxiety levels significantly modulated con-
correlations were also observed between the nectivity between the amygdala and only
amygdala and areas, including the superior two regions. High state anxious individu-
frontal gyrus, bilateral middle frontal gyrus, als showed negative amygdala-vmPFC con-
posterior cingulate cortex, precuneus, and nectivity contrasting with positive func-
parietal and occipital lobes. Further anal- tional connectivity between these regions
yses were conducted to profile the con- in low state anxious individuals. In addi-
nections of different amygdala subnuclei. tion, they also showed an absence of the
These provided some suggestion of nega- negative connectivity between the amyg-
tive connectivity between the basolateral dala and dorsal medial PFC observed in low
nucleus and central nucleus of the amyg- state anxious volunteers. The effects of anx-
dala, as well as opposing patterns of con- iety in this study are particularly of inter-
nectivity between these subnuclei with the est because of their selectivity. In advanc-
medial frontal gyrus and anterior cingulate ing our understanding of trait vulnerability
cortex. to affective disorder, a limitation is that
These findings are intriguing, but have the primary analyses presented used state
several limitations. First, as noted by anxiety measures, with analyses using trait
the authors, it is extremely difficult to anxiety measures being described as show-
specify amygdala subnuclei on echoplanar ing similar but weaker trends. One hopes
images. Although the probabilistic approach that future studies will further explore
adopted by Roy and colleagues can provide the extent to which anxiety-related vari-
an estimate, it is not clear if the level of ability in amygdala-frontal functional con-
accuracy achieved is sufficient for analyses nectivity at rest reflects stable individual
568 SONIA BISHOP & SOPHIE FORSTER
cross-group studies of normative brain func- voxel and genetic polymorphism array
tion and between-group studies of specific size?
affective disorders far outnumber studies
using continuous trait measures to study the
brain basis of vulnerability to affective dis-
order. These latter studies face a number of References
challenges. In addition to the need for well-
validated trait measures of affective style and Amin, Z., Constable, R. T., & Canli, T. (2004).
Attentional bias for valenced stimuli as a
vulnerability, rigor in neuroimaging design
function of personality in the dot-probe task.
and analysis is likely to be an important
Journal of Research in Personality, 38, 15–23.
determinant of progress in this field. Excit- Amir, N., Weber, G., Beard, C., Bomyea, J., &
ing advances are being made in the meth- Taylor, C. T. (2008). The effect of a single-
ods and techniques available, both within session attention modification program on
neuroimaging and converging approaches. response to a public-speaking challenge in
By incorporating these advances and by socially anxious individuals. Journal of Abnor-
drawing on models derived from both the mal Psychology, 117(4), 860–68.
human cognitive and basic neuroscience lit- Anderson, A. K., Christoff, K., Panitz, D., De
eratures, it will be possible to test increas- Rosa, E., & Gabrieli, J.D. (2003). Neural cor-
ingly sophisticated hypotheses regarding the relates of the automatic processing of threat
facial signals. Journal of Neuroscience, 23(13),
brain basis of vulnerability to affective
5627–33.
disorder.
Ansari, T. L., & Derakshan, N. (2011). The neu-
ral correlates of impaired inhibitory control in
anxiety. Neuropsychologia,49(5),1146–53.
Outstanding Questions and Future
Arnsten, A. F. T. (2009). Stress signaling path-
Directions ways that impair prefrontal cortex structure
r How do different dimensions of personal- and function. Nature Reviews Neuroscience, 10,
410–22.
ity relate to vulnerability to affective dis- Arnsten, A. F. T., Mathew, R., Ubriani, R., Tay-
order? lor, J. R., & Li, B.-M. (1999). α-1 noradrenergic
r Are there multiple “pathways” by which receptor stimulation impairs prefrontal cor-
dysregulation of amygdala/frontal cir- tical cognitive function. Biological Psychiatry,
cuitry confers vulnerability to affective 45, 26–31.
disorder? Barrett, J., & Armony, J. L. (2009). Influence of
r To what extent does this dysregulation trait anxiety on brain activity during the acqui-
sition and extinction of aversive conditioning.
reflect genetic influences, the effects of
Psychological Medicine, 39(2), 255–65.
early life stress on gene expression, or the
Beck, A. X, Ward, C. H., Mendelson, M., Mock,
direct effect of chronic or acute stress on J., & Erbaugh, J. (1961). An inventory for mea-
this circuitry? suring depression. Archives of General Psychi-
r On a methodological front, how can we
atry, 4, 561–71.
best balance hypothesis-driven research Bieling, P. J., Antony, M. M., & Swinson, R. P.
with exploratory investigations? Within (1998). The state–trait anxiety inventory, trait
the area of neuroimaging, what are the version: Structure and content re-examined.
respective limitations of different meth- Behaviour Research and Therapy, 36,
ods of data acquisition and styles of anal- 777–88.
ysis (e.g., region-of-interest approaches Bishop, S. J. (2008). Neural mechanisms underly-
ing selective attention to threat. Annals of the
versus whole-brain analyses)?
r If we seek to understand genetic influ- New York Academy of Sciences, 1129, 141–52.
Bishop, S. J. (2009) Trait anxiety and impover-
ences on brain mechanisms implicated in ished prefrontal control of attention. Nature
vulnerability to affective disorder, how Neuroscience, 12, 92–98.
can we deal with the multiple com- Bishop, S. J., Duncan, J., Brett, M., &
parisons problem due to both brain Lawrence, A. D. (2004). Prefrontal cortical
570 SONIA BISHOP & SOPHIE FORSTER
function and anxiety: Controlling attention to Canli, T. (2004). Functional brain mapping of
threat-related stimuli. Nature Neuroscience, extraversion and neuroticism: Learning from
7(2),184–88. individual differences in emotion processing.
Bishop, S. J., Duncan, J., & Lawrence, A. (2004b). Journal of Personality, 72, 1105–32.
State anxiety modulation of the amygdala Canli, T., Amin, Z., Haas, B., Omura, K., &
response to unattended threat-related stimuli. Constable, R. T. (2004). A double dissociation
Journal of Neuroscience, 24, 10364–68. between mood states and personality traits
Bishop, S. J., Jenkins, R., & Lawrence, A. (2007). in the anterior cingulate. Behavioral Neuro-
The neural processing of task-irrelevant fear- science, 18, 897–904.
ful faces: Effects of perceptual load and indi- Canli, T., Sivers, H., Whitfield, S. L., Gotlib,
vidual differences in trait and state anxiety. I. H., & Gabrieli, J. D. (2002). Amygdala
Cerebral Cortex, 17, 1595–1603. response to happy faces as a function of
Botvinick, M. M., Cohen, D. D., & Carter, C. S. extraversion. Science, 296, 2191.
(2004). Conflict monitoring and anterior cin- Canli, T., Zhao, Z., Desmond, J. E., Kang, E.,
gulated cortex: An update. Trends in Cognitive Gross, J., & Gabrieli, J. D. E. (2001). An fMRI
Sciences, 12, 539–46. study of personality influences on brain reac-
Bradley, B. P., Mogg, K., Falla, S. J., & Hamilton, tivity to emotional stimuli. Behavioral Neu-
L. R. (1998). Attentional bias for threatening roscence, 115, 33–42.
facial expressions in anxiety: Manipulation of Carter, C. S., Macdonald, A. M., Botvinick,
stimulus duration. Cognition Emotion, 12, 737– M., Ross, L. L., Stenger, V. A., Noll, D.,
53. & Cohen, J. D. (2000). Parsing executive
Britton, J. C., Ho, S. H., Taylor, S. F., & Liberzon, processes: Strategic vs. evaluative functions
I. (2007). Neuroticism associated with neural of the anterior cingulated cortex. Proceed-
activation patterns to positive stimuli. Psychi- ings of the National Academy of Sciences, 97,
atric Research – Neuroimaging, 156(3), 263–67. 1944–48.
Brown, T. A. (2007). Temporal course and struc- Chan, S. W., Goodwin, G. M., & Harmer,
tural relationships among dimensions of tem- C. J. (2007). Highly neurotic never-depressed
perament and DSM-IV anxiety and mood students have negative biases in informa-
disorder constructs. Journal of Abnormal Psy- tion processing. Psychological Medicine, 37,
chology, 116, 313–28. 1281–91.
Brown, T. A., & Barlow, D. H. (2009). A proposal Chan, S. W. Y., Norbury, R., Goodwin, G. M., &
for a dimensional classification system based Harmer C. J. (2009). Risk for depression and
on the shared features of the DSM-IV anxiety neural responses to fearful facial expressions
and mood disorders: Implications for assess- of emotion. British Journal of Psychiatry, 194,
ment and treatment. Psychological Assessment, 139–45.
21(3), 256–71. Clark, L. A., & Watson D. (1991). Tripartite
Brown, T. A., & Rosellini, A. J. (2011). The direct model of anxiety and depression: Psychomet-
and interactive effects of neuroticism and life ric evidence and taxonomic implications. Jour-
stress on the severity and longitudinal course nal of Abnormal Psychology, 100(3), 16–36.
of depressive symptoms. Journal of Abnormal Costa, P. T, Jr., & McCrae, R. R. (1995). Domains
Psychology, 120(4), 844–56 and facets: Hierarchical personality assess-
Bush, G. (2010). Attention-deficit/hyperactivity ment using the Revised NEO Personality
disorder and attention networks. Neuropsy- Inventory. Journal of Personality Assessment,
chopharmacology, 35, 278–300. 64, 21–50.
Bush, G., Frazier, J. A., Rauch, S. L., Seidman, Costa, P., & McCrae, R. R. (1992). Normal
L .J., Whalen, P. J., Jenike, M. A., & Bie- personality assessment in clinical practice:
derman, J. (1999). Anterior cingulate cortex The NEO Personality Inventory. Psychological
dysfunction in attention-deficit/hyperactivity Assessment, 4, 5–13.
disorder revealed by fMRI and the Counting Cremers, H. R., Demenescu, L. R., Aleman, A.,
Stroop. Biological Psychiatry, 45(12), 1542–52. Renken, R., van Tol, M.J., van der Wee, N. J.,
Bush, G., Luu, P., & Posner, M. I. (2000). Cogni- & Roelofs, K. (2010). Neuroticism modulates
tive and emotional influences in anterior cin- amygdala-prefrontal connectivity in response
gulated cortex. Trends in Cognitive Sciences, to negative emotional facial expressions. Neu-
4(6), 215–22. roimage, 49, 963–70.
TRAIT ANXIETY, NEUROTICISM, AND THE BRAIN BASIS OF VULNERABILITY TO AFFECTIVE DISORDER 571
Davidson, R. J. (2002). Anxiety and affective Eysenck, M. W., Derakshan, N., Santos, R.,
style: Role of prefrontal cortex and amygdala. & Calvo, M. G. (2007). Anxiety and cogni-
Biological Psychiatry, 51(1), 68–80. tive performance: Attentional control theory.
Debiec, J., & LeDoux, J. E. (2006). Noradrenergic Emotion, 7, 336–53.
signaling in the amygdala contributes to the Francis, D. D., Champagne, F. A., Liu, D., &
reconsolidation of fear memory: Treatment Meaney, M. J. (1999). Maternal care, gene
implications for PTSD. Annals of the New York expression, and the development of individ-
Academy of Sciences, 1071, 521–24. ual differences in stress reactivity. Annals
Deckersbach, T., Miller, K. K., Klibanski, A., of the New York Academy of Sciences, 896,
Fischman, A., Dougherty, D. D., Blais, M. 66–84.
A., & Rauch, S. L. (2006). Regional cere- Gazzaniga, M. S. Ivry, R. B., & Mangun, G. R.
bral brain metabolism correlates of neuroti- (2009). Emotion. In M.S. Gazzaniga, R. B.
cism and extraversion. Depression and Anxi- Ivry, & G. R. Mangun (Eds.), Cognitive neu-
ety, 23(3), 133–38. roscience – the biology of the mind (3rd ed., pp.
Deco, G., Jirsa, V. K., & McIntosh, A. R. (2011). 364–281). New York: Norton.
Emerging concepts for the dynamical orga- Goldstein, L. E., Rasmusson, A. M., Bunney,
nization of resting-state activity in the brain. S. B., & Roth, R. H. (1996). Role of the
Nature Reviews Neuroscience, 12(1), 43–56. amygdala in the coordination of behav-
Delgado, M. R., Nearing, K. I., Ledoux, J. E., & ioral, neuroendocrine and prefrontal cortical
Phelps, E. A. (2008). Neural circuitry underly- monoamine responses to psychological stress
ing the regulation of conditioned fear and its in the rat. Journal of Neuroscience, 16, 4787–98.
relation to extinction. Neuron., 59(5), 829–38. Gotlib, I. H., MacLachlan, A., & Katz, A. (1988).
Derryberry, D., & Reed, M .A. (2002). Anxiety- Biases in visual attention in depressed and
related attentional biases and their regulation nondepressed individuals. Cognition Emotion,
by attentional control. Journal of Abnormal 2, 185–200.
Psychology, 111, 225–36. Gotlib, I. H., & McCann, C. D. (1984). Construct
Desimone, R., & Duncan, J. (1995). Neural mech- accessibility and depression: An examination
anisms of selective attention. Annual Review of cognitive and affective factors, Journal of
of Neuroscience, 18, 193–222. Personality and Social Psychology, 47, 427–39.
Eisenberger, N. I., Lieberman, M. D., & Satpute, Haas, B. W., Omura, K., Constable, R. T., &
A. B. (2005). Personality from a controlled Canli, T. (2007). Emotional conflict and neu-
processing perspective: An fMRI study of neu- roticism: Personality-dependent activation in
roticism, extraversion, and self-consciousness. the amygdala and subgenual anterior cingu-
Cognitive Affective & Behavioral Neuroscience, late. Behavioral Neuroscience, 121, 249–56.
5, 169–81. Hakamata, Y., Lissek, S., Bar-Haim, Y., Britton,
Etkin, A., Egner, T., Peraza, D. M., Kandel, E. R., J. C., Fox, N. A., Leibenluft, E., & Pine, D. S.
& Hirsch, J. (2006). Resolving emotional con- (2010). Attention bias modification treatment:
flict: A role for the rostral anterior cingulate A meta-analysis toward the establishment of
cortex in modulating activity in the amygdala. novel treatment for anxiety. Biological Psychi-
Neuron, 51, 871–82. atry, 68(11), 982–90.
Etkin, A., Klemenhagen, K. C., Dudman, J. T., Hettema, J. M., An, S. S., Bukszar, J., van
Rogan, M. T., Hen, R., Kandel, E. R., & den Oord, E. J., Neale, M. C., Kendler,
Hirsch, J. (2004). Individual differences in trait K. S., & Chen, X. (2008). Catechol-O-
anxiety predict the response of the basolateral methyltransferase contributes to genetic sus-
amygdala to unconsciously processed fearful ceptibility shared among anxiety spectrum
faces. Neuron, 44(6), 1043–55. phenotypes. Biological Psychiatry, 64(4), 302–
Etkin, A., & Wager, T. D. (2007) Functional neu- 10.
roimaging of anxiety: A meta-analysis of emo- Hill, A. B., & Dutton, F. (1989). Depression and
tional processing in PTSD, social anxiety dis- selective attention to self-esteem threatening
order, and specific phobia. American Journal words. Personality and Individual Differences,
of Psychiatry, 164(10), 1476–88. 10, 915–17.
Eysenck, H. J., & Eysenck, S. B. G. (1975). Eysenck Hill, A. B., & Knowles, T. H. (1991). Depression
Personality Questionnaire manual. San Diego: and the emotional Stroop effect. Personality
Educational and Industrial Testing Service. and Individual Differences, 12, 481–85.
572 SONIA BISHOP & SOPHIE FORSTER
Holmes, A., & Wellman, C. L (2009). Stress- LeDoux, J. E. (2000). Emotion circuits in the
induced prefrontal reorganization and execu- brain. Annual Review of Neuroscience, 23, 155–
tive dysfunction in rodents. Neuroscience and 84.
Biobehavioral Reviews, 33, 773–83. Luteijn, F., & Bouman, T. K. (1988). The con-
Huang-Pollock, C. L., Carr, T. H., & Nigg, J. cepts of depression, anxiety, and neuroticism
T. (2002). Development of selective attention: in questionnaires. European Journal of Person-
Perceptual load influences early versus late ality, 2, 113–20.
attentional selection in children and adults. MacDonald, A. W., Cohen, J. D., Stenger, V. A.,
Developmental Psychology, 38, 363–75. & Carter, C. S. (2000). Dissociating the role
Hyde, L. W., Bogdan, R., & Hariri, A. R. of dorsolateral prefrontal cortex and anterior
(2011). Understanding risk for psychopathol- cingulate cortex in cognitive control. Science,
ogy through imaging gene-environment inter- 288, 1835–38.
actions. Trends in Cognitive Sciences, 15(9), 417– MacLean, P. D. (1949). Psychosomatic disease
27. and the “visceral brain”: Recent developments
Indovina, I., Robbins, T. W., Núñez-Elizalde, A. bearing on the Papez theory of emotion. Psy-
O., Dunn, B. D., Bishop, S. J. (2011). Fear- chosomatic Medicine, 11, 338–53
conditioning mechanisms associated with trait MacLeod, C., & Hagan, R. (1992). Individual dif-
vulnerability to anxiety in humans. Neuron, ferences in the selective processing of threat-
69(3), 563–71. ening information, and emotional responses to
John, O. P. (1990). The “Big Five” factor taxon- a stressful life event. Behaviour Research and
omy: Dimensions of personality in the natu- Therapy , 30, 151–61.
ral language and in questionnaires. In L. A. MacLeod, C., & Mathews, A. (1988). Anxiety and
Pervin (Ed.), Handbook of personality theory the allocation of attention to threat. Quarterly
and research (pp. 66–100). New York: Guil- Journal of Experimental Psychology, 40, 653–70.
ford Press. Maren, S., & Quirk, G. J. (2004). Neuronal sig-
Kastner, S., & Ungerleider, L.G. (2000). Mecha- naling of fear memory. Nature Reviews Neu-
nisms of visual attention in the human cortex. roscience, 5, 844–52.
Annual Review of Neuroscience, 23, 315–41. Mathews, A., & Mackintosh, B. (1998). A cogni-
Kendler, K. S., Gardner, C. O., Gatz, M., & tive model of selective processing in anxiety.
Pedersen, N. L. (2007). The sources of co- Cognitive Therapy Research, 22, 539–60.
morbidity between major depression and gen- Mathews, A., Ridgeway, V., & Williamson, D.
eralized anxiety disorder in a Swedish national A., (1996). Evidence for attention to threaten-
twin sample. Psychological Medicine, 37(3), ing stimuli in depression. Behaviour Research
453–62. and Therapy, 34, 695–705.
Kim, M. J., Gee, D. G., Loucks, R. A., Davis, Mayberg, H. S., Liotti, M., Brannan, S. K.,
F. C., & Whalen, P. J. (2011). Anxiety dissoci- McGinnis, S., Mahurin, R. K., Jerabek, P.
ates dorsal and ventral medial prefrontal cor- A., & Fox, P. T. (1999). Reciprocal limbic-
tex functional connectivity with the amygdala cortical function and negative mood: Con-
at rest. Cerebral Cortex, 21(7), 1667–73. verging PET findings in depression and normal
Kim, H., Somerville, L. H., Johnstone, T., sadness. American Journal of Psychiatry, 156,
Alexander, A. L., & Whalen, P. J. (2003). 675–82.
Inverse amygdala and medial prefrontal cor- Maylor, E., & Lavie, N. (1998). The influence of
tex responses to surprised faces. Neuroreport, perceptual load on age differences in selective
14(18), 2317–22. attention. Psychology and Aging, 13, 563–73.
Kim, M. J., & Whalen, P. J. (2009). The structural Milad, M. R., Pitman, R. K., Ellis, C. B., Gold,
integrity of an amygdala-prefrontal pathway A. L., Shin, L. M., Lasko, N. B., & Rauch,
predicts trait anxiety. Journal of Neuroscience, S. L. (2009). Neurobiological basis of failure
29(37), 11614–18. to recall extinction memory in posttraumatic
Kosslyn, S. M., Cacioppo, J. T., Davidson, R.J., stress disorder. Biological Psychiatry, 66(12),
Hugdahl, K., Lovallo, W. R., Spiegel, D., & 1075–82.
Rose, R.(2002). Bridging psychology and biol- Milad, M. R., Quirk, G. J., Pitman, R. K., Orr,
ogy. American Psychologist, 57, 341–51. S. P., Fischl, B., & Rauch, S.L. (2007). A role
Lavie, N. (2005). Distracted and confused?: Selec- for the human dorsal anterior cingulate cortex
tive attention under load. Trends in Cognitive in fear expression. Biological Psychiatry, 62(10),
Sciences, 9, 75–82. 1191–94.
TRAIT ANXIETY, NEUROTICISM, AND THE BRAIN BASIS OF VULNERABILITY TO AFFECTIVE DISORDER 573
Milad, M. R, Wright, C. I., Orr, S. P., Pitman, R. disorders: From the laboratory to the clinic.
K., Quirk, G. J., & Rauch, S. L. (2007). Recall Nature Neuroscience, 10, 1116–24.
of fear extinction in humans activates the ven- Richards, A., & Millwood, B. (1989). Colour-
tromedial prefrontal cortex and hippocam- identification of differentially valenced words
pus in concert. Biological Psychiatry, 62(5), in anxiety. Cognition Emotion, 3, 171–76.
446–54. Rijsdijk, F. V., Riese, H., Tops, M., Snieder, H.,
Mogg, K., & Bradley, B. P. (1998). A cognitive- Brouwer, W. H, Smid, H. G. O. M., & Ormel,
motivational analysis of anxiety. Behaviour J. (2009). Neuroticism, recall bias and atten-
Research and Therapy, 36, 809–48. tion bias for valenced probes: A twin study.
Morris, J. S., Frith, C. D., Perrett, D. I., Rowland, Psychological Medicine, 39(1), 45–54.
D., Young, A. W., Calder, A. J., & Dolan, R. Roy, A .K., Shehzad, Z., Margulies, D. S., Kelly,
J. (1996). A differential neural response in the A. M., Uddin, L. Q., Gotimer, K., Biswal, B.
human amygdala to fearful and happy facial B., & Milham, M. P. (2009). Functional con-
expressions. Nature, 383(6603), 812–15. nectivity of the human amygdala using resting
Nomura, M., Iidaka, T., Kakehi, K., Tsukiura, state fMRI. Neuroimage, 45(2), 614–26.
T., Hasegawa, T., Maeda, Y., & Matsue, Y. Sehlmeyer, C., Dannlowski, U., SchÃning, S.,
(2003). Frontal lobe networks for effective pro- Kugel, H., Pyka, M., Pfleiderer, B., Zwitser-
cessing of ambiguously expressed emotions in lood, P., & Konrad, C. (2011). Neural correlates
humans. Neuroscience Letters, 348(2), 113–16. of trait anxiety in fear extinction. Psychological
Ochsner, K. N., Bunge, S. A., Gross, J. J., & Medicine, 41(4), 789–98.
Gabrieli, J. D. (2002). Rethinking feelings: Shackman, A. J., Salomons, T. V., Slagter, H.
An FMRI study of the cognitive regulation A., Fox, A. S., Winter, J. J., & Davidson, R.
of emotion. Journal of Cognitive Neuroscience, J. (2011). The integration of negative affect,
14(8), 1215–29. pain and cognitive control in the cingulate
Papez, J. W. (1937). A proposed mechanism of cortex. Nature Reviews Neuroscience, 12(3),
emotion. Archives of Neurology and Psychiatry, 154–67.
38, 725–74. Shin, L .M., & Liberzon, I. (2010). The neuro-
Pessoa, L., & Adolphs, R. (2010). Emotion pro- circuitry of fear, stress, and anxiety disorders.
cessing and the amygdala: From a “low road” Neuropsychopharmacology, 35(1), 169–91.
to “many roads” of evaluating biological sig- Shin, L. M., Whalen, P. J., Pitman, R. K., Bush,
nificance. Nature Reviews Neuroscience, 11(11), G., Macklin, M. L., Lasko, N. B., Orr, S.
773–83. P., & Rauch, S. L. (2001). An fMRI study of
Pessoa, L., McKenna, M., Gutierrez, E., & Unger- anterior cingulate function in posttraumatic
leider, L.G. (2002). Neural processing of emo- stress disorder. Biological Psychiatry, 50(12),
tional faces requires attention. Proceedings 932–42.
of the National Academy of Sciences, 99(17), Sotres-Boyen, F., Bush, D. E. A., & LeDoux, J. E.
11458–63. (2004). Emotional perseveration: An update
Phelps, E. A., Delgado, M.R., Nearing, K. I., & on prefrontal–amygdala interactions in fear
LeDoux, J. E. (2004). Extinction learning in extinction. Learning and Memory, 11, 525–35.
humans: Role of the amygdala and vmPFC. Spielberger C. D., Gorsuch R. L., Lushene P. R.,
Neuron, 43(6), 897–905. Vagg P. R., & Jacobs, A. G. (1983). Manual
Radloff, L. S. (1977). The CES-D Scale: A self- for the State-Trait Anxiety Inventory (Form Y).
report depression scale for research in the Palo Alto, CA: Consulting Psychologists Press.
general population. Applied Psychological Mea- Stein, M. B. (2009). Neurobiology of generalized
surement, 1, 385–401. anxiety disorder. Journal of Clinical Psychiatry,
Reed, M. A., & Derryberry, D. (1995). Tempera- 70(Suppl. 2), 15–19.
ment and attention to positive and negative Stein, M. B., Simmons, A. N., Feinstein, J. S.,
trait information. Personality and Individual & Paulus, M. P. (2007). Increased amygdala
Differences, 18, 135–47. and insula activation during emotion process-
Rees, G, Frith, C. D., & Lavie, N. (1997). Modu- ing in anxiety-prone subjects. American Jour-
lating irrelevant motion perception by varying nal of Psychiatry, 164(2), 318–27.
attentional load in an unrelated task. Science, Tamietto, M., & de Gelder, B. (2010). Neural
278(5343), 1616–19. bases of the non-conscious perception of emo-
Ressler, K. J., & Mayberg, H.S. (2007). Targeting tional signals. Nature Reviews Neurosciences,
abnormal neural circuits in mood and anxiety 11(10), 697–709.
574 SONIA BISHOP & SOPHIE FORSTER
Taylor, J. A. (1953). A personality scale of man- aversive emotional states. Psychological Bul-
ifest anxiety. Journal of Abnormal and Social letin, 96, 465–90.
Psychology, 48, 285–90. Watson, D., & Clark, L. A. (1991). The Mood and
Underwood, B. J. (1975). Individual differences Anxiety Symptom Questionnaire. Iowa City:
as a crucible in theory construction. American University of Iowa.
Psychologist, 30, 128–34. Weems, C. F., Pina, A. A., Costa, N. M., Watts, S.
Vrtička, P., Sander, D., & Vuilleumier, P. (2011). E., Taylor, L. K., & Cannon, M. F. (2007). Pre-
Effects of emotion regulation strategy on brain disaster trait anxiety and negative affect pre-
responses to the valence and social content of dict posttraumatic stress in youth after Hurri-
visual scenes. Neuropsychologia, 49(5), 1067– cane Katrina. Journal of Consulting and Clini-
82. cal Psychology, 75, 154–59.
Vuilleumier, P., Armony, J. L, Driver, J., & Whalen, P. J. (2007). The uncertainty of it all.
Dolan, R. J. (2001). Effects of attention and Trends in Cognitive Sciences, 11(12), 499–500.
emotion on face processing in the human Whalen, P. J., Kagan, J, Cook, R. G., Davis, F. C.,
brain: An event-related fMRI study. Neuron, Kim, H., Polis, S., McLaren, D. G., & John-
30, 829–41. stone, T. (2004). Human amygdala respon-
Vul, E., Harris, C., Winkielman, P., & Pashler, H. sivity to masked fearful eye whites. Science,
(2009). Puzzlingly high correlations in fMRI 306(5704), 2061.
studies of emotion, personality, and social cog- Whalen, P. J., Rauch, S. L., Etcoff, N. L., McIn-
nition. Perspectives on Psychological Science, 4, erney, S. C., Lee, M. B., & Jenike, M. A
274–290. (1998). Masked presentations of emotional
Vyas, A., Mitra, R., Shankaranarayana Rao, facial expressions modulate amygdala activity
B. S., & Chattarji, S. (2002). Chronic without explicit knowledge. Journal of Neuro-
stress induces contrasting patterns of den- science, 18(1), 411–18.
dritic remodeling in hippocampal and amyg- Williams, J. M. G., & Nulty, D.D. (1986). Con-
daloid neurons. Journal of Neuroscience, 22, struct accessibility, depression and the emo-
6810–18. tional Stroop task: Transient mood or stable
Watson, D., & Clark, L. A. (1984). Nega- structure? Personality and Individual Differ-
tive affectivity: The disposition to experience ences, 7, 485–491.
CHAPTER 25
Ahmad R. Hariri
Individual differences in trait affect, per- Chapter 5). Thus, much like their behav-
sonality, and temperament are critical in ioral counterparts, patterns of brain activa-
shaping complex human behaviors, success- tion represent enduring, trait-like phenom-
fully navigating social interactions, and over- ena that in and of themselves may serve as
coming challenges from our ever changing important markers of individual differences
environments. Such individual differences as well as disease liability and pathophysiol-
may also serve as important predictors of ogy.
vulnerability to neuropsychiatric disorders As neuroimaging studies continue to il-
including depression, anxiety, and addic- lustrate the predictive relationship between
tion, especially after exposure to environ- regional brain activation and trait-like be-
mental adversity. Accordingly, identifying haviors (e.g., increased amygdala reactiv-
the biological mechanisms that give rise ity predicts trait anxiety; see Chapter 24),
to trait individual differences affords an an important next step is to systemati-
unique opportunity to develop a deeper cally identify the underlying mechanisms
understanding of complex human behav- driving variability in brain circuit function.
iors, disease liability, and treatment. Having In this regard, recent neuroimaging stud-
established multiple modal neural processes ies employing pharmacological challenge
supporting specific aspects of complex paradigms, principally targeting monoamine
behavioral processes, human neuroimag- neurotransmission, have revealed that even
ing studies – especially those employ- subtle alterations in dopaminergic, nora-
ing blood-oxygen-level-dependent (BOLD) drenergic, and serotonergic signaling can
fMRI – have now begun to reveal the neu- have a profound impact on the functional
ral substrates of interindividual variability response of brain circuitries supporting
in these and related constructs. Moreover, affect, personality, and temperament. Simi-
recent studies have established that BOLD larly, multimodal neuroimaging approaches
fMRI measures represent temporally stable have provided evidence for directionally spe-
and reliable indices of brain function (see cific relationships between key components of
575
576 AHMAD R. HARIRI
Individuals with high trait anxiety exhibit happy facial expressions (Stein, Simmons,
a propensity to more frequently appraise Feinstein, & Paulus, 2007). Consistent with
situations as more threatening than do this pattern of normal variability, various
others and are generally more sensitive to mood and anxiety disorders (e.g., unipolar
social cues, including those representing and bipolar depression, generalized anxiety
both explicit and implicit threat (e.g., disorder, social phobia) have been linked
angry and fearful facial expressions). In with greater amygdala responses to facial
turn, these individuals are at increased risk expressions depicting fear and anger, as well
for developing neuropsychiatric disorders as sadness and disgust, and, more variably,
characterized by abnormal social and to emotionally neutral facial expressions.
emotional behaviors such as depression and Such findings demonstrate that anxiety-
that are often precipitated by exposure related psychopathology is associated with
to chronic or severe stressors. Examining a heightened amygdala response to diverse
the neural correlates of individual vari- affective stimuli. More importantly, in the
ability in dispositional temperament such absence of such disorders, variability in
as trait anxiety represents an important the magnitude of threat-related amygdala
step in understanding key socioemotional reactivity is an important predictor of
behaviors, as well as an effective means of individual differences in trait anxiety.
elucidating pathophysiological processes Having first established a predictive link
contributing to related disordered states. between amygdala reactivity and trait anx-
Converging evidence from animal and iety, factors that drive such behaviorally
human studies clearly demonstrates that relevant variability in brain function can
the amygdala is centrally involved in medi- be now be identified in the broader con-
ating both physiological (e.g., autonomic text of detailing the biological mechanisms
reactivity) and behavioral (e.g., realloca- mediating individual differences in temper-
tion of attentional resources) effects that amental anxiety. Converging preclinical and
allow an individual to respond adaptively to clinical evidence indicates that amygdala
varied environmental and social challenges. functioning is sensitive to the effects of
A large corpus of human neuroimaging central serotonin, whose principal forebrain
research reveals that the amygdala is innervation is provided by the midbrain
robustly engaged by varied biologically dorsal raphe nuclei (DRN). Available data
salient stimuli, most notably emotional from animal studies indicate that relative
facial expressions representing threat (see increases in local 5-HT result in potentia-
Chapter 7). However, individuals differ tion of amygdala activation and associated
appreciably in the magnitude of amyg- behavioral phenomena, such as fear con-
dala activation on exposure to emotionally ditioning (Amat, Matus-Amat, Watkins, &
expressive facial expressions, and these indi- Maier, 1998; Amat et al. 2004; Burghardt,
vidual differences appear to be stable over Sullivan, McEwen, Gorman, & LeDoux,
time (David et al., 2005; Manuck, Brown, 2004; Forster et al., 2006; Maier & Watkins,
Forbes, & Hariri, 2007). Thus, they may con- 2005).
tribute to the emergence of stable differ- As presented in the introduction of
ences in temperament such as trait anxiety. this chapter, recent neuroimaging studies
Recent neuroimaging studies have using multimodal PET/fMRI or pharmaco-
reported positive relationships between logical challenge BOLD fMRI have pro-
the magnitude of amygdala reactivity to vided direct evidence for parallel effects
affective, especially threatening, stimuli and of 5-HT in humans. Specifically, in vivo
interindividual variability in indices of trait PET has revealed that decreased endoge-
and state anxiety. In one study, Stein and nous capacity for local 5-HT reuptake
colleagues reported that high trait anxiety is (Rhodes et al., 2007) is associated with rel-
associated with greater amygdala reactivity atively increased amygdala reactivity. Acute
not only to angry and fearful but also to IV administration of a selective serotonin
MAPPING NEUROGENETIC MECHANISMS OF INDIVIDUAL DIFFERENCES IN AFFECT 579
reuptake inhibitor, which reduces the ca- carrying the -1019G allele (Parsey et al.,
pacity for 5-HT reuptake, during BOLD 2006). However, a similar effect was not
fMRI is likewise associated not only with observed in an earlier PET study (David
increased amygdala reactivity but also with et al., 2005). Regardless, the in vitro effects
decreased habituation of amygdala reactiv- of the HTR1A -1019G allele and the more
ity over time (Bigos et al., 2008). These data general relationship documented between
clearly indicate that variability in the reg- increased 5-HT1A autoreceptor density and
ulation of 5-HT signaling is an important decreased amygdala reactivity (Fisher et
source of individual differences in amygdala al., 2006) suggest that this common func-
reactivity. tional genetic variation may contribute sig-
Crucial among components regulating nificantly to the emergence of interindivid-
5-HT neurotransmission and its subsequent ual variability in serotonin signaling that, in
modulation of brain function is the activa- turn, biases amygdala reactivity.
tion of somatodendritic 5-HT1A autorecep- Consistent with the existing data (i.e.,
tors, which mediate negative feedback on increased 5-HT1A autoreceptors leading to
DRN neurons resulting in decreased 5-HT increased negative feedback inhibition of
release at postsynaptic targets in the fore- DRN and decreased 5-HT release), we
brain. Using multimodal PET/fMRI, we pre- recently demonstrated that the HTR1A -
viously reported that the density of 5-HT1A 1019G allele is associated with significantly
autoreceptors accounts for 30–44% of the decreased threat-related amygdala reactiv-
variability in amygdala reactivity in healthy ity (Fakra et al., 2009). In addition, we found
adults (Fisher et al., 2006), confirming the that HTR1A genotype effects on trait anxiety
important role of 5-HT1A autoreceptors in were mediated through its impact on threat-
modulating the activity of serotonergic tar- related amygdala reactivity, which presum-
get regions. Given the critical role of 5-HT1A ably reflects the genotype’s modulation
autoreceptors in regulating 5-HT signaling of postsynaptic 5-HT release. Specifically,
and its resulting influence on the functioning although path models revealed no signifi-
of major brain targets, such as the amygdala, cant direct genotype effect on trait anxiety,
as well as complex behavioral processes, it they demonstrated that HTR1A C(-1019)G
is important to identify sources of emergent and amygdala reactivity indirectly predicted
variability in 5-HT1A function. a significant proportion (9.2%) of individ-
Common sequence variation in the ual differences in trait anxiety through their
human 5-HT1A gene (HTR1A) represents respective indirect and direct paths. The
one potential source of such interindivid- data from this study are remarkably consis-
ual variability. Recently, a relatively fre- tent with that reported for other common
quent single nucleotide polymorphism, C(- functional polymorphisms that are also asso-
1019)G, in the promoter region of HTR1A ciated with relatively increased 5-HT signal-
was demonstrated to affect transcriptional ing, most notably the 5-HTTLPR short allele
regulation of the gene through altered bind- (Hariri, Mattay, Tessitore, Kolachana, et al.,
ing of the transcription factors. Specifi- 2002; Munafo, Brown, & Hariri, 2008). These
cally, the -1019G allele abolishes or impairs findings represent an important step in this
transcriptional repression of the promoter avenue of research by providing empiri-
and, as a consequence, is associated with cal documentation for the basic premise
increased 5-HT1A expression (Lemonde that genetic variation in neural signaling
et al., 2003), a phenomenon that appears cascades indirectly affects emergent behav-
to be specific to autoreceptors (Czesak, ioral processes by biasing the response of
Lemonde, Peterson, Rogaeva, & Albert, underlying neural circuitries. More gener-
2006). Consistent with this finding, in ally, however, the predictive links among
vivo human PET has revealed specifically genes, amygdala function, and anxiety will
increased 5-HT1A autoreceptor density in require further investigations, including the
both healthy adults and depressed patients elucidation of developmental effects of
580 AHMAD R. HARIRI
genetically driven variability in serotoniner- such behaviors but also the pathophysi-
gic signaling on amygdala reactivity, as well ology of addiction and related disorders.
as patterns of structural and functional con- Through reciprocal cortical and subcorti-
nectivity between the amygdala and regula- cal connections, the nucleus accumbens
tory regions of prefrontal cortex (see Chap- (NAcc) and, more broadly, the ventral stria-
ter 24). tum (VS) contribute to the motivational
salience of stimuli and abet appetitive or
reward-dependent behaviors. Activity of the
Impulsivity, the Ventral Striatum, and VS increases in response to both the antic-
Dopamine ipation and receipt of rewarding stimuli,
including primary (e.g., food) and secondary
Discounting future outcomes underlies (e.g., money) reinforcers (see Chapter 19).
much of human decision making (see Chap- Moreover, in addiction, craving and compul-
ter 17) and figures prominently in several sive drug seeking and sensitivity to drug cues
overlapping psychological constructs, such are associated with dysregulated increases
as self-regulation, impulse-control, delay in VS activity (Kalivas & Volkow, 2005).
of gratification, and intertemporal choice Because the response of the VS involves an
(Manuck, Flory, Muldoon, & Ferrell, 2003). immediate response to rewards, the magni-
Moreover, individuals who strongly prefer tude of VS activity may contribute to indi-
immediate over deferred rewards of larger vidual differences in a relative preference for
nominal value are often generally impul- immediate, compared to delayed, rewards.
sive or lacking in self-control and at risk Using BOLD fMRI, we have demon-
for addictive disorders, such as pathological strated that the magnitude of VS reactiv-
gambling, cigarette smoking, and drug and ity predicts individual differences in a sim-
alcohol abuse (Alessi & Petry, 2003; Bickel, ple laboratory measure of DD (Hariri et al.,
Odum, & Madden, 1999; Kirby, Petry, & 2006). Our analyses revealed that individual
Bickel, 1999; Madden, Petry, Badger, & differences in DD correlate positively with
Bickel, 1997). In experimental research on the magnitude of VS activation in response
intertemporal choice, discounting of future to both positive and negative feedback,
rewards or delay discounting (DD) is a well- as well as with differential reward-related
characterized behavioral measure of prefer- VS activation in response to positive com-
ence for immediate over delayed rewards pared with negative feedback. Consistent
and provides an index of impulsive ten- with the strong general correlation between
dencies in humans. Behavioral tests used to DD and traditional self-report measures of
derive estimates of DD commonly ask par- impulsivity (de Wit, 2009), we have also
ticipants to choose between multiple imme- found that reward-related VS reactivity is
diate rewards that vary in value and a con- positively correlated with scores from the
stant, larger reward available after varying Barratt Impulsiveness Scale (Forbes et al.,
intervals of delay. In such tasks, rates of dis- 2009). Collectively, our results suggest that
counting often differ appreciably and consis- increased self-reported impulsivity and the
tently among individuals. Thus, DD repre- preference for smaller immediate over larger
sents a potentially important psychometric delayed rewards reflect both a relatively
index of individual differences in present- indiscriminate and hyper-reactive VS cir-
versus future-oriented tendencies. cuitry. Similar variability in VS function
Similar to the research on trait anxi- has also been associated with more com-
ety and amygdala reactivity, explication of plex measures of incentive-based decision
the underlying neural processes that give making (Knutson, Rick, Wimmer, Prelec,
rise to such interindividual variability has & Loewenstein, 2007). Moreover, dysregula-
the potential to allow for a more compre- tion of the VS contributes to addiction, per-
hensive understanding of the mechanisms haps by affecting impulsive decision mak-
leading not only to normal variability in ing (Kalivas & Volkow, 2005). As such,
MAPPING NEUROGENETIC MECHANISMS OF INDIVIDUAL DIFFERENCES IN AFFECT 581
VanNess, Owens, & Kilts, 2005). We hypo- ual differences in behavioral attributes (e.g.,
thesized that there would be relatively anxious or impulsive temperament) associ-
greater VS reactivity associated with the ated with an increased risk for psychiatric
9-repeat allele, which is linked with reduced disorders.
DAT expression and presumably greater After their biosynthesis from arachidonic
striatal synaptic DA, in comparison with the acid, eCBs such as anandamide (AEA)
10-repeat allele. Consistent with our hypoth- and 2-arachidonoylglycerol (2-AG), typi-
esis, the DAT1 9-repeat allele was associ- cally modulate synaptic neurotransmission
ated with relatively greater VS reactivity and through stimulation of CB1, the principal
accounted for nearly 12% of the interindivid- CNS cannabinoid receptor widely expressed
ual variability. In contrast, genetic variation on multiple neuronal subtypes and their dis-
directly affecting DA signaling only in the tributed circuitries. In turn, the duration
prefrontal cortex (i.e., COMT Val158Met) and intensity of eCB signaling, especially
was not associated with variability in VS for AEA, are regulated by two complemen-
reactivity. These results highlight an impor- tary mechanisms: enzymatic degradation via
tant role for a genetic polymorphism fatty acid amide hydrolase (Cravatt et al.,
affecting striatal DA neurotransmission in 1996) and active synaptic clearance via the
mediating interindividual differences in AEA transporter (Piomelli et al., 1999). The
reward-related VS reactivity. They further psychotropic and THC-like effects of AEA,
suggest that altered VS reactivity may repre- however, appear to be coupled with fatty
sent a key neurobiological pathway through acid amide hydrolase (FAAH), but not AEA
which these polymorphisms contribute to transporter function (Solinas, Tanda. et al.,
variability in behavioral impulsivity and the 2007). Thus, FAAH, an integral membrane
related risk for substance use disorders. enzyme, may uniquely regulate behaviorally
relevant eCB signaling by mediating the
hydrolytic breakdown of AEA into arachi-
Endocannabinoids, Threat-, and donic acid and ethanolamine.
Reward-Related Brain Function Again, common genetic variation (i.e.,
polymorphisms) affecting the functioning of
Modern neuroscience methodologies have components involved in eCB neurotransmis-
greatly advanced our understanding of the sion (e.g., AEA, CB1, FAAH) may represent
intrinsic mechanisms mediating and reg- a significant potential source of interindivid-
ulating endogenous cannabinoid or endo- ual variability in eCB signaling that medi-
cannabinoid (eCB) signaling in the CNS ates emergent differences in emotion- and
(Piomelli, 2003). Such eCB signaling has reward-related behaviors. Because of its crit-
emerged as a potent modulator of neural cir- ical role in regulating the signaling duration
cuitries mediating both basic physiological and intensity of AEA, and its selective con-
and advanced behavioral responses. Experi- tribution to AEA’s psychotropic effects, we
mental manipulation of these mechanisms examined the neurobiological and behav-
has revealed significant behavioral effects, ioral effects of a common functional nonsyn-
especially in threat- and reward-related onymous single nucleotide polymorphism
domains, which are generally consistent (SNP) resulting in the conversion of a con-
with the effects of Cannabis intoxica- served proline residue to threonine (P129T)
tion, which are largely driven by the con- in the amino acid sequence of FAAH (Hariri
stituent chemical 9 -tetrahydrocannabinol. et al., 2009). In vitro, FAAH 385A is associ-
The elucidation of molecular mechanisms ated with normal catalytic properties, but
regulating eCB signaling, akin to that for reduced cellular expression of FAAH, pos-
serotonin and dopamine, has motivated sibly through enhanced sensitivity to prote-
attempts to understand its possible contri- olytic degradation (Chiang, Gerber, Sipe, &
bution to the emergence of variability in Cravatt, 2004; Sipe, Chiang, Gerber, Beut-
brain circuit function and related individ- ler, & Cravatt, 2002). Moreover, the C385A
MAPPING NEUROGENETIC MECHANISMS OF INDIVIDUAL DIFFERENCES IN AFFECT 583
is the only common mutation in FAAH mon occurrence when working with rel-
(Flanagan, Gerber, Cadet, Beutler, & Sipe, atively small samples; it possibly reflects
2006), and the 385A, which putatively aug- the minimal effect that a proximal biolog-
ments AEA signaling via decreased enzy- ical impact associated with any genotype
matic degradation, has been associated with has on any distal behavioral phenotype,
reward-related pathologies including street as well as the importance of environ-
drug use and problem drug/alcohol abuse, as mental stressors in unmasking genetically
well as being overweight and obese (Flana- driven effects on behavior (Caspi & Moffitt,
gan et al., 2006; Sipe et al., 2002). 2006). However, there were robust differ-
In animal models, both pharmacological ences in the relationships between regional
and genetic disruption of FAAH function brain function and complex behaviors as
result in decreased anxiety-like behaviors, as a function of the FAAH C385A geno-
well as increased consumption and prefer- type. These observed brain-behavior pat-
ence for ethanol (Basavarajappa, Yalaman- terns may reflect the influence of FAAH
chili, Cravatt, Cooper, & Hungund, 2006; C385A-associated differences in endogenous
Blednov, Cravatt, Boehm, Walker, & Harris, eCB tone on stimulus-driven neural cir-
2007; Kathuria et al., 2003; Moreira, Kaiser, cuit function mediating complex behavioral
Monory, & Lutz, 2008; Solinas, Yasar, & processes. Relatively higher levels of AEA
Goldberg, 2007). Moreover, a recent phar- in the amygdala of FAAH 385A carriers
macological fMRI study in human subjects may reduce the responsiveness of this struc-
has reported that acute oral administration ture to salient input (possibly through CB1-
of THC is associated with reduced amygdala mediated potentiation of local GABAergic
reactivity to threat-related facial expressions interneurons) and, as a consequence, lead to
of emotion (Phan et al., 2008). Consistent reduced anxiety-like behaviors predicted by
with these effects, we hypothesized that amygdala function. In contrast, higher lev-
the FAAH 385A would be associated with els of AEA may increase the responsiveness
relatively decreased threat-related amygdala of the VS in FAAH 385A carriers (possibly
reactivity, but increased reward-related reac- through CB1-mediated increased dopamine
tivity in the ventral striatum (VS). Analyses release and potentiation of VS neuron activ-
revealed that carriers of the FAAH 385A, ity), leading to increased reward-sensitivity
associated with reduced enzyme expres- predicted by VS function. Support for this
sion and, presumably, increased AEA signal- speculation exists in studies reporting a fail-
ing, have decreased threat-related amygdala ure of restraint stress to effect changes in
reactivity. In contrast, carriers of the FAAH amygdala activation in knockout mice lack-
385A exhibited increased reward-related VS ing FAAH or animals treated with FAAH
reactivity in comparison to C385 homozy- inhibitors (Patel, Cravatt, & Hillard, 2005),
gotes. Moreover, divergent effects of the and increased food intake as a result of local
FAAH C385A genotype on brain function FAAH inhibition in the nucleus accum-
were manifest in a consistent manner at the bens (Sorice-Gomez et al., 2007). Thus, the
level of brain-behavior relationships. Rel- endogenous state of eCB signaling associ-
ative to C385 homozygotes, FAAH 385A ated with either constitutive genetic varia-
carriers showed a diminished relationship tion such as the FAAH C385A or acute phar-
between amygdala reactivity and trait anxi- macological manipulation likely biases the
ety. In contrast, they exhibited a markedly responsiveness of neural circuits to behav-
increased relationship between VS reactiv- iorally relevant information and their subse-
ity and delay discounting, a behavioral index quent regulation of complex behaviors.
of impulsivity and reward sensitivity. Decreased threat-related amygdala reac-
It is important to note that there were tivity and associated trait anxiety may con-
no direct associations between FAAH geno- tribute to the emergence of pathologies
type and behavioral phenotypes (i.e., anx- such as addiction and obesity, previously
iety or impulsivity) in this study, a com- associated with the FAAH 385A genotype
584 AHMAD R. HARIRI
(Flanagan et al., 2006; Sipe et al., 2002; genetic polymorphisms are now being used
Tyndale, Payne, Gerber, & Sipe, 2007), to understand the biological origins of vari-
by reducing the sensitivity of these indi- ability in component processes of molecular
viduals to potential environmental threat signaling pathways and to efficiently model
or harm. In fact, blunted amygdala reac- how such emergent variability affects behav-
tivity has been reported in individuals at iorally relevant brain function. Such ongoing
high familial risk for alcoholism and has efforts to understand the detailed mecha-
been interpreted as possibly contributing nisms that mediate individual differences in
to decreased threat sensitivity and subse- complex behavioral traits and related neu-
quently increased risk-taking behaviors in ropsychiatric diseases at the level of brain
these genetically predisposed individuals circuit function, molecular signaling path-
(Glahn, Lovallo, & Fox, 2007). An increase ways, and functional genetic polymorphisms
in reward-related VS reactivity and associ- have the potential to inform clinically rele-
ated impulsivity (e.g., steeper discounting vant issues and provide guiding principles
of future relative to immediate rewards) for the development of more effective and
may likewise contribute to disinhibitory psy- individually tailored treatment regimes. In
chopathologies through heightened reward addition, the elucidation of such mecha-
sensitivity and impulsive decision making. nisms, especially those mapped to functional
Studies in addicted patients have generally genetic polymorphisms, can lead to identifi-
reported a sensitization of the neural cir- cation of predictive risk markers that inter-
cuitry for reward, including the VS (Kali- act with unique environmental factors to
vas & Volkow, 2005). And increased behav- precipitate disease.
ioral impulsivity and reward sensitivity are Although the three examples highlighted
significant risk factors for addiction (de Wit in this chapter are evidence for the poten-
& Richards 2004). Thus, through divergent tial of an informed and integrated research
effects on both threat- and reward-related strategy to identify the neurobiology of indi-
brain function, the influence of FAAH vidual differences in complex behavioral
C385A on eCB signaling may have a com- traits and their related clinical endpoints,
pound and accelerated effect on risk for much work remains to be done. First, to
related pathologies. allow for tractable experimental designs and
testable hypotheses in existing samples, the
studies highlighted here have focused on
Conclusions and Future Directions the effects of a single signaling pathway
on behaviorally relevant brain circuitry. Of
As detailed in this chapter, neuroimag- course, it is very clear that are numer-
ing technologies, especially BOLD fMRI, ous complex interactions between signal-
have begun to identify how variability in ing pathways and that more than one path-
neural substrates associated with process- way contributes to the regulation of any
ing specific forms of information contribute brain circuitry. For example, we know that
to emergent individual differences in sta- DA plays an important role in modulating
ble and enduring aspects of human behav- amygdala function and anxiety (Hariri, Mat-
iors such as personality and temperament. tay, Tessitore, Fera, et al., 2002; Tessitore
In parallel, the application of pharmaco- et al., 2002) and that 5-HT can influence
logical fMRI and multimodal PET/fMRI reward-related brain circuitry and impulsiv-
is allowing for an understanding of how ity (Manuck et al., 1998). However, existing
variability in specific molecular signaling studies lack the power and sophistication
pathways influences individual differences to model such complex interactions while
in the function of these behaviorally rele- effectively controlling for other important
vant brain circuitries. Moreover, informa- modulatory factors (e.g., age, gender) in the
tion on DNA sequence variation in humans context of BOLD fMRI, pharmacological
and the related identification of functional MRI, or multimodal PET/fMRI protocols.
MAPPING NEUROGENETIC MECHANISMS OF INDIVIDUAL DIFFERENCES IN AFFECT 585
To do so, we must aggressively expand the ulations with different genetic backgrounds
scale and scope of our studies to include (e.g., Asian).
hundreds and, preferably, thousands of sub- A third important consideration for the
jects. This will afford opportunities to effec- future of this research is the need to conduct
tively examine interactions between signal- large-scale prospective studies beginning in
ing pathways (e.g., 5-HT and DA) on brain childhood to determine any developmental
function and behavior through modeling shifts in neurogenetic pathways mediating
of multiple functional polymorphisms (e.g., individual differences in behavior, as well
HTR1A -1019 and DAT1) and thereby exam- as their predictive utility in identifying neu-
ine the effects of genetically driven variation ropsychiatric disease risk as a function of
in signaling pathways on multiple behav- environmental or other stressors (see also
iorally relevant brain circuitries. Chapter 27). All of the studies described in
A second important consideration is that this chapter and most of the studies avail-
existing studies have been largely conducted able in the literature have been conducted
in ethnically and racially homogeneous pop- in adults carefully screened for the absence
ulations. Thus, the observed effects may not of psychopathology. Therefore, these find-
generalize to other populations. This is espe- ings identify mechanisms contributing to
cially true of studies using functional genetic variability only in the normative range of
polymorphisms because the potential effect behavior. The utility of these markers of
of any single genetic variant on a com- individual differences in behavior – be they
plex biological and behavioral phenotype is neural, molecular, or genetic – in predict-
likely small against the background of the ing vulnerability to neuropsychiatric disor-
approximately 20,000–25,000 human genes der is unclear. Such predictive utility is
and the multitude of other neurobiologically ideally tested through prospective studies
relevant functional variants they likely har- beginning with premorbid populations that
bor. In fact, we have already seen that the account for the moderating effects of envi-
well-replicated effects of a common func- ronmental stress in the emergence of clinical
tional polymorphism affecting 5-HT signal- disorder over time (Caspi & Moffitt, 2006;
ing on amygdala reactivity in Caucasian sub- Viding, Williamson, & Hariri, 2006).
jects may be reversed in those of Asian A fourth issue is the need to further inte-
ancestry (Munafo et al., 2008). Importantly, grate pharmacological challenge protocols
our most recent studies have experimentally with multimodal PET/fMRI to determine
controlled for occult genetic stratification if variability in molecular components of
(i.e., differences in the genetic backgrounds signaling pathways mediates the effects of
of individuals who superficially appear sim- specific neurotransmitters or neuromodula-
ilar), regardless of self-reported race or eth- tors on individual differences in behaviorally
nicity, as well as the independence of the relevant brain circuit function. For exam-
target genotype from other functional poly- ple, despite the remarkable convergence of
morphisms affecting the brain functions findings implicating variability in eCB sig-
under study. Although such efforts allow naling in threat- and reward-related brain
for the attribution of emergent variability function, the exact nature of the down-
in brain and behavior to the candidate vari- stream signaling pathways through which
ant of interest and not to other possible FAAH C385A may modulate neuronal and
polymorphisms or more general differences neural circuit function cannot be deter-
between genotype groups in genetic back- mined from the available results. FAAH cat-
ground, it is important to explicitly test alyzes the hydrolysis of other biologically
the independence of functional polymor- active endogenous fatty acid amides (e.g.,
phisms through rigorous statistical model- oleamide & oleoylethanolamide), which
ing in larger samples and also to test the affect threat- and reward-related behaviors
validity of any associations derived in one independently of AEA. Although, FAAH
sample population (e.g., Caucasian) to pop- has high selectivity for AEA (Desarnaud,
586 AHMAD R. HARIRI
Cadas, & Piomelli, 1995) the effects of FAAH gene) and the production of technologies
C385A cannot be specifically linked to AEA supporting efficient high-throughput iden-
neurotransmission without additional data. tification of such variation in individuals
If the neural and behavioral effects of FAAH have dramatically accelerated the discovery
C385A are mediated by genotype-driven dif- of genes involved in the emergence of com-
ferential availability of AEA, then these plex disease processes (Fellay et al., 2007;
effects should be sensitive to manipulation Link et al., 2008) as well as normal variabil-
of CB1 receptors. An interesting test of this ity in continuous traits (Lettre et al., 2008).
putative mechanism would be to exam- Many of the genes identified in such studies
ine the impact of CB1 antagonists, such as have illuminated novel pathways not previ-
rimonabant, on neural phenotypes associ- ously implicated in these processes or traits,
ated with the FAAH C385A genotype using spurring intensive efforts to understand the
pharmacological fMRI. The availability of a potential biological effects of the proteins
PET radiotracer for CB1 (Burns et al., 2007) produced by these genes.
also allows for the determination of any As such, these “genome-wide” screens
FAAH C385A effects on endogenous recep- represent an opportunity to leap forward
tor concentrations. If this polymorphism beyond the available pool of candidate
biases brain function through AEA stimu- molecules and pathways in parsing the
lation of CB1, then antagonism of the recep- mechanisms of complex biological proces-
tor should eliminate the divergent effects ses. Because neuroimaging-based measures
on amygdala and VS reactivity documented of brain function reveal key mechanisms
here. Any genotype-related alterations in involved in the emergence of individual
AEA concentrations may also be reflected differences in behavioral traits and are closer
in relative up- or down-regulation of CB1 to the biological effects of functional genetic
receptors assayed via PET. If CB1 antago- polymorphisms, they are ideal substrates for
nism fails to abolish the differential effects genome-wide screens. For example, BOLD
of FAAH C385A on brain function or if fMRI estimates of amygdala reactivity pre-
there are no differences in CB1 concentra- dicting variability in temperamental anxiety
tions based on the genotype, then the exist- can be used as the continuous trait in a
ing effects are likely mediated by non-eCB genome-wide screen. Any significant asso-
fatty acid amides. In addition to testing ciations that emerge between genetic vari-
this mechanistic hypothesis with pharma- ation and amygdala reactivity may confirm
cological fMRI and multimodal PET/fMRI, existing relationships (e.g., the importance
future studies with substantially increased of genes biasing 5-HT signaling) or, more
sample sizes can model allele load effects importantly, reveal unexpected candidate
of FAAH 385A, as well as potential FAAH molecules or pathways (e.g., a gene produc-
interactions with functional genetic poly- ing a molecule that is expressed in the brain
morphisms affecting other components of and may function in second-messenger
eCB neurotransmission. signaling cascades). Once identified and,
Finally, there is tremendous potential in ideally, replicated in large-scale databases
developing large databases (again preferably that effectively address confounds common
with thousands of subjects) with detailed to genome-wide screens (e.g., controlling for
measures of behavioral traits, neuroimaging- multiple comparisons resulting from testing
based measures of multiple brain circuitries, the association of a phenotype with hun-
and extensive genotyping. One of the most dreds of thousands or millions of SNPs), the
exciting applications of molecular genetics is impact of variation in novel genes associated
in identifying novel biological pathways con- with amygdala reactivity can be explored at
tributing to the emergence of complex traits. each level of the biological cascade leading
The continued refinement of a detailed to trait anxiety (i.e., be fed back into the
map of sequence variation across the entire discovery loop outlined in Figure 25.1). In
human genome (i.e., SNPs that “tag” every addition to exponentially improving our
MAPPING NEUROGENETIC MECHANISMS OF INDIVIDUAL DIFFERENCES IN AFFECT 587
understanding of neurobiological pathways cortin 2 into the dorsal raphe nucleus activates
leading to individual differences in complex serotonergic neurons and increases extracellu-
behavioral traits, these efforts may lead to lar serotonin in the basolateral amygdala. Neu-
the discovery of novel therapeutic strategies roscience, 129, 509–19.
targeting related disease processes. Bannon, M. J., Michelhaugh, S. K., Wang, J.,
& Sacchetti, P. (2001). The human dopamine
transporter gene: gene organization, transcrip-
tional regulation, and potential involvement
Outstanding Questions and Future in neuropsychiatric disorders. European Neu-
Directions ropsychopharmacology, 11, 449–55.
Basavarajappa, B. S., Yalamanchili, R., Cravatt,
r Are studies finding relationships between B. F., Cooper, T. B., & Hungund, B. L.
(2006). Increased ethanol consumption and
single brain areas (e.g., the amygdala) and
preference and decreased ethanol sensitivity
behavior that are the result of more com-
in female FAAH knockout mice. Neurophar-
plex interactions among multiple brain macology, 50, 834–44.
structures that we do not yet understand? Bickel, W. K., Odum, A. L., & Madden, G. J.
r To what extent do genetic variants affect
(1999). Impulsivity and cigarette smoking:
behavior through their influence on func- Delay discounting in current, never, and ex-
tion versus structure versus connectivity smokers. Psychopharmacology (Berl), 146, 447–
in the brain? 54.
r When and how do most genes of interest Bigos, K. L., Pollock, B. G., Aizenstein, H.,
have their effect on brain and behavior? Fisher, P. M., Bies, R. R., & Hariri, A. R.
r Are there experiences that interact dif- (2008). Acute 5-HT reuptake blockade poten-
tiates human amygdala reactivity. Neuropsy-
ferently with genetic polymorphisms
chopharmacology, 33(13), 3221–25.
depending on when they occur?
r Can intermediate neural phenotypes and, Blednov, Y. A., Cravatt, B. F., Boehm, S. L.,
2nd, Walker, D., & Harris, R. A. (2007). Role
ultimately, the genetic polymorphisms by of endocannabinoids in alcohol consumption
which they are predicted usefully inform and intoxication: studies of mice lacking fatty
diagnosis and treatment? acid amide hydrolase. Neuropsychopharmacol-
ogy, 32, 1570–82.
Brown, S. M., & Hariri, A. R. (2006). Neuroimag-
Acknowledgments ing studies of serotonin gene polymorphisms:
Exploring the interplay of genes, brain, and
This chapter is largely based on Hariri, behavior. Cognitive, Affective, & Behavioral
Neuroscience, 6, 44–52.
Annual Review of Neuroscience, 2009. Some
Burghardt, N. S., Sullivan, G. M., McEwen, B. S.,
material is also based on Hyde, Bogdan, &
Gorman, J. M., & LeDoux, J. E. (2004). The
Hariri, Trends in Cognitive Sciences, 2011. selective serotonin reuptake inhibitor citalo-
pram increases fear after acute treatment but
reduces fear with chronic treatment: A com-
References parison with tianeptine. Biological Psychiatry,
55, 1171–78.
Alessi, S. M., & Petry, N. M. (2003). Pathological Burns, H. D., Van Laere, K., Sanabria-
gambling severity is associated with impulsiv- Bohorquez., S., Hamill, T. G., Bormans, G.,
ity in a delay discounting procedure. Behav- et al. (2007). [18F]MK-9470, a positron emis-
ioral Processes, 64, 345–54. sion tomography (PET) tracer for in vivo
Amat, J., Matus-Amat, P., Watkins, L. R., & human PET brain imaging of the cannabinoid-
Maier, S. F. (1998). Escapable and inescapable 1 receptor. Proceedings of the National Academy
stress differentially alter extracellular levels of of Sciences, 104, 9800–5.
5-HT in the basolateral amygdala of the rat. Caspi, A., & Moffitt, T. E. (2006). Gene-
Brain Research, 812, 113–20. environment interactions in psychiatry: Join-
Amat, J., Tamblyn, J. P., Paul, E. D., Bland, S. T., ing forces with neuroscience. Nature Reviews
Amat, P., et al. (2004). Microinjection of uro- Neuroscience, 7, 583–90.
588 AHMAD R. HARIRI
Cheon, K. A., Ryu, Y. H., Kim, J. W., & HT1A -mediated autoregulation predicts amyg-
Cho, D. Y. (2005). The homozygosity for 10- dala reactivity. Nature Neuroscience, 9. 1362–
repeat allele at dopamine transporter gene 63.
and dopamine transporter density in Korean Flanagan, J. M., Gerber, A. L., Cadet, J. L., Beut-
children with attention deficit hyperactivity ler, E., & Sipe, J. C. (2006). The fatty acid
disorder: Relating to treatment response to amide hydrolase 385 A/A (P129T) variant: hap-
methylphenidate. European Neuropsychophar- lotype analysis of an ancient missense muta-
macology, 15, 95–101. tion and validation of risk for drug addiction.
Chiang, K. P., Gerber, A. L., Sipe, J. C., & Cra- Human Genetics, 120, 581–58.
vatt, B. F. (2004). Reduced cellular expression Forbes, E. E., Brown, S. M., Kimak, M., Fer-
and activity of the P129T mutant of human rell, R. E., Manuck, S. B., & Hariri, A. R.
fatty acid amide hydrolase: Evidence for a link (2009). Genetic variation in components of
between defects in the endocannabinoid sys- dopamine neurotransmission impacts ventral
tem and problem drug use. Human Molecular striatal reactivity associated with impulsivity.
Genetics, 13, 2113–19. Molecular Psychiatry, 14, 60–70.
Cravatt, B. F., Giang, D. K., Mayfield, S. P., Forster, G. L., Feng, N., Watt, M. J., Korzan, W.
Boger, D. L., Lerner, R. A., & Gilula, N. J., Mouw, N. J., et al. (2006). Corticotropin-
B. (1996). Molecular characterization of an releasing factor in the dorsal raphe elicits tem-
enzyme that degrades neuromodulatory fatty- porally distinct serotonergic responses in the
acid amides. Nature, 384, 83–87. limbic system in relation to fear behavior.
Czesak, M., Lemonde, S., Peterson, E. A., Neuroscience, 141, 1047–55.
Rogaeva, A., & Albert, P. R. (2006). Cell- Glahn, D. C., Lovallo, W. R., & Fox, P. T. (2007).
specific repressor or enhancer activities of Reduced amygdala activation in young adults
Deaf-1 at a serotonin 1A receptor gene poly- at high risk of alcoholism: Studies from the
morphism. Journal of Neuroscience, 26, 1864– Oklahoma family health patterns project. Bio-
71. logical Psychiatry, 61, 1306–9.
David, S. P., Murthy, N. V., Rabiner, E. A., Hariri, A. R., Brown, S. M., Williamson, D. E.,
Munafo, M. R., Johnstone, E. C., et al. (2005). Flory, J. D., de Wit, H., & Manuck, S. B.
A functional genetic variation of the serotonin (2006). Preference for immediate over delayed
(5-HT) transporter affects 5-HT1A receptor rewards is associated with magnitude of ven-
binding in humans. Journal of Neuroscience, tral striatal activity. Journal of Neuroscience,
25, 2586–90. 26, 13213–17.
de Wit, H. 2009. Impulsivity as a determinant and Hariri, A. R., Gorka, A., Hyde, L. W., Kimak,
consequence of drug use: A review of under- M., Halder, I., et al. (2009). Divergent effects
lying processes. Addictive Biology, 14: 22–31. of genetic variation in endocannabinoid sig-
de Wit, H., & Richards, J. B. (2004). Dual deter- naling on human threat- and reward-related
minants of drug use in humans: Reward and brain function. Biological Psychiatry, 66, 9–
impulsivity. Nebraska Symposium on Motiva- 16.
tion, 50, 19–55. Hariri, A. R., Mattay, V. S., Tessitore, A., Fera,
Desarnaud, F., Cadas, H., & Piomelli, D. (1995). F., Smith, W. G., & Weinberger, D. R. (2002a).
Anandamide amidohydrolase activity in rat Dextroamphetamine modulates the response
brain microsomes. Journal of Biological Chem- of the human amygdala. Neuropsychopharma-
istry, 270, 6030–35. cology, 27, 1036–40.
Fakra, E., Hyde, L. W., Gorka, A., Fisher, P. M., Hariri, A. R., Mattay, V. S., Tessitore, A., Fera, F.,
Munoz, K. E., et al. (2009). Effects of HTR1A Smith, W. G., & Weinberger, D. R. (2002b).
C(-1019)G on amygdala reactivity & trait anx- Serotonin transporter genetic variation and
iety. Archives of General Psychiatry, 66(1), 33– the response of the human amygdala. Science,
40. 297, 400–3.
Fellay, J., Shianna, K. V., Ge, D., Colombo, Heinz, A., Goldman, D., Jones, D. W., Palmour,
S., Ledergerber, B., et al. (2007). A whole- R., Hommer, D., et al. (2000). Genotype influ-
genome association study of major determi- ences in vivo dopamine transporter availabil-
nants for host control of HIV-1. Science, 317, ity in human striatum. Neuropsychopharma-
944–47. cology, 22, 133–39.
Fisher, P. M., Meltzer, C. C., Ziolko, S. K., Price, Hyman, S. E., Malenka, R. C., & Nestler, E. J.
J. C., & Hariri, A. R.. (2006). Capacity for 5- (2006). Neural mechanisms of addiction: the
MAPPING NEUROGENETIC MECHANISMS OF INDIVIDUAL DIFFERENCES IN AFFECT 589
role of reward-related learning and memory. Manuck, S. B., Flory, J. D., Muldoon, M. F.,
Annual Review of Neuroscience, 29, 565–98. & Ferrell, R. E. (2003). A neurobiology of
Kalivas, P. W., & Volkow, N. D. (2005). The neu- intertemporal choice In G. Loewenstein, D.
ral basis of addiction: A pathology of motiva- Read, & R. F. Baumeister (Eds.), Time and
tion and choice. American Journal of Psychia- decision: Economic and psychological perspec-
try, 162, 1403–13. tives on intertemporal choice (pp. 139–72). New
Kathuria, S., Gaetani, S., Fegley, D., Valino, F., York: Sage.
Duranti, A., et al. (2003). Modulation of anx- Menon, M., Jensen, J., Vitcu, I., Graff-Guerrero,
iety through blockade of anandamide hydrol- A., Crawley, A., et al. (2007).Temporal dif-
ysis. National Medicine, 9, 76–81. ference modeling of the blood-oxygen level
Kirby, K. N., Petry, N. M., & Bickel, W. K. dependent response during aversive condi-
(1999). Heroin addicts have higher discount tioning in humans: Effects of dopaminergic
rates for delayed rewards than non-drug-using modulation. Biological Psychiatry, 62(7), 765–
controls. Journal of Experimental Psychology: 72.
General, 128, 78–87. Mill, J., Asherson, P., Browes, C., D’Souza, U., &
Knutson, B., Rick, S., Wimmer, G. E., Prelec, D., Craig, I. (2002). Expression of the dopamine
& Loewenstein, G. (2007). Neural predictors transporter gene is regulated by the 3’ UTR
of purchases. Neuron, 53, 147–56. VNTR: Evidence from brain and lymphocytes
Lemonde, S., Turecki, G., Bakish, D., Du, L., using quantitative RT-PCR. American Journal
Hrdinam P. D., et al. (2003). Impaired repres- of Medical Genetics, 114, 975–99.
sion at a 5-hydroxytryptamine 1A receptor Moreira, F. A., Kaiser, N., Monory, K., & Lutz,
gene polymorphism associated with major B. (2008). Reduced anxiety-like behaviour
depression and suicide. Journal of Neuro- induced by genetic and pharmacological
science, 23, 8788–99. inhibition of the endocannabinoid-degrading
Lettre, G., Jackson, A. U., Gieger, C., Schu- enzyme fatty acid amide hydrolase (FAAH) is
macher, F. R., Berndt, S. I., et al. (2008). Iden- mediated by CB1 receptors. Neuropharmacol-
tification of ten loci associated with height ogy, 54, 141–50.
highlights new biological pathways in human Munafo, M. R., Brown, S. M., & Hariri, A.
growth. Nature Genetics, 40, 584–91. R. (2008). Serotonin transporter (5-HTTLPR)
Link, E., Parish, S., Armitage, J., Bowman, L., genotype and amygdala activation: A meta-
Heath, S., et al. (2008). SLCO1B1 variants analysis. Biological Psychiatry, 63, 852–57.
and statin-induced myopathy – a genomewide Parsey, R. V., Oquendo, M. A., Ogden, R. T,
study. New England Journal of Medicine, 359, Olvet, D. M., Simpson, N., et al. (2006).
789–99. Altered serotonin 1A binding in major depres-
Madden, G. J., Petry, N. M., Badger, G. J., sion: A [carbonyl-C-11]WAY100635 positron
& Bickel, W. K. (1997). Impulsive and self- emission tomography study. Biological Psychi-
control choices in opioid-dependent patients atry, 59, 106–13.
and non-drug-using control participants: Drug Patel, S., Cravatt, B. F., & Hillard, C. J. (2005).
and monetary rewards. Experimental and Clin- Synergistic interactions between cannabinoids
ical Psychopharmacology, 5. 256–62. and environmental stress in the activation of
Maier, S. F., & Watkins, L. R. (2005). Stres- the central amygdala. Neuropsychopharmacol-
sor controllability and learned helplessness: ogy, 30, 497–507.
The roles of the dorsal raphe nucleus, sero- Phan, K. L., Angstadt, M., Golden, J.,
tonin, and corticotropin-releasing factor. Neu- Onyewuenyi, I., Popovska, A., & de Wit,
roscience and Biobehavioral Reviews, 29, 829–41. H. (2008). Cannabinoid modulation of amyg-
Manuck, S. B., Brown, S. M., Forbes, E. E., & dala reactivity to social signals of threat in
Hariri, A. R. (2007). Temporal stability of humans. Journal of Neuroscience, 28, 2313–
individual differences in amygdala reactivity. 19.
American Journal of Psychiatry, 164, 1613–14. Piomelli, D. (2003). The molecular logic of endo-
Manuck, S. B., Flory, J. D., McCaffery, J. M., cannabinoid signalling. Nature Reviews Neuro-
Matthews, K. A., Mann, J. J., & Muldoon, science, 4, 873–84.
M. F. (1998). Aggression, impulsivity, and cen- Piomelli, D., Beltramo, M., Glasnapp, S.,
tral nervous system serotonergic responsivity Lin, S. Y., Goutopoulos, A., et al. (1999).
in a nonpatient sample. Neuropsychopharma- Structural determinants for recognition and
cology, 19, 287–99. translocation by the anandamide transporter.
590 AHMAD R. HARIRI
Proceedings of the National Academy of Sci- cannabinoid system in the nucleus accumbens
ences, 96, 5802–7. shell stimulates food intake and increases c-
Rhodes, R. A., Murthy, N. V., Dresner, M. A., Fos expression in the hypothalamus. British
Selvaraj, S., Stavrakakis, N., et al. (2007). Journal of Pharmacology, 151, 1109–16.
Human 5-HT transporter availability predicts Stein, M. B., Simmons, A. N., Feinstein, J. S.,
amygdala reactivity in vivo. Journal of Neuro- & Paulus, M. P. (2007). Increased amygdala
science, 27, 9233–37. and insula activation during emotion process-
Siessmeier, T., Kienast, T., Wrase, J., Larsen, ing in anxiety-prone subjects. American Jour-
J. L., Braus, D. F., et al. (2006). Net influx nal of Psychiatry, 164, 318–27.
of plasma 6-[18F]fluoro-l-DOPA (FDOPA) to Tessitore, A., Hariri, A. R., Fera, F., Smith, W.
the ventral striatum correlates with prefrontal G., Chase, T. N., et al. (2002). Dopamine
processing of affective stimuli. European Jour- modulates the response of the human amyg-
nal of Neuroscience, 24, 305–13. dala: A study in Parkinson’s disease. Journal of
Sipe, J. C., Chiang, K., Gerber, A. L., Beut- Neuroscience, 22, 9099–1003.
ler, E., & Cravatt, B. F. (2002). A missense Tyndale, R. F., Payne, J. I., Gerber, A. L, &
mutation in human fatty acid amide hydrolase Sipe, J. C. (2007). The fatty acid amide
associated with problem drug use. Proceed- hydrolase C385A (P129T) missense variant in
ings of the National Academy of Sciences, 99: cannabis users: Studies of drug use and depen-
8394–99. dence in Caucasians. American Journal of Med-
Solinas, M., Tanda, G., Justinova, Z., Wertheim, ical Genetics B: Neuropsychiatric Genetics, 144,
C. E., Yasar, S., et al. (2007). The endogenous 660–66.
cannabinoid anandamide produces delta-9- VanNess, S. H., Owens, M. J., & Kilts, C.
tetrahydrocannabinol-like discriminative and D. (2005). The variable number of tandem
neurochemical effects that are enhanced by repeats element in DAT1 regulates in vitro
inhibition of fatty acid amide hydrolase but dopamine transporter density. BMC Genetics,
not by inhibition of anandamide transport. 6, 55.
Journal of Pharmacology and Experimental Viding, E., Williamson, D. E., & Hariri, A. R.
Therapy, 321, 370–80. 2006. Developmental imaging genetics: Chal-
Solinas, M., Yasar, S., & Goldberg, S. R. 2007. lenges and promises for translational research.
Endocannabinoid system involvement in brain Developmental Psychopathology, 18, 877–92.
reward processes related to drug abuse. Phar- Volkow, N. D., Fowler, J. S., & Wang, G.
macological Research, 56, 393–405. J. (1999). Imaging studies on the role of
Sorice-Gomez, E., Matias, I., Rueda-Orozco, P. dopamine in cocaine reinforcement and addic-
E., Cisneros, M., Petrosino, S., et al. (2007). tion in humans. Journal of Psychopharmacol-
Pharmacological enhancement of the endo- ogy, 13, 337–45.
CHAPTER 26
Annett Schirmer
“Women are never disarmed by thymic than women (Figueira et al., 2008;
compliments. Men always are. That is the Pompili et al., 2008). These temperamental
difference between the sexes.” differences emerge in infancy, where girls
–Oscar Wilde are typically less active and novelty seek-
ing than boys (Maccoby & Jacklin, 1974).
As children, girls are better than boys
in inhibiting inappropriate responses and
Among the existing stereotypes, those behaviors, but tend to show more fear (Else-
regarding sex differences appear to be most Quest, Hyde, Goldsmith, & Van Hulle,
pervasive. Across time and culture, women 2006). As a consequence, stressful life events
are typically viewed as the fairer but also the more readily evoke internalizing problems,
weaker sex. Emotional sensitivity and empa- marked by withdrawal, somatic complaints,
thy are more readily ascribed to women fear, or sadness, in girls, whereas boys
than to men, who in turn are perceived as tend to show more externalizing problems,
more knowledgeable and competent (Kim- marked by delinquent or aggressive behavior
mel, 2000). Moreover, whereas sex-specific (Crijnen, Achenbach, & Verhulst, 1997). In
behavioral norms endorse socially respon- adulthood, this translates into sex differ-
sible and accommodating interaction styles ences in the prevalence of psychiatric dis-
in women, they endorse dominant and orders. Specifically, anxiety and depression
assertive interaction styles in men. occur about twice as often in women than
In line with these popular perceptions, in men (Hamann, 2005; Nolen-Hoeksema,
and possibly because of them, women and 2001). Conversely, conduct disorder and psy-
men differ in their temperament. Across chopathy are more prevalent and severe
different geographical regions, women are in men (Rogstad & Rogers, 2008; Simonoff
found to be more depressive, anxious, and et al., 1997).
cyclothymic than men, who in contrast Both female and male typical psychi-
are found to be more irritable and hyper- atric conditions have marked social and
591
592 ANNETT SCHIRMER
economic consequences not only for the in emotional processing and behavioral pat-
affected individuals but also for their imme- terns, allowing us to link these differences to
diate social circle and society at large. There- psychopathology.
fore, it is very important to characterize
sex-specific vulnerabilities and their devel-
opment. In this chapter, I first review Prosocial Emotions
research that addresses such sex-specific
Fear
vulnerabilities through the investigation of
emotional responses, emotion regulation Fear is an emotion felt in response to threats
strategies, and influences of emotion on cog- that, if confronted, entail a great risk of
nition in healthy women and men. I then immediate physical or social damage. Fright-
discuss the biological and environmental fac- ened individuals appraise these threats as
tors that produce sex-specific resources and more powerful than themselves and thus
challenges and consider how these resources seek to avoid them. The brain system sup-
and challenges engender male and female porting threat appraisal has been studied
minds. extensively. From these studies, the amyg-
dala has emerged as a key structure. Located
in the medial temporal lobe, the amyg-
Emotional Perception and Expression dala receives sensory information from cor-
tical and subcortical structures and modi-
As apparent from the different tempera- fies activity in these and other brain regions
ments, behavioral problems, and psychi- according to appraisal outcomes. Appraisal
atric conditions of men and women, emo- by the amygdala is believed to be automatic
tional differences between the sexes are not and may thus occur independently of cogni-
one-dimensional. Rather than being simply tive or reflective processes (see Chapters 14
“more emotional,” women differ qualita- and 15 for reviews). However, such pro-
tively from men in their emotional expe- cesses, if they occur, can influence amygdala
riences. Moreover, certain emotions, such function (see Chapter 16 for a review).
as fear, sadness, and happiness, appear to Once activated by threat, the amyg-
play a greater role in women, whereas oth- dala initiates a cascade of physiological
ers, such as anger, contempt, and disgust, and cognitive responses aimed at escap-
play a greater role in men. Researchers have ing or minimizing damage. The sympa-
categorized these emotions as prosocial or thetic adrenal medullary system and the
powerless and antisocial or powerful emo- hypothalamic-pituitary-adrenal (HPA) axis
tions, respectively (Hess, Adams, & Kleck, become activated and issue the release
2005; Safdar et al., 2009). Expressions of of adrenaline/noradrenaline and corticos-
fear, sadness, and happiness are thought to teroids, respectively. These hormones then
appeal to others and to communicate depen- attach to target receptors, thereby regulat-
dency or a willingness to cooperate and, ing activity of the autonomous nervous sys-
in that sense, can be considered as “proso- tem, the metabolic system, and the immune
cial.” Expressions of anger, contempt, and system. In the short term, they prepare the
disgust, in contrast, signal the rejection of body for action by facilitating physiological
something or someone and a willingness processes such as heart rate, blood pressure,
to defend one’s own interests, thus having and sweating. In the long term, they deplete
more “antisocial” functions. As seen later, the body’s energy stores and weaken the
this classification fails to fully map onto immune system. In addition to these physio-
existing sex differences: Women do not con- logical effects, the release of stress hormones
sistently score higher on prosocial and lower feeds back to the central nervous system
on antisocial emotions than men. Neverthe- and modulates brain activity. Among other
less, this classification represents a useful regions, this feedback reaches the amygdala,
approach for the analysis of sex differences thereby influencing the effect of amygdala
SEX DIFFERENCES IN EMOTION 593
activity on cognition (McGaugh, 2004). At Vaitl, 2005). Thus, it seems that informa-
the cost of other ongoing processes, the tion relating to physical combat may have
amygdala facilitates attention to threatening more implicit relevance for men, whereas
stimuli and, together with peripheral feed- social threat outside of combat has more rel-
back, makes their representation and eval- evance for women. This accords with the
uation a priority (McGaugh, 2004; see also roles of men and women in hunter/gatherer
Chapter 14). Thus, threat typically “pops societies and the proposed sex differences in
out” and is retained more easily in mem- the nature of the threat response: Whereas
ory than other mundane information (see this response has been associated with fight-
Chapter 20 for a review). or-flight in men, recent evidence suggests
Although neural, physiological, and cog- a greater significance of tend-and-befriend
nitive responses associated with fear are behavior in women (Taylor et al., 2000).
largely comparable in women and men, dif- According to this evidence, women, because
ferences nevertheless exist and translate into of their reproductive responsibilities, are less
behavioral differences. At the neural level, willing to engage in physical combat. Instead
sex differences have been observed for both they engage in nurturing activities protect-
structure and function. Specifically, struc- ing the self and offspring and form alliances
tural studies have found the amygdala to be with other women and, to a lesser extent,
smaller in women than in men (when con- men that can be tapped on in times of need.
trolled for total brain size; Goldstein et al., As mentioned earlier, once activated, the
2001). Additionally, there are differences in amygdala triggers a cascade of physiolog-
the structural covariation of the amygdala ical, cognitive, and behavioral responses;
and other brain regions. Gray matter density again these responses are qualified by sex.
of the left amygdala and the right angular In particular, physiological indices of defen-
gyrus is more positively correlated in women sive activation to threat, such as heart rate
than in men, whereas gray matter density of deceleration, startle reflex, or facial muscle
the left amygdala and bilateral anterior infe- activity, are greater in women than in men
rior temporal cortex is more positively cor- (Anokhin & Golosheykin, 2010; Bradley,
related in men than in women (Mechelli, Codispoti, Cuthbert, & Lang, 2001). At a
Friston, Frackowiak, & Price, 2005). Fur- cognitive level, women more readily notice
thermore, during the resting state, amyg- threat – particularly if it is social in nature
dala functional connectivity with other brain (Schirmer et al., 2005, 2007). Additionally,
structures is left-lateralized in women but there are sex differences in the relationship
right-lateralized in men (Kilpatrick, Zald, between threat and learning. After a stres-
Pardo, & Cahill, 2006). sor, learning in males is enhanced, whereas
Given these structural and functional learning in females is impaired (Wood &
differences, it is not surprising that brain Shors, 1998). Nonhuman animal studies, fur-
responses to threat differ between women thermore, have revealed that males fear-
and men. Threat perceived from angry condition more readily if freezing is used
(McClure et al., 2004) or frightened faces as the dependent measure, whereas females
(Williams et al., 2005) elicits greater amyg- fear-condition more readily if the startle
dala activity in women than in men. Like- reflex is used as the dependent measure (for
wise, vocal threat has been shown to excite a review see Dalla & Shors, 2009). This find-
greater neural responses in women as com- ing, as well as that females learn more readily
pared with men (Schirmer, Simpson, & to avoid a footshock, has been taken to imply
Escoffier, 2007; Figure 26.1). Conversely, pic- baseline behavioral differences that prop-
tures portraying physical threat from human agate sex-specific fear responses. Greater
or animal attack elicit lower amygdala and baseline activity levels may make defensive
fusiform activity in women as compared to or avoidance behaviors more accessible in
men, despite women’s greater self-reported females as compared with males (Dalla &
fear (Schienle, Schäfer, Stark, Walter, & Shors, 2009).
594 ANNETT SCHIRMER
CZ -5 µV MMN CZ -5 µV MMN
P300
P300
loud vocal soft nonvocal
soft vocal loud nonvocal
piggybacked onto the evolutionary older and pain and implicating sex steroids in the
pain system. moderation or mediation of sex differences
Sex differences in pain and sadness can (Martin, 2009).
again be observed at different levels of anal-
ysis. Research comparing the morphology of
Happiness and Attraction
the anterior cingulate found it to be larger in
women than in men (Goldstein et al., 2001). Like other emotions, the positive feel-
Furthermore, both pain and sadness recruit ings evoked when anticipating or obtaining
this structure in sex-specific ways. Activa- something longed for are fundamental moti-
tion of the anterior cingulate by noxious vators of behavior. Unlike other emotions,
stimuli is more likely in women than in men however, they are critical in giving life a
(Paulson, Minoshima, Morrow, & Casey, sense of meaning or fulfillment. The brain
1998) and is associated with sex-specific pat- system that mediates positive feelings is
terns of regional connectivity (Labus et al., commonly referred to as the reward system
2008). Women also activate this structure and overlaps with the dopaminergic system
more automatically than men when perceiv- (for a review, see Chapter 19). The dopamin-
ing someone else who is in pain (Singer et al., ergic system originates in the midbrain,
2006). Activation of the anterior cingulate by where neurons in the ventral tegmental area
social exclusion does not differ between the and substantia nigra project upward to sub-
sexes. However, only in women does this cortical and cortical targets. Most important
activation correlate with increased immune for reward is the mesolimbic pathway pro-
system activity (Eisenberger, Inagaki, Rame- jecting to the nucleus accumbens and with
son, Mashal, & Irwin, 2009). The absence of more recently identified projections to the
such a correlation in men has been linked amygdala. Once activated, these projections
to potentially stronger regulatory activity trigger the release of dopamine, which binds
aimed at reducing social distress. Such activ- to receptors located in various parts of the
ity may buffer against the coupling between brain. Dopamine activity, particularly in the
social exclusion, social pain, and immune nucleus accumbens, is associated with sev-
system activity and thus reduce the risk for eral life-sustaining activities like eating or
depression. reproduction. It creates pleasurable feelings
Given the lower perceptual and neural and thereby makes these activities rewarding
thresholds for pain and possibly sadness in (Wise, 2004). Pleasurable feelings can also be
women (Paulson et al., 1998), it is not sur- elicited through the stimulation of dopamin-
prising that they differ from men in a key ergic structures via implanted electrodes or
physiological response: crying. Compared to the application of drugs that act as dopamin-
men, women cry more frequently, longer, ergic agonists. However, whether experi-
and more intensely, especially in situations enced through the consumption of natu-
involving “tender feelings,” criticisms from ral rewards, electrical stimulation, or drugs,
or disputes with others, and problems at pleasurable feelings entice humans and non-
work (Williams & Morris, 1996). This is human animals to repeat temporally linked
also true for depressed women relative to behaviors and therefore contribute to the
depressed men (Romans, Tyas, Cohen, & development of addiction.
Silverstone, 2007) and appears to emerge As already mentioned, a morphometric
early in infancy (Fuller, 2002). A survey analysis identified the amygdala as being
across 33 countries revealed that female cry- larger in men than in women. Analysis
ing increases during the luteal phase of the of the nucleus accumbens, however, failed
menstrual cycle, when estrogen is high (but to reveal gross morphological differences
note cultural differences; see van Tilburg, (Goldstein et al., 2001). Nevertheless, sexual
Becht, & Vingerhoets, 2003). Likewise, pain dimorphisms in these and other dopamin-
perception increases during the luteal phase, ergic structures have been identified at a
again corroborating the link between sadness smaller level in nonhuman animal models.
596 ANNETT SCHIRMER
Here, females have been shown to differ erotica activates the nucleus accumbens
from males in the number of dopamin- regardless of sex, men show enhanced activ-
ergic neurons and receptors and in the ity in an extended network including the
amount of dopamine released during acti- visual cortex (Sabatinelli et al., 2004) and
vation. For example, females, compared the sexually dimorphic amygdala (Hamann,
to males, demonstrate a chronically lower Herman, Nolan, & Wallen, 2004). Thus,
dopaminergic tone and a smaller number this evidence suggests that, whereas certain
of D1 receptors in the striatum and nucleus affiliative cues from other humans may be
accumbens (for a review, see Becker, 2009). more important for women, cues related to
Researchers use these and similar obser- sexual intercourse may be more important
vations to explain why women and men for men. Given the role of female gonadal
differ in their response to rewarding stim- hormones in reward, these effects are likely
uli. Such differences have been documented to vary across the menstrual cycle, which
for addiction. Although more men than may explain why some sex differences have
woman are drug abusers and have addic- not been reported consistently.
tions, women, if exposed to drugs, become Among the behaviors that are associated
addicted more easily (for a review, see Kuhn with happiness, one has emerged as being
et al., 2010). This finding has been repli- markedly different between women and
cated in rodent models and linked to both men. Despite similarities in the reported
dopamine and estrogen. In particular, the feelings of happiness, across different
addictive effect of drugs and sexual dimor- cultures women smile more frequently than
phisms in the dopamine system increases men (Safdar et al., 2009). Facial displays,
with increasing levels of estrogen. Sex dif- similar to the human smile, exist in other
ferences in addictive behaviors emerge dur- primates and play a role in regulating
ing adolescence, when gonadal hormones social intercourse by signaling appeasement
bring about sexual maturation and introduce or submission. It is believed that these
females to heightened and cyclically modu- functions are still preserved to some extent
lating levels of estrogen (Kuhn et al., 2010). in the human smile, and that it thus
Of course, the human reward system reflects an individual’s power (LaFrance
evolved not for our appreciation of recre- & Henley, 1994). This belief, however,
ational drugs, but as a means to motivate has received mixed support, and more
self-sustenance. Through this system, recent evidence points to a relationship
the approach or consumption of stimuli between smiling and affiliation: Compared
that serve the purpose of self-sustenance to nonsmiling individuals, smiling indi-
becomes rewarding. Among others, such viduals are perceived as more affiliative
stimuli include food, peaceful nature rather than less dominant (Hess et al.,
scenes, other, positively spirited humans, 2005). Thus, women’s greater propen-
and opportunities for reproduction. Given sity to smile might reflect their greater
the effects observed for drug addiction, interest in affiliation. Notably, a recent
one may venture that the response to these study identified another contributing factor
evolutionary more appropriate rewards to women’s greater frequency of smiles
differs between women and men. However, (Becker, Kenrick, Neuberg, Blackwell, &
studies investigating brain activation pat- Smith, 2007). In a clever series of exper-
terns to positive scenes or faces failed to find iments it was demonstrated that female
sex differences (e.g., Sabatinelli, Flaisch, faces differ from male faces in looking hap-
Bradley, Fitzsimmons, & Lang, 2004). Yet, pier – even without a smile (Figure 26.2).
such differences were observed for vocal Thus, one may speculate that this sexual
expressions of happiness (Schirmer et al., dimorphism of the face evolved to make
2005), with greater neuronal responses in females look more approachable and males
women relative to men. Sex differences look less approachable, thereby facilitating
also exist for responses to erotica. Although different interactional styles.
SEX DIFFERENCES IN EMOTION 597
Figure 26.2. Male and female faces differ in their structure, and this
difference correlates with how happy or angry they are perceived.
Illustrated here are stimuli and results employed by a study by
Becker and colleagues (2007). In their study, more masculine faces
were rated as angrier and as less happy than more feminine faces.
Masculinity and anger ratings are presented in the first and second
data row, respectively. Reprinted from Journal of Personality and
Social Psychology, 92, Becker et al., The confounded nature of angry
men and happy women, 179–190, Copyright (2007), with permission
from the American Psychological Association.
levels of dopamine and testosterone have to down-regulate their anger feelings (see
been linked to an increase in aggression. For Chapter 16).
example, the anticipation of a fight has been Research studying the emotional and
show to reduce serotonin secretion (Ferarri, behavioral consequences of sex-specific
Van Erp, Tornatzky, & Miczek, 2003) while brain mechanisms for aggression and anger
enhancing both testosterone (Oliveira et al., has found that women are similar to men
2001) and dopamine secretion (Ferarri et al., in the former but not the latter. In par-
2003). ticular, surveys assessing felt anger found
Research on the neural underpinnings of that this emotion is experienced equally
anger and aggression has revealed sex differ- often and intensely across the sexes (Archer,
ences at various levels. For one, there are sex 2004). Nevertheless, one may argue that
differences in brain structure. In addition to this finding cannot fully discount the pos-
the previously mentioned larger amygdala, sibility of sex differences in feeling anger.
men have on average smaller orbitofrontal Because many emotions are elicited auto-
cortices relative to women (Goldstein matically and may be experienced without
et al., 2001). Furthermore, women and men conscious reflection, explicit emotion judg-
differ in the neurochemical systems impli- ments may be difficult and unreliable. For
cated in aggression, such as the dopamine example, researchers have observed sex dif-
system (see preceding section). Addition- ferences in cardiac responses to emotion
ally, there are sex differences in circulat- induction in the absence of sex differences
ing testosterone levels at different stages in self-report (Labouvie-Vief, Lumley, Jain,
of development. As such, testosterone has & Heinze, 2003). Moreover, there is some
both sexually dimorphic organizational and evidence that men control their anger more
activational effects. Early surges of testos- frequently than women, suggesting the pos-
terone in males effect neuronal changes that sibility that anger may be perceived more
promote later aggressive behavior (Bronson frequently and/or more intensely by men
& Desjardin, 1968). In adulthood, increases relative to women (Doster, Purdum, Mar-
in testosterone have been shown to down- tin, Goven, & Moorefield, 2009).
regulate GABAergic activity, thereby reduc- Although the jury is still out on sex differ-
ing the brain’s main inhibitory mechanism ences in the feelings of anger, there is unan-
and increasing the immediate likelihood of imous support for sex differences in aggres-
physical attack (Pinna, Costa, & Guidotti, sion. In the United States, men commit
2005). Additionally, testosterone levels 10 times more murders than women and are
appear to be linked to differences in the 5 times more likely to receive “correctional
activation of brain structures implicated in supervision” (Craig & Halton, 2009). Sim-
aggression. Whereas in women testosterone ilar statistics have been observed in other
is unrelated to the activation of brain regions nations and suggest a greater readiness of
in the context of angry faces, such a relation- men to engage in violent acts. Given the
ship exists in men. Non-intuitively, how- lack of self-reported differences in anger,
ever, testosterone in men correlates posi- these findings have been linked to men’s
tively with vmPFC activation and negatively smaller orbitofrontal cortex and attributed
with amygdala activation as evoked by to a greater difficulty in inhibiting aggres-
angry faces (Stanton, Wirth, Waugh, & sive impulses (Jones, 2008). Nevertheless,
Schultheiss, 2009). One possible explanation it is possible that such impulses are up-
for this might be that these faces indeed regulated by sex-specific dopaminergic and
trigger anger-related patterns of processing testosterone activity.
more readily with increasing levels of testos-
terone. However, the constrained situation
Disgust and Contempt
in the fMRI scanner and/or the realiza-
tion that these are only pictures and not Disgust is experienced in response to repul-
real interaction partners may induce men sive objects whose presence may be revealed
SEX DIFFERENCES IN EMOTION 599
by their sight, smell, taste, touch, or sound. gesting a dissociation between disgust and
Thus, although disgust, like anger, is a neg- its social counterpart, contempt.
ative emotion, it is tied more closely to These neuroimaging results nicely align
the various senses and promotes avoidance with self-reported disgust sensitivity. Across
rather than approach. Researchers believe several cultures, women feel greater disgust
disgust to be an adaptive response to disease- than men, and this difference appears to
salient stimuli (Curtis, Aunger, & Rabie, be relatively consistent across the lifespan
2004). It prevents us from contaminating (Curtis et al., 2004). Comparatively little
ourselves with viruses, bacteria, or parasites research is available on subjective experi-
that feast on bad food, cadavers, and the ences of contempt. What is available, how-
sick. The notion of contamination has been ever, suggests a greater role of this emotion
extended to foul acts performed by indi- for men. Observers perceive a man more
viduals with whom we share a social cir- readily as contemptuous than a woman, par-
cle. Noticing such acts is believed to elicit a ticularly if he is also perceived as high in
social variant of disgust referred to as con- dominance and low in affiliation (Hess et al.,
tempt. 2005). Moreover, self-reported social dom-
Of the senses tied to the experience of inance has been shown to correlate with
disgust, the sense of taste is particularly brain responses to contemptuous faces, lead-
important. Even if experienced from a dis- ing researchers to speculate that social domi-
tance, disgusting stimuli activate a region in nance mediates greater contempt sensitivity
the anterior insula called the primary gusta- in men relative to women (Aleman & Swart,
tory cortex and may thus literally leave a bad 2008).
taste in one’s mouth. Brain imaging stud-
ies find that the anterior insula is activated
more strongly to disgusting stimuli or human Emotion Regulation
nonverbal signals of disgust than to neutral
information (Wicker et al., 2003). Additional Emotions are fundamental motivators of
evidence comes from neurological patients. behavior. If we do not feel, we do not
For example, NK, a patient with a lesion to act. Sometimes, however, a full-blown emo-
the insula extending into the basal ganglia, tional response may be inappropriate and
was no longer disgusted by normally repul- even impair behavioral outcomes. For exam-
sive, disease-salient stimuli. He also had dif- ple, being overwhelmed by fear may pre-
ficulties recognizing expressions of disgust vent individuals from succeeding in a job
in other individuals (Calder, Keane, Manes, interview. Being overwhelmed by anger may
Antoun, & Young, 2000). cause people to turn small disputes into
Gross morphometric analysis of the physical violence. Thus, although nature
insula and basal ganglia found no meaning- designed emotions to trigger important
ful sex differences (Goldstein et al., 2001; but behavioral impulses, it also built in a
see Welborn et al., 2009, who found a larger brake to stop these impulses from running
putamen and globus pallidus in women). unchecked.
Nevertheless, there appear to exist sex dif- Researchers refer to nature’s brake as
ferences in function. Several neuroimaging emotion regulation (see Chapter 16). In
studies revealed greater neuronal activity the past decade, they identified numerous
in response to disgust-relevant images in mechanisms through which emotion regu-
female relative to male participants. These lation may be achieved and developed a
activity differences have been reported in range of classification systems. Two of the
the bilateral insula and frontal and tempo- best studied mechanisms with respect to
ral cortex (Aleman & Swart, 2008). Interest- their neural underpinnings are (1) cognitive
ingly, facial expressions of contempt elicit reappraisal, an emotion regulation strategy
less activity in female as compared to male that moderates attention and/or knowledge,
participants (Aleman & Swart, 2008) sug- thereby reevaluating an emotional event in
600 ANNETT SCHIRMER
a goal-oriented manner, and (2) emotion level of cortisol (Buchanan et al., 2010).
expression suppression, an emotion regu- Sex differences have also been observed
lation strategy that has bodily targets such for measures of brain activity during emo-
as the facial muscles and, like cognitive tion regulation tasks. For example, Mak and
reappraisal, may serve a situational goal colleagues (2009) found that men engage
(Welborn et al., 2009). the dorsal emotion regulation system to a
Research investigating both regulation greater extent than women, who engaged
strategies suggests the existence of two ded- the ventral system more strongly. Unfortu-
icated brain systems (Ochsner & Gross, nately, this study and the earlier mentioned
2005). A ventral system situated in the brain lesion study failed to specify which
orbitofrontal cortex is believed to support regulation strategy participants should use
the implicit and context-sensitive appraisal during the task. It is hence possible that
of emotional stimuli and the selection of an the observed sex differences reflect differ-
appropriate response. A dorsal system sit- ences in the strategies typically used by men
uated in the dorsal PFC presumably sup- and women. For example, men, being more
ports explicit reasoning about an event and likely to engage in expression suppression,
may affect emotion through influencing the may more strongly rely on the dorsal sys-
ventral system or through biasing percep- tem. Moreover, regulation through this sys-
tual and memory systems. As with other tem may become easier with impairment
functions that rely on the dorsal prefrontal to the potentially competing ventral sys-
cortex, with practice or experience explicit tem. In contrast, women may gravitate to
regulation attempts may become implicit, emotion regulation via the ventral system.
and the contribution of the dorsal system to When this system is impaired, emotional
emotion regulation may decrease. challenges may be experienced as more
Of the brain systems implicated in emo- intense.
tion regulation, the ventral system has been Studies investigating sex differences for
repeatedly shown to differ between men a specified emotion regulation strategy
and women. Across several morphometric are scarce. In one such study, McRae and
studies, men were found to have smaller colleagues (2008) presented participants
orbitofrontal cortices than women, partic- with negative pictures and asked them to
ularly in the right lateral and ventrome- either “look” or “regulate” their emotional
dial aspects (Goldstein et al., 2001; Welborn response. In the look condition, partici-
et al., 2009). Furthermore, the volume of pants passively viewed the pictures. In the
the vmPFC correlates negatively with the regulate condition, participants were asked
self-reported frequency of using expression to reduce a potentially negative response
suppression in emotionally challenging sit- using cognitive reappraisal. Compared to
uations and as such may mediate the sex the look condition, the regulate condition
differences that have been observed for the elicited greater activity in the ventral and
latter measure (Gross & John, 2003). Men dorsal system and reduced activity in the
report using expression suppression more amygdala. Sex differences were observed
frequently than women, and this difference in both the ventral and dorsal system, with
can be explained by differences in the size greater activity in women as compared
of the vmPFC (Welborn et al., 2009). to men. In the amygdala, the effect was
In addition to differences in size, research reversed. Here men but not women showed
suggests that the ventral system is differently the reported reduction of activity as a
engaged by men and women during emo- function of regulation. Notably, women
tion regulation. In men, lesions to this sys- showed an effect of emotion regulation in
tem result in decreased stress responses as the ventral striatum that was absent in men.
measured by blood levels of cortisol during The authors thus speculated that women,
a stress challenge task. In contrast, compa- instead of down-regulating a negative emo-
rable lesions in women increase the blood tional response, reappraise events to elicit
SEX DIFFERENCES IN EMOTION 601
emotions and memory. Although emotions nurture responsibility and authority in men
benefit subsequent memory in both men and to curtail similar aptitudes in women.
and women, there are different mecha- The assertion “Give me a child and I’ll shape
nisms underlying this benefit. Across sev- him into anything,” famously attributed to
eral functional imaging studies, emotion B. F. Skinner, nicely reflects these senti-
effects during memory encoding elicited ments.
right-lateralized amygdala activity in men As scientists overcame the dogmas of
and left-lateralized amygdala activity in behaviorism and started to explore the
women (Cahill et al., 2001). These dif- human mind and its substrate, the brain, sev-
ferences have been linked to differences eral biological differences between men and
in amygdala connectivity during rest (Kil- women emerged. As reviewed earlier, both
patrick et al., 2006) and to differential forms structural and functional brain differences
of memory. With respect to the forms of were discovered that could be linked to dif-
memory, beta-blockers administered to men ferences in mental processes and behavior.
and women prior to an emotional memory Yet, these discoveries failed to fully con-
task were found to impair the subsequent vince of the importance of nature. After
retrieval of global/gist and local/detail infor- all, gendered environments provide differ-
mation, respectively (Cahill & van Stegeren, ent challenges and resources to developing
2003). Thus, it has been proposed that right girls and boys, thereby shaping their brains
amygdala recruitment in men supports the differently (Kimmel, 2000). Moreover, skep-
storage of global/gist information, whereas tics held that differences between the sexes
left amygdala recruitment in women sup- are negligible at birth and that if boys and
ports the storage of local/detail information. girls were treated in the same way the dif-
These and other memory effects have been ferences would remain negligible into adult-
shown to depend on the female menstrual hood.
cycle (Andreano & Cahill, 2008), suggesting That this belief is wrong became clear
hormonal mediation. when researchers identified cases in which
the environment failed to successfully shape
male and female typical behaviors. One
Factors Contributing to Sexual such case is that of Bruce Reimer (Colap-
Differentiation into, 2000). As an infant, he lost his genitals
during circumcision, and his parents, fol-
Scientists and philosophers have long lowing medical advice at the time, raised
debated the cause of sex differences in emo- Bruce as a girl. Despite this upbringing
tion and other aspects of the human mind. and female hormonal treatments, however,
For example, Aristotle argued that “women Bruce experienced difficulties with his gen-
are [generally] defective by nature” (Whit- der identity. When his parents later revealed
back, 1976). In contrast, his mentor, Plato, his biological sex to him, he decided to
held that, despite their physical inferior- return to being a male and underwent
ity, women may not be intellectually infe- the necessary medical treatments. Yet, his
rior and, if instructed in the same way as childhood experiences left a deep mark,
men, may be capable of similar pursuits and he failed to achieve life satisfaction
(Plato, 2000). The idea of natural gender as an adult. The resulting psychological
equality was also popular in Western cul- difficulties were so strong that, even though
tures of the 20th century. With the emer- he had become a husband and adoptive
gence of behaviorism as a school of thought, father, he took his life at the age of 38.
researchers interpreted apparent differences Other observations that underline the
between men and women as the result of role of nature have been made in children
the different environments in which they are with congenital adrenal hyperplasia (CAH).
placed. The more responsible and authorita- CAH is a genetic disorder that affects the
tive positions that men held were thought to enzymes responsible for the production of
SEX DIFFERENCES IN EMOTION 603
Figure 26.3. Vervet monkeys (Cercopithecus aethiops sabaeus) show sex-specific toy preferences
(Alexander & Hines, 2002). Female monkeys spend more time with toys that are typically preferred
by girls (e.g., dolls), whereas male monkeys spend more time with toys typically preferred by boys
(e.g., cars). Reprinted from Evolution and Human Behavior, 23, Alexander & Hines, Sex differences
in response to children’s toys in nonhuman primates (Cercopithecus aethiops sabaeus), 467–479,
Copyright (2002), with permission from Elsevier.
cortisol by the adrenal gland. One of the of the emotion phenomena described earlier
consequences of this disorder is the hyper- depend on the levels of testosterone, estro-
or hypo-production of sex steroids. CAH gen, and progesterone, which vary across
girls with increased concentration of andro- the day and an individual’s lifetime and
gens, the male sex steroid, show more male- in women across the menstrual cycle. For
typical preferences for toys than girls with- example, early cortical responses to unat-
out the disorder. When given a choice, they tended vocal deviants differ as a function
select building blocks or guns over dolls and of emotion and estrogen (Schirmer et al.,
beauty items. Importantly, they make these 2008). Moreover, estrogen and progesterone
selections despite parental encouragement seem to enhance amygdala and hippocam-
to play with female-typical toys (Pasterski pus activation to negative images (Andreano
et al., 2005). The fact that such toy pref- & Cahill, 2010). Together with testosterone,
erences can also be observed in male and these hormones also affect emotion regula-
female nonhuman primates further under- tion. Among others, a review by van Wingen
lines the importance of nature in deter- and colleagues (2011) suggests that proges-
mining sex-specific behaviors (Alexander & terone enhances connectivity between the
Hines, 2002; Figure 26.3). amygdala and the medial prefrontal cortex,
Finally, evidence is accumulating for a whereas testosterone decreases connectivity
role of gonadal hormones in the emotional between the amygdala and the orbitofrontal
responses of healthy human adults. Several cortex, thus potentially promoting emotion
604 ANNETT SCHIRMER
regulation and interfering with behavioral ing of HPA activity following initial encoun-
inhibition, respectively. Lastly, there is evi- ters with a stressor. Licking and grooming
dence for gonadal hormones influencing have also been demonstrated to influence
the learning that results from an emotional the expression of estrogen receptors in the
event (Milad et al., 2006, 2010). Importantly, medial preoptic area and the associated
women are more likely to show sponta- development of oxytocin receptors (Zhang
neous recovery after fear conditioning and & Meaney, 2010). Female offspring experi-
extinction when estrogen is low as com- encing frequent licking and grooming show
pared to high. Researchers speculate that a greater number of oxytocin receptors and
such hormonal modulation of learning could more readily and intensely care for pubs that
be implicated in the high incidence of anxi- are not their own (Zhang & Meaney, 2010).
ety disorders in women. These effects are remarkable because
Despite the overwhelming evidence for they demonstrate sustained changes in brain
organizational and activational effects of function that result from experiences dur-
gonadal hormones, it would be inappropri- ing the first days of life. Moreover, recent
ate to conclude that they as opposed to envi- evidence of such experiences influencing
ronmental influences fully explain sex dif- methylation and gene expression in the
ferences in emotions. Instead, it appears that human brain indicates that the epigentic
sex differences result from both innate bio- effects observed in nonhuman animals are
logical mechanisms and the environments also present in humans (McGowan et al.,
that these mechanisms meet. Evidence for 2009). It is thus reasonable to consider envi-
this claim comes from a research field called ronmental factors, such as parental care,
epigenetics, which investigates gene expres- as crucial variables in the emergence of
sion as a function of the information that is sex differences. In particular, there is abun-
stored together with the genome. Methyla- dant evidence that parents interact differ-
tion is one such type of information. It refers ently with boys and girls. For example,
to the methyl groups that bind to parts of a some researchers found that parents talk
cell’s DNA, thereby regulating transcription less about feelings with boys than with girls
factor binding and cell-specific protein syn- and restrict the expression of sadness more
thesis. strongly in boys than in girls (Maccoby,
Research has demonstrated that methy- 1998). Others reported that women tend
lation changes as a function of environmen- to smile more at female as compared to
tal conditions. Important for the present male infants, select different toys (Will, Self,
discussion is evidence from nonhuman ani- & Datan, 1976), and touch them in differ-
mals that parental care affects brain function ent ways (Fleishman, 1983). Compared with
through gene expression. Early tactile inter- infant girls, infant boys receive less ventral-
actions between mother and her offspring to-ventral contact with their mothers. How-
have been shown to affect the expression of ever, during the first 3 months of life, they
glucocorticoid receptors in the hippocam- receive more general touch than infant girls.
pus. Offspring experiencing frequent licking At the age of 6 months, the pattern reverses.
and grooming from their biological or surro- From then on into adulthood, girls engage
gate mothers develop a greater number of in tactile contact more frequently than boys
glucocorticoid receptors in the hippocam- (for a review, see Fleishman, 1983).
pus than offspring experiencing little licking Given the role of early tactile experi-
and grooming (Zhang & Meaney, 2010). The ences for gene expression, it is easy to con-
hippocampus plays a critical role in regu- ceive that sex-specific parenting affects brain
lating activity of the HPA axis, because it development. Moreover, one may speculate
receives feedback about the level of gluco- that greater tactile stimulation of girls pro-
corticoids released from the adrenal gland. A motes the development of the brain’s social
greater number of glococorticoid receptors system and associated prosocial emotions.
therefore translates into a greater dampen- Conversely, a restriction of tactile contact
SEX DIFFERENCES IN EMOTION 605
with boys may curtail their social devel- tial harm to one’s life. The same applies to
opment. Thus, together nature and nur- territorial behavior, which in humans and
ture shape the human mind and create two other primates is more strongly developed
individuals with different emotional dispo- in males. Thus, emotional responses that
sitions. facilitated aggression (e.g., anger/contempt)
may have been of greater benefit to males
than emotions that reduced aggression (e.g.,
Conclusions fear/sadness). Conversely, early female typ-
ical tasks such as food gathering and child
The emotions of men and women differ care were less confrontational and dan-
in many ways. For some emotions, men gerous. Hence, here aggression may not
show stronger subjective feelings, cognitive, have been as important and potentially
and/or behavioral effects than women (e.g., impeded group cohesion. Given their lesser
anger, contempt), whereas for others we physical strength and greater investment
find the opposite (e.g., sadness, fear, dis- in reproduction, females likely relied more
gust). Moreover, we also observe sex differ- than males on social bonds for protection
ences for a range of emotions that depend (Taylor et al., 2000). As such, emotional
not so much on the emotion itself but on the responses linked to self-preservation (e.g.,
eliciting stimulus (e.g., social vs. nonsocial). fear/disgust) and the facilitation of prosocial
Different emotion regulation mechanisms in behavior (e.g., fear/sadness/happiness) may
men and women further contribute to this have been of greater benefit to women than
sexual specialization. emotional responses that triggered aggres-
Although the sex differences reported sive impulses.
here are highly complex, an organizing prin- One consequence of such an emotional
ciple may be inferred from other aspects of specialization is that men and women posi-
sexual differentiation. Specifically, humans tioned themselves at different points on
like most animals show sexual dimorphisms different “basic emotion continua.” Thus,
that go together with sex-specific roles in if they present with an emotional distur-
reproduction and the care of offspring. In bance, they are more likely to be considered
human evolution, these dimorphisms poten- abnormal when they digress away from the
tially assisted in the emergence of different mean of both sexes as compared to when
lifestyles and the inception of the division of they digress away from the mean of their
labor – a key feature of modern human soci- own sex toward the mean of the other sex
eties. According to some, the propensity of (Figure 26.4). Therefore, the diagnosis of
early human males to engage in hunting and emotional disorders may vary between the
for early human females to engage in food sexes. Women present more often than men
gathering and child care developed into a with disorders of prosocial emotions. That
more diversified division of labor that even- is they are more likely than men to suf-
tually enabled modern humans to domes- fer from intense and prolonged feelings of
ticate animals, cultivate plants, and create fear or sadness. Men, in contrast, present
what we call “civilization” (Kuhn & Stiner, more often with disorders of antisocial emo-
2006). tions. They are more likely than women to
The evidence reviewed in this chapter be diagnosed with sociopathy or contact dis-
suggests that emotional specialization could order. To deal successfully with these condi-
have been part and parcel of this develop- tions, it is important to understand healthy
ment. Moreover, one may speculate that the emotional processing and its variants in men
emergence of male and female typical tasks and women. Although for a long time emo-
leveraged on sex-specific emotional disposi- tion research disregarded sex as an impor-
tions. For example, hunting, particularly of tant variable, this position has changed sig-
large game, required a certain readiness to nificantly in the last decade. During this
engage in aggression and to tolerate poten- time, researchers have identified chemical,
606 ANNETT SCHIRMER
Number of Individuals
cellular, gross structural, and functional dif- is a valid principle and whether there are
ferences between male and female brains other principles contributing to sex dif-
and shed light on the interplay between ferences are still open questions.
nature and nurture. From this evidence, r Currently we do not understand the
we now know that both innate mecha- relationship between sex differences in
nisms and environmental conditions shape gross brain morphology and brain func-
the human mind in a sex-specific manner. tion. Moreover, we know little about
Although interactions between both deter- sex differences in neuronal connectiv-
minants have been reported primarily for ity and neurochemical signaling. These
early developmental periods, there is indica- issues need to be addressed before we can
tion that they remain active throughout the fully appreciate why men and women feel
lifespan (Weaver, Meaney, & Szyf, 2006). differently.
Thus, a better understanding of the nature- r Our capacity to automatically regulate
nurture interplay promises new perspectives emotions based on goals or situational
on the prevention and treatment of both constraints creates a major challenge for
child and adult psychopathology. the investigation of emotions and associ-
ated sex differences. Future research on
emotional processing needs to consider
Outstanding Questions and Future emotion regulation effects and find means
Directions to model their influence.
Andreano, J. M., & Cahill, L. (2008). Menstrual tion and experience of disgust following brain
cycle modulation of the relationship between injury. Nature Neuroscience, 3, 1077–78.
cortisol and long-term memory. Psychoneu- Colapinto, J. (2000). As nature made him: The boy
roendocrinology, 33, 874–82. who was raised as a girl. New York: Harper-
Andreano, J. M., & Cahill, L. (2010). Menstrual Collins.
cycle modulation of medial temporal activity Craig, I. W., & Halton, K. E. (2009). Genetics of
evoked by negative emotion. Neuroimage, 53, human aggressive behaviour. Human Genet-
1286–93. ics, 126, 101–13.
Aleman, A., & Swart, M. (2008). Sex differ- Crijnen, A. A. M., Achenbach, T. M., & Ver-
ences in neural activation to facial expressions hulst, F. C. (1997). Comparisons of problems
denoting contempt and disgust. PLoS One, 3, reported by parents of children in 12 cultures:
e3622. Total problems, externalizing, and internaliz-
Alexander, G. M., & Hines, M. (2002). Sex ing. Journal of the American Academy of Child
differences in response to children’s toys in and Adolescent Psychiatry, 36, 1269–77.
nonhuman primates (Cercopithecus aethiops Curtis, V., Aunger, R., & Rabie, T. (2004). Evi-
sabaeus). Evolution and Human Behavior, 23, dence that disgust evolved to protect from risk
467–79. of disease. Proceedings of the Royal Society of
Archer, J. (2004). Sex differences in aggression in London, 271, 131–33.
real-world settings: A meta-analytic review. Dalla, C., & Shors, T. J. (2009). Sex differences
Review of General Psychology, 8, 291–322. in learning processes of classical and operant
Anokhin, A. P., & Golosheykin, S. (2010). Startle conditioning. Physiology & Behavior, 97, 229–
modulation by affective faces. Biological Psy- 38.
chology, 83, 37–40. Doster, J. A., Purdum, M. B., Martin, L. A.,
Becker, D. V., Kenrick, D. T., Neuberg, S. L., Goven, A. J., & Moorefield, R. (2009). Gen-
Blackwell, K. C., & Smith, D. M. (2007). The der differences, anger expression, and cardio-
confounded nature of angry men and happy vascular risk. Journal of Nervous and Mental
women. Journal of Personality and Social Psy- Disease, 197, 552–54.
chology, 92, 179–90. Drabant, E. M., McRae, K., Manuck, S. B., Hariri,
Becker, J. B. (2009). Sexual differentiation of A. R., & Gross, J. J. (2009). Individual differ-
motivation: A novel mechanism? Hormones ences in typical reappraisal use predict amyg-
and behavior, 55, 646–54. dala and prefrontal responses. Biological Psy-
Bradley, M. M., Codispoti, M., Cuthbert, B. N., chiatry, 65, 367–73.
& Lang, P. J. (2001). Emotion and motivation I: Eisenberger, N. I., Inagaki, T. K., Rameson,
Defensive and appetitive reactions in picture L. T., Mashal, N. M., & Irwin, M. R. (2009).
processing. Emotion, 1, 276–98. An fMRI study of cytokine-induced depressed
Bronson, F. H., & Desjardins, C. (1968). Aggres- mood and social pain: The role of sex differ-
sion in adult mice: Modification by neonatal ences. NeuroImage, 47, 881–90.
injections of gonadal hormones. Science, 161, Eisenberger, N. I., Lieberman, M. D., & Williams,
705–6. K. D. (2003). Does rejection hurt? An
Buchanan, T. W., Driscoll, D., Mowrer, S. M., fMRI study of social exclusion. Science, 302,
Sollers III, J. J., Thayer, J. F., Kirschbaum, C., 290–92.
et al. (2010). Medial prefrontal cortex damage Else-Quest, N. M., Hyde, J. S., Goldsmith, H. H.,
affects physiological and psychological stress & Van Hulle, C. A. (2006). Gender differences
responses differently in men and women. Psy- in temperament: A meta-analysis. Psychologi-
choneuroendocrinology, 35, 56–66. cal Bulletin, 132, 33–72.
Cahill, L., Haier, R. J., White, N. S., Fallon, J., Eugène, F., Lévesque, J., Mensour, B., Leroux,
Kilpatrick, L., Lawrence, C., et al. (2001). Sex- J. M., Beaudoin, G., Bourgouin, P., et al.
related difference in amygdala activity during (2003). The impact of individual differences
emotionally influenced memory storage. Neu- on the neural circuitry underlying sadness.
robiology of Learning and Memory, 75, 1–9. Neuroimage, 19, 354–64.
Cahill, L., & van Stegeren, A. (2003). Sex-related Federmeier, K. D., Kirson, D. A., Moreno,
impairment of memory for emotional events E. M., & Kutas, M. (2001). Effects of transient,
with ß-adrenergic blockade. Neurobiology of mild mood states on semantic memory orga-
Learning and Memory, 79, 81– 88. nization and use: An event-related potential
Calder, A. J., Keane, J., Manes, F., Antoun, N., investigation in humans. Neuroscience Letters,
& Young, A. W. (2000). Impaired recogni- 305(3), 149–52.
608 ANNETT SCHIRMER
Ferrari, P. F., Van Erp, A. M. M., Tornatzky, W., Kilpatrick, L. A., Zald, D. H., Pardo, J. V., &
& Miczek, K. A. (2003). Accumbal dopamine Cahill, L. (2006). Sex-related differences in
and serotonin in anticipation of the next amygdala functional connectivity during rest-
aggressive episode in rats. European Journal ing conditions. Neuroimage, 30, 452–61.
of Neuroscience, 17, 371–78. Kuhn, C., Johnson, M., Thomae, A., Luo, B.,
Figueira, M. L., Caeiro, L., Ferro, A., Sev- Simon, S. A., Zhou, G., et al. (2010). The
erino, L., Duarte, P. M., Abreu, M., et al. emergence of gonadal hormone influences on
(2008). Validation of the Temperament Eval- dopaminergic function during puberty. Hor-
uation of Memphis, Pisa, Paris and San Diego mones and Behavior, 58, 122–37.
(TEMPS-A): Portuguese-Lisbon version. Jour- Kuhn, S. L., & Stiner, M. C. (2006). What’sa
nal of Affective Disorders, 111(2–3), 193–203. mother to do? The division of labor among
Fleishman, E. G. (1983). Sex-role acquisition, Neandertals and modern humans in Eurasia.
parental behavior, and sexual orientation: Current Anthropology, 47, 953–80.
Some tentative hypotheses. Sex Roles, 9, 1051– Kimmel, M. S. (2000). The gendered society. New
59. York: Oxford University Press.
Fuller, B. F. (2002). Infant gender differences Labouvie-Vief, G., Lumley, M. A., Jain, E., &
regarding acute established pain. Clinical Heinze, H. (2003). Age and gender differ-
Nursing Research, 11, 190–203. ences in cardiac reactivity and subjective emo-
Garnefski, N., Teerds, J., Kraaij, V., Legerstee, tion responses to emotional autobiographical
J., & van den Kommer, T. (2004). Cognitive memories. Emotion, 3, 115–26.
emotion regulation strategies and depressive Labus, J. S., Naliboff, B. N., Fallon, J., Berman, S.
symptoms: Differences between males and M., Suyenobu, B., Bueller, J. A., et al. (2008).
females. Personality and Individual Differences, Sex differences in brain activity during aver-
36, 267–76. sive visceral stimulation and its expectation in
Goldstein, J. M., Seidman, L. J., Horton, N. patients with chronic abdominal pain: A net-
J., Makris, N., Kennedy, D. N., Caviness Jr, work analysis. Neuroimage, 41, 1032–43.
V. S., et al. (2001). Normal sexual dimorphism LaFrance, M., & Henley, N. M. (1994). On
of the adult human brain assessed by in vivo oppressing hypotheses: Or differences in non-
magnetic resonance imaging. Cerebral Cortex, verbal sensitivity revisited. In H. L. Radtke &
11, 490–97. H. J. Stam (Eds.), Power/gender: Social rela-
Gross, J. J., & John, O. P. (2003). Individual tions in theory and practice. Inquiries in social
differences in two emotion regulation pro- construction (pp. 287–311). London: Sage.
cesses: Implications for affect, relationships, Maccoby, E. E. (1998). The socialization compo-
and well-being. Journal of Personality and nent. In E. E. Maccoby (Ed.), The two sexes:
Social Psychology, 85, 348–62. Growing up apart, coming together (pp. 118–52).
Hamann, S. (2005). Sex differences in the Cambridge, MA: Belknap Press.
responses of the human amygdala. Neurosci- Maccoby, E. E., & Jacklin, C. N. (1974). The psy-
entist, 11, 288–93. chology of sex differences. Stanford, CA: Stan-
Hamann, S., Herman, R. A., Nolan, C. L., & ford University Press.
Wallen, K. (2004). Men and women differ in Mak, A. K., Hu, Z., Zhang, J. X., Xiao, Z., & Lee,
amygdala response to visual sexual stimuli. T. M. (2009). Sex-related differences in neural
Nature Neuroscience, 7, 411–16. activity during emotion regulation. Neuropsy-
Harmon-Jones, E., & Peterson, C. K. (2009). chologia, 47, 2900–08.
Supine body position reduces neural response Martin, V. T. (2009). Ovarian hormones and
to anger evocation. Psychological Science, 20, pain response: A review of clinical and basic
1209–10. science studies. Gender Medicine: Official
Hess, U., Adams Jr, R., & Kleck, R. (2005). Journal of the Partnership for Gender-Specific
Who may frown and who should smile? Dom- Medicine at Columbia University, 6(Suppl. 2),
inance, affiliation, and the display of hap- 168–92.
piness and anger. Cognition & Emotion, 19, McClure, E. B., Monk, C. S., Nelson, E. E.,
515–36. Zarahn, E., Leibenluft, E., Bilder, R. M.,
Jones, D. (2008). Killer instincts: What can evo- et al. (2004). A developmental examination of
lution say about why humans kill – and about gender differences in brain engagement dur-
why we do so less than we used to? Nature ing evaluation of threat. Biological Psychiatry,
451, 512–15. 55, 1047–55.
SEX DIFFERENCES IN EMOTION 609
McGaugh, J. L. (2004). The amygdala modulates during noxious heat stimulation in humans.
the consolidation of memories of emotionally Pain, 76, 223–29.
arousing experiences. Annual Review of Neu- Pinna, G., Costa, E., & Guidotti, A. (2005).
roscience, 27, 1–28. Changes in brain testosterone and allopreg-
McGowan, P. O., Sasaki, A., D’Alessio, A. C., nanolone biosynthesis elicit aggressive behav-
Dymov, S., Labonté, B., Szyf, M., et al. (2009). ior. Proceedings of the National Academy of Sci-
Epigenetic regulation of the glucocorticoid ences, 102, 2135–40.
receptor in human brain associates with child- Plato (2000). The republic. Mineola, NY: Dover
hood abuse. Nature Neuroscience, 12, 342–48. Publications
McRae, K., Hughes, B., Chopra, S., Gabrieli, Pompili, M., Girardi, P., Tatarelli, R., Iliceto, P.,
J. D., Gross, J. J., & Ochsner, K. N. (2010). The De Pisa, E., Tondo, L., et al. (2008). TEMPS-
neural bases of distraction and reappraisal. A (Rome): Psychometric validation of affec-
Journal of Cognitive Neuroscience, 22, 248–62. tive temperaments in clinically well subjects
McRae, K., Ochsner, K. N., Mauss, I. B., Gabrieli, in mid- and south Italy. Journal of Affective
J. J., & Gross, J. J. (2008). Gender differ- Disorders, 107, 63–75.
ences in emotion regulation: An fMRI study of Rogstad, J. E., & Rogers, R. (2008). Gender dif-
cognitive reappraisal. Group Processes & Inter- ferences in contributions of emotion to psy-
group Relations, 11, 143–62. chopathy and antisocial personality disorder.
Mechelli, A., Friston, K. J., Frackowiak, R. S., Clinical Psychology Review, 28, 1472–84.
& Price, C. J. (2005). Structural covariance in Romans, S. E., Tyas, J., Cohen, M. M., & Sil-
the human cortex. Journal of Neuroscience, 25, verstone, T. (2007). Gender differences in the
8303–10. symptoms of major depressive disorder. Jour-
Milad, M. R., Goldstein, J. M., Orr, S. P., Wedig, nal of Nervous and Mental Disease, 195, 905–11.
M. M., Klibanski, A., Pitman, R. K., & Rauch, Sabatinelli, D., Flaisch, T., Bradley, M. M.,
S. L. (2006). Fear conditioning and extinction: Fitzsimmons, J. R., & Lang, P. J. (2004). Affec-
Influence of sex and menstrual cycle in healthy tive picture perception: gender differences in
humans. Behavoural Neuroscience, 120, 1196– visual cortex? Neuroreport, 15, 1109–12.
1203. Safdar, S., Matsumoto, D., Kwantes, C.T.,
Milad, M. R., Zeidan, M. A., Contero, A., Pit- Friedlmeier, W., Yoo, S.H., Kakai, H., et al.
man, R. K., Klibanski, A., Rauch, S. L., & (2009). Variations of emotional display rules
Goldstein, J. M. (2010). The influence of within and across cultures: A comparison
gonadal hormones on conditioned fear extinc- between Canada, USA, and Japan. Canadian
tion in healthy humans. Neuroscience, 168, 652– Journal of Behavioural Science, 41, 1–10.
58. Schienle, A., Schäfer, A., Stark, R., Walter, B.,
Nelson, R. J. & Trainor, B. C. (2007). Neu- & Vaitl, D. (2005). Gender differences in the
ral mechanisms of aggression. Nature Reviews processing of disgust- and fear-inducing pic-
Neuroscience, 8, 536–46. tures: An fMRI study. Neuroreport, 16, 277–80.
Nolen-Hoeksema, S. (2001). Gender differences Schirmer, A., Escoffier, N., Li, Q. Y., Li, H.,
in depression. Current Directions in Psycholog- Strafford-Wilson, J., & Li, W.-I. (2008). What
ical Science, 10, 173–76. grabs his attention but not hers? Estrogen
Ochsner, K. N., & Gross, J. J. (2005). The cog- correlates with neurophysiological measures
nitive control of emotion. Trends in Cognitive of vocal change detection. Psychoneuroen-
Sciences, 9, 242–49. docrinology, 33, 718–27.
Oliveira, R. F., Lopes, M., Carneiro, L. A., & Schirmer, A., Kotz, S. A., & Friederici, A. D.
Canário, A. V. (2001). Watching fights raises (2002). Sex differentiates the role of emo-
fish hormone levels. Nature, 409, 475. tional prosody during word processing. Cog-
Pasterski, V. L., Geffner, M. E., Brain, C., Hind- nitive Brain Research, 14, 228–33.
marsh, P., Brook, C., & Hines, M. (2005). Pre- Schirmer, A., Simpson, E., & Escoffier, N. (2007).
natal hormones and postnatal socialization by Listen up! Processing of intensity change dif-
parents as determinants of male-typical toy fers for vocal and nonvocal sounds. Brain
play in girls with congenital adrenal hyperpla- Research, 1176, 103–12.
sia. Child Development, 76, 264–78. Schirmer, A., Striano, T., & Friederici, A.D.
Paulson, P. E., Minoshima, S., Morrow, T. J., & (2005). Sex differences in the pre-attentive
Casey, K. L. (1998). Gender differences in pain processing of vocal emotional expressions.
perception and patterns of cerebral activation Neuroreport, 16, 635–39.
610 ANNETT SCHIRMER
Simonoff, E., Pickles, A., Meyer, J. M., Silberg, affect. Social Cognitive and Affective Neuro-
J. L., Maes, H. H., Loeber, R., et al. (1997). science, 4, 328–39.
The Virginia Twin Study of Adolescent Whitbeck, C. (1976). Theories of sex difference.
Behavioral Development: Influences of age, In C. C. Gould, & M. W. Wartofsky (Eds.),
sex, and impairment on rates of disorder. Women and philosophy: Toward a theory of lib-
Archives of General Psychiatry, 54, 801–8. eration (pp. 54–80). New York: Putnam.
Singer, T., Seymour, B., O’Doherty, J. P., Wicker, B., Keysers, C., Plailly, J., Royet, J. P.,
Stephan, K. E., Dolan, R. J., & Frith, C. D. Gallese, V., & Rizzolatti, G. (2003). Both of us
(2006). Empathic neural responses are mod- disgusted in My insula: The common neural
ulated by the perceived fairness of others. basis of seeing and feeling disgust. Neuron, 40,
Nature, 439, 466–69. 655–64.
Stanton, S. J., Wirth, M. M., Waugh, C. E., & Will, J. A., Self, A., & Datan, N. Maternal behav-
Schultheiss, O. C. (2009). Endogenous testos- ior and perceived sex of infant. American Jour-
terone levels are associated with amygdala and nal of Orthopsychiatry, 46(1), 135–39.
ventromedial prefrontal cortex responses to Williams, D. G., & Morris, G. H. (1996). Crying,
anger faces in men but not women. Biological weeping or tearfulness in British and Israeli
Psychology, 81, 118–22. adults. British Journal of Psychology, 87, 479–
Taylor, S. E., Klein, L. C., Lewis, B. P., Grue- 505.
newald, T. L., Gurung, R. A. R., & Updegraff, Williams, L. M., Barton, M. J., Kemp, A. H.,
J. A. (2000). Biobehavioral responses to stress Liddell, B. J., Peduto, A., Gordon, E., et al.
in females: Tend-and-befriend, not fight-or- (2005). Distinct amygdala-autonomic arousal
flight. Psychological Review, 107, 411–29. profiles in response to fear signals in healthy
van Tilburg, M. A., Becht, M. C., & Vingerhoets, males and females. Neuroimage, 28, 618–26.
A. J. (2003). Self-reported crying during the Wise, R. A. (2004). Dopamine, learning and moti-
menstrual cycle: Sign of discomfort and emo- vation. Nature Reviews Neuroscience, 5, 483–
tional turmoil or erroneous beliefs? Journal of 94.
Psychosomatic Obstetrics and Gynaecology, 24, Wood, G. E., & Shors, T. J. (1998). Stress
247–55. facilitates classical conditioning in males,
van Wingen, G.A., Ossewaarde, L., Bäckström, but impairs classical conditioning in females
T., Hermans, E. J., Fernández, G. (2011). through activational effects of ovarian hor-
Gonadal hormone regulation of the emotion mones. Proceedings of the National Academy
circuitry in humans. Neuroscience, 191, 38–45. of Sciences, 95, 4066–71.
Weaver, I. C., Meaney, M. J., & Szyf, M. (2006). Zhang, T.Y. & Meaney, M. J. (2010). Epigenet-
Maternal care effects on the hippocampal ics and the environmental regulation of the
transcriptome and anxiety-mediated behav- genome and its function. Annual Review of
iors in the offspring that are reversible Psychology, 61, 439–66.
in adulthood. Proceedings of the National Zubieta, J. K., Ketter, T. A., Bueller, J. A., Xu, Y.,
Academy of Sciences, 103, 3480–85. Kilbourn, M. R., Young, E. A., et al. (2003).
Welborn, B. L., Papademetris, X., Reis, D. L., Regulation of human affective responses by
Rajeevan, N., Bloise, S. M., & Gray, J. R. anterior cingulate and limbic mu-opioid neu-
(2009). Variation in orbitofrontal cortex vol- rotransmission. Archives of General Psychiatry,
ume: Relation to sex, emotion regulation and 60, 1145–53.
CHAPTER 27
611
612 ESSI VIDING, CATHERINE L. SEBASTIAN, & EAMON J. MCCRORY
of the emotion brain systems can be found 16 and 24). The areas that have received most
in Chapters 19 (striatum), 18 (amygdala), 16, attention in affective neuroscience studies
and 24 (prefrontal cortex), this section pro- of children include the orbitofrontal cor-
vides only a schematic overview for the pur- tex (OFC); ventromedial prefrontal cor-
poses of anchoring the developmental find- tex (vmPFC); ventrolateral prefrontal cor-
ings presented in this chapter. tex (VLPFC); and dorsolateral prefrontal
cortex (DLPFC). The OFC is thought
to implement rapid stimulus-reinforcement
The Amygdala
associations and the correction of these asso-
The amygdala is a subcortical region that is ciations when the contingencies of rein-
important for processing the current value forcement change (e.g., Mitchell, Richell,
of stimul (Adolphs, 2010). This structure Pine, & Blair, 2008; O’Doherty, 2004; Rolls
has a critical role in several affective pro- & Grabenhorst, 2008), whereas the vmPFC
cesses, such as mediating conditioned emo- is thought to, among other functions, rep-
tional responses, responding to various emo- resent the elementary positive and negative
tional stimuli (including facial expressions affective states in the absence of immedi-
of emotion), and influencing social behavior ately present incentives, as well as being
toward conspecifics (see, e.g., Adolphs, 2010; important for encoding outcome expecta-
Sergerie, Chochol, & Armony, 2008; and tions (e.g., Davidson & Irwin, 1999; Mitchell,
Chapter 1). 2011). A more rostral region of the medial
PFC has been implicated in the process-
ing of more complex social emotions such
The Striatum
as guilt and embarrassment (Burnett, Bird,
The striatum is a subcortical region that Moll, Frith, & Blakemore, 2009; Takahashi et
plays a role in modulating behavior toward al., 2004). Current research suggests that the
potentially rewarding stimuli, particularly VLPFC integrates affective information and
stimuli that hold a high subjective reward supports response selection by increasing
value to an individual (see, e.g., Peters & the salience of alternative motor response
Buchel, 2010; Rosen & Levenson, 2009; and option representations through interactions
Chapter 19). with the striatum (e.g., Mitchell, 2011;
Mitchell et al., 2008). It is also associated
with effortful regulation of negative affect
The Anterior Cingulate Cortex
via connections with subcortical structures
The anterior cingulate cortex (ACC) is including the amygdala (Ochsner & Gross,
thought to play a distinct role in com- 2005; Wager, Davidson, Hughes, Lindquist,
plex aspects of emotion, such as processing & Ochsner, 2008). The DLPFC, in turn, is
moral emotions (e.g., guilt), self-regulation thought to increase attentional control of
of negative emotions, and action reinforce- task-relevant stimulus features and repre-
ment (the route by which reward history sent goal states toward which more elemen-
influences action choice; see, e.g., Kédia, tary positive and negative affective states are
Berthoz, Wessa, Hilton, & Martinot, 2008; directed (Mitchell, 2011).
Levesque et al., 2004; Rushworth, Behrens, This, of course, represents only an
Rudebeck & Walton, 2007; and Chapters 16 extremely succinct overview of the brain
and 24). areas (and their putative functions) that
form the brain’s affective circuitry. We
alluded only briefly to the phenomenon of
The Prefrontal Cortex
connectivity in the PFC section. However,
Various sectors of the prefrontal cortex most of the brain’s affective processes are
(PFC) have been implicated in emotion achieved by functional integration across
(see e.g., Davidson, Jackson, & Kalin, 2000; several brain areas. That is, although this
Vuilleumier & Pourtois, 2007; and Chapters section has presented broad functions for
DEVELOPMENT OF AFFECTIVE CIRCUITRY 613
individual brain areas, many of these areas social signals such as emotional facial expres-
work together to achieve an appropriate sions (Leppanen & Nelson, 2009). Other
behavioral outcome for the individual. For people’s facial expressions can provide both
example, the amygdala, together with the salient social communication and vital clues
PFC and ventral striatum, forms a network about the nature of the surrounding environ-
of structures involved in processing the cur- ment (see Chapter 7). For example, some-
rent value of stimuli, and various PFC/ACC one displaying a fearful expression might sig-
regions are connected either directly or nal to an observer the presence of a potential
indirectly with the amygdala to achieve and previously unnoticed threat. Being able
emotion regulation via numerous mecha- to detect and attach appropriate valence to
nisms, such as reappraisal. such cues is likely to have strong adaptive
value (Leppanen & Nelson, 2009).
dip” in performance on a match-to-sample OFC more than did adults (aged 25–36).
task in which participants had to match Similarly, when participants were asked to
emotional faces to words. Reaction times attend to a nonemotional aspect of the stim-
were 10–20% longer at the onset of puberty uli (e.g., “how wide is the nose?”), ACC
for both males and females than in younger activation was greater in the adolescents
children. Performance only regained its pre- than in the adults. However, when partic-
pubertal level at 16–17 years. This dip may ipants were instructed to rate how afraid
be caused by neural reorganization occur- they felt on viewing each face, adults acti-
ring around the onset of puberty. However, vated the right OFC more than adolescents.
the pubertal dip effect has not been repli- The authors suggested that adults were able
cated with simple emotion expression recog- to modulate their neural responses based
nition, and the differential effects of puberty on task demands, with the OFC selectively
and chronological age on the development engaged in response to the requirement to
and functioning of emotional brain circuitry focus on emotional content. By contrast,
in humans are not yet known. adolescents’ responses were modulated by
Several recent studies have used fMRI the emotional nature of the stimulus across
to identify the neural substrates associated tasks, suggesting that this age group may find
with affective processing in children and it harder to disengage from affectively salient
adolescents. One of the first studies com- information, regardless of task demands.
pared young adolescents (mean age 11 years) These findings highlight the importance of
with adults using fMRI during the pas- considering the context in which an emo-
sive viewing of fearful and neutral faces tional response is measured, because differ-
(Thomas et al., 2001). It was found that, ent task demands may differentially engage
whereas adults activated the amygdala more subcortical and cortical affective structures.
in response to fearful faces than neutral ones, Sex differences further complicate the
adolescents showed the opposite response. emerging picture of emotional brain devel-
This was possibly because neutral faces opment (see Chapter 26). At a behavioral
were interpreted as more ambiguous and level, there is a small female advantage
therefore more threatening in the adoles- in emotional expression recognition across
cent group or possibly because the amyg- development, from infancy to adolescence
dala is less selective earlier in development. (McClure, 2000). Some studies have found
However, there are inconsistent findings evidence of a differential response to emo-
regarding amygdala response in adolescence. tional faces between the sexes in the amyg-
For example, Guyer and colleagues found dala and PFC, although they disagree on
greater amygdala activation to fearful faces the precise nature of these differences and
in adolescents (aged 9–17 years) compared to the timing with which they emerge. For
adults (Guyer et al. 2008). This finding is in example, one study reported an increase
line with recent theories of adolescent neu- in PFC response to fearful faces with age
rocognitive development that suggest that in girls aged 9 to 17, but no relationship
activity in subcortical structures such as with age in boys (Killgore, Oki, & Yurgelun-
the amygdala and ventral striatum are not Todd, 2001). Another study found no differ-
adequately regulated by prefrontal struc- ences in the neural response to facial expres-
tures during early and mid-adolescence (e.g., sions between adolescent girls and boys aged
Nelson et al., 2005). 9–17, but did find a greater response to angry
It should be noted that task demands faces in adult females relative to males in
influence the nature of differential responses the OFC and amygdala (McClure et al.,
between adolescents and adults during facial 2004), suggesting that sex differences emerge
expression processing (Monk et al., 2003). between adolescence and adulthood. Study-
During passive viewing of fearful faces (rel- ing the possible neural basis of sex differ-
ative to neutral faces), adolescents (aged ences in facial emotion perception across
9–17) activated the amygdala, ACC, and the entire developmental time course and
616 ESSI VIDING, CATHERINE L. SEBASTIAN, & EAMON J. MCCRORY
Figure 27.1. Recent studies suggest that development of the functional neural bases of complex
affective processing continues between adolescence and adulthood. This figure shows a region of the
ventromedial prefrontal cortex (peak voxel: −10 46 8) that responded to a greater extent in adolescent
males (mean age 14.1 years; n = 15) than in matched adult controls during the presentation of cartoon
scenarios requiring affective Theory of Mind (understanding emotions in a social context) relative to
those requiring cause-and-effect reasoning (physical causality). Group differences in this region were
not seen in response to scenarios requiring cognitive Theory of Mind (mentalizing), suggesting that
integrating affective and social cognition may represent a unique challenge for the developing
adolescent brain. Figure adapted from Sebastian et al. (2011). See color plate 27.1.
as well as broaden the range of paradigms affect regulation, because it depends on con-
that are used; for example, to routinely nectivity between subcortical and cortical
include stimuli from a wide range of modali- structures, such as the amygdala and pre-
ties (such as emotional voices and body pos- frontal cortex.
tures). Increasing use should be made of eco-
logically valid paradigms measuring social
interaction. Furthermore, most functional Development of Neural Circuitry
studies have tended to be cross-sectional in Subserving Affect Regulation
nature; future work should include longi-
tudinal studies that are able to chart the It has been suggested that “emotion regu-
development of emotion processing within lation consists of the extrinsic and intrinsic
an individual over time. It is also necessary to processes responsible for monitoring, eval-
investigate developmental changes in con- uating and modifying emotional reactions”
nectivity between affective brain regions, (Thompson, 1994, p. 27). As such, it is some-
and not just regional differences with age. what artificial to separate a discussion of
One area of research in which this approach emotion regulation from that of emotion
will be particularly informative is that of itself, because these processes “co-evolve”
618 ESSI VIDING, CATHERINE L. SEBASTIAN, & EAMON J. MCCRORY
via bidirectional connections between lim- ful faces, which may signify threat (Nelson
bic and cortical regions (Lewis & Stieben, & de Haan, 1996). However, this component
2004). The following section explores the reflects only an automatic response to poten-
development of emotion regulation, from tial threat. It has been argued that delib-
the inflexible automatic resources available erate self-regulation does not develop until
to the infant to the emergence of more the third year of life and that it depends
effortful and consciously directed cogni- to a large extent on the development of
tive strategies that can modulate affective effortful control, or the ability to inhibit
responses (for a fuller discussion of emotion a prepotent response in order to achieve
regulation in adults, see Chapters 16 and 24). a goal (Posner & Rothbart, 2000). Tasks
that require conflict resolution, such as the
Stroop task or the go/no-go task, are com-
Infancy
monly used as an index of inhibitory control.
In terms of the definition of emotion regu- Successful performance on age-appropriate
lation just stated, infants are more depen- versions of these tasks emerges at around 30
dent on extrinsic compared with intrinsic months and is associated with lower nega-
resources than at any other point in the tive affect even at this early age (Gerardi-
lifespan. It therefore follows that the rela- Caulton, 2000). The next section focuses on
tionship between the infant and the primary two key approaches to emotion regulation:
caregiver is likely to play a critical role, with the development of inhibitory control and
the primary caregiver helping to “scaffold” its neural bases, and the use of explicit emo-
the development of the infant’s affective and tion regulation strategies in childhood and
regulatory circuitry (Fonagy, Gergely, Jurist adolescence.
& Target, 2002). For example, a caregiver
may be able to reduce distress by holding or
Childhood and Adolescence
rocking the infant or by distracting his or her
attention away from the cause of distress. Inhibitory control is mediated by a number
With time, the infant builds on this train- of prefrontal structures including the dorso-
ing and is able to self-sooth, possibly via the lateral, ventrolateral, and ventromedial PFC
development of a mid-frontal cognitive con- and the ACC. Behavioral and neuroimag-
trol system (Posner & Rothbart, 2000). Thus, ing studies demonstrate that inhibitory
the quality of the infant-caregiver relation- control continues to develop throughout
ship is typically crucial to the successful childhood and adolescence (e.g., David-
early development of affect regulation sys- son, Amso, Anderson, & Diamond, 2006)
tems. Future research should be tailored to and that the prefrontal response during
explicitly test, as well as advance, our cur- inhibitory tasks becomes more efficient
rent models of attachment, reflective func- between childhood and adolescence (Lewis,
tioning, affect mirroring, and agency to pro- Lamm, Segalowitz, Stieben, & Zelazo, 2006)
vide a coherent framework within which we and then between adolescence and adult-
can make sense of the neurobiological cor- hood (Luna et al., 2001). However, there
relates of affective response and regulation is some debate as to the trajectory of
(Fonagy et al., 2002). this development. Studies using standard
By the middle of the first year, there is inhibitory control tasks without an affec-
evidence of a rudimentary system for ori- tive component, such as the go/no-go and
enting attention to salient affective stimuli. Simon tasks, have demonstrated a linear
The evidence we have cited in relation to improvement with age (Davidson et al.,
affective processing in infancy suggests that, 2006). However, as discussed later, there is
by the age of 7 months, cognitive control evidence that a linear trend may not ade-
regions such as the ACC (as indexed by the quately describe the developmental trajec-
Nc component) seem to be engaged in the tory of inhibitory control of affect, par-
preferential allocation of attention to fear- ticularly in paradigms designed to emulate
DEVELOPMENT OF AFFECTIVE CIRCUITRY 619
emotionally charged situations with “real- ing adolescence, in the absence of a sim-
world” validity (Somerville & Casey, 2010). ilarly up-regulated response in regulatory
Recent models of adolescence suggest regions. These data are complemented by
that development of prefrontal regulatory studies that have shown a decreased reg-
structures lags behind that of the limbic ulatory response in prefrontal regions dur-
structures that mediate an initial affective ing affective tasks in adolescents relative to
response (e.g., Nelson et al., 2005). These adults. Both Eshel, Nelson, Blair, Pine, and
models would predict that inhibitory control Ernst (2007) and Sebastian et al. (2011) have
of an affective response follows a U-shaped found evidence for a reduced response in
trajectory, with less effective PFC regulation ventrolateral PFC in adolescents compared
of subcortical structures such as the amyg- with adults: the former during a gambling
dala in adolescence relative to childhood and task and the latter during a task measuring
adulthood. One recent fMRI study inves- neural responses to social rejection.
tigated age-related differences in PFC and These studies have focused on the pre-
amygdala function while participants (aged frontal modulation of subcortical structures
7–32) completed an emotional go/no-go task during tasks requiring effortful control in the
involving fearful, happy, and calm facial presence of affective stimuli. Although par-
expressions (Hare et al., 2008). Amygdala ticipants must explicitly direct their atten-
reactivity to fearful faces was greater in ado- tion to a particular aspect of the stimuli to
lescents than in children or adults, and it cor- successfully complete these tasks, regulation
related with reaction time delays to fearful of their own affective response is an implicit
relative to happy expressions. Activity in the requirement. Individuals are also able to
ventral PFC, a regulatory region, was nega- consciously engage these neural systems for
tively correlated with reaction time differ- the explicit purpose of emotion regulation:
ence, but did not vary with age. The authors Learning to consciously modulate an emo-
suggested that increased limbic activation tional response is vital for successful social-
relative to ventral PFC regulation could con- ization (Posner & Rothbart, 2000). Accord-
tribute to the increased emotional reactivity ing to Gross’s (1998) process-oriented
and poor decision making associated with approach, a number of strategies for emo-
adolescence. tion regulation are available, of which
Similar conclusions have been reached by two, suppression and reappraisal, have
studies that have used gambling tasks to received considerable empirical attention.
investigate the striatal response to reward Suppression involves directly inhibiting an
across development. One such study (Van emotional response, whereas reappraisal
Leijenhorst et al., 2010) found an exag- involves interpreting an emotion-eliciting
gerated response to anticipated reward in experience in a more positive light and is
the striatum in 14–15 year olds, relative to thought to be more successful in repairing
both 10–12 year olds and 18–23 year olds. negative moods than suppression.
Another study found that this increased stri- Developmentally, the use of a suppres-
atal response was driven by greater predic- sion strategy has been shown to decrease
tion error signals in the striatum in adoles- with age between 9 and 15 (Gullone,
cents relative to children or adults (Cohen Hughes, King, & Tonge, 2010), in line with
et al., 2010). The authors suggested that the idea that more adaptive emotion regula-
this increased response may render ado- tion strategies are employed with increas-
lescents more sensitive to potentially pos- ing maturation. Surprisingly, Gullone and
itive outcomes (or less sensitive to poten- colleagues found that self-reported use of
tially negative outcomes) and thus may reappraisal also decreased over this same age
motivate them to take more risks than at range. It may be that such strategies become
other ages. The pattern emerging from these more automatic and less effortful with
studies is that of an increased subcortical age and therefore become less readily
response to emotion-eliciting stimuli dur- amenable to investigation with self-report
620 ESSI VIDING, CATHERINE L. SEBASTIAN, & EAMON J. MCCRORY
measures. Few studies have investigated of analyses (see the case study in the next
neurocognitive development of these strate- section). Finally, we consider how affective
gies. One fMRI study by Levesque et al. neuroscience can have a translational impact
(2004) investigated the neural response to in the field of developmental psychopatho-
sad film clips in 8- to 10-year-old girls logy.
during both passive viewing and under
instructions to use a reappraisal strategy.
The Impact of Genotype Differences
Reappraisal was associated with greater
recruitment of several prefrontal loci, In the context of individual differences in
including the lateral, ventrolateral, orbital, typical development, as well as in devel-
and medial PFC, as well as ACC. A similar opmental psychopathology, genes represent
earlier study involving adult women found the cornerstone of mechanisms that either
that fewer prefrontal loci were active in this directly or in concert with environmen-
contrast (Levesque et al., 2003), and it may tal events ultimately result in disease (see
be that this more widespread response in Chapter 25). It is thought that these genetic
the developmental sample reflects immatu- variants act across the lifespan by biasing the
rity of prefrontal control. functioning of several brain and hormonal
In summary, there is evidence for pro- circuits that are crucial for effecting a stress
tracted behavioral and neural development response. Although most human behav-
of affective processing and emotion regula- iors cannot be explained by genes alone,
tion. The following section explores indi- and certainly much variance in aspects of
vidual differences in affective processing affective processing is not genetically deter-
during development and discusses how vul- mined directly, it is anticipated that vari-
nerabilities in these systems may give rise ations in the genetic sequence that affect
to developmental disorders of affective cir- gene function will contribute an apprecia-
cuitry, with special reference to conduct dis- ble amount of variance to these resultant
order and anxiety. complex behavioral phenomena. This con-
clusion is implicit in the results of studies of
twins that have revealed heritabilities rang-
Individual Differences in Affective ing from 40 to 70% for various aspects of cog-
Development nition, temperament, and personality. Gene
effects on behavior are mediated by their
Clearly there are individual differences in molecular and cellular effects on informa-
emotion processing and regulation styles and tion processing in the brain. Thus, examin-
abilities and in how these styles and abilities ing gene effects on the brain represents a
develop (e.g., Braver, Cole, & Yarkoni, 2010). critical step in understanding their ultimate
Disturbances in emotion processing and reg- contribution to variability in behavior and in
ulation and in the related circuitry are also the development of psychopathology.
a hallmark of many forms of developmental Because genes are directly involved in
psychopathology. In this section, we first dis- the development and function of brain
cuss how genetic and environmental risk fac- regions subserving specific cognitive and
tors can alter the development of those brain emotional processes, functional polymor-
areas involved in affect processing and regu- phisms in genes may be strongly related to
lation. We also outline periods of vulnerabil- the function of these specific neural systems
ity within different parts of the affective cir- and in turn, may mediate/moderate their
cuitry – to the extent that they are known for involvement in behavioral outcomes. This is
humans. Antisocial behavior in childhood is the underlying assumption of investigations
used as an example of how affective neu- examining the relation between genes and
roscience research can inform the develop- neural systems, known as “imaging genetics”
mental psychopathology evidence base, par- (see Chapter 25). Imaging genetics within
ticularly when combined with other levels the context of a candidate gene association
DEVELOPMENT OF AFFECTIVE CIRCUITRY 621
long before the physiological associations ferences in the development of affective cir-
were captured via neuroimaging in adult- cuits and consequent individual differences
hood (i.e., they represent “ghosts in the in vulnerability to affective psychopathol-
machine”). One avenue of fruitful research ogy. They may be particularly useful in pro-
into these developing systems is the use of viding a better understanding of “dormant
longitudinal studies beginning in childhood. vulnerabilities” and how such vulnerabilities
This approach represents an ideal way to may expose an individual to a maladaptive
examine the impact of genetic and environ- outcome when he or she experiences envi-
mental effects on the developing neural cir- ronmental risk.
cuitry that supports behavior and confers
risk for psychopathology (environmental
A Case Study: Childhood Antisocial
effects on affective circuitry are reviewed
Behavior
in the next section of the chapter). Such
an approach allows for the determination of Differences in several brain areas and
genetically driven variation on structural and information-processing functions associated
functional brain development during win- with perception and the regulation of emo-
dows of time that reflect critical matura- tions have been implicated in adult pop-
tional processes (e.g., myelination, synaptic ulations with antisocial behavior (Blair,
pruning). 2010). In particular, the OFC/ventral PFC,
A study by Shaw et al. (2009) examined ACC, amygdala, and interconnected regions
the effects of the 7-repeat microsatellite in have shown both structural and functional
the dopamine receptor 4 (DRD4) gene on abnormalities in these individuals (Blair,
clinical outcome and cortical development 2010). These abnormalities likely reflect both
in ADHD throughout childhood and ado- genetic vulnerability and environmental risk
lescence. They found that possession of the for antisocial behavior.
DRD4 7-repeat allele was associated with In children, both structural MRI and
a thinner right OFC/inferior prefrontal and fMRI have only been available relatively
posterior parietal cortex. Among the chil- recently for the study of antisocial behav-
dren with ADHD, those carrying the DRD4 ior; as a result, only a handful of devel-
7-repeat allele had a better clinical outcome opmentally informative studies have been
and a distinct trajectory of cortical devel- conducted to date. Findings from these
opment. They showed normalization of the studies are generally in line with studies
right parietal cortical region, a pattern that of adult antisocial populations and indi-
has been linked with better clinical outcome cate that abnormalities in several regions,
in earlier studies. The authors concluded including the amygdala, ACC, and OFC, are
that the DRD4 7-repeat allele, which has already observable in childhood (Sterzer &
been associated with a diagnosis of ADHD Stadler, 2009).
and better clinical outcome, is associated Imaging genetic research on adults sug-
with cortical thinning in regions important gests that genetic variants known to predis-
in attentional control. This regional thinning pose to antisocial behavior, such as MAOA,
was most apparent in childhood and largely also influence the structure and functioning
resolved during adolescence. Although it did of several key areas in the emotion process-
not directly assess the development of neu- ing circuitry, including the amygdala, ACC,
ral circuitry supporting affective process- and OFC (Buckholtz & Meyer-Lindenberg,
ing or affect regulation, this study high- 2009). In addition, research with child sam-
lighted the wealth of information that can be ples suggests that environmental risk factors
gleaned from longitudinal imaging genetic known to predispose to antisocial behav-
data. ior, such as maltreatment, also affect the
Developmental imaging genetic ap- structural development of the brain’s affec-
proaches hold substantial promise in in- tive circuitry, including the OFC (McCrory,
creasing our understanding of individual dif- DeBrito, & Viding, 2010).
DEVELOPMENT OF AFFECTIVE CIRCUITRY 623
It is clear from behavioral data that chil- to others’ distress (Jones, Laurens, Herba,
dren who display antisocial behavior cannot Barker, & Viding, 2009; Marsh et al., 2008).
be regarded as a homogeneous group (Vid- Both studies employed an implicit emotion-
ing, McCrory, Blakemore, & Frederickson, processing task (gender recognition) and
2011). Affective neuroscience research can indeed found amygdala hypo-reactivity to
help establish whether there are distinct tra- fearful faces in antisocial children with CU+
jectories to antisocial behavior and whether compared to typically developing children
they are underpinned by different neu- and/or children with ADHD. This variable
rocognitive vulnerabilities. Such research pattern of findings across studies and stimu-
has the genuine scope to provide researchers lus types warrants further investigation and
and practitioners with a developmentally may partly reflect the heterogeneous nature
informed model of persistent antisocial of the samples that have been studied. A
behavior that can in turn shape approaches recent study from our group demonstrated
to intervention (Viding et al., 2011). that amygdala activity to emotional stim-
The current evidence base suggests that at uli can vary as a function of CU traits in
least two subgroups of children with early- children with antisocial behavior (Sebastian
onset antisocial behavior can be delineated et al., 2012).
on the basis of their affective processing These preliminary findings from brain
style; one characterized by emotional under- imaging studies are promising because they
reactivity (callous-unemotional; CU+) and help illuminate the neurological basis of
another by emotional over-reactivity (non- some of the affective differences found
callous, CU−; Viding et al., 2011). Exist- across the different developmental path-
ing neuroimaging studies provide tenta- ways to antisocial behavior (e.g., CU+ and
tive support for this distinction (Sterzer CU−). For example, they can help make
& Stadler, 2009; Viding et al., 2011). A sense of the different patterns of antisocial
selective overview of some findings relat- behavior seen in the CU+ and CU− groups,
ing to amygdala functioning in children with the CU+ group showing significantly
with antisocial behavior illustrates the cur- more premeditated aggression and the
rent state of the field. Sterzer et al. (2005) CU− group having a predominant problem
were the first to conduct fMRI research in controlling reactive aggression (Viding
in youth with antisocial behavior. They et al., 2011). Refining the affective neuro-
reported that as a group youth with anti- science phenotypes for different subtypes of
social behavior showed amygdala hypo- antisocial behavior is important in under-
reactivity to passively viewed negative and standing the neurocognitive mechanisms
threatening pictures, but this result emerged through which inherited predispositions and
only when anxiety scores were used as a environmental adversity can place a child at
covariate. Another fMRI study using a simi- risk for the development of antisocial traits.
lar paradigm found increased amygdala acti- Combining affective neuroscience data with
vation, partly related to comorbid anxiety, other levels of analyses (e.g., genetics and
in children with antisocial behavior (Her- the environment) and using longitudinal
pertz et al., 2008). A more recent study using data offer great promise for advancing our
emotional face stimuli found evidence of knowledge of the different pathways to the
amygdala hypo-reactivity to sad, but hyper- same diagnostic outcome.
reactivity to neutral faces in adolescents
and young adults with antisocial behav-
ior (Passamonti et al., 2010). In addition to Environmental Influences on Affective
these studies focusing on antisocial behav- Development
ior in general, two studies to date have
explicitly recruited antisocial children with It is now well established from both animal
CU+ and tested the hypothesis that this and human studies that psychosocial stress
group would show amygdala hypo-reactivity and poor caregiving can have a significant
624 ESSI VIDING, CATHERINE L. SEBASTIAN, & EAMON J. MCCRORY
2008, for a comprehensive review). One treated individuals presenting with depres-
possibility is that exposure to early adver- sion. Both effects may reflect adaptations of
sity in these children is associated with stress the HPA axis in response to different forms
habituation over time, a pattern that may of maltreatment and perhaps different peri-
be linked to their difficulties in emotional ods of onset and chronicity.
and behavioral regulation; equally, reduced
stress responsivity may emerge as a result of
Maltreatment and Brain Structure: the
genetic factors or gene–environment inter-
Importance of Timing
actions (van Goozen & Fairchild, 2008). A
recent meta-analysis has suggested that pat- A growing number of MRI and DTI studies
terns of diurnal and morning cortisol lev- have now been conducted both in children
els may vary depending on the type of who have experienced maltreatment and in
antisocial behavior, patterns of internaliz- adults with histories of childhood abuse;
ing comorbidity, and early environmental this work, reviewed more fully elsewhere,
adversity (Hawes, Brennan, & Dadds, 2009). provides evidence of measurable changes in
In instances when parenting is compromised both gray and white matter in these groups
but where there is no maltreatment – for (McCrory et al., 2010, 2011). There is now
example, in the context of maternal depres- reliable evidence that a reduction in the
sion – there appears to be a link with atyp- volume of the corpus callosum is associ-
ical HPA activity. Halligan and colleagues ated with maltreatment; this white matter
found that morning salivary cortisol in ado- structure controls interhemispheric com-
lescents who had been exposed to maternal munication of a host of processes, includ-
postnatal depression was elevated and more ing, but not limited to, arousal, emotion,
variable compared to nonexposed peers, a and higher cognitive abilities (Giedd et al.,
pattern associated with an increased risk of 1996). Findings for the amygdala are more
depression (Halligan, Herbert, Goodyer, & mixed. Although two recent studies have
Murray, 2004). reported an increase in amygdala volumes
There is now increasing support for the (Mehta et al., 2009; Tottenham et al.,
hypothesis that that the association between 2010) a meta-analysis of children with
early adversity (in the form of compromised maltreatment-related PTSD did not find sig-
maternal care, including maltreatment) and nificant differences between maltreated and
a heightened risk for later psychopathology non-maltreated children (Woon & Hedges,
is associated with atypical HPA functioning 2008). It is notable that increased amyg-
(McCrory et al., 2010). For example, Heim dala volume has been reported in the con-
and colleagues in a convincing review of the text only of institutional rearing in infancy
evidence make a strong case that childhood before being adopted into normative envi-
maltreatment increases the risk of develop- ronments (Tottenham et al., 2010). These
ing depression in adulthood due to the sen- children were therefore likely to have been
sitization of the HPA system (Heim, New- exposed to very early experiences of stress
port, Mletzko, Miller, & Nemeroff, 2008). at a time when the amygdala was still struc-
By contrast, increased risk for posttrau- turally immature. This interpretation is con-
matic stress disorder (PTSD) may be asso- sistent with findings from the animal liter-
ciated with hypocortisolism; Meewisse and ature. For example, in monkeys, maternal
colleagues noted the relationship between separation at 1 week of age had differen-
lower cortisol levels and PTSD in the con- tial effects on behavior and gene expression
text of physical and sexual forms of abuse patterns in the amygdala compared with
(Meewisse, Reitsma, De Vries, Gersons, & maternal separation at 1 month (Sabatini
Olff, 2007). These findings indicate a possi- et al., 2007), although both groups of sep-
ble dissociation, with HPA hypo-activity in arated monkeys showed aberrant social and
those with maltreatment-related PTSD, but emotional behavior relative to nonseparated
hyper-activity of the HPA system in mal- monkeys. This finding suggests not only that
626 ESSI VIDING, CATHERINE L. SEBASTIAN, & EAMON J. MCCRORY
the amygdala is sensitive to stress early in ods in development; that is, a time when
life but also that even a difference in timing a neural system or brain region is espe-
of a few weeks is enough to produce mea- cially plastic and therefore more suscepti-
surable effects on its integrity. It is possi- ble to environmental influence (Tottenham
ble that such experience-dependent alter- & Sheridan, 2010). Previously we noted that
ations to the amygdala’s structure early in the impact of amygdala damage on affect
life play an adaptive role in preparing an perception was associated with the devel-
individual for future adverse environments opmental timing of the lesion; the expe-
(Tottenham & Sheridan, 2010). rience of maltreatment and environmental
Evidence for structural differences in the stress is likely to have a similar differen-
prefrontal cortex after early adversity or tial impact on brain development depend-
maltreatment is more mixed (see McCrory ing on the child’s stage of development. A
et al., 2010, for a review). One study reported recent MRI study, for example, investigated
decreased volume of the OFC in a group the effects of sexual abuse on brain struc-
of children and young adolescents who had ture in women who had experienced abuse
experienced physical abuse, possibly over at varying ages (Andersen et al., 2008). Rel-
an extended time period (Hanson et al., ative to controls, reduced hippocampal vol-
2010). Notably, smaller volumes in the right umes were seen in young adult women who
OFC observed in the physically abused chil- had experienced sexual abuse between the
dren predicted greater problems in chil- ages of 3 and 5 years, whereas frontal cor-
dren’s functioning across a range of social tex volumes were reduced in those who had
domains. This finding suggests that envi- experienced abuse between the ages of 11
ronmental adversity in the form of physical and 13 years of age. This finding fits with evi-
abuse is associated with structural changes dence regarding the developmental trajec-
in the affective circuitry and that these in tories of these regions, with the hippocam-
turn have implications for the child’s social pus thought to be largely mature by age 4
and emotional functioning. (Giedd et al., 1996), whereas the PFC con-
Each of these structures – the corpus tinues to develop into adolescence (Giedd et
callosum, the amygdala, and the OFC – al., 1999; Gogtay et al., 2004). No group dif-
are likely to be differentially susceptible ferences were seen in the amygdala, which is
to maltreatment and psychosocial stress consistent with the view that the amygdala
depending on when the environmental is particularly susceptible to environmental
stressor was experienced (e.g., Totten- stress very early in life. By contrast, regions
ham & Sheridan, 2010). We already know with a longer developmental trajectory such
that the experience of maltreatment at as the prefrontal cortex appear most suscep-
different ages is likely to be associated tible later in development, such as during
with heightened risk for different forms of adolescence. This is one of the first studies
later psychopathology. In a study of adults to directly show unique windows of vulner-
who had experienced abuse or neglect ability for different brain regions following
in childhood, Kaplow and Widom (2007) a common environmental stressor, in this
reported that, whereas earlier maltreatment case, sexual abuse.
was associated with increased anxiety and A complicating factor in investigating the
depression in adulthood, later maltreatment effects of stress at different ages is that
was associated with increased behavioral they may not be immediately apparent and
problems. Although this study did not look instead may emerge only later in develop-
at the neural underpinnings of these effects, ment. For example, although most devel-
it is likely that these differential outcomes opmental indices improve once a child is
are at least in part accounted for by the adopted away from institutional care, one
differential impact of psychosocial stress on study found that children who had been
brain development at different ages. adopted in infancy showed increased emo-
In other words, there is a strong likeli- tional problems at age 11 than at age 6
hood that neural systems have sensitive peri- (Colvert et al., 2008). Another study showed
DEVELOPMENT OF AFFECTIVE CIRCUITRY 627
that, although clinical depression is common Pollak and colleagues has demonstrated that
in individuals who have experienced sexual these children develop broader perceptual
abuse, the average length of time between boundaries for categorizing anger (Pollak &
the onset of abuse and the onset of depres- Kistler, 2002), are able to identify angry facial
sion is 11.5 years, with most cases of depres- expressions with less perceptual informa-
sion occurring during adolescence (Widom, tion than their peers, and accord greater
DuMont, & Czaja, 2007). It may be that attentional resources to the processing of
stress modifies the trajectory of connec- angry faces (Pollak, Klorman, Thatcher, &
tions during an “incubation period,” after Cicchetti, 2001). A recent fMRI study from
which the effects of stress are seen (Lupien our group has demonstrated that children
et al. 2009). This line of argument might who have experienced family violence show
account for the discrepancy between a con- amygdala and anterior insula hyper reactiv-
sistent pattern of reduced hippocampal vol- ity to angry faces (McCrory et al., 2011). It
ume in adults with maltreatment-related is suggested that this pattern of differential
PTSD and the absence of such a pattern responsiveness to anger may reflect an adap-
in children. In other words, it is possible tation to an environment where these threat
that the hippocampal volume reduction evi- signals predict harm (Shackman, Shackman
dent in adults with histories of maltreat- & Pollak, 2007). It is known that experi-
ment is a delayed consequence of a chronic ences of maltreatment are associated with
period of stress during childhood. Consis- a general increase in risk for psychopathol-
tent with this hypothesis, Carrion and col- ogy, including anxiety disorders. One pos-
leagues (2007) reported that cortisol levels sibility is that a pattern of hyper-vigilance
and PTSD symptoms at baseline predicted to social threat cues in the environment
the degree of hippocampal volume reduc- (such as anger) may, at least in part, mediate
tion over an ensuing 12- to 18-month interval the relationship between physical abuse and
in 15 maltreated children with PTSD (Car- anxiety symptoms.
rion, Weems, & Reiss, 2007). Alternatively, Pollak and colleagues have used event-
it is possible that a smaller hippocampal vol- related potentials (ERPs) to investigate this
ume is a predisposing risk factor for PTSD possibility. They examined the extent to
that is present in some individuals prior to which early traumatic experiences were
any traumatic experience (Gilbertson et al., associated with children’s ability to regu-
2002). Longitudinal studies or studies taking late voluntary and involuntary attention to
advantage of identical twins discordant for threat and how this in turn could account
maltreatment exposure are required to pro- for levels of anxiety symptoms (Shackman
vide empirical support for incubation effects et al., 2007). Relative to controls, abused
at the neural level and distinguish these kind children were found to over-attend to visual
of competing accounts. and auditory anger cues. For example, the
presence of a task-irrelevant vocal anger cue
(but not happiness or sadness) elicited a neg-
Maltreatment and the Functioning of
ative frontal ERP component, occurring at
the Affective System
approximately 400 ms, that was larger in
In addition to investigating the neuroen- abused children than in controls. The N2
docrine and structural correlates of early component has typically been interpreted to
adversity and maltreatment, researchers reflect enhanced cognitive control; therefore
have studied how such experiences may lead it is possible that, as a consequence of greater
to functional differences in the way that the processing of auditory anger cues, abused
affective system responds to social cues that children must exert greater cognitive control
vary in emotional valence. Children who to resolve conflicting signals and maintain
have experienced physical abuse in partic- their attention on the task. Abused children
ular have been reported to show a consis- also showed a greater 3Pb response to their
tent pattern of facilitated or enhanced pro- mother’s angry face and voice, which likely
cessing of anger. For example the work of indexes greater voluntary processing of these
628 ESSI VIDING, CATHERINE L. SEBASTIAN, & EAMON J. MCCRORY
potentially threatening cues. These higher overlaps to a great extent with neural regions
3Pb amplitudes were associated with higher implicated in affective processing in adults;
levels of anxiety symptoms and in fact signif- however, we have seen that affective pro-
icantly mediated the relationship between cessing and regulation tasks can elicit dif-
the children’s abuse experience and anxiety ferent patterns of activation depending on
symptoms. This finding is important because age. This is to be expected. There is now
it demonstrates that not only does early envi- an extensive social and experimental psy-
ronmental adversity lead to an adaptation of chological literature documenting the devel-
the affective processing system at both the opment of social and emotional processing
neural and behavioral levels but also that across infancy, childhood, and adolescence.
adaptation contributes directly to a height- Integrating these data with emerging evi-
ened vulnerability for psychopathology. dence from affective neuroscience research
In summary, there is accumulating evi- represents a core task for the next decade of
dence pointing to a variety of neurobiologi- research. We hope that this endeavor will
cal changes associated with childhood mal- not simply be characterized by the search
treatment. Such changes can, on the one for “neural correlates” of specific cognitive
hand, be viewed as a cascade of deleteri- processes, but rather will reflect a mutual
ous effects that are harmful for the child; exchange of ideas and questions, with affec-
however, a more evolutionary and develop- tive neuroscience serving to generate novel
mentally informed view would suggest that developmental hypotheses about how emo-
such changes are in fact adaptive responses tional capacities change across childhood.
to an early environment characterized by Conduct disorder is an example where neu-
threat. If a child is to respond optimally roimaging is helping shed new light on an
to the challenges posed by his or her sur- old problem. Specifically, the neuroimag-
roundings then early stress-induced changes ing findings of atypical amygdala activation
in affective systems can be seen as a means to distress in a subgroup of children with
of “programming” or calibrating those sys- conduct problems are helping inform psy-
tems to match the demands of a hostile chological and clinical models of antisocial
environment. From a clinical perspective, behavior, including the emergence of psy-
such adaptation may heighten vulnerabil- chopathy. However, further questions are
ity to psychopathology later in life, partly raised by this research, including the degree
due to the changes in how emotional and to which any atypical patterns of activa-
cognitive systems mediate social interac- tion reflect functional impairments or atyp-
tion. For example, early-established patterns ical processing styles that are capable of
of hyper-vigilance, although adaptive in an change.
unpredictable home environment, may be The range of research in the field of devel-
maladaptive in other settings, thus increas- opmental affective neuroscience increas-
ing vulnerability for behavioral, emotional, ingly acknowledges the complex interplay
and social difficulties. Investigation of atyp- among genetic, neural, and environmental
ical affective neural development in popu- factors and the need to bear in mind that
lations of children exposed to such aversive these interactions may vary depending on
early stress experiences provides one way to the age of the child. For example, we have
advance our understanding of the role and seen that the impact of sexual abuse on neu-
importance of early caregiving in shaping a ral structures may be contingent on the age
child’s affective development. at which it was experienced. Neural sys-
tems not only show global maturation across
development but also there are regional
Conclusions variations in the rates of maturation of dif-
ferent brain regions, and specific environ-
The neural circuitry associated with affec- mental influences are likely to affect this
tive processing in children is becoming maturation in different ways. This interac-
increasingly well delineated. This circuitry tion in turn will be shaped by individual
DEVELOPMENT OF AFFECTIVE CIRCUITRY 629
differences in genetic factors, such that not development is needed to better under-
all children exposed to the same experience stand brain correlates of emotional devel-
at the same time will be influenced in the opment.
same way. r What are the neural correlates of com-
It is worth noting the increasing empha- plex social interactions that often have
sis on the functional integration of neural considerable affective content? For exam-
systems, rather than the functioning of spe- ple, we know relatively little about “real-
cific regions. A more sophisticated model life” affect regulation in children.
of how limbic and frontal systems interact, r Do atypical patterns of neural structure
for example, is essential if we are to inter- or function at one stage of develop-
pret our functional imaging findings in a ment predict later impaired functioning
more meaningful way. Research in the field or psychopathology? We currently have
of emotion regulation is pertinent in this very limited information about “affective
regard, given that the concept of emotion biomarkers” that may be important for
regulation necessarily requires a considera- understanding developmental vulnerabil-
tion of regions that support both bottom-up ity to psychopathology.
and top-down processes.
Future research in this field faces several
significant challenges. First, there is a need Acknowledgments
to develop a much more detailed “develop-
mental map” of how affect processing and This work was supported by ESRC (RES-
regulation typically change across develop- 062–23-2202) and British Academy (BARDA
ment and how such maturational changes 53229) grants to E. V. and an ESRC grant
interface with the development of other (RES-061–25-0189) to E. M. C. We thank
cognitive and social processes. This work Patricia Lockwood for her assistance in
has started, but there is much still to do preparing this manuscript.
in relation to nearly all stages of develop-
ment. Second, we need to develop a more
sophisticated, ecologically valid, and rela- References
tional set of paradigms. To date many stud-
ies have simply used visual stimuli to look Adolphs, R. (2010). What does the amygdala con-
at evocative effects of different forms of tribute to social cognition? Annals of the New
emotion. Although helpful in mapping the York Academy of Sciences, 1191, 42–61.
basic bottom-up processes, this approach Adolphs, R., Tranel, D., Damasio, H., & Dama-
sio, A. (1994). Impaired recognition of emo-
tells us little about how affect is regulated
tion in facial expressions following bilateral
and may be influenced by context, includ- damage to the human amygdala. Nature, 372,
ing the influence of current (and past) social 669–72.
interactions. Finally, we suggest the need for Andersen, S. L., Tomada, A., Vincow, E. S.,
more longitudinal studies to examine the Valente, E., Polcari, A., & Teicher, M. H.
possible causal role that regions may play (2008). Preliminary evidence for sensitive peri-
in relation to child functioning. Such longi- ods in the effect of childhood sexual abuse on
tudinal research has the potential to open up regional brain development. Journal of Neu-
new approaches to intervention and indeed ropsychiatry and Clinical Neuroscience, 20, 292–
to prevention. 301.
Anderson, S. W., Bechara, A., Damasio, H.,
Tranel, D., & Damasio, A. R. (1999).
Impairment of social and moral behavior
Outstanding Questions and Future related to early damage in human pre-
Directions frontal cortex. Nature Neuroscience, 2, 1032–
37.
r What are the developmental trajectories
Blair, R. J. (2010). Neuroimaging of psychopa-
of affect processing and affect regulation? thy and antisocial behavior: A targeted review.
Longitudinal research on functional brain Current Psychiatry Reports, 12, 76–82.
630 ESSI VIDING, CATHERINE L. SEBASTIAN, & EAMON J. MCCRORY
Bornstein, M. H., & Arterberry, M. E. (2003). rior cingulate cortices. Neuropsychologia, 45,
Recognition, discrimination and categoriza- 1270–79.
tion of smiling by 5-month-old infants. Devel- Fonagy, P., Gergely, G., Jurist, E., & Target, M.
opmental Science, 6, 585–99. (2002). Affect regulation, mentalization and the
Braver, T. S., Cole, M. W., & Yarkoni, T. (2010). development of the self. New York: Other Press
Vive les differences! Individual variation in Francis, D. D., Caldji, C., Champagne, F.,
neural mechanisms of executive control. Cur- Plotsky, P. M., & Meaney, M. J. (1999).
rent Opinions in Neurobiology, 20(2), 242–50. The Role of corticotrophin-releasing factor–
Buckholtz, J. W., & Meyer-Lindenberg, A. (2009) norepinepherine systems in mediating the
Gene-brain associations: The example of effects of early experience on the develop-
MAOA. In S. Hodgins & E. Viding (Eds.), Per- ment of behavioral and endocrine Responses
sistent violent offenders: Neuroscience and reha- to stress. Biological Psychiatry, 46(9), 1153–66.
bilitation (pp. 265–86). Oxford: Oxford Uni- Fusar-Poli, P., Placentino, A., Carletti, F., Landi,
versity Press. P., Allen, P., Surguladze, S., . . . Politi, P.
Burnett, S., Bird, G., Moll, J., Frith, C., & Blake- (2009). Functional atlas of emotional faces
more, S. J. (2009). Development during ado- processing: A voxel-based meta-analysis of 105
lescence of the neural processing of social functional magnetic resonance imaging stud-
emotion. Journal of Cognitive Neuroscience, 21, ies. Journal of Psychiatry and Neuroscience, 34,
1736–50. 418–32.
Carrion, V. G., Weems, C. F., & Reiss, A. L. Gerardi-Caulton, G. (2000). Sensitivity to spa-
(2007). Stress predicts brain changes in chil- tial conflict and the development of self-
dren: A pilot longitudinal study on youth regulation in children 24–36 months of age.
stress, posttraumatic stress disorder, and the Developmental Science, 3, 397–404.
hippocampus. Pediatrics, 119, 509–16. Giedd, J. N., Blumenthal, J., Jeffries, N.
Cohen, J. R., Asarnow, R. F., Sabb, F. W., Bilder, O., Castellanos, F. X., Liu, H., Zijdenbos,
R. M., Bookheimer, S. Y., Knowlton, B. J., & A., . . . Rapoport, J. L. (1999). Brain develop-
Poldrack, R. A. (2010). A unique adolescent ment during childhood and adolescence: A
response to reward prediction errors. Nature longitudinal MRI study. Nature Neuroscience,
Neuroscience, 13, 669–71. 2, 861–63.
Colvert, E., Rutter, M., Kreppner, J., Beck- Giedd, J. N., Rumsey, J. M., Castellanos,
ett, C., Castle, J., Groothues, C., Hawkins, F.X., Rajapakse, J.C., Kaysen, D., Vaituzis,
A., . . . Sonuga-Bark, J. S. (2008). Do the- A. C., . . . Rapoport, J. L. (1996). A quanti-
ory of mind and executive function deficits tative MRI study of the corpus callosum in
underlie the adverse outcomes associated with children and adolescents. Developmental Brain
profound early deprivation? Findings from Research, 91, 274–80.
the English and Romanian adoptees study. Gilbertson, M. W., Shenton, M. E., Ciszewski,
Journal of Abnormal Child Psychology, 36(7), A., Kasai, K., Lasko, N. B., Orr, S. P., &
1057–68. Pitman, R. K. (2002). Smaller hippocampal
Davidson, M. C., Amso, D., Anderson, L. C., & volume predicts pathologic vulnerability to
Diamond, A. (2006). Development of cogni- psychological trauma. Nature Neuroscience, 5,
tive control and executive functions from 4 1242–47.
to 13 years: Evidence from manipulations of Gogtay, N., Giedd, J. N., Lusk, L., Hayashi,
memory, inhibition, and task switching. Neu- K. M., Greenstein, D., Vaituzis, A. C., . . .
ropsychologia, 44, 2037–78. Thompson, P. M. (2004). Dynamic map-
Davidson, R., & Irwin, W. (1999). The functional ping of human cortical development during
neuroanatomy of emotion and affective style. childhood through early adulthood. Proceed-
Trends in Cognitive Sciences, 3(1), 11–21. ings of the National Academy of Sciences, 101,
Davidson, R. J., Jackson, D. C., & Kalin, N. H. 8174–79.
(2000). Emotion, plasticity, context, and reg- Gross, J. J. (1998). The emerging field of emotion
ulation: Perspectives from affective neuro- regulation: An integrative review. Review of
science. Psychological Bulletin, 126(6), 890–909. General Psychology, 2, 271–299.
Eshel, N., Nelson, E. E., Blair, R. J., Pine, D. S., & Gullone, E., Hughes, E. K., King, N. J., &
Ernst, M. (2007). Neural substrates of choice Tonge, B. (2010). The normative development
selection in adults and adolescents: Develop- of emotion regulation strategy use in children
ment of the ventrolateral prefrontal and ante- and adolescents: A 2-year follow-up study.
DEVELOPMENT OF AFFECTIVE CIRCUITRY 631
Journal of Child Psychology and Psychiatry, 51, J., . . . Herpertz-Dahlmann, B. (2008). Emo-
567–74. tional processing in male adolescents with
Guyer, A. E., McClure-Tone, E. B., Shiffrin, N. childhood-onset conduct disorder. Journal of
D., Pine, D. S., & Nelson, E. E. (2009). Prob- Child Psychology and Psychiatry, 49(7), 781–91.
ing the neural correlates of anticipated peer Johnson, M. H. (2005). Subcortical face process-
evaluation in adolescence. Child Development, ing. Nature Reviews Neuroscience, 6, 766–74.
80(4), 1000–15. Jones, A. P., Laurens, K. R., Herba, C. M.,
Guyer, A. E., Monk, C. S., Clure-Tone, E. Barker, G. J., & Viding, E. (2009). Amygdala
B., Nelson, E. E., Roberson-Nay, R., Adler, hypoactivity to fearful faces in boys with con-
A. D., . . . Ernst, M. (2008). A developmen- duct problems and callous-unemotional traits.
tal examination of amygdala response to facial American Journal of Psychiatry, 166(1), 95–102.
expressions. Journal of Cognitive Neuroscience, Kaplow, J. B. & Widom, C. S. (2007) Age of
20, 1565–82. onset of child maltreatment predicts long-
Halligan, S. L., Herbert, J., Goodyer, I. M., term mental health outcomes. Journal of
& Murray, L. (2004). Exposure to postnatal Abnormal Psychology, 116(1), 176–87.
depression predicts elevated cortisol in ado- Kaufman, J., Birmaher, B., Perel, J., Dahl, R.
lescent offspring. Biological Psychiatry, 55(4), E., Moreci, P., Nelson, B., . . . Ryan, N. D.
376–81. (1997). The corticotropin-releasing hormone
Hamann, S. B., Stefanacci, L., Squire, L. R., challenge in depressed abused, depressed
Adolphs, R., Tranel, D., Damasio, H., & nonabused, and normal control children. Bio-
Damasio, A. (1996). Recognizing facial emo- logical Psychiatry, 42(8), 669–79.
tion. Nature, 379, 497. Kédia, G., Berthoz, S., Wessa, M., Hilton, D., &
Hanson, J. L., Chung, M. K., Avants, B. B., Shirt- Martinot, J.-L. (2008). An agent harms a vic-
cliff, E. A., Gee, J. C., Davidson, R. J., & tim: A functional magnetic resonance imaging
Pollak, S. D. (2010). Early stress is associated study on specific moral emotions. Journal of
with alterations in the orbitofrontal cortex: Cognitive Neuroscience, 20(10), 1788–98.
A tensor-based morphometry investigation of Kestenbaum, R., & Nelson, C. A. (2010). The
brain structure and behavioral risk. Journal of recognition and categorization of upright and
Neuroscience, 30, 7466–72. inverted emotional expressions by 7-month-
Hare, T. A., Tottenham, N., Galvan, A., Voss, H. old infants. Infant Behavior and Development,
U., Glover, G. H., & Casey, B. J. (2008). Bio- 13, 497–511.
logical substrates of emotional reactivity and Killgore, W. D., Oki, M., & Yurgelun-Todd, D.
regulation in adolescence during an emotional A. (2001). Sex-specific developmental changes
go-no go task. Biological Psychiatry, 63, 927–34. in amygdala responses to affective faces. Neu-
Hart, J., Gunnar, M., & Cicchetti, D. (1995). Sali- roreport, 12, 427–33.
vary cortisol in maltreated children: evidence Larsen, J. T., To, Y. M., & Fireman, G. (2007).
of relations between neuroendocrine activity Children’s understanding and experience of
and social competence. Development and Psy- mixed emotions. Psychological Science, 18, 186–
chopathology, 7, 11–26. 91.
Hawes, D. J., Brennan, J., & Dadds, M. R. (2009). Lau, J. Y. F., Goldman, D., Buzas, B., Fromm,
Cortisol, callous-unemotional traits, and path- S. J., Guyer, A. E., Monk, C. S., Nelson, E.
ways to antisocial behavior. Current Opinion E., . . . Ernst, M. (2009). Amygdala function
in Psychiatry, 22(4), 357–62. and 5-HTT gene variants in adolescent anx-
Heim, C., Newport, D. J., Mletzko, T., Miller, iety and major depressive disorder. Anxiety,
A. H., & Nemeroff, C. B. (2008). The link 65(4), 349–55.
between childhood trauma and depression: Leppanen, J. M., & Nelson, C. A. (2009). Tuning
Insights from HPA axis studies in humans. the developing brain to social signals of emo-
Psychoneuroendocrinology, 33, 693–710. tions. Nature Reviews Neuroscience, 10, 37–47.
Herba, C., & Phillips, M. (2004). Annotation: Levesque, J., Eugene, F., Joanette, Y., Paquette,
Development of facial expression recognition V., Mensour, B., Beaudoin, G., . . . Beauregard,
from childhood to adolescence: Behavioral M. (2003). Neural circuitry underlying volun-
and neurological perspectives. Journal of Child tary suppression of sadness. Biological Psychi-
Psychology and Psychiatry, 45, 1185–98. atry, 53, 502–10.
Herpertz, S. C., Huebner, T., Marx, I., Vloet, Levesque, J., Joanette, Y., Mensour, B., Beau-
T. D., Fink, G. R., Stoecker, T., Shah, N. doin, G., Leroux, J. M., Bourgouin, P. &
632 ESSI VIDING, CATHERINE L. SEBASTIAN, & EAMON J. MCCRORY
Beauregard, M. (2004). Neural basis of emo- McGivern, R. F., Andersen, J., Byrd, D., Mutter,
tional self-regulation in childhood. Neuro- K. L., & Reilly, J. (2002). Cognitive efficiency
science, 129, 361–69. on a match to sample task decreases at the
Lewis, M. D., Lamm, C., Segalowitz, S. J., onset of puberty in children. Brain and Cogni-
Stieben, J., & Zelazo, P. D. (2006). Neuro- tion, 50, 73–89.
physiological correlates of emotion regulation Mechelli, A., Tognin, S., McGuire, P. K., Prata,
in children and adolescents. Journal of Cogni- D., Sartori, G., Fusar-Poli, P., De Brito, S.,
tive Neuroscience, 18, 430–43. Hariri, A. R., Viding, E. (2009). Genetic Vul-
Lewis, M. D., & Stieben, J. (2004). Emotion reg- nerability to Affective Psychopathology in
ulation in the brain: Conceptual issues and Childhood: A Combined Voxel-Based Mor-
directions for developmental research. Child phometry and Functional Magnetic Reso-
Development, 75, 371–76. nance Imaging Study. Biol Psychiatry 66(3),
Luna, B., Thulborn, K. R., Munoz, D. P., Mer- 231–237.
riam, E. P., Garver, K. E., Minshew, N. Meewisse, M. L., Reitsma, J. B., De Vries, G. J.,
J., . . . Sweeney, J. A. (2001). Maturation of Gersons, B. P. R., & Olff, M. (2007). Cortisol
widely distributed brain function subserves and post-traumatic stress disorder in adults:
cognitive development. Neuroimage, 13, 786– Systematic review and metaanalysis. British
93. Journal of Psychiatry, 191, 387–92.
Lupien, S. J., McEwen, B. S., Gunnar, M. R., & Mehta, M. A, Golembo, N. I., Nosarti,
Heim, C. (2009). Effects of stress throughout C., Colvert, E., Mota, A., Williams, S.
the lifespan on the brain, behavior and cogni- C., . . . Sonuga-Barke, E. (2009). Amygdala,
tion. Nature Reviews. Neuroscience, 10(6), 434– hippocampal and corpus callosum size fol-
45. lowing severe early institutional deprivation:
Marsh, A. A., Finger, E. C., Mitchell, D. G. V., The English and Romanian Adoptees study
Reid, M. E., Sims, C., Kosson, D. S., Tow- pilot. Journal of Child Psychology and Psychia-
bin, K. E., et al. (2008). Reduced amygdala try, 50(8), 943–51.
response to fearful expressions in children and Mitchell, D. G. V. (2011). The nexus between
adolescents with callous-unemotional traits decision making and emotion regulation: A
and disruptive behavior disorders. American review of convergent neurocognitive sub-
Journal of Psychiatry, 165(6), 712–20. strates. Behavioral Brain Research, 217(1), 215–
McClure, E. B. (2000). A meta-analytic review 31.
of sex differences in facial expression process- Mitchell, D. G. V., Richell, R. A., Pine, D., &
ing and their development in infants, chil- Blair, R. J. R. (2008). The contribution of ven-
dren, and adolescents. Psychological Bulletin, trolateral and dorsolateral prefrontal cortex to
126, 424–53. response reversal. Behavioral Brain Research,
McClure, E. B., Monk, C. S., Nelson, E. 187(1), 80–87.
E., Zarahn, E., Leibenluft, E., Bilder, R. Minagawa-Kawai, Y., Matsuoka, S., Dan, I.,
M., . . . Pine, D. S. (2004). A developmental Naoi, N., Nakamura, K., & Kojima, S. (2009).
examination of gender differences in brain Prefrontal activation associated with social
engagement during evaluation of threat. Bio- attachment: Facial-emotion recognition in
logical Psychiatry, 55, 1047–55. mothers and infants. Cerebral Cortex, 19, 284–
McCrory, E. J., De Brito, S. A., Sebastian, C. L., 92.
Mechelli, A., Bird, G., Kelly, P. A., & Vid- Monk, C. S., McClure, E. B., Nelson, E. E.,
ing, E. (2011). Heightened neural reactivity to Zarahn, E., Bilder, R. M., Leibenluft, E., et al.
threat in child victims of family violence. Cur- (2003). Adolescent immaturity in attention-
rent Biology, 21(23), R947–R948. related brain engagement to emotional facial
McCrory, E., De Brito, S. A., & Viding, E. expressions. Neuroimage, 20, 420–28.
(2010). Research review: The neurobiology Nelson, C. A., & de Haan. M. (1996). Neural
and genetics of maltreatment and adversity. correlates of infants‘ visual responsiveness to
Journal of Child Psychology and Psychiatry, 10, facial expressions of emotion. Developmental
1079–95. Psychobiology, 29, 577–95.
McCrory, E., De Brito, S. A., & Viding, E. (2011). Nelson, C. A., Morse, P. A., & Leavitt, L. A.
The impact of childhood maltreatment: A (1979). Recognition of facial expressions by
review of neurobiological and genetic factors. seven-month-old infants. Child Development,
Frontiers in Psychiatry, 2, 48. 50, 1239–42.
DEVELOPMENT OF AFFECTIVE CIRCUITRY 633
Nelson, E. E., Leibenluft, E., McClure, E. B., Rolls, E. T., & Grabenhorst, F. (2008). The
& Pine, D. S. (2005). The social re-orientation orbitofrontal cortex and beyond: From affect
of adolescence: A neuroscience perspec- to decision-making. Progress in Neurobiology,
tive on the process and its relation to 86(3), 216–44.
psychopathology. Psychological Medicine, 35, Rosen, H. J., & Levenson, R. W. (2009). The emo-
163–74. tional brain: Combining insights from patients
Ochsner, K. N., & Gross, J. J. (2005). The cog- and basic science. Neurocase, 15(3), 173–81.
nitive control of emotion. Trends in Cognitive Rushworth, M. F. S., Behrens, T. E. J., Rudebeck,
Sciences, 9, 242–49. P. H., & Walton, M. E. (2007). Contrasting
O’Doherty, J. P. (2004). Reward representations roles for cingulate and orbitofrontal cortex in
and reward-related learning in the human decision and social behavior. Trends in Cogni-
brain: Insights from neuroimaging. Current tive Sciences, 11, 168–76.
Opinions in Neurobiology, 14(6), 769–76. Sabatini, M. J., Ebert, P., Lewis, D. A, Levitt,
Passamonti, L., Fairchild, G., Goodyer, I. M., P., Cameron, J. L., & Mirnics, K. (2007).
Hurford, G., Hagan, C. C., Rowe, J. B., Amygdala gene expression correlates of social
& Calder, A. J. (2010) Neural abnormalities behavior in monkeys experiencing maternal
in early-onset and adolescence-onset conduct separation. Journal of Neuroscience, 27(12),
disorder. Archives of General Psychiatry, 67(7), 3295–3304.
729–38. Sánchez, M. M., Ladd, C. O., & Plotsky, P. M.
Paus, T., Keshavan, M., & Giedd, J. N. (2008). (2001). Early adverse experience as a develop-
Why do many psychiatric disorders emerge mental risk factor for later psychopathology:
during adolescence? Nature Reviews Neuro- Evidence from rodent and primate models.
science, 9, 947–57. Development and Psychopathology, 13, 419–49.
Peltola, M. J., Leppanen, J. M., Palokangas, T., & Schumann, C. M., Hamstra, J., Goodlin-Jones,
Hietanen, J. K. (2008). Fearful faces modulate B. L., Lotspeich, L. J., Kwon, H., Buonocore,
looking duration and attention disengagement M. H., . . . Amaral, D. G. (2004). The amyg-
in 7-month-old infants. Developmental Science, dala is enlarged in children but not adolescents
11, 60–68. with autism; the hippocampus is enlarged
Peters, J., & Büchel, C. (2010). Episodic future at all ages. Journal of Neuroscience, 24, 6392–
thinking reduces reward delay discounting 6401.
through an enhancement of prefrontal- Sebastian, C. L., Fontaine, N. M. G., Bird, G.,
medio-temporal interactions. Neuron, 66(1), Blakemore, S.-J., De Brito, S. A., McCrory,
138–48. E. J. P., & Viding, E. (2012). Neural processing
Posner, M. I., & Rothbart, M. K. (2000). Devel- associated with cognitive and affective Theory
oping mechanisms of self-regulation. Develop- of Mind in adolescents and adults. Social Cog-
ment and Psychopathology, 12, 427–41. nitive and Affective Neuroscience, 7(1), 53-63.
Pollak, S. D., Cicchetti, D., Klorman, R., & Bru- doi: 10.1093/scan/nsr023
maghim, J. T. (1997). Cognitive brain event- Sebastian, C. L., McCrory, E. J., Cecil, C. A.,
related potentials and emotion processing in Lockwood, P. L., De Brito, S. A., Fontaine,
maltreated children. Child Development, 68, N. M., & Viding, E. (2012). Neural responses
773–87. to affective and cognitive theory of mind in
Pollak, S. D., & Kistler, D. J. (2002). Early expe- children with conduct problems and varying
rience is associated with the development of levels of callous-unemotional traits. Archives
categorical representations for facial expres- of General Psychiatry, 69(8), 814–22.
sions of emotion. Proceedings of the National Sebastian, C. L., Tan, G. C. Y., Roiser, J. P.,
Academy of Sciences, 99, 9072–76. Viding, E., Dumontheil, I., & Blakemore, S. J.
Pollak, S. D., Klorman, R., Thatcher, J. E., & Cic- (2011). Developmental influences on the neu-
chetti, D. (2001). P3b reflects maltreated chil- ral bases of responses to social rejection: Impli-
dren’s reactions to facial displays of emotion. cations of social neuroscience for education.
Psychophysiology 38, 267–74. Neuroimage, 57(3), 686–94.
Reynolds, G. D., & Richards, J. E. (2005). Famil- Sergerie, K., Chochol, C., & Armony, J. L. (2008).
iarization, attention, and recognition memory The role of the amygdala in emotional pro-
in infancy: An event-related potential and cor- cessing: A quantitative meta-analysis of func-
tical source localization study. Developmental tional neuroimaging studies. Neuroscience and
Psychology, 41, 598–615. Biobehavioral Reviews, 32(4), 811–30.
634 ESSI VIDING, CATHERINE L. SEBASTIAN, & EAMON J. MCCRORY
Shackman, J. E., Shackman, A. J., & Pollak, S. D. Tottenham, N., Hare, T. A., Quinn, B. T.,
(2007). Physical abuse amplifies attention to McCarry, T. W., Nurse, M., Gilhooly,
threat and increases anxiety in children. Emo- T., . . . Casey, B. J. (2010). Prolonged institut-
tion, 7(4), 838–52. ional rearing is associated with atypically large
Shaw, P., Kabani, N. J., Lerch, J. P., Eckstrand, amygdala volume and difficulties in emotion
K., Lenroot, R., Gogtay, N., . . . Wise, S. P. regulation. Developmental Science, 13(1), 46–61.
(2008). Neurodevelopmental trajectories of Tottenham, N., & Sheridan, M. A. (2010). A
the human cerebral cortex. Journal of Neu- review of adversity, the amygdala and the hip-
roscience, 28, 3586–94. pocampus: A consideration of developmen-
Shaw, P., Lalonde, F., Lepage, C., Rabin, tal timing. Frontiers in Human Neuroscience,
C., Eckstrand, K., Sharp, W., Greenstein, 3(68), 1–18.
D., . . . Rapoport, J. (2009). Development of van Goozen, S. H. M., & Fairchild, G. (2008).
cortical asymmetry in typically developing How can the study of biological processes help
children and its disruption in attention- design new interventions for children with
deficit/hyperactivity disorder. Archives of Gen- severe antisocial behavior? Development and
eral Psychiatry, 66(8), 888–96. Psychopathology, 20(3), 941–73.
Somerville, L. H., & Casey, B. J. (2010). Devel- Van Leijenhorst, L., Zanolie, K., Van Meel, C. S.,
opmental neurobiology of cognitive control Westenberg, P. M., Rombouts, S. A. R. B., &
and motivational systems. Current Opinions Crone, E. A. (2010). What motivates the ado-
in Neurobiology, 20, 236–41. lescent? Brain regions mediating reward sensi-
Sterzer, P., & Stadler, C. (2009). Neuroimaging tivity across adolescence. Cerebral Cortex, 20,
of aggressive and violent behavior in children 61–69.
and adolescents. Frontiers in Behavioral Neu- Viding, E., McCrory, E. J., Blakemore, S.-J., &
roscience, 3(35), 1–8. Frederickson, N. (2011). Behavioral problems
Sterzer, P., Stadler, C., Krebs, A., Kleinschmidt, and bullying at school: Can cognitive neuro-
A., & Poustka, F. (2005). Abnormal neural science shed new light on an old problem?
responses to emotional visual stimuli in ado- Trends in Cognitive Sciences, 15(7), 289–91.
lescents with conduct disorder. Biological Psy- Vuilleumier, P., & Pourtois, G. (2007). Dis-
chiatry, 57(1), 7–15. tributed and interactive brain mechanisms
Takakahashi, H., Yahata, N., Koeda, M., Mat- during emotion face perception: Evidence
suda, T., Asai, K., & Okubo, Y. (2004). Brain from functional neuroimaging. Neuropsycholo-
activation associated with evaluative processes gia, 45(1), 174–94.
of guilt and embarrassment: An fMRI study. Wager, T. D., Davidson, M. L., Hughes, B.
Neuroimage, 23, 967–74. L., Lindquist, M. A., & Ochsner, K. N.
Thomas, K. M., Drevets, W. C., Whalen, P. J., (2008). Prefrontal-subcortical pathways medi-
Eccard, C. H., Dahl, R. E., Ryan, N. D., et ating successful emotion regulation. Neuron,
al. (2001). Amygdala response to facial expres- 59, 1037–50.
sions in children and adults. Biological Psychi- Widom, C. S., DuMont, K., & Czaja, S. J. (2007).
atry, 49, 309–16. A prospective investigation of major depres-
Thomas, L. A., De Bellis, M. D., Graham, R., sive disorder and comorbidity in abused and
& LaBar, K. S. (2007). Development of emo- neglected children grown up. Archives of Gen-
tional facial recognition in late childhood and eral Psychiatry, 64(1), 49–56.
adolescence. Developmental Science, 10, 547– Woon, F. L., & Hedges, D. W. (2008). Hip-
58. pocampal and amygdala volumes in chil-
Thompson, R. (1994). Emotion regulation: A dren and adults with childhood maltreatment-
theme in search of a definition. In N. A. related posttraumatic stress disorder: A
Fox (Ed.), Emotion regulation: Biological and meta-analysis. Hippocampus, 18, 729–36.
behavioral considerations. Monographs of the Zarrow, M. X., Philpott, J. E., & Denenberg,
Society for Research in Child Development V. H. (1970). Passage of 14-C-4 corticosterone
(pp. 25–52). Chicago: University of Chicago from the rat mother to the fetus and neonate.
Press. Nature, 226, 1058–59.
CHAPTER 28
635
636 PEGGY L. ST. JACQUES, AMY WINECOFF, & ROBERTO CABEZA
Two Accounts of Spared Emotional structure and visual cortex, the medial tem-
Processing in Old Age poral lobes, and the frontal lobes (Amaral &
Price, 1984).
The fact that emotional processing is rela- The structural integrity of the amygdala
tively spared by aging is surprising,. given remains relatively preserved in aging (Brabec
evidence of substantial neural decline in et al., 2010; Cherubini, Peran, Caltagirone,
healthy aging (for a review, see Dennis Sabatini, & Spalletta, 2009). Although a
& Cabeza, 2008). One possible explana- few studies have reported age-related vol-
tion is that age-related anatomical deteriora- ume reductions (Mu, Xie, Wen, Weng,
tion is less pronounced in the brain regions & Shuyun, 1999), this effect has been
mediating emotional processing (anatomi- attributed to the inclusion of white mat-
cal preservation hypothesis). Another possi- ter within the amygdalar region of inter-
ble explanation is that age-related anatom- est (Brabec et al., 2010). In contrast with
ical and physiological decline is partly healthy aging, substantial amygdalar atro-
counteracted by compensatory increases phy has been found in individuals with
in brain activity (functional compensation Alzheimer’s disease (AD) or at genetic risk
hypothesis). Evidence for these two hypothe- for AD (Honea, Vidoni, Harsha, & Burns,
ses, which are not incompatible with each 2009).
other, is reviewed next.
STRIATUM
The striatum can be divided into the dor-
Anatomical Preservation
sal striatum, which includes the caudate
Age-related anatomical decline is most pro- and putamen nuclei, and the ventral stria-
nounced in the lateral prefrontal cortex tum, which includes the nucleus accum-
(PFC). The lateral PFC shows the steepest bens. In addition to important contributions
rate of age-related atrophy (Raz et al., 2005), to motor and executive functions, the stria-
and this atrophy has been linked to cogni- tum has been associated with the processing
tive deficits, particularly in executive func- of reward (see Chapter 19). Unlike the amyg-
tions (Gunning-Dixon & Raz, 2003). Age- dala, striatal nuclei have been associated
related atrophy is also substantial in the with substantial atrophy in healthy aging
hippocampus (e.g., Raz, 2005), which also (Cherubini et al., 2009). Whereas the dor-
shows age-related reductions in memory- sal striatum, particularly the caudate, shows
related brain activity (for a review, see substantial shrinkage as a function of aging
Dennis et al., 2008). In contrast with the (Raz et al., 2003), there is evidence that the
decline of regions mediating executive and ventral striatum shows very little age-related
memory functions, brain regions support- atrophy (Cherubini et al., 2009).
ing emotional and reward processing, such It is worth noting, however, that the stria-
as the amygdala, ventral striatum, and the tum is affected by considerable age-related
medial PFC (O’Doherty, 2004; Phan, Wager, decline in dopamine function, including
Taylor, & Liberzon, 2002) are relatively well reductions in dopamine receptors (e.g.,
preserved in healthy older adults. Antonini & Leenders, 1993; Wang et al.,
1998) and dopamine transporter (e.g., van
AMYGDALA Dyck et al., 1995). Dopamine decline plays
The amygdala has been associated with the a major role in age-related cognitive deficits
automatic detection of emotions, as well as and has been strongly linked to reductions
with the generation of emotion and associ- in episodic memory and executive function
ated physiological responses (Phelps, 2006). (e.g., Erixon-Lindroth et al., 2005). Substan-
It is also assumed to mediate emotional tial decline in dopamine function leads to
modulation of perception, emotion, and the prediction that striatal contributions to
decision making, an idea consistent with reward processing are likely to be disrupted
dense anatomical connections between this by aging (Mohr, Li, & Heekeren, 2009), even
EMOTION AND AGING 637
activity. This pattern has been observed spared emotional processing in old age: the
in many cognitive domains, including per- anatomical preservation hypothesis and the
ception, attention, working memory, prob- functional compensation hypothesis. Both
lem solving, and episodic memory encoding hypotheses have some empirical support.
and memory retrieval (Dennis & Cabeza, The anatomical preservation hypothesis is
2008). PASA was first reported by Grady consistent with evidence that, in contrast
et al. (1994), who suggested that older adults with substantial atrophy in lateral PFC
compensated for visual processing deficits regions, the structural integrity of emotional
(occipital decrease) by recruiting higher processing regions such as the amygdala,
order cognitive processes (PFC increase). ventral striatum, and medial PFC is rela-
Consistent with the compensation hypoth- tively well maintained in older adults. The
esis, there is evidence that the PFC increase anatomical preservation hypothesis is less
in older adults is positively correlated with consistent with dorsal striatal atrophy and
cognitive performance and negatively cor- declines in dopamine function, as well as
related with the occipital decrease (Davis with the atrophy of the lateral PFC, which
et al., 2008). plays a role in some forms of emotion regula-
tion. The functional compensation hypoth-
HEMISPHERIC ASYMMETRY REDUCTION IN esis is indirectly supported by evidence that
OLDER ADULTS (HAROLD) age-related neural decline can be partly
The second brain activity pattern in older counteracted by changes in brain activity,
adults found consistently across cognitive such as PFC over-recruitment. Two patterns
domains is a more bilateral (less asymmet- of PFC over-recruitment – the posterior-
ric) pattern of PFC activity. This pattern, anterior shift with aging (PASA) and the
which is known as the hemispheric asymme- hemispheric asymmetry reduction in older
try reduction in OLDer adults (HAROLD) adults (HAROLD) – have been linked to
has been reported in the domains of percep- improved cognitive performance, opening
tion, attention, working memory, episodic the possibility that PFC over-recruitment
memory encoding and memory retrieval, could also explain spared emotional process-
and inhibitory control (Cabeza, 2002) The ing in old age. More direct evidence for the
HAROLD pattern was originally described contribution of functional compensation to
by Cabeza et al. (1997) and was also improved emotional processing in aging is
attributed to a compensatory mechanism. provided by functional neuroimaging stud-
This compensation account is consistent with ies in the emotion domain, as reviewed in
evidence that bilateral activity in older the next section.
adults is positively correlated with success-
ful cognitive performance (Reuter-Lorenz
et al., 2000) and is found in high-performing Behavioral and Functional
rather than in low-performing older adults Neuroimaging Studies of Emotion
(Cabeza, 2002). The alternative dedifferenti- and Aging
ation account is that HAROLD reflects an
age-related difficulty in engaging specialized The following sections review behavioral
neural mechanisms (e.g., Logan, Sanders, and functional neuroimaging studies on the
Snyder, Morris, & Buckner, 2002). In gen- effects of aging on emotional processing in
eral, the available evidence tends to be different domains, including emotional per-
more consistent with the compensation than ception, emotional episodic memory, emo-
with the dedifferentiation account (Dennis tional decision making, and emotion regu-
& Cabeza, 2008). lation. The most consistent pattern is an
increase in frontal activity that is some-
times coupled with decreases in amyg-
Summary of Current Accounts
dalar responses to negative stimuli; that is,
In this first section, we considered evidence the pattern we have called frontoamygdalar
for two cognitive neuroscience accounts of age-related differences in emotion (FADE;
EMOTION AND AGING 639
St. Jacques, Bessette-Symons, & Cabeza, tures. For example, older adults tend to
2009). The nature of this pattern will rate positive pictures as more positive when
become clear when illustrated by specific compared to young adults and to standard-
findings in the following sections. One ques- ized ratings (e.g., Gruhn & Scheibe, 2008).
tion to keep in mind is whether age-related In sum, although emotional salience might
PFC increases could partly account for generally remain intact during healthy aging,
spared emotional processing in older adults there are some age-related differences in
(functional compensation hypothesis). the perception of negative versus positive
emotions.
Emotional Perception: Behavior
EMOTIONAL PERCEPTION: FUNCTIONAL
In young adults, emotional stimuli are more NEUROIMAGING
salient such that they are perceived and The majority of functional neuroimaging
attended to with greater ease and fluency studies on the effects of aging on emotional
when compared to neutral stimuli (for a processing have focused on perception of
review, see Kensinger, 2004, and Chap- emotional stimuli. One of the most con-
ter 14). Older adults also benefit from emo- sistent findings from functional neuroimag-
tion during emotional perception, such that ing studies of emotional perception is an
emotional salience remains relatively sta- age-related reduction in amygdalar activa-
ble with age (for a meta-analysis, see Mur- tion for negative stimuli (see Table 28.1). For
phy & Isaacowitz, 2008). For example, com- example, in the first fMRI study of emo-
pared to young adults, older adults perform tion and aging, Iidaka et al. (2002) found
similarly on attention (Mather & Knight, less left amygdalar activity for negative faces
2006; Samanez-Larkin, Robertson, Mikels, in older adults when compared to young
Carstensen, & Gotlib, 2009) and percep- adults, and other studies have observed simi-
tion of emotional stimuli (for a review, lar findings for both emotional faces and pic-
see Kensinger, 2009). These results sug- tures (Fischer et al., 2005; Gunning-Dixon
gest that the automatic processing of emo- et al., 2003; Leclerc & Kensinger, 2011; Tes-
tion remains relatively intact with age (for sitore et al., 2005). However, in some of the
reviews, see Kensinger & Leclerc, 2009; aforementioned studies (Gunning-Dixon et
Mather, 2006). al., 2003; Iidaka et al., 2002; Tessitore et al.,
Subtle age-related effects on emo- 2005) older adults’ reported negative stim-
tional perception, however, might emerge uli valence ratings that were less negative
depending on the particular quality and type than the ones given by young adults, con-
of emotional stimulus. Compared to young sistent with the behavioral findings show-
adults, older adults have greater difficulty ing age-related reductions in the recognition
perceiving negative stimuli, but their per- of negative faces. Thus, one potential inter-
ception of positive stimuli is less impaired. pretation is that the age-related reduction
The age-related reduction for negative stim- in amygdala activity reflects an inability of
uli and the increase or comparable perfor- these types of stimuli to elicit strong nega-
mance for positive stimuli are known as the tive emotions in older adults, rather than a
positivity effect (for a review, see Mather & deficit in amygdala function per se. Consis-
Carstensen, 2005). For example, in a recent tent with this idea, we and others have found
meta-analysis examining age-related differ- that amygdala activity can be as strong in
ences in emotional perception, Ruffman, older as in younger adults when stimuli were
Henry, Livingston, and Phillips (2008) found classified according to participants’ own rat-
that older adults had impaired recognition ings (Leclerc & Kensinger, 2008; Mather et
of angry and sad faces compared to young al., 2004; Ritchey, Bessette-Symons, Hayes,
adults, but were less impaired during the & Cabeza, 2011; St. Jacques, Dolcos, &
recognition of happy faces. These results Cabeza, 2010; Wright, Wedig, Williams,
are similar to age-related effects observed Rauch, & Albert, 2006) or when there
for other emotional stimuli such as pic- were no age-related differences in emotional
640 PEGGY L. ST. JACQUES, AMY WINECOFF, & ROBERTO CABEZA
Perception
Iidaka 02 Neg faces neg (a, s) > •
control; gender
discrimiation
Mather 04* Neg pictures neg vs baseline; •
emotion rating
Fischer 05 Neg faces neg(a) > neut; •
passive viewing
Tessitore 05 Neg faces neg (a, afr) > ◦ •
control;
perceptual
matching
Williams 06* Neg faces neg (f) > neut; ◦ • •
passive viewing
Wright 06* Neg faces neg (f) +novel > ❖
neut; passive
viewing
Wright 07* Neg faces neg (f) + novel > ❖
neut; passive
viewing
St.Jacques 10 Neg pictures neg > neut; ◦ ◦ ❖ ◦
emotion rating
LeClerc 08 Neg objects neg > pos; • •
semantic rating
Iidaka 02 Pos faces pos (h) > control;
gender
discrimiation
Mather 04* Pos pictures neg vs baseline; ❖
emotion rating
Williams 06* Pos faces pos (f) > neut; • • •
passive viewing
LeClerc 08 Pos objects pos > neg; ◦ ◦ ❖
semantic rating
Gutchess 07 Pos words (pos self>pos ◦ ◦
other)>(neg
self>neg other);
self/other rating
Episodic Memory
Fischer 10 Neg faces neg (f) > neut; ◦ •
emotion rating
St. Jacques 09 Neg pictures neg vs. neut; ◦ ❖ •
emotion rating
Murty 09 Neg pictures neg vs. neut; ◦
semantic rating
Murty 09 Neg pictures neg > neut; ◦ ◦ •
old/new
recognition
Kensinger 08 Neg objects neg > pos & neut;
semantic rating
EMOTION AND AGING 641
ratings (Fischer et al., 2005). Furthermore, stimuli to elicit strong negative emotions in
we (St. Jacques et al., 2010) found that amyg- older adults.
dala activity was indeed reduced for negative In contrast with the amygdala, functional
stimuli that older adults subjectively rated as neuroimaging studies of emotional percep-
neutral. Interestingly, age-invariant activity tion have generally found an age-related
in the amygdala is typically observed for pos- increase in PFC recruitment. This effect is
itive stimuli (Gutchess, Kensinger, Yoon, & often more pronounced in the medial PFC
Schacter, 2007; Leclerc & Kensinger, 2011; (see Table 28.1), but it can be also observed
Mather et al., 2004; although see Williams in lateral PFC regions (Gunning-Dixon et
et al., 2006) and might reflect a shift in the al., 2003; Gutchess, Kensinger, & Schacter,
preference of amygdala activation in aging 2007; Tessitore et al., 2005). Some studies
(Mather et al., 2004; also see Wright et al., have observed an age-related increase in the
2006). In sum, amygdalar responses to nega- recruitment of the medial PFC during the
tive stimuli tend to be reduced by aging, but perception of negative versus neutral pic-
this reduction may reflect a failure of the tures (Leclerc & Kensinger, 2011; St. Jacques,
642 PEGGY L. ST. JACQUES, AMY WINECOFF, & ROBERTO CABEZA
Dolcos, & Cabeza, 2010; Williams et al., the perception of emotional stimuli. The
2006), whereas other studies have observed fMRI studies reviewed here suggest that per-
that the age-related increases in medial PFC ceptual differences for positive versus nega-
recruitment are greater for positive versus tive valence in aging are sometimes observed
negative stimuli (Gutchess, Kensinger, & because these tasks rely on controlled pro-
Schacter, 2007; Leclerc & Kensinger, 2008, cesses associated with the PFC that are pref-
2011; Ritchey, Bessette-Symons, Hayes, & erentially engaged in older adults.
Cabeza, 2011; also see Leclerc & Kensinger,
2010). These apparent differences might be
Emotional Episodic Memory: Behavior
due to the dissociable effects of arousal
and valence in dorsal versus ventral aspects In young adults, memory is typically bet-
of the medial PFC (e.g., Dolcos, LaBar, & ter for emotional than neutral stimuli
Cabeza, 2004). For example, Leclerc and (emotionally enhanced memory, EEM; see
Kensinger (2008) found age-invariant activ- Chapter 20). Although EEM is relatively
ity in dorsomedial regions reflecting emo- preserved in healthy aging there is an age-
tional arousal, whereas the age-related dif- related alteration in the extent of EEM,
ferences observed in ventromedial regions especially for negative information (for a
(specifically ventral ACC) reflected differ- meta-analysis, see Murphy & Isaacowitz,
ences in positive versus negative valence. 2008). Some studies have reported that
Additional research is needed to tease apart EEM in older adults is similar for posi-
potential age-related differences in the neu- tive and negative emotions (e.g., Gruhn,
ral correlates subserving emotional arousal Smith, & Baltes, 2005; St. Jacques & Levine,
and valence. 2007), whereas, others have reported dif-
The age-related increase in PFC activ- ferential effects of valence (e.g., Mather &
ity may reflect greater recruitment of con- Carstensen, 2003). One interpretation of the
trolled processes during emotional per- different findings is that age-related effects
ception. Evidence for this hypothesis was on valence are only evident under certain
recently provided by Ritchey et al. (2011). conditions. According to Mather (2006) the
First, they found that age-related increases positivity effect is less likely to be generated
in medial PFC and ventrolateral PFC activity when there are tasks during encoding that
depended on the engagement of controlled interfere with older adults’ ability to direct
processes during perception. Second, they limited cognitive resources to emotional
found that individual differences in execu- stimuli. In a series of experiments, Mather
tive function predicted the recruitment of and Knight (2005) demonstrated that older
these PFC regions. The increased recruit- adults engage in greater elaborative process-
ment of controlled processes could poten- ing for positive than negative stimuli, that
tially explain age-related differences in the the positivity bias is correlated with perfor-
perception of emotional stimuli. Consistent mance on cognitive control, and that older
with this idea, we (St. Jacques et al., 2010) adults who are distracted during the encod-
found that negative to neutral shifts in older ing of emotional stimuli do not show a pref-
adults’ ratings of pictures modulated the erence in memory for positive items. Simi-
involvement of the medial PFC and its cou- larly, Emery and Hess (2008) manipulated
pling with the amygdala. viewing instructions during encoding and
In sum, consistent with FADE, emotional found that older adults’ emotional mem-
perception studies of aging converge on ory was better when stimuli were processed
the observation of an age-related reduction in an emotionally meaningful way com-
in amygdala activation for negative stim- pared to perceptual processing or to pas-
uli coupled with an increase in PFC activa- sive viewing. Taken together these results
tion for both negative and positive stimuli. suggest that age-related differences in mem-
The latter effect suggests that older adults ory for positive and negative stimuli are less
engage more controlled processing during likely to occur when encoding places greater
EMOTION AND AGING 643
demands on cognitive resources that reduce ful memory encoding of objects, regardless
attention to emotionally salient aspects of of the particular negative or positive valence,
the stimuli. but older adults elicited greater medial PFC
Alternatively, the mixed findings regard- activity during the successful encoding of
ing the positivity effect in aging could be positive objects.
due to differential levels of arousal elicited Furthermore, we and others have obser-
across studies (cf. Gruhn et al., 2005). ved an age-related decrease in functional
Online ratings of arousal are sometimes not connectivity between the amygdala and the
acquired in studies, or they are not ana- areas that typically support memory for-
lyzed with respect to potential age-related mation (i.e., the hippocampus) for negative
interactions with valence when obtained. stimuli, but an age-related increase in func-
Kensinger (2008) showed that the positivity tional connectivity with PFC areas involved
effect is more likely for low-arousing words in controlled processing (Murty et al.,
compared to high-arousing stimuli. In sum, 2009; St. Jacques, Dolcos, & Cabeza, 2009;
interpretations of a positivity effect or nega- although see Addis, Leclerc, Muscatell, &
tive reduction in episodic memory are prob- Kensinger, 2010). Thus, increased frontal
lematic without a careful consideration of recruitment could reflect compensatory
task demands and the amount of arousal processes that support successful memory
elicited by emotional stimuli. formation. Based on age-related reductions
in EEM observed in these studies, how-
EMOTIONAL MEMORY: FUNCTIONAL ever, this age-related frontal shift is less
NEUROIMAGING effective compared to the modulation of
Only recently have functional neuroimaging the hippocampus by the amygdala, which
studies begun to explore the neural bases boosts consolidation processes and is likely
of the age-related changes in EEM. Despite to persist over time (Ritchey, Dolcos, &
differences in methodology, functional neu- Cabeza, 2008).
roimaging studies of memory encoding gen- Addis et al. (2010) recently showed that
erally show increases in PFC activity and the pattern of connectivity during the for-
alterations in the amygdala. For example, mation of positive memories also differs in
we (St. Jacques, Dolcos, & Cabeza, 2009) older adults. Using a structural equation
found that amygdala activity predicted sub- model to examine the effective connectiv-
sequent memory of negative versus neutral ity in the emotional network, they found
pictures in both young and older adults, but that older adults had stronger positive con-
that older adults recruited additional frontal nections in the top-down influence of the
activity to support memory formation as amygdala and ventromedial PFC on the hip-
visual cortex activity declined. Recently, Fis- pocampus, whereas in young adults these
cher (2010) observed age-related increases same regions had a negative influence on
in left dorsolateral PFC (DLPFC) for sub- the hippocampus. Further, young adults had
sequent memory of fearful versus neutral greater positive influence of the thalamus on
faces. Although both age groups recruited the hippocampus. Addis and colleagues sug-
bilateral amygdala during successful encod- gested that the age-related changes in effec-
ing of fearful faces, young adults recruited tive connectivity during successful encod-
greater right amygdala. Similarly, Murty ing of positive memories might reflect
et al. (2009) found age-related increases in greater self-referential processing in the
the recruitment of left DLPFC during the older adults, which could also explain their
encoding of negative versus neutral picture increased memory for these same stimuli.
blocks, but they did not observe age-related Currently, only one study has examined
differences in recruitment of the amygdala. age-related effects during memory retrieval.
Complementing these findings, Kensinger The aforementioned study by Murty et
and Schacter (2008) observed age-invariant al. (2009) found that older adults showed
recruitment of the amygdala during success- a reduction in the recruitment of the
644 PEGGY L. ST. JACQUES, AMY WINECOFF, & ROBERTO CABEZA
amygdala during memory retrieval of nega- older adults are susceptible to framing
tive versus neutral blocks, but an age-related effects when gains are emphasized but not
increase in the right DLPFC. Further, older when losses are emphasized (Mikels & Reed,
adults had greater amygdalar-DLPFC cou- 2009), and they spend more time review-
pling, but less amygdalar-hippocampal cou- ing positive relative to negative informa-
pling during negative retrieval blocks, sug- tion when making decisions about physi-
gesting a possible compensatory mechanism cians and health care plans than do young
to aid memory retrieval or potentially the adults (Lockenhoff & Carstensen, 2007).
engagement of controlled processes that reg- These results suggest that older adults’ deci-
ulate emotional responses during retrieval. sions are influenced by goals to actively man-
In sum, the results of these initial fMRI age their emotional experiences. In line with
studies suggest that aging leads to an this idea, although young and older adults
increased reliance on controlled processes seek similar levels of variety in choices for
in the PFC that support and maintain current consumption, older adults’ choices
enhanced memory for emotional materials. for future consumption show less variety.
These results are in line with the behav- By selecting options that are known to be
ioral studies reviewed earlier that suggest preferred, older adults may be attempting
that age-related changes in memory for emo- to regulate their future emotional experi-
tion stimuli depend on the recruitment of ences (Novak & Mather, 2007). Given that
controlled processes that influence emotion. emotion regulatory processes can be used to
Future research will need to more closely reduce behavioral reports and physiological
examine how both valence and arousal con- metrics of loss (Sokol-Hessner et al., 2009)
tribute to potential age-related differences and risk aversion (Heilman, Crisan, Houser,
in the neural mechanisms supporting suc- Miclea, & Miu, 2010), the motivation to reg-
cessful memory encoding and retrieval. ulate emotions during decision making may
not necessarily produce poorer decision out-
comes in older adults in all contexts.
Affective Decision Making: Behavior
Motivational influences play a critical role
Information processing during decision in decision making. As a result, several stud-
making has often been characterized as rely- ies have investigated age-related differences
ing on two separate systems: one deliber- in decision making by examining how older
ative and cognitive, and the other affec- adults learn the associations between partic-
tive and experiential (Peters, Hess, Västfjäll, ular stimuli and rewarding or punishing out-
& Auman, 2007; see Chapter 17). Changes comes (e.g., Denburg, Recknor, Bechara, &
in both deliberative and affective aspects Tranel, 2006); however, it is not exactly clear
of information processing in decision mak- how aging affects information processing in
ing occur with age (Kennedy & Mather, reinforcement learning. On the one hand,
2007); however, the degree to which these older adults sometimes show preferential
changes act in concert to produce mal- learning from positive rewards. For exam-
adaptive choices may depend on a number ple, Denburg et al. (2006) found that high-
of contextual factors (Peters et al., 2007). performing older adults were more likely
Nonetheless, age-related differences in emo- to generate anticipatory skin conductance
tional processing may result in suboptimal responses when selecting from advantageous
real-world financial behavior in older adults decks on the Iowa Gambling Task, whereas
(Weierich et al., 2011). Thus, characterizing low-performing older adults did not show
the ways in which aging affects affective pro- differential anticipatory responses between
cessing is critical to understanding decision advantageous and disadvantageous decks.
making in older adulthood. This finding suggests that older adults poten-
Consistent with studies of emotion per- tially learn better from positive than nega-
ception and memory, older adults’ deci- tive reinforcement, perhaps representing an
sions are differentially influenced by positive age-related shift consistent with changes in
versus negative information. For example, emotional focus in later life (Denburg et al.,
EMOTION AND AGING 645
2006). On the other hand, older adults may these findings were interpreted as evidence
be biased toward negative outcomes in some that older adults do not activate the stria-
contexts. Using a probabilistic learning task, tum in a systematic way to guide subse-
Frank and Kong (2008) assessed whether quent reward learning. This is consistent
older adults’ initial learning was guided by with the hypothesis that, although older
approach or avoidance strategies and found adults do not show deficits in the processing
that the oldest group of participants learned of reward outcomes, they do not effectively
more from negative than positive outcomes. integrate this information into predictions
However, this age-related increase in avoid- about future rewards (Schott et al., 2007).
ance learning does not appear to hold for Similarly, increased temporal variability in
younger samples of older adults, suggesting signals arising from the nucleus accumbens
that the effect may not generalize to all older in older adults predicts poorer decision per-
adults. formance (Samanez-Larkin, Kuhnen, Yoo,
In sum, like young adults, older adults’ & Knutson, 2010). Thus, differences in stri-
decisions are biased by emotional infor- atal functioning in older adults are consistent
mation; however, the valence and nature with deficits in reinforcement learning pro-
of emotional information in decision mak- cesses that rely on reward-signal processing
ing and reward tasks may have different in the basal ganglia.
impacts on behavior in later adulthood. Sim- Some evidence, however, points to a
ilar to studies examining age-related changes valence-specific shift in neural reward pro-
in emotional perception and memory, this cessing in older adulthood, providing con-
research suggests that older adults poten- sistency with findings in the domains of
tially prefer positive information and rein- perception and memory. When anticipating
forcers when making and processing emo- monetary gains, older adults demonstrate
tional decisions; however, the influence of preserved striatal responses to gains but
context and the extent to which an emo- diminished responses to losses (Samanez-
tional bias is present during decision making Larkin et al., 2007). Also, although older
in older adults are not yet fully elucidated. adults show similar striatal responses to the
presentation of monetary outcomes when
AFFECTIVE DECISION MAKING: the entire outcome period is modeled (Cox,
FUNCTIONAL NEUROIMAGING Aizenstein, & Fiez, 2008; Samanez-Larkin
Consistent with age-related declines in et al., 2007), they show a reduced effect
dopaminergic function and the volume of negative outcomes on striatal signaling
of the dorsal striatum, as reviewed ear- initially after outcomes are presented (Cox
lier, functional neuroimaging studies have et al., 2008). Together, these results are
reported reduced striatal activity in older suggestive of the role of altered affective
adults during reward processing while learn- processing during decision making in later
ing associations (Mell et al., 2009) and when adulthood, particularly when older adults
anticipating rewards (Dreher et al., 2008; are considering potential economic losses.
Schott et al., 2007; although see Samanez- Consistent with evidence of altered PFC
Larkin et al., 2007). For example, Mell function during affective processing in later
et al. (2009) found differential age effects life, older adults show changes in the
during early versus late stages of reward PFC during reward learning and decision
learning from feedback on a probabalistic making. In one investigation, older adults
object reversal task. During the late phase showed decreased activation of the DLPFC,
of learning, older adults showed reduced but less deactivation in the medial PFC
activation in the striatum to rewards com- during the receipt of reward, compared
pared to young adults. In the early learn- to younger adults (Dreher et al., 2008).
ing stage, however, older adults showed Along the same lines, older adults showed
greater ventral striatal response to reward greater activation in the DLPFC during
processing. Given that behavioral perfor- decision making when reward contingencies
mance in older adults was worse overall, were already well learned, which potentially
646 PEGGY L. ST. JACQUES, AMY WINECOFF, & ROBERTO CABEZA
suggests that older adults over-recruit dor- experienced, or how an emotion is expressed
solateral aspects of the PFC to maintain (Gross, 1998b; Chapter 16). Aging is asso-
and monitor performance even after learn- ciated with increased control over emo-
ing has already taken place (Mell et al., tional experiences in everyday life (Gross
2009). Because in vivo positron emission et al., 1997). Within laboratory settings,
tomography research has shown that the older adults differentially employ both rel-
relationship between mesolimbic and pre- atively automatic and controlled forms of
frontal dopamine systems is altered in aging, emotion regulation. First, older adults seem
these differences may reflect age-related to automatically engage emotional regula-
dopaminergic decline (Dreher et al., 2008), tion strategies when not instructed to do
which could account for age-related alter- so. For example, studies of spatial attention
ations in the reward system as well as in have shown that, despite a preserved abil-
the PFC. Interestingly, one study has linked ity to perceive threat-related stimuli more
changes in the PFC during hedonic pro- quickly than neutral stimuli (Mather &
cessing to experience of gustatory stimuli. Knight, 2006), older adults are less likely
Regardless of the specific taste, older adults to attend to faces displaying negative emo-
showed greater responses not only in brain tion (Isaacowitz, Wadlinger, Goren, & Wil-
circuitry involved in affective processing, son, 2006; Mather & Carstensen, 2003) and
such as the caudate and amygdala, but also more likely to attend to faces displaying pos-
in the anterior cingulate and medial pre- itive emotion (Isaacowitz et al., 2006), when
frontal cortex (Jacobson, Green, & Murphy, compared to young adults. This attention
2010). These results suggest that age affects deployment to positive versus negative stim-
the processing of motivational stimuli both uli is a type of antecedent-focused emotion
in limbic and prefrontal regions. regulation strategy (Gross, 1998b), in which
In sum, the handful of neuroimaging older adults seem to be experts (Charles &
studies of reward processing and decision Carstensen, 2007).
making in later life show that older adults Older adults also differ in their imple-
demonstrate altered patterns of reward pro- mentation of controlled emotion regula-
cessing and that these patterns may result tion. However, the age-related differences
not only from dopaminergic decline but in emotion regulation under explicit instruc-
also from altered emotional functioning in tions are not fully understood. For exam-
aging. Although studies of reward learning ple, contrary to studies in which they
and decision making in older adults have deploy automatic emotion regulation, older
typically interpreted findings of differential adults show a decreased ability to emotion-
PFC recruitment as representing cognitive ally detach themselves from negative stim-
decline, these studies cannot rule out the uli (Shiota & Levenson, 2009; Winecoff,
potential influence of altered emotional con- LaBar, Madden, Cabeza, & Huettel, 2011).
trol processes acting during decision making Despite this finding, older adults also show
as a source of PFC engagement. It will be a preserved ability to suppress behavioral
important in future research to dissociate responses to emotional stimuli (Kunzmann,
the effects of declining cognitive and neural Kupperbusch, & Levenson, 2005; Shiota &
function from changes in emotional biases Levenson, 2009) and an enhanced ability to
in later life because they may affect decision regulate emotions by creating more positive
making in older adults in different ways. interpretations of negative stimuli compared
to younger adults (Shiota & Levenson, 2009).
Some researchers have speculated that emo-
Emotion Regulation: Behavior
tionally detaching from stimuli relies more
Emotion regulation is a process by which on fluid intelligence, whereas creating posi-
individuals attempt to manage their emo- tive reinterpretations of stimuli relies more
tional experiences by altering which specific on expertise, which potentially explains the
emotion is experienced, when an emotion is age-related variance in performance based
EMOTION AND AGING 647
on emotion regulation strategy (Shiota & amygdala responses for negative stimuli. For
Levenson, 2009); however, this idea does example, we examined age-related differ-
not account for the finding that older adults ences in the functional connectivity with the
are equally as effective as younger adults in amygdala and found an age-related increase
emotionally detaching themselves from pos- in the functional connectivity between the
itive stimuli (Winecoff et al., 2011). Thus, ventral ACC and the amygdala (St. Jacques
the relationship between age and controlled et al., 2010). Importantly, we found a nega-
emotion regulation is not yet fully eluci- tive correlation between these regions dur-
dated. ing the perception of negative pictures that
The engagement of both automatic and older adults subjectively rated as neutral
controlled forms of emotional regulation and also a subsequent decrease in amygdala
may also depend on the availability of cogni- activity, which suggests the engagement of
tive resources. For example, although selec- emotion regulation.
tive attention is not very cognitively taxing Implementing emotion regulation strate-
(e.g., Allard & Isaacowitz, 2008), whether gies via increased PFC recruitment (Ochsner
older adults use selective attention as an & Gross, 2005) can also reduce striatal acti-
emotional regulation strategy may depend vation to anticipated gains (Delgado, Gillis,
on intact cognitive resources (Mather & & Phelps, 2008). A clear link between altered
Carstensen, 2005). This poses an interest- striatal function and automatic forms of
ing problem for the study of aging: To the emotion regulation in aging has not yet
extent that emotion regulatory strategies been established. However, in one recent
rely on cognitive resources, how is emo- investigation using emotional images, young
tional functioning maintained in later life? adults showed increased functional connec-
Because functional neuroimaging allows for tivity between the medial PFC and the
the investigation of the underlying mecha- striatum during the presentation of nega-
nisms of emotion regulation, it provides one tive stimuli, whereas older adults showed
promising avenue for answering this ques- greater functional connectivity between
tion. In particular, functional neuroimaging these regions during the presentation of pos-
studies can help us understand how the neu- itively valenced stimuli (Ritchey et al., 2011).
ral mechanisms of automatic versus con- Though this study did not focus on reward
trolled regulation differ across the lifespan. processing or decision making per se, it sug-
gests that a FADE-like pattern might extend
EMOTIONAL REGULATION: FUNCTIONAL more widely to reward regions of the brain.
NEUROIMAGING Functional neuroimaging studies have
Functional neuroimaging studies provide also begun to investigate age-related changes
some evidence that older adults may during controlled emotion regulation. One
automatically engage emotional regulation form of emotion regulation that has been
strategies although not instructed to do so. studied extensively in young adults is reap-
Consistent with FADE, in many studies, praisal, whereby the emotional meaning of
the age-related increase in frontal activity a stimulus is transformed through the use
was coupled with a reduction in amyg- of some cognitive strategy. This antecedent-
dala activity during perception (Gunning- focused emotion regulation strategy, as
Dixon et al., 2003; Tessitore et al., 2005; also does attentional deployment, operates early
see Fischer et al., 2005; Iidaka et al., 2002; in the emotion-generation period (Gross,
Samanez-Larkin et al., 2007; St. Jacques 1998b) and is therefore more likely to reduce
et al., 2010; although see Williams et al., emotionally evoked physiological arousal
2006) and retrieval (Murty et al., 2009) of (Gross, 1998a). Given that older adults show
negative stimuli. Thus, one possibility is that declines in cognitive control but preserved
older adults’ enhanced emotion regulation or improved emotional functioning (Mather
strategies lead to the recruitment of PFC- & Carstensen, 2005), one hypothesis might
mediated control processes that dampen be that older adults rely on different
648 PEGGY L. ST. JACQUES, AMY WINECOFF, & ROBERTO CABEZA
neural networks for voluntary emotion regu- well as potential age-invariant or age-related
lation than younger adults; however, prelim- differences in the emotion regulation of
inary data suggest that this is not the case. In positive and negative valence.
one study of reappraisal in an older cohort, Frontoamygdalar Age-Related Differences
older adults engaged diverse aspects of the in Emotion (FADE) Evidence from age-
PFC during emotion regulation, and par- related functional neuroimaging studies
ticipants showing the greatest reappraisal- across emotional perception, memory, and
related decreases in amygdala activation emotional regulation domains generally
showed increased activation in the medial shows the activity pattern we called FADE;
PFC relative to those who did not show namely, age-related increases in PFC activity
this effect (Urry et al., 2006). Similarly, in that are sometimes coupled with decreases
a study directly comparing older and young in amygdalar responses to negative stim-
adults’ neural responses to reappraisal, few uli. Few functional neuroimaging stud-
age-related differences emerged (Winecoff ies have investigated age-related changes
et al., 2011). In both young and older adults, in decision making and reward; however,
cognitive reappraisal recruited PFC regions the available evidence points to a simi-
previously observed in emotion regulation lar age-related increase in frontal recruit-
(Kim & Hamann, 2007; Ochsner et al., ment and reduced function in the stria-
2002; Ochsner et al., 2004), whereas the tum, suggesting a more general pattern of
experience of emotion recruited the amyg- age-related cortical increases and subcortical
dala. Further, across age groups, task-related decreases (Samanez-Larkin & Carstensen,
decreases in amygdala activation were func- 2011). Although FADE refers to func-
tionally coupled with increased activation tional changes in brain activity, it can
in the dorsolateral prefrontal cortex dur- also be related to structural changes in
ing reappraisal. The only age-related differ- anatomy. Thus, FADE can be linked to both
ences to emerge were in the left inferior the anatomical preservation hypothesis and
frontal gyrus and left medial and superior the functional compensation hypothesis, as
temporal sulcus, where older adults showed described in the following two subsections.
reduced recruitment when actively attempt-
ing to regulate responses to negative (but not FADE AND THE ANATOMICAL
positive) stimuli. Consistent with the sug- PRESERVATION HYPOTHESIS
gestion that cognitive control is necessary Although findings of age-related reductions
for emotion regulation (Mather & Knight, in amygdalar activity during perception (Fis-
2005), higher performance on a battery of cher et al., 2005; Iidaka et al., 2002; Tes-
cognitive tests predicted greater reappraisal- sitore et al., 2005) and memory (Fischer,
related decreases in the amygdala even after 2010; Murty et al., 2009) involving nega-
controlling for the effects of age (Winecoff tive emotions may appear as inconsistent
et al., 2011; also see Ritchey et al., 2011). with evidence of little age-related atrophy in
Collectively, these results suggest that the amygdala, the anatomical preservation
automatic engagement of emotional regu- of the amygdala can explain why activity
lation strategies may underlie some of the reductions can vary as a function of stimulus
age-related changes in the neural correlates type and are less frequent for positive stim-
supporting emotional processing, whereas uli. The fact that age-related atrophy is min-
controlled emotion regulation relies on imal in the amygdala supports the idea that
generally similar brain systems across the age-related reductions in amygdala activity
lifespan. Future research is needed to better for negative stimuli reflect a change in
understand the boundary conditions of emotional-processing strategies rather than
these effects in order to reconcile discrep- simple anatomical deterioration.
ancies in the neural mechanisms supporting A similar argument can be made about
age-related changes in automatic versus age-related increases in PFC activity. The
controlled forms of emotion regulation, as fact that these increases occur both in
EMOTION AND AGING 649
lateral PFC regions that show substantial PFC increases compensate for the amygdalar
age-related atrophy and in medial PFC decreases. In fact, if one interprets FADE
regions that show relatively less atrophy sug- in terms of emotion regulation, the effect
gests that these increases reflect a change would be in the opposite direction, with
in the way older adults process emotions, PFC-mediated control processes dampening
rather than a simple reflection of anatomi- amygdala-mediated processing of negative
cal preservation vs. decline. emotions. At any rate, further research is
Finally, the anatomical preservation required to determine if the FADE pattern
hypothesis cannot easily account for age within the emotion domain fits with global
effects on striatal activity, which often age-related activation patterns observed in
shows reductions in both dorsal and ven- other cognitive domains, such as PASA.
tral regions, despite anatomical evidence As noted earlier, the fact that age-related
that age-related atrophy is minimal in ven- PFC increases during emotional process-
tral regions. Although global striatal reduc- ing occur both in regions showing substan-
tions could be attributed to age-related tial age-related atrophy, such as the lateral
dopamine deficits, the latter cannot explain PFC, and in regions showing relatively less
why some studies (e.g., Jacobson et al., anatomical decline, such as the medial PFC,
2010; Samanez-Larkin et al., 2007) found age- suggests that these increases reflect changes
related increases in striatal activity in some in processing strategies rather than anatom-
conditions. ical changes. What is unclear is what these
In sum, age-related changes in neural processing strategy changes are.
activity during emotional processing cannot One possibility is that age-related PFC
be simply attributed to anatomical preser- increases could reflect emotion regulation.
vation or decline. To better link changes in Emotional well-being in aging has been asso-
anatomy to the functional changes described ciated with a shift from automatic process-
by FADE, future studies could investi- ing to more controlled processing of emo-
gate the relationship between volumetric tions via the recruitment of the medial PFC
changes in regions of interest with changes (Williams et al., 2006). As reviewed earlier,
in the activity of these regions. the pattern of activation across a number
of studies suggests that older adults poten-
FADE AND THE FUNCTIONAL tially engage automatic emotion regulation
COMPENSATION HYPOTHESIS strategies when processing emotional stim-
The PFC component of FADE is clearly uli by recruiting frontal control regions that
consistent with evidence of age-related PFC may dampen emotional responses mediated
increases in various cognitive domains, as by the amygdala. Interestingly, when the
summarized by the posterior-anterior shift differential engagement of these strategies
in aging (PASA) and hemispheric asymme- is equated across age groups via explicit
try reduction in older adults (HAROLD) instructions, few age-related differences are
patterns (Dennis & Cabeza, 2008). FADE observed (Winecoff et al., 2011).
fits particularly well with the PASA pat- Second, the frontal increase could be
tern, which comprises not only age-related an instance of PASA, which is frequently
increases in anterior brain regions but also observed in nonemotional domains (Den-
age-related decreases in posterior brain nis & Cabeza, 2008) and thus could reflect a
regions. Although it is tempting to see the compensatory strategy. Consistent with this
age-related amygdalar reductions in FADE idea, we and others have found that the
as an example of the posterior reductions increase in PFC activity during emotional
in PASA, there is an important difference processing was coupled with a decrease in
in how these reductions are conceptual- the recruitment of posterior regions (St.
ized. In PASA, PFC increases are interpreted Jacques, Dolcos, et al. 2009; St. Jacques
as compensating for the posterior reduc- et al., 2010; Tessitore et al., 2005; also see
tions, whereas FADE does not assume that Gunning-Dixon et al., 2003; Iidaka et al.,
650 PEGGY L. ST. JACQUES, AMY WINECOFF, & ROBERTO CABEZA
2002). Further, in keeping with the compen- necessarily an effective regulation strategy.
satory account of PASA, we also found a sig- Thus, understanding the specific contribu-
nificant correlation between the age-related tions of each of these processes and their
increase in frontal activity and the reduction interactions is a major challenge for future
in visual cortex activity during emotional research.
perception (St. Jacques et al., 2010) and
that the age-related frontal increase was pre-
dictive of subsequent memory for negative Linking Brain Data to Theories
stimuli (St. Jacques, Dolcos, et al., 2009). of Emotional Aging
Another possibility is that age-related
increases in the medial PFC could reflect The number of functional neuroimaging
an augmentation of self-referential processes studies of emotion and aging is growing
(Kensinger & Leclerc, 2009). One line of rapidly; however, many of the aforemen-
evidence supporting this interpretation is tioned studies do not directly link age-
that medial PFC recruitment in older adults related changes in neural activation to cog-
has been shown to vary as a function of nitive and social theories of emotional aging.
valence (Kensinger & Schacter, 2008; Leclerc There are at least four different theories of
& Kensinger, 2008). For example, Leclerc emotional changes in aging: the socioemo-
and Kensinger (2008) found an age-related tional selectivity theory (Carstensen, Mikels,
reversal in the medial PFC, such that older & Mather, 2006), dynamic integration theory
adults recruited this region more for pos- (Labouvie-Vief, 2003, 2009), learning and
itive stimuli and less for negative stim- practice theory (Blanchard-Fields, 2007),
uli (although see Williams et al., 2006). and the byproduct of biological decline the-
They suggested that older adults might ory (Cacioppo, Berntson, Bechara, Tranel,
interpret positive stimuli in a more self- & Hawkley, 2011). Several of these theo-
relevant way. In keeping with this interpre- ries were originally proposed to account
tation, a study directly interrogating age- for behavioral data, and hence, they do
related differences in self-referential pro- not include assumptions about brain mecha-
cessing for emotional stimuli found that nisms and cannot make predictions for func-
older adults recruited medial PFC regions tional neuroimaging. However, using cur-
to a greater extent for self-related positive rent knowledge regarding the neural bases
words (Gutchess, Kensinger, & Schacter, of various cognitive and emotional pro-
2007). Given that older adults tend to engage cesses, one may expand these theories with
the default network more during cogni- assumptions about neural mechanisms and
tive tasks (Grady, Springer, Hongwanishkul, derive predictions for functional neuroimag-
McIntosh, & Winocur, 2006), one possibility ing. We next describe each of these theories,
is that the positivity shift observed in per- how they could be expanded to incorporate
ception and memory studies might result hypotheses regarding brain function, and
from an age-related increase in the tendency whether the derived predictions fit available
to interpret information in a self-relevant functional neuroimaging evidence.
manner (cf. Kensinger & Leclerc, 2009).
It is worth noting that emotion regula-
Socioemotional Selectivity Theory
tion, compensation, and self-referential pro-
cessing accounts are not incompatible with BASIC THEORY AND EXPANDED THEORY
each other. In fact, emotional regulation WITH NEURAL ASSUMPTIONS AND
could be seen as a form of compensation PREDICTIONS
(St. Jacques et al., 2010), and self-referential Socioemotional selectivity theory (SST)
processing could be seen as an emotion reg- postulates that aging is associated with a
ulation strategy (Kensinger & Leclerc, 2009). limited perspective on time, which leads
Yet, not all forms of emotion regulation to motivational differences in allocating
are beneficial for performance (i.e., compen- attention to information (Carstensen, Fung,
satory), and self-referential processing is not & Charles, 2003; Mather & Carstensen,
EMOTION AND AGING 651
2005). Specifically, the theory has two (also see Samanez-Larkin & Carstensen,
predictions: (1) Aging involves the greater 2011; St. Jacques, Bessette-Symons, et al.,
allocation of cognitive resources to emo- 2009). Emerging findings from the domain
tional stimuli, and (2) older adults are more of decision making suggest that a more gen-
likely to allocate these limited resources to eral pattern of age-related reduction in sub-
information that enhances their mood and cortical activation and increase in cortical
well-being. Because of this mood-enhancing activation could explain both the amyg-
goal, older adults are expected to be more dalar and striatal findings (Samanez-Larkin
sensitive to positive information and less & Carstensen, 2011). These studies are in line
sensitive to or avoidant of negative informa- with the expanded SST, which suggests that
tion, a phenomenon known as the positivity older adults’ enhanced emotion regulation
effect (Carstensen & Mikels, 2005; Carstensen strategies lead to the recruitment of PFC-
et al., 2006). In the context of studies com- mediated control processes that dampen
paring attention, memory, or decisions amygdala or striatal responses for negative
involving positive versus negative stim- stimuli and losses. Further supporting this
uli, the positivity effect is defined as an idea are studies that have examined cou-
age-related shift in the overall ratio of pling between frontal and amgydalar regions
positive-to-negative material attended (Murty et al., 2009; St. Jacques et al., 2010;
to, remembered, or selected (Scheibe & St. Jacques, Dolcos, et al., 2009). Some of
Carstensen, 2010). Further, SST proposes the studies that have explicitly queried emo-
that the positivity effect is more likely to tion regulation are also consistent with the
occur on tasks that require greater con- expanded SST. For example, Urry et al.
trolled emotional processing and less so on (2006) directly asked older adults to regu-
more automatic emotional tasks (Mather, late emotions while viewing negative pic-
2006). In sum, SST suggests that older tures and found that, compared to pas-
adults are more likely to allocate cognitive sive viewing, when older adults were asked
resources to regulate their emotions. to decrease their emotional responses they
The fact that functional neuroimaging recruited greater vmPFC activity coupled
studies of emotional aging frequently discuss with a reduction in the recruitment of the
the popular SST suggests that this theory amygdala. In fact, emotional well-being in
is amenable to brain-based interpretations aging has been associated with a shift from
(for related reviews, see Knight & Mather, automatic processing to more controlled
2006; Mather, 2006; Samanez-Larkin & processing of emotions via the recruit-
Carstensen, 2011; Scheibe & Carstensen, ment of the medial PFC (Williams et al.,
2010). Given that SST assumes that older 2006). Moreover, consistent with predictions
adults use controlled processes to up- from SST, the frontal over-recruitment may
regulate responses to positive stimuli and/or occur only on controlled emotional tasks
down-regulate responses to negative stim- but not on automatic ones (Ritchey et al.,
uli, this theory predicts that during emo- 2011).
tional processing older adults should show
(1) increased control-related PFC activity
Dynamic Integration Theory
and (2) the effects of this control in the form
of shifts in amygdalar responses to positive BASIC THEORY AND EXPANDED THEORY
vs. negative stimuli or in striatal responses WITH NEURAL ASSUMPTIONS AND
to gains vs. losses. PREDICTIONS
According to dynamic integration the-
FUNCTIONAL NEUROIMAGING EVIDENCE ory (DIT; Labouvie-Vief, 2009) age-related
Consistent with the expanded SST, sev- changes in emotional processing depend on
eral functional neuroimaging studies of emo- the interaction between situational circum-
tional aging have found the FADE pattern: stances and individual differences, which
an age-related increase in frontal recruit- determines the effectiveness of emotion
ment and altered amygdala recruitment regulation. DIT suggests that the level of
652 PEGGY L. ST. JACQUES, AMY WINECOFF, & ROBERTO CABEZA
considering the link between brain mecha- relevant fMRI study, Gutchess, Kensinger,
nisms and DIT leads to some novel questions and Schacter (2007) asked young and older
for future research. In particular, a greater adults to judge whether positive and neg-
number of studies are needed to examine the ative adjectives described themselves or
influence of emotional activation, emotional Albert Einstein. Both the self and other
complexity, cognitive demands, and individ- conditions are relatively familiar situations;
ual differences in performance, which are however, presumably referencing the self
critical for understanding age-related differ- is more familiar than referencing Einstein.
ences in the neural correlates of emotion. The fMRI results revealed that, compared to
young adults, older adults engaged a region
in the dorsal medial PFC to a greater extent
Learning and Practice Theory
for positive versus negative self-referencing,
BASIC THEORY AND EXPANDED THEORY but there was no interaction with valence
WITH NEURAL ASSUMPTIONS AND when making a judgment about another
PREDICTIONS person. The Gutchess et al. (2007) findings
According to the theory of learning and provide preliminary partial support for the
practice, older adults have developed exper- expanded learning and practice theory, in
tise in emotion regulation through sub- that they suggest that older adults poten-
stantial experience across the lifespan tially engage increased controlled process-
(Blanchard-Fields, 2007; Hess, 2005). Older ing for more familiar positive versus neg-
adults’ expertise in dealing with socioe- ative stimuli. However, this same contrast
motional situations results in more com- revealed no activation in subcortical regions;
plex, flexible, and mature emotion regula- thus, it is not clear whether the recruitment
tion strategies compared to young adults. of controlled processes was in the service
The increasing ease with which older adults of emotion regulation. Further research is
apply emotion regulation strategies in their needed to directly test age-related differ-
daily lives potentially underlies age-related ences in emotional processes of familiar ver-
reductions in the perceptions of stress due to sus unfamiliar situations.
health problems, bereavement, and environ-
mental catastrophes (for reviews see Charles
Byproduct of Biological Decline
& Carstensen, 2007, 2010).
Theory
Given the emphasis on experience, the
learning and practice theory suggests that BASIC THEORY AND EXPANDED THEORY
older adults will be most effective in apply- WITH NEURAL ASSUMPTIONS AND
ing emotion regulation strategies to familiar PREDICTIONS
situations. Thus, the expanded learning and Unlike the previous cognitive and social the-
practice theory predicts that older adults ories of emotional aging, biological decline
should show (1) increased control-related theories directly link age-related differences
PFC activity and (2) the effects of this con- to brain function. According to one such
trol in the form of up- or down-regulation theory, the aging-brain model (Cacioppo
of amygdala or striatal activity to a greater et al., 2011), age-related changes in emotion
extent in familiar versus unfamiliar situa- are the byproduct of biological decline. The
tions. aging-brain model postulates that there is
a valence shift in the response of amygdala
FUNCTIONAL NEUROIMAGING EVIDENCE activation due to age-related attenuations in
To date there is little functional neuroimag- arousal for negative stimuli. The age-related
ing evidence supporting the expanded the- reduction in arousal response to negative
ory of learning and practice because very few stimuli reduces the emotional enhancement
studies have examined age-related differ- of memory (EEM) for negative stimuli, but
ences during emotional processing for famil- increases subjective well-being. Further, the
iar versus novel situations. In one potentially aging-brain model suggests that attenua-
654 PEGGY L. ST. JACQUES, AMY WINECOFF, & ROBERTO CABEZA
tion in arousal responses to negative stimuli lower arousing emotional stimuli. However,
potentially impairs decision-making pro- we (St. Jacques et al., 2010) showed that
cesses that rely on negative feedback. Propo- amygdala activation was reduced in older
nents of the aging-brain model suggest that adults for the subset of less arousing negative
the pattern of age-related changes in emo- stimuli, but not for more arousing negative
tion is similar to that in patients with selec- stimuli, which is the exact opposite of the
tive amygdala lesions, implying that amyg- aging-brain model hypothesis. Further, as
dala dysfunction is the basis of emotional suggested earlier, age-related differences in
changes in aging. the amygdalar response to negative stimuli
Taking directly from the aging-brain tended to occur when there were also age-
model hypothesis, there will be age-related related differences in behavioral ratings for
reductions in amygdala activation for neg- negative stimuli, suggesting that these stim-
ative stimuli, but not for positive stimuli. uli were ineffective in engendering a strong
Unlike the previous theories, the aging-brain emotional response in older adults (although
model is agnostic with respect to age-related see Ritchey et al., 2011). It will be important
changes in the PFC because it posits that the for future research to test the brain hypothe-
attenuation of arousal responses to negative ses of the aging-brain model using a range of
stimuli are due to dysfunction of the amyg- arousing stimuli.
dala. The aging-brain model relies on the The current research does not appear
assumption that arousal drives age-related to provide much support for the hypoth-
changes; thus, one additional hypothesis is esis that age-related changes in the neural
that age-related differences in the amygdalar mechanisms of emotional aging are merely
responses for low-arousing emotional stim- a byproduct of biological decline (Scheibe
uli will be minimal. & Carstensen, 2010). Cacioppo et al. (2011)
argue that patients with amygdalar lesions
FUNCTIONAL NEUROIMAGING EVIDENCE seem to show similar findings to those
Several functional neuroimaging studies of older adults, suggesting a possible link
have observed age-related reductions in the between amygdalar damage and age-related
recruitment of the amygdala for negative changes in emotional processing. However,
stimuli. Although very few studies have the studies reviewed here generally suggest
examined age-related changes for positively that the structure and function of the amyg-
valenced stimuli, the handful of studies to dala remain relatively intact across the lifes-
date generally show age-invariant recruit- pan, although there is some debate regarding
ment for positive stimuli (e.g., Leclerc & the preference of this region for positive ver-
Kensinger, 2008; Mather et al., 2004). Con- sus negative stimuli. Further, let us consider
trary to the aging-brain model, however, the findings from patients with Alzheimer’s
several studies have also observed age- disease, a neurodegenerative disease that
invariant activation in the amygdala for affects the structure and function of the
negative stimuli (e.g., Ritchey et al., 2011; amygdala (for a review, see Chow &
St. Jacques et al., 2010). Based on the aging- Cummings, 2000). In contrast with healthy
brain model one explanation of these dis- older adults, patients with Alzheimer’s dis-
crepant findings is that the level of arousal ease show a disruption of emotional pro-
differed in studies that failed to find an age- cesses (for a review, see Kensinger, 2009).
related difference in amygdala response for In sum, these findings bring into question
negative stimuli. According to the aging- whether emotional changes in healthy aging
brain model, age-related differences in the are due to biological decline in emotional
recruitment of the amygdala should be min- regions such as the amygdala. It is possible
imal when emotional stimuli are low arous- that future research will reveal subtle differ-
ing. Thus, studies that did not find age- ences in age-related biological decline that
related differences in amygdalar responses could account for the findings in emotional
for negative stimuli may have included processes.
EMOTION AND AGING 655
Frank, M., & Kong, L. (2008). Learning to avoid in Heilman, R., Crisan, L., Houser, D., Miclea,
older age. Psychology and Aging, 23(2), 392–98. M., & Miu, A. (2010). Emotion regulation and
Grady, C. L., Maisog, J. M., Horwitz, B., Unger- decision making under risk and uncertainty.
leider, L. G., Mentis, M. J., Salerno, J. A., Emotion, 10(2), 257–65.
et al. (1994). Age-related changes in cortical Hess, T. M. (2005). Memory and aging in context.
blood flow activation during visual processing Psychological Bulletin, 131(3), 383–406.
of faces and location. Journal of Neuroscience, Honea, R. A., Vidoni, E., Harsha, A., & Burns,
14(3, Pt. 2), 1450–62. J. M. (2009). Impact of APOE on the healthy
Grady, C. L., Springer, M. V., Hongwanishkul, aging brain: A voxel-based MRI and DTI
D., McIntosh, A. R., & Winocur, G. (2006). study. Journal of Alzheimers Disease, 18(3),
Age-related changes in brain activity across 553–64.
the adult lifespan. Journal of Cognitive Neu- Idika, T., Okada, T., Murata, T., Omori, M.,
roscience, 18(2), 227–41. Kosaka, H., Sadato, N., et al. (2002). Age-
Gross, J. (1998a). Antecedent-and response- related differences in the medial temporal
focused emotion regulation: Divergent con- lobe responses to emotional faces as revealed
sequences for experience, expression, and by fMRI. Hippocampus, 12(3), 352–62.
physiology. Journal of Personality and Social Isaacowitz, D., Wadlinger, H., Goren, D., &
Psychology, 74, 224–37. Wilson, H. (2006). Is there an age-related pos-
Gross, J. (1998b). The emerging field of emotion itivity effect in visual attention? A comparison
regulation: An integrative review. Review of of two methodologies. Emotion, 6(3), 511–16.
General Psychology, 2(3), 271–99. Jacobson, A., Green, E., & Murphy, C. (2010).
Gross, J., Carstensen, L., Pasupathi, M., Tsai, Age-related functional changes in gustatory
J., Skorpen, C., & Hsu, A. (1997). Emotion and reward processing regions: An fMRI
and aging: Experience, expression, and con- study. Neuroimage, 53, 602–10.
trol. Psychology and Aging, 12, 590–99. Kennedy, Q., & Mather, M. (2007). Aging, affect
Gruhn, D., & Scheibe, S. (2008). Age-related and decision making. In R. Baumeister &
differences in valence and arousal ratings of G. Lowenstein (Eds.), Do emotions help or
pictures from the International Affective Pic- hurt decision making? A hedgefoxian perspective
ture System (IAPS): Do ratings become more (pp. 245–65). New York: Russel Sage Founda-
extreme with age? Behavioral Research Meth- tion.
ods, 40(2), 512–21. Kensinger, E. A. (2004). Remembering emotional
Gruhn, D., Smith, J., & Baltes, P. B. (2005). experiences: The contribution of valence
No aging bias favoring memory for positive and arousal. Reviews of Neuroscience, 15(4),
material: Evidence from a heterogeneity- 241–51.
homogeneity list paradigm using emotionally Kensinger, E. A. (2008). Age differences in mem-
toned words. Psychology and Aging, 20(4), 579– ory for arousing and nonarousing emotional
88. words. Journal of Gerontology Series B: Psycho-
Gunning-Dixon, F. M., Gur, R. C., Perkins, logical and Social Sciences, 63(1), P13–18.
A. C., Schroeder, L., Turner, T., Turetsky, Kensinger, E. A. (2009). Emotional memory across
B. I., et al. (2003). Age-related differences in the adult lifespan. New York: Taylor & Francis.
brain activation during emotional face pro- Kensinger, E. A., & Leclerc, C. M. (2009). Age-
cessing. Neurobiology of Aging, 24(2), 285–95. related changes in the neural mechanisms
Gunning-Dixon, F. M., & Raz, N. (2003). Neu- supporting emotion processing and emotional
roanatomical correlates of selected executive memory. European Journal of Cognitive Psy-
functions in middle-aged and older adults: chology, 21, 192–215.
A prospective MRI study. Neuropsychologia, Kensinger, E. A., & Schacter, D. L. (2008). Neural
41(14), 1929–41. processes supporting young and older adults’
Gutchess, A. H., Kensinger, E. A., & Schac- emotional memories. Journal of Cognitive Neu-
ter, D. L. (2007). Aging, self-referencing, and roscience, 20(7), 1161–73.
medial prefrontal cortex. Social Neuroscience, Kim, S., & Hamann, S. (2007). Neural correlates
2(2), 117–33. of positive and negative emotion regulation.
Gutchess, A. H., Kensinger, E. A., Yoon, C., & Journal of Cognitive Neuroscience, 19(5), 776–
Schacter, D. L. (2007). Ageing and the self- 98.
reference effect in memory. Memory, 15(8), Knight, M., & Mather, M. (2006). The affec-
822–37. tive neuroscience of aging and its implications
EMOTION AND AGING 659
for cognition. In T. Canli (Ed.), The biological Mather, M., & Carstensen, L. L. (2003). Aging and
bases of personality and individual differences attentional biases for emotional faces. Psycho-
(pp. 159–183). New York: Guilford Press. logical Science, 14(5), 409–15.
Krienen, F. M., Tu, P. C., & Buckner, R. L. (2010). Mather, M., & Carstensen, L. L. (2005). Aging
Clan mentality: Evidence that the medial pre- and motivated cognition: The positivity effect
frontal cortex responds to close others. Journal in attention and memory. Trends in Cognitive
of Neuroscience, 30(41), 13906–15. Sciences, 9(10), 496–502.
Kunzmann, U., Kupperbusch, C. S., & Levenson, Mather, M., & Knight, M. (2005). Goal-directed
R. W. (2005). Behavioral inhibition and ampli- memory: The role of cognitive control in
fication during emotional arousal: A compar- older adults’ emotional memory. Psychology
ison of two age groups. Psychology & Aging, and Aging, 20(4), 554.
20(1), 144–58. Mather, M., & Knight, M. (2006). Angry faces
Labouvie-Vief, G. (2003). Dynamic integration: get noticed quickly: Threat detection is not
Affect, cognition and the self in adulthood. impaired among older adults. Journals of
Current Directions in Psychological Science, Gerontology Series B: Psychological Sciences and
12(6), 201–6. Social Sciences, 61(1), P54.
Labouvie-Vief, G. (2009). Dynamic integration Mell, T., Wartenburger, I., Marschner, A., Vill-
theory: Emotion, cognition and equilibrium ringer, A., Reischies, F., & Heekeren, H.
in later life. In V. L. Bengtson, D. Gans, N. (2009). Altered function of ventral striatum
M. Putney, & M. Silverstein (Eds.), Handbook during reward-based decision making in old
of theory of aging (2nd ed., pp. 277–93). New age, Frontiers in Human Neuroscience, 3, 34.
York: Springer. Mikels, J. A., & Reed, A. E. (2009). Monetary
Leclerc, C. M., & Kensinger, E. A. (2008). Age- losses do not loom large in later life: Age
related differences in medial prefrontal activa- differences in the framing effect. Journals of
tion in response to emotional images. Cogni- Gerontology Series B: Psychological Sciences and
tive, Affective, & Behavioral Neuroscience, 8(2), Social Sciences, 64B(4), 457–60.
153–64. Miller, E. K., & Cohen, J. D. (2001). An inte-
Leclerc, C. M., & Kensinger, E. A. (2010). Age- grative theory of prefrontal cortex function.
related valence-based reversal in recruitment Annual Reviews in Neuroscience, 24, 167–202.
of medial prefrontal cortex on a visual search Mohr, P.N., Li, S. C., & Heekeren, H. R. (2009)
task. Social Neuroscience, 5(5–6), 560–76. Neuroeconomics and aging: Neuromodula-
Leclerc, C. M., & Kensinger, E. A. (2011). Neural tion of economic decision making in old
processing of emotional pictures and words: A age. Neuroscience & Biobehavioral Reviews, 34,
comparison of young and older adults. Devel- 6878–88.
opmental Neuropsychology, 36, 519–38. Mu, Q., Xie, J., Wen, Z., Weng, Y., & Shuyun,
Lockenhoff, C., & Carstensen, L. (2007). Aging, Z. (1999). A quantitative MR study of the
emotion, and health-related decision strate- hippocampal formation, the amygdala, and
gies: Motivational manipulations can reduce the temporal horn of the lateral ventricle in
age differences. Psychology and Aging, 22(1), healthy subjects 40 to 90 years of age. Ameri-
134–46. can Journal of Neuroradiology, 20(2), 207–11.
Logan, J. M., Sanders, A. L., Snyder, A. Z., Murphy, N. A., & Isaacowitz, D. M. (2008). Pref-
Morris, J. C., & Buckner, R. L. (2002). erences for emotional information in older
Under-recruitment and nonselective recruit- and younger adults: A meta-analysis of mem-
ment: Dissociable neural mechanisms associ- ory and attention tasks. Psychology and Aging,
ated with aging. Neuron, 33, 827–40. 23(2), 263–86.
Mather, M. (2006). Why memories may become Murty, V. P., Sambataro, F., Das, S., Tan, H. Y.,
more positive as people age. In B. Uttl & A. L. Callicott, J. H., Goldberg, T. E., et al. (2009).
Ohta (Eds.), Memory and emotion: Interdisci- Age-related alterations in simple declarative
plinary perspectives (pp. 135–57). Malden, MA: memory and the effect of negative stimu-
Blackwell. lus valence. Journal of Cognitive Neuroscience,
Mather, M., Canli, T., English, T., Whitfield, S., 21(10), 1920–33.
Wais, P., Ochsner, K., et al. (2004). Amyg- Novak, D., & Mather, M. (2007). Aging and vari-
dala responses to emotionally valenced stim- ety seeking. Psychology and Aging, 22(4), 728.
uli in older and younger adults. Psychological Ochsner, K., Bunge, S., Gross, J., & Gabrieli, J.
Science, 15(4), 259–63. (2002). Rethinking feelings: An fMRI study of
660 PEGGY L. ST. JACQUES, AMY WINECOFF, & ROBERTO CABEZA
the cognitive regulation of emotion. Journal of revealed by PET. Journal of Cognitive Neuro-
Cognitive Neuroscience, 14(8), 1215–29. science, 12(1), 174–87.
Ochsner, K., & Gross, J. (2005). The cognitive Ritchey, M., Bessette-Symons, B., Hayes, S., &
control of emotion. Trends in Cognitive Sci- Cabeza, R. (2011). Emotion processing in the
ences, 9(5), 242–49. aging brain is modulated by semantic elabora-
Ochsner, K. N., & Gross, J. J. (2007). The neural tion. Neuropsychologia, 49, 640–50.
architecture of emotional regulation. In J. J. Ritchey, M., Dolcos, F., & Cabeza, R. (2008).
Gross & R. Buck (Eds.), The handbook of emo- Role of amygdala connectivity in the per-
tion regulation (pp. 87–109). New York: Guil- sistence of emotional memories over time:
ford Press. An event-related FMRI investigation. Cerebral
Ochsner, K. N., Ray, R. D., Cooper, J. C., Cortex, 18(11), 2494–2504.
Robertson, E. R., Chopra, S., Gabrieli, J. D., Ruffman, T., Henry, J. D., Livingstone, V., &
et al. (2004). For better or for worse: Neural Phillips, L. H. (2008). A meta-analytic review
systems supporting the cognitive down- and of emotion recognition and aging: Implica-
up-regulation of negative emotion. Neuroim- tions for neuropsychological models of aging.
age, 23(2), 483–99. Neuroscience & Biobehavioral Reviews, 32(4),
O’Doherty, J. P. (2004). Reward representations 863–81.
and reward-related learning in the human Salat, D. H., Kaye, J. A., & Janowsky, J. S.
brain: Insights from neuroimaging. Current (2001). Selective preservation and degener-
Opinions in Neurobiology, 14(6), 769–76. ation within the prefrontal cortex in aging
Peters, E., Hess, T. M., Västfjäll, D., & Auman, and Alzheimer disease. Archives of Neurology.,
C. (2007). Adult age differences in dual infor- 58(9), 1403–8.
mation processes: Implications for the role of Samanez-Larkin, G. R., & Carstensen, L. L.
affective and deliberative processes in older (2011). Socioemotional functioning and the
adults’ decision making. Perspectives on Psy- aging brain. In J. Decety & J. T. Cacioppo
chological Science, 2(1), 1–23. (Eds.), The handbook of social neuroscience
Phan, K. L., Wager, T., Taylor, S. F., & Liberzon, (pp. 507–21). New York: Oxford University
I. (2002). Functional neuroanatomy of emo- Press.
tion: A meta-analysis of emotion activation Samanez-Larkin, G., Gibbs, S., Khanna, K.,
studies in PET and fMRI. Neuroimage, 16(2), Nielsen, L., Carstensen, L., & Knutson, B.
331–48. (2007). Anticipation of monetary gain but not
Phelps, E. A. (2006). Emotion and cognition: loss in healthy older adults. Nature Neuro-
Insights from studies of the human amygdala. science, 10(6), 787–91.
Annual Review of Psychology, 57, 27–53. Samanez-Larkin, G. R., Kuhnen, C. M., Yoo, D.
Raz, N. (2005). The aging brain observed in vivo: J., & Knutson, B. (2010). Variability in nucleus
Differential changes and their modifiers. In accumbens activity mediates age-related sub-
R. Cabeza, L. Nyberg, & D. C. Park (Eds.), optimal financial risk taking. Journal of Neuro-
Long-term memory and aging: A cognitive neu- science, 30(4), 1426–34.
roscience perspective (pp. 19–57). New York: Samanez-Larkin, G. R., Robertson, E. R., Mikels,
Oxford University Press. J. A., Carstensen, L. L., & Gotlib, I. H. (2009).
Raz, N., Lindenberger, U., Rodrigue, K. M., Selective attention to emotion in the aging
Kennedy, K. M., Head, D., Williamson, A., brain. Psychology and Aging, 24(3), 519–29.
et al. (2005). Regional brain changes in aging Scheibe, S., & Carstensen, L. L. (2010). Emotional
healthy adults: General trends, individual dif- aging: Recent findings and future trends. Jour-
ferences and modifiers. Cerebral Cortex, 15(11), nal of Gerontology Series B: Psychological Sci-
1676–89. ences and Social Sciences, 65, 135–44.
Raz, N., Rodrigue, K. M., Kennedy, K. M., Head, Schott, B., Niehaus, L., Wittmann, B., Schtze,
D., Gunning-Dixon, F., & Acker, J. D. (2003). H., Seidenbecher, C., Heinze, H., et al.
Differential aging of the human striatum: Lon- (2007). Ageing and early-stage Parkinson’s dis-
gitudinal evidence. American Journal of Neu- ease affect separable neural mechanisms of
roradiology, 24(9), 1849–56. mesolimbic reward processing. Brain, 130(9),
Reuter-Lorenz, P. A., Jonides, J., Smith, E. E., 2412.
Hartley, A., Miller, A., Marshuetz, C., et al. Shiota, M., & Levenson, R. (2009). Effects
(2000). Age differences in the frontal lateral- of aging on experimentally instructed
ization of verbal and spatial working memory detached reappraisal, positive reappraisal, and
EMOTION AND AGING 661
emotional behavior suppression. Psychology Urry, H., van Reekum, C., Johnstone, T., Kalin,
and Aging, 24(4), 890–900. N., Thurow, M., Schaefer, H., et al. (2006).
Sokol-Hessner, P., Hsu, M., Curley, N., Delgado, Amygdala and ventromedial prefrontal cortex
M., Camerer, C., & Phelps, E. (2009). Think- are inversely coupled during regulation of neg-
ing like a trader selectively reduces individu- ative affect and predict the diurnal pattern of
als’ loss aversion. Proceedings of the National cortisol secretion among older adults. Journal
Academy of Sciences, 106(13), 5035. of Neuroscience, 26(16), 4415.
St. Jacques, P. L., Bessette-Symons, B., & van Dyck, C. H., Seibyl, J. P., Malison, R.
Cabeza, R. (2009). Functional neuroimag- T., Laruelle, M., Wallace, E., Zoghbi, S. S.,
ing studies of aging and emotion: fronto- et al. (1995). Age-related decline in striatal
amygdalar differences during emotional per- dopamine transporter binding with iodine-123-
ception and episodic memory. Journal of the beta-CITSPECT. Journal of Nuclear Medicine,
International Neuropsychological Society, 15(6), 36(7), 1175–81.
819–25. Wang, Y., Chan, G. L., Holden, J. E., Dobko,
St. Jacques, P. L., Conway, M. A., Lowder, M. T., Mak, E., Schulzer, M., et al. (1998). Age-
W., & Cabeza, R. (2011). Watching my mind dependent decline of dopamine D1 receptors
unfold versus yours: An fMRI study using in human brain: A PET study. Synapse, 30(1),
a novel camera technology to examine neu- 56–61.
ral differences in self-projection of self versus Weierich, M., Kensinger, E., Munnell, A., Sass,
other perspectives. Journal of Cognitive Neu- S., Dickerson, B., Wright, C., et al. (2011).
roscience, 23, 1275–84. Older and wiser? An affective science per-
St. Jacques, P. L., Dolcos, F., & Cabeza, R. (2009). spective on age-related challenges in financial
Effects of aging on functional connectivity of decision making, Social Cognitive & Affective
the amygdala for subsequent memory of nega- Neuroscience, 6, 195–206.
tive pictures: A network analysis of fMRI data. Williams, L. M., Brown, K. J., Palmer, D., Lid-
Psychological Science, 20(1), 74–84. dell, B. J., Kemp, A. H., Olivieri, G., et al.
St. Jacques, P., Dolcos, F., & Cabeza, R. (2010). (2006). The mellow years? Aeural basis of
Effects of aging on functional connectivity of improving emotional stability over age. Jour-
the amygdala during negative evaluation: A nal of Neuroscience, 26(24), 6422–30.
network analysis of fMRI data. Neurobiology Winecoff, A., LaBar, K. S., Madden, D. J.,
of Aging, 31(2), 315–27. Cabeza, R., & Huettel, S. A. (2011). Cognitive
St. Jacques, P. L., & Levine, B. (2007). Ageing and neural contributors to emotion regulation
and autobiographical memory for emotional in aging. Social Cognitive and Affective Neuro-
and neutral events. Memory, 15(2): 129–44. science, 6, 165–76.
Tessitore, A., Hariri, A. R., Fera, F., Smith, W. Wright, C. I., Wedig, M. M., Williams, D.,
G., Das, S., Weinberger, D. R., et al. (2005). Rauch, S. L., & Albert, M. S. (2006). Novel
Functional changes in the activity of brain fearful faces activate the amygdala in healthy
regions underlying emotion processing in the young and elderly adults. Neurobiology of
elderly. Psychiatry Research, 139(1), 9–18. Aging, 27(2), 361–74.
Index
663
664 INDEX