Edmund T. Rolls Emotion Elicited by Primary Reinforcers and Following Stimulus-Reinforcement Association Learning ‘What are emotions? Why do we have emotions? What are the rules by which emotion operates? What are the brain mechanisms of emotion, and how can disorders of emotion be understood? What motivates us to work for particular rewards, such as food when we are hungry o water when ‘we are thirsty? How do these motivational control systems operate to ensure that we eat approximately the correct amotint of food to maintain our body weight or to replenish our thirst? What factors account for the overeating and obe- sity that some humans show? Why is the brain built to in- clude reward and punishment systems, rather than in some other way? In this chapter I describe an approach to these issues that leads to ways based on fundamental principles to investigate emotions by using and understanding the processing of pri- mary (unleamed) reinforcers and how learning of associa- tions of other stimuli to these primary reinforcers oceurs. 1 pay special attention to analyses of the operation of these systems in nonhuman primates, because of their close rel- evance to understanding the function of these reward-and- punishment systems in humans. A Theory of Emotion and Some Definitions Emotions can usefully be defined as states elicited by rewards ‘and punishers, including changes in rewards and punishers (Rolls, 1999a, 2005; see also Rolls, 1986a, 1986b, 1990, 20008). A reward is anything for which an animal will work A punisher is anything that an animal will work to escape oF avoid. Some examples of emotion include happiness pro- duced by being given a reward, such as a pleasant touch, praise, ora prize ofa large sum of money, or fear produced by the sound of a rapidly approaching bus, or the sight of an, angry expression on someone's face. We work to avoid such stimuli, Other examples include frustration, anger, or sad ness produced by the omission of an expected reward, such as. prize, or the termination of a reward, such as the death, of loved one; or relief produced by the omission or termi- nation ofa punisher, such as the removal ofa painful stimau- tus or sailing out of danger. These examples indicate how emotions can be produced by the delivery, omission, or ter- rmination of rewards or punishers, and they go some way to indicate how different emotions could be produced and clas sified in terms of the rewards and punishers received, omit ted, or terminated. A diagram that summarizes some of the ‘emotions associated with the delivery of a reward or pun- {sher or a stimulus associated with them or with the omis- sion of a reward or punisher is shown in Figure 9.1. Itshoald, be understood that this diagram summarizes the states that, could be produced by manipulation of the reinforcement ‘contingencies for any one reinforcer. The brain is built to have a large set of different primary reinforcers, and the reinforcer space is in this sense high dimensional Before accepting this approach, we should consider whether there are any exceptions to the proposed rale. Are 37138 Emotion fiicitation ane Ang Frat ae at Saree 3 Figure 9.1. Some ofthe emotions associated with different reinforcement contingencies are indicated. Intensity increases (on # continuous scale away from the center ofthe diagram. The classification scheme created by the different reinforcement contingencies consists, with respect tothe action, of (1) the delivery presentation ofa reward (S+), (2) the presentation of a punisher (5-), 3) the omission ofa reward (S+. extinction) or the termination of a reward (Se time out), and (4) the omission of a punisher (S-; avoidance) or the termination ofa punisher (5+; escape). Adapted from Rolls, 1999a, Figure 3.1. any emotions caused by stimuli, events, or remembered ‘events that are not rewarding or punishing? Do any reward- ing or punishing stimuli not cause emotions? We consider these questions in more detail later. The point is that if there are no major exceptions, or if any exceptions can be clearly encapsulated, then we may have a good working definition atleast of what causes emotions, and this leads to principled ways for eliciting and studying emotion. next consider a slightly more formal definition than rewards or punishers, in which the concept of reinforcers is, introduced, and show how there has been a considerable history in the development of ideas along this line ‘The proposal that emotions can be usefully seen as states produced by instrumental reinforcing stimut follows earlier work by Millenson (1967), Weiskrantz (1968), Gray (1975, 1987) and Rolls (1986a, 1986b; 1990). Instrumental rein- forcers are stimuli that, if their occurrence, termination, ot ‘omission is made contingent on the making of an action, alter, the probability of the future emission of that action (see Dickinson, 1980; Gray, 1975; Lieberman, 2000; Mackintosh, 1983), Rewards and punishers are instrumental reinforcing stimuli, The notion of an action here is that an arbitrary ac- tion, for example, turning right versus turning left, will be performed in order to obtain the reward or avold the pun- Isher, so that there is no prewired connection between the response and the reinforcer. Some stimull are primary (un- leaned) reinforcers (e.g., the taste of food if the animal is Inungry, pain), whereas others may become reinforcing by learning because of their association with such primary re- inforcers, thereby becoming “secondary reinforcers.” This, type of learning may thus be called stmuls-reinforcement association, and it occurs via an associative learning process. A positive reinforcer (such as food) increases the probability ‘of emission of a response on which it is contingent; the pro- cess is termed positive reinforcement, and the outcome is a reward (such as food), & negative reinforcer (such as a pain- ful stimulus) increases the probability of emission of a re- sponse that causes the negative reinforcer to be omitted (as, in active avoidance) or terminated (as in escape), and the procedure is termed negative reinforcement. In contrast, pun- ishment refers to a procedure in which the probability of an, action is decreased. Punishment thus describes procedures in which an action decreases in probability if tis followed bya painful stimulus, asin passive avoidance. Punishment, can also be used to refer toa procedure involving the omis- sion or termination of a reward (‘extinction* and “time out,” respectively), both of which decrease the probability of re sponses (see, further, Dickinson, 1980; Gray, 1975; Lieber man, 2000; Mackintosh, 1983). ‘My argument is that anaffectively positive or “appetitive” stimulus (which produces a state of pleasure) acts operation- ally asa reward, which instrumentally, when delivered, acts a5 a positive reinforcer or, when not delivered (omitted or terminated), acts to decrease the probability of responses on, which itis contingent. Conversely, I argue that an affectively negative or aversive stimulus (which produces an unpleasant state) acts operationally as.a punisher, which instrumentally, when delivered, acts to decrease the probability of responses, on which itis contingent or, when not delivered (escaped from or avoided), acts as a negative reinforcer. The link being made between emotion and instrumental reinforcers s partly an operational link. Most people find that it is not easy to think of exceptions to the statements that emotions occur after rewards or punishers are given (some- times continuing for long afer the eliciting stimulus hasended, as in a mood state); or that rewards and punishers, but Not ‘other stimul, produce emotional states. But the link is deeper than this, as we will ee, in that the theory has been developed that genes specify primary reinforcers in order to encourage the animal to perform arbitrary actions to seek particular goals, thas increasing the probability oftheir own (the genes’ survival into the next generation (Rolls, 1999, 2005). The emotional states elicited by the reinforcers have a number of functions, described later, related to these processes. This foundation has been developed (see Rolls, 1999a, 2005, 2000, 1986a, 1986b, 1990) to show how very wide range of emotions can be accounted for as a result of the ‘operation of a number of factors, including the following all ‘of which can be manipulated to influence the elicitation of emotion: 1. The reinforcement contingency (eg, whether reward-or punishment i given of withheld: see Figure 9.1)Emotion Eliited by Primary Reinforcers and Following Stimulus-Reinforcement Association Learning 139 The intensity ofthe reinforcer (see Figure 9.1). Any environmental stimulus might have a number of different reinforcement associations. (For example, a stiraulus might be associated both with the presenta- tion of a teward and of a punisher, allowing states, such as conflict and guilt to arise.) Emotions elicited by stimuli associated with different primary reinforcers will be different. Emotions elicited by diferent secondary reinforcing stimuli willbe different from each other (even ifthe. primary reinforcer is similar). For example, iftwo different people were each associated with the same primary reinforcer, then the emotions would be different. This ts in line with my hypothesis that ‘emotions consist of states elicited by reinforcers and that these states include whatever representations are needed for the eliciting stimulus, which could be cognitive, and the resulting mood change (Rolls, 1999a, 2005). Moods then may continue in the absence of the eliciting stimulus; or they can some- times be produced, as in depression, in the absence of an eliciting stimulus, perhaps due. to dysregulation in the system that normally enables moods to be long lasting (see Rolls, 19992, 2005). ‘The emotion elicited can depend on whether an active ora passive behavioral response is possible. (For example, if an active behavioral response can occur to the omission of a positive reinforcer, then anger—a state that tends to lead to action—might be produced; bucif only passive behavior is possible, then sadness, depression, or grief might occur.) By combining these six factors, itis possible to account - and also elicit, a very wide range of emotions (for elabo- jon, see Rolls, 1990, and Rolls, 1999a, 2005). It is also nth nocing that emotions can be elicited by the recall of nforcing events as much as by external reinforcing strmuli® 4 that cognitive processing (whether conscious or not) is portant in many emotions, for very complex cognitive acessing may be required to determine whether or not vironmental events are reinforcing, Indeed, emotions nor- Illy consist of cognitive processing that analyzes the stinmu- +, determines its reinforcing valence, and elicits a mood ange ifthe valence is positive or negative. In that an emo- nis produced by a stimulus, philosophers say that emo- ns have an object in the world and that emotional states intentional in that they are about something, We note that ‘ood or affective state may occur in the absence of an ex- nal stimulus, as in some types of depression, but that rally the mood or affective state is produced by an ex- nal stimulus, with the whole process of stimulus tepre- nation, evaluation in terms of reward or punishment, and resulting mood or affect being referred to as emotion. The ermal stimulus may be perceived consciously, but stimuli ww are not perceived consciously may also produce emo- tion, Indeed, there may be separate routes to action for con- scious and unconscious stimuli (Rolls, 19992, 2005, 2004) Three issues receive discussion here (see Rolls, 19992, 2005, 2000a). One is that rewarding stimuli such as the taste of food are not usually described as producing emotional states (though there are cultural differences). Its useful here toseparate rewards related to internal homeostatic need states associated with regulation of the internal milieu—for ex- ample, hunger and thirst—and to nore that these rewards are not universally described as producing emotional states Incontrast, the great majority of rewards and punishers are external stimuli not related to intemal need states such as hunger and thirst, and these stimuli do produce emotional responses. An example is fear produced by the sight of @ stimulus that is about to produce pain. A second issue ts that some philosophers categorize fear, bbut not pain, in the example as an emotion. The distinction, they make may be that primary (unlearned) reinforcers do not, produce emotions, whereas secondary reinforcers (stimuli associated by stimulus-reinforcement leaming with primary reinforcers) do. They describe the pain as a sensation. But neutral stimuli (Such asa table) can produce sensations when touched. Itaccordingly seemsto be much more useful toca egorize stimuli according to whether they are reinforcing (in which case they produce emotions) or are not reinforcing (in, ‘which case they do not produce emotions), Clearly, there isa difference between primary reinforcers and learned reinforc~ ets; but this is most precisely caught by noting that this is the difference, and that operationally itis whether a stimulus is, reinforcing that determines whether itis related to emotion. A third issue is that, as Lam about to discuss, emotional states (ie, those elicited by reinforcers) have many functions, and the implementations of only some of these functions by the brain are associated with emotional feelings, that is, with, conscious emotional states (Rolls, 19992, 2005). Indeed, there is evidence for interesting dissociations in some patients with brain damage between actions performed to reinforc- ‘ng stimuli and what is subjectively reported. In this sense it is biologically and psychologically useful to consider emo- tional states to include more than those states associated with conscious feelings of emotion (Rolls, 1999a, 2005). ‘The Functions of Emotion ‘The functions of emotion also provide insight into the in- vestigation, elicitation, and measurement of emotion in that Faystem is that itis aimed directly toward obtaining sensed sr expected reward by virtue of connections to brain systems, auch asthe basal ganglia, that are concerned with the initia- ion of actions (see Figure 9.2). The expectation may, of vourse, involve behavior to obtain stimuli associated with ‘eward, which might even be present in a chain. This expec- ation is built by stimutus-reinforcement association learn ng in the amygdala and orbitofrontal cortex and reversed ay learning in the orbitofrontal cortex, from which signals nay reach the dopamine system (Rolls, 1999s, 2005) ‘With this first route, behavior is partly controlled by re- ‘vard value of the outcome. Atthe same time, the animal may vvork for the reward only if the costis not too high. Indeed, nthe field of behavioral ecology, animals are often thought >fas performing optimally on some cost-benefit curve (see, 28. Krebs & Kacelnik, 1992), This does not at all mean that he animal thinks about the rewards and performs a cost- >enefit analysis with alot of thoughts about the costs, other cewards available and their costs, and so forth, Instead, it should be taken ro mean thatthe system has evolved in such {way that che variations in the rewards that occur with the Jifferent energy densities or amounts of food and the delay defore itis received can be used as part of the input to a rrechanism that has also been built zo track the costs of ob- aining the food (e.g., energy loss in obtaining i, tisk of pre- ation, etc.) and then to select the current behavior that provides the most “net rewatd,” given many such types of reward and the associated cost Part of the value of having the computation expressed in this reward-minus-cost form {sthat there is then a sultable “currency.” or net reward valve, toenable the animal to select the behavior with currently the ‘most net reward gain (or minimal aversive outcome) The second route in humans involves a computation, ‘with many “if. then’ statements, to implement a plan to obtain a reward, In this case, the reward may actually be deferred as part of the plan, which might involve working fst to obtain one reward and only then to work Tor a second, more highly valued reward, if such a strategy was thought to be optimal overall in terms of resource usage (e.g. ume). In this case, syntax is required, because the many symbols {e.g names of people) that are par ofthe plan rust be cor- rectly linked or bound, Such linking might be of the form: “IFA does this, then B is liely to do this, and this will cause Co do this...” The requirement of syntax for this type of planning implies that an output to language systems that at Teast can implement syntax in the brain is required for this type of planning (see Figure 9.2, and Rolls, 2004a), Thus the explicit language system in humans may allow working for deferred rewards by enabling use ofa one-off, individual plan appropriate for each situation. Another building block for such planning operationsiin the brain may be the type of short- term memory in which the prefrontal cortex fs involved, This short-term memory may be, for example in nonhuman pri- ‘mates, a memory of where in space a response has just been made. A development of this type of short-term-response memory system in humans to enable multiple short-term memories to be held in place correctly, preferably with the temporal order of the different items in the short-term memory coded correctly, may be another building block for the rmultiple-step “if... then” type of computation in order to form a multiple-step plan. Such short-term memories are implemented in the (dorsolateral and inferior convexity) prefrontal cortex of nonhuman primates and humans (see Goldman-Rakic, 1996; Petrides, 1996; Rolls & Deco, 2002) ‘and may be part ofthe reason that prefrontal cortex damage impaits planning (see Rolls & Deco, 2002; Shallice & Bur- 55, 1996). Of these two routes (see Figure 9.2), itis the second, involving syntax, that I suggested earlier is related to con- sciousness (see Rolls, 19994, 2005) The question then arises of how decisions are made in animals such as humans that have both the implitt, direct reviard-based and the explicit, rational planning systems (see Figure 9.2), One particular situation in which the first, impli, system may be especially important is a case in which rapid reactions to stimuli with reward or punishment value must ‘bemade, forthen the direct connections from structures such asthe orbitofrontal cortex to the basal ganglia may allow rapid actions, Anotherisa situation in which there may be too many factors to be taken into account easily by the explicit, rational ‘Planning system, so that therefore the implicit system may beEmotion Elicited by Primary Reinforcers and Following Stimulus-Reinforcement Association Learning 145, used to guide action. 1p contrast, when the implicit system continually makes errors, it would then be beneficial for the ‘organism to switch from automatic, direct action based on dobtaiging what the orbitofrontal cortex system decodes as being the mos: positively reinforcing choice currently avail- able to the explicit, conscious control system that can evalu- ate, with ts long-term planning algorithms, what action should be performed next, Indeed, it would be adaptive forthe ex- plicit system to regularly be assessing performance by the more automat system and to switch itself on to control behavior quite frequently, as otherwise the adaptive value of having the cexplicitsystem would be less than optimal. Another factor that ‘may influence the balance between control by the implicit and explicit systemsis the presence of pharmacological agents such asalcohol, which may alter the balance toward control by the implicit system, may allow the implicit system to influence ‘more the explanations made by the explicit system, and may, within the explicit system, alter the relative value it places on caution and restraint versus commitment to a risky action or plan, ‘There may also be a flow of influence from the explicit verbal system to the implicit system, in that the explicit sys- tem may decide on a plan of action or strategy and exert an influence on the implicit system that will alter the reinforce- ‘ment evaluations made by and the signals produced by the implicit system. As an example, if a pregnant woman feels that she would like to escape a cruel mate but is aware that she may not survive inthe jungle, then it would be adaptive if the explicit system could suppress some aspects of her irmplicit behavior toward her mate so that she does not give signals that she is displeased with her situation, (In the li erature on self-deception, it has been suggested that uncon- scious desires may not be made explicit in consciousness—or may be actually repressed—so as not to compromise the explicit system in what it produces; see, e.g, Alexander, 1975, 1979; Trivers, 1976, 1985; and the review by Nesse & Lloyd, 1992). As another example, the explicit system might, because of its long-term plans, influence the implicit system to increase its response to, for example, a positive reinforcer. One way in which the explicit systera might in- fluence the implicit system is by setting up the conditions in which, for example, when a given stimulus (e.g. person) is present, positive reinforcers are given to facilitate stimulus- reinforcement association learning by the implicit system of the person receiving the positive reinforcers. Conversely, the implicit system may influence the explicit system, for ex- ample, by highlighting certain stimu in the environment that are currently associated with reward to guide the attention of the explicit system to such stim However, there is often a conflict between these systems in that the first, traplicit, system is able to guide behavior particularly to obtain the greatest immediate reinforcement, ‘whereas the explicit system can potentially enable immedi- ate rewards to be deferred and longer-term multistep plans tobe formed. Thistype of conflict wall occur in animals with a syntactic planning ability (as described earlier), that is, in humans and any other animals that have the ability to pro- cessaseries of if. .. then" stages of planning, Thistsa prop- erty of the human language system, andthe extent to which itisa property of nonhuman primates is not yet fully cleat. In any case, such conflict may be an important aspect of the operation of at least the human mind, because itis so essen- tial for humans to correctly decide, at every moment, whether to invest ina relationship or a group that may offer longe-term benefits or whether to directly pursue immediate benefits (Nesse & Lloyd, 1992), ‘Within this framework, we now consider the investiga tion of emotion using primary reinforcers, including the mechanisms by which previously neutral stimulate associ- ated by learning with primary reinforcers, and how this earn- ing can be rapidly reversed The Use of Primary Reinforcers to Elicit Emotion Taste Stim ind Taste Processing in the Primate Brain Pathways Diagrams of the taste and related olfactory pathways in pri- ‘mates are shown in Figures 9.3 and 9.4. Ic is of particular interest that primates have a direct projection from the ros tral part of the nucleus of the solitary tract (NTS) tothe taste thalamus and thus to the primary taste cortex in the frontal operculua and adjoining insula, with no pontine taste area and associated subcortical projections as in rodents (Norgren, 1984; Pritchard et al., 1986). This emphasis on cortical pro- cessing of taste in primates may be related to the great devel- coptnent ofthe cerebral cortex in primates and to the advantage of usingextensive and similar cortical analysis of inputs froma ‘every sensory modality before the analyzed representations from each modality are brought together in multimodal re- _lons, as is documented later. The Secondary Taste Cortex A secondary cortical taste area in primates was discovered by Rolls, Yaxley, and Sienkiewicz (1990) in the caudolateral orbitofrontal cortex, extending several millimeters in front ofthe primary taste cortex. One principle of taste processing ‘is that the tuning of neurons by the secondary taste cortex can become quite specific, with some neurons responding, for example, only to sweet taste, This specific tuning (espe- cially when combined with olfactory inputs) helps to pro- vide a basis for changes in appetite for some but not other foods eaten during a meal Five Prototypical Tastes, including Umami The primary and secondary taste cortices contain many neu- rons that respond best to each ofthe four classical prototypi- cal tastes—sweet, salt, bitter, and sour (Rolls, 1997)—but also many neurons that respond best to umami tastants such146 Emotion Elicitation VISION ‘Thatamus VPL Striatum Behavior Lateral Hypothalamus Autonomic Gato responses ay | fnetion ‘Hunger neuron controted by, glucase uiization, seach de, roy weight Primary SSmatoensory Coto (1.23) Figure 9.3. Schematic diagram showing some of the gustatory, olfactory visual, and somatosen- sory pathways to the orbitofrontal cortex and amygdala and some ofthe outputs of the orbito- frontal conex and amygdala. The secondary taste cortex and the secondary olfactory comtex are within the orbitofrontal cortex. VI « primary visual cortex: V4 = visual cortical area as ghutamate (which is present in many natural foods such 5 tomatoes, mushrooms, and milk; Baylis 6 Rolls, 1991) and inosine monophosphate (which is present in meat and some fish, such as tuna; Rolls, Critchley, Wakeman, &¢ Mason, 1996), This evidence, taken together with the identification of a glutamate taste receptor (Chaudhari, Landin, 6: Roper, 2000), leads to the view that there are five prototypical types of taste information channels, with umami contributing, of ten in combination with corresponding olfactory inputs (Rolls, Critchley, Browning, & Hernadi, 1998), to the flavor of protein In humans studies using functional magnetic resonance, imaging ((MRI) have shown that taste activates an area of the anterior insula/frontal operculum, which is probably the ‘primary taste cortex, and part of the orbitofrontal cortex, which is probably the secondary taste cortex (Francis etal, 1999; O'Doherty, Rolls, etal.,2001). The orbitofrontal cor- tex taste area is distinct from areas activated by odors and by pleasant rouch (Francis et al., 1999). It has been shown that within individual participants separate areas of the orbitofrontal cortex are activated by sweet (pleasant) and by salt (unpleasant) tastes (O'Doherty, Rolls, etal, 2001) Francis et al. (1999) also found activation of the human amygdala by the taste of glucose. Extending this study, O'Doherty, Rolls, et al. (2001) showed that the human amygdala was as much activated by the affectively pleasant taste of glucose as by the aflectively negative taste of NaCl and thus provided evidence that the human amygdala is notEmotion Blicited by Primary Reinforcers and Following Stimulus-Reinforcement Association Learming 147 10mm Figure 9.4, Some of the pathways involved in emotion described in the text are shown on this lateral view of the brain of the macaque monkey. Connections from the primary waste and olisctory cortices to the orbitofrontal corsex and amyguals are show. Connections are also shown inthe ventral visual system from VI to V2, V4, the inferior temporal visual cortex, and £0 forth, with some connections reaching the amygdala and orbitofrontal cortex. Also shown are connections from the somatosensory cortical ateas 1,2, and 3 that reach the orbitofrontal cortex directly and via the insular cortex and that reach the amygdala via the insular corcex. Abbreviations as, arcuate sulcus; cl, calearine sulcus; lun, lunate suleus; ps, principal sulcus; to, Inferior occipital sulcus; ip, intraparietal sulcus (which has been opened to reveal some of the areas it contains); FST, visual motion processing area; LIP, lateral intraparietal area: MST, visual ‘motion processing area; MT. visual motion processing area (also called V5). STP, superior temporal plane, cortex, architectonic area including subareas (TEa and Tem) of high-order visual association cortex; TG, architectonic area inthe temporal pole; VI, V2 and Vi, visual areas 1, 2 and 4: VIP, ventral intraparital area, The numerals refer vo architectonic areas and have the (ollowing epproximate functional equivalence: 1,2,3, somatosensory cortex (posterior tothe central suleus); 4, motor cortex; 5, superior parietal lobule; 7, inferior parietal lobule, visual part; 7b, inferior parietal lobule, somatosensory part; 6, lateral premotor cortex; 8, frontal eye Field; 12, part of orbitofrontal cortex, 46, dorsolateral prefrontal corte. especially involved in processing aversive as compared with _5'-monophosphate (IMP) was added to MSG (0.05 M), the rewarding stimu. Another study has recently shown that ‘umami taste stimuli, an exemplar of which is monosodium glutamate (MSG) and which capture what is described asthe taste of protein, activate similar cortical regions of the zhuman taste system to those activated by a prototypical taste stimulus, glucose (De Araujo, Kringelbach, Rolls, & Hobden, 2003). A part of the rostral anterior cingulate cortex (ACO) ‘was also activated. When the nucleotide 0.005 M inosine BOLD (blood-oxygenation-level-dependent) signal in an anterior part of the orbitofrontal cortex showed supralinesr additivity, and this may reflect the subjective enhancement of umami taste that has been described when IMP isadded to MSG. In neuroimaging studies of taste, it ishelpful to use asa comparison in a" contrast” tasteless solution, containing the tain ionic components of saliva (e.g,, 25 mM KCl-+ 2.5 mM148 Emotion Elicitation NaCO,) to control for non-taste, somatosensory, inputs, 35 ‘well as the single tongue movement that is made when the tastant is introduced into the mouth, The use of this was introduced in the studies described earlier and enabled us to go onto show that water in the mouth activates cortical taste areas, with the primary cortex representing taste inde- pendently of motivational state and the orbitofrontal cortex representing the reward value of taste, in that its activation by water when thirsty was decreased after water was con- sumed to satiety (De Araujo, Kringelbach, Rolls, & MeGlone, 2003). The use of a single-event design enables subjective ratings made after every trial of the affective value of the stimulus to be correlated with the activation of diferent brain areas, and indeed inthis study (De Araujo, Kringelbach, Rolls, & MeGlone, 2003) and others (Kringelbach, O'Doherty, Rolls, & Andrews, 2003) it was found that the subjective pleasantness of the taste correlates with the activation of the orbitofrontal cortex produced by the stimulus. When brain areas such asthe orbitofrontal cortex are imaged with IMRT, Joss of signal may be a problem due to magnetic susceptibil ity introduced by the air-filled spaces in the frontal sinuses, and distortion may be introduced. We have developed 2 ‘number of techniques to minimize these problems, includ- {ng imaging in the coronal plane, minimizing voxel size in the plane ofthe imaging, using as high a gradient switching frequency as possible (960 Hz), using a short echo time of 25 ms (as described in the aforementioned studies), and using local shimming for the inferior frontal area using methods described by Wilson et al. (2002). The Pleasantness of the Taste of Food The modulation of the reward value of a sensory stimulus sich as the taste of food by motivational stae, for example, Junger, is one important way in which motivational behav- ior iscontrolled (Rolls, 1999a, 2005), The subjective corre- late of this modulation is that food tastes pleasant when one is hungry and tastes hedonically neutral when one has eaten to satiety. We have found that the modulation oftaste-evoked signals by mativation is not a property found in early stages ‘of the primate gustatory system, The responsiveness of taste neurons in the nucleus of the solitary tract (Yaxley, Rolls, Sienkiewicz & Scott, 1985) and in the primary taste cortex ((ronial opercular, Rolls, Scott, Sienkiewicz, & Yaxley, 1988; insular; Yaxley, Rolls, & Sienkiewicz, 1988) is not attenu- ated by feeding to satiety. In contrast, in the secondary taste cortex in the caudolateral part of the orbitofrontal cortex, it hhas been shown that the responses of the neurons tothe taste of the glucose decreased to zero as a monkey ate it to satiety, ring the course of which the behavior tured from avid acceptance to active rejection (Rolls, Sienkiewicz, & Yaxley, 1989). This modulation of responsiveness of the gustatory responses of the orbitofrontal cortex neurons by satiety could not have been due to peripheral adaptation in the gustatory system or to altered efficacy of gustatory stimulation after satiety was reached, because modulation of neuronal respon- siveness by satiety was not seen at the earlier stages of the gustatory system, including the nucleus of the solitary tract, the frontal opercular taste cortex, and the insulat taste cor- tex, The evidence that these neurons, in closely correlating with the decrease in preference for a food as it is eaten to satiety, encode the preference for a food has been supported by Schultz, Tremblay, and Hollerman (2000; see also Soelch etal, 2001) Sensory Specific Satiety In the secondary taste cortex, it was also found that the de- creases in the responsiveness of the neurons were relatively specific to the food with which the monkey had been fed to satiety. For example, in seven experiments in which amon- key was fed ghucose solution, neuronal responsiveness de- creased to the taste ofthe glucose but not to the taste of black currant juice (see the example in Figure 9.5). Conversely, in ‘wo experiments in which the monkey was fed to satiety with Fruit Juice, the responses of the neurons decreased to fruit juice but net to ghucose (Rolls, Sienkiewicz, & Yaxley, 1989) These findings provide evidence that the reduced accep. tance of food that occurs when food is eaten to satiety and the reduction in the pleasantness of its taste (Cabanae, 1971, Rolls, Rolls, Rowe, & Sweeney, 1981; Rolls, Rowe, Rolls, Kingston, & Megson, 1981; Rolls, Rowe, & Rolls, 1982; B J. Ralls, Ros, & Rowe, 1983; Rolls & Rolls, 1997) are not produced by a reduction in the responses of neurons in the ‘mucleus of the solitary tract or frontal opercular or insular gustatory cortices to gustatory stimuli, Indeed, after feeding to satiety, humans reported that the taste of the food on which they had been satiated tasted almost as intense as when they were hungry, though much less pleasant (E. T. Rolls, Rolls, & Rowe, 1983). This comparison is consistent with the possibility that activity in the frontal opercular and in sulartaste cortices, aswell as the nucleus ofthe solitary tract, does not reflect the pleasantness of the taste of a food but rather its sensoty qualities, independent of motivational state (Onthe other hand, the responsesofthe neuronsin the caudo- lateral orbitofrontal cortex taste area and in the lateral hypo. thalamus (Rolls, Murzi, Yaxley, Thorpe, & Simpson, 1986) are modulated by satiety, and itis presumably in areas such as these that neuronal activity may be telated to whether a food tastes pleasant and to whether the food should be eaten Gee also Critchley & Rolls, 1996b; Rolls, 1996, 1999a, 2005, 2000, 2000¢; Rolls & Scott, 2003). Itisan important principle that the identity ofa taste and its intensity are represented separately from its pleasaniness ‘Thus itis possible to represent what a taste is and to leam about it even when we are not hungry. ‘The Representation of the Pleasantness of Odor in the Brain thas been possible to investigate whether the olfactory tepre- sentation found in the primate orbitofrontal cortex is af-Emotion Eicited by Primary Reinforcers and Following Stimulus Reinforcement Association Learning 149) 2 FC ccte7 mo 80100180 cc170 Figure 9.5. The effect of feeding wo satiety with glucose solution on the responses of two neurons in the secondary taste cortex tothe tate of ghicose and of black currant juice (B)). The spontaneous firing rate i also indicated (SA), The behavioral rmeaguce of the acceptance or rejection ofthe solution on a scale from +2 to ~2 (see text) Is shown below the neuronal response data for each experiment. A 20% glucose solution was used to Feed to satiety. The monkey was fed 50 mlof the solution at cach stage ofthe experiment, as indicated along the abscissa until it was satiated, as shown by whether it accepted or rejected the solution. Pre » the fing rate ofthe neuron before the satiety experiment started. The values shown are the mean firing rate and its standard error. From Rolls, Sienkiewicz, & Yaxley, 1989. fected by hunger and thus whether the pleasantness of odor is represented in the orbitofrontal cortex. In satiety experi- ‘ments, Critchley and Rolls (1996b) showed that che te- sponses of some olfactory neurons to a food odor are decreased during feeding to satiety with a food (e.g, fruit Jjuice) containing that odor. In particular, seven of nine ol- factory neurons that were responsive to the odors of foods, such as black currant juice, were found to decrease their responses tothe odor of the satiating food. The dectease was typically at east partly specific to the odor of the food that hhad been eaten to satiety, potentially providing part of the basis for sensory-specific satiety Inhumans, in addition to activation ofthe pyriform (ol- factory) conex (Poellinger etal, 2001; Sobel etl, 1998, Zald & Pardo, 1997), there is strong and consistent activation of the orbitofrontal cortex by olfactory stimuli (Francis etal 1999; Zatorre, Jones-Gotman, Evans, &r Meyer, 1992). Inan investigation of where the pleasantness of olfactory stimuli ‘might be represented in humans, O'Doherty et al. (2000) showed that the activation of an area of the orbitofrontal cortex to a banana odor was decreased (relative to control vanilla odor) after bananas were eaten to satiety. Thus activ ity ina part of the human orbitofrontal cortex olfactory area is telated to sensory-specific satiety, and this is one brain region in which the pleasantness of odor is represented We have also measured brain activation by whole foods before and after the food is eaten to satiety (Kringelbach, ‘O'Doherty, Rolls, & Andzews, 2003). The aim is to show, using a food that has olfactory, taste, and texture compo- nents, the extent of the region that shows decreases when the food becomes less pleasant in oder to identify the dif- ferent brain areas in which the pleasantness ofthe odor, taste, and texture of food are represented, The foods eaten to sat ety were either chocolate milk or tomato juice. A decrease in activation by the food eaten to satiety relative to the other food was found in the orbitofrontal cortex (Kringelbach et a 2003) but not in the primary taste cortex. This study pro- vided evidence that the pleasanitness of the flavor of food is represented in the orbitofrontal cortex. ‘An important issue is whether there are separate regions of the human brain, discriminable with [MRI, that represent pleasant and unpleasant odors, To investigate this, we mea- sured the brain activations produced by three pleasant and three unpleasant odors. The pleasant odors chosen were linalyl acetate (floral, sweet), geranyl acetate (oral), and alpha-ionone (woody, slightly food-related). The unpleasant odors chosen were hexanoic acid, octanol, and isovalertc acid ‘We found that they activated dissociable pans of the human brain (Rolls, Kringelbach, & De Araujo, 2003) Pleasant, but not unpleasant, odors were found to activate 4 medial region ofthe rostral orbitofrontal cortex (see Figure 9.6). The method that was used to demonstrate this utilized conjunction analy- sis to reveal brain regions activated by all the pleasant odors, Further, there wasa correlation becween the subjective pleas antness ratings of the six odors given duringthe investigation and activation of a medial region of the rostral orbitofrontal conex (see Figute 9.7). Thisevent-telated approach—in which correlation can be investigated between the BOLD signal in different brain regions and subjective ratings of its affective value—is a powerful way to investigate the neural basis of the affective states elicited by different types of stimuli. in contrast, a correlation between the subjective unpleasantness150 Emotion Elicitation Figure 9.6, The represen: of pleasant and un ‘odors in the hurn Above: Group conjunction results for the ehiee pleasant ‘odors, Sagittal, horizontal, and coronal views are shown at the Tevels indiaced, al including the same acavation in the medial orbitofrontal cortex, OFC, y,2=0, 54, -12;2 5.23), Also shoven is activation forthe che pleasant od the anterior cingulate cortex, ACC Gy, 2 2,20,32;2= 5.44), These activations were significant at p-<.05 fully conected for multiple compar sons. Below: Group conjunction results forthe three unpleasant odors, The sagial view (lf) shows an activated region of the anterior cingolate carex (xy, 0,18, 36:2 = 442, p< 05, SVC). The coronal view (ight) shows an activated region ofthe lateral ‘orbitofrontal eorex (~36, 27, -8:2=423,p< 055V.C).All the activations were thresholded at p <.00003 10 show che extent fof the activations, From Rolls, Kringelbach, & De Araujo, 20032. See color insert. ratings of the six odors was found in regions of the left and more lateral orbitofrontal cortex. Activation was also found in the anterior cingulate cortex, with a middle part ofthe terior cingulate activated by both pleasant and unpleasant ‘odors (Figure 9.7), and in amore anterior part ofthe anterior cingulate cortex showing a correlation with the subjective santness ratings of the odors. These results provide evi- dence fora hedonic map of the sense of smell in brain regions, szuch asthe orbicofroncal cortex and cingulate cortex. ‘The Representation of Flavor: Convergence of Olfactory and Taste Inputs At some stage in taste processing, itis likely that taste cepre- sentations are brought together with inputs from different modalities, forexample, with olfactory inputs to form a rep- Ploasant Odors ayes Porno aes peed resentation of flavor (see Figure 9.3). We found (Rolls & Baylis, 1994) chat, of 12 single neurons inthe orbitofrontal cortex tate areas that responded to any of these modalities, many were unimodal (caste, 34%; olfactory, 13%; visual 2198) but were found in close proximity co each other. Some single neurons showed convergence, responding, for ex ample, to taste and visual inputs (13%). taste and olfactory inpuss (13%), and olfactory and visual inputs (5%). Some ofthese multimodal single neurons had corresponding s sitivities in the two modalities in that they responded best to sweet tastes (e.g. IM glucose) and responded more im a visual discrimination task to the visual stimulus that sign fied sweet Fruit juice than to the stinnulus that signified saline or responded to sweet taste and, in an olfactory discrimi nation task, to fruit odor. The dilferent types of neurons, (unimodal in different modalities and mul:imodal) were fre-Emotion Elicited by Primary Reinforcers and Following Stimulus-Reinforcement Association Learning 151 BOLD signal (4 change) 195 0 05 1 15 ve Pleasartness Ratings 04 Boa x02 a & wot a ee is 1 05 0 0 Subjective Pleasariness Ratings Figure 9.7. The representation of pleasant and unpleasant odors in the human brain. Random effets group analysis correlation analysis of the BOLD signal with the subjective pleasancness ratings. On the cop let is shown the region of the mediarasteal orbitofrontal (peak at [-2, 52, 10); z= 4.28) correlating positively with pleasantness ratings, a8 well asthe region of the anterior cingulate cortex inthe top middle. On the far top right of the figure is shown the relation between the subjective pleasantness ratings and the BOLD signal from this cluster (in the medial orbitofrontal cortex at Y= 52), together withthe regression line. The means and standard error of measurement across participants are shown, At the borior ofthe figure is shown the regions of left more lateral orbitofrontal cortex (peaks at {-20, 54, -I4]; 2 = 4.26 and [-16, 28, ~18!: 4.08) correlating negatively with pleasantness ratings. On the far bottom right ofthe figure is shown the relation berween the subjective pleasaniness ratings and the BOLD signal from the frst, cluster (in the lateral orbitofrontal cortex at ¥ = 54), together with the regression line, The means and standard err across participants are shown, The activations were thresholded at p < 0001 rom Rolls, Kringelbach, & De Araujo, 20032, See color insert for extent there isonly alow proportion of bimodal taste and olfactory neurons in the primary taste cortex (Rolls & Baylis, 1994). To investigate the rules underlying the affective represen quently found close to one another in tracks made into this region, consistent with the hypothesis that the multimodal representations are actually being formed from unimodal inputs to this region, thus appears to be in these orbitofrontal cortex areas thac flavor representations are built, where favor is taken to mean a representation that is evoked best by a combination cof gustatory and olfactory input. This orbitofrontal region does appear to be an important region for convergence, for tation of odors, many of which may not be primary reinforcers, investigations have been performed of whether the repr tation of odor depends on the taste (a primary reinforcer) ‘with which itis associated during learning, Critchley and, Rolls (1996a) showed that 35% of orbitofrontal cortex ol- factory neurons categorized odors based on their taste ass0-452 Emotion Elicitation tation in an olfactory-to-taste-discrimination task. Rolls, ritchley, Mason, and Wakeman (1996) found that 68% of rbitofrontal cortex odor-responsive neurons modified their :sponses in some way following changes inthe taste reward ssoctations of the odorants during olfactory-taste discrimi- ation learning and its reversal. In an olfactory discrimina- fon experiment, fa lick response to one odor, the St is rade, a drop of glucose taste reward is obtained; if incor- sctly,alick response is made to another odor, the S-,adrop faversive saline is obtained. At some time in the experiment, xe contingency between the odor and the taste is reversed, rnd when the “meaning” ofthe two odors alters, so does the havior. It is of interest to investigate in which parts of the factory system the neurons show reversal, for where they 6, it can be conchided that the neuronal response to the dor depends on the taste with which itis associated and ces not depend primarily on the physicochemical structure fhe odor. Full reversal of the neuronal responses was seen, 125% of the neurons analyzed. (In full reversal, the odor > which the neuron responded reversed when the taste with, ‘ich it was associated reversed.) Extinction of the difleren- al neuronal responses after task reversal was seen in 43% of 1ese neurons, (These neurons simply stopped discriminat- ag between the wo odors after the reversal.) These findings emonstrate directly a coding principle in primate olfaction ‘hereby the responses of some orbitofrontal cortex olfactory curonsare modified by and depend on the taste with which re odor is associated (Rolls, 2001, 20022, 2002). ‘twas of interest, however, that this modification was less omplete, and much slower, than the modifications found or orbitofrontal visual neurons during visval-taste reversal Rolls, Critchley, Mason, & Wakeman, 1996). This relative lexbilty of olfactory responses is consistent with the need br some stability in odor-taste associations to facilitate the mation and perception of flavors. Inaddition, some orbito- ontal cortex olfactory neurons did not code in relation to re taste with which the odor was associated (Critchley & alls, 1996a), so that there is also a taste-independent rep- 2sentation of odor in this region Tmaging studies in bumans are consistent with what has. cen discovered in maceques in that both taste and olfactory inmuli can activate parts ofthe orbitofrontal cortex and the joining most anterior part of the agranular insula but nei- rer activates earlier cortical areas such as the taste insula (De rayjo, Rolls, Kringelbach, McGlone, & Phillips, 2003). he Representation of the Affective Value f Somatosensory Stimuli in the Brain leasant touch, pain, an environmental temperature that romotes homeostasis (Cabanac, 1971), and probably the -xcure of food are primary reinforcers, Some single neurons, «the primate orbitofrontal cortex represent the viscosity of vod in the mouth (shown using a methyl cellulose series 1 the range of 110,000 centiPoise), others encode the par- ticulate quality of food in the mouth (Rolls, Verhagen, & Kadohisa, 2003), and others code for the texture of fat (Rolls, Critchley, Browning, Hemadi, & Lenard, 1999; Verhagen, Rolls, & Kadohisa, 2003). At least some of the neurons that ‘encode the texture of fat encode its reward value, in that feed- ing to satiery with fat decreases their responses to fatto zero Dbut not their responses to other rewards, such as glucose (Rolls et al., 1999). The temperature of oral stimuli ate also represented in the macaque orbitofrontal cortex (Kadohisa, Rolls, & Verhagen, 2004). In designing investigations of the representation of temperature, it is of interest to include stimuli that are within the region transduced by different {genetically identified receptor systems and to include cheini- cal stimuli that activate the different classes of temperature receptor (Patapoutian, Peier, Story, & Viswanath, 2003) Plessant touch (administered in an {MRI experiment using velvet on the hand) and pain (administered using a stylus applied to the hand) produced relatively more activa- tion of the human orbitofrontal cortex than a neutral (wood texture) stimulus, with the somatosensory cortex asa refer- ence, providing evidence that the affective aspects of touch are represented in the orbitofrontal cortex (Rolls, Doherty, ct al., 2003). The investigation of the brain pathways that transmit information from pain receptors is well developed (Craig, 2003), and, centrally, cingulate activation is invari- ably observed, but there isno full understanding ofthe brain regions that are crucial for the affective aspects of pain. Stud- ses of patients wit brain lesions suggest that the orbitofrontal cortex is one crucial area, as patients with lesions to this re- gion may report that they can discriminate a stimulus asbeing. painful, yet they do not have a strong aflective response to it Rolls, 1999a, 2005) Visual and Auditory Stimuli That Are Primary Reinforcers Although the majority of visual and auditory stimuh are not primary reinforcers, some, such as vocal and face expression, warning calls, and even correlates of attractiveness such as face symmetry, may be (see Table 9.1; Rolls, 1999a, 2005) “Tounderstand the neural bass of emotion, itis therefore of ‘interest to investigate where in the brain the affective repre- sentations of these stimuli may be. It has been shown that patients with lesions of the orbitofrontal cortex can be im- paired at identifying the face and/or voice expression in the absence of lace or voice identity discrimination impairments (Homak, Rolls, & Wade, 1996). This can occur even with lesions restricted to a medial and posterior part of the orbito- frontal cortex (Hornak et al., 2003) and occurs for a range of different face and voice expressions (such as happiness, sadness, anger, fear, etc.; Homnak etal, 2003; Rolls, 1999b). Face expression is encoded by neurons in the cortex in the anterior part of the superior temporal sulcus of macaques (Hasselmo, Rolls, & Baylis, 1989), but whether these neu- tons are perceptual analyzers that send information to theEm orbitofrontal cortex (Rolls, Critchley, Browning, & Inoue, 2006) and amygdala (Leonard, Rolls, Wilson, & Baylis, 1985), where the affective value is represented, or whether the af- fective value is represented in the cortex in the superior tem- poral sulcus is not yet known. The attractiveness of a face has a brain correlate in the BOLD response in orbitofrontal cortex (O'Doherty et al,,2003). Phillips and colleagues (1998) have produced evidence that activation of a part of the hu- rman insula occurs to the facial expression of disgust, although ic isnot clear whether this is the taste insula, as taste stimuli hhave not been used in the same series of experiments. It has analogously been suggested that amygdala activation occurs to fearful facial expressions and that lesions of the amygdala impair the identification of facial expressions of fear (Calder, Lawrence, & Young, 2001), although caution in ascribing particular emotions to particular brain regions is appropriate (Rolls 1999a, 2005, 1999b; Winston, O'Doherty, & Dolan, 2003), Learning and Reversing Associations to Primary Reinforcers Given the importance of rewards and punishers in emotion, the learning of associations from previously neutral stimult to primary reinforcers is the type of learning that is funda- ‘mental in emotion. Itis also very important that such asso- cations can be rapidly and flexibly reversed. For example, {n social interactions, even a short change in a facial expres- sion may indicate that a change of behavior toward the per- son is appropriate, The processes that underlie this type of, associative leaming, which is stimulus-stimulus (in that the primary reinforcer isa stimulus), are therefore of consider- able interest and clinical relevance. Given the anatomical connections shown in Figure 9.3, there would not seem to be a basis in high-order visual cor- tical areas such asthe inferior temporal visual cortex for a5- sociations between visual stimuli and primary reinforcers to be learned, as taste and somatosensory stimuli do not reach these cortical areas. This has been directly demonstrated: ‘When the reward that a monkey normally obtains from lick- {nga viswal stimulusis alered in a visual discrimination task so that the monkey obtains only aversive salt, no alteration takes place in the responses of inferior temporal cortex neu- rons, which continue to fire tothe physical simutus to which they respond independently of itsaffective significance (Rolls, Aggelopoulos, & Zheng, 2003; Rolls, Judge, & Sanghera, 1977) In contrast, some neurons in the macaque orbitofrontal cortex lear in as litle as one trial to respond toa visual stimu- lus associated with a taste reward and can reverse this re- sponse when the reinforcement contingencies are reversed Rolls, Critchley, Mason, & Wakeman, 1996; Thorpe, Rolls, & Maddison, 1983). These neurons reflect the reward (or affective) value of the visual stimulus in that they stop re- Elicited by Primary Reinforcers and Following Stimulus-Reinforcement Association Leaming 153 sponding to the visual stimulus gradually asthe monkey is fed to satiety (Critchley & Rolls, 1996b). Also consistent with this, lesions of the macaque orbitofrontal cortex im- pair stimulus-reward reversal leaming and extinction (But ter, 1969; Iversen & Mishkin, 1970), and deficits in this type of learning are also found in humans after orbitofrontal cor- tex lesions (Rolls, Horak, Wade, & McGrath, 1994; Hornak etal., 2004). In humans, activation of the orbitofrontal cor- tex can reflect quite abstract rewards, though in the end as- sociated with primary reinforcers, such as monetary reward. (medially) and loss (laterally; O'Doherty, Kringelbach, Rolls, Homak, & Andrews, 2001). A probabilistic presentation of the magnitude of reward was used in this investigation and in a complementary investigation with the same task in pa- tients with lesions of the orbitofrontal cortex (Homak etal, 2004) so that the magnitude of the brain activation could be correlated in a single-event (MRI design with the amount of, monetary reward or loss on each trial and to minimize the use of verbal strategies and encourage the use of implicit associative emotional learning processes. The special role of the orbitofrontal cortex in rapid reversal of stimulus reinforcement associations is expressed also in the presence in the orbitofrontal cortex of neurons that respond when an expected reward is not obtained (Thorpe et al., 1983), and these can be called reward error neurons or frustrative non- reward neurons. They may have a special roe in reversal (Deco & Rolls, 2005; Rolls, 2004a). Consistent with this evidence. in macaques, Kringelbach and Rolls (2003) found activation of alateral part of the human orbitofrontal cortex specifically, when a changed facial expression was used asa cue to signal reversal in a visual discrimination task (see also Kringelbach & Rolls, 2004). ‘The amygdala receives information about primary rein- forcers (Guch as taste and touch) and also about visual and auditory stimuli from higher cortical areas (such asthe infe- ‘or temporal cortex) that can be associated by learning with, primary reinforcers (Figure 9.3). Bilateral removal of the amygdala in monkeys produces tameness,alack of emotional, responsiveness; excessive examination of objects, often with the mouth; and eating of previously rejected items, such as, ‘meat (the Kluver-Bucy syndrome). In analyses of the bases, of these behavioral changes, it has been observed that there are deficits in learning to associate stimuli with primary re- inforcement, including both punishments and rewards (see Rolls, 2000c). The association-learning deficit is present when the associations must be leamed from a previously neutral stimulus (.g., the sight of an object) to a primary reinfore- ing stimulus (such as the taste of food), Further evidence that links the amygdala to reinforcement mechanisms includes the facts that monkeys will work in order to obtain electri- cal stimulation of the amygdala, that single neurons in the amygdala are activated by brain-stimulation reward of a number of different sites, and that some amygdala neurons respond mainly to rewarding stimuli and others to punish- ing stimuli (see Rolls, 1999, 2005, 2000). The association154 Emotion Elicitation learning in the amygrlala may be implemented by assoctia- Lively modifiable synapses from visual and auditory neurons conto neurons that receive inputs from taste, olfactory, ot somatosensory primary reinforcers (LeDoux, 1996, 2000) ‘Consistent with this, Davis (2000) has found that at least one type of associative learning in the amygdala can be blocked by local application to the amygdala of an N-methyl-D-2s partate (NMDA) receptor blocker, which blocks long-term potentiation (LTP), a model of the synaptic changes that underlie learning (see Rolls & Treves, 1998). Davis (2000) used a lear-potentiated startle response as a measure of the conditioned fear. Consistent with the hypothesis that the learned incentive (conditioned reinforcing) effects of previ- ously neutral stimuli paired with rewards are mediated by the amygdala acting through the ventral striaturn, amphet- amine injections into the ventral striatum enhanced the effects of a conditioned reinforcing stirmulus only if the amygdala was intact (see also Everitt, Cardinal, Hall, Parkin- son, & Robbins, 2000). In an imaging study in humans, Ploghaus et al. 2000) showed that fear conditioning pro- duced activation of the amygdala. A difference between the amygdala and the orbitofrontal cortex in stimulus-reinforee- ‘ment association learning may be that the rapid reversal of the associations may require the orbitofrontal cortex (Rolls, 1999a, 2005, 20026), which may implement this rapid re- versal at the neuronal level more efficiently (Rolls, 19992, 2005; Rolls & Deco, 2002), perhaps because it has rule-based attractor networks that can be rapidly switched by nonreward (Deco & Rolls, 2005). Acknowledgments | have worked on some ofthe experiments described here with G.C. Baylis, LL. Baylis, M. J. Burton, H.C. Crichley, M.E. Hasselmo, J. Homa, M:Keingelbach, C.M. Leonard, F. Mora, 4J.0' Doherty. D. Perret, MK. Sanghera, TR Scott, S.J ‘Thorpe, and F. A. W. Wilson, and thei collaboration and helpful discussions with oF communications from M. Davies (Cocpus Christi College, Oxford and M.S, Dawkins are sincerely acknowledged. Some of the research deseribed was supported by the Medical Research Council Notes 1. Rewards and punishers are generally external, that i, extcroceptive, stimuli, such a the sight, smell, and taste of food when one is hungry, Interoceptive stimall, even when produced by rewards and punishers after ingesting foods and including Algestive processes and the reduction of the drive (hunger) state, are not good reinforcers, Some of the evidence for this is that the taste of fod isan excellent reinforcer, but placing food into the stomach isnot. This imporant distinction is described. by Rolls (1999, 2005). 2. Part of the basis for this is that when memories are recalled, top-down conneetions into the higher perceptual and cognitive cortical areas lead to reinstatement of activity in those areas (Rolls & Deco, 2002; Treves & Rolls, 1994), which in turn can produce emotional states via onward connections to the orbitofrontal cortex and amygdala (Rolls, 1999, 2003) References Alexander, RD. (1975). The search fora general theory of behaviour. Behavioral Sciences, 20, 77-100. Alexander, R.D. (1979). Darwinism and human afar. Seattle University of Washington Press. Baylis, L L., & Rolls, ET. (1991). Responses of neurons in the primate taste cortex to glutamate. Physiology and Behaviour 49, 973-979. Booth, D. A.(1983). Food:conditioned eatin preferences and versions with interooeptive elements: Learned appetites and satieties, Annals ofthe New York Academy of Sciences, 443, 22-37 Buck, L (2000) Smell and taste: The chemical senses In E.R Kandel, J. H. Schwartz 6 H, Jessel (Eds), Principles of neural science (4th ed, pp. 625-647). New York: MeGraw-Hil Butter, C. M. (1969). Perseveration in extinction end in discrimination reversal tasks following selective prefrontal ablations in Macaca matatta. Physiology and Behavior, 4, 163-171 Cabansc, M. (1971). Biological role of pleasure. Science, 173, 1103-1107. Calder, A.J, Laweence, A. D., & Young, A. W. (2000). Neuropsychology of fear and loathing, Nature Reviews. Neuroscience, 2, 352-363, Chaudhari, N., Landin, AIM., & Roper, $. D. (2000). A metabotropic glutamate recepror variant functions asa taste receptor. Nature Neuroscience, 3, 113-119. Craig, A D. (2003), Pain mechanisms: labelled fines versus convergence in central processing. Annual Review of Neuroscience, 26, 130. tchley, H. D., & Rolls, E. T. (1996a). Olfactory neuronal responses in the primate orbitofrontal cortex: Analysis in an ‘olfactory discrimination task, Journal of Neurophysiology, 75, 1659-1672 Critchley, H. D., & Rolls, E. T. (19966). Hunger and satiety modify the responses of ollactory and visual neurons in the primate orbitofrontal corcex. Joural of Neurophysiology, 75, 1673-1686 Critchley, H. D. & Roll, E:T. (1996c). Responses of primate taste cortex neurons to the astringent tastant tannic acid. Chemical Senses, 21, 135-148. Darwin, C. (1998). The expression ofthe emotions in man and ‘animals Grd ed, P. Ekman, Ed). Chicago: Univesity of Chicago Press. (Original work published 1872) Davis, M. (2000). The role of the amygdala in conditioned and unconditioned fear and anxiety. In J.P. Aggleton (Ed), The amygdala: A functional analysis 2nd e@., pp. 213-228) Oxford, UK: Osferd University Press De Araujo, LE. T., Kringelbach, M. L, Rolls, E. T.,& Hobden, . (2003), Representation of umam taste in the human brain, Journal of Neurophysiology, 90, 313-319. De Araujo IE. T., Kringelbach, M.L., Rolls, E. 7, & McGilone, F. (2003). Human conical responses to water in the mouth,Emotion Blicited by Primary Reinforcers and Following Stimulus-Reinforcement Association Leaming 155 and the eects of thirst. journal of Neurophysiology, 90, 1865-1876. De Araujo... T, Rolls, E.T., Kringelbach, M.L., MeGlone, F., ‘& Philips, N. (2003). Tase-olfectory convergence, and the representation ofthe pleasantness ef Mavour, in the heaman brain. European journal of Newroscience, 18, 2059-2068, Deco, G., & Rolls, ET. 2005), Synapric and spiking dynamics undetlying ceward reversal inthe orbitofrontal cortex Corebval Cortex, 15. 15-30. Dieleinsom, A. (1980). Contemporary animal learning theory Cambridge, UK: Cambridge University Press Ekman, P, (1982). Emotion inthe human face (2nd ed), Cam- bridge, UK: Cambridge University Press Ekman, P. (1993). Facial expression and emotion, American Paychoogist, 48, 384-392 Everit,B J. Cardinel, R.N., Hall). Parkinson, J. A. & Robbins, T. W. (2000), Differential involvement of amygéala subsystems in appettive conditioning and drug addiction, In}. . Aggleton (Ed.), The amygdala: A func- tiona analysis (2nd ed., pp. 353-390). Oxford, UK: Oxford ‘University Press Francis, 5. Rolls E.T., Bowtell, R, McGlone, F., Doherty... Browning, A’, etal (1999). The representation of pleasant touch inthe brain and its relationship with taste and clfactory areas. Neuroreport, 10, 453-459. Goldman-Rakic, B.S. (1996). The prefrontal landscape: Implications of functional architecture for understanding, ‘human mentation and the central executive, Philosophical Transactions ofthe Royal Society of London: Series B,Bolgicl Sciences, 351, 1445-1453, Gray, J. A. (1975). Elements ofa swo-processCheory of learning, London: Academic Press Gay, A. (1987). The psychology of ear and stress (2nd ed.) Cambridge, UK: Cambridge University Pres. Hasselmo, M. E, Rolls, ET. & Baylis, G. C, (1989), The roe of expression and identity in the face-selective responses of neurons inthe temporal visual cortex ofthe monkey. Behavioural Brain Research, 32, 203-218. “Bramham,}, Rolls, E.T., Mertis, RG., O'Doherty, ‘J. Bullock, 2. Ret al. (2003). Changes in emotion after cireumseribed surgical lesions ofthe orbitofrontal and cingulate cortices. Brain, 126, 1691-1712 Horak, J, Doheny, , Bramham., J, Rolls, F.T., Moris, RG, Bullock, PR, eval. (2004), Reward-related reversal learning after surgical excisions n orbitohiontal and dorsolateral prefrontal cortex in humans. Journal of Cognitive Neuroscience, 16, 463-478. Horak, J, Rolls, E.T., & Wade, D. (1996), Face and voice ‘expression identification in patients with emotional and behavioural changes following ventral frontal lobe damage. Neuropsychologa, 34, 247-261 Iversen, $., & Mishkin, M, (1970). Perseveratve interference ‘in monkey following selective lesions of the inferior prefron- tal convenity. Experimental Brain Research, 11, 376-386, Kadohisa, M., Rolls E.T., & Verhagen, J. V. (2008). Orbito- {ronal cortex neuronal representation of temperature and capsaicin in the mouth. Newrosclence, 127, 207-221. Xrebs, JR. & Kacenik, A (1991), Decision making, In). R Krebs & N. B. Davies (Eds), Behavioural ecology 3rd ed, ppp. 105-136), Oxford, UK: Blackall XKringelbach, M. L, ODoherty J, Roll, E.T.. & Andrews, C (2003), Activation ofthe human orbitofrontal cortex co a Liquid food stimulus is correlated with its subjective pleasantness. Cerebral Cortex, 13, 1064-1071 Keingelbach, M. L, & Rolls, E.T. (2003), Neural correlates of rapid reversal learning ina simple model of human social interaction. Newrofmage, 20, 1371-1383, Kringelbach, M. L, & Rolls, ET. (2004) The functional neuroanatomy of the human orbitofrontal cortex: Evidence from neuroimaging and neuropsychology. Progress i Neurobiology, 72, 341-372. LeDoux, JE (1996). The emotional brain. New York: Simon & Schuster, LeDoux, J. E, (2000). The amygdala and emotion: A view ‘hrough fear In J.P. Aggleton (Bd), The amygdala: A Junctional analysis (2nd ed, pp. 289-310). Oxlord, UK (Oxford University Press. Leonard, C. M., Rolls, E.., Wilson, FA. W., 6 Baylis,G. (1985). Neurons inthe araygdala ofthe monkey with responses selective for faces. Behavioural Brain Research, 25, 159-176. Lieberman, D. A. (2000). Learning (3rd ed). Belmont, CA: Wadsworth, “Mackintosh, N. J. (1983). Conditioning and associative learning Oxford, UK: Oxford University Pres. Millenson, J. R. (1967). Principles of behavioral analysis. New Yorks Macmillan esse, R.M., 6 Lloyd, A. T. (1992). The evolution of psycho- dynamic mechanisms, In J. H. Barlow, L, Cosmides, & 4. Tooby (Eds.), The adapied mind (pp. 601-624). New York: Oxford University Press [Norgren, R.(1984). Cental neural mechanisms of taste. In I Darien-Smith (Series Ed), J. Brookhart, &V. B Mounteastle (Vol. Eds.) Handbook of physiology: Voli. The nervous system: Sensory processes (pp. 1087-1128). Wash: ington, DC: American Physiological Society ODohery, J, Kringelbech, M.L.,Rells, ET, Homak, J. & ‘Andrews, C. (2001). Abstract reward and punishment representations in the human orbitofrontal corex. Nature Neuroscience, 4,95-102. ODohery, J. Rolls, E.T,, Francis, S., Bowell, R., 6 MGlone. F (2001), The representation of pleasant and aversive taste in the human brain. Journal of Neurophysiology, 85, 1315-1321 ODoheny, J, Rolls, E.T., Francis, S., Bowell, R, MeGlone, F Kobal, Get al (2000) Sensary-specilic satiety related olfacory activation ofthe human orbitofrontal cortex. ‘Newrorepor, 11, 893-897. Doherty, J, Winston, J., Critchley, H, Perret D., Bur D. M.,& Dolan. J. (2003), Beauty ina smile: The role of medial orbitofrontal cortex in facial attractiveness Neuropsychologia, 41, 147-155, Oatley, K, & Jenkins, J. M, (1996) Understanding emotions Oxford: Blackwell Paspoutian, A. Peier, A.M. Story, G. M, &¢Viswanath, V. (2003) ‘ThermoT RP channels and beyond! Mechanisms oftempera- ture sensation. Nature Reviews. Neuroscience, 4, 529-538. Petrides, M, (1996). Specialized systems for the processing of rmuemonic information within the primate frontal cortex. Philosophical Transactions ofthe Royal Society of London: Series B Biological Sciences, 351, 1455-1462.156 Emotion Elicitation Phillips, M.L., Young, A. W., Seat, $.K., Calder, A.J. Andrew, C., Giampetto,V., etal. (1998), Proceedings of the Royal Society of London: Series B. Biological Sciences, 265, 1809 1817. Ploghaus, A. Tracey, , Clare, S., Gat J.S.,Rawlins,J.N,,& Patthews, P.M. (2000). Learning about pain: The neural substrate ofthe prediction error for aversive events Proceedings ofthe National Academy of Sciences of the USA, 97, 5281-9286. Pocllinger, A, Thomas, R, Lio, P., Lee, A., Makris, N., Rosen, B.R etal. (2001), Activation and habituation in olactin: An IMRI study. Newrolmage, 13, 547-560 Pritchard, T.C., Hamilton, R.B., Morse, J.R., & Norgren, R. (1986). Projections of thalamic gustatory and lingual areas inthe monkey, Macaca fascicular Journal of Comparative Newology, 244, 213-228. Rolls, BJ, & Rolls, ET. (1973), kffects of tesions in the basolateral amygdala on fluid tntake inthe rt. Journal of Comparative and Physiological Psychology, 83, 240-247. Rolls, BJ. Rolls, ET, & Rowe, E.A, (1983), Body fat control and obesity. Behavioral and Brain Sciences, 4, 744-745. Rolls, B.)., Rolls, E.T., Rowe, E. A., & Sweeney, K. (1981). Sensory specific satiety in man, Physciogy and Behavior, 27, 174 Rolls, B.J., Rowe, E.A., 6 Rolls, ET. (1982). How sensory properties of foods affect human feeding behavior. Physial- gy and Behavior, 29, 409-417. Rolls, B.J., Rowe, EA. Rolls, E.., Kingston. B, & Megson, A (1981), Variety in a meal enhances food intake in man, Physiology ard Behavior, 26, 215-221 Rolls, ET, (1986a), Neural systems involved in emotion tn primates. to R, Plutchik & H. Kellerman (Eds), Emotion: Theory, research and experience: Vol. 3. Biological foundations of emation (pp. 125-143), New York: Acadernic Pres. Rolls, ET. (1986b). A theory of emotion, and its application to understanding the neural basis of emotion In Y. Oormura (Ea), Emotion: Neural and chemical control (pp. 325-344), ‘Tokyo: Japan Scientific Societies Press. Rolls, E.T. (1990). A theory of emotion and its application to understanding the neural bass of emotion. Cognition and Eatin, 4, 161-190 Rolls, ET. (1996). The orbitofrontal cortex. Philosophical ‘Transactions of the Royal Society of London: Series B. Bicogical Sciences, 351, 1433-1444, Rolls, ET, (1907). Taste and ollactory processing in the brain and its relation tothe control of eating, Critical Reviews in Neurobiology, 1, 263-287, Ralls, ET, (19992). The brain and emotion. Oxlord, UK: Oxlord. University Press Rolls, ET. (19996). The functions ofthe orbitofrontal cortex. Newrocase, 5, 301-312. Rolls, E, T. (20008). Préis of the brain end emotion. Behavioral ‘and Brain Sciences, 23, 177-234. Rolls, ET. (20008). The orbitofrontal cortex and reward. Cerebral Cortex, 10, 284-294 Rolls, ET, (2000c). Neurophysiology and functions of the ‘primate amygdala, and the neural basis of emotion. In J.P Aggleton (Ed), The amygdala: A functional analysis (2nd ed pp: 447-478). Oxford, UX: Oxlord University Press. Rolls, £, T (2001), The rules of formation of the olfactory representations found in the orbitofrontal cortex olfactory aceas in primates, Chemical Senses, 26, 595-60 Rolls, E.T. (2002a). The cortical representation of taste snd smell In G. Rouby,B. Schaal, D. Dubois, R. Gervais, & A. Holley (Eds), Olfaction (pp. 367-388). New York: Cam- bridge University Press. Rolls, E, T. (2002b). The functions ofthe orbitofrontal coxtex. nD. T. Stuss & R. T, Knight (Eds), Principles of froma obe ‘futon (pp. 354-375). New York: Oxford University Press Rolis, ET. (2004a) The functions ofthe orbitofrontal cortex Brain and Cognition, 55, 11-29. Rolls, E. T2004). A higher order syntactic thought (HOST) theory of consciousness. In RJ. Gennaro (Ed), Higher order theories of consciousness (pp. 137-172). Amsterdam: John Benjamins Rolls, ET. (2005), Emotion Explained. Oxford, UK: Oxford University Press. Rolls, E.T., Aggelopoulos, N. C., &¢ Zheng, F. (2003). The receptive fields of inferior temporal cottex neurons in natal seenes, Journal of Neuroscience, 23, 339-348 Rolls, E.T., & Baylis, LL, (1994), Gustatory, olfactory and visual convergence within the primate orbitofrontal cortex. Journal of Neuroscience, 14, 5437-5452 Rolis, ET, Crtehley, H. D., Browning, A, & Hemadi, | (1998). The neurophysiology of taste and olfaction in pritnates, and umami flavor. Annals of the Nev York Academy of Sciences, 855, 426-437. Rolls, ET. Critchley, H. D., Browning, A. S., Hemadi, A. & Lenard, L, (1999). Responses to the sensory properties of fat of neurons in the primate orbitofrontal cortex. journal of Newrosclence, 19, 1532-1540 Rolls ET, Citchley, H.D., Browning, A. S., Inoue, K (2006). Face-selecrve and auditory neurons in the primate ‘otbitofvontal cortex. Experimental Brain Research, 170, 74-87. Rolls, ET, Critchley, H., Mason, R., & Wakeman, EA (2996). Orbitofrontal cortex neurons: Role in olftory and visual association learning, Journal of Neurophysiology, 75, 1970-1981 Rolls, ET. Critchley, H.. Wakeman, E, A, & Mason, R (1996). Responses of neurons inthe primate taste cortex to the glutamate ion and to inosine 5-monophosphate. Physiology and Bekaviar, 99, 991-1000. Rolls, E.T., & Deco, G. (2002). Computational neuroscience of vision, Oxford, UK: Oxford University Pres Rolls, E.T., Horak, J, Wade, D., 6 McGrath, J. (1994) ‘Emotion-telated Ieaming n patients with social and ‘emotional changes associated with frontal lobe damage. Journal of Neurology, Neurosurgery, and Psychiatry, 57, 1318- 1524 Rolls, E.T. Judge, S.J. 6 Sanghera, M. (1977). Activity of ‘neurones i the inferotemporel cortex of the alert monkey. Brain Research, 130, 229-238. Rolls, E.T., Kringelbech, ML, & De Araujo, LE, T. (2003). Different representations of pleasant and unpleasant odors fn the human brain, Furepean Journal of Neuroscience, 18, 695-703, Rolls, £.T., Mazi, E, Yaxley,S., Thorpe, 5.) & Simpson, 5. (1986) Seasory-specific satiety: Food specific reduction ta ‘esponsiveness of ventral forebrain neurons after feeding in the monkey. Brain Researc, 368, 79-86Emotion Ei Rolls, E.T., ODeherty.J., Kringelbach, ML, Francs, $., Bowell, R, & MeGilone, F (2003). Representations of pleasant and painful touch in the human orbitofrontal and cingulate cortices. Cerebral Cortex, 13, 308-317 Rolls, E.T., Rolls, BJ. & Rowe. E. A, (1983). Sensory-specific and moxivation-Specific satiety forthe sight and taste of food and water in man. Physilogy and Behavior, 0, 185-192. Rolls, ET, 6 Rolls, J. H, (1997), Ollactory sensory-spectfic satiety in humans. Physiology and Behavior, 61, 461-473. Rolls ET, & Seow, T. R, (2003). Central taste anatomy and neurophysiology. in R. L, Doty (Ed.), Handbook of olfaction and gustaion (2nd ed., pp. 679-705). New York: Dekker. Rolls, ET, Scot, T. R., Sienkiewiez, Z.), & Yaxley, 5. (1988) ‘The responsiveness of neurones in the frontal operculse gustatory cortex of the macaque monkey is independent of IRunger. Journal of Physiology, 397, 1-12. Rolls, ET, Sienkiewicz, Z.]., 6 Yaxley, 5. (1989). Hunger ‘modilats the responses to gustatory stimu of single neurons inthe orbitofrontal cortex. European Journal of Neuroscience, 1, 53-60, Rolls, ET, & Stringer. S. M. (2001). A model ofthe interaction between mood and memory. Network: Computation in Newral Systems, 12, 89-109. Rolls, ET, & Treves, A. (1998). Neural networks and brain function. Oxford, UK: Oxford University Press Rolls, E.T., Verhagen, J. V, & Kadohisa, M. (2003). Represen- tations ofthe texture of food in the primate orbitofrontal cortex: Neurons responding to viscosity, grttness, and capsaicin. journal of Neurophysiology, 90. 3711-3724 Rolls, E.T., Yaxley, 5, & Sienkiewicz, Z. (1990). Gusatory responses of single neurons in te orbitofrontal comex of the rmacaque monkey. Jounal of Neurophysiology, 64, 1055-1066. Schatz, W., Tremblay, Lf Hollerman,].R. (2000), Reward processing in primate orbitofrontal cortex and basal ganglia. Cerebrai Corte, 10, 272-284. Shallice .,& Burgess, P. (1996). The domain of supervisory processes and temporal organization of behaviour. Philo- sophical Transactions ofthe Royal Society of Landon: Series B Biological Sciences, 351, 1405-1411. Sobel, N, Prabhakaran, V., Desmond, }.E., Glover, GH., Goode, RL, Sullivan, FV. et al. (1998). Saifing and smelling’ separate subsystems inthe human ollatory cortex. Nature, 392, 282-286. Soelch, M., Leenders, K.L., Chevalley, A.F., Missimer,.. unig, G., Magyar, S., al (2001). Reward mechanisms in -d by Primary Reinforcers and Following Stimulus-Reinforcement Association Learning 157 che brain and ther role in dependence: Evidence from neurophysiological and neuroimaging studies. rain Research Reviews, 36, 139-149, Thorpe, S.J, Rolls, E.T., & Maddison, S. (1983). The orbitofrontal coreex: Neuronal activity in the behaving, monkey. Experimental Brain Research, 49, 93-115. Tinbergen, N. (1951). Te study of lstnct. Oxford, UK Clarendon Pres. Treves, A, & Rolls, E.T. (1994). A computational analysis of the role of the hippocampus in memory. Hippocampus, 4 374-391 Trivers, RL. (1976), Foreword. la R Dawkins, The sesh gene Oxford, UK: Oxford University Press Trivers, R.L-(1985). Social evolution, Menlo Park, CA: Benjamin Cummings Verhagen, J.V., Rolls, E.T.,6¢ Kadohisa, M. (2003), Neurons in the primate orbitofontal cortex respond to fat texture independently of viscosity. Journal of Neurophysiology, 90, 1514-1525, Weiskeantz, L. (1968). Emotion. In L Weiskrantz Ed), ‘Analysis of behavioral change (pp. 50-90). New York: Harper & Row. Wilson, J.L, Jenkinson, M., de Araujo, 1, Kringelbach, ML, Rolls, ET, Jezard,P. (2002), Fas, flly automated gicbal and local magnet field optimisation for (MRI ofthe human brain, Neurolmage, 17, 967-976 Winston}. 5, ©Dohexy, J, & Dela, B J (2003). Common, and distinct neural responses during dieet and incidental processing of mutple facial emotions. Neurolmage, 20, 84 97, Yaxley, 5, Rolls, E.T.,& Sienkiewicz Z. J (1988). The responsiveness of neurones in the msular gustatory cortex of the macaque money is independent of hunger. Physalagy and Behavior, 42, 223-229. Yarley, S, Roll, ET, Sienkiewicz, .}, 6 Scott, T. (1985) Satlety does not alfet gustatory acy inthe nucleus of the solitary tract ofthe alert monkey. Brain Research 347, 85-93, Zald, D. HL, 6 Pardo, JV, (1997). Emotion, ollacton, and the hhuman amygdala: Amygdala activation during aversive olfactory stimulation, Proceedings o the National Academy of Sciences ofthe USA, 94, 4119-4124 Zatore, RJ, Jones-Gotman, M., Evans, A.C. & Meyer, (1992), Functional localization and lateralisation of the human olfactory cortex. Natur, 360, 339-340.