Physiology & Behavior 103 (2011) 51–58

Contents lists available at ScienceDirect

Physiology & Behavior

j o u r n a l h o m e p a g e : w w w. e l s ev i e r. c o m / l o c a t e / p h b

Learning and the persistence of appetite: Extinction and the motivation to

eat and overeat☆
Mark E. Bouton ⁎
Department of Psychology, University of Vermont, 2 Colchester Ave., Burlington, VT 05405-0134, United States

a r t i c l e i n f o a b s t r a c t

Article history: The modern world is saturated with highly palatable and highly available food, providing many opportunities
Received 15 September 2010 to associate food with environmental cues and actions (through Pavlovian and operant or instrumental
Received in revised form 31 October 2010 learning, respectively). Basic learning processes can often increase the tendency to approach and consume
Accepted 19 November 2010
food, whereas extinction, in which Pavlovian and operant behaviors decline when the reinforcer is withheld,
weakens but does not erase those tendencies. Contemporary research suggests that extinction involves an
Keywords:
Eating
inhibitory form of new learning that appears fragile because it is highly dependent on the context for
Overeating expression. These ideas are supported by the phenomena of renewal, spontaneous recovery, resurgence,
Pavlovian and operant conditioning reinstatement, and rapid reacquisition in appetitive learning, which together may help explain why
Extinction overeating may be difficult to suppress permanently, and why appetitive behavior may seem so persistent.
© 2010 Published by Elsevier Inc.

Human food intake is influenced by a large and very complex set of
factors (e.g., [1]). However, there is little doubt that learning and
conditioning processes are important among them. As Stephen
Woods and others have emphasized, conditioning can play a very
important role in determining the onset or timing of meals,
preferences and aversions for different foods, and other aspects of
eating and appetite (e.g., [2–4]; see also [5–7]). The purpose of this
article is to selectively review some basic research on learning with an
emphasis on extinction, a major inhibitory process in conditioning,
and discuss some of the possible implications for understanding the
persistence of appetite, difficulties in losing weight and maintaining
weight loss, and the contemporary obesity epidemic.
The modern world is saturated with food and with opportunities
to consume it. As noted by Kessler [8], we are constantly exposed to
highly palatable, high-calorie foods that have been engineered with
multiple layers of sugar, salt, and fat. From the perspective of learning
theory, the high abundance of such tasty foods means that the world
is full of very effective conditioning trials, i.e., opportunities to
associate the food with cues in the environment (through Pavlovian
conditioning) or with voluntary actions that lead to it (through
operant or instrumental learning). Table 1 illustrates some of the
many cues and operant behaviors that are available for association
with food. In the Pavlovian case, it seems hard to avoid exposure to
colorful food packaging, advertising logos, and the sight, smell and
taste of foods. Any or all such stimuli can serve as Pavlovian
☆ For the Special Issue of Physiology & Behavior, The implications of conditioning and
metabolism for addiction and obesity: A Festschrift honoring Stephen C. Woods.
⁎ Tel.: +1 802 656 4164; fax: +1 802 656 8783.
E-mail address: mark.bouton@uvm.edu.
conditional stimuli (CSs), and as CSs, they would be expected to
engage approach behavior (e.g., [9]) and any of a number of
conditioned responses that function to get the system ready for the
meal (e.g., [10]). They might also trigger eating, even when the
individual is full or satiated (e.g., [11–15]). Table 1 also illustrates
some voluntary, operant behaviors available for reinforcement by the
pleasures of eating—choosing a restaurant at the mall, purchasing
food, opening the packaging to get at the food, and of course handling
and eating it. Presentation of a Pavlovian CS would further excite or
motivate these behaviors through Pavlovian-instrumental transfer
(e.g., [16,17]), for example, by eliciting the “wanting” of food (e.g.,
[18,19]). Given the abundance and high palatability of the food in the
environment today, basic learning processes could easily support
strong appetite and food-motivated behavior.
Table 1 also lists a few examples of the kinds of “contexts” in which
food consumption occurs. Contexts are usually “background” cues in
which Pavlovian CSs are presented and operant behaviors are emitted.
Thus, as illustrated in Table 1, CSs and operant behaviors never occur
in a vacuum; they always occur in a context like a shopping mall, an
airport, the couch in front of the television, or a social event such as a
Superbowl party. Contexts like these may be directly associated with
food, and have all the behavioral consequences of CSs noted above.
Contexts can also provide a kind of hierarchical control over behavior
that occurs in them (e.g., [20]), as I will illustrate below. In that case,
they can engage behavior not by directly eliciting it, but by signaling
whether or not the CS or the operant behavior will be associated with
food.
Research in my laboratory and in others has discovered that
contexts play an especially important role in extinction. In extinction,
the reinforcer (food) is withdrawn, and the Pavlovian CS or the

0031-9384/$ – see front matter © 2010 Published by Elsevier Inc.

doi:10.1016/j.physbeh.2010.11.025
52 M.E. Bouton / Physiology & Behavior 103 (2011) 51–58
Table 1
Some natural conditioning events available for association with food.
Conditional stimuli (CSs)
Advertising logos
Food packaging
Sight of food
Smell of food
Taste of food
Operant (instrumental) actions
Approaching food court or restaurant
Purchasing food
Handling food
Contexts:
Airports
Shopping malls
Sporting events
Superbowl parties
Couch in front of the television
operant response consequently occurs without it. The strength of
Pavlovian or operant behavior therefore declines. An important fact
about extinction is that it does not erase the original learning; a great
deal of research suggests that the CS or behavior's original meaning is
still available in the brain, even after extensive exposure occasions in
which the CS or the behavior has occurred without the reinforcer (e.g.,
[21,22]). This means that an individual may find it hard to unlearn
behaviors and habits that have accumulated over years of drinking
soda and eating junk food. Instead, research suggests that extinction
depends at least partly on the organism actively learning something
new that inhibits performance based on the original learning. As I
illustrate below, the organism seems to learn that the CS or the
behavior is no longer paired with the reinforcer in the present context.
Behavior can thus return in other contexts, and this fact (among
others) makes extinguished behavior seem persistent and vulnerable
to relapse. I will illustrate the idea by reviewing several phenomena in
extinction. The phenomena and their implications have been
discussed elsewhere (e.g., [23,24]); what follows emphasizes recent
research with food reinforcers and attempts to develop some specific
implications for appetite and overeating.
1. Renewal of extinguished appetitive behavior
One phenomenon that is especially central to the contemporary
understanding of extinction is the renewal effect. Renewal has been
studied most extensively in Pavlovian learning. In its simplest form, if
a CS is associated with a reinforcer in one context (e.g., Context A),
and then extinguished in a second one (Context B), the CS elicits
behavior again when it is returned to the original context (Context A).
This is called ABA renewal. Renewal can also be observed if
conditioning, extinction, and testing respectively occur in Contexts
A, B, and C (ABC renewal) and A, A, and B (AAB renewal). Although the
latter forms of renewal are often somewhat weaker than the classic
ABA form, they are interesting because the final return of the behavior
occurs in a context other than the original conditioning context. ABC
and AAB renewals thus indicate that simple removal from the context
of extinction can be sufficient to cause a return of conditioned
responding. Something learned in, and specific to, the extinction
context inhibits or suppresses the conditioned response. Hence,
extinction is at least partly a context-dependent form of learning.
Many well-known studies of renewal used fear conditioning
techniques, where a tone is first paired with footshock and then
consequently evokes fear (e.g., [25,26]). The renewal of fear has
implications for the treatment and understanding of anxiety disorders
(e.g., [23,27]). However, we have also demonstrated various forms of
renewal in appetitive conditioning, where a CS like a tone is paired
with food pellets (e.g., [28–30]). In this case, food responding is
elicited by the CS again when it is removed from the extinction
context. The parallel between aversive and appetitive conditioning is
strong. In either case, there is often surprisingly little effect on the
strength of the response to the CS when the CS is switched from
Context A to Context B at the beginning of extinction. That is, the first
context switch has little influence on conditioned responding. Like the
ABC and AAB renewal phenomena themselves, this feature of
behavior suggests that conditioned responding can transfer surpris-
ingly well to new contexts, while extinction is more context-specific.
The asymmetry or imbalance may be fundamental to the persistence
of motivated behavior and appetite.
Weight loss is notoriously difficult to maintain after clinical
treatment, and Wilfley et al. [31] have recently noted how renewal
effects might contribute to making maintenance difficult. The renewal
effect might also work in more natural settings, outside the domain of
clinical treatment. For example, a friend of mine was able to eat
sensibly and extinguish a number of habitual appetitive behaviors
while she was at school for several months, away from a home
environment in which the family often gathered socially with great
quantities of food. At school, she stopped shopping for (and cooking)
large amounts of food when she was expecting guests, and refrained
from over-indulging at potlucks and dinner parties. On return home to
the family over the holidays, however, all the original behaviors
returned. In a similar way, those of us who feel cravings in the
presence of the odor of cinnamon buns may learn to suppress the urge
to eat in their presence when we visit the shopping mall. But the odor
may trigger the desire again when we encounter it at the airport.
The renewal effect is clearly evident in operant as well as Pavlovian
conditioning. Interestingly, its importance in operant learning has
been a focus of several behavioral pharmacology laboratories
investigating drug self-administration. The “compulsion” that some
individuals have to take drugs might bear some resemblance to the
“compulsion” that others have to eat food. ABA renewal effects have
now been documented after extinction when rats have been
reinforced for lever pressing with injections of heroin (e.g., [32]),
cocaine (e.g., [33]), a mixture of heroin and cocaine (e.g., [34]), or oral
delivery of ethanol (e.g., [35,36]). Specifically, when the rat learns to
take the drug via lever pressing in one context and then is
extinguished in another, drug seeking returns when the animal is
simply returned to the original context. ABA renewal has also been
demonstrated in operant experiments in which rats have learned to
lever press for food pellets instead of drugs (e.g., [37,38,76]).
Nakajima et al. [37] also found the renewal effect in discriminated
operant conditioning, where rats first learned to perform the operant
response in the presence of a light discriminative stimulus. In this
case, renewal took the form of increased responding in the presence of
the light in Context A after it had been fully extinguished in Context B.
The context thus controlled the tendency of the light to instigate food-
seeking behavior.
Most of the initial operant work focused on ABA renewal, ignoring
the ABC and AAB forms. In fact, there were a number of failures to
produce the AAB renewal effect [32,34,37]. This created a puzzle,
because as noted above, ABC and AAB renewals are important in
suggesting that removal from the extinction context is sufficient to
renew behavior. However, my colleagues and I have recently
produced good evidence of each of the renewal effects in free operant
learning [39]. For example, in one experiment rats were first trained
to lever press in Context A on a variable-interval (VI) 30-s schedule of
food-pellet reinforcement. After several daily sessions, they received
four sessions of extinction training in either Context A, the same
context, or Context B, a second context. After extinction was complete,
the rats were tested in extinction in both Contexts A and B
(counterbalanced). As shown in Fig. 1, when the rats extinguished
in Context B were tested in Context A (the ABA renewal condition),
there was a robust return of extinguished lever pressing. And when
the rats extinguished in Context A were tested in Context B (the AAB
renewal condition), there was also an increase in responding.
Although the AAB effect was significantly weaker than the ABA effect,
 M.E. Bouton / Physiology & Behavior 103 (2011) 51–58 53

Fig. 1. The renewal effect after the extinction of operant conditioning. Left: Mean lever-press responding in rats for a food-pellet reinforcer during each 30-min session of acquisition
(Context A) and extinction. Right: Mean responding during two 10-min test sessions in the extinction context and the non-extinction (renewal) context (counterbalanced; note
change in the y axis). AAB = renewal in Context B after extinction in Context A (n = 16); ABA = renewal in Context A after extinction in Context B (n = 16). Both groups increased
responding when moved to the nonextinction context (ps b 0.005); there was significantly more responding in the renewal context in the ABA condition than in the AAB condition
(p b 0.005).
From Bouton et al. [39], Experiment 1. Reprinted with permission.

100% of the animals in either condition responded more in the
renewal context than in the extinction context. In another exper-
iment, we found that tripling the amount of extinction in the AAB
design did not weaken the strength of the AAB renewal effect. And in a
third experiment, we found evidence of ABC renewal—when operant
conditioning occurred in Context A, extinction occurred in B, and
testing occurred in B and C, the rats recovered some responding in C.
In this case, 94% of the subjects showed more responding in the
nonextinction context than in the extinction context. Thus, there is no
doubt that mere removal from a context of extinction can be sufficient
to cause recovery of an extinguished appetitive operant. The
extinguished craving for a cinnamon bun, and the urge to get up
and buy one, can be renewed at the airport even if I've never eaten a
cinnamon bun there. One does not have to return to the original
context in which the operant behavior was learned.
Although the confirmation of operant ABC and AAB renewal is
important, there is little doubt that ABA is a stronger effect than
either AAB or ABC. One obvious reason is that testing in the ABA
design occurs in a context (Context A) that has been directly
associated with food during the original operant training. Thus, the
rat is returned to a food-associated context that can motivate
responding (through Pavlovian-instrumental transfer) via its direct
Pavlovian association with food (e.g., [17]). To test this idea, we ran
an experiment in which two groups of rats received operant lever
press-food training in Context A before receiving extinction (lever
press-no food) in Context B. A control group received the same
training that the ABA group in Fig. 1 had. But in an experimental
group, the rats were returned to Context A (with response lever
removed and unavailable) four times (30-min sessions each) after
every extinction session in Context B. This treatment created
substantial exposure to the food-associated box without food.
Although all that exposure should have weakened the influence of
the context-food association through extinction, as shown in Fig. 2, it
had no discernible impact on the strength of the renewal effect that
occurred there during a final test. Although we know that a direct
association between a context and a reinforcer can in fact augment
extinguished operant responding ([40], see below), the role of
Context A in the ABA renewal experiment may depend on another
mechanism. It seems likely that the context is playing the role of an
occasion setter, a stimulus that signals that the response–reinforcer
association is now in force, rather than simply that the context
means food. In Pavlovian experiments, such occasion setting is not
diminished by simple exposure to the occasion setting stimulus itself
(e.g., [41]), but instead requires disconfirmation of a target CS–
reinforcer relation in the presence of the occasion setter in order to
extinguish it. In the analogous operant case, simple passive exposure

Fig. 2. Lack of an effect of exposure to Context A on the ABA renewal effect in operant conditioning. Left: Mean lever-press responding during each 30-min session of acquisition
(Context A) and extinction (Context B). Right: Mean responding during the 10-min test sessions in the extinction context (Context B) and the non-extinction (renewal) context
(Context A; note change in y axis). EXP = extra exposures to Context A (n = 16) during the extinction phase; NoEXP = no such exposures (n = 16). There was a reliable increase in
responding in either group in Context A (p b 0.005), and no difference between the groups or group × test session interaction (Fs b 1).
From Bouton et al. [39], Experiment 4. Reprinted with permission.
54 M.E. Bouton / Physiology & Behavior 103 (2011) 51–58
to the cues of a fast food restaurant might not be as good at
weakening operant food-seeking as extinction of at least one
operant behavior in that context. We need more basic research on
how to get rid of occasion setting in both Pavlovian and operant
learning.
It is worth explicitly summarizing what renewal might suggest for
our understanding of overeating. First, the renewal effect tells us that
extinction is not unlearning. The potential for old behaviors may
remain, perhaps indefinitely, in the memory system. Second, it
illustrates and supports the idea that extinction depends on learning
about the extinction context. Thus, as the overeater learns not to eat in
the presence of a trigger cue, he or she really learns not to eat in the
presence of the trigger cue in that context. Third, the ABC and AAB
effects further tell us that removal from the extinction context is
sufficient to cause a return of the behavior—it is not necessary to
return to the context in which indulgence has occurred in the past.
Finally, although a return to the original context (ABA) does produce
the strongest renewal effect, simple exposure to the context without
emission of the operant behavior (as in the experiment shown in
Fig. 2) is not sufficient to eliminate the renewal effect. This might be
one reason why simple exposure to cues associated with drugs and
perhaps food might be ineffective at reducing appetite (see [42]).
More research is needed to further understand how to eliminate
behavioral control by occasion setting.
2. Spontaneous recovery of extinguished appetitive behavior
Spontaneous recovery, another post-extinction behavior recovery
phenomenon, was first described by Pavlov ([43], e.g., p. 58). In either
Pavlovian or operant learning, allowing time to elapse after extinction
has occurred can allow the behavior to at least partially return. In the
animal lab, the CS elicits responding again, or the subject now returns
to the lever and emits some responding. Extinguished appetitive
behavior can thus return after a delay.
Several explanations of spontaneous recovery are available (e.g.,
[44]). One idea is that the organism pays less and less attention to the
CS (or perhaps its own behavior) as extinction training continues, and
this attention recovers spontaneously over time (e.g., [45]). However,
spontaneous recovery also occurs after counterconditioning, where a
CS associated with food is associated with a qualitatively different
event (e.g., footshock) in the second phase [46]. The subject shows, by
its appropriate shock-related behavior when the CS is presented, that
it is still attending to the CS during Phase 2. My students and I have
instead emphasized that time may be part of the context. Just as the
organism might learn that the CS or operant response is no longer
paired with food here, it might learn that the CS is no longer paired
with food now. Spontaneous recovery is essentially the renewal effect
that occurs when testing occurs in a temporal context that is different
from the temporal context of extinction. Consistent with this view, we
have recently shown that temporal intervals can indeed play the role
of context and control performance to CSs embedded in them [47–49].
In a sense, the animal fails to retrieve the extinction learning (or
counterconditioning) as time elapses after extinction. Consistent with
this possibility, we have shown that presentation of a retrieval cue
associated with extinction can reduce the strength of either
spontaneous recovery caused by changing the temporal context
(e.g., [50]) or renewal caused by changing the physical context [30].
Similar reminders could be built into weight loss programs to help
individuals remember to choose newly-learned healthy behaviors,
rather than older less healthy ones, in new contexts [31].
The general implication for appetite is that the urge to eat or seek
food may return sometime after the temptation has been successfully
suppressed. Once again, the conditioning processes that trigger
motivated behavior seem to generalize better across contexts than
the extinction process that suppresses them.
3. Resurgence of extinguished appetitive behavior
Another post-extinction phenomenon is worth addressing at this
point, because it seems especially relevant to the issue of overeating.
In resurgence, an organism first learns one operant response (e.g.,
pressing a lever in a Skinner box) and then, while that behavior is
being extinguished, a second “replacement” behavior is reinforced
(e.g., pressing a second lever in the box). In a third phase, the second
behavior is then extinguished, and the first response recovers (e.g.,
[51–53]). Once again, extinction—in this case coupled with the
explicit reinforcement of a new behavior—does not erase the original
memory, but only suppresses it. An example of resurgence is
presented in Fig. 3 (Winterbauer and Bouton, unpublished). Rats
were first trained to press on a lever (Lever 1) reinforced on a VI 30
schedule (Panel A). Then, presses on Lever 1 were extinguished while
presses on a second lever were reinforced (or, in a control group,
ignored; Panel B). When responses on Lever 2 were then in turn
extinguished, responses on Lever 1 returned in the experimental
group (Panel D at right in Fig. 3). New experiments (in progress) have
found that resurgence is even stronger when the first response
receives more training than the rather minimal amount shown. The
phenomenon also has some generality. For example, a number of
different schedules of reinforcement on Levers 1 and 2 support the
effect [53], and resurgence has also been shown to occur in rats
reinforced in Phase 1 with other reinforcers, like alcohol [55].
In the world outside the laboratory, reinforcement of other
behaviors during extinction of a target behavior is probably the rule
rather than the exception. A dieter who is reducing his or her calorie
intake might be encouraged to take up new behaviors, such as
walking or running or playing volley ball. Any such replacement
behavior has its own payoff according to its own natural schedule of
reinforcement. Extinction of the new behavior could allow the bad
habits to resurge. In the clinic, weight loss and self-regulatory
behaviors can be explicitly reinforced via social contact with a
clinician, or even more effectively, with money (e.g., [56]). Although
the evidence suggests that such strategies help with weight loss, the
implication of resurgence is that the extinguished appetitive behavior
should be expected to lapse to some extent if the new behavior is put
on extinction. Consistent with this, weight is regained at least partially
in the first few months after monetary incentives are withdrawn (e.g.,
[56]).
Resurgence can result from several behavioral processes. An early
idea [57] was that the reinforcement of the replacement behavior
might reduce the frequency of the behavior it is replacing so radically
that the individual has fewer opportunities to learn about extinction.
The replacement behavior, or the availability of the incentive
motivating it, suppresses the original response so much that the
subject insufficiently samples the new extinction contingency. We
have recently shown, however, that resurgence readily occurs even
when the new behavior is reinforced on a reinforcement schedule that
is lean enough so that it does not suppress the original behavior
beyond the suppression that occurs through extinction alone [53]. As
an alternative, we have therefore proposed that resurgence can be
considered a subtle example of the renewal effect (see also [58]). The
original behavior is extinguished in the “context” of the new one
being reinforced. Then, when reinforcement of the new behavior is
discontinued, and its frequency begins to decline, the context changes
again. Resurgence might thus be interpreted as another example of
the ABC renewal effect—conditioning generalizes more than extinc-
tion does to the new context.
Perhaps the most straightforward implication of this contextual
analysis of resurgence is that it highlights the importance of the
transition between the extinction plus replacement behavior training
and the test in which the replacement is put on extinction. A
procedure that makes the transition less abrupt, e.g., by fading in the
extinction phase so that the new reinforcer gradually becomes less
 M.E. Bouton / Physiology & Behavior 103 (2011) 51–58 55

Fig. 3. Resurgence after extinction in operant conditioning. A target operant behavior (presses to Lever 1, or L1) was first reinforced on a VI 30 reinforcement schedule over several
30-min acquisition sessions (Panel A). Then, in a 4-day extinction phase, presses to L1 were extinguished under normal conditions (Group EXT, n = 7), or while a second behavior
(presses to Lever 2, or L2) was also reinforced on a fixed-ratio 10 schedule (Group FR10, n = 7). Responding on Lever 1 is shown in Panel B, whereas responding on Lever 2 is shown
in Panel C. In a final 30-min test session, presses on Lever 2 were then extinguished. As shown in Panel D, responding on Lever 1 increased during this session compared to the final
extinction session, F (1,6) = 39.09, p b 0.005; there was no such increase in responding in the control group (EXT), which pressed Lever 1 reliably less than Group FR10 during the
resurgence test, F (1,12) = 11.00, p b 0.01. Lever 2 responses are shown in Panel E. Unpublished data by Winterbauer and Bouton.

frequent, would produce a less salient context change and ultimately
allow extinction to be connected with contexts beyond the artificial
one in which monetary incentives are earned. We are currently
running experiments in the animal laboratory that investigate such
methods.
4. Reinstatement of extinguished appetitive behavior
We also know that extinguished behavior can readily return if the
organism is exposed to the reinforcer itself again after extinction. In
Pavlovian learning, the food reinforcer can be presented and
consumed again, independently of the CS, after extinction. When
the CS is then tested, it can trigger responding again [28,59,60].
Reinstatement also occurs after operant extinction. If the reinforcer is
simply presented after lever pressing has been extinguished, the rat
may return to lever pressing [40,61,62]. In the laboratory, the
reinstating reinforcers are usually presented somewhat artificially;
they are delivered freely, that is, not contingent on a particular
behavior. Humans probably rarely receive rewards that are truly free
and noncontingent. But we have recently shown that reinforcers
presented contingent on a second behavior also reinstate a separate
extinguished operant [54]. Because of reinstatement, it is easy to see
why an overeater might begin bingeing again if he or she eats some
tasty guacamole at a Superbowl party. The presentation of the
reinforcer would increase Pavlovian responding as well as voluntary,
food-seeking operant behavior.
There are several mechanisms underlying reinstatement. In the
drug self-administration literature, the operating assumption is
usually that presentation of the drug primes neural reward circuits
(but see [63]). A mechanism we have emphasized is the fact that
presentation of the reinforcer allows it to be associated with the
context, and this contextual conditioning triggers the extinguished
response. Consistent with a role for context conditioning, in both
Pavlovian and operant learning, presentation of the reinforcer in a
different context has very little impact on the extinguished response
in the context where it has been extinguished (e.g., [26,28,40,59,64]).
Similarly, extended exposure to the context after presentation of the
reinforcer can reduce its reinstating impact [40,64]. Theoretically, a
context made “hot” by recent association with the reinforcer might
motivate and invigorate the Pavlovian or instrumental response
(exposure to the context without the reinforcer would effectively
“cool” the context down). Or a hot context might set the occasion for
the response, because the context was also hot when the response
was originally reinforced, and thus provides a kind of contextual
stimulus for another ABA renewal effect.
Another mechanism behind reinstatement, though, is direct
occasion setting by the reinforcer itself. When the rat lever presses
in an operant conditioning session, or the human takes a tasty chip
from the bag, the taste and feel of the reinforcer provide a stimulus
that is associated with reinforcement of the next response: The rat
therefore returns to the lever, and the human reaches into the bag
again. Presentation of food reinforcers after extinction might
reinvigorate appetitive behavior because they are thus part of the
context that supported them. One implication is that providing
occasional reinforcers during extinction, so they are now decoupled
from reinforcement of the next response, can reduce the reinstate-
ment effect (e.g., [54,62]). I will return to this somewhat paradoxical
idea in the next section.
56 M.E. Bouton / Physiology & Behavior 103 (2011) 51–58
5. Rapid reacquisition of extinguished appetitive behavior
We (e.g., [23]) have previously noted that the phenomena
discussed above can cause extinguished behavior to lapse or “slip.”
But one very important feature of eating and appetitive behavior in
the natural world is that the act of eating during a lapse will inevitably
bring the stimuli and behavior into contact with the reinforcer again.
Response recovery by renewal, spontaneous recovery, resurgence, or
reinstatement will bring the individual back in contact with new CS–
reinforcer or action–reinforcer pairings.
This observation is important, because we know that behaviors
that have received extensive training, like those presumably involved
in overeating, are quick to return to full force when the CS or the
operant behavior is paired with the reinforcer again [65]. Such a
return to the full-blown, reinforced, behavior would be the start of
true relapse. Many investigators have been satisfied to assume that
rapid reacquisition is merely evidence that the original learning is
“saved” or retained after extinction. But this doesn't really explain
why the behavior rapidly returns. Instead, at a more mechanistic level,
the reintroduction of the CS–reinforcer or action–reinforcer contin-
gency may return the subject to the original acquisition context,
because conditioning originally occurred in the presence of a memory
of recent reinforced trials. In this sense, rapid reacquisition may be an
ABA renewal effect [65–67].
An important implication of this analysis is analogous to one we
discussed a few paragraphs ago. If we are to protect an individual from
relapse via rapid reacquisition, it is best to try to associate the
renewing cues—CS–reinforcer or action–reinforcer pairings—with
extinction. One can do this by inserting occasional CS–reinforcer or
action–reinforcer pairings into an extinction procedure. In Pavlovian
experiments [66], rats received extensive extinction in which the CS
was paired with the reinforcer very infrequently, in the midst of many
extinction trials. The idea was to associate the reinforced trials with
ensuing extinction trials. Consistent with our hypothesis, the
treatment slowed down the reacquisition that occurred when CS
and US were again paired relative to a group that had received simple
extinction. The new pairings were a feature of extinction. Fig. 4 shows
the results of a test phase in an analogous experiment done in operant
conditioning [67]. Here, after initial reinforcement of lever pressing
Fig. 4. Reacquisition of an operant lever-pressing response following eight 60-min
sessions of either simple extinction (EXT) or extinction with occasional reinforced trials
(ns = 8) (lever pressing had originally been reinforced on a VI 30 schedule during five
30-min acquisition sessions). Responding is shown as the percent of the response rate
achieved during the last acquisition session. Baseline behavior immediately before the
beginning of reacquisition is shown at left (“before”). The remaining data are response
rates during successive 4-min periods after each new response–reinforcer pairing.
Notice that the rats given simple extinction responded substantially more after each
new pairing than the rats given extinction with occasional reinforced trials (ps b 0.05).
Adapted from Woods and Bouton [67], Experiment 2, with permission from the
publisher (El).
with food pellets on a VI 30 schedule, the rats received either simple
extinction or extinction with occasional response–reinforcer pairings
(the reinforcement schedule was gradually made leaner, until the rats
were responding on a VI 32-min schedule). Responding declined with
either method, although the occasional response–reinforcer pairings
led to less complete decline (as illustrated by “before” at left in Fig. 4).
Importantly, when the response was then paired with the reinforcer
again in a reacquisition phase, the partial reinforcement procedure
reduced the impact of each new pairing. That is, while rats that had
received simple extinction increased their responding during the
minutes after each new action–reinforcer pairing, the partially
reinforced rats did not. In this way, occasional reinforcement of the
action during extinction may slow down relapse when the action
regularly leads to the reinforcer again.
The implications for overeating (and for drug taking) are
interesting. If a person has learned to eat unhealthy foods (or take
drugs) repeatedly in binges, then eating a snack (or taking a drink)
becomes a cue for reinforcement of the next one. A snack is a stimulus
for additional intake. Therefore, an extinction procedure that includes
the consumption of occasional snacks in a way that allows them to be
associated with an overall reduction in intake may help reduce the
risk of a lapse becoming a relapse.
6. Discussion and conclusion
Each of the phenomena described above illustrates at least three
points. First, extinction is not the same as erasure; the tendency to
respond can readily return. Second, each phenomenon suggests a
possible mechanism or lapse and/or relapse. And third, the overall
message is that extinguished responding is highly sensitive to
manipulations of the context. In this regard, it is worth noting that
our discussion of “context” has assumed that the concept is very
broad. Although many experiments in the animal learning literature
define context as the box or immediate environment in which
learning and remembering take place, I have discussed the role of a
number of possible background stimuli as part of the context,
including time (in explaining spontaneous recovery), other behaviors
and their reinforcement (in explaining resurgence), presentations of
the reinforcer or context–reinforcer associations (in explaining
reinstatement), and CS–reinforcer and action–reinforcer pairings (in
explaining rapid reacquisition). Other research indicates that emo-
tions and drug states can provide contexts too (e.g., see [23]). And it is
noteworthy that Davidson [68,69] has argued that the state of food
deprivation can also provide a contextual stimulus; this may help
explain how hunger can modulate behavior (see also [16]). In our
view, it is advisable to think of “context” as any of a large number of
background stimuli.
It is also appropriate to think more broadly about the inhibitory
processes that are engaged by extinction. Is the context important at
influencing all forms of inhibitory learning? The answer to this
question turns out to be complex. Learning theorists often study
inhibition by means of so-called feature-negative discriminations in
which one CS (X) is paired with a reinforcer when it is presented alone
(X+), but without the reinforcer when it is combined with a second
CS (Y) (XY−). In the world of eating and appetite, a child might learn
that the sight of a bright blue box of sugary cereal is associated with
something wonderful (X+), but not if a disapproving adult is also
present (XY−). The box will therefore generate more excitement
when the disapproving adult is not around. Learning theories
attribute the behavioral pattern to the development of conditioned
inhibition to stimulus Y (the adult) (X might also develop a little
inhibition of its own after it first acquires an association with food).
Interestingly, laboratory experiments on feature-positive discrimina-
tions suggest that inhibition to Y is not specific to the original context;
inhibition to Y transfers without loss when tested in a new context
[70,71]. But surprisingly, in the new context, responding to X becomes
 M.E. Bouton / Physiology & Behavior 103 (2011) 51–58
more difficult to inhibit. Thus, the bright blue box may stimulate more
excitement and eating at the vacation cottage than at home, even in
the presence of the disapproving adult.
A more general implication of this digression is that inhibition per
se is not context-specific. It turns out that the second thing learned
about a stimulus is often more context-specific than the first thing
learned [72]—as if the memory system treats the second thing learned
about a stimulus as a conditional, context-specific exception to the rule
[23]. Extinction is thus relatively context-specific because it is the
second thing the organism learns about the CS or the operant action. In
counterconditioning, where a CS is first associated with one reinforcer
and then a second reinforcer in a subsequent phase, the principle is the
same: the CS's association with the second reinforcer is more context-
specific than the association with the first [73]. It is thus retroactive
inhibition (or retroactive interference), not conditioned inhibition, that
is specific to its context. What I have been claiming for extinction could
apply to any situation in which new learning replaces old learning.
As I stated earlier, the causes of eating and overeating are numerous
and enormously complex. Yet, a learning analysis of conditioning and
extinction may provide a useful perspective on them. Although
conditioning mechanisms have almost certainly evolved to allow
organisms to cope effectively with biologically-significant events (e.g.,
[2,74,75]) the bets are off in the modern world, given the hyperpalat-
ability of modern food, its abundance, and the ease with which it can
be eaten and ingested [8]. Given the number of palatable foods out
there, and the known behavioral effects of Pavlovian and operant
learning processes, it is not surprising that we eat, and are tempted to
eat, in the presence of so many CSs and contexts.
The idea developed here is that extinction, whether it is
deliberately produced through weight loss regimes or more naturally
by naturally-occurring extinction trials, does not permanently undo
the effects of conditioning. At the most general level, we should not
suppose that it erases, even partially, the learning allowed by our
food-saturated world. Instead, in both Pavlovian and instrumental
situations, extinction is best thought of as new learning that is at least
partly context-dependent. There seems to be a bias or imbalance in
basic learning processes such that conditioning tends to generalize
across contexts, whereas extinction generalizes less completely.
Renewal, spontaneous recovery, resurgence, reinstatement, and
rapid reconditioning all suggest that lapse and relapse are under-
standable (and perhaps inevitable) after extinction. And they may
contribute to why behavior, once learned, is so persistent. This point
has been made before concerning anxiety disorders and drug
dependence. The arguments may apply equally to our understanding
of eating, appetite, and their apparent potency today.
Acknowledgments
Supported by Grant R01 MH064847 from the National Institute of
Mental Health. I thank Leonard Epstein, Lilya Sitnikov, Travis Todd,
Drina Vurbic, and Neil Winterbauer for their discussion and
comments, and Stephen C. Woods for his friendship and inspiration.
References
[1] Woods SC. The control of food intake: behavioral versus molecular perspectives.
Cell Metab 2009;9:489–98.
[2] Woods SC. The eating paradox: how we tolerate food. Psychol Rev 1991;98:
488–505.
[3] Strubbe JH, Woods SC. The timing of meals. Psychol Rev 2004;111:128–41.
[4] Woods SC, Strubbe JH. The psychobiology of meals. Psychon Bull Rev 1994;1:
141–55.
[5] Capaldi ED, editor. Why we eat what we eat. Washington, DC: American
Psychological Association; 1996.
[6] Holland PC, Petrovich GD. A neural systems analysis of the potentiation of feeding
by conditioned stimuli. Physiol Behav 2005;86:747–61.
[7] Seeley RJ, Ramsay DS, Woods SC. Regulation of food intake: interactions between
learning and physiology. In: Bouton ME, Fanselow MS, editors. Learning,
57
motivation, and cognition: the functional behaviorism of Robert C. Bolles.
Washington, DC: American Psychological Association; 1997. p. 99–115.
[8] Kessler DA. The end of overeating: taking control of the insatiable American
appetite. New York: Rodale; 2009.
[9] Hearst E, Jenkins HM. Sign-tracking: the stimulus-reinforcer relation and directed
action. Austin, TX: The Psychonomic Society; 1974.
[10] Bouton ME. Learning and behavior: a contemporary synthesis. Sunderland, MA:
Sinauer Associates, Inc.; 2007.
[11] Birch LL. Obesity and eating disorders: a developmental perspective. Bull Psychon
Soc 1991;29:265–72.
[12] Holland PC, Petrovich GD, Gallagher M. The effects of amygdale lesions on
conditioned stimulus-potentiated eating in rats. Physiol Behav 2002;76:117–29.
[13] Petrovich GD, Ross CA, Gallagher M, Holland PC. Learned contextual cue
potentiates eating in rats. Physiol Behav 2007;90:363–7.
[14] Weingarten HP. Conditioned cues elicit feeding in sated rats: a role for learning in
meal initiation. Science 1983;220:431–3.
[15] Weingarten HP. Meal initiation controlled by learned cues: basic behavioral
properties. Appetite 1984;6:387–401.
[16] Balleine BW. Neural bases of food-seeking: affect, arousal and reward in
corticostriatolimbic circuits. Physiol Behav 2005;86:717–30.
[17] Rescorla RA, Solomon RL. Two-process learning theory: relationships between
Pavlovian conditioning and instrumental learning. Psychol Rev 1967;74:151–82.
[18] Berridge KC. ‘Liking’ and ‘wanting’ food rewards: brain substrates and roles in
eating disorders. Physiol Behav 2009;97:537–50.
[19] Berridge KC, Robinson TE, Aldridge JW. Dissecting components of reward: ‘liking’,
‘wanting’, and learning. Curr Opin Pharmacol 2009;9:65–73.
[20] Holland PC. Occasion setting in Pavlovian conditioning. In: Medin DL, editor. The
psychology of learning and motivation, Vol. 28. San Diego, CA: Academic Press;
1992. p. 69–125.
[21] Bouton ME. Context and behavioral processes in extinction. Learn Mem 2004;11:
485–94.
[22] Rescorla RA. Experimental extinction. In: Mowrer RR, Klein SB, editors. Handbook
of contemporary learning theories. Mahwah, NJ: Erlbaum; 2001. p. 119–54.
[23] Bouton ME. Context, ambiguity, and unlearning: sources of relapse after
behavioral extinction. Biol Psychiatry 2002;52:976–86.
[24] Bouton ME, Woods AM. Extinction: behavioral mechanisms and their implica-
tions. In: Byrne JH, Sweatt D, Menzel R, Eichenbaum H, Roediger H, editors.
Learning and memory: a comprehensive reference (vol. 1, learning theory and
behaviour). Oxford: Elsevier; 2008. p. 151–71.
[25] Bouton ME, Bolles RC. Contextual control of the extinction of conditioned fear.
Learn Motiv 1979;10:445–66.
[26] Bouton ME, King DA. Contextual control of the extinction of conditioned fear: tests
for the associative value of the context. J Exp Psychol Anim Behav Process 1983;9:
248–65.
[27] Bouton ME. Context and ambiguity in the extinction of emotional learning:
implications for exposure therapy. Behav Res Ther 1988;26:137–49.
[28] Bouton ME, Peck CA. Context effects on conditioning, extinction, and reinstatement
in an appetitive conditioning preparation. Anim Learn Behav 1989;17:188–98.
[29] Bouton ME, Ricker ST. Renewal of extinguished responding in a second context.
Anim Learn Behav 1994;22:317–24.
[30] Brooks DC, Bouton ME. A retrieval cue for extinction attenuates response recovery
(renewal) caused by a return to the conditioning context. J Exp Psychol Anim
Behav Process 1994;20:366–79.
[31] Wilfley DE, Van Buren DJ, Theim KR, Stein RI, Saelens BE, Ezzet F, et al. The use of
biosimulation in the design of a novel multilevel weight loss maintenance
program for overweight children. Obesity 2010;18:S91–8.
[32] Bossert JM, Liu SY, Lu L, Shaham Y. A role of ventral tegmental area glutamate in
contextual cue-induced relapse to heroin seeking. J Neurosci 2004;24:10726–30.
[33] Hamlin AS, Clemens KJ, McNally GP. Renewal of extinguished cocaine-seeking.
Neuroscience 2008;151:659–70.
[34] Crombag HS, Shaham Y. Renewal of drug seeking by contextual cues after
prolonged extinction in rats. Behav Neurosci 2002;116:169–73.
[35] Hamlin AS, Newby J, McNally GP. The neural correlates and role of D1 dopamine
receptors in renewal of extinguished alcohol-seeking. Neuroscience 2007;146:
525–36.
[36] Zironi I, Burattini C, Aicardi G, Janak PH. Context is a trigger for relapse to alcohol.
Behav Brain Res 2006;167:150–5.
[37] Nakajima S, Tanaka S, Urushihara K, Imada H. Renewal of extinguished lever-press
responses upon return to the training context. Learn Motiv 2000;31:416–31.
[38] Nakajima S, Urushihara K, Masaki T. Renewal of operant performance formerly
eliminated by omission or noncontingency training upon return to the acquisition
context. Learn Motiv 2002;33:510–25.
[39] Bouton ME, Todd TP, Vurbic D, Winterbauer NE. Renewal after the extinction of
free operant behavior. Learn Behav 2011;39:57–67.
[40] Baker AG, Steinwald H, Bouton ME. Contextual conditioning and reinstatement of
extinguished instrumental responding. Q J Exp Psychol 1991;43B:199–218.
[41] Rescorla RA. Extinction of facilitation. J Exp Psychol Anim Behav Process 1986;12:
16–24.
[42] Conklin CA, Tiffany ST. Applying extinction research and theory to cue-exposure
addiction treatments. Addiction 2002;97:155–67.
[43] Pavlov IP. Conditioned reflexes. Oxford, UK: Oxford University Press; 1927.
[44] Rescorla RA. Spontaneous recovery. Learn Mem 2004;11:501–9.
[45] Robbins SJ. Mechanisms underlying spontaneous recovery in autoshaping. J Exp
Psychol Anim Behav Process 1990;16:235–49.
[46] Bouton ME, Peck CA. Spontaneous recovery in cross-motivational transfer
(counterconditioning). Anim Learn Behav 1992;20:313–21.
58 M.E. Bouton / Physiology & Behavior 103 (2011) 51–58
[47] Bouton ME, García-Gutiérrez A. Intertrial interval as a contextual stimulus. Behav
Process 2006;71:307–17.
[48] Bouton ME, Hendrix MC. Intertrial interval as a contextual stimulus: further
analysis of a novel asymmetry in temporal discrimination learning. J Exp Psychol
Anim Behav Processes 2011;37:79–93.
[49] Todd TP, Winterbauer NE, Bouton ME. Interstimulus interval as a discriminative
cue: evidence of the generality of a novel asymmetry in temporal discrimination
learning. Behav Process 2010;84:412–20.
[50] Brooks DC, Bouton ME. A retrieval cue for extinction attenuates spontaneous
recovery. J Exp Psychol Anim Behav Process 1993;19:77–89.
[51] Leitenberg H, Rawson RA, Bath K. Reinforcement of competing behavior during
extinction. Science 1970;169:301–3.
[52] Lieving GA, Lattal KA. Recency, repeatability, and reinforcer retrenchment: an
experimental analysis of resurgence. J Exp Anal Behav 2003;80:217–33.
[53] Winterbauer NE, Bouton ME. Mechanisms of resurgence of an extinguished
instrumental behavior. J Exp Psychol Anim Behav Process 2010;36:343–53.
[54] Winterbauer NE, Bouton ME. Mechanisms of resurgence II: response-contingent
reinforcers can reinstate a second extinguished behavior. Submitted for
publication. Learn Motiv.
[55] Podlesnik CA, Jimenez-Gomez C, Shahan TA. Resurgence of alcohol seeking
produced by discontinuing non-drug reinforcement as an animal model of drug
relapse. Behav Pharmacol 2006;17:369–74.
[56] Volpp KG, John LK, Troxel AB, Norton L, Fassbender J, Loewenstein G. Financial
incentive-based approaches for weight loss: a randomized trial. J Am Med Assoc
2008;300:2631–7.
[57] Rawson RA, Leitenberg H, Mulick JA, Lefebvre MF. Recovery of extinction
responding in rats following discontinuation of reinforcement of alternative
behavior: a test of two explanations. Anim Learn Behav 1977;5:415–20.
[58] Bouton ME, Swartzentruber D. Sources of relapse after extinction in Pavlovian and
instrumental learning. Clin Psychol Rev 1991;11:123–40.
[59] Bouton ME. Differential control by context in the inflation and reinstatement
paradigms. J Exp Psychol Anim Behav Process 1984;10:56–74.
[60] Rescorla RA, Heth CD. Reinstatement of fear to an extinguished conditioned
stimulus. J Exp Psychol Anim Behav Process 1975;1:88–96.
[61] Reid RL. The role of the reinforcer as a stimulus. Br J Psychol 1958;49:202–9.
[62] Rescorla RA, Skucy JC. Effect of response-independent reinforcers during
extinction. J Comp Physiol Psychol 1969;67:381–9.
[63] Wise RA, Wang B, You Z-B. Cocaine serves as a peripheral interoceptive
conditioned stimulus for central glutamate and dopamine release. PLoS ONE
2008;3:1–9.
[64] Bouton ME, Bolles RC. Role of conditioned contextual stimuli in reinstatement of
extinguished fear. J Exp Psychol Anim Behav Process 1979;5:368–78.
[65] Ricker ST, Bouton ME. Reacquisition following extinction in appetitive condition-
ing. Anim Learn Behav 1996;24:423–36.
[66] Bouton ME, Woods AM, Pineño O. Occasional reinforced trials during extinction
can slow the rate of rapid reacquisition. Learn Motiv 2004;35:371–90.
[67] Woods AM, Bouton ME. Occasional reinforced responses during extinction can
slow the rate of reacquisition of an operant response. Learn Motiv 2007;38:56–74.
[68] Davidson TL. The nature and function of interoceptive signals to feed: toward
integration of physiological and learning perspectives. Psychol Rev 1993;100:
640–57.
[69] Davidson TL. Hunger cues as modulatory stimuli. In: Schmajuk NA, Holland PC,
editors. Occasion setting: associative learning and cognition in animals.
Washington, DC: American Psychological Association; 1998. p. 223–48.
[70] Bouton ME, Nelson JB. Context-specificity of target versus feature inhibition in a
feature-negative discrimination. J Exp Psychol Anim Behav Process 1994;20:
51–65.
[71] Nelson JB, Bouton ME. The effects of a context switch following serial and
simultaneous feature-negative discriminations. Learn Motiv 1997;28:56–84.
[72] Nelson JB. Context specificity of excitation and inhibition in ambiguous stimuli.
Learn Motiv 2002;33:284–310.
[73] Peck CA, Bouton ME. Context and performance in aversive-to-appetitive and
appetitive-to-aversive transfer. Learn Motiv 1990;21:1–31.
[74] Hollis KL. Pavlovian conditioning of signal-centered action patterns and
autonomic behavior: a biological analysis of function. In: Rosenblatt RA, Hinde
RA, Beer C, Busnel MC, editors. Advances in the study of behavior, Vol. 12. New
York: Academic Press; 1982. p. 1–64.
[75] Hollis KL. Contemporary research on Pavlovian conditioning: a “new” functional
analysis. Am Psychol 1997;52:956–65.
[76] Welker RL, McAuley K. Reductions in resistance to extinction and spontaneous
recovery as a function of changes in transportational and contextual stimuli. Anim
Learn Behav 1978;6:451–7.