Behavioural Brain Research 335 (2017) 215–222

Contents lists available at ScienceDirect

Behavioural Brain Research

journal homepage: www.elsevier.com/locate/bbr

Research report

Operant models of relapse in zebrafish (Danio rerio): Resurgence, renewal, MARK

and reinstatement
⁎
Toshikazu Kurodaa, , Yuto Mizutanib, Carlos R.X. Cançadoc, Christopher A. Podlesnikd,e
a
Aichi Bunkyo University, 5969-3 Okusa, Komaki, Aichi, 485-8565, Japan
b
Aichi Gakuin University, 12 Araike Iwasaki, Nisshin, Aichi, 470-0195, Japan
c
Universidade de Brasília, Campus Universitário Darcy Ribeiro, 70.910-900, Brasília, DF, Brazil
d
Florida Institute of Technology and The Scott Center for Autism Treatment, 150 W University Boulevard, Melbourne, FL 32901, USA
e
The University of Auckland, New Zealand

A R T I C L E I N F O A B S T R A C T

Keywords: Zebrafish are a widely used animal model in biomedical research, as an alternative to mammals, for having
Resurgence features such as a fully sequenced genome, high fecundity, and low-cost maintenance, but behavioral research
Renewal with these fish remains scarce. The present study investigated whether zebrafish could be a new animal model
Reinstatement for studies on the relapse of behavior (e.g., addiction and overeating) after the behavior has been extinguished.
Relapse
Specifically, we examined whether zebrafish would show three different types of relapse commonly studied with
Operant conditioning
Zebrafish
other species: resurgence, renewal, and reinstatement. For resurgence, a target response (i.e., approaching a
sensor) was established by presenting a reinforcer (i.e., shrimp eggs) contingent upon the response in Phase 1;
the target response was extinguished while introducing reinforcement for an alternative response in Phase 2;
neither response produced the reinforcer in Phase 3. For renewal, a target response was established under
Context A in Phase 1 and was extinguished under Context B in Phase 2; the fish were placed back in Context A in
Phase 3, where extinction remained in effect. For reinstatement, a target response was established in Phase 1 and
was extinguished in Phase 2; the reinforcer was presented independently of responding in Phase 3. Each type of
relapse occurred in Phase 3. These results replicate and extend previous findings on relapse to a new species and
suggest that zebrafish can be a useful animal model for studying the interactions of biological and environmental
factors that lead to relapse.

1. Introduction zebrafish are available (e.g., National BioResource Project in Japan).

Operant behavior results from the relations between behavior and
its consequences and has been used extensively as a model of voluntary
behavior [1,2]. The systematic analysis of environmental and biological
factors underlying operant behavior provides understanding and po-
tential treatments for a range of behaviors considered maladaptive,
such as addictions [3] and overeating [4]. Most research examining
biological and environmental factors in learning processes come from
studies with mammalian laboratory animals. As an alternative, zebra-
fish (Danio rerio) are ideal for examining environmental and biological
factors involved in learning because of their high fecundity, low-cost
maintenance, transparent embryo, and fully sequenced genome
[e.g.,5–8]. Key neurobiological structures mediating the reinforcing
effects of rewards (e.g., drugs) appear conserved in zebrafish
[e.g.,9,10]. Moreover, many different types of genetically modified
These features offer biomedical and behavioral researchers opportu-
nities to examine how genes and environment interact to determine
operant behavior. Nevertheless, studies of operant behavior with zeb-
rafish remain few [11–13].
One area of research relevant to understanding operant behavior
generally, and treatments for maladaptive behavior specifically, is with
laboratory models of relapse. A persistent tendency to relapse even after
prolonged periods of abstinence is a defining feature of addictive be-
havior [14]. Therefore, relapse of previously learned behavior has been
investigated extensively during the past few decades [15–18]. Most
models of relapse arrange reinforcement of operant responding in Phase
1 to simulate engaging in maladaptive behavior and then, in Phase 2,
model a basic form of behavioral treatment by eliminating those re-
inforcers through extinction of operant responding. Models of relapse
primarily differ in Phase 3 as a result of arranging different factors

⁎
Corresponding author at: Aichi Bunkyo University, Faculty of Humanities, 5969-3 Okusa, Komaki, Aichi, 485-8565 Japan.
E-mail addresses: toshikazu.kuroda@gmail.com (T. Kuroda), yuto.mizutani3@gmail.com (Y. Mizutani), carlos.cancado@gmail.com (C.R.X. Cançado),
cpodlesnik@fit.edu (C.A. Podlesnik).

http://dx.doi.org/10.1016/j.bbr.2017.08.023
Received 25 July 2017; Received in revised form 3 August 2017; Accepted 14 August 2017
Available online 19 August 2017
0166-4328/ © 2017 Elsevier B.V. All rights reserved.
T. Kuroda et al.
Table 1
A summary of typical procedures for assessing resurgence, renewal, and reinstatement. “Target” and “Alt” stand for target and alternative responses, respectively.
Phase 1 Phase 2
Resurgence Target reinforced; Target extinguished;
Alt not reinforced Alt reinforced
Renewal Target reinforced in Context A Target extinguished in Context B
Reinstatement Target reinforced Target extinguished
contributing to the relapse. The relapse of previously reinforced and
extinguished responding in these models indicates that decreases in
operant responding during extinction do not represent an elimination of
prior learning. Instead, relapse indicates new learning involving the
extinction contingency [see 15].
Relapse of operant behavior has been demonstrated with a wide
variety of species in the three laboratory models of relapse serving as
the focus in this investigation (see Table 1). First, resurgence is the re-
lapse of a previously reinforced and then extinguished response when
reducing or eliminating reinforcement for a more recently trained re-
sponse [e.g.,19–22]. Hereafter, the response reinforced and then ex-
tinguished will be referred to as the target response and the more re-
cently trained response as the alternative response. Resurgence has been
demonstrated with rats [e.g.,20,23], pigeons [e.g.,24,25], Siamese
fighting fish [26], monkeys [27], and humans [e.g.,28]. Recently, our
laboratory demonstrated resurgence with zebrafish for the first time
[29].
Second, renewal is relapse due to changes in environmental context.
Specifically, renewal is the return of a target response when changing
the global environmental context compared to the extinction context
[e.g.,30,31,32]. Renewal occurs, for example, when reinforcing and
extinguishing the target response in different contexts and then (1)
returning to the training context (ABA renewal) or (2) transitioning to a
novel context (e.g., ABC renewal). Renewal of operant behavior has
been demonstrated with rats [30,33], pigeons [e.g.,34,35], and humans
[36] but not yet with zebrafish.
Finally, reinstatement is relapse resulting from exposure to a range of
events due to learning (e.g., cues) or biological relevance (e.g., stress)
[37]. One commonly used method for assessing reinstatement of an
extinguished target response is to present the event previously serving
as the reinforcer independently of responding [38,39]. Reinstatement
has been demonstrated with rats [e.g.,40], pigeons [e.g.,41,42], mon-
keys [43,44], and humans [e.g.,45,46]. Thus far, there have been no
demonstrations of reinstatement of operant behavior in zebrafish −
only reinstatement of a Pavlovian conditioned approach response to
drug (cocaine or nicotine) [10].
All these studies suggest relapse of previously reinforced and then
extinguished operant behavior is general across species. These three
procedures serve as experimental models for understanding the condi-
tions under which relapse occurs and might be used as platforms to
examine biological influences in relapse, including developmental,
pharmacological, and neurobiological factors [16]. Nevertheless, re-
search on laboratory models of relapse using zebrafish has been limited.
Therefore, the goals of the present study were to systematically re-
plicate resurgence and to extend the analysis to renewal, and re-
instatement of extinguished operant behavior in zebrafish using an
automated device to record discrete responses (i.e., approaching sen-
sors) and deliver decapsulated brine shrimp eggs as reinforcers [11,12].
2. General methods
2.1. Subjects
Forty experimentally naïve zebrafish (National Bioresource Project
of Japan, Riken Brain Science Institute), 7–8 months old at the start of
the experiments, served as subjects. They were wild-type fish (i.e., not
216
Behavioural Brain Research 335 (2017) 215–222
Phase 3
Target extinguished;
Alt extinguished
Target extinguished in Context A
Target extinguished; Response-independent delivery of reinforcer
genetically modified). Each fish was housed individually in a room with
a 14hr:10hr light-dark cycle (lights on at 6:00 am) in an aquarium
(13.0-cm wide by 16.8-cm long by 11.3-cm high) connected to a pump
and tank where water was filtered, aerated, and maintained at 28 °C
with a heater. Water was kept at a pH level of 7.5 and half of the water
in the tank was replaced weekly. When a fish produced fewer than 30
reinforcers (decapsulated brine shrimp eggs) in a session, postsession
feedings of supplemental food (Kyorin, Hikari Labo 270) occurred
30 min after sessions.
2.2. Material
Two types of automated device, similar to that described by Manabe
et al. [11], were used. Fig. 1 shows the device used to study resurgence
in Experiment 1. The device was placed in the aquarium of each fish
during sessions and consisted of three sensors and two feeders fixed on
Plexiglas. The three sensors were for the target, alternative, and control
responses, as described below. Each sensor consisted of a 2-mm dia-
meter polymeroptical fiber (Mitsubishi Rayon Co., LTD, Esca CK80), a
fiber sensor (Keyence FU-23X), an amplifier (Keyence, FS-V21X), and a
tri-color LED (Oput-Supply OSTA5131A-C). The tip of each sensor was
set 5 mm below the water surface and 5 mm away from the front wall of
the aquarium. The sensor tips were 4.5 cm apart from each other and
could be illuminated red, green, or blue. A response was registered
when the fish approached and was at least within 5 mm below the
sensor tip.
The feeders were located 2.25 cm to the left and right of the middle
sensor. Each feeder consisted of a servo motor (Futaba S3005), a brass
tube with the inner diameter of 600 μm, a 400-μm-diameter piano wire
inserted in the brass tube, a vibrator, and a white LED (i.e., the feeder
light). During reinforcement, the feeder light turned on immediately
upon a response, the servo motor pulled the piano wire, the vibrator
operated, decapsulated brine shrimp eggs were loaded into the brass
Fig. 1. A photo of the automated device used in Experiment 1. See text for details.
T. Kuroda et al.
tube, and the servo motor pushed the eggs out of the tube and into the
water (this whole series of events occurred approximately in 0.5 s). The
feeder light then flashed 5 times at a 0.2-s interval. Thereafter, the
device remained inoperative until 5 s had elapsed since the onset of
reinforcement.
The other type of the device, used to study renewal and reinstate-
ment in Experiments 2 and 3, respectively, was similar to that used in
Experiment 1 except that there were only two sensors (left and right),
9 cm apart from each other, and a single feeder in between. The two
sensors were for the target and control responses, as described below.
The feeder was set on the midline of the front wall of the aquarium.
Except in Experiment 2, in which changes in contextual stimuli were
necessary, a white partition covering each side of the aquarium mini-
mized visual distraction during sessions. A Lenovo® laptop computer
with Visual Basic 2015® controlled the experiment and recorded data
through an I/O board (Pololu, Mini Maestro servo controller).
2.3. Procedure
After acclimation in the home aquarium for 3-4 days, the fish were
trained to consume the reinforcer delivered from the feeder. At this
stage, a single feeder was set on the midline of the front wall of the
aquarium and delivered the reinforcer randomly three times per min
(i.e., 20 s on average). The sensors were removed to preclude re-
sponding to the sensors being reinforced adventitiously. Sessions ended
after 40 reinforcers.
When each fish consumed the reinforcer reliably, response training
initiated with the sensors and feeder(s) being set as described pre-
viously and by reinforcing responses to a target sensor according to a
fixed-ratio (FR) 1 schedule. In this schedule, every target response re-
sulted in a reinforcer delivery. When the fish obtained 40 reinforcers
within 20 min, the schedule was changed from an FR 1 to a variable-
interval (VI) 10 s, with the latter being in effect for three 20-min ses-
sions. Under the VI 10-s schedule, the first response after intervals with
the average of 10 s resulted in a reinforcer delivery. The VI schedules at
this stage and in subsequent phases in each experiment consisted of 10
intervals [47], randomly selected without replacement. Reinforcement
time (5 s per reinforcer) was excluded from the session time. Sessions
were conducted during the light period of the room’s light-dark cycle,
around the same time of the day, seven days a week.
2.4. Data analysis
The primary dependent variable was response rate (i.e., responses
per min). A one-way repeated-measures ANOVA was conducted in each
experiment with session as a within-subject independent variable and
target response rate as a dependent variable. To assess whether target
responding was sensitive to reinforcement and extinction con-
tingencies, the analysis included data from the last baseline (Phase-1)
session and from extinction (Phase-2) sessions. To determine the pre-
sence of relapse, the analysis included data from the last extinction
(Phase-2) session and from the five relapse-test (Phase-3) sessions.
These sessions were consecutive and were evenly spaced in time; thus,
the first-order autocorrelation (i.e., correlation between at a time, Tn,
and the one before, Tn-1) was expected. For this reason, the first-order
autoregressive structure was set for the covariance matrix when the
ANOVA was conducted using the proc mixed function on SAS®.
Following the ANOVA, the Bonferroni method was used for post hoc
tests (t tests) to control the familywise error rate at 0.05. This method
could be too conservative for a large number of contrasts between
means. The primary question of the present study was whether target
responding was higher in the five relapse-test (Phase-3) sessions than in
the last extinction (Phase-2) session so that the mean contrast was
limited to those five comparisons. Thus, the alpha level was set at 0.01
(i.e., 0.05 divided by 5) in each of the post hoc tests for assessing re-
lapse. Likewise, for assessing sensitivity to reinforcement and extinction
217
Behavioural Brain Research 335 (2017) 215–222
contingencies, the alpha level was set at 0.01, 0.007, and 0.006 for
Experiments 1, 2, and 3 where there were five, seven, and eight Phase-2
sessions, respectively.
3. Experiment 1
The first experiment investigated resurgence, which involved re-
inforcement of a target response in Phase 1, extinction of the target
response while reinforcing an alternative response in Phase 2, and ex-
tinction of both responses in Phase 3. In addition to the target and al-
ternative responses, a control response that was never reinforced
throughout the experiment was measured to dissociate resurgence from
a possible increase in general activity induced by extinction (i.e., ex-
tinction-induced variability) [48].
We recently demonstrated resurgence with zebrafish using a similar
experimental setup [29] but the study was limited in several aspects.
First, the magnitude of the resurgence effect was meager, perhaps due
to the reinforcement schedules used. We arranged equal VI 20-s sche-
dules in Phases 1 and 2, but previous studies of resurgence indicate
greater rates of alternative reinforcement in Phase 2 produce greater
resurgence [19]. Therefore, we doubled the rate of alternative re-
inforcement in Phase 2 relative to Phase 1. Second, the experimental
apparatus was not optimal for assessing resurgence. Specifically, three
sensors and one feeder were aligned as follows: left sensor, middle
sensor, the feeder, and right sensor with equal distances between each
pair. Either the left or right sensor was for the target response while the
other sensor was for the control response (counterbalanced across fish),
and the middle sensor was for the alternative response for each subject.
Thus, the distance between the target and alternative sensors, as well as
the distance between the control and alternative sensors, varied across
subjects. This might have unnecessarily increased between-subject
variability in the magnitude of resurgence. This problem was elimi-
nated in the present study by setting the three sensors equally apart
with a feeder being placed between each pair of the sensors, as de-
scribed in the Material section above.
3.1. Methods
3.1.1. Subjects and material
Sixteen experimentally naïve fish (8 males and 8 females) initially
served as subjects. Three male fish failed to acquire either a target or an
alternative response and their results are not reported. Two sets of the
device with three sensors and two feeders were used. In a feeder test
with 100 operations of the first set, a mean of 19.70 (SD = 6.99) and
24.03 eggs (SD = 7.62) were delivered per operation from the left and
right feeders, respectively. For the second set, a mean of 20.88
(SD = 6.98) and 23.67 eggs (SD = 7.88) were delivered per operation
from the respective feeders.
3.1.2. Procedure
3.1.2.1. Phase 1: acquisition. The target response was reinforced
according to a VI 20-s schedule. Alternative and control responses
were recorded but had no programmed consequence. Approaching
either the left or right sensor served as the target response, and the
opposite sensor was for the alternative response, counterbalanced
across subjects; the middle sensor served as the control response for
all the subjects. The LED colors (red, blue, and green) illuminating the
tips of the sensors were counterbalanced across the subjects. Only the
feeder on the same side as the target sensor operated. Sessions ended
after 20 min and this phase lasted for 15 sessions.
3.1.2.2. Phase 2: extinction and alternative reinforcement. The target
response was extinguished and the alternative response was
reinforced according to a VI 10-s schedule. Reinforcement rate was
doubled between phases because previous studies with other species
T. Kuroda et al.
showed higher reinforcer rates for alternative responding increase
resurgence [e.g.,18,19,49]. The control response remained
unreinforced. Only the feeder on the same side as the alternative
sensor operated. Sessions ended after 20 min and this phase lasted for 5
sessions.
3.1.2.3. Phase 3: resurgence test. Extinction was in effect for the target,
alternative, and control responses and neither feeder operated. Sessions
ended after 20 min and this phase lasted for 5 sessions. Resurgence was
defined as an increase in target response rates compared to target
response rates during the last session of Phase 2; additionally, this
increase in target response rate had to exceed that of control response
rates in a given session [see 48].
3.2. Results and discussion
The left graph in Fig. 2 shows mean response rates for the target,
alternative, and control responses in each phase. During acquisition in
Phase 1, target response rates were higher than alternative and control
response rates, indicating that decapsulated brine shrimp eggs indeed
served as a reinforcer when presented upon a target response. Control
response rates were slightly but consistently higher than alternative
response rates, most likely because the control (middle) sensor was
physically closer than the alternative sensor to the operable feeder. In
Phase 2, when removing reinforcement for target responding while
introducing reinforcement for the alternative response, target response
rates decreased and alternative response rates increased across sessions.
The decrease in target response rates from the last Phase-1 session to
Phase-2 sessions was verified with an ANOVA, F(5, 60) = 56.12,
p < 0.0001, with all Phase-2 sessions differing from the last Phase-1
session in post hoc tests (p’s < 0.0001). There was no reliable change
in control response rates from Phase 1–2. During the test of resurgence
in Phase 3, removing alternative reinforcement increased target re-
sponding but not control responding; alternative responding decreased
across sessions.
Mean obtained reinforcement rate (per min; SEM in parenthesis) for
target responding across fish, in each of the last five sessions of Phase 1,
were 2.07 (0.17), 2.24 (0.07), 2.17 (0.09), 2.27 (0.07), and 2.12 (0.12),
respectively. Mean obtained reinforcement rates for alternative re-
sponding in each session of Phase 2 were 0.86 (0.31), 1.47 (0.33), 2.21
(0.30), 2.54 (0.29), and 2.66 (0.26), respectively. Thus, the obtained
reinforcement rates in Phase 2 successively increased across sessions of
this phase. This trend corresponded to the change in alternative re-
sponse rates.
The right graph in Fig. 2 shows mean response rates in the last
session of Phase 2 and all sessions of Phase 3. The increase in target
response rates during Phase-3 sessions relative to the last Phase-2 ses-
sion was statistically significant, F(5, 60) = 6.14, p = 0.0001. Post hoc
tests indicated that the difference in target response rates between the
last Phase-2 session and the first Phase-3 session, but not for other
comparisons, was statistically significant, t(60) = 5.00, p < 0.0001.
This suggests that resurgence occurred in zebrafish and its magnitude
218
Behavioural Brain Research 335 (2017) 215–222
Fig. 2. Mean responses per min across fish in the last five
sessions of Phase 1 (Acquisition) and all sessions of Phase 2
(Extinction and alternative reinforcement) and Phase 3
(Resurgence test) in Experiment 1 (left graph). Closed sym-
bols, open symbols and a dashed line without symbols re-
present the target, alternative, and control responses, re-
spectively. Error bars show ± SEM. The right graph shows
target and control response rates in the last session of Phase 2
and each session of Phase 3 during resurgence tests.
was greatest in the first Phase-3 session.
Overall, the present findings demonstrate with zebrafish the re-
versal of performance with changes in contingencies in Phases 1 and 2
[see also 50–52] and the resurgence of target responding when extin-
guishing a more recently reinforced alternative response in Phase 3 [see
17]. The absence of systematic change in control response rates sug-
gests the increase in target response rates in Phase 3 was a function of a
previous reinforcement history from Phase 1 and the removal of al-
ternative reinforcers when transitioning from Phase 2–3.
The present result was consistent with the finding of our previous
study [29] despite the modification in the arrangement of sensors and
feeders. We attempted to increase the magnitude of resurgence in the
present study by arranging a greater reinforcement rate for alternative
responding in Phase 2. Although the magnitude was comparable in the
present study, resurgence effects were replicated and therefore indicate
the generality and robustness of the effect. Further increasing the dif-
ference in reinforcement rate between phases might increase the mag-
nitude of resurgence with zebrafish but the magnitude might be limited
due to generally low target response in Phase 1 compared to other
species [e.g.,20,23,24,25].
4. Experiment 2
The second experiment investigated ABA renewal, which involved
reinforcement of a target response under Context A in Phase 1 and
extinction of the response under Context B in Phase 2. In Phase 3, ex-
tinction remained in effect but the context was changed back to Context
A. As in Experiment 1, a control response that was never reinforced was
measured throughout the experiment to dissociate renewal from a
possible increase in general activity when the context was changed in
Phase 3. Contexts were visual stimuli covering the sides of the aquarium
because previous studies reported that zebrafish behavior is sensitive to
this modality of stimuli [53,54].
4.1. Methods
4.1.1. Subjects and material
Twelve experimentally naïve fish (6 males and 6 females) initially
served as subjects. Three male fish and one female fish failed to acquire
responding and their results are not reported. The device with two
sensors and a single feeder was used. In a feeder test with 100 opera-
tions, a mean of 24.9 eggs (SD = 6.09) were delivered per operation.
4.1.2. Procedure
4.1.2.1. Phase 1: acquisition. The target response was reinforced in
Context A according to a VI 20-s schedule. The control response was
recorded but had no programmed consequence. Locations of the target
and control sensors, as well as the LED colors associated with each
sensor, were counterbalanced across subjects. Context A was either a
white partition covering each side of the aquarium or a black partition
covering each side of the aquarium with a green silicon sheet
T. Kuroda et al.
underneath, counterbalanced across fish. Sessions ended after 20 min
and this phase lasted for 15 sessions.
4.1.2.2. Phase 2: extinction. The target response was extinguished in
Context B. Contextual stimuli for Context B were opposite to the one
used for Context A in Phase 1 for each fish. The control response
remained unreinforced. Sessions ended after 20 min and this phase
lasted seven sessions.
4.1.2.3. Phase 3: renewal test. Extinction remained in effect for the
target and control responses and each fish was re-exposed to Context A.
Sessions ended after 20 min and this phase lasted five sessions. Renewal
was defined as an increase in target response rates compared to target
response rates during the last session of Phase 2 and this increase had to
exceed that of control response rates in a given session.
4.2. Results and discussion
The left graph in Fig. 3 shows mean response rates for the target and
control responses in each phase. Target response rates were higher than
control response rates in Phase 1 (Context A). Obtained reinforcement
rates for target responding, calculated as in Experiment 1, in each of the
last five sessions of Phase 1 were 1.96 (0.21), 2.11 (0.08), 2.04 (0.13),
2.13 (0.05), and 2.12 (0.09) in, respectively. Target response rates
decreased to near zero while control response rates remained near zero
in Phase 2 (Context B). The decrease in target response rates from the
last Phase-1 session to Phase-2 sessions was verified, F(7, 49) = 36.08,
p < 0.0001, with all Phase-2 sessions differing from the last Phase-1
session (p’s < 0.0001).
When placed back to Context A in Phase 3, the right graph in Fig. 3
shows target response rates initially increased and then decreased while
control response rates remained low. The ANOVA indicated that the
change in target response rates from the last session of Phase 2 to Phase
3 was statistically significant, F(5, 35) = 4.71, p = 0.0021. Post hoc
test indicated that the difference between the last Phase-2 session and
the first Phase-3 session was statistically significant, t(35) = 4.10,
p = 0.0002. Differences in other comparisons were not statistically
significant.
In the present experiment, we observed renewal in zebrafish upon
returning to the training context after extinction and its magnitude was
greatest in the first Phase-3 session. The absence of systematic change
in control response rates suggests the increase in target response rates
in Phase 3 was a function of a previous reinforcement history (Phase 1)
and the change in context from Phase 2 (Context B) to Phase 3 (Context
A). Overall, these findings extend others showing sensitivity of zebra-
fish learning to context [53,54] by demonstrating renewal of operant
behavior with this species [32].
5. Experiment 3
The third experiment investigated reinstatement [38,55], which
219
Behavioural Brain Research 335 (2017) 215–222
Fig. 3. Mean responses per min across fish in the last five
sessions of Phase 1 (Acquisition) and all sessions of Phase 2
(Extinction) and Phase 3 (Renewal test) in Experiment 2 (left
graph). Closed symbols and a dashed line without symbols
represent the target and control responses, respectively. Error
bars show ± SEM. The right graph shows target and control
response rates in the last session of Phase 2 and each session
of Phase 3 during renewal tests.
involved reinforcement of a target response in Phase 1 and extinction of the
response in Phase 2. Phase 3 arranged response-independent presentations
of the stimulus that served as the reinforcer for the target response during
Phase 1. As in Experiments 1 and 2, a control response was measured to
dissociate reinstatement from general increases in activity [56].
5.1. Methods
5.1.1. Subjects and material
Twelve experimentally naïve fish (6 males and 6 females) served as
subjects. One male fish failed to acquire responding and its results are
not reported. As in Experiment 2, the device with two sensors and a
single feeder was used. In a feeder test with 100 operations, a mean of
24.2 (SD = 5.58) were delivered per operation.
5.1.2. Procedure
5.1.2.1. Phase 1: acquisition. The target response was reinforced
according to a VI 20-s schedule. The control response was recorded
but had no programmed consequence. Locations of the target and
control sensors, as well as the LED colors associated with each sensor,
were counterbalanced across fish. Sessions ended after 20 min and this
phase lasted for 15 sessions.
5.1.2.2. Phase 2: extinction. The target response was extinguished and
the control response remained unreinforced. Sessions ended after
20 min and this phase lasted for 8 sessions.
5.1.2.3. Phase 3: reinstatement test. Neither target nor control responses
resulted in a reinforcer delivery. However, the reinforcer was delivered
independently of responding twice per min on average, according to a
variable-time 30-s schedule. This rate approximated the obtained rate
of reinforcement in Phase 1. Sessions ended after 20 min and this phase
lasted for five sessions. Reinstatement was defined as an increase in
target response rates compared to target response rates during the last
session of Phase 2, and this increase had to exceed that of control
response rates in a given session.
5.2. Results and discussion
The left graph in Fig. 4 shows mean response rates for target and
control responses in each phase. During acquisition in Phase 1, target
response rates were higher than control response rates. Obtained re-
inforcement rates for target responding, calculated as in Experiment 1,
in each of the last five sessions of Phase 1 were 1.92 (0.08), 1.96 (0.06),
1.80 (0.13), 1.81 (0.10), and 1.90 (0.10), respectively. During extinc-
tion in Phase 2, target response rates decreased to near zero and control
responses remained at this level for all sessions of this phase. The de-
crease in target response rates from the last Phase-1 session to Phase-2
sessions was verified, F(8, 80) = 28.88, p < 0.0001, with all Phase-2
sessions differing from the last Phase-1 session (p’s < 0.0001). When
T. Kuroda et al.
reinforcers were delivered independently of responding during re-
instatement tests in Phase 3, target response rates initially increased
and then decreased but control response rates remained at low levels.
The right graph in Fig. 4 shows response rates in the last session of
Phase 2 and each session of Phase 3. The change in target response rates
from the last session of Phase 2 to Phase 3 was statistically significant, F
(5, 50) = 4.20, p = 0.0029. Post hoc tests indicated that the last Phase-
2 session differed from the first Phase-3 session, t(50) = 0.0003, and
from the second Phase-3 session, t(50) = 0.0074, but not for other
comparisons. Thus, reinstatement occurred in zebrafish and its magni-
tude was greatest in the first two Phase-3 sessions.
The present findings confirmed reinstatement of operant behavior in
zebrafish, thereby extending the generality of reinstatement of operant
behavior [e.g.,39]. The present findings further extend those demon-
strating reinstatement of Pavlovian conditioning in zebrafish [10] and
were the first to show operant reinstatement reliably in any fish species
[cf. 13].
6. General discussion
The present experiments provided the first evidence of relapse of
operant behavior in zebrafish using resurgence, renewal, and re-
instatement models. In each model, we used the apparatus developed
by Manabe et al. [11] to study operant behavior with zebrafish, in
which a discrete approach response served as the operant response to
dispense food reinforcers automatically. Target response rates exceeded
control responses during Phase 1 of all three experiments, indicating
reliable discrimination of the reinforcement contingencies. Extinction
arranged during Phase 2 of all three experiments reduced target re-
sponding to near-zero levels. In Phase 3 of all three experiments, relapse
of target responding exceeded target response rates at the end of Phase
2 and control response rates in Phase 3. In Experiment 1, removing
alternative reinforcement produced resurgence of target responding. In
Experiment 2, returning to the training context (A) after extinguishing
in a novel context (B) produced renewal of target responding. In Ex-
periment 3, presenting the food reinforcer response-independently
produced reinstatement of target responding. These findings are con-
sistent with relapse of previously reinforced and extinguished target
responding assessed across a range of species and experimental condi-
tions [see 15,18,16,57].
Relapse examined in these models reveal learning between re-
sponding and reinforcement established during training does not un-
dergo erasure, or unlearning, during extinction [15]. Instead, decreases
in operant responding during extinction reveal new learning that target
responding no longer produces reinforcers. Specifically, relapse of
target responding in Phase 3 of each of these models indicates the ab-
sence of responding during extinction is a performance deficit – original
learning is preserved throughout extinction across a wide range of
species. Extending these findings to zebrafish demonstrates the robust
generality of these operant learning processes. In the zebrafish, the
return of extinguished behavior could reflect effective foraging strate-
gies through a return to a previously reinforced response when a more
220
Behavioural Brain Research 335 (2017) 215–222
Fig. 4. Mean responses per min across fish in the last five
sessions of Phase 1 (Acquisition) and all sessions of Phase 2
(Extinction) and Phase 3 (Reinstatement test) in Experiment 3
(left graph). Closed symbols and a dashed line without sym-
bols represent the target and control responses, respectively.
Error bars show ± SEM. The right graph shows target and
control response rates in the last session of Phase 2 and each
session of Phase 3 during reinstatement tests.
recent source depletes (resurgence), when returning to a previously
bountiful context (renewal), or when an apparently depleted food
source shows signs of remaining plentiful (reinstatement).
Observing relapse reliably in these laboratory models paves the way
to examine interactions between environmental and biological influ-
ences on learning processes involved in addiction. Thus far, studies of
relapse in zebrafish have been limited to resurgence [29] and to re-
instatement examined in the conditioned-place preference model [10].
In the latter study, training involved adding response-independent
ethanol or nicotine to the water when restricting fish to one side of a
two-sided chamber cued with distinct visual stimuli. After observing
preference for the side of the chamber paired with drug, preference
extinguished to basal levels with repeated unrestrained exposure to
both sides of the conditioning chamber. Finally, exposing fish to drug in
a separate tank before transferring to the testing tank reinstated pre-
ference for the drug-paired side of the chamber. Such findings im-
portantly reveal conservation of effects from mammalian studies of
place conditioning with drugs [58–60].
Nevertheless, all findings using conditioned-place preference are
limited by ambiguity in interpreting the behavioral processes involved
in the conditioned effects [see 60,61]. In particular, the absence of a
clearly defined operant response in the production of drug effects and
the accompanying cues suggest a clearer role for Pavlovian stimulus-
stimulus learning. Conversely, drug use itself might be greatly influ-
enced and motivated by Pavlovian cues [62] but is generally considered
operant behavior involving learning between behavior and drug effects
[3]. Thus, drug self-administration typically has been used to model
drug use and abuse in humans and laboratory animals [63]. Further
exploration of factors influencing relapse of operant behavior with
zebrafish, as in the present study, but using drug self-administration
procedures [cf. 64] with zebrafish would be a useful step in examining
factors involved in drug addiction.
Theories concerned with the environmental causes of relapse of
operant behavior differ in the processes proposed to underlie relapse
but suggest resurgence, renewal, and reinstatement occur due to
common learning processes. Context theory suggests new learning oc-
curring during extinction inhibits operant behavior learned previously
during training [15,65]. Relapse occurs because excitatory learning
from training generalizes to new situations better than inhibitory
learning from extinction, as suggested by renewal occurring when
transitioning to novel contexts during testing (e.g., ABC and AAC).
Further, removal of alternative reinforcement during resurgence pro-
cedures and response-independent reinforcer presentations during re-
instatement are assumed to constitute contextual changes removing
inhibitory extinction learning [15]. Conversely, behavioral momentum
theory is a quantitative model asserting that reinforcers delivered
during training strengthen behavioral mass and extinction serves as a
force disrupting operant behavior [66]. Changing stimulus context with
renewal, removing alternative reinforcers with resurgence, and re-
sponse-independent reinforcer presentations during reinstatement all
remove disruptive forces in place during extinction, resulting in a
temporary increase in the previously reinforced responding. Although
T. Kuroda et al.
the learning processes involved in relapse of operant behavior remain
debated, these theories provide specific approaches to guide future
research examining biological and environmental interactions of re-
lapse in zebrafish.
Acknowledgements
This research was approved by the Research Ethics Committee at
Aichi Bunkyo University, where the experiment was conducted, and
was supported by a grant from the Japan Society for the Promotion of
Science (#16K17366, 2016) to Toshikazu Kuroda. The research was in
accordance with the Act on Welfare and Management of Animals in
Japan. We thank Dr. Manabe Kazuchika at Nihon University for de-
signing the automated feeder. Carlos Cançado was supported by a
postdoctoral fellowship (PNPD-CAPES) at Universidade de Brasília,
Brazil, during the writing of this article.
References
[1] B.F. Skinner, Science and Human Behavior, Free Press, 1953.
[2] B.F. Skinner, Selection by consequences, Science 213 (1981) 501–504, http://dx.
doi.org/10.1126/science.7244649.
[3] G.M. Heyman, Addiction: A disorder of choice, Harvard University Press, 2009.
[4] M.E. Bouton, Why behavior change is difficult to sustain, Prev. Med. 68 (2014)
29–36, http://dx.doi.org/10.1016/j.ypmed.2014.06.010.
[5] J.M. Bailey, A.N. Oliveri, E.D. Levin, Pharmacological analyses of learning and
memory in zebrafish (Danio rerio), Pharmacol., Biochem. Behav. 139 (2015)
103–111, http://dx.doi.org/10.1016/j.pbb.2015.03.006.
[6] R.E. Blaser, D.G. Vira, Experiments on learning in zebrafish (Danio rerio): a pro-
mising model of neurocognitive function, Neurosci. Biobehav. Rev. 42 (2014)
224–231, http://dx.doi.org/10.1016/j.neubiorev.2014.03.003.
[7] A.V. Kalueff, A.M. Stewart, R. Gerlai, Zebrafish as an emerging model for studying
complex brain disorders, Trends Pharmacol. Sci. 35 (2014) 63–75, http://dx.doi.
org/10.1016/j.tips.2013.12.002.
[8] R. Spence, G. Gerlach, C. Lawrence, C. Smith, The behaviour and ecology of the
zebrafish, Danio rerio, Biol. Rev. 83 (2008) 13–34, http://dx.doi.org/10.1111/j.
1469-185X.2007.00030.x.
[9] L.J.M. Kily, Y.C.M. Cowe, O. Hussain, S. Patel, S. McElwaine, F.E. Cotter,
C.H. Brennan, Gene expression changes in a zebrafish model of drug dependency
suggest conservation of neuro-adaptation pathways, J. Exp. Biol. 211 (2008)
1623–1634, http://dx.doi.org/10.1242/jeb.014399.
[10] C.H. Brennan, A. Parmar, L.K.M. Kily, A. Ananthathevan, A. Doshi, S. Patel,
Conditioned place preference models of drug dependence and relapse to drug
seeking: studies with nicotine and ethanol, in: A.V. Kalueff, J.M. Cachat (Eds.),
Zebrafish Models in Neurobehavioral Research, Springer, 2011, pp. 163–179.
[11] K. Manabe, R.J. Dooling, S. Takaku, An automated device for appetitive con-
ditioning in zebrafish (Danio rerio), Zebrafish 10 (2013) 518–523, http://dx.doi.
org/10.1089/zeb.2012.0776.
[12] K. Manabe, R.J. Dooling, S. Takaku, Differential reinforcement of an approach re-
sponse in zebrafish (Danio rerio), Behav. Processes 98 (2013) 106–111, http://dx.
doi.org/10.1016/j.beproc.2013.05.013.
[13] A. Valente, K.H. Huang, R. Portugues, F. Engert, Ontogeny of classical and operant
learning behaviors in zebrafish, Learn. Mem. 19 (2012) 170–177, http://dx.doi.
org/10.1101/lm.025668.112.
[14] S. Bowen, K. Witkiewitz, S.L. Clifasefi, J. Grow, N. Chawla, S.H. Hsu, H.A. Carroll,
E. Harrop, S.E. Collins, M.K. Lustyk, M.E. Larimer, Relative efficacy of mindfulness-
based relapse prevention, standard relapse prevention, and treatment as usual for
substance use disorders: a randomized clinical trial, JAMA Psychiatry 71 (2014)
547–556, http://dx.doi.org/10.1001/jamapsychiatry.2013.4546.
[15] M.E. Bouton, N.E. Winterbauer, T.P. Todd, Relapse processes after the extinction of
instrumental learning: renewal, resurgence, and reacquisition, Behav. Processes 90
(2012) 130–141, http://dx.doi.org/10.1016/j.beproc.2012.03.004.
[16] N.J. Marchant, X. Li, Y. Shaham, Recent developments in animal models of drug
relapse, Curr. Opin. Neurobiol. 23 (2013) 675–683, http://dx.doi.org/10.1016/j.
conb.2013.01.003.
[17] C.A. Podlesnik, M.E. Kelley, Resurgence response competition, stimulus control,
and reinforcer control, J. Exp. Anal. Behav. 102 (2014) 231–240, http://dx.doi.org/
10.1002/jeab.102.
[18] D. Pritchard, M. Hoerger, F.C. Mace, Treatment relapse and behavioral momentum
theory, J. Appl. Behav. Anal. 47 (2014) 814–833, http://dx.doi.org/10.1002/jaba.
163.
[19] A.R. Craig, T.A. Shahan, Behavioral momentum theory fails to account for the ef-
fects of reinforcement rate on resurgence, J. Exp. Anal. Behav. 105 (2016) 375–392,
http://dx.doi.org/10.1002/jeab.207.
[20] C.A. Podlesnik, C. Jimenez-Gomez, T.A. Shahan, Resurgence of alcohol seeking
produced by discontinuing non-drug reinforcement as an animal model of drug
relapse, Behav. Pharmacol. 17 (2006) 369–374, http://dx.doi.org/10.1097/01.fbp.
0000224385.09486.ba.
[21] S.L. Quick, A.D. Pyszczynski, K.A. Colston, T.A. Shahan, Loss of alternative non-
drug reinforcement induces relapse of cocaine-seeking in rats: role of dopamine D1
221
Behavioural Brain Research 335 (2017) 215–222
receptors, Neuropsychopharmacology 36 (2011) 1015–1020, http://dx.doi.org/10.
1038/npp.2010.239.
[22] T.A. Shahan, A.R. Craig, Resurgence as choice, Behav. Processes (2016), http://dx.
doi.org/10.1016/j.beproc.2016.10.006.
[23] H. Leitenberg, R.A. Rawson, K. Bath, Reinforcement of competing behavior during
extinction, Science 169 (1970) 301–303, http://dx.doi.org/10.1126/science.169.
3942.301.
[24] C.R.X. Cançado, K.A. Lattal, Resurgence of temporal patterns of responding, J. Exp.
Anal. Behav. 95 (2011) 271–287, http://dx.doi.org/10.1901/jeab.2011.95-271.
[25] J.Y.H. Bai, S. Cowie, C.A. Podlesnik, Quantitative analysis of local-level resurgence,
Learn. Behav. 45 (2017) 76–88, http://dx.doi.org/10.3758/s13420-016-0242-1.
[26] S.P. da Silva, C.R.X. Cancado, K.A. Lattal, Resurgence in Siamese fighting fish, Betta
splendens, Behav. Processes 103 (2014) 315–319, http://dx.doi.org/10.1016/j.
beproc.2014.01.004.
[27] J.A. Mulick, H. Leitenberg, R.A. Rawson, Alternative response training, differential
reinforcement of other behavior, and extinction in squirrel monkeys (Saimiri
sciureus), J. Exp. Anal. Behav. 25 (1976) 311–320, http://dx.doi.org/10.1901/jeab.
1976.25-311.
[28] T. Kuroda, C.R.X. Cançado, C.A. Podlesnik, Resistance to change and resurgence in
humans engaging in a computer task, Behav. Processes 125 (2016) 1–5, http://dx.
doi.org/10.1016/j.beproc.2016.01.010.
[29] T. Kuroda, Y. Mizutani, C.R.X. Cançado, C.A. Podlesnik, Reversal learning and re-
surgence of operant behavior in zebrafish (Danio rerio), Behav. Processes 142
(2017) 79–83, http://dx.doi.org/10.1016/j.beproc.2017.06.004.
[30] M.E. Bouton, T.P. Todd, D. Vurbic, N.E. Winterbauer, Renewal after the extinction
of free operant behavior, Learn. Behav. 39 (2011) 57–67, http://dx.doi.org/10.
3758/s13420-011-0018-6.
[31] A.L. Willcocks, G.P. McNally, An extinction retrieval cue attenuates renewal but not
reacquisition of alcohol seeking, Behav. Neurosci. 128 (2014) 83–91, http://dx.doi.
org/10.1037/a0035595.
[32] C.A. Podlesnik, M.E. Kelley, C. Jimenez-Gomez, M.E. Bouton, Renewed behavior
produced by context change and its implications for treatment maintenance: a re-
view, J. Appl. Behav. Anal. 50 (2017) 675–697, http://dx.doi.org/10.1002/jaba.
400.
[33] S. Nakajima, S. Tanaka, K. Urshihara, H. Imada, Renewal of extinguished lever-
press responses upon return to the training context, Learn. Motiv. 31 (2000)
416–431, http://dx.doi.org/10.1006/lmot.2000.1064.
[34] M.S. Berry, M.M. Sweeney, A.L. Odum, Effects of baseline reinforcement rate on
operant ABA and ABC renewal, Behav. Processes 108 (2014) 87–93, http://dx.doi.
org/10.1016/j.beproc.2014.09.009.
[35] C.A. Podlesnik, T.A. Shahan, Behavioral momentum and relapse of extinguished
operant responding, Learn. Behav. 37 (2009) 357–364, http://dx.doi.org/10.3758/
LB.37.4.357.
[36] M.E. Kelley, C.J. Liddon, A. Ribeiro, A.E. Greif, C.A. Podlesnik, Basic and transla-
tional evaluation of renewal of operant responding, J. Appl. Behav. Anal. 48 (2015)
390–401.
[37] D. Epstein, K. Preston, J. Stewart, Y. Shaham, Toward a model of drug relapse: an
assessment of the validity of the reinstatement procedure, Psychopharmacology 189
(2006) 1–16.
[38] R.L. Reid, The role of the reinforcer as a stimulus, Br. J. Psychol. 49 (1958)
202–209, http://dx.doi.org/10.1111/j.2044-8295.1958.tb00658.x.
[39] U. Shalev, J.W. Grimm, Y. Shaham, Neurobiology of relapse to heroin and cocaine
seeking: a review, Pharmacol. Rev. 54 (2002) 1–42.
[40] G.J. Franks, K.A. Lattal, Antecedent reinforcement schedule training and operant
response reinstatement in rats, Anim. Learn. Behav. 4 (1976) 374–378, http://dx.
doi.org/10.3758/BF03214424.
[41] A.H. Doughty, P. Reed, K.A. Lattal, Differential reinstatement predicted by pre-
extinction response rate, Psychon. Bull. Rev. 11 (2004) 1118–1123, http://dx.doi.
org/10.3758/BF03196746.
[42] L. Miranda-Dukoski, J. Bensemann, C.A. Podlesnik, Training reinforcement rates,
resistance to extinction, and the role of context in reinstatement, Learn. Behav. 44
(2016) 29–48, http://dx.doi.org/10.3758/s13420-015-0188-8.
[43] C.A. Paronis, M. Gasior, J. Bergman, Effects of cocaine under concurrent fixed ratio
schedules of food and IV drug availability: a novel choice procedure in monkeys,
Psychopharmacology 163 (2002) 283–291, http://dx.doi.org/10.1007/s00213-
002-1180-5.
[44] R. Stretch, G.J. Gerber, S.M. Wood, Factors affecting behavior maintained by re-
sponse-contingent intravenous infusions of amphetamine in squirrel monkeys, Can.
J. Physiol. Pharmacol. 49 (1971) 581–589, http://dx.doi.org/10.1139/y71-075.
[45] T.S. Falcomata, K.J. Hoffman, S. Gainey, C.S. Muething, D.M. Fienup, A preliminary
evaluation of reinstatement of destructive behavior displayed by individuals with
autism, Psychol. Rec. 63 (2013) 453–466, http://dx.doi.org/10.11133/j.tpr.2013.
63.3.004.
[46] J.E. Spradlin, D.L. Fixsen, F.L. Girarbeau, Reinstatement of an operant response by
the delivery of reinforcement during extinction, J. Exp. Child Psychol. 7 (1969)
96–100, http://dx.doi.org/10.1016/0022-0965(69)90088-5.
[47] M. Fleshler, H.S. Hoffman, A progression for generating variable-interval schedules,
J. Exp. Anal. Behav. 5 (1962) 529–530, http://dx.doi.org/10.1901/jeab.1962.5-
529.
[48] R. Epstein, Resurgence of previously reinforced behavior during extinction, Behav.
Anal. Lett. 3 (1983) 391–397.
[49] M.M. Sweeney, T.A. Shahan, Effects of high, low, and thinning rates of alternative
reinforcement on response elimination and resurgence, J. Exp. Anal. Behav. 100
(2013) 102–116, http://dx.doi.org/10.1002/jeab.26.
[50] D. Arthur, E.D. Levin, Spatial and non-spatial visual discrimination learning in
zebrafish (Danio rerio), Anim. Cogn. 4 (2001) 125–131, http://dx.doi.org/10.1007/
T. Kuroda et al.
s100710100111.
[51] R.M. Colwill, M.P. Raymon, L. Ferreira, H. Escudero, Visual discrimination learning
in zebrafish (Danio rerio), Behav. Processes 70 (2005) 19–31, http://dx.doi.org/10.
1016/j.beproc.2005.03.001.
[52] M.O. Parker, J. Gaviria, A. Haigh, M.E. Millington, V.J. Brown, F.J. Combe,
C.H. Brennan, Discrimination reversal and attentional sets in zebrafish (Danio rerio),
Behav. Brain Res. 232 (2012) 264–268, http://dx.doi.org/10.1016/j.bbr.2012.04.
035.
[53] L. Al-Imari, R. Gerlai, Sight conspecifics as reward in associative learning in zeb-
rafish (Danio rerio), Behav. Brain Res. 189 (2008) 216–219, http://dx.doi.org/10.
1016/j.bbr.2007.12.007.
[54] A. Avdesh, M.T. Martin-Iverson, A. Mondal, C. Mengqi, S. Askraba, N. Morgan,
M. Lardelli, D.M. Groth, G. Verdile, R.N. Martins, Evaluation of color preference in
zebrafish for learning and memory, J. Alzheimer’s Dis. 28 (2012) 459–469, http://
dx.doi.org/10.3233/JAD-2011-110704.
[55] R.A. Rescorla, J.C. Skucy, Effect of response-independent reinforcers during ex-
tinction, J. Comp. Physiol. Psychol. 67 (1969) 381–389, http://dx.doi.org/10.
1037/h0026793.
[56] S.G. Nair, S.M. Gray, U.E. Ghitza, Role of food type in yohimbine- and pellet-
priming-induced reinstatement of food seeking, Physiol. Behav. 88 (2006) 559–566,
http://dx.doi.org/10.1016/j.physbeh.2006.05.014.
[57] C.A. Podlesnik, T.A. Shahan, Extinction, relapse, and behavioral momentum, Behav.
Processes 84 (2010) 400–411, http://dx.doi.org/10.1016/j.beproc.2010.02.001.
[58] T.M. Tzschentke, Measuring reward with the conditioned place preference para-
digm: a comprehensive review of drug effects, recent progress and new issues, Prog.
Neurobiol. 56 (1998) 613–672, http://dx.doi.org/10.1016/S0301-0082(98)
00060-4.
222
Behavioural Brain Research 335 (2017) 215–222
[59] T.M. Tzschentke, Measuring reward with the conditioned place preference (CPP)
paradigm: update of the last decade, Addict. Biol. 12 (2007) 227–462, http://dx.
doi.org/10.1111/j.1369-1600.2007.00070.x.
[60] M.T. Bardo, R.A. Bevins, Conditioned place preference: what does it add to our
preclinical understanding of drug reward? Psychopharmacology 153 (2000) 31–43,
http://dx.doi.org/10.1007/s002130000569.
[61] J.P. Huston, M.A. de Souza Silva, B. Topic, C.P. Müller, What’s conditioned in
conditioned place preference? Trends Pharmacol. Sci. 34 (2013) 162–166, http://
dx.doi.org/10.1016/j.tips.2013.01.004.
[62] K.C. Berridge, T.E. Robinson, Liking, wanting, and the incentive-sensitization
theory of addiction, Am. Psychol. 71 (2016) 670–679, http://dx.doi.org/10.1037/
amp0000059.
[63] L.V. Panlilio, S.R. Goldberg, Self-administration of drugs in animals and humans as
a model and an investigative tool, Addiction 102 (2007) 1863–1870, http://dx.doi.
org/10.1111/j.1360-0443.2007.02011.x.
[64] S. Bretaud, Q. Li, B.L. Lockwood, K. Kobayashi, E. Lin, S. Guo, A choice behavior for
morphine reveals experience-dependent drug preference and underlying neural
substrates in developing larval zebrafish, Neuroscience 146 (2007) 1109–1116,
http://dx.doi.org/10.1016/j.neuroscience.2006.12.073.
[65] M.E. Bouton, T.P. Todd, A fundamental role for context in instrumental learning
and extinction, Behav. Processes 104 (2014) 13–19, http://dx.doi.org/10.1016/j.
beproc.2014.02.012.
[66] J.A. Nevin, A.R. Craig, P.J. Cunningham, C.A. Podlesnik, T.A. Shahan,
M.M. Sweeney, Quantitative models of persistence and relapse from the perspective
of behavioral momentum theory: fits and misfits, Behav. Processes 141 (2017)
92–99, http://dx.doi.org/10.1016/j.beproc.2017.04.016.