You are on page 1of 2

doi:10.

1093/jmcb/mjp010 Journal of Molecular Cell Biology (2009), 1, 15 –16 j 15


Published online September 21, 2009

Research Highlight
Do Substantia Nigra Dopaminergic Neurons
Differentiate Between Reward and Punishment?
Michael J. Frank1 and D. James Surmeier2,*
1 Departments of Cognitive & Linguistic Sciences, Psychology, and Psychiatry, Brown University, 190 Thayer Street, Providence, RI 02912-1978, USA
2 Department of Physiology, Northwestern University, 303 E. Chicago Ave., Chicago, IL 60611, USA
* Correspondence to: D. James Surmeier, E-mail: j-surmeier@northwestern.edu

The activity of dopaminergic neurons are thought to be increased by stimuli that predict reward and decreased by stimuli that predict
aversive outcomes. Recent work by Matsumoto and Hikosaka challenges this model by asserting that stimuli associated with either

Downloaded from http://jmcb.oxfordjournals.org/ at National Li brary of Serbia on May 23, 2013


rewarding or aversive outcomes increase the activity of dopaminergic neurons in the substantia nigra pars compacta.

A wealth of evidence over the past two It is widely assumed that all midbrain unexpected. Thus, these cells appear to
decades has implicated the dopaminergic dopaminergic cells, from the ventral teg- encode something akin to salience,
system in encoding ‘reward prediction mental area (VTA) to the substantia nigra perhaps reflecting the absolute value of
errors’ (RPEs). That is, when animals and pars compacta (SNc), convey a common, RPE.
humans experience a reward or an event global RPE signal. In a recent Nature One limitation of the Matsumoto and
that is better than expected, midbrain article, Matsumoto and Hikosaka (2009) Hikosaka study is that the neurons
dopaminergic neurons exhibit phasic have challenged this assumption. They recorded were not histochemically ident-
burst firing, with the magnitude of the trained monkeys with a classical condition- ified. As in most in vivo studies, neurons
firing-rate increase correlated with the ing procedure in which some stimuli were were identified using electrophysiological
degree to which the outcome is better strongly or weakly predictive of a liquid criteria and their phasic excitation to free
than expected. Conversely, events that reward, whereas other stimuli were reward. Although this strategy has been
are worse than expected are accompanied strongly predictive of an aversive airpuff used widely, there are reasons to question
by pauses of dopaminergic firing, with the to the face. They found that one class of whether it is water-tight. For example,
duration of pause correlating with the neurons, located more ventromedially in Ungless et al. (2004) found mesencephalic
degree to which events are worse than the region of the VTA, responded just as neurons excited by aversive stimuli that
expected. These findings, originally predicted by the RPE hypothesis: phasic were non-dopaminergic interneurons, in
reported by Schultz and colleagues firing increases were elicited by the spite of having electrophysiological fea-
(1997) have now been demonstrated reward-predictive stimuli, and phasic tures like dopaminergic neurons. The
across multiple labs and species, including depressions were elicited by the airpuff- problem with this interpretation is that
monkeys, rats and humans. This biphasic predictive stimuli. When the outcomes Matsumoto and Hikosaka did not find a
modulation of dopamine cell activity is themselves were experienced, phasic population of neurons with a ‘normal’
thought to act as a ‘teaching signal’ by spiking was only observed when the (RPE) pattern of response intermingled
modifying synaptic connections in the outcome was more rewarding or less aver- with this novel group of neurons in the
striatum to promote the selection of sive than expected, whereas transient SNc, as would be expected from sampling
actions that produce positive outcomes pauses were seen only when expected of dopaminergic and non-dopaminergic
and to diminish the selection of those rewards were withheld. neurons. A more recent study in rats by
that do not (Wickens et al., 2003; Cohen However, in contrast to the current Brishcoux et al. (2009) using a combi-
and Frank, 2009). Indeed, recent optoge- model, Matsumoto and Hikosaka (2009) nation of identification strategies, has cor-
netic studies have confirmed that phasic, identified presumptive dopaminergic roborated the Matsumoto and Hikosaka
but not tonic, stimulation of dopaminergic neurons in the SNc region that were phasi- finding that there are mesencephalic dopa-
cells induces behavioral conditioning: cally activated by conditioned stimuli minergic neurons activated by aversive
animals spend more time in locations in associated with both rewarding and aver- stimuli, but they are rare, possibly
which they had received such stimulation sive outcomes. This unidirectionality was because of the use of anesthetic. Another
compared with those in which they had also observed during the outcome itself, issue is location. Brischoux et al. provided
not (Tsai et al., 2009). particularly if its presentation was strong evidence that there are neurons

# The Author (2009). Published by Oxford University Press on behalf of Journal of Molecular Cell Biology, IBCB, SIBS, CAS. All rights reserved.
16 j Journal of Molecular Cell Biology Frank and Surmeier

(even if rare) excited by aversive stimuli the conditioned stimulus allowed the might be—thalamostriatal signals, inter-
within the VTA, whereas Matsumoto and monkey to blink and reduce the unplea- neurons—remains to be defined, but
Hikosaka contend based upon less santness of the airpuff. This interpretation the intriguing work by Matsumoto and
precise recording depth measurements helps explain the partial dissociation Hikosaka forces us to ask the question.
that these neurons are largely within the between the responses of dopaminergic
SNc. This could be a species-related neurons to the conditioned and uncondi-
difference. tioned aversive stimuli: there was a much References
If true, this finding necessitates a para- smaller percentage (11%) of neurons Beninger, R.J., Mason, S.T., Phillips, A.G., and
digm shift in thinking how experience excited by unconditioned aversive stimuli Fibiger, H.C. (1980). The use of conditioned sup-
pression to evaluate the nature of neuroleptic-
engages dopaminergic neurons to reshape than by conditioned aversive stimuli
induced avoidance deficits. J. Pharmacol. Exp.
neural circuitry. In the striatal regions regu- (37%) (and a far greater proportion— Ther. 213, 623 –627.
lated by the VTA (ventromedial striatum), 47%—were inhibited by the unconditioned Brischoux, F., Chakraborty, S., Brierley, D.I., and
phasic dopamine release in response to aversive stimuli). Moreover, avoidance of Ungless, M.A. (2009). Phasic excitation of dopa-
positive RPE should preferentially activate an expected negative outcome, or termin- mine neurons in ventral VTA by noxious stimuli.
Proc. Natl. Acad. Sci. U.S.A. 106, 4894 –4899.
low affinity D1 dopamine receptors, promot- ation of an experienced one, is associated
Cohen, M.X., and Frank, M.J. (2009).
ing the response of striatonigral medium with increases in dopamine and activates

Downloaded from http://jmcb.oxfordjournals.org/ at National Li brary of Serbia on May 23, 2013


Neurocomputational models of basal ganglia
spiny neurons to sensorimotor-linked corti- brain areas associated with reward (Kim function in learning, memory and choice. Behav.
cal activity, both by increasing synaptic et al., 2006; Brischoux et al., 2009). Fur- Brain Res. 199, 141–156.
strength and postsynaptic excitability; this thermore, stimuli that are predictive of Kim, H., Shimojo, S., and O’Doherty, J.P. (2006). Is
avoiding an aversive outcome rewarding? Neural
should promote selection of actions associ- this release from aversion may acquire
substrates of avoidance learning in the human
ated with reward. Conversely, transient some positive motivational values in that brain. PLoS Biol. 4, e233.
drops in dopamine release in response to they represent ‘safety signals’—i.e. the Matsumoto, M., and Hikosaka, O. (2009). Two
negative RPE should deactivate tonically knowledge that an aversive outcome can types of dopamine neuron distinctly convey posi-
active, high affinity D2 receptors, promoting be avoided. Accordingly, the ability to tive and negative motivational signals. Nature
459, 837 –841.
the activity of striatopallidal mediums avoid aversive outcomes is dopamine-
Moutoussis, M., Bentall, R.P., Williams, J., and
spiny neurons, both by increasing synaptic dependent (Beninger et al., 1980; Mou- Dayan, P. (2008). A temporal difference
strength and postsynaptic excitability; this toussis et al., 2008). One problem with account of avoidance learning. Network
should promote the suppression of actions this explanation is it is not clear why (Bristol, England) 19, 137 – 160.
associated with aversive outcomes (Shen these stimuli should be interpreted differ- O’Doherty, J., Dayan, P., Schultz, J., Deichmann, R.,
Friston, K., and Dolan, R.J. (2004). Dissociable
et al., 2008; Cohen and Frank, 2009). In ently by SNc and VTA neurons. It is possible
roles of ventral and dorsal striatum in instrumen-
the striatal regions regulated by the SNc that the VTA neurons are primarily con- tal conditioning. Science 304, 452 –454.
(dorsolateral striatum), the response to cerned with Pavlovian states (the CS Schultz, W., Dayan, P., and Montague, P.R. (1997).
positive RPEs should be the same. But the state signals that there is potential for an A neural substrate of prediction and reward.
results of Matsumoto and Hikosaka predict upcoming aversive outcome) which are Science 275, 1593 –1599.
Shen, W., Flajolet, M., Greengard, P., and Surmeier,
that stimuli with a negative RPE should encoded in ventral striatum, whereas SNc
D.J. (2008). Dichotomous dopaminergic control
evoke the same alterations in striatal neurons participate in reinforcing instru- of striatal synaptic plasticity. Science 321,
circuitry—leading to action selection rather mental actions (e.g. blinking) encoded in 848 –851.
than action suppression. dorsolateral striatum that would act to Tsai, H.-C., Zhang, F., Adamantidis, A., Stuber, G.D.,
A possible resolution to this conundrum improve the predicted state (O’Doherty Bonci, A., de Lecea, L., and Deisseroth, K. (2009).
Phasic firing in dopaminergic neurons is suffi-
may lie in the uncertainty about whether et al., 2004).
cient for behavioral conditioning. Science 324,
the conditioned stimulus associated with The alternative explanations aside, the 1080– 1084.
the aversive airpuff in the Matsumoto and study by Matsumoto and Hikosaka clearly Ungless, M.A., Magill, P.J., and Bolam, J.P. (2004).
Hikosaka study had positive or negative challenges our ideas about how dopamine Uniform inhibition of dopamine neurons in the
motivational valence. The most straightfor- signaling is regulating the striatonigral and ventral tegmental area by aversive stimuli.
Science 303, 2040 – 2042.
ward inference about the airpuff-associated striatopallidal circuitry in the dorsolateral
Wickens, J.R., Reynolds, J.N.J., and Hyland, B.I.
conditioned stimulus is that it had a nega- striatum and suggests that there may be (2003). Neural mechanisms of reward-related
tive RPE. But it is possible that it had a other factors in play that we do not motor learning. Curr. Opin. Neurobiol. 13,
positive RPE in this paradigm because understand. What those other factors 685 – 690.