Value computation & choice

Dongil Chung, PhD

Department of Biomedical Engineering
UNIST
 Saliency of stimuli

https://www.youtube.com/watch?v=Hw88MWoqenQ
2
Value and Saliency

• Salience: the quality of being particularly noticeable or important

• Saliency signals are related to motivation, attention, and arousal

Strong no No Yes Strong yes

Value

Saliency
3
A task design to control for saliency
signal from value signal

• 60 food items
• Liking-rating: –2 (not at all) ~ 2 (very much)
• “How much would you like to eat this item at the end of the experiment”

• Not eating for ~4 hours

• Choosing whether or not they want to eat the item at the
end of the experiment
• One randomly selected choice is actually implemented
• Response: Strong No, No, Yes, Strong Yes
Litt et al., 2011, Cerebral Cortex

4
Hypothesizing a separate salience
signal from stimuli value signal
• The pattern we are looking for
from the brain:

SV

SV

5
Hypothesizing a separate salience
signal from stimuli value signal
• The pattern we are looking for
from the brain:

Salience

SV

SV

Saliency

6
 mOFC, rACC, dPCC responses were
correlated with stimuli value signal, rather
than saliency signal

= slope
How well the brain responses follow
Rostral anterior cingulate
cortex (rACC)
Dorsal posterior
cingulate cortex

the “SV” shape

(dPCC)

n.s. =
not significant
Medial orbitofrontal cortex (mOFC)

Looking for
SV

Litt et al., 2011, Cerebral Cortex 7

 mOFC, rACC, dPCC responses were
correlated with stimuli value signal, rather
than saliency signal

= slope
How well the brain responses follow
Rostral anterior cingulate
cortex (rACC)
Dorsal posterior
cingulate cortex

the “SV” shape

(dPCC)

n.s. =
not significant
Medial orbitofrontal cortex (mOFC)

Looking for
SV

Litt et al., 2011, Cerebral Cortex 8

 Responses from dACC, insula, and
fusiform gyrus are correlated with saliency,
rather than stimuli value

Dorsal anterior cingulate Fusiform gyrus

cortex (dACC) Insula

How well the brain responses

follow the “saliency” shape

Looking for
saliency

Litt et al., 2011, Cerebral Cortex 9

 Responses from dACC, insula, and
fusiform gyrus are correlated with saliency,
rather than stimuli value

Dorsal anterior cingulate Fusiform gyrus

cortex (dACC) Insula

How well the brain responses

follow the “saliency” shape

Looking for
saliency

Litt et al., 2011, Cerebral Cortex 10

 Ventral Striatum encodes both value
and saliency signals

Salience
SV

Litt et al., 2011, Cerebral Cortex 11

Separating goal, decision values from
prediction errors
• Goal value: the predicted reward that results from the outcome generated by each
of the actions under consideration

• Decision value: the net value of taking the different actions

• Prediction error: deviations from individuals’ previous reward expectations

Three different types of value-related signals are usually highly correlated!

12
 Measure goal value separately before a
choice task
Hare et al., 2008, J Neurosci

1. 50 bidding trials
2. 300 trials of a
forced-choice
task (Yes/No)

• Not to eat for 4hr • Stay for 30 min and eat the snack
• $60 participation fee
• $5 extra to purchase a snack (familiar chips or candy bars)
• Keep any money by oneself if not used

• Goal value: Bid amount

13
 Providing choice independent cost and
gain/loss separates different value signals
Hare et al., 2008, J Neurosci

• Price:
-3~3

• Action independent
gain/loss:
-3~3

• Goal value = the price participants reported

at the bidding (not presented)

• Decision value = goal value - price + gain/loss

• Decision to make: whether to select the trial or not

14
 Providing choice independent cost and
gain/loss separates different value signals
Hare et al., 2008, J Neurosci

• Price:
-3~3

• Action independent
gain/loss:
-3~3
• Goal value trial
no price; no gain/loss

• Goal value = the price participants reported • Decision value trial

at the bidding (not presented) no goal value; no gain/loss
• Decision value = goal value - price + gain/loss
• Prediction error trial
• Decision to make: whether to select the trial or not no goal value; no price

15
 Decisions are made based on the
combination of value signals
Hare et al., 2008, J Neurosci

• Price: -3~3 • Goal value

• Action independent
gain/loss: -3~2

• Decision value = goal value - price + gain/loss

16
Neural substrates of value and PE
signals
Hare et al., 2008, J Neurosci

Canonical hemodynamic function

17
Neural substrates of value and PE
signals

Decision value Prediction error

Goal value

19
Does the brain encode values of different
classes of goods in the same manner?

• Amount of money

• Nonfood items (“trinkets”; Caltech memorabilia and DVDs)

• Food (sweet and salty snack foods, candy bars and chips)

20
 Task procedure for common value
signals
Chib et al., 2009, J Neurosci

• 20 money
• 20 trinkets
• 20 foods

• Not to eat for 4hr • Stay for 30 min and eat the snack
• $60 participation fee
• $4 extra to purchase each classes of goods ($12 in total)

21
 Choice task after measuring
willingness-to-pay

80% chance

Chib et al., 2009, J Neurosci 22

 Choice task after measuring
willingness-to-pay

80% chance

$4 Food item

Chib et al., 2009, J Neurosci 23

 Choice task after measuring
willingness-to-pay

80% chance

The median bid over all items made in the pre-scanning session

Chib et al., 2009, J Neurosci 24

 Participants WTP explains their object
choice patterns

• Group distributions of WTP • WTP explains participants’

choice pattern

Chib et al., 2009, J Neurosci 25

 “Common Currency”

• Medial orbitofrontal cortex commonly encodes the decision value of multiple

classes of goods

Experiment 1: Experiment 2:
All choices against fixed monetary bid All choices against fixed snack item

Chib et al., 2009, J Neurosci 26

 vmPFC encodes subjective value difference
between attended and unattended stimulus

Lim et al., 2011, J Neurosci

• Not to eat for 4h before the task

• One liking-rating task (60 trials; 5 point scale)
• 14 binary choice task runs (30 trials/run)
• Fixate their eyes on the food marked with the target color frame

27
28