Value computation & choice

Dongil Chung, PhD

Department of Biomedical Engineering
UNIST
 Would you play this game?

Reward Effort

Choices

2
 Would you play this game?

• Stimulus value = value of an orange • Action cost = force

you have to put
on a hand grip

Reward Effort

Choices

• Action value = integrating the StimV and AC

or Decision value
3
Each piece of value signal affect
probability of a choice

4
Choices might be made based on the
integrated value signal

YES

or

NO

5
To find out how we make choices,

• Investigate whether or not action value is

represented in the brain!

• Hypothesis 1: Action value indeed exists and

represents integrated signal of stimulus value
and action cost
• Hypothesis 2: Action value is not encoded.
Instead, stimulus value and action cost are
separately used for decision-making

6
To find out how we make choices,

• Investigate whether or not action value is

represented in the brain!

• Hypothesis 1: Action value indeed exists and

represents integrated signal of stimulus value
and action cost
• Hypothesis 2: Action value is not encoded.
Instead, stimulus value and action cost are
separately used for decision-making

7
 How to identify stimulus value signals?

• Easiest method: ASK!

• How much would you like to get this good at the
end of the experiment?
• “Likert scale”

Negative Positive

I must not I don’t It doesn’t So so Very much

get it have to matter
get it

8
How to identify stimulus value signals?

• Becker-DeGroot-Marschack (BDM) auction

• “how much would you be willing to pay to get this
good at the end of the experiment?”

9
BDM auction

• A great method to examine a true willingness-to-pay of

participants

Steps:
• A participant formulates a bid (B)
• A random number generator determines a price (P)
• If the bid (B) is higher than the determined price (P),
the participant pays the price (P) and receives the item
• If the bid is lower than the price, the participant pays
nothing and receives nothing
10
Let’s try some examples. Bid!
(up to $50)

READY?

11
Let’s try some examples. Bid!
(up to $50)

12
BDM results?

13
Let’s try some examples. Bid!
(up to $50)

14
BDM results?

15
We will get to see a few fMRI studies as
examples
• Functional modules in the brain
• We can find a brain region that has a key role in a certain
cognitive (or affective) function (e.g., occipital lobe in visual
information processing)

• Brain is more complex than that!

• It is NOT a one-to-one match between regions and functions
• E.g., Amygdala is not only involved in processing fear, but also
in valuation

• All scientific results must be interpreted with caution!!

16
 Measuring willingness-to-pay

Plassmann et al., 2007, J Neurosci

• Not to eat for 4hr • Stay for 30 min and eat the snack

• $35 participation fee

• $3 extra to purchase a snack (familiar chips or candy bars)
• Bid = $0, $1, $2, or $3
• Keep any money by oneself if not used

17
Measuring willingness-to-pay

• Bidding amount is
already determined
in forced bid trials
• Bidding computation
is not performed in
forced bid trials

• All the other

computations, such
as the anticipated
taste of food is
equally performed

Plassmann et al., 2007, J Neurosci

18
 Parametric modulation

• Assuming that size of BOLD responses in the brain reflect the size of stimuli

Neural response

$5 $10 $15 Stimuli

Parametric modulation

19
 BOLD responses in the vmPFC and
dlPFC were correlated with the bids

Medial orbitofrontal
cortex (mOFC) or
ventromedial
prefrontal cortex
(vmPFC) Free bid modulated by bid
– Forced bid modulated by bid

Dorsolateral
prefrontal cortex
(dlPFC)

Plassmann et al., 2007, J Neurosci

20
 Stimuli value can be modulated as a
function of its delivered time
Kable & Glimcher, 2007, Nature Neurosci
• “Intertemporal choice”
or “delay discounting task”
• Immediate fixed reward
($20) vs delayed reward

$20 $40
vs
Now 30 days

$35
10 days
• Participants were
…

6 different delays ~ 6h-180 days responding whether to

6 different amounts ~ $20.25-$110 take an immediate or a
delayed reward
36 pairs x 4 times = 144 trials
21
Delayed reward is less preferred =
delayed reward is discounted

$40 $20 SV = (objective value)/ [1 + kD]

=
30 days Now
D: delays
k: discount rate
Subjective value

When k = 0: no discounting
$40 Larger k: larger discounting
= Larger valuation for the
immediate reward
$20
= impulsive
Delays
30
days

22
 Individual differences in discount rates

Use k to calculate individuals’

subjective values

Posterior cingulate
cortex

Ventromedial
prefrontal cortex Ventral
striatum

Kable & Glimcher, 2007, Nature Neurosci

23
Example

SV = (Obj V)/ [1 + kD]

$40 $?
=
30 days Now D: delays
k: discount rate

Adam has a discount rate k = 0. 1

Fill in the question mark.

? = 40 x 1/ [1 + 0.1 x 30] = 40 x ¼ = 10

$40 $10
For Adam: =
30 days Now

24
 Stimuli value with gains and losses

• 50/50 gambles involving a gain and a loss

• Gain: $10-$40 (in $2 increments)
+12 –14
• Loss: $5-$20 (in $1 increments)
• Question: will you accept or reject the
gamble?
Tom et al., 2007, Science

+12 –14 vs 0

25
Size of potential loss or gain are
tracked in a network of brain regions

• Parametric modulation analyses • Conjunction

vlPFC vStr

vmPFC
vStr vStr

vmPFC vmPFC

Loss
Tom et al., 2007, Science
BOLD

BOLD

26
Gain
Size of potential loss or gain are
tracked in a network of brain regions

• Parametric modulation analyses • Conjunction

• Slope is the neural sensitivity (how the brain tracks gains or losses)
• Blue is steeper than Red
= Neural loss aversion (higher sensitivity to loss)

• This pattern was found in several regions, including bilateral

ventral striatum, bilateral lateral and superior PFC, right inferior
parietal cortex

Loss
BOLD

BOLD

27
Gain
• Neural loss aversion (higher sensitivity to loss)

• This pattern was found in several regions, including bilateral

ventral striatum, bilateral lateral and superior PFC, right inferior
parietal cortex

28
 Neural and behavioral loss aversion
show a consistent pattern

ßgain : Neural loss aversion

ßloss

• If v > 0, U(v) = v rho

• If v < 0, U(v) = – λ (–v) rho

λ: Behavioral loss aversion

measured from ventral striatum

Tom et al., 2007, Science

29
30