1

RESCORLA-WAGNER MODEL



What are some characteristics of a good model?

1. Variables well-described and manipulatable.
2. Accounts for known results and able to predict non-trivial results of new experiments.
3. Dependent variable(s) predicted in at least relative magnitude and direction.
4. Parsimonious (i.e., minimum assumptions for maximum effectiveness).

The principal theoretical variable in the model is called associative strength, usually
denoted by the capital letter V. This variable is not directly observable or measurable but
is presumably related to a measured variable, for example, the magnitude of a conditional
response after some number of conditioning trials. This is a common feature of models,
especially mathematical models in psychology, where the variables and other parameters
entering into the model represent some hypothetical or intervening constructs putatively
related to specific observed and measured events. Moreover, almost all models in
psychology are only relatively predictive, that is, they might be able to predict that an
outcome of one experiment will be bigger or smaller than that of another experiment, but
not how big or small or what the actual obtained values would be. Models that can make
precise quantitative predictions could be called parameter predictive. Some models in
physics, for example, can predict experimental values to better than 15 decimal places!
The Rescorla-Wagner model of Pavlovian conditioning is, at best, only relatively
predictive. Yet, it has been remarkably successful in its domain in spite of (or because
of) the fact that it is probably the simplest possible plausible model of conditioning.


Assumptions of the Rescorla-Wagner model:

1. When a CS is presented its associative strength, Vcs, may increase (CS+), decrease
(CS-), or remain unchanged.

2. The asymptotic strength () of association depends on the magnitude (I) of the UCS:
= f (UCS
I
).

3. A given UCS can support only a certain level of associative strength, .

4. In a stimulus compound, the total associative strength is the algebraic sum of the
associative strength of the components. [ex. T: tone, L: light. V
T+L
= V
T
+ V
L
]

5. The change in associative strength, V, on any trial is proportional to the difference
between the present associative strength, Vcs, and the asymptotic associative strength,
.
2

The last assumption yields the basic model:

1
1
; 01.
n
ni
i
VKVK

=
||
=<<
|
\.



This equation says that the change in associative strength on trial n (Vn) is proportional
to the difference between the asymptotic associative strength () and the cumulative
associative strength on trial n 1. The summation expression is sometimes simply called
Vsum.

Equations of this form are called difference equations. They are the discrete counterparts
of differential equations with which most of you are at least somewhat familiar. If
associative strength were treated as a continuous variable, the model would have been a
differential equation. (What would it have looked like and what would have been a
solution?)

Notice that Vi is the cumulative associative strength over trials. This value is
sometimes called Vsum. The constant of proportionality, K, called the learning rate
parameter is actually comprised of two constants, one related to properties of the CS and
the other to properties of the UCS, multiplied together. In the following examples, we
will only concern ourselves with changes in saliency in the CS, but keep in mind the
assumption about the relation between the magnitude of the UCS and the asymptote .
To keep things simple, think of K as related to the CS and related to the UCSthe
more intense the CS, the bigger K is; the more intense the UCS is, the bigger is. An
important aspect of this model is that the amount of change in behavior that can take
place is proportional to the discrepancy between the current behavior supported by the
contingency and some maintenance or asymptotic level that could be attained. That level
should not be seen as some impenetrable ceiling, but a function of the prevailing
contingencies as well as initial and boundary conditions.

Now, work very carefully through the following examples to see how the model works.
Plot the results to get a better feel for what is happening. To learn about a model you
must play with it.


Example 1: Acquisition

Suppose we pair a light L with a mild shock and record changes, say, in the galvanic skin
response. So, L => SH. Assume that K = 0.2, = 100, and on trial 1 Vsum = 0. How
will conditioning proceed?

Trial 1: V1 = 0.2 (100 - 0) = 20
3
Trial 2: V2 = 0.2 (100 - 20) = 16
Trial 3: V3 = 0.2 (100 - 36) = 12.8
Trial 4: V4 = 0.2 (100 - 48.8) = 10.24, etc.

Just after Trail 4, V4(sum) = 59.04, a bit more than half-way up to the asymptote of 100.

What would V10 (i.e., Vsum on the 10
th
trial) be? Is there a short-cut way to find out?
Show that, in general,

1
1
1
1
1 and that
(1)1(1).
n
n
n
in
n
i
V
K
V
VKKK
+

(
( ==
(

Using the last expression we find that V10 = 89.26.


Example 2: Extinction

Work through this on your own, starting with the results for V4 and V10 obtained above.
What will be now?


Example 3: Overshadowing
With stimulus compounds you have to be careful to work with each stimulus separately
on each trial and then add the associative strengths together (Assumption 4) to get the
Vsum on that trial.
Suppose you started with two potential CSs, a tone, T and light, L where the L > T, that
is, the light is significantly more salient than the tone. This difference would be captured
by differences in the Ks. Let K
L
= 0.5 and K
T
= 0.2. Let = 100.
Trial 1: V
L
= 0.5 (100 - 0) = 50 V
L
= 0.2 (100 - 0) = 20

V
T + L
= 70

Trial 2: V
L
= 0.5 (100 - 70) = 15 V
T
= 0.2 (100 - 70) = 6

V
T

+ L
= 91

Trial 3: V
L
= 0.5 (100 - 91) = 4.5 V
T
= 0.2 (100 - 91) = 1.8

V
T+L
= 97.3
4

Total V
L
= 69.5 Total V
T
= 27.8

Thus, the light is overshadowing the tone.


Example 4: Blocking

Assume two stimuli L and T are equally salient, that is K
L
= K
T
. We pair L only with
the UCS for three trials. On the fourth trial we combine L with T (L + T) and pair them
with the UCS. Let K = 0.5 and = 100. What will V
L3
be? What will V
L4
and V
T4
be?
Why is this called blocking?


Example 5: Conditioned inhibition

The term inhibition is controversial, but in the context of the Rescorla-Wagner model, a
stimulus that through conditioning comes to have negative associative strength is called a
conditioned inhibitor. How can this come about?
Suppose we have paired a light, L, with shock to develop a conditioned response, CR.
After the CR is clearly manifest, we shift to extinction but combine the light with an
equally salient tone, T. After a number of extinction trials what happens to V
L
? What
about V
T
? What is ? Heres an example:
Let K
L
= K
T
= 0.2 and assume that V
L
starts at 90 (why do we assume that V
L
>>0?).
Initially V
T
= 0 (why?). Also = 0 (why?).

Trial 1: L: V
L
= 0.2 (0 - 90) = -18
T: V
T
= 0.2 (0 - 90) = -18

V
L
= 72 (i.e., 90 - 18); V
T
= -18 (i.e., 0 -18) thus, V
L+T
= 54.

Trial 2: V
L
= 0.2 (0 - 54) = -10.8
V
T
= 0.2 (0 - 54) = -10.8

V
L
= 61.2; V
T
= -28.8 thus, V
L+T
= 32.4.

etc.

What will be the limiting values of V
L
and V
T
? How could you test to see if the tone
were a conditioned inhibitor, that is, what procedures might allow you to functionally
define an inhibitor?

5
There are a number of other examples possible showing the applications and limitations
of the Rescorla-Wagner model. Try to make up some cases yourself. Think about
conditions where Ks or s might be manipulated or where CSs are separately
conditioned and then combined, etc. Finally, how might you use the assumptions of the
model to show that if the P (UCS | CS) = P (UCS | ~CS), then V
CS
= 0?

The Rescorla- Wagner model exemplifies in perhaps the simplest possible way a set of
characteristics common to many other models of learning, some embodying much more
sophisticated mathematical structures, for example, neural networks, genetic algorithms,
and various dynamical programming models. Such models, at a minimum, capture three
essential aspects of behavioral change: (1) some form of selective association; (2) some
form of memory; and (3) some variation of a delta rule or feedback process that drives the
system toward stable or meta-stable equilibriain the terminology of dynamical
systemsto fields of attractors. Can you also see the relations between these
characteristics of learning models and processes involved in natural selection driving
biological evolution?

M. Jackson Marr, Psych. 3031/6016