Chapter 9: Operants: The Selection of Behaviour

- An organism’s behaviour consists of a repertoire of responses, each with a different

probability. We can make some parts of the repertoire more likely than others by
reinforcing some and not reinforcing others.
- Shaping - gives us away to generate new responses. Shaping will also help us to see how
reinforcement can create classes of responses. These classes are our units of behaviour
which we call, operants.
- Reinforcement can’t work on a response that is never emitted. Instead of waiting for the
response, we can generate it by successively reinforcing others that more and more
closely approximate it.

Shaping: Differential Reinforcement of Successive Approximations

- An experienced experimenter can shape a pigeon’s key peck with just 10 or 15 reinforcer
deliveries.
- Shaping usually compromises between express of request and infrequent deliveries of
reinforcers. Frequent leads to quicker satiation and may overly strengthen some early
responses that lager wont’ be hand, infrequent delivery may reduce responding in general
and once the organism becomes inactive the progress in shaping up to that point may be
lost.
- A shaper who can work close to the limits of extinction, thereby not reinforcing lots of
responses now that will have to be extinguished later when the distribution of responses if
close to the target response, will probably finish shaping more quickly than one who is
generous with reinforcers, but risks satiation before reaching the target.
- It is crucial to be sensitive to the moment to moment interactions of the organism’s
behaviour with the reinforcers you deliver.
- Shaping can be put to either good or bad and many uses it without even knowing they are
doing so.
- Shaping is based upon differential reinforcement. At successive stages, some responses
are reinforced, and others aren’t. The criteria for differential reinforcement change as
responding changes in successive approximations to the response to be shaped. The
property of behaviour that makes shaping effective is that behaviour is variable. No two
responses are the same and reinforcement of one response produces a spectrum of
responses, each differing from the reinforced response along such dimensions of
topography (force, magnitude, and direction).
- There is a paradox to shaping. Reinforcement is said to raise the probability of the
reinforced response but no response I ever repeated exactly.
- We can’t deal with single responses; we must deal with classes of responses.

Natural and Artificial Selection in Shaping

- Shaping is a variety of selection that provides an ontogenic parallel to the phylogenic
selection that occurs in biological evolution.
- Teaching the skills to a seeing eye dog – artificial selection
- Shaping can also occur as a result of natural contingencies
- Evolution is then not a matter of “waiting” for a new mutation, but the preferential
increase or decrease of alternative forms in response to changes in conditions.
 - Shaping typically involves quantitative changes along one or more dimensions of an
organism’s behaviour, but sometimes it appears to produce qualitative changes.

- In phylogenic evolution, the latter changes are sometimes called saltations.

- One dimension of behaviour that may be shaped is variability itself. When behaviour is
more variable, it provides more variants that may be reinforced and therefore may make
shaping quicker and easier.

- One more dimension of shaping is illustrated by the rat and the counterweighted lever.
Ordinarily in shaping, you watch current behaviour and deliver reinforcers based on
when you see.
o In shaping procedures that set thresholds, your judgements are not which
responses to reinforcer but rather how quickly you can move the threshold up.
o If you move up too quickly and responding stops, you might have to drop the
threshold down pretty far to being back responding before you can start moving it
up again.

Shaping as Signal Detection

- Shaping is a skill you might acquire through instruction or through the nonverbal
contingencies of actual practice.
- Signal detection theory assumes that an observer responds or doesn’t respond to a
stimulus that consists of either a signal in noise or noise alone.
- Responding to the signal is a hit but responding to the noise alone is a false alarm, not
responding to the signal is a miss, but not responding to the noise alone is a correct
rejection.
- TABLE 9-1 – signal detection contingencies in shaping pg. 116.
- Example – if a false alarm matters more than a miss, you’ll be biased toward not
reinforcing rather than toward reinforcing – this might make you more likely to lose the
behaviour to extinction.
-

Shaping with Pets and with People

- Animals used in combat and espionage and police work have a long history.
- The evolved or intuitive skills of existing trainers have gradually been integrated with the
methods of behaviour analysts to create effective new techniques.
- Skinners discovery of shaping was a by-product of World War II etc.
- Clicker training has become an increasingly favored method for training pets.

Differentiation and Induction

- Responses are instances of behaviour, so each can occur only once; responses can have
common properties, but they can’t be identical in all respects.
- We can’t group all responses together without distinction because we would be left with
nothing to speak about but behaviour in general.
- We speak of classes of responses defined by common properties.
Response Classes
- Defining response classes in terms of common environmental effects is the basis for both
recording responses and arranging their consequences.
- Operant class – an operant class is a class of responses affected by the way in which it
operates on or works on the environment.
- An operant is a response class modifiable by the consequences of the responses in it.
- The behavioural properties of operant classes are based on the operation called
differential reinforcement, the reinforcement only of responses that fall within a specified
class. This operation makes subsequent responding conform more and more closely to the
defining properties of the class.
- The essential feature of an operant is the correspondence between a class of responses
defined by its consequences and the spectrum of responses generated by these
consequences.

Some Examples of Differential Reinforcement

- Induction – the spread of the effect of reinforcement to other responses not included in
the reinforced class (response generalization).
- When responding stays within the boundaries of the reinforced class we speak of
differentiation, when it calls outside, we speak of induction. Nevertheless, the same
procedure generates responding outside as well as inside
- A fundamental dimension of any class of reinforced responses is the degree of
correspondence between (1) the behaviour that is reinforced and (2) the behaviour
generated by reinforcement.
- The behaviour that is reinforced is sometimes called the descriptive or nominal class; the
behaviour generated by reinforcement is sometimes called the functional class.
- Operants must be defined in terms of the relation between (1), the environment (the
consequences it arranges for responses) and (2) behaviour (the responding produced by
these consequences).
- Reinforcement inevitably involves differentiation.

Function Versus Topography

- We’ve seen how classes of behaviour can be created through differential reinforcement. It
might seem at first that these classes are defined by their topographical properties (what
they look like). But that is not so. Consider even a superficially simple response like the
lever press. A rat might press the lever with its left paw or with its right paw or with both
paws. It might also push the lever down with its chin or jump on it or sit on it. These
responses look very different, but they all count as lever presses; they’re all members of
the same operant class. Despite their differences, they’re members of that class because
they share a common function; They all produce the same consequences.
- In other words: operants are defined by their function and not by their form.
- E.g. self-injurious behaviour of three children – looks similar but the function is different
(i.e. escape/ attention/ sensory etc.)
o The analysis of their behaviour would recommend different treatment programs
for each child.
 o The point is, it is more important to define behaviour classes by their
consequences than by their topographies.
- We can’t define response classes by what they look like – we must define a treatment
program based on the function
o E.g. in this case, we must define a treatment program that uses attention to
reinforce more effective and appropriate behaviour.
- The self-injurious behaviour was one class of behaviour embedded in the larger class of
attention-getting behaviour
- The larger class was held together by the common consequences of its members
- Common consequences are the glue that holds classes of behaviour together.
- When a class of responses seems insensitive to its consequences, as when the first child’s
self-injurious behaviour seemed not to extinguish, we must entertain the possibility that
we’ve improperly defined the class, and that it is part of a larger class
- The issue of function versus form is relevant at many levels. The DSM has long
emphasized form over function – a great cost to patients and to our society, many
behavioural disorders continue to be defined in terms of what they look like rather than in
terms of their function and many continue to be characterized in terms of the disease
models rather than in terms of the contingencies that may have emerged them.

Study Guide Chapter 9

Chapter 9

1) In your own words describe how shaping is used to establish new behaviors.
Describe in your own words the key terms used in the process (i.e., differential
reinforcement and successive approximation).
a. Differential reinforcement – reinforcing the behavior you want to have and
not reinforcing the behavior that you don’t want.
b. Shaping is based upon differential reinforcement. At successive stages, some
responses are reinforced, and others aren’t
c. The criteria for differential reinforcement change as responding changes, in
successive approximations to the response being shaped.

2) In your own words describe how the characteristics of an organism under

study influence the shaping process.
a. What can be shaped has limits and depend on anatomies and other
physiological constraints (don’t both trying to shape flying in either rats or
elephants.
b. E.g. impossible to do big shaping with snakes such as pythons – animals that
only eat one swallow meals spaced days – few opportunities for delivering
reinforcement.
 3) Define and provide examples of a response class. What is the central feature
which determined an operant class? Provide an example to illustrate the
central feature.
a. Defining response classes in terms of common environmental effects is the
basis for both recording responses and arranging their consequences.
b. OPERANT class – a class of responses affected by the way in which it operates
on or works on the environment
i. An operant is a response class modifiable by the consequences of the
responses its in.
c. The essential feature of an operant is the correspondence between a class of
responses defined by its consequences and the spectrum of responses
generated by these consequences.
i.

4) In your own words describe the response parameter. What is the difference
between the function class and descriptive (nominal) class? Provide examples.
a. Distinction between the behavior that is reinforced and the behavior that is
generated by reinforcement
b. DESCRIPTIVE/NOMINAL – THE BEHAVIOUR THAT IS REINFORCED
c. FUNCTIONAL CLASS – THE BEHAVIOUR GENERATED BY REINFORCEMENT

5) According to Catania, it is more important to define behavior classes by their

consequences than their topographies. Why is this the case? Use an example
from your own life to support this argument.
a. OPERANTS ARE DEFINED BY THEIR FUNCTION AND NOT BY THEIR FORM.
b. We’ve seen how classes of behaviour can be created through differential
reinforcement. It might seem at first that these classes are defined by their
topographical properties (what they look like). But that is not so. Consider even a
superficially simple response like the lever press. A rat might press the lever with
its left paw or with its right paw or with both paws. It might also push the lever
down with its chin or jump on it or sit on it. These responses look very different,
but they all count as lever presses; they’re all members of the same operant class.
Despite their differences, they’re members of that class because they share a
common function; They all produce the same consequences.
c. In other words: operants are defined by their function and not by their form.
d. E.g. self-injurious behaviour of three children – looks similar but the function is
different (i.e. escape/ attention/ sensory etc.)

Chapter 10: The Structure of Operants

- What can be shaped has limits and successful shaping sometimes depends on knowing
more than just when to deliver a reinforcer.
 - Responding can vary not only in location or force but also in topography or form,
direction and so on.
- Differential reinforcement can be based on any response dimension, so any dimension or
combination of dimensions might provide the defining properties of an operant class.

Differential Reinforcement of Temporal Organization

- With responses reinforced only if preceded by a minimum time without a response is
called a differential reinforcement of low rate or DRL schedule – sometimes called a
schedule of interresponse time or IRT reinforcement.
- In general, the longer the interresponse time require for reinforcement, the lower the rate
of responding.
- DRL contingencies differentially reinforce a complex operant consisting of an IRT plus a
response, in that order.
- Differential reinforcement of high rate (DRH).
- We must be cautious about taking responses rate as a fundamental measure of the effects
of reinforcement.
- Other classes of differential reinforcement schedules set other temporal criteria
o The differential reinforcement of paced responding sets both upper and lower
limits on the IRTs that can precede reinforced responses and tends to maintain a
fairly constant response rate.
o The differential reinforcement of other behaviour (DRO) delivers a reinforcement
if a specified time elapses without a response. This is in fact the technical name
used to reinforce alternative response during extinction in applied settings.

Response Sequences: Chains versus Chunks

- Once we break down a behaviour sequence into components, we can treat the sequence
as a succession of different operants, each defined by the reinforcing consequence of
producing an opportunity to engage in the next one until the sequence ends with a
reinforcer.
- Such sequences are called response chains.
- a discriminative stimulus that serves such a reinforcing function is sometimes called a
conditional reinforcer.
- Chained sequences are inevitable products of some environments. If I’m on my way up to
my campus office, I must open the door of the building before I can enter, when I get to
the elevator, I must press the elevator button unless the elevator car is already there, I can
only enter when its door opens, once inside I must press the bottom for my floor, I leave
only when the elevator door opens on my floor and so on.
- Each new stimulus allows the next response of the chain, and some of those responses
cannot occur without them: I can’t pass through a door, I haven’t opened yet.
- Some behaviour sequences can be reduced to smaller units in this way, and the analysis
into such components can be confirmed experimentally by checking how independent the
components are from one another.
- Some researchers hold that sequential behaviour could always be interpreted in terms of
an ordering of independent units, others holding that sequential behaviour could not be
adequately interpreted in such terms.
 - Some sequential patterns cannot be reduced to a succession of stimulus-response of S-R
units.
- Some sequences clearly can be put together so that each response produces stimulus
conditions that set the occasion for the next (chaining), whereas others must be integrated
so that responses appear in the proper order without each depending on the consequences,
of the last (chunking).
- For any given sequences, the issue is deciding whether it is made up of chains or chunks
of some combination.

Mediating Behavior
- When we observe sequences, another issue is the function of the responses we see in the
sequence
- When responses make sequential contingencies more likely to be met even though they
are not required by the contingencies, they are sometimes referred to as instances of
mediated generalizations.
- In research settings, mediating behaviour it typically unplanned and its function is
determined only by experimental analysis.
- Natural environments provide many opportunities for establishing such sequences,
especially where situations require waiting.
- If mediating behaviour during a wait extends the time and therefore changes the
likelihood that some response at the end of the wait will be reinforced, that behaviour
may be properly regarded as a product of the contingencies.

Variable Behaviour
- The close correspondence between the class of responses with consequences and the class
of responses generated by these consequences is the criterion for speaking of an operant
class.
- Our major interest is with the dimensions along with responding conforms to the class of
responses that is reinforced.
- the structure of behaviour is such that we can’t always define such dimensions
independently of reinforcement contingencies.
- Example, if a porpoise’s backflips were reinforced in one session, slapping the water with
its tail reinforced in the next and beaching itself at the side of the pool reinforced in the
next one, after several sessions, the porpoise began to emit responses in each new
sessions, such as leaping up from the water with a corkscrew spins, that the
experimenters had never seen before.
o Novel behaviour must be emitted before it can be incorporated into other
behaviour. But the fact that we have difficulty measuring it doesn’t rule out
novelty or other complex dimensions of behaviour as properties that define
operant classes.
- Another dimension that cannot be defined independently of reinforcement contingencies
is the variability of responding, which has also been a target for differential
reinforcement.
- Study showed that reinforcement could in fact maintain highly variable behaviour and
these results led to a research program that demonstrated that variable responding was a
dimension of behaviour that could be reinforced.
 - Novelty and variability can only be properties of responses in the context of other
responses that occurred earlier.

Chapter 10

1) Describe and provide examples of the dimensions in which responses can vary
a. Responding can vary not only in location or force but also in topography or form,
direction etc.
b. Differential reinforcement is based on any response dimension, so any
dimension or combination or dimensions might provide the defining properties
of any operant class.

2) Describe the difference between differential reinforcement of 1) low rates of behavior,

2) high rates of behavior, 3) other behavior. Provide examples of when each of these
schedules might be used in real life.
a. DRL – sometimes called a schedule of interresponse time (IRT) reinforcement –
reinforcement is based on the spacing in time of individual responses rather
than on the average rate generated by many responses over an extended time. In
general, the longer the interresponse time required for reinforcement, the lower
the rate of responding.
i. DRL contingencies differentially reinforce a complex operant consisting
of an IRT plus a response, in that order.
b. DRH – this schedule has received less research attention than the DRL schedule,
mostly because it is harder to work with DRH than DRL.
c. DRO – differential reinforcement of other behavior – reinforce when the problem
behaviors it NOT occurring e.g. when John does not take vitamins in the
morning.
i. Delivers a reinforcer if a specified time elapses without a response. This
is fact the technical name for a procedure often use to reinforce
alternative responses during extinctions in applied settings.

The significance of these examples is that we must be cautious about taking response rate as a
fundamental measure of the effects of reinforcement.

3) Describe the important characteristics of a behavior chain. Provide an example of a

sequence of behavior which does and does not constitute a behavior chain.
a. Important characteristics of a behavior chain
 Behavior chain – e.g. going up the stairs to your office every morning. –
can’t really change the order of the sequence – the order is determined by
the environment
o Not every sequence can be broken down in smaller units – are chains the only
way to put behavior sequences together? – code digit lock on door.
o Temporally extended units – Catania name for sequences that aren’t exactly
response chains – or a chunk
o Sequences clearly built up as successive links in which each response produces a
discriminative stimulus that occasions the next response will be called chains
 o Those consisting of integrated components the links of which cannot be broken
down in that way will be called chunks
o Sometimes chunks will be embedded within chains, as when opening a
combination lock is one integration link within a chain.
o For any given sequence, the issue is deciding whether it is made up of chains or
chunks or some combination

LECTURE
- Learned helplessness
o When people know they can control their environment they do much better.
o When people perceive they have the control of the situation they do much better
overall
o Offering choice opportunities – perform better – much less problem behaviours

- Shaping – AKA- successive approximations

- Response class – group of responses
- Shaping process – several behaviors by step
- Terminal behaviour – final behaviour
- Gradual teaching until it meets the terminal target behaviour
o Each step is a part of a response class.

- Shaping – systematically and differentially reinforcing successive approximations to a

terminal behaviour

- DR – differential reinforcement – reinforcement is provided for responses that have a

certain predetermined quality
- Reinforcement is withheld for responses that don’t have that quality (extinction)
- Provide reinforcement to shape the initial behaviours
- Once previous steps have met the criteria you no longer reinforce those behaviours
- They are going into extinction because you are shaping the next step

- DR effects – response similar to those that have been reinforced occur more frequently.
Responses similar to those that have NOT been reinforced occur less frequently
(Extinction)

- Extinction typically leads to new behaviours

Dimensions of behaviour
- Topography – form of the behaviour
- Frequently
- Latency
- Duration
- Amplitude (magnitude)
Successive Approximations – usually when shaping the child must have some prereq in
repertoire

Shaping across and within

- Across response topography – topography of behaviour change but behaviours are
still members of the same response class e.g. doing laundry
- Within response topographies – topography of behaviours remains constant –
another measurable dimension of behaviour is changed e.g. duration of the
behaviour (practice piano from 5m to 10m to 20m and so on – until the terminal
behaviour of 1 hr)

Generalizations
- Behaviours not being reinforced are generalizations when they continue without
reinforcement

Descriptive/nominal – behaviour that is reinforced – the behaviour has been taught

- E.g. Calling your dad – daddy is learned through reinforcement
- E.g. a child sees a German Shepard and says dog – mom reinforces but praising son

Functional class – response generalization or induction – behaviour generated by reinforcement.

It is learned but not through direct contact with reinforcement – generalization – induction
behaviours are learned without direct contact with R+
- E.g. 1. Now calling everybody daddy who is a man is a behaviour generated by
reinforcement but there has been no direct contact with reinforcement – an undesirable
generalization (“overgeneralization”)
- E.g. 2 – the child sees a dalmatian, lab, retriever and says dog, and the child is also
praised – generalization

Response class vs. operant class

- RESPONSE CLASS
o E.g. a child may point to milk, say milk, sign milk etc. – these behaviours are all
in the response class – they all have the same effect of getting milk
- If in this response class only some behaviours will result in getting milk and some with
not, then you may not classify in an operant class – they are a response class
o E.g. – say someone does not recognize the sign language for signing milk, so they
will not be able to get milk – that behaviour does not belong to the operant class
- OPERANT CLASS – is defined by consequence – if one or more behaviours doesn’t
involve getting milk it cannot belong to the operant class but still can to the response
class
- Response class has nothing to do with the consequence – it is broader
- Members in an operant class have the same consequences

BEHAVIOUR CHAIN
- Sequence of responses in which the result of each response (except for the last) is a
conditioned reinforcer for that response
- E.g. putting on pants – completion of step 1 is an SD for the next step
- 3 types of chaining
o Forward chaining
o Total task chaining
 Teacher prompts when necessary
 Continue teaching until mastered all steps
o Backward chaining
 Puzzle example