Chapter 3

Receptive fields
of a sample of
Given that cells with all the receptive field types
simple cells with shown in 3.6-3.7, each for a full range of orienta-
differing preferred tions, are needed for each of the myriad retinal
orientations but patches, one can see that the striate cortex is a
all receiving their
inputs from the very elaborate biological image processing device
same patch of indeed. We will examine its structute and function
retinal receptors in Ch 9.
A patch
of retinal
receptors Are Simple Cells Feature Detectors?

If you look at the optimal stimuli shown in 3.6-

3.7 then you can easily see why these cells have
often been dubbedfoature detectors. That is, it has
3.10 Many simple cells analyze each patch of retina commonly been assumed that if a cell has a specific
Beware: this is a highly schematic diagram-it does not
edge feature as its optimal stimulus then that cell
show the cells intervening between the receptors in the
retina and the simple cells in the brain. The properties of must be a template recognition mechanism for
these intervening cells are described in Ch 6. signaling the presence of an edge feature in a given
position on the retina.
But computer experiments that make use of
by different cells. That is, any given patch of retina
simple cell models to perform image processing
is "looked at" by a whole range of cells, each cell
have shown this simple-minded "feature detector"
dealing with one or other orientation. This is illus-
idea must be wrong. The reason is that the output
trated in 3.10 for cells with "bar detecting" fields.
of any given simple cell is far too ambiguous to be
The angle to which each cell is tuned is determined
regarded as a reliable feature detector on its own.
by the pattern of its excitatory and inhibitory
For example, consider a cell with a preferred
regions. 0
orientation of 90 (vertical) with the tuning curve
"Slit" and "edge" simple cells are also found for
shown in 3.lla. Such a cell would by defini-
a full range of orientations. Cells of those types are
tion respond most strongly when a high contrast
also "looking at" any given patch of the retina and
vertical black edge falls in the appropriate position
hence the associated region of the input image.
in its receptive field. If the contrast of the edge
Thus, a wide range of cell types is found analysing
is reduced, by making the black zone a dark gray
what is present in each patch of retina. These cells
and the white zone a light grey, then the cell still
all share the same receptors and hence work from
responds but less strongly, 3.llb. The response is
the same gray level description. But the fibers go-
lowered because less excitation and more inhibition
ing from the receptors to each cell are different and
is being fed to the cell than before.
the patterns of excitatory and inhibitory connec-
tions are thereby different. Now consider the cell's response to a high con-
trast edge rotated by, say, ±20° or so away from the
Different Cells for Different Places optimal vertical orientation, 3.llc. The cell would
respond just as well to this non-optimally oriented
So far we have considered analyzing the image in
stimulus as it would to the optimally oriented one
just one patch of retina. But the mammalian retina
oflower contrast. If activity in this cell is taken
consists of millions of receptors and each patch of
simply and directly as the neural representation
receptors needs to be analyzed . Convolution is the
of a vertical edge then we would be suscepti-
technical term for applying a given receptive field
ble to some very awkward illusions. We would
type all over an image. We will explain that con-
confuse faint vertical edges with high contrast
cept in detail in Ch 5. For the present we note that
just-off-vertical ones, a quite unsatisfactory state of
a convolution is implemented in biological visual
affairs which fortunately does not arise. The brain
systems by having many cells of the same type,
has mechanisms for solving this problem, and we
each one devoted to its patch of retina.
describe these next.
64
 Seeing with Receptive Fields

Different stimuli Maintained Stimuli on during Maintained a

on same discharge this period discharge Low contrast
receptive field (faint) vertical
a edge stimulus

I 1"1111111111111111111111111111 I
Preferred stimulus:
very brisk response Row of"",
Some -s showing more
inhibition than in a preferred
IJt t /'
Fewer +s showing less orientations Cells with
b excitation than in a The most active very low
cell (firing at, activity (say,
I I 1111111111111111111 II II I say, 20 impulses 0-5 impulses
per second) per second)
Preferred stimulus orientation but
low contrast: weaker response b
than in a High contrast
just-off-vertical
edge stimulus
Balance of excitation and
c / inhibition the same as in b

I I I 11111 II 11111111111111 II I Same row

Not preferred stimulus orientation of cells as
but high contrast: similar above
response to that in b

Active at 20 impulses per second, just as in The most

All responses are from the same simple striate cell active cell (say,
response to the faint edge above, but still much less
which has a vertical preferred stimulus orientation 100 impulses
active than the neighboring cell which is tuned to
prefer the just-off-vertical edge orientation per second)

3.11 Simple cell ambiguity 3.12 Interpreting simple cell responses in context

active one, and register the input feature as having

Many Cells Make Light Work
The general answer to the ambiguity problem just
introduced is to consider each cell's output, not on
its own, but in the context of the activities of other
cells examining the same patch of retina.
For example, suppose mechanisms were present
that identified the most active neuron when assess-
ing the orientation of an input feature . (A better
scheme will be described shortly but this example
is a helpful starting point.) In this case, for the
faint vertical edge in 3.11b, the most active cell
would still be vertically tuned, 3.12a. This cell
would not be firing as briskly as it would have been
if the edge were a high contrast vertical one. Nev-
ertheless, it would still "come out on top" against
the opposition from other, even less active cells.
Hence, the hypothetical pattern decision
mechanism would note that this cell was the most
a vertical orientation.
Now consider responses to a high contrast edge
0
oriented about 20 from vertical, 3.12b. This
non-vertical stimulus stimulates the vertically ori-
ented receptive field just as well as the vertical but
faint edge shown in 3.12a. But in 3.12b the cell
0
tuned to 20 from vertical fires at an even greater
0
rate than the vertically tuned one because 20 from
vertical is its preferred orientation.
Consequently, the mechanism which detects the
most active neuron would register the input feature
not as a vertical one but at its true orientation of
0
20 from vertical.
In this way, by taking advantage of the context
in which any given cell's activity occurs, the brain
need not be fooled by the intrinsically ambiguous
responses of individual simple cells. (If cell outputs
are free of any jitter or noise, the outputs of only

65
Chapter 3
two cells are sufficient to correctly resolve this
ambiguity; Ch 11.)
This orientation-contrast example illustrates
very nicely the idea of regarding each simple cell
as delivering image measurements for interpretation,
and not as an "edge detectors" pure and simple.
They can't serve as feature detectors because their
individual responses are far too ambiguous.
The example of ambiguity just discussed in-
volves two stimulus dimensions, orientation and
contrast. We examine in detail in Ch 11 how to
resolve ambiguities of that sort. We now consider
how to compute the orientation of a stimulus
when its contrast is held constant. Our goal is to
show that there is a much better way of using the
responses of a range of cells with different preferred
orientations than simply to seek the most actjve
one.
Computing Stimulus Orientation
Single cell recordings show that there are simple
cells for only 18-20 different preferred orienta-
tions within each patch of retina. Hence, if cell
responses were interpreted as suggested in 3.12,
with the most active cell signalling the orientation
of the input feature, then we could only see 18-20
different orientations. This implies that we would
be limited to discriminating between lines differing
in orientation only if their orientations differed by
about 10°. That is, the 180° of orientation would
be shared between, say, 18 orientation detectors
with the peaks of their tuning curves (3.13,3.14)
separated by 180°/18 =10°. But our perceprual ca-
pabilities are much better than this: we can manage
discriminations of less than 0.25°. Clearly, there is
a need for some method of interpolation between
neighboring orientation measurements. By this we
mean there must be a way for the brain to estimate
stimulus orientations lying in between the 18 pre-
ferred orientations encoded by simple cells.
For example, compare the two situations illus-
trated schematically in 3.13 and 3.14. The input
feature in 3.13 is vertical and this causes a symmet-
ric distribution of simple cell firing rates with peak
firing shown for the vertically-tuned cell. Either
side of this peak response, simple cells whose pre-
ferred stimuli are bars with orientations of 80° and
100° are shown firing quite briskly-but not as fast
as the vertical ly (90°) tuned cell. Cells with optimal
66
orientations of 70° and 110° also fire, but relatively
weakly, and the responses from cells tuned to 60°
and 120° are very small indeed. For cells with
optimal orientations further away from vertical
than about ±30° the firing rates quickly reduce to
resting discharge levels. This symmetric pattern of
firing rates centered on the vertically tuned cell can
be regarded as a "signature" profile of activity for
the feature representation verticaL bar. It would be
the signature noted by the interpretive mechanisms
whose task it is to decide what the simple cell
measurements convey about the input.
Now study 3.14, which illustrates firing rates
that arise for an input bar whose orientation is 92°.
As you can see, the distribution is slightly skewed,
not symmetric. The vertically tuned cell is still
firing fastest but it has almost been caught up by
the 100° cell, which is now firing well above the
level of the 80° cell. That wasn't the case for the 90°
input. Equally, the 110° cell is firing more briskly
than the 70° cell. This skewed activity profile is the
signature profile for a 92° bar.
This is a much more sophisticated approach
to the interpretation of simple cell measurements
(outputs) than simply finding the most active
neuron and assuming its preferred orientation is
the orientation of the input feature. What we now
have is a system in which simple cells can be said
to sample the stimulus dimension oforientation,
and infer what is going on in the image from the
pattern of simple cell outputs.
One advantage of this scheme for finding edge
orientation is that it is a very neat way of avoiding
more simple cells than are necessary to do the job.
This makes it economical in terms of number of
cells required. Another clever brain trick.
Integrating Channel Outputs: Weighted Means
A channel is defined as a population of cells that
all have the same preferred value of a particular
stimulus property. For example, if all the cells in a
given population have the same preferred orienta-
tion then together they constitute a single orienta-
tion channel. Note that the cells in such a channel
are not necessari ly located close to o ne another.
In fact, they can be distributed widely across the
striate cortex (Ch 9). The examples shown in 3.13
and 3.14 illustrate orientation channels, but show
only one single cell taken from each channel with
 Seeing with Receptive Fields

Same input 90° to each cell

I: ,:
:
. I .I .1 1 I

ltl itt:
! 1l::
I
I :
I
I
I
I
:

:
I
'
I
I
I
I
I
I
I
I
I
I
I
I
j'm
i All I
I
I
I
I
I
I
I
I
I
I
I
: .
•
•
•
•
I
I
I

. ..I...L80......
Tuning of
Cells

Each line in this

graph shows
the response
. ... ... Profile of channel activities to
vertical bar (90°)

to a vertical
(90°) stimulus of
just one of the LOw L-____________ _____________
differently tuned
cells shown above
.... .. ..... . .. .. Row of cells
Receptive fields

3.13 Activities of simple cells to a vertically oriented stimulus

The upper graphs show the tuning functions of cells with preferred orientations from 60° to 120°. The striped bars under each tuning curve
represent the receptive field of the cell: their orientations show the preferred orientations of each cell. Above each tuning curve is shown a
vertical bar as input. Following down the dotted lines from the input bars leads to the firing rate , shown by the thin vertical line under each tun-
ing curve, for each cell to the vertical bar stimulus. The vertically tuned (90°) channel fires most vigorously. Cells with preferred orientations
close to verti ca l also become activated by the vertical input bar though to a lesser extent. The overall pattern of activity is symmetrical around
vertical. (Drawn schematically.)

Same Input 92° to each cell

: A:
:
\ .\ .\ \ \. \. \
A:
itl ' I :

:
I
I
I
I
:
I

:
I
•
•
I

:
I
•
•
I

I
I
I
I
I

:
I
•
•
All l 1j i !
I

:
I
•
•
•

: :
•
' .
•
I

Tuning of
Cells

Profile of channel activities to

Each line in this graph Activity
92° bar
shows the response of level
one of the differently in each cell
tuned cells shown above
to a 92° stimulus- a Low
bar slightly rotated
anti-clockwise from
vertical
.... .. ... .. Row of cells
Receptive fields

3.14 Activities of simple cell to a stimulus oriented at 92°

As in 3.13, the vertically tuned (90°) cell fires most vigorously but to a slightly lesser extent. However, the ce ll tuned to 100° fires much more
strongly than in 3.13, indeed almost as strongly as the 90° cell. Also, the cell tuned to 80° fires less briskly than it did when the stimulus was a
vertical bar (compare 3.14). The overall pattern of activity is thus asymmetrical , being skewed toward the cells tuned to orientations over 90°.
(Drawn schematically to emphasize main points .)

67
Chapter 3
a particular orientation tuning (90°, 80°, 70°, and
so on).
We now explore further the idea of interpret-
ing simple cell outputs by giving an example of
one particularly simple way of doing it. This is to
regard all the activities of the cells as a mass of data
and work out the weighted mean of all these data.
Slcip these details if you prefer and go to the next
section.
To keep things arithmetically straightforward,
let each impulse in a given time interval of, say, 1
second be regarded as one data item. Also, we will
consider computing a weighted mean from just
three cells, with preferred orientations of 80°,90°
and 100°. The basic idea is to let each cell contrib-
ute to the computation according to its firing rate
(output) in comparison with the other cells.
Let's first take the situation in which these cells
are responding to a vertical bar (90°), and let's sup-
pose their firing rates are as shown in the second
column of 3.15. The total number of impulses in
1 second is 35+50+35= 120 impulses per sec-
ond. We now weight the contribution of each cell
taking into account how much each cell is firing
in comparison with the other cells. Thus" we
weight the output of the 80° cell by the fraction
35/120=0 .29, the 90° cell by 501120=0.42 and
the 100° cell by 35/120=0.29. You can think of
this weighting as reflecting how much influence is
to be given to each of the cells in computing the
stimulus orientation they are dealing with.
The final column of the table multiplies the pre-
ferred orientation of each cell by its weighted firing
rate, which sum to give the weighted mean of 90°.
This is exactly as it should be, of course, for a 90°
input- this what we want the feature code to be
saying when this particular symmetrical "signature
tune" is "playing" in the orientation channels.
But now consider 3.16 which shows asymmetri-
cal firing rates in the same three channels for a
stimulus just-off-vertical, 92°. The weighted mean
is now 92°. This output can be regarded as the re-
sult of interpolating between the preferred orienta-
tions of the three orientation channels to find the
orientation of the stimulus. Progress.
This strategy of using weighted outputs from a
set of channels has the advantage that it averages
out the effects of noise in responses. Obviously,
this will be better if more channels are used, for
68
examp le, the full range shown in 3.13 and 3.14.
We say more about the problems of noise in eh 5.
[This simple scheme for calculating a weighted
mean is used as an example to exp lain the basic
idea. However, it would need to be refined in a
practical system, as it collapses arithmetical ly for
any orientation coded as 0°. This problem can be
fixed by expressing angles trigonometrically, but we
will not go into detail here.]
But you might say: the brain doesn't have a cal-
culator for doing arithmetic, so how might it use
its neurons to implement this type of interpolation
calculation? The general answer is: the brain can
"do" arithmetic using the processes of excitation
and inhibition in combi nation.
Suppose the brain did do something along the
lin es of a weighted mean calculation , and then
used just one neuron to encode each discriminable
orientation. Given that we can distinguish orienta-
tions as little as 0.25° apart then that would entail
having a few hundred neurons to encode the orien-
tation of every edge feature in each patch of retina.
Each such neuron would then be said to be a local
code for just one particular orientation.
Alternatively, perhaps the brain doesn't do a
weighted means calculation to decide which one
of a set of neurons should become activated as the
code for a given orientation. Perhaps instead the
patterns of simple cell responses shown in 3.13
and 3.14 are used as a population code for the
feature representation vertical bar present.
After all, our simple weighted mean calculation
has demonstrated that the population of simple
cells taken together has the orientation of the
bar encoded in its activity pattern. Perhaps this
distributed representation is sufficient as it stands
for the uses the brain has for orientation data. If
so, why bother going a step further and malcing
the bar orientation explicit in a local code? This
question raises some fundamental issues in trying
to understand seeing and the brain. We return to
them in detail in later chapters.
Coarsely Tuned Channels Are a Good Idea
We said above that simple cells can be viewed as
channels sampling the stimulus dimension oforienta-
tion. It turns out that the basic principle underly-
ing this SOrt of san1pling scheme applies generally
in vision.
 Seeing with Receptive Fields

Cellfiring One consequence of having a limited

Preferred Orientations number of samples of a stimulus dimension
rates Weighted
Orientation Weighted is that the cells taking each measurement
(impulses per firing rates
of Cells (degrees) need to be broadly tuned. This is why using
sec)
such chann els is called coarse coding. The
80° 35 35/ 120 = 0.29 80°xO.29 = 23.43°
sampling idea would not work for color
90° 50 50/ 120 = 0.42 90° x0.42 = 37.50° vision if each retinal co ne were exquisitely
100° 35 35/120 = 0.29
0
100 xO .29 = 29.17° sensitive to just one wavelength .
The reason is that as soon as a wavelength
Total = 120 Total = 90.0°
appeared that was outside its narrowly tuned
This is the
range it would fall silent. It would literal ly
Weighted Mean
have nothing to say about that wavelength.
3.15 Calculating a weighted mean from the responses Thus, we would either have to be blind to
of three channels to a vertical stimulus (90°) that wavelength, or we wou ld need myriad
sharply tuned cones, each specializing in
just one wavelength. That is unworkable. A
Cellfiring
Preferred Orientations massive number of cones would be needed
rates Weighted
Orientation Weighted for each small patch of the fovea (the central
(impulses per firing rates
of Cells (degrees) region of the retina that mediates highest
sec)
resolution vision). How could this myriad
80° 24 24/118 = 0.20 80°xO.20 = 16.27° be packed in without unacceptable loss of
90° 46 46/1 18 = 0.39 90° xO.39 = 35 .08° spatial resolution for fine details?
100° 48 4811 18 = 0.40 100 xO.40 = 40 .68°
0 Broad tuning is, th en, an im portant
requirement for exploiting the economy
Total= 118 Total = 92.0°
offered by using channels. However, broad
This is the
tuning means that the coarsely coded cells
Weighted Mean
in question cannot serve as a local code for a
3.16 Calculating a weighted mean from the responses feature assertion. As stated above, the output
of three channels to a 92° stimulus of each broadly tuned cell is far too ambigu-
ous to serve that role. Either the activities
in the whole set of chan nels need to be used
Take color vision for example. Have you ever as a population code, or a process of interpretation
had to replace colored inks in a color printer and needs to be performed to create a local code with
wondered: how can just three in ks (cyan, magenta, the required resolution.
and yel low) be sufficient to create al l the colors Whichever of these coding schemes, local or
that you see on the printed page? population, turns out to be used by the brain, it is
The short answer is: evolution has "discovered" clear that we need to break away entirely from the
that having just three types of color receptors idea that simple cells are feature detectors, pure
(often referred to as red, green , and blue cones; Ch and simple. It is this realization wh ich lay behind
17) is a sufficient sampling of the stimul us dimen- our earlier remarks about the inappropriateness of
sion of wavelength of light. That is, just three calling such cells detectors: "slit detectors, "
measurements obtained usi ng red, green, and blue "bar detectors, " and so on.
cones are sufficient to infer all the colors that mat- This concl usion was drawn by Marr and others
ter to us for survival. The three colored inks of a from computer-based image processing experi-
printer are sufficient to trigger these three receptor ments that modelled simple cells. The ambigu-
types appropriately for almost all the colors we can ity in the responses of their models of simple
see. The result is a clever eco nomy in stimulus crea- cells mean t that these cells must be regarded as
tion (the printer) and in stimulus sampling (the image-measuring devices which provide useful data
eye and brain). about features of the input image but either this

69
 Chapter 3

A region where output A region where output changes a lot

changes relatively little as stimulus orientation changes
as stimulus orientation
changes
3.17 How a change in stimulus
orientation is related to the
change in cell response
100 The solid curve depicts the tuning
curve of a cell. The dashed curves
show how the tuning curve slope
u-
Q)
80 changes with stimulus orientation ;
en
en
Q)
these curves therefore depicts the
-'"
.6. 60 sensitivity of the cell to changes
.!!l.- of orientation at points along the
Q)
orientation continuum . Note that,
ro
a::: 40 surprisingly at first sight, the cell
Ol is least sensitive to changes at
c I
·c
i.L 20
I its preferred orientation (here 0°)
where its firing rate is highest.
I
\ I It has similarly poor sensitivity
0 \ I at the extremes (i.e. , +60 and
0

_60°). The cell is most sensitive at

-60 -40 -20 o 20 40 60 around +20° and -20°.
Orientation (degrees)

data needs to be interpreted before a proper feature
description can be asserted, or their responses have
to be treated as a population code.
One method for reducing the effects of noise is
to average responses from many cells. The underly-
ing assumption here is that the noise in any given
cell will be independent of that affecting other
cells. This means that the noise variations will tend
to cancel out when an average is taken.
This is one reason why it may be a good idea
for the brain to consider the responses of entire
populations of cells when attempting to recover
the parameters of the stimulus that caused those
cells to respond. Using averaging to get around
the problem of noise is explored in detail Ch 5 in
connection with the task of edge detection from
noisy images.
Problem of Parameter Resolution
A question that we discuss in Ch 11 is: how few
channels are needed to resolve the ambiguity in
simple cells responses? It turns out that, in princi-
ple, only two, very broadly tuned. However, if we
had only two cells to span the entire range of 180
0 because at the tops the change in response as retinal
then each cell would be very insensitive to most
orientation changes. This is because a large part of
each cell's orientation range would be on regions
of the tuning curve that change very slowly with
changes in input orientation, so that changes in
stimulus orientation would cause very little change
in output, 3.17. The output of such a very broadly
tuned cell changes very little unless retinal line
orientation falls on the flanks of the tuning curve.
These flanks are where the slope of the tuning
curve is greatest, and therefore where the change in
cell output per degree of change in line orientation
is greatest.
One way to ensure that most retinal line ori-
entations coincide with this' sensitive" part of a
tuning curve is to use a large number of cells with
tuning curves which, taken together, tile the space
of all possible orientations to yield adequate resolu-
tion, as in 3.18. This represents another reason for
using many cells to encode orientation, which is
independent of the noise ptoblem above.
A curious side effect of this way of extracting
orientation is that it predicts peaks and troughs
in the system's sensitivity to orientation. A peak
of high sensitivity should be found in the re-
gion where the two response curves are changing
sharply. Troughs should arise in the regions covered
by the top of each cell's response curve. This is
orientation changes is not as great.
This is paradoxical. It would, at first sight, be
natural to expect maximum sensitivity for orienta-
tions falling on the highest point of each cell's tun-
ing curve. But on careful examination, each cell is

70
 Seeing with Receptive Fields

Examp les of regions of peak sensitivity to stimulus orientation changes because these are regions
with steepest changes in channel outputs

100 -I -I -I -
U-
Q) 80
en
u;
Q)

"""
.6.
.e 60
2
ell
a:
OJ 40
.'u:::c" 3.18 Using many simple
20 cells with different
preferred orientations
to "tile" the full stimulus
0 orientation range

Orientation (degrees)

most sensitive to changes in orientation about half
way down from the top.
So it is reasonable to ask: does human vision
show these predicted peaks and troughs in orien-
tation sensitivity? The answer is yes. Regan and
Beverley reported experiments in 1985 on human
orientation discrimination which confirmed the
prediction (see Regan, 2000).
Can Templates Ever Work as Recognizers?
We started this chapter by considering the use
of bar templates to detect stimulus bars. We
discovered that the problem of response ambigu-
ity bedevils their use, but this led us to a general
principle: ambiguities can be resolved by drawing
inferences from the outputs of many templates
(channels).
Even so, simple templates can be made to work
well as pattern recognition devices in some special
number template has a much more complex pat-
tern than the simple bar feature. Moreover, bank
check numbers are made more readily distinguish-
ab le one from another by using specially designed
numerals with lines of different thicknesses to
facilitate recognition.
But that trick alone would not be enough to get
such templates to serve as pattern recognizers. The
crucial added ingredient is using a special check
scanning device which prevents comp lications
arising from large variations in the input images in
terms of the brightness, contrast, shape, size, and
orientation of numerals. This permits a template
recognition system that works well for the task it
tackles.
N. _ _ a.ntPlc

limited contexts. Consider, for example, a bank

check number recognition system, 3.19. One way Cheque No.

that the numbers can be recognized is to build into

the number-recognizing machine a set of tem- lI'ooooa ?II'
plates, one for each numeral. Then the task is be to
note which template best fits the number on the
check being analyzed. 3.19 Bank check number recognition
Using a template in this way is similar to the
template bar detector in 3.4, except that the

71
Chapter 3
However, that task is a very si mple one by the
standards of biological vision systems. They have
to cope with all manner of variations in the way
objects appear in retinal images, variations over
which they have no control. How human vision
copes with some of these variations is an issue that
we will address in later chapters, particularly eh
8, Seeing Objects. But we pursue here the topic of
template recognition a bit further by way of intro-
d ucing some basic facts that illuminate why the
seeing problem is so hard.
Templates and the Combinatorial Explosion
You might be wondering: could a template recog-
nition system be made to work by having a range
of different templates, each tuned to deal with one
or other source of image variation?
The way we coped with the problem of vari-
able image bar orientation illustrates this idea: we
found it a good idea to have 18 or so differently
oriented bar templates, and to use these coarsely
coded measurements of orientation to work out
bar orientation using weighted means. What if this
ap proach was extended for other sources of image
variation, such as color and size?
Each such variable, often called a parameter as
already noted, would need its own set of coarsely
coded templates, each one dealing with a limited
range of the parameter's possible values.
The trouble with this idea is that it immediately
hits a major snag, called the combinatorial explo-
sion. If we need 18 templates for orientation then
for each one of these we will need a suitable range
of templates for size. Let's say for simplicity that
this would also be 18. Hence, 18 x 18 templates
would be needed to deal with al l combinations of
orientation and size.
Now consider adding another variable, such
as contrast, and suppose that too needed 18
templates. This takes us to 18 x 18 x 18 = 5,832
templates for one numeral.
Well, the brain has a lot of neurons so perhaps
that isn't so bad. But things rapidly get worse when
other sources of image variation are brought into
the equation .
Take object position on the retina for example.
Again to keep things simple, let's suppose the
parameter of vertical position needs 18 templates
and similarly for the horizontal position parameter.
72
We now have 18 x 18 x 18 x 18 x 18 = 1,889,568
templates.
And, once again, this is for just one object, for
example just one of the numerals that our bank
check number template recognizer would have to
deal with if it was stripped of careful control of
variations in the input image.
Imagine needing this huge number of tem-
plates for all the different objects that we so readily
recognize-numerals, letters, birds, trees, chairs,
people, and so on.
There is a general formula for working out the
number of combinations of parameters involved
in this combinatorial explosion: the total number
equals N where N is the number of templates per
parameter (18 in our example), and the exponent
k is the number of parameters. Don't worry if
this exponential formula seems a bit opaque: if
you want to know more about it then read e h 11
where we discuss its implications at length .
The combinatorial explosio n reveals just how
hard the problem vision is. Any attempt to solve it
using simple-minded templates doesn't work: the
brain just doesn't have enough neurons.
Binding Problem
You might think at this point this is si lly; surely
there is no need to have a cell for every combina-
tion of parameter values? Why not simply have one
population of cells that encodes only one param-
eter exclusively, for all objects.
For example, one population could encode only
color, and another could encode only orientation .
Using this scheme we would require k popula-
tions to encode k parameters. If each population
consisted of one million cells then no more than k
million cells would be required in total. The brain
may do something along these lines (as we will
see), but this raises another fundamental vision
problem: how do we attach parameter values to
objects? This is known as the bindingproblem and
we discuss it in eh 11.
Back to the Jumping Spider
We can now see that a spider would have a very
hard time using templates as a means of deciding
the questio n: is the object over there mate or prey?
Such a spider would also be subject to the combi-
natorial explosion (further details in eh 11).

However, it may be that the jumping spider has,
as suggested earlier, evolved some special-purpose
visual mechanisms that are quite unlike our own.
Land has suggested that these spiders use scanning
movements of their boomerang-shaped retinae to
align them with the orientations of leg-like bars in
the input image. This trick might allow the spider
to avoid the 18 or so different orien tation tuned
channels that monkeys and humans seem to pos-
sess.
These retinal scanning movements might also
solve the problem of variable object position in the
image. They would do that if they ensured that the
object's image falls on exactly the right spot for the
spider's limited number of templates to be able to
recognize Mate or Prey. Perhaps these and other
special-purpose adaptations give the spider a work-
ing template-based recognition system.
The general idea here is that perhaps the spider
has not evolved a general-purpose vision system,
such as our own, but one specialized for its particu-
lar ecological niche. It can survive if it can capture
insect prey and find mates. Perhaps it has a visual
system set up to do those tasks and very little else.
In this respect it may be a bit like the simple-mind-
ed but highly specialised bank check recognition
system described above.
Concluding Remarks
This chapter has explored the task of building a bar
detector using templates to discover what problems
have to be overcome. AJong the way we defined
some essential technical terms, many to do with
basic facts about the "hardware" of biological visual
systems. A key concept has been that of a receptive
fieLd with excitatory and inhibitory regions.
Linked to this is the idea of receptive fields of
various types, organised as channels analysing each
patch of retina and providing measurements about
a stimuLus dimension (such as orientation) from
which the brain can work out which features are
present in the input image. In the next chapter, Ch
4, we use these ideas in showing how certain illu-
sions called aftereffects have revealed orientation
channels in the human visual system without need
for invasive single cell recordings.
This chapter has also introduced a fundamental
concept in vision research: the combinatoriaL expLo-
73
Seeing with Receptive Fields
sion arising from the exponential IV" formula. We
examine that problem in considerable depth in
Ch 11 to explain in more detail why vision is such
a hard problem.
Finally, armed with core ideas introduced in this
chapter, we are ready to have a much closer look
in Ch 5 at the task of edge detection. We consider
there and in Ch 6 the tasks of how ro recover fea-
ture properties other than orientation, such as bar
width, and whether an edge is a sharp or fuzzy.
Ch 5, Seeing Edges, will also remedy a nagging
irritation that may have formed in your mind. We
emphasized in Chs 1 and 2 the need to be very
clear about the computationaL theory, aLgorithmic,
and hardware Levels of task analysis when studying
vision. But we have blatantly ignored our own ad-
vice in this chapter. We have jumped straight into
considering a particular sort of algorithm, applying
templates, without any guidance from a computa-
tional theory as to the design of those templates.
You might feel a bit cheated. But we have done this
simply to introduce a wide range of basic concepts
and terms that it is best to get out of the way first.
In any event, if you do feel a bit cheated then
you have drawn the right conclusion because this
chapter illustrates just how unsatisfactory it can be
to start addressing an image processing task with-
out a clear computational theory of that task. Tem-
plate matching is a species of algorithm. The design
and use of templates demands the clarity afforded
by a decent theory of the task. We investigate in
Ch 5 how we can get a much better understanding
of what the receptive fields of simple cells are doing
by thinking a lot harder about the task of feature
detection. That will be seen to be the moral of the
story of simple cells told here.
Further Reading
This main function of this early chapter has been
to explore some core ideas needed to understand
seeing, such as receptive fields, channels, coarse
and fine coding. We do not recommend much fur-
ther reading at this stage on this material. We will
be making suggestions for further reading for later
chapters that use the core concepts dealt with here.
Hence, it is not suggested you consult the
sources overleaf at this juncture but we provide
them so that you can follow them up if you wish.
Chapter 3

Barlow HB (1972) Single units and sensation:
A neuron doctrine for perceptual psychology,
Perception 1 37 1-394 Comment C lassic paper that
discusses the relationship between the firing of
single neurons in sensory systems and subjectively
experienced sensations. Co mm entaries celebrating
this landmark paper are in Perception 38 795-807.
It is probably best tackled after reading Ch 11 .
See pp.l16-120. Comment: An excellent book
that we recommend strongly for readers who want
to pursue various topics in human and animal vi-
sio n at an advanced level, and specifically the work
of Regan and Beverley on peaks and troughs in
orientation sensitivity.

Drees 0 (1952) Untersuchungen liber die an-

geborrenen Verhaltensweisen bei Springspinnen
(Salticidae). Z. TierpsychoL. 9 169-209.

Hubel DH and Wiesel TN (1962) Receptive fields,

binocular interaction and functional architecture
in the ca t's striate co rtex. Jou rnal ofPhysiology 160
106-154.
Hubel DH (1988) Eye, Brain and Vision. Scien-
tific American Library. New York. WH Freeman.
Comment: This gives a highly readable overview of
Hubel and Wiesel 's Nobel Prize-winning research.
It is the source of the quotation from Hubel given
in this chapter. However, it is best left until after
reading C hs 8 and 9.
Jonsso n E (2008) Channel-Coded Feature Maps for
Computer Vision and Machine Learning. Linkop-
ing Studies in Science and Technology, Linkoping
University, Sweden. ISBN 978-91-7393-988-l.
Comment: An advan ced mathematical treatment of
theory underl ying the use of channels in computer
vision. We cite it because it is up-to-date and of
possib le interest to students of computer vision.
Land M (1969a) Structure of the retinae of the
principal eyes of jumping spiders (Salticidae: Den-
dryphantinae) in relation to visual optics. Jou rnal of
Experimental Biology 51 443-470.
Land M (l969b) Movements of the retina of
jumping spiders (Salticidae: Dendryphantinae) in
response to visual stimuli. Journal ofExperimental
Biology 51 47 1-493.
Land MF and Nilsson D-E (2001) Animal Eyes.
Oxford University Press. Comment: A short,
erudite, and fascin ating account of animal eyes in
all their diversity. Subsumes papers by Land cited
above.
Regan D (2000) Human Perception of Objects. Sin-
auer Associates, Inc. Publi shers: Sunderland, Mass.

74