C126: Perception

Lectures 3B-
4A
Pattern &
Shape 1
 What does a
Hypercolumn do?

one hypercolumn
contains all
measurements for
one local area

one hypercolumn

“Is there a set of every possible orientations

for simple AND complex cells for every area
in the visual field, so we can perceive every
possible ‘thing’ in that area?” 2
 Today we’ll learn:
Human perception Cat Perception

3
1. Line detector vs spatial frequency
2. Contrast sensitivity function (CSF)
3. Different frequency scales

4
 Background

David Hubel Torsten Wiesel

5
 Simple Cells

square
flash

orientation and position selectivity

predictable from the arrangement of on
and off areas
6
 Line detectors

David Hubel Torsten Wiesel

7
1. Line detector vs spatial frequency

What is a line detector?

How would it code for shape?
What is the alternative?

8
 What the frog’s eye tells the
frog’s brain

“bug detectors”
retinal ganglion
cells that respond
to small moving
spots, trigger
motor response

9
Anything that’s a moving spot is a fly

10
Anything that’s a moving spot is a fly

11
 Gaping in baby seagulls
Super
gaping mom
strength

12
 My mom is a waving stick

Super
gaping mom
strength

13
Do feature detectors also encode more complex shapes
e.g. Hubel and Weisel suggest that a shape is a
piecewise set of adjacent bars and corners

R
Letter as a set of
Attneave’s cat connected line segments

Plausible but is it consistent with the data?

14
For example, is a single line encoded as a set of
adjacent line segments?

Each detected

= by neurons with
nonoverlapping
receptive fields?

Stimulus Individual
segments
15
 But we know this is not the case
columns
of neurons with
receptive fields
centered over
The picture can't
be displayed.

the line

Each hypercolumn has

many receptive fields of
different preferred size
and orientation

a simple shape is a superposition of components

(like a sound) 16
Common analogy: Decomposing
sounds into frequency

1st Harmonic
(Fundamental)
(Pure tone = sine wave)

+ 2nd Harmonic
=
+ 4th Harmonic

17
 Clarinet Frequency Spectrum

Amplitude
Pressure
Sound

250 750 1500

Time Frequency (Hz)

Frequency Spectrum
Frequency Spectrum
Amplitude

Amplitude

500 1000 2000 4000

Frequency (Hz)
Frequency (Hz) 18
 Complex spatial patterns can also be
decomposed into sine waves
Fundamental

Luminance
3rd Harmonic + Space

5th Harmonic +

19
+

20
21
 Complex patterns can be made of the sum of simple
sine patterns of different frequency (Fourier
decomposition/synthesis)
But receptive fields don’t correspond to sine waves
What do they look like?
Receptive

+ + -
- +
Field

+ +
- + -
+ + - -+
+
Profile

+
— —

Bar detector Real receptive field

Realistic RFs have extra excitatory and inhibitory lobes

22
RFs are like small patches of sine waves
(wavelets/Gabors)
The same summation property holds
Arbitrary spatial patterns can be created by
summing small patches of sine waves of many
different orientations and sizes

23
 +
-

Any spatial pattern (e.g., a

photograph) can be created by
summing small patches of sine
waves (wavelets) of many
different orientations and sizes
24
 +
-

Note: cell not explicitly

measuring edges/bars. Instead,
measure local patches of sine
wave luminance variations

25
 What’s the point here?

Individual cell NOT explicitly

measuring feature like rim of
hat; rather hypercolumn
contains many cells with
overlapping RFs that code local
spatial frequency 26
Adding up wavelets to make an image

27
 Hubel & Wiesel missed the extra lobes

H&W Simple Cell Actual Simple Cell

Receptive Field Receptive Field

-- + - -+
- + - + -- + --
- + --
-
- ++ - - -+ + - - +
-
-
- - +

Bar detector Measures patches

of sine waves

Small difference?
No. Bar detector vs. image measurement 28
Bar detector vs. image measurement

1 neuron
population
29
Why measure patches of sine waves and not bars/edges?
Rapid/efficient transmission of data

37 seconds (using a 56K modem)

Full picture 512x512 = 262.144K

Next 6 slides: decreasing data transmission from

100% to 0.1% of wavelet coded information 30
 262K

37 sec 31
 65K
25%

9.25 sec

32
 6.25% 16K

2.4 sec

33
 1.6%
4K

0.6 secs

34
0.4 %

1K

.15 sec

35
0.1%

256
.04 sec bytes

36
 25% of the
0.4% of the sine bar/edge
wave patches: information

Coding images with sine wave

patches is more efficient 37
 Evidence for wavelets?
• spatial frequency tuning of cortical neurons
– Narrow tuning for specific frequencies
– Selectivity to bar width vs grating spatial
frequency
• spatial frequency adaptation in humans
(psychophysics)

38
Measuring simple cell responses to sine waves
Contrast Sensitivity
for a bunch of cells

+ - + - +

(Cell’s response)
+

+ Low High
+ - - + 39
 Cells more selective to spatial
frequency than to bar width

spatial frequency bar width

40
2. Contrast Sensitivity Function

41
 What is the limit of our perception?
How do individual cells contribute to those limits?

Envelope of peak sensitivities of

(Cell’s response)

individual neurons =
Limits of our perception =
Contrast Sensitivity Function

Low High

42
 Low Limits of visibility: the Contrast Sensitivity Function (CSF)

INVISIBLE INVISIBLE

VISIBLE
Contrast
High

Spatial Frequency
43
Low High
 Low Limits of visibility: the Contrast Sensitivity Function (CSF)

20
Dayyr(photopic)
old

80 yr old
Contrast

Night (scotopic)
High

Spatial Frequency
44
Low High
CSF, Age, & Art Monet, Parc Monceau, 1876
Monet, ~38 yrs old
Monet, Parc Monceau, 1878

Monet, House seen from the rose garden, 85 yrs old, loss of high SF information
1925, 85 yrs old 45
 Monet, Water Lilies and Japanese Bridge, 1899
CSF, Age, & Art

Monet, The Japanese Bridge, 1925

46
CSF & Night Driving

Day (photopic)

Night (scotopic)

47
 Low Limits of visibility: the Contrast Sensitivity Function (CSF)

Human
Cat
Contrast

Goldfish
High

Spatial Frequency
48
Low High
Human CSF limit Cat CSF limit

Human CSF limit Goldfish CSF limit

49
 What is the limit of our perception?
How do individual cells contribute to those limits?

Envelope of peak sensitivities of

(Cell’s response)

individual neurons =
Limits of our perception =
Contrast Sensitivity Function

Low High

50
 Adaptation experiments
(can we desensitize some wavelet
units?)
• view a fixed SF grating for a
prolonged period
• then measure contrast sensitivity
function in its entirety
• subtract adapted CSF from
unadapted CSF to estimate
characteristics of underlying
mechanisms

51
 Adapt patterns

test patterns
adapt
test

time -->
52
 Blakemore and Campbell (1969) experiments

Plot difference between before and after

53
 Support for spatial
frequency analysis--wavelet
hypothesis (summary so far)
• Narrow spatial frequency tuning in single
cortical neurons
• Adaptation selective to restricted range of
spatial frequencies around the adapted SF

54
V1 hypercolumns revisited

Right Left
Eye Eye

55
Hubel & Wiesel Hypercolumn model
Ocular dominance slabs
Orientation columns

One orientation column

One ocular dominance slab; R = right eye

56
Ocular dominance slabs

Right Left
Eye Eye
One ocular dominance slab; R = right eye
57
 DeValois revised hypercolumn model

RF width
decreasing
with distance
from center

58
 Orientation columns:
Pinwheels not ice trays

59
 3. Different frequency ranges

Low, mid, and high spatial frequency ranges =

Large scale, mid range and fine detail of an image
In periphery, only large scale features are registered

60
Gross and fine detail of image

+ =

61
Artists use the different ranges of spatial
frequencies to camouflage patterns

Julesz- Dali
Harmon 62
Chuck
Close
63
Artists use the different ranges of spatial
frequencies to camouflage patterns

Tina Fey - Hasselhoff Condi - Hasselhoff

64
Different spatial frequencies here carry
different information
Vary distance, vary which information you
are most sensitive to 65
66