PSC100Y Perception 3 – Object Recognition!

Introduction to Cognitive Psychology

Perception 3-1!
Bayes Equation!
“Fan Blowing Bubbles”!

Knowledge!
Top-Down Processing!
Perception!
Bottom-Up Processing!

Sensory Inputs!

Bayes’s Theorem!
P(H|E) = P(H) x P(E|H)!
•  Is the object circled in red a flat circle or a sphere?!
! ! P(E)!

•  Our visual system tests these hypotheses and

chooses the one that best fits with both the
sensory evidence and our prior experience of the
world.!

Thomas Bayes (1701–1761)!

Conditional Probabilities! Bayes’s Theorem!

P(A|B) Probability that A is true given that B is true! H is a hypothesis about the world!
(e.g., that a circular object is present)!
!
Example 1!
E is the current sensory evidence!
Probability that a person is > 6 feet tall for adult males and females:!
(e.g., the current visual input)!
P(over 6’ | adult male) > P(over 6’ | adult female)!

Example 2! !P(H|E)!
Probability that a circular object is present in the world given the
image shown below!
P(circular object | sensory input)!
Probability that the
•  The sensory input may be hypothesis is true
consistent with multiple given the evidence
interpretations, so prior knowledge (posterior probability)!
is important to determine the most
likely interpretation!

© S. J. Luck

All rights reserved
1

 Bayes’s Theorem! PSC100Y
Introduction to Cognitive Psychology
Probability that the Probability that the
hypothesis is true evidence would occur if
regardless of the evidence the hypothesis is true!
(prior probability)!

!P(H|E) = P(H) x P(E|H)!

! ! ! P(E)!

Probability that the

Probability that the
evidence would occur
hypothesis is true
whether or not the
given the evidence
(posterior probability)!
hypothesis is true
(normalizing constant)!
Perception 3-2!
Bayes Examples!

Variations in Evidence! Plugging in Some Numbers!

•  Which is more likely, H1 or H2?! Which is more likely, H1 (sphere) or H2 (disk)?!
–  H1 = sphere! P(E|H1) = .8! (80% chance that a sphere would produce this image)!
–  H2 = flat disk with fancy shading! P(E|H2) = .2! (20% chance that a flat disk would produce this image)!
•  This pattern of shading is likely for a sphere! P(H1) = P(H2) = .5! (Assume that, without any stimulus, we have equal
reason to predict a sphere or a flat disk)!
–  P(E|H) is high for H1!
P(E) = 1! (Not relevant for this example)!
•  This pattern of shading is unlikely for a flat disk!
!
–  P(E|H) is low for H2!
P(H1|E) = P(H1) x P(E|H1) = .5 x .8 = .4!
!
! ! P(E) 1!
!P(H|E) = P(H) x P(E|H)!
P(H2|E) = P(H2) x P(E|H2) = .5 x .2 = .1!
! ! ! P(E)!
! ! P(E) 1!
! Probabilities are always between 0 and 1!
!
Note: Probabilities are always between 0 and 1! Note:

Variations in Prior Probability! Plugging in Some Numbers!

•  Which is more likely, H1 or H2, given Which is more likely, H1 (CAT) or H2 (CHT)?!
as the sensory input! P(E|H1) = P(E|H2) = .5! (Evidence is equally strong for CAT and CHT)!
–  H1 = CAT! P(H1) = .0001! (Frequency of occurrence of CAT in written English)!
–  H2 = CHT!
P(H2) = .000001! (Frequency of occurrence of CHT in written English)!
•  Evidence is equally consistent with H1 and H2!
P(E) = .0000505! (Probability of this stimulus pattern occurring)!
–  P(E|H) is the same for H1 and H2!
!
•  CAT occurs more often than CHT in English!
–  Prior probability P(H) is higher for CAT than for CHT! P(H1|E) = P(H1) x P(E|H1) = .0001 x .5 = .99!
! ! ! P(E) .0000505!
!P(H|E) = P(H) x P(E|H)! P(H2|E) = P(H2) x P(E|H2) = .000001 x .5 = .01!
! ! ! P(E)! ! ! P(E) .0000505!
Note: Probabilities are always between 0 and 1! ! Probabilities are always between 0 and 1!
Note:

!
!

© S. J. Luck

All rights reserved
2

 PSC100Y Stages of Form Perception!
Introduction to Cognitive Psychology
Parsing: breaking the
complex scene up into
individual objects.!
Surface interpretation:
figuring out the surface
properties of each object
(color, lightness, 3D-shape)!
Recognition: link lower level
perceptual properties with
the objects stored in
memory.!
Perception 3-3!
Parsing and Filling In!

Parsing! Finding Edges!

The visual input is a 2-dimensional
array of brightness and color
values -- where are the object
boundaries?!
30,0,90 32,0,90 33,50,90 40,50,90 38,50,94

20,5,90 23,0,80 20,50,90 18,50,98 20,50,99

10,10,90 14,10,23 10,10,90 12,8,92 12,12,94

10,0,90 23,8,26 26,12,71 23,9,90 21,8,92

12,0,85 36,6,38 39,18,72 33,7,85 31,2,85

•  The process of finding edges is accomplished by early stages of
13,0,80 49,9,80 48,7,71 47,6,80 45,9,80
visual cortex, which apply a variety of complex algorithms.!

•  Finding edges! •  The result is something like the image above to the right of the
•  Filling in behind occluders! original.!

•  Grouping elements together!

Filling In Behind Occluders! Filling In Behind Occluders!

•  We actually do more than simply eliminate occluding objects; we also

make a guess about what lies behind the occluders, filling in the
missing information.!

•  We don’t have conscious experience of the information we fill in, but

•  In the image above, the bubbles that were once covering (occluding)
research has demonstrated that we unconsciously represent
various areas of the original image have been removed!
information we infer lies behind occluders.!

© S. J. Luck

All rights reserved
3

 APPLE APPLE
•  This is what the world looks like with the ink still visible; it is much
•  Occluding ink was spilled on this image and was removed; this easier to figure out that the word is “apple.”!
makes it fairly difficult to read the word. !
•  Our visual system doesn’t just strip away the occluding ink; it actually
fills in missing information, allowing us to perceive the partially
occluded object better.!

•  How does the visual system know that these shapes continue in straight lines

APPLE
behind the ink to form this shape?!

APPLE •  The visual system uses assumptions about what things are likely
to happen in the world and what things are unlikely to happen.!

APPLE
Bayes’s Theorem!

APPLE
P(H|E) = P(H) x P(E|H)!
!! ! P(E)!

•  Hypothesis: big thick line that extends behind the ink.!

•  Evidence: the fact that the edge on one side of the ink lines of
with the other edge.!
!-These edges are collinear!
•  What we are computing: the probability that a big thick
continuous line is present given the fact that these two edges are Bayes’s Theorem!
collinear.!
P(H|E) = P(H) x P(E|H)!
! ! P(E)!

© S. J. Luck

All rights reserved
4

 PSC100Y

APPLE
Bayes’s Theorem! Introduction to Cognitive Psychology
P(H|E) = P(H) x P(E|H)!
! ! P(E)!

•  P(E|H): the probability that the two edges would be collinear if we

had a single thick, continuous line behind the spilled ink.!

•  P(E): the probability that these two edges would be collinear just by
chance, even if they weren’t part of the same continuous line.!

Perception 3-4!
Gestalt!

Gestalt Psychology!
“Gestalt” is a German word that means “figure” in the sense of a
unified or meaningful whole!

•  Studied the rules that our perceptual systems use to

group together the different elements of a sensory
input to make meaningful objects.!

•  They believed that the perception of an object is more

than just the perception of the individual parts.!

•  Arranging the disorganized elements on the left into the image on

the right allows us to perceive something beyond the individual parts
themselves; we see a 3-dimensional sphere.!

•  There is no actual sphere here, but the arrangement of the parts

makes us perceive one.!

•  This is an illusion that usually gives us the right answer.!

Gestalt Principles: Pragnanz
 Gestalt Principles: Similarity!

Literally, “pregnant” (with meaning)

Often translated as “good figure”


•  Does this solid black shape consist of a single complicated shape, or Similar shapes are
does it consist of two shapes that happen to overlap each other.! grouped together,
causing perception
•  Pragnanz says that we should choose the simplest possible
of vertical groups!
interpretation of the sensory input, which would be two simple
shapes.!

© S. J. Luck

All rights reserved
5

 Gestalt Principles: Similarity! Gestalt Principles: Good Continuation!

Similar shapes are Easily perceived as + +

grouped together, either horizontal or
causing perception vertical groups!
of vertical groups! Preferred Nonpreferred
Interpretation Interpretation

Gestalt Principles: Good Continuation! Gestalt Principles: Good Continuation!

+ + +
•  In this image, we group together all of the bumps that fall along
Preferred Nonpreferred a single curve.!

Interpretation Interpretation

PSC100Y Stages of Form Perception!

Introduction to Cognitive Psychology
Parsing!
Surface interpretation:
figuring out the surface
properties of an object,
such as its color,
texture, and 3D shape.!
Recognition!

Perception 3-5!
Recognition!

© S. J. Luck

All rights reserved
6

 Surface Interpretation! Surface Interpretation!
Blue Green Red

Relative
Absorption

400 500 600 700

Wavelength (nm)

•  The perception of an object’s color is a result of our red, green,

and blue photoreceptors.!

•  There are also color constancy mechanisms we use to factor out the •  Using motion cues to figure out the 3D form of the exterior of an
color of the light source from the reflectance properties of the object is also an example of Surface Interpretation.!
surface of the object.!

Stages of Form Perception! Template Theory Example!

Template for "A" Template for "B"

Recognition: the
process of linking a
sensory input to a
representation in
memory!
!
Theories of Recognition!
Structural Description
Theories! •  Template Theory: we have a picture-like representation of each
Template Theories object that we know in memory.!
(View-Based Theories)! •  These picture-like memory representations are called templates.!
!
•  We compare the incoming sensory input with our templates to see
which template is best matched by the sensory input.!

AAAAAAA
Template Example!
Correlation = 100% Correlation = 30%

AAA AAAA
A!
•  Not all A’s look the same; however, we can still recognize the
•  We compare our sensory input with the template in memory to see letter if given to us in a different font.!
which fits best.!

© S. J. Luck

All rights reserved
7

 Template Example! Rotation & Scaling!

Correlation = 85% Correlation = 30%

•  Templates still work even if they do not match perfectly.!

•  Our brain also mentally adjusts the size and rotation of the
•  As long as one template matches well enough, and matches better image so that it can be matched against the template.!
than any other template, we can recognize the object.!

•  We also can have multiple templates of a given object, such as that

of the letter A.!

Mental Rotation! Mental Rotation!

Task: Is it a normal letter or a mirror-image letter?! Task: Is it a normal letter or a mirror-image letter?!

Slow Slow

•  In this task, subjects are shown a letter of the alphabet.!

Reaction
•  Sometimes it is a normal letter and sometimes it is mirror- Reaction
Time Time
reversed.!

•  Subjects must press one button if it is normal and a different

button if it is mirror-reversed.!
Fast Fast
0 180 0 180
Degrees of Rotation from Upright Degrees of Rotation from Upright

Mental Rotation! Mental Rotation!

1400 Task: Are the left and right objects the same?!
Reaction Time
(milliseconds)

1200

1000

800
0º ±72º ±144º
Rotation angle
Hyun, J.-S., & Luck, S.J. (2007). Visual working memory as the substrate for Shepard & Metzler, 1971!
mental rotation. Psychonomic Bulletin & Review, 13, 154-158. !

© S. J. Luck

All rights reserved
8

 Mental Rotation! Mental Rotation!
Task: Are the left and right objects the same?! Task: Are the left and right objects the same?!

Shepard & Metzler, 1971! Shepard & Metzler, 1971!

Mental Rotation! PSC100Y

Introduction to Cognitive Psychology
Task: Are the left and right objects the same?!

Perception 3-6!
Shepard & Metzler, 1971! Structural Descriptions!

Structural-Description Theories! Structural-Description Theories!

T •  In structural-description
theories, objects are
T T
Contains Contains represented abstractly as Contains Contains Contains Contains
parts and relations between
parts. !
Vertical Horizontal Vertical Horizontal Vertical Horizontal
Line Line Line Line Line Line
•  Recognition consists of
forming a structural
Supports description of the input and Supports Supports
comparing it with structural
Bisects descriptions in memory.! Bisects Bisects

Structural Description Structural Description of

in Memory! Sensory Input!

© S. J. Luck

All rights reserved
9

 Structural-Description Theories! PSC100Y
Introduction to Cognitive Psychology

T
Contains Contains

Vertical Horizontal
Line Line

Supports

Bisects
Perception 3-7!
Structural Description Recognition by Components!
in Memory!

Recognition-By-Components! Recognition-By-Components!

•  Common objects can be broken down into a set of simple 3D parts,

•  Structural descriptions work well for simple objects, such as T’s,
that Biederman called geons.!
but it becomes more difficult for 3D objects.!
•  20 different geons, such as bricks, pyramids, and cones.!
•  Irv Biederman’s theory, Recognition by Components, provides a
solution for the problem of figuring out shapes of the parts of
•  Real world objects can be defined by these geons and the
3D objects, given that the shape of the retinal image depends
relationships between them.!
on the 3D viewpoint from which we view an object.!

Recognition-By-Components! Recognition-By-Components!

•  How can we tell what geons are present in an object?!

•  Some properties of an object are stable and diagnostic, whereas

•  Some objects use the same geons, but have different spatial others depend on the particular viewing conditions and are not
relationships, such as a cup or pail.! diagnostic.!

•  The non-diagnostic features are termed “accdiental properties,” and

the diagnostic features are termed “non-accdiental properties.”!

© S. J. Luck

All rights reserved
10

 Recognition-By-Components! Recognition-By-Components!

Brick:! Brick:!
3 parallel edges! 3 parallel edges!
Inner Y vertex! Inner Y vertex!
3 outer arrow vertices! 3 outer arrow vertices!

•  These properties can be

seen from almost any
viewpoint.!

Recognition-By-Components! Recognition-By-Components!

Brick:!
3 parallel edges!
Inner Y vertex!
3 outer arrow vertices!

•  However, there are some unusual viewpoints where this does not
always work.!

•  We cannot see all of the non-accidental properties that define a brick

from this viewpoint.!

•  It also turns out that the ability of people to recognize objects from
unusual viewpoints is not very good.!

Recognition-By-Components! Recognition-By-Components!

Brick:! Brick:!
3 parallel edges! 3 parallel edges!
Inner Y vertex! Inner Y vertex!
3 outer arrow vertices! 3 outer arrow vertices!

Cylinder:! Cylinder:!
2 parallel edges! 2 parallel edges!
2 tangent Y vertices! 2 tangent Y vertices!
2 parallel curves! 2 parallel curves!

© S. J. Luck

All rights reserved
11

 Recognition-By-Components! Recognition-By-Components!

Brick:! Brick:!
3 parallel edges! 3 parallel edges!
Inner Y vertex! Inner Y vertex!
3 outer arrow vertices! 3 outer arrow vertices!

Cylinder:! Cylinder:!
2 parallel edges! 2 parallel edges!
2 tangent Y vertices! 2 tangent Y vertices!
2 parallel curves! 2 parallel curves!

Recognition-By-Components!

Y Junction!

Guitar! Luggage!

•  Recognition-By-Components theory is designed to do basic-

level categorization, where we figure out what basic category
an object belongs to.!

Template for "A" Template for "B"

‘56 Strat!
T
Contains Contains

Vertical Horizontal
Line Line

Supports

PRS Custom 22! Bisects

•  It is not designed to explain how we

can tell the difference between two
members of the same category, such
as a PRS custom 22 and ‘56 strat
guitars.!
•  We don’t yet know which type of theory is correct.!
•  One possibility is that the brain uses both kinds of mechanisms
because each has its strengths and weaknesses.!

© S. J. Luck

All rights reserved
12