86 The Perceptual Organization of Depth
ROLAND FLEMING AND BARTON L. ANDERSON
T    perception is to identify the spatial layout
of the objects and surfaces that constitute our surroundings.
One important observation about the world around us that
influences the way we see depth is that physical matter is not
distributed randomly, with arbitrary depths at every location.
On the contrary, the environment is generally well organized:
the world consists mainly of tightly bound objects in a dis-
cernible layout. This order results from countless forces and
processes (such as gravity and biological growth) which tend
to organize matter into objects and place those objects in
certain spatial relations. The central thesis of this chapter is
that our perception of depth mirrors this organization. We
argue that because the world consists of objects and surfaces,
our perception of depth should likewise be represented in
terms of the functionally valuable units of the environment,
namely, surfaces and objects. As we shall see, this has pro-
found consequences for the processing of depth information.
In particular, there is more to depth perception than simply
measuring the distance from the observer of every location
in the visual field. Rather, the perception of depth is the active
organization of depth estimates into meaningful bodies. Depth
constrains the formation of perceptual units, and, recipro-
cally, the figural relations between depth measurements allow
the visual system to parse its representation of depth into eco-
logically valuable structures.
There are many sources of information about depth from
“pictorial” perspective to motion parallax. An exhaustive
review of all these sources of information is beyond the
scope of this chapter (although see Bruce et al., 1996;
Palmer, 1999, for introductory reviews). Instead, we discuss
three key domains in which the visual system “organizes”
our perception of depth into meaningful units to emphasize
the intimate relationship between depth processing and
perceptual unit formation.
In the first section, we discuss how the visual system infers
the layout of surfaces from local measurements of depth. We
will argue that local estimates of depth are ambiguous but
that the geometry of occlusion critically constrains the legal
interpretations. Occlusion occurs when one opaque object
partly obscures the view of a more distant object, as happens
frequently under normal viewing conditions. Occlusion is
important because it occurs at object boundaries, and
therefore the depth discontinuities introduced by occlusion
1284
provide ideal locations for the segmentation of depth into
objects. Moreover, as we will show in the first section, the
geometry of occlusion causes relatively near and relatively
far depths to play different roles in the inference of surface
structure.
In the second section we discuss the visual representation
of environmental structures that are hidden from view. If the
visual system is to organize depth into meaningful bodies, it
must represent whole objects, not only those fragments that
happen to be visible. In order to do this, the visual system
must interpolate across gaps in the image to complete its
representation of form. We argue that by considering the
particular environmental conditions under which structures
become invisible (specifically occlusion and camouflage), we
can make predictions about the mechanisms underlying
visual completion. We also discuss how visual completion
influences the representation of depth.
Finally, we discuss what happens when the scene contains
transparent surfaces, and thus multiple depths are visible
along a single line of sight. We argue that this introduces a
second segmentation problem in the perceptual organization
of depth. The visual system not only needs to segment “per-
pendicular” to the image plane, such that neighboring loca-
tions are assigned to different objects; with transparency,
the visual system also has to segment depth “parallel” to the
image plane by separating a single image intensity into
multiple depths, a process known as scission (Koffka, 1935).
We discuss the conditions under which the visual system
performs scission and how the ordering of the surfaces in
depth is resolved.
We argue that the ambiguity of local depth measure-
ments, the representation of missing structure, and the
depiction of multiple depth planes are three of the major
problems faced by a visual system if it is to organize depth
into surfaces and objects. Through systematic explanations
of example stimuli, we discuss some of the ways in which
the visual system overcomes these problems.
Interpreting local depth measurements: the contrast depth
asymmetry principle
In this section we discuss how occlusion constrains the inter-
pretation of local depth estimates. Specifically, we show that
occlusion enforces a crucial asymmetry between relatively
near and relatively distant structures that can have profound
implications for the representation of surface layout.
Although the principles are discussed in terms of binocular
disparity, the fundamental logic relates to the geometry of
occlusion and therefore applies to any local estimate of
depth.
B S   C P
Binocular stereopsis is the most thoroughly studied source
of information about depth. Binocular depth perception
relies on the fact that the two eyes receive slightly dif-
ferent views of the same scene. Because of the horizontal
parallax between the two views, a given feature in the
world often projects to two slightly different locations on
the two retinas (Fig. 86.1). These small differences in retinal
location, or binocular disparities, vary systematically with
distance in depth from the point of convergence and can
thus be used to triangulate depth. For a thorough treatment
of stereopsis, see Howard and Rogers (1995) and Chapter
87.
In order to determine the disparity of a feature in the
world, the visual system must localize that feature in the two
retinal images. Once it has identified matching image
features, the difference in retinal location is the binocular
disparity, which can then be scaled to estimate depth. The
visual system must not measure the disparity between fea-
tures that do not belong together; otherwise, it will derive
spurious depth estimates (Fig. 86.1). Because of this, the
accuracy of the matching process is critical to binocular
depth perception. The problem of identifying matching fea-
tures in the two eyes’ views (i.e., features that originate from
a common source in the world) is known as the correspondence
problem.
If the features that the visual system localizes in the two
images are very simple, such as raw intensity values (or pixels),
then in principle there could be many distracting features
that do not in reality share a common origin in the
world. Under these conditions the correspondence problem
would be difficult, as the visual system would have to iden-
tify the one true match from among a large number of false
targets.
However, there is considerable debate about what types
of image features the visual system matches to determine
disparity ( Jones and Malik, 1992; Julesz 1960, 1971;
Marr and Poggio, 1976, 1979; Pollard et al., 1985; Prazdny,
1985; Sperling, 1970). Psychophysically, at least, it now
seems unlikely that the visual system matches raw lumi-
nances. Rather, the visual system seems to match local
contrast signals, that is, localizable variations in intensity,
such as luminance edges (Anderson and Nakayama, 1994;
Smallman and McKee, 1995). This seems an almost
inevitable consequence of early visual processing, which
  :    
P
(a)
Vieth-Müller Circle
α
Q
Q'
P' Q' P'
(b) (c)
A B A B
dA
d*
B'
A' A' A'
F 86.1. a, The two eyes converge by angle a on a point P.
Therefore, by definition, P projects to the foveae of both eyes (P¢).
The Vieth-Müller circle is one of the geometrical horopters, that
is, it traces a locus of points in space that project to equivalent
retinal locations in the two eyes and thus carry no interocular
disparity. Point Q is closer to the observer than P (as it falls inside
the horopter). Therefore, it projects to different locations on the
two retinas (Q¢). The difference in the locations of Q¢ is the binoc-
ular disparity, which can be scaled by the vergence angle, a, to
derive depth. b, When the visual field contains many points, there
is a potential ambiguity concerning which image features corre-
spond in the two eyes. Correct matches yield correct depth esti-
mates, such as dA. c, By contrast, false matches yield erroneous
depth estimates. Here, the image of point A has been incorrectly
matched with the image of point B, leading to an incorrect depth
estimate, d*.
maximizes sensitivity to contrasts rather than to absolute
luminances (Cornsweet, 1970; Hartline, 1940; Ratliff, 1965;
Wallach, 1948). By the time binocular information converges
in V1, the visual field appears to be represented in terms of
local measurements of oriented contrast energy (De Valois
and DeValois, 1988; Hubel and Wiesel, 1962), and thus it
is likely that these are the features from which disparity is
computed.
If this is true, then the image features that carry disparity
information are local contrasts such as luminance edges.
1285
However, this poses a problem for the visual system, for in (a)
order to capture the functional units of the environment, the
visual representation of depth should be tied to surfaces and
objects, not to local image features. There is therefore a poten-
tial discrepancy between the image features that carry dis-
parity information (i.e., local contrasts) and the perceptual
structures to which depth is assigned (i.e., regions) in the
ultimate representation of environmental layout. This
discrepancy plays a critical role in the theoretical discussion
that follows (see Anderson, in press). (b) (c)
A local image feature, such as an edge, has only one true
match in the other eye’s image. Therefore, the edge carries
only one disparity. However, depth is ultimately assigned to
the two regions that meet to form the edge. This results in a
problem: in order to represent surface structure, the
visual system must assign depth to both sides of an edge,
even though the edge carries only one disparity (Fig. 86.2).
How does the visual system infer the depths of two regions
from every local disparity signal? We will show that the
geometry of occlusion imposes an inviolable constraint on F 86.3. Asymmetries in depth interpolation. a, When the
the interpretation of local disparity-carrying features. To left stereopair is cross-fused, the diamonds appear to float inde-
pendently in front of the Kanizsa triangle, as schematized in
anticipate, we show that the simple fact that near surfaces
b. When the disparity of the diamonds is inverted (by cross-fusing
can occlude more distant ones, but not vice versa, has pro- the right stereopair), the diamonds drag their background with
them, creating the percept of a triangular hole, even though
only the disparity of the diamonds has changed (c). This asym-
(a)
metrical change in surface structure can be explained by the con-
trast depth asymmetry principle (see text). (Adapted from Takeichi
et al., 1992.)

? ? found consequences for the assignment of depth to whole

regions.

A  D: A D By way of

(b)
wo
rld
ima
ge
F 86.2. a, The image of a square occluding a diamond.
A receptive field of limited extent (the ellipse) captures only
local information about the scene, here a vertical luminance
edge. This local information is ambiguous, as many different scenes
could have resulted in the same image feature. b, If disparity is
calculated by matching local contrasts, then the edge carries
only a single disparity. However, in this case, the light and dark
sides of the edge result from two distinct objects, and therefore
different depths have to be assigned to the two sides of the
edge.
motivation for the theoretical discussion that follows, con-
sider Figure 86.3, which is based on a figure developed by
Takeichi et al. (1992). The figure consists of a Kanizsa illu-
sory triangle and three diamonds. When disparity places the
diamonds closer to the observer than the triangle and induc-
ers (by cross-fusing the stereopair on the left of Fig. 86.3),
the diamonds appear to float independently in front of the
background, and the Kanizsa triangle tends to be seen as a
figure in front of the circular inducers; this percept is
schematized in Figure 86.3B. The disparities in the display
can be inverted simply by swapping the left and right
eyes’ views, as can be seen by cross-fusing the stereopair on
the right side of Figure 86.3. In this case, what was previ-
ously distant becomes near and vice versa, such that the
diamonds are placed behind the plane of the inducers. In
both versions of the display, the triangle itself carries no dis-
parity relative to the circular inducers; only the disparity of
the diamonds changes from near to far. This simple inver-
sion leads to a change in surface representation that is more

1286 , ,   

 Possible depth interpretations
Continuous Surfaces
Occluding Surfaces
matching,
disparity
computation
Local Image Data
F 86.4. A contour which carries a depth signal (e.g., dis-
parity) is inherently ambiguous. Two main classes of world states
could have given rise to the contour: the contour could have orig-
inated from a single continuous surface (e.g., a reflectance edge or
cast shadow), or it could have originated from an occlusion event.
In the occlusion case, the border ownership of the contour (i.e.,
which side is the occluder) is ambiguous. Nonetheless, in all con-
figurations, both sides of the contour are constrained to be at least
as far as the depth signal carried by the contour. This introduces a
fundamental asymmetry in the role of near and far contours in
determining surface structure (see text for details). (Adapted from
Anderson, in press; see also Anderson et al., 2002.)
complex than a simple reversal in the depth ordering of the
perceptual units (as schematized in Fig. 86.3B). When the
diamonds recede, they drag their background back with
them, such that the triangle appears as a hole through which
the observer can see a white surface; the three black dia-
monds lie embedded in the more distant white surface. This
recession of the background has a secondary effect of
increasing the strength of the illusory contour (the border of
the triangle).
The important observations with regard to the theory are
the following. First, when the diamonds are in front, they are
freely floating and separate, while when they recede, they
drag the background with them. Second, when the dia-
monds are forward, the Kanizsa triangle tends to be seen as
a figure (rather than ground), but when the diamonds are
more distant, the triangle is seen as a hole. And yet all that
changed in the display was the disparity of the diamonds.
Why does this simple reversal in depth lead to an asymmet-
ric change in the surface representation? Why does the dis-
parity of the diamonds influence the appearance of the
triangle? These are the asymmetries of depth to which the
following discussion pertains.
F F  S: I  L
D S Let us assume that the visual system
has located a luminance edge and derived a disparity, d0,
from that edge. What possible surface configurations are
consistent with the local disparity measurement? Broadly,
the legal interpretations fall into two classes, as shown in
  :    
Figure 86.4. The first class consists of surface events in
which both sides of the edge meet at the depth of the edge,
d0. There are many surface events for which this is the
case: reflectance edges, cast shadows, and creases in the
surface, to name just three. When the feature originates from
a continuous manifold, as in these cases, interpretation is
simple, as both sides of the edge are assigned the same
depth, d0.
The second class of interpretations occurs when the edge
corresponds to an object boundary and therefore represents
a depth discontinuity (Fig. 86.4). In this case, one side of the
edge lies at the depth of the occluding object, and the other
side of the edge lies at the depth of the background. There-
fore, the visual system must assign different depths to the two
sides of the edge. How can the visual system assign two
depths, when it is given only one disparity, d0? The answer
is that it only assigns a unique depth to the occluding
side. The critical insight is the following: the depth mea-
surement acquired at an occluding edge only specifies the depth
of the occluding surface. The visual system assigns depth d0 to
the occluding surface. All that it knows about the other side
is that it must be more distant than the occluding surface. If
the more distant surface is untextured, then it could be at
any depth behind the occluder and the local image data
would remain the same. By contrast, if the depth of the
occluding surface varies, the disparity carried by the object
boundary must also change, because the occluding surface
“owns” the contour (Koffka, 1935; Nakayama et al., 1989)
and is therefore responsible for the disparity associated with
the edge.
Although the visual system cannot uniquely derive the
depth of the occluded side (i.e., the background) from the
local disparity computation, there is one critical piece of
information that it does have: the occluded side is more
distant than the occluder. There is no way for an occluding
object to be more distant than the background that it
occludes. If the background is brought closer than the
object, then the background becomes the occluding surface and
carries the edge with it. In this way, occlusion introduces a
fundamental asymmetry into the interpretation of disparity-
carrying edges: the occluded side of the edge can be at
any distance greater than d0, but neither side can be nearer
than d0.
We can summarize the possible depth assignments
(from the occlusion and nonocclusion classes just described)
in the form of a constraint on the interpretation of local
disparity-carrying contrasts, termed the contrast depth asymme-
try principle (Anderson, in press; see also Anderson et al.,
2002):
Both sides of an edge must be situated at a depth that is greater than
or equal to the depth carried by that edge.
1287
 Although this geometric fact is simple in form, it can have
pronounced effects on the global interpretation of images
when the constraint applies to all edges simultaneously. We
will now run through an example to show how the principle
can explain the asymmetric changes in perceived surface
structure that occur when near and far disparities are
inverted.
A   C D A P-
 In order to demonstrate the explanatory power of
the contrast depth asymmetry principle (hereafter CDAP),
we will now use it to account for the demonstration in
Figure 86.3. Recall that when the diamonds carry near
disparity, they float freely in front of the background, and
the illusory triangle tends to be seen as figure. When the
disparity is reversed, however, the diamonds drag the
background back with them, and the triangle appears as a
hole. This asymmetry in surface layout is depicted in Figure
86.3B.
Let us first consider the case in which the diamonds
appear to float in front. The visual system has to interpret
the disparity signals carried by the edges of the diamonds.
The CDAP requires both sides of the diamonds’ edges (i.e.,
the black inside and the white outside of the diamonds) to
be at least as distant as the edges. Now consider the
“pacman” inducers, which are more distant than the dia-
monds. The constraint requires both sides of these edges to
be at least as distant as their edges. This means that all of
the black interior of the inducers must be at least this distant
and, more importantly, all of the white background must be at
least this distant, which is farther than the disparity of the
diamonds. If all of the white background is farther than the
diamonds, then the edges of the diamonds must be occluding
edges, and the black interior of the diamonds must be an
occluding surface. This explains why the diamonds are seen
as independent occluders, floating in front of the large white
background and black inducers: the edges of the “pacman”
inducers drag the white background back, leaving the
diamonds floating in front.
Now consider the case in which the diamonds are more
distant than the inducers. Again, the CDAP requires both
the inside and the outside of the diamonds to be at least as
far back as their disparity dictates. This means that both the
diamonds and their white background are dragged back to the
more distant disparity. Now consider the “pacman” induc-
ers, which carry a relatively near disparity. Because the white
background behind the diamonds has been dragged back
with the diamonds, the inducers and their white background
must be occluding surfaces. This means that the background
immediately surrounding the diamonds must be visible
through a hole in the occluding surface. The edges of this
hole are the illusory contours of the Kanizsa figure. Note
again: the fact that both sides of every edge have to be
1288 , ,   
at least as far as the edge leads to asymmetrical surface
structures when disparities are inverted.
This is just one example that shows how the CDAP can
account for asymmetrical effects of relatively near and rela-
tively far disparities on perceived surface layout. Because the
CDAP is derived from the geometry of occlusion, it can
account for a very large number of displays and can be
used to generate surprising new displays (Anderson, 1999,
Anderson, in press).
Occlusion and camouflage: hallucinating the invisible
The central thesis of this chapter is that the visual
system does not merely record depth at each location in
the visual field; rather, it actively organizes its depth mea-
surements into functionally valuable units. In the previous
section, we discussed how occlusion plays a key role in
this organization. In this section, we discuss how the
visual system handles what is arguably the hardest problem
posed by occlusion: the visual representation of structures
that are hidden and are therefore completely invisible. If
seeing depth is about representing the actual layout of
objects in the environment, then all portions of the objects
must be represented, even those that are hidden from
view: hidden portions do not disappear from the environ-
ment just because they do not appear in the image. There-
fore, the visual system has to go beyond local image data
to construct representations of hidden structures. We will
now discuss how the environmental conditions of occlusion
and camouflage predict properties of the construction
process.
M  A C We will consider two
major ways in which parts of the scene can become
invisible. The first is simple occlusion, in which an opaque
object obscures part of a more distant object. When
this happens, the occluded structures of the more distant
object have no corresponding features in the image, and thus
the visual system must somehow “reconstruct” the missing
data. The second way that viewing conditions can lead to
invisible structures is through camouflage. In camouflage
it is the nearer, occluding surface that is rendered invisible
because it happens to match the color of its background.
Because the boundaries of the camouflaged object do
not project any contrast, they have no corresponding
features in the image, and thus the nearer object is effec-
tively invisible. Under these circumstances, the visual system
must actively “hallucinate” the invisible structures. In both
cases, the visual system interpolates missing data, a process
known as visual completion. This process is important to
depth perception because it is one of the means by which
the visual system organizes its depth measurements into
meaningful bodies. We argue that depth perception and unit

(a)
(b) (c)
F 86.5. a, Modal completion. Most observers report seeing
a vivid white triangle in front of three discs and a black triangular
outline. The contours of the white triangle are subjectively distinct,
resembling real contours, even though there is no corresponding
image contrast, and hence the triangle is illusory. b, Amodal com-
pletion. Most observers report seeing a single continuous black
shape, part of which is hidden from view by the gray occluder, even
though the parts that are hidden from view are, by definition, invis-
ible. c, A self-splitting object. Even though the shape is uniformly
black, it tends to be seen as two forms, one in front of the other.
Which form tends to complete modally, and which amodally,
depends in part on the distance that must be spanned by the com-
pletion (Petter’s law).
formation are intimately intertwined, for depth constrains
the perceptual units that are formed, and perceptual
organization influences the interpretation of local depth
measurements.
The phenomenal quality of completed structures differs,
depending on whether it is near (camouflaged) or far
(occluded) structures that are interpolated. In the case of
camouflage, the interpolation leads to a distinct impression
of a contour or surface across the region of missing data.
This is referred to as modal completion (Michotte et al.,
1964/1991) because the experience is of the same phe-
nomenal modality as ordinary visual experience. An illusory
contour, for example, is crisp and subjectively similar to a
real contour, as can be seen in Figure 86.5A. In contrast, the
sense of completion experienced with occluded structures is
less distinct. The black form in Figure 86.5B tends to be seen
as a single object, part of which is hidden, rather than as
two distinct objects, whose boundaries coincide with the
boundary of the gray occluder. There is a compelling sense
that the two visible portions of the black form belong to the
same object and that that object continues in the space
behind the occluder. However, this impression, although
visual in origin, is not of the same phenomenal mode as
normal and modal contours and is therefore referred to as
  :    
amodal completion (Michotte et al., 1964/1991). In general, the
regions of the image which are visible, and lead to visual
completion, are referred to as inducers.
T I H There is a vast literature on
visual completion; a thorough discussion of all the issues is
beyond the scope of this chapter. One important issue that
is discussed in greater detail in Chapter 75 is whether visual
completion occurs relatively early or late in the putative pro-
cessing hierarchy. However, the perceptual organization of
depth has a direct bearing on another current debate, specif-
ically the extent to which modal and amodal completion are
the consequences of a single process. This issue is intimately
bound to depth perception because it determines the extent
to which depth processing and perceptual organization are
independent.
The debate runs roughly as follows. On the one hand, a
strong claim has been made that a single mechanism is
responsible for both modal and amodal completion. Accord-
ing to this account, perceptual organization (including visual
completion) produces perceptual units, and an independent
process places those units in depth. The theory states that
psychological differences between modal and amodal com-
pletion result from the final depth ordering of the completed
forms (Kellman and Shipley, 1991; Kellman et al., 1998;
Shipley and Kellman, 1992) rather than a difference
between the completion processes themselves. This is known
as the identity hypothesis. On the other hand, the two processes
could be largely independent, subject to different constraints
and subserved by distinct neural mechanisms. A strong
form of this dual mechanism hypothesis would be that the
two processes are fundamentally different in nature—for
example, that modal completion is largely data driven,
while amodal completion is essentially cognitive. To anticipate,
although we do not subscribe to the strongest form of
the dual mechanism hypothesis, we will provide evidence
that modal and amodal completion follow different
constraints and argue that they are subserved by distinct
neural processes. Central to the arguments that we present
are the geometric and photometric conditions under
which occlusion and camouflage actually occur in the
environment.
The principal evidence for the identity hypothesis has
been that subjects perform similarly with modally and
amodally completed figures in a variety of tasks. In one task,
Shipley and Kellman (1992) varied the spatial alignment of
the inducing elements in both modally and amodally com-
pleted squares. Such misalignment is known to weaken the
sense of completion, as the completed boundary is forced to
undergo an inflection. Subjects were asked to rate the sub-
jective strength of visual completion as a function of the
degree of misalignment for modal and amodal versions of
the display. Shipley and Kellman (1992) found that ratings
1289
declined at the same rate as a function of misalignment for
both modal and amodal figures. This has been interpreted
as evidence that a single mechanism is responsible for both
forms of completion.
Using a more rigorous method, Ringach and Shapley
(1996) performed a shape discrimination task with modal
and amodal versions of a Kanizsa figure. By rotating the
inducing elements, the vertical contours of the completed
square can be made to bow out (creating a “Fat” Kanizsa)
or curve in (creating a “Thin” Kanizsa). Subjects were asked
to discriminate between Fat and Thin versions of the display
while the angle through which the inducers were rotated was
varied. Ringach and Shapley found that discrimination per-
formance as a function of rotation was nearly identical for
modal and amodal versions of the display, a finding which
is consistent with the identity hypothesis.
One problem with this type of evidence is that it relies on
negative results, that is, a failure to detect a difference, which
could be due to the method rather than to a fundamental
property of the system being studied. If positive evidence
could be provided that modal and amodal completion are
subject to different constraints, or result in different percep-
tual units, then the identity hypothesis would no longer be
tenable.
There are two major reasons for believing that modal
and amodal completion should be subject to different con-
straints, both of which are related to the environmental
conditions under which occlusion and camouflage occur.
First, occlusion occurs over greater distances across images
because it only requires that one object be in front of
another. Camouflage, on the other hand, requires a perfect
match in color between the near surface and its background,
and thus occurs less frequently in general. This difference is
reflected in a constraint on the image distances over which
modal and amodal completion occur, which was first docu-
mented by Petter (1956). Petter used a class of stimuli now
known as spontaneously splitting objects (SSOs), which consist of
a single homogeneously colored shape, such as the one
shown in Figure 86.5C, that tends to be interpreted as two
independent shapes, one behind the other. Which object is
seen in front tends to oscillate with prolonged viewing.
However, which shape is seen in front first, and which tends
to be seen in front for a greater proportion of the time, can
be predicted rather well from the lengths of the contours
that must be interpolated. Petter’s rule states that longer con-
tours tend to be completed amodally, while shorter contours
tend to be completed modally. Thus, which figure is seen in
front can be predicted from the length of the contours that
must be completed. If the two types of completion are
subject to different constraints on the distances over which
they occur, this opens the possibility that they are subserved
by different mechanisms.
1290 , ,   
A second reason for believing that modal and amodal
completion are subject to different constraints relates to
the color conditions that are required for occlusion and
camouflage to occur. Again, occlusion can happen between
objects of any color. The reflectance of the near object is
unrelated to the fact that it hides the more distant one from
view. This suggests that amodal completion should not
be sensitive to the luminance relations between the image
regions involved. Camouflage, by contrast, requires a
perfect match in luminance between the near and far sur-
faces. This implies that modal completion should be sensitive
to the luminance relations between the image regions
involved.
Recent experimental work has shown that this luminance
sensitivity can lead to large differences between modal
and amodal displays (Anderson et al., 2002). Anderson et al.
created displays consisting of two vertically separated
circles filled with light and dark stripes, as shown in Figure
86.6. The binocular disparity of the circles was kept con-
stant, but the disparity of the light-dark contours inside the
circles was altered to place the stripes behind or in front of
the circular boundaries. When the stripes were further than
the circles, the top and bottom stripes tended to complete
amodally to form a single continuous dark and light surface,
which appeared to be visible through two circular holes, as
schematized in Figure 86.6D. This percept occurred irre-
spective of the luminance of the region surrounding the
circles.
By contrast, when the disparity placed the contours in
front of the circles, the dark and light stripes separated
into different depth planes. The way in which the stripes
separated from one another depended on the luminance of
the surround. When the surround was the same color as the
light stripes, the light stripes appeared to float in front and
completed modally across the gap between the two circles.
In this condition, the dark stripes completed amodally
underneath the light stripes to form complete circles. This
led to an impression of light vertical stripes in front of
dark circles, as schematized in Figure 86.6E. However,
when the surround was the same luminance as the dark
stripes, the percept inverted, such that the dark stripes ap-
peared to float in front of light discs. This demonstrates
a fundamental dependence on luminance that was not
present in the amodal version of the display. Furthermore,
if the surround was an intermediate gray, then the display
was not consistent with camouflage, as neither the light
nor the dark stripes perfectly matched the luminance of the
background. Under these conditions, there was no modal
completion across the gap, and the percept was difficult
to interpret. This demonstrates that modal completion is
sensitive to luminance relations, while amodal completion
is not.
(a)
(d) Amodal depth ordering
(b)
(e) Modal depth ordering
(c)
F 86.6. Demonstration of dependence of modal comple-
tion on surround luminance. When the left stereopairs of a, b, and
c are cross-fused, the stripes tend to complete amodally between the
gaps between the circular hole, creating the impression of a single
striped surface (like wallpaper) viewed through two apertures, as
depicted in d. This occurs irrespective of the luminance of the sur-
round. However, when the right stereopairs are cross-fused, thus
inverting the disparity, only two stripes appear to complete modally,
and which stripes complete depends critically on the surround
luminance, as depicted in e. When the surround is dark, as in a, the
dark stripes complete modally; when the surround is light, as in b,
the light stripes complete modally; and when the surround is inter-
mediate, no completion is visible. This demonstrates that modal
completion is luminance dependent, while amodal completion is
not. (Adapted from Anderson et al., 2002.)
Anderson et al. showed that this luminance sensitivity
could affect performance on basic visual tasks such as vernier
acuity. The stripes in the top and bottom circles can be
horizontally offset (i.e., misaligned slightly) without destroy-
ing the sense of completion. Subjects were asked to report
in which of two displays the contours were slightly mis-
aligned. Both modal and amodal completion facilitate per-
formance in this task. However, in the amodal case,
performance was unaffected by the luminance of the
surround, while in the modal case, performance was
much worse when the luminance of the surround was an
intermediate gray (the condition in which the stripes do not
complete across the gap). Thus, modal and amodal com-
  :    
pletion are subject to different constraints, both on the dis-
tance over which they occur and on the luminance condi-
tions that are required to induce them. This positive
evidence for a difference between modal and amodal com-
pletion uses essentially the same types of task as the nega-
tive evidence that had previously been used to support the
identity hypothesis.
V C   P O
 D The geometric and photometric differences
between modal and amodal completion are derived
directly from the environmental conditions of occlusion
and camouflage. Because occlusion and camouflage occur
under different circumstances, they have different conse-
quences for the organization of depth into meaningful
bodies. In fact, the differences can be exploited to generate
stimuli in which modal and amodal completion lead to dif-
ferent shapes. This is important, as it shows that unit for-
mation is intimately bound to the placement of structures in
depth.
The greater “promiscuity” of amodal completion is the
key in the generation of these displays. Figure 86.7 is a
recently developed stereoscopic variant of the Kanizsa
configuration in which the inducing elements are rotated
outward (Anderson et al., 2002). When the straight segments
(the “mouths” of the “pacmen”) are placed in front of the
circular portions of the inducers, the impression is of five
independent illusory fragments that float in front of five
black discs on a white background. However, when the
two eyes’ views are interchanged, and thus the straight
contours are placed behind the circular segments, the impres-
sion is rather dramatically altered. With the disparity
inverted, the impression is of a single amodally completed,
irregularly shaped, black figure on a white background,
which is visible through five holes in a white surface
(these percepts are schematized in Figs. 86.7B and 86.7C).
Thus, the former case consists of a total of 11 surfaces (5
fragments + 5 discs + white background), while the latter
case consists of 3 (1 white surface with 5 holes + 1 black
shape + white background). Clearly, the placement in depth
has a considerable effect on what perceptual units are
formed.
Anderson et al. also provided evidence that differences
between modal and amodal interpolation can lead to differ-
ences in the very shapes of completed contours themselves.
When the left-hand stereopair in Figure 86.8A is uncross
fused, the resulting percept consists of six circular discs that
are partly occluded by a jagged white surface on the right-
hand side, as schematized in Figure 86.8B. However, when
the disparities are inverted (by uncross-fusing the right pair
of Fig. 86.8A), the modal completion across the regions
between the four black blobs tends to take the form of a
1291
 (a)
(b) (c)
F 86.7. a, Relative depth alters perceptual organization.
When the left stereopair is cross-fused, the figure tends to appear
as five discs occluded by five distinct image fragments, as depicted
in b; the transparency in b is included only so that both depth planes
can be depicted simultaneously. When the depth ordering is
reversed by cross-fusing the right stereopair, a single irregular black
“star” appears to lie on a continuous white background, which is
visible through five holes in a continuous overlying layer (c). In this
depth ordering the black shape tends to appear as figure. (Adapted
from Anderson et al., 2002.)
continuous wavy contour that runs down the center of the
display. This percept is schematized in Figure 86.8C. The
importance of this demonstration is that it shows that modal
and amodal completion can not only result in different
surface structures, but even in differently shaped contours.
It is difficult to see what the concept of a single completion
mechanism serves to explain if the two processes can result
in different completed forms.
Ultimately, the identity hypothesis is a claim about mech-
anism and can therefore be assessed physiologically. There
is a considerable body of evidence for extrastriatal units that
are sensitive to illusory but not to amodally completed
contours (see Chapter 76 for a review). A critical additional
piece of evidence was provided recently by Sugita
(1999), who found cells in V1 that respond to amodal
completion across their receptive fields but not to modal
completion. Cells responded weakly when presented with
two unconnected edges; holes and occluding surfaces on
their own; and stimuli in which two unconnected edges
were separated by a hole. However, when the cells were pre-
sented with two edge fragments separated by an occluder (a
1292 , ,   
(a)
(b) Serrated edge near (c) Serrated edge far
F 86.8. The serrated-edge illusion. When the left stereopair
in a is uncross-fused, the resulting percept consists of six circular
disks that are partly occluded by a jagged white surface on the right,
as depicted in b. When the right stereopair is uncross-fused, the
modal completion of these four black blobs tends to take the form
of a single wavy contour that runs vertically down the center of the
display, as depicted in c. Although other percepts are possible, this
is an existence proof that depth inversion alone can alter the shape
of modally and amodally completed contours. (Adapted from
Anderson et al., 2002.)
stimulus that leads to amodal completion of the edge), the
cells responded vigorously. This shows that at the earliest
stages of cortical processing, there is a double dissociation
between the representations of modal and amodal struc-
tures, a conclusion which supports the dual mechanism
hypothesis.
Transparency, scission, and the representation of multiple
depth planes
Transparency poses a particularly interesting problem in the
perceptual organization of depth. With transparency, one
object is visible through another, and thus two distinct depths
lie along the same line of sight (Fig. 86.9). If the visual system
is to represent depth in terms of the actual surfaces of the
environment, it has to depict two distinct depths at a single loca-
tion in the visual field. The process of projection compresses

(a) (b)
(c) (d)
(e) (f)
F 86.9. Perceptual transparency. The figure in a tends to be
seen as a light gray transparent surface in front of a bipartite back-
ground, as depicted in b, and thus two distinct surfaces are visible
along the same line of sight. Transparency is seen only when
certain relations hold between the various regions of the display. In
c the central region is higher in contrast than its surround and thus
is not seen as transparent. In d, the polarity of the contrasts is
reversed, and again transparency is not seen. In e, the contour of
the underlying layer is not continuous inside and outside the central
region, eliminating the percept of transparency. In f, the contour
of the overlying layer is not continuous, which also reduces the
percept of transparency.
the light arriving from the transparent surface and the light
arriving from the more distant surface into a single image
intensity on the retina. In order to represent both surfaces,
the visual system has to separate a single luminance value
into multiple contributions, a process known as scission
(Koffka, 1935). We argue that scission is a type of percep-
tual segmentation, as it parses the representation of depth
into distinct surfaces. However, rather than segmenting
neighboring locations into distinct objects, scission separates
depth into layers, or planes, and thus operates “parallel” to
the image plane.
Scission poses the visual system with two principal prob-
lems. The first is to identify when a single luminance results
  :    
from two distinct depths. The second is to assign surfaces
properties correctly at the two depths. By studying when and
how we see transparency, we can learn how the visual system
scissions depth into layers.
Much of the seminal work on perceptual transparency
was conducted by Metelli (1970, 1974a, 1974b; see also
Metelli et al., 1985), who provided a quantitative analysis of
the color mixing that occurs when one surface is visible
through another. When a background is visible through a
transparent sheet, only certain geometrical and luminance
relations can hold between the various regions of the display
(Fig. 86.9). From these relations Metelli derived constraints
that determine whether a region will look transparent or not,
and how opaque it will appear if it does look transparent.
This is important, as it determines the conditions under
which the visual system scissions a single image intensity into
multiple layers, and thus how the visual system stratifies its
representation of depth.
Broadly, the conditions required for perceptual scission fall
into two classes. The first are the photometric conditions for
transparency, which detail the relations between the light
intensities of neighboring regions that are necessary for
scission. The second set of conditions for perceptual scission
are geometrical, or figural. Depth separates into layers only
when these relations hold between the various regions of the
display.
P C  S Consider the
display shown in Figure 86.9A, which tends to be seen as a
bipartite background that is visible through a transparent
filter. The vivid separation of the central region into two
depths occurs only when certain luminance relations hold.
Metelli derived two constraints on the photometric condi-
tions required for perceptual scission.
The intuition behind the first constraint, which we refer
to as the magnitude constraint, is that a transparent medium
cannot increase the contrast of the structures visible through
it. The consequence of this constraint is that the central
diamond must be lower or equal in contrast than its sur-
round in order to appear transparent, as shown in Figure
86.9A. This constraint is important, as it restricts the condi-
tions under which scission occurs: a region can scission only
if its contrast is less than or equal to the contrast of its flank-
ing regions. As can be seen in Figure 86.9C, infringement of
this constraint with respect to the central diamond prevents
the central region from undergoing scission. However, in this
display, the constraint is satisfied for the region surrounding
the diamond; thus, the display can be seen as a bipartite
display viewed through a transparent filter with a diamond-
shaped hole in the center.
The intuition behind the second luminance constraint,
which we refer to as the polarity constraint, is that a transpar-
ent medium cannot alter the contrast polarity of the
1293
(a) (b) (c)
F 86.10. The polarity constraint means that transparency
manifests itself in distinctive local ordinal relations in luminance.
The only difference between the three figures is the luminance
of the region of overlap. In a, the region is dark and the image
is bistable, as either square can be seen in front. When this occurs,
a line that passes progressively from brighter to darker regions
creates a Z shape. In b, the overlap is intermediate, such that
the line that joins regions of decreasing brightness is C-shaped.
When this happens, exactly one of the surfaces appears transpar-
ent. In c, the overlap is light, creating a crisscross pattern. In
this case, neither square appears transparent, as the polarity con-
straint is infringed for both squares. (Adapted from Beck and Ivry,
1988.)
structures visible through it. Put another way, if a dark-light
edge passes underneath a transparent medium, the dark side
will remain darker than the light side, no matter what the
absolute luminances are. As can be seen in Figure 86.9D,
infringement of this constraint prevents perceptual scission,
demonstrating that the visual system respects this optical
outcome of transparency. This constraint is particularly
important in determining the depth ordering in transparent
displays.
The polarity constraint enforces certain restrictions on the
ordinal relationships between the luminances of neighbor-
ing regions. This means that, in principle, we can classify
the locations where neighboring regions meet to determine
whether scission is or is not possible in each region. This pro-
vides the visual system with a local signature of transparency.
Beck and Ivry (1988) noted that if one draws a series of lines
running progressively from the brightest to the darkest
regions, there are three possible shapes that result, as shown
in Figure 86.10. The only difference between the three
figures is the luminance of the region of overlap between
the two squares. In the first instance (Fig. 86.10A), the image
is bistable, as either square can be seen as a transparent
overlay. In these circumstances, the lines linking regions of
1294 , ,   
increasing luminance form a Z configuration. When the
lines form a C shape (Fig. 86.10B), only one of the squares
is seen as transparent, and when the lines crisscross (Fig.
86.10C), the polarity constraint is infringed for all regions
and neither square scissions. Adelson and Anandan (1990)
provided a similar taxonomy based on the number of polar-
ity reversals. A number of lightness illusions demonstrate
that scission can be predicted from the class of X-junctions
in the display and that these X-junctions can have powerful
effects on many qualities of our experience (see, e.g.,
Adelson, 1993, 1999).
The magnitude and polarity constraints can be unified
as a single rule that describes a powerful local cue to scis-
sion. Anderson (1997) phrased the rule as follows: “When
two aligned contours undergo a discontinuous change in
contrast magnitude, but preserve contrast polarity, the lower-
contrast region is decomposed into two causal layers.” There
are two valuable consequences of this rule. The first is that
it unifies the two Metelli constraints. The second is that it
provides a local signature of transparency that can be applied
to any meeting of contours. This includes those T-junctions
that are in fact degenerate X-junctions, that is, those in
which two neighboring regions happen to have exactly the
same luminance. Anderson also demonstrated that a
number of traditional lightness phenomena, including
White’s effect and its variants, and neon color spreading,
can be accounted for as cases of scission rather than as
the consequence of traditional “contrast” or “assimilation”
processes.
Having identified that a location contains two surfaces,
the visual system has to partition the luminance at that
location between the two depths. How much of the light is
due to reflectance of the underlying surface, and how much
is due to the properties of the overlying layer? The opacity
of the overlying layer determines how the luminance is
divided between the two depths. Metelli’s model makes
explicit predictions about the perceived opacity and light-
ness of the transparent layer. The equations predict that
two surfaces with identical transmittance should look
equally opaque irrespective of their lightness. However,
Metelli himself noted that dark filters tend to look
more transparent than light filters with the same transmit-
tance. Why does the visual system confuse lightness and
transmittance in partitioning luminance between two
depths?
In a series of matching experiments, Singh and Ander-
son (2002) recently resolved this issue. Subjects adjusted
the opacity of one filter until it matched the perceived
opacity of another filter with a different lightness. Singh
and Anderson found that perceived transmittance is
predicted almost perfectly by the ratio of Michelson contrasts
inside and outside the transparent region, even though
such a measure is actually inconsistent with the optics of
transparency. As discussed above, there is a general con-
sensus that the early visual processing tends to optimize
sensitivity to contrast rather than absolute luminance.
Hence, in assigning transmittance, the visual system appears
to use the readily available contrast measurements, even
though they are not strictly accurate measurements of
opacity.
F C  S In addition to the lumi-
nance conditions, certain geometrical relations must hold
between the various regions of the display in order for depth
stratification to occur (Kanizsa, 1995/1979; Metelli, 1974a).
These figural conditions fall into two broad classes. The first
class requires good continuation of the underlying layer.
Specifically, the contours that are in plain view should be
continuous with the contours viewed through the region of
presumed transparency. As can be seen in Figure 86.9E,
infringement of this condition interrupts the percept of
transparency. The second figural condition requires good
continuation of the transparent layer. Figure 86.9F shows that
infringement of this condition weakens or eliminates the
percept of transparency.
There are conditions in which the figural cues to trans-
parency are so strong that they can override the luminance
cues. Beck and Ivry (1988) showed subjects displays like
the one shown in Figure 86.10C, in which the region of
overlap between the two figures is the wrong contrast
polarity for either figure to be seen as transparent. Despite
this, naive subjects did occasionally report seeing such figures
as transparent, demonstrating that the sense of figural
overlap is a central aspect of the percept of transparency.
Certainly most observers are willing to agree that the
region of overlap in Figure 86.10C appears to belong to
two figures simultaneously, an impression that can be
enhanced with stereo and relative motion. However, it
should be noted that the gray of the overlap region does not
appear to scission into two distinct sources, at least not in the
same way as the overlap of a normal transparency display
does (as in Figs. 86.10A and 86.10B). This leads to the
possibility of two distinct neural processes in the percep-
tion of transparency. One is driven by relatively local
cues and leads to phenomenal color scission. The other is
driven by more global geometrical relations and leads to
stratification in depth. Under normal conditions of trans-
parency, the two processes operate in concert to produce the
full impression of transparency. However, using carefully
designed cue-conflict stimuli, such as those used by Beck and
Ivry, these two factors in the representation of transparent
surfaces can be distinguished. An open question, however, is
how these processes are instantiated neurally. All we can
conclude is that the representation of depth is much more
  :    
sophisticated than a mere two-dimensional map of depth
values.
S   P O  D
Scission can have pronounced effects on perceptual organi-
zation. For example, Stoner et al. (1990) demonstrated that
perceived transparency can alter the integration of motion
signals into coherent moving objects. When a plaid is drifted
at constant velocity across the visual field, it is typically
seen as a single coherent pattern that moves at the velocity
of the intersections between the two component gratings.
However, with prolonged viewing, the plaid appears to sep-
arate into two component gratings that slide across each
other, each of which appears to move in the direction per-
pendicular to its orientation. When the plaid is coherent, it
appears to occupy a single depth plane, but when it sepa-
rates into its components, the gratings tend to appear at dif-
ferent depths.
Stoner et al. varied the intensity of the intersections of the
plaids and measured the proportion of time for which
the plaid was seen as coherent. They found that when the
color of the intersection was consistent with one grating
being seen through the other (i.e., when the junctions
were consistent with transparency), the proportion of the
time for which the plaid appeared to separate into gratings
was greatly increased. By contrast, when the color of the
intersections infringed the polarity constraint, such that
neither grating could be seen as transparent, the pattern
tended to be seen as a coherent plaid rather than undergo-
ing scission into distinct layers. This demonstrates that
scission has important consequences for the representation
of visual structure. When an image region scissions, the
effects can spread to regions distant from the local cues to
scission.
Scission acts as a nexus between depth and other visual
attributes. Scission of depth can cause regions to change in
apparent lightness; conversely, changes in luminance can
cause changes in depth stratification. Figure 86.11 demon-
strates this close relationship between luminance, scission,
and the perceptual organization of depth. Three circular
patches of a random texture were placed on a uniform back-
ground. Critical to the demonstration is that disparity is
introduced between the circular boundaries and the texture
inside the circles. When the disparity places the texture
behind the circular boundaries, the circles appear as holes
through which the texture is visible. The texture tends to
appear as a single plane with continuously stochastically
varying lightness. However, when the disparity places the
texture in front of the circular boundaries, the percept
changes considerably. The texture separates into two distinct
layers: a near layer made up of clouds with spatially varying
transmittance and a far layer that is visible through the
1295

F 86.11. Scission and the perceptual organization of depth.
The top and bottom figures are identical apart from the brightness
of the surround. When the right stereopair is cross-fused, the
figure appears as a single textured plane that is visible through
three circular holes. This is seen irrespective of the luminance of
the surround. However, when the disparity is reversed (by cross-
fusing the left stereopair), the texture appears to separate into two
depth planes. The near layer contains near clouds that vary
spatially in thickness or opacity. Through these clouds can be seen
three more distant disks, which appear more or less uniform in
lightness. For this depth ordering, the structure completely reverses
with a change in the luminance. In the top case, the dark portions
of the texture form the clouds; in the bottom case, the light por-
tions of the texture form the clouds. Scission makes these percepts
possible by allowing the visual system to separate the intermediate
grays into two distinct contributions. (Adapted from Anderson,
1999.)
clouds, which consists of uniform discs on a uniform
background (see Anderson, 1999; Anderson, in press).
Another interesting property of this display is that the
lightness and spatial structure of the clouds and discs reverse
completely when the luminance of the surround varies. In
Figure 86.11, the top and bottom displays are identical
except for the lightness of the surround. When the surround
is dark, the texture scissions into dark, smoke-like clouds in
front of white discs. However, when the surround is white,
it is the light portions of the texture that move forward, float-
ing like mist in front of dark discs. One final observation
about the display is that when the texture carries near dis-
parity, and thus undergoes scission, the clouds that float in
front tends to complete modally across the gaps in between
the discs. This is in part due to the fact that the conditions
for camouflage are satisfied, as discussed in the second
section.
When the depth is reversed in the display, two asymme-
tries occur. The first is geometrical in that it alters the struc-
ture of the depths in the scene. In the near case the texture
scissions into two layers, while in the far case the texture
1296 , ,   
appears relatively uniform in depth by comparison. The
second asymmetry that occurs with depth inversion is photo-
metric in that it is driven by the luminance of the surround
and determines the lightness of the cloud and discs. When
the texture is distant, the percept changes very little with
changes to the luminance of the surround; by contrast, when
the texture is near, the luminance of the surround critically
determines how the scission occurs, as well as the lightness
of the cloud and discs. In what follows, we will use the CDAP
discussed in the first section and the concept of scission to
explain theses asymmetries. For a more thorough discussion,
see Anderson (in press).
Let us first consider the case in which the texture carries
far disparity relative to the circular boundaries. Because the
texture is continuously varying in luminance, it carries local-
izable disparity signals at almost every location. Put another
way, if disparity is carried by contrast, as argued in the first
section, then patterns that are richly structured bear the
densest distribution of disparities. Recall that the CDAP
requires both sides of every contrast to be at least as distant
as the disparity carried by the contrast. This means that
when the texture is given far disparity (or, more precisely,
when the contrasts of the texture are given far disparity), both
the light and dark matter in the texture recede to this depth.
In turn, the depth placement of the texture uniquely deter-
mines the border ownership of the boundaries of the discs,
which carry relatively near disparity. If the insides of the discs
(i.e., the texture) carry far disparity, then the outsides (i.e., the
region surrounding the discs) must be at the depth carried
by the circular boundary. Thus, the circles are seen as holes
in the surrounding surface; it is through these holes that the
texture is visible.
The situation is more complex when the depth is reversed,
that is, when the contrasts of the texture are nearer than the
contrast of the circular boundaries. Crucial to the following
argument is that it is contrasts that carry disparity, while it is
the light and dark regions that make up the contrasts to
which depth is assigned. First, let us consider the circular
boundary between the surround and the texture. When
the surround is light, it is the dark portions of the texture
(inside the circles) that contrast with the surround. Thus, the
disparity of the circular boundary is carried by the contrast
between the light matter of the surround and the dark
portion of the textured disc. The CDAP requires both of
these regions to be at least as distant as the disparity carried
by the boundary. This means that the light surround is
dragged back to this depth, and the dark matter of the
texture is also dragged back to this depth. Now consider the
contrasts between the dark and light portions within the
texture. These contrasts carry relatively near disparity. But
the contrast between the dark matter and the surround has
already constrained the dark matter to be at least as distant
as the circular boundary. This means that it must be the light
matter of the texture that is responsible for the near disparity
of the texture, that is, the light matter is a near surface
that partly obscures the dark matter. This explains why
the texture splits into two depths: the dark matter is drag-
ged back by forming a contrast that carries far disparity
(i.e., the boundary of the disc) and the light matter floats
in front, as its boundaries with the dark matter carry near
disparity.
The final logical step in the explanation involves scission.
The texture does not consist of only two luminances, but of
a continuous range of luminances from light to dark. How
can we explain the appearance of the intermediate lumi-
nances in the texture? Scission makes it possible to separate
the intermediate luminances into two distinct components:
dark “stuff ” and light “stuff,” which have been compressed
into a single luminance by the process of projection onto the
retina. These two components lie in different depth planes.
Put another way, scission allows the visual system to inter-
pret the gray regions as dark matter viewed through light matter.
The critical insight is that it is the dark stuff in the texture
that forms the contrast with the surround. Therefore, all of
the dark stuff belongs to the more distant depth, including
the dark stuff within the grays. All of the remaining light-
ness in the grays belongs to the transparent clouds that float
in front of the discs. In this way, the intermediate luminances
are interpreted as varying degrees of transmittance of the
overlying layer. The lighter the gray, the thicker the cloud;
the darker the gray, the sparser. This explains why the disc
appears as a uniform dark disc: all of the dark is “sucked
out” of intermediate regions and is dragged back to form
the disc. The “leftover” lightness is attributed to the trans-
parent clouds.
The whole argument reverses when we change the sur-
round from light to dark. When the surround is dark, it is
the light portions of the texture that contrast with the sur-
round, and therefore, it is the light portions of the texture
that are dragged back. The near disparity of the texture
must therefore be due to the dark regions, and thus dark
clouds are seen to float in front of white discs. Again, as it
is the whiteness of the texture that is dragged back, all of the
whiteness in the intermediate luminances is attributed to the
more distant discs. The remaining darkness in the grays is
attributed to the dark clouds that float in front. In this way,
changing the luminance of the surround changes which
contrasts carry the disparities and thus which regions are
dragged back by virtue of the CDAP. Scission enables the
visual system to separate luminances into multiple contribu-
tions and thus segment the intermediate grays into two dis-
tinct depth planes.
This demonstration and others like it are important, as
they show how multiple processes interface to determine our
  :    
percepts of depth and material quality. It is through the
CDAP and scission that the visual system interprets local
variations in luminance as meaningful surfaces located in
depth. Depth stratification complements traditional seg-
mentation as an important process through which the visual
system organizes its representation of depth into ecologically
valid structures.
Conclusions
It is common to think that depth perception involves
little more than determining the depth at each location in
the visual field. We have argued, to the contrary, that the
visual system mirrors the structural organization of the
environment by tying its representation of depth to
surfaces and objects. Thus, depth perception is an active
process of perceptual organization, as well as a passive
process of acquiring depth estimates. We have argued
that luminance, disparity, and contrast are some of the
basic image features that carry local information about
depth, while scission, visual completion, and the CDAP
are some of the means by which depth is organized into
surfaces.
In the first section, we introduced the CDAP and argued
that
1. Disparity is carried by local contrasts (e.g., luminance
edges) but depth is assigned to the regions that meet to form
the contrasts.
2. Occlusion introduces a critical constraint on the inter-
pretation of local disparity signals, the CDAP. This con-
straint requires that both sides of a contrast are at the depth
specified by the contrast, or one side could be a more distant
occluded surface. In the latter case, the disparity determines
the depth of the occluding side.
3. The CDAP imposes a fundamental asymmetry
between near and far structures. When simultaneously
applied to all edges in a display, the CDAP can explain
a number of asymmetrical changes in perceived surface
layout that occur with simple inversion of the disparity
field.
In the second section, we discussed how the visual system
deals with structures that are invisible because they are
hidden by occlusion or camouflaged against their back-
ground. We argued that
1. The visual system has to actively complete the
missing data if it is to segment depth accurately into
objects.
2. Consideration of the environmental conditions of
occlusion and camouflage predicts (a) that modal comple-
tion is sensitive to luminance, while amodal completion is
1297
not, and (b) that modal completion tends to occur over
shorter distances than amodal completion.
3. As predicted from the environmental differences, dis-
tinct mechanisms are responsible for the two types of com-
pletion. The differences can be used to generate displays in
which the completed forms differ when the disparity field is
inverted.
Finally, in the third section, we discussed how scission
allows the visual system to represent two depths along the
same line of sight and thus organize depth into layer. We
argued that
1. Certain luminance and figural relations must obtain in
order for a region to undergo scission.
2. Scission can have pronounced effects on perceptual
organization in regions distant from the local signatures of
transparency.
REFERENCES
Adelson, E. H., 1993. Perceptual organization and the judgment
of brightness, Science, 262:2042–2044.
Adelson, E. H., 1999. Lightness perception and lightness illusions,
in The New Cognitive Neurosciences (M. Gazzaniga ed.-in-chief ),
Cambridge, MA: MIT Press.
Adelson, E. H., and P. Anandan, 1990. Ordinal characteristics of
transparency, Boston: AAAI-90 Workshop on Qualitative Vision,
July 29.
Anderson, B. L., 1997. A theory of illusory lightness and trans-
parency in monocular and binocular images: the role of contour
junctions, Perception, 26:419–453.
Anderson, B. L., 1999. Stereoscopic surface perception, Neuron,
24:919–928.
Anderson, B. L., in press. The role of occlusion in the perception
of depth, lightness, and opacity, Psychological Review, (in
press).
Anderson, B. L., and K. Nakayama, 1994. Towards a general
theory of stereopsis: binocular matching, occluding contours and
fusion, Psychol. Rev., 101: 414–445.
Anderson, B. L., M. Singh, and R. W. Fleming, 2002. The
interpolation of object and surface structure, Cogn. Psychol.,
44:148–190.
Beck, J., and R. Ivry, 1988. On the role of figural organiza-
tion in perceptual transparency, Percept. Psychophys., 44:585–
594.
Bruce, V., P. R. Green, and M. A. Georgeson, 1996. Visual Percep-
tion, 3rd ed., Hove, East Sussex, UK: Psychology Press.
Cornsweet, T. N., 1970. Visual Perception, New York: Academic
Press.
DeValois, R. L., and K. K. DeValois, 1988. Spatial Vision, New York:
Oxford University Press.
Hartline, H. K., 1940. The receptive fields of optic nerve fibres,
Am. J. Physiol., 130:690–699.
Howard, I. P., and B. J. Rogers, 1995. Binocular Vision and Stereopsis,
New York: Oxford University Press.
Hubel, D. H., and T. N. Wiesel, 1962. Receptive fields, binocular
interaction and functional architecture of monkey striate cortex,
J. Physiol., 160:106–154.
1298 , ,   
Jones, J., and J. Malik, 1992. A computational framework for deter-
mining stereo correspondence from a set of linear spatial filters,
Image Vis. Comput., 10:699–708.
Julesz, B., 1960. Binocular depth perception of computer gener-
ated patterns, Bell Syst. Tech. J., 39:1125–1162.
Julesz, B., 1971. Foundations of Cyclopean Perception, Chicago: Univer-
sity of Chicago Press.
Kellman, P. J., and T. F. Shipley, 1991. A theory of visual interpo-
lation in object perception, Cogn. Psychol., 23:141–221.
Kellman, P. J., C. Yin, and T. F. Shipley, 1998. A common mech-
anism for illusory and occluded object completion, J. Exp. Psychol.
Hum. Percept. Perform., 24:859–869.
Kanizsa, G., 1955/1979. Organization in Vision, New York: Praeger.
Koffka, K., 1935. Principles of Gestalt Psychology, Cleveland:
Harcourt, Brace and World.
Marr, D., and T. Poggio, 1976. Cooperative computation of stereo
disparity, Science, 194:283–287.
Marr, D., and T. Poggio, 1979. A computational theory of human
stereo vision, Proc. R. Soc. Lond B, 204:301–328.
Metelli, F., 1970. An algebraic development of the theory of per-
ceptual transparency, Ergonomics, 13:59–66.
Metelli, F., 1974a. The perception of transparency, Sci. Am.,
230:90–98.
Metelli, F., 1974b. Achromatic color conditions in the perception
of transparency, in Perception: Essays in Honor of J. J. Gibson (R. B.
MacLeod and H. L. Pick, eds.), Ithaca, NY: Cornell University
Press.
Metelli, F., O. da Pos, and A. Cavedon, 1985. Balanced and unbal-
anced, complete and partial transparency, Percept. Psychophys.,
38:354–366.
Michotte, A., G. Thines, and G. Crabbe, 1964/1991.
Amodal completion of perceptual structures, in Michotte’s
Experimental Phenomenology of Perception (G. Thines, A. Costall,
and G. Butterworth, eds.), Hillsdale, NJ: Erlbaum, pp. 140–
167.
Nakayama, K., S. Shimojo, and G. H. Silverman, 1989.
Stereoscopic depth. Its relation to image segmentation, group-
ing, and the recognition of occluded objects, Perception, 18:55–
68.
Palmer, S. E., 1999. Vision Science, Cambridge, MA: MIT Press.
Petter, G., 1956. Nuove ricerche sperimentali sulla totalizzazione
percettiva, Riv. Psicol., 50:213–227.
Pollard, S. B., J. E. W. Mayhew, and J. P. Frisby, 1985. A stereo cor-
respondence algorithm using a disparity gradient limit, Perception,
14:449–470.
Prazdny, K., 1985. Detection of binocular disparities, Biol. Cybern.,
52:93–99.
Ratliff, F., 1965. Mach Bands: Quantitative Studies on Neural Networks in
the Retina, San Francisco: Holden-Day.
Ringach, D. L., and R. Shapley, 1996. Spatial and temporal prop-
erties of illusory contours and amodal boundary completion, Vis.
Res., 36:3037–3050.
Shipley, T. F., and P. J. Kellman, 1992. Perception of partly
occluded objects and illusory figures: evidence for an
identity hypothesis, J. Exp. Psychol. Hum. Percept. Perform., 18:106–
120.
Singh, M., and B. L. Anderson, 2002. Toward a perceptual theory
of transparency, Psychol. Rev., 109(3):492–519.
Smallman, H. S., and S. P. McKee, 1995. A contrast ratio
constraint on stereo matching, Proc. R. Soc Lond. B, 260:265–
271.
Sperling, G., 1970. Binocular vision: a physiological and neural
theory, Am. J. Psychol., 83:461–534.
Stoner, G. R., T. D. Albright, and V. S. Ramachandran, 1990.
Transparency and coherence in human motion perception,
Nature, 344:153–155.
Sugita, Y., 1999. Grouping of image fragments in primary visual
cortex, Nature, 401:269–272.
Takeichi, H., T. Watanabe, and S. Shimojo, 1992. Illusory occlud-
ing contours and surface formation by depth propagation, Per-
ception, 21:177–184.
Wallach, H., 1948. Brightness constancy and the nature of achro-
matic colors, J. Exp. Psychol., 38:310–324.

  :     1299