Slide 1

An Empirical Vie w

Slide 2

Que stions
How do we se ? e Why is conte im xt portant? How do we se color? e How do we se depth? e How doe this a re ? s ll late

How do we see? Why is context important? How do we see color? How do we see depth? How does this all relate/why should I care  tie to purves, or should this be done at the beginning?

Slide 3

How do we see ?
Jacob talks about brain mechanisms of seeing color


Slide 4

How do wesee rea , lly?

But I don’t know Jacob, that might be good description of some of the mecahanisms behind vision, but I want to know how we really see. Everyone think back to when you were a little kid, and try to remember the first idea you had about our eyes, and how they see. What do you remember? When we thought of seeing, we thought of the retina like a camera, taking a picture of the scene in front of us, and decoding wavelengths of light to determine color. I guess this is more of a reconstructionist point of view. But we are going to see that it does not work that way, that context is important. So before we jump into color, I want to talk about the first group of people to realize that context is important in vision, the gestalt theorists.

Slide 5

Gestalt Theory
The first the that re ory cognize the d im portance of conte xt

This theory was developed in the 1920’s, and these guys were the first group of psychologists to systematcially study perceptual organisation . I think they were a bunch of German guys :P So we’ll just run through these principles quickly because they serve as a good starting base to show how important context is in deciphering what we see, and then connecting this idea to color and depth. And I think thee are fun. There are six principles

Slide 6

Principle 1:

Proxim ity

Principle one relates to proximity, Who can tell me what different groups they see on this screen? 1 and 2 3 and 4 3 groups dots in three lines What happens with the evenly spaced dots? So on the left we see horizontal groups, and on the right we see vertical groups. So the first principle of proximity or contiguity says that things that are closer together are seen as belonging together.

Slide 7

Principle 2:

Sim ilarity

Principle 2 relates to similarity. Now where do we see the groups? THat’s riiight! We tend to see groups that have similar characteristics, so blacks and white. And how do we see these groups arranged? In lines… The principle of similarity says that things that share visual characteristics like shape, size, color, texture, value or orientation are seen as belonging together.

Slide 8

Principle 3:

Com on Fate m

Principle three is basically combining the first two, it’s called common fate. This is a subtle illusion, but cool nonetheless. When both the principles of proximity and similarity are used, then we actually see a little bit of movement/. Do you guys see that? Why do you think that is? I think it’s because we jump back and forth from grouping based on proximity, and grouping based on similarity. So I want to group the ones on the top based on proximity, but not this guy down here, but then my mind shifts to wanting to group based on similarity, which includes them all.

Slide 9

Principle 4:

Good Continuation

The fourth principle relates to the idea that we prefer to see things as unified. So in this picture we actually have 4 lines, but instead of seeing A to O and O to D, we just see A to D, and the same with instead of seeing C to O and O to B we see C to B lines. So this is called the principle of continuity, and it predicts our preference for continuous figures. Theres no way that there are 4 crosses lines here, there are two crossed lines.

Slide 10

Principle 5:


Principle five is related to the last one in the idea that not only to we want continuity, we want to see closed, simple figures. So our visual system fills in the gaps. So the circle at the top its easy to see this, especially if you look slightly away. In the second figure we see two overlapping rectangles as opposed to two rectangles with chunks cut out touching corners, or you could just see three shapes touching, which I actually had a hard time seeing. The third can be looked at like a curve with three squares, or just as three random, irregular shapes touching. What do you guys see?

Slide 11

Principle 6:

Are and Sym e a m try

The sixth and last principle, the principle of area and symmetry, says that the smaller of two overlapping figures is perceived as a figure while the bigger one is seen as ground. And with symmetrical figures we tend to see closed figure, and the Symmetrical contours tend to separate the defined figure from the ground. And we will be looking at some cool examples of this a bit later.

Slide 12

Color Pe rception
An Empirical Vie w

So let’s apply this stress on context first to color! some of the demonstrations I am going to show you today, and asked us to think a little outside of the box, so to speak.

Slide 13



Before we jump in I want to refresh your memory and define some terms that are important to know when talking about color… hue, saturation, and brightness. Hue is like is it red, is it blue, etc. Saturation is the INTENSITY of a specific color, which may look familiar to those of you that like to play around in photoshop, the more highly saturated is towards the top, and brightness is the luminesence of a visual target, which is a little hard to show on a computer screen.

Slide 14

thecolor a obse r se s is e n rve e ntire de rm d by the ly te ine proba bilitydistribution of the possiblesource of the s stim ulus Conte is im xt portant!

EMPIRICAL STRATEGY OF PERCEPTION: So purves applied this empirical strategy of vision on color, luminesence, and suggested that it could be applied to all aspects of perception. Im going to focus exclusively on color, color constancy and color contrast and try to argue that the color an observer sees is entirely determined by the probability distribution of the possible sources of the stimulus So let’s dumb this down a bit to a jargon that I can understand. we have a scene, and we have light, and that makes us see color, right? Well one thing that makes this perception so hard is the fact that they are influenced by the context of the scene. So let’s first look at a broad example.

Slide 15

Why is context importa nt?

So first I want to try and convince you that context is important. This picture answered that really well for me. So these stimuli, pointed out in the diagram at the bottom, appear to differ in respect to distance from the viewer, luminesence, surface reflectivity, and obviously, color. But guess what… we use a little photoshop magic to get rid of the rest of the stuff and… they really are the same

Slide 16

Color Contrast

So let’s try to explain this phenomenon… and I’m going to start by talking about color contrast. Here you see two targets on an orangish and a nasty yellow background, and how do they look to you in terms of color? So I know that you know that althugh they look different, I am going to reveal to you that we are actually looking at the same color. Ta da! They only look difference is in the context of the two targets. Differences aren’t in the color, but in the perceived hue and saturation of the targets in the context of the background. I chose this example in particular because these colors arent different in terms of brightness and luminesence, so we are seeing a phenomenon just related to hue and saturation, color contrast.

Slide 17

Color Constancy

So that was color contrast, conversely we have color constancy. So id like to draw your attention to these squares on the left, and these on the right. What color do they look like to you guys? (I guess that is sort of leading you on by saying color… because when we add a mask, we see that they are very different colors, so this Is called color contansy, and that is taking different colors and putting them in different contexts so they look similar (example) So This presents an obvious problem for explanations of color vision in physiological terms. Obviously whatever is there isnt just taken a shapshot of and uploaded into our brains.

Slide 18

So when we have a situation of the same spectral targets eliciting different color sensations Land, who came up with the first theory of color perception called retinex theory in the 1980’s, said that this could be ascribed to 'adaptation' of the color system to the average spectral content of the overall stimulus or to computations of spectral ratios across chromatic contrast boundaries. And here is an example of that. Focus your eyes on this black dot, and Ill time 30 seconds. So what did you guys see?? That’s right, so you saw a lack of green in the picture on the left and a lack of red in the picture on the right, and then it took you a bit a staring to realize that they are actually exactly the same. The idea is that our visual system adapted to the red and the green at the input stages. (show counterexample picture with t t) So great, Land say that we adapt, but this idea doesn’t provide a good explanation, or biological rationale really.

Slide 19

Color Contra a constancy: an e pirical e na st nd m xpla tion

Phyloge tic ne

So let’s bring a cool guy named Purves into the picture, and give an explanation in empirical terms for color constancy and color contrast. Now the idea behind the empirical strategy of vision is that everything that we see is dependent on the historical success or failure of the interactions between what I see and me. So when I look at a scene the way that I see it is dependent on my past perceptions. So you can think about it as your brain seeing what probabilistically makes most sense in the environment. So it isn’t that my mind breaks it up into little pieces and shoots everything out like a camera, but that what we see is literally and figuratively colored by what we’ve percevied in the past. So let’s give an emprical explanation of color contrast (and constancy). So Id like to argue that the visual stimuli that I’ve shown you up to thi point, are actually pretty ambiguous in the sene that many combinations of reflectances, conditions of illumination and influences of transmittance could generate the same perception. It is the job of the visual system to take in all of these patterns that are returned to the eye and create a behavioral response that makes sense. What we see is based on the successes and failures of what we have seen before. I like to think of the idea like a neural network, that you have back propogration and the weights of the nodes are adjusted to promote more appropriate reactinos to the stimuli.

We can call this a phylogenetic process, phylogenetic meaning there is evolutionary relatedness amongst members of a species., you can sort of think of it like lineage. You experience a visual stimuluus, there is a certain neuronal response, and those responses get linked with specific spectral profiles so they have significance. So this means that then, the pattern of neuronal activity that you experience in response to a visual stimulus, the strength of that network, is determined by the number of occurrences of the combinations of colors, reflectances, illuminants and transmittances that have lead to successful behavior in response to that visual stimulus in the past. Ok, methinks we need more examples!

Slide 20

So Jacob is going to talk about brightness and luminesence in a second, and I think I’d like to use an example with color and some brightness to explain this idea of how past experiences influence what we see. So if this is true, then when we put targets of the same color on two contexts, two different colors, we might experience them differently based on their context. So this is very cool, here we have similar surfaces under similar illuminants, and you can see that the purple target in the middle looks, purple in both cases.

Slide 21

Here we have the exact same surfaces BUT they are seen under different illuminants, the left one a blue light, and the right one a purple light. What do you guys see now? Yeah I see purple on the left and more of a blue on the right. So as human beings we have to be able to make appropriate responses, behavior wise, to many thing that we see. And of course this isn’t something that you consciously think about. And what a challenge that is because the variety of different combinatinos of color, brightness, the number of different contexts that we can see is basically infinite. So the only way to do this is to use successes and failures of the past, and then adjust the weights in this neural network. This is the main idea of the empirical strategy of vision. This idea applys to brightness, luminence, and reflectivity as well, which Jacob is going to cover next.

Slide 22

Depth Perception
How do we see depth?

Slide 23

Four Key Ide as
• • • • Binocula Dispa r rity Ne corre s ural late Ve ntral vs dorsa stre ms . l a Ambly , squint, a a re ss opia nd wa ne

Slide 24

What is binocular disparity?
• Each e obta a diffe nt ima of the world ye ins re ge • Diffe nce be en the two ima s is calle re twe ge d binocular dispa rity • Re quire high accuracy from CNS to re s giste r diffe nce be en the two im ge re s twe a s
– Diffe nceis usually sm lle than the width of a re a r singlephotore ptor ce

-each eye has separate image due to their horizontal separation

Slide 25

cont. binocular disparity
• How doe the brain e s xtract the diffe nce se re s?
– Individua fe l ature in the le e ’s im m be s ft ye age ust m atche with the corre d spondingim ge on the rig a ht e ye – Ex. For obje re nition, e ’s m m ct cog ye ust atch up a nd also m atch se nsory input with what is sore in d mm e ory

-complexity deals with not only matching visual stimuli, but also making sense of that stimuli

Slide 26

Ne ural Correlate s
• Absolute vs re tive disparity . la • Binocula a r nti-corre lation • Ambiguity in ste om re atching

Slide 27

• Suppose two e s looking at one point ye • The angular diffe nce in the proje re ctions of that point onto the le a right e s with re re ft nd ye fe nce to e ch e ’s fove a ye a

Slide 28

Relative Disparity
• Ta s into account two points ke • Diffe nce in the absolute disparity of both e s re ye with re ct to both points spe
– Elim inate fove as are re point s a fe nce – Diffe ncebe e the a le subte d by the two re twe n ng nde points on the le e a the a le be e those ft ye nd ng twe n point on therig e ht ye

Slide 29

-point to the Alpha minus Beta -explain that fovea is not reference point, but rather r.d. is a reflection of the spacing of objects with varying depth

Slide 30

• Huma ste oscopic syste re s m on n re m lie ore re lative dispa whe a ssing depth rity n sse • V2 a as show m se re ore nsitivity to com x ple re lative dispa rity; V1 shows no se nsitivity to re lative dispa rity • Diffe nt spatial configura re tions re in the sult sam num rical re tive dispa e e la rity
– MT shows som se e nsitivity for re tive dispa for la rity ce rtain stim config tions uli ura – Spa l la out for re tive dispa s is im tia y la ritie portant

-MT shows sensitivity to rel. disp. When a single plane is tiled to the observer and has a relative disparity between the nearer and further edges of the plane; in same region, neurons show no snesitivity to relative disparity when center-surround configuration is used -also highly responsive to introduction of a relative disparity in a rotating transparent cylinder stimulus -it is impossible to apply such a label as ‘relative disparity’ to a particular brain region without being more specific about the kinds of relative disparity referred to

Slide 31

Slide 32

Binocular anti-correlation
• Stim pre nte visual fe uli se d ature of opposite s contra to the le and right e s st ft ye • Random dot ste ogram re :
• Pa of pictures, one for ea eye; ra ir ch ndom com ly posed of bla a white elem ck nd ents • Stereopsisrevea previously hidden figure (think: m gic ls a eyebooks)

-stereopsis: perception of depth that occurs when information from the right and left eyes are encoded

Slide 33

With ste oscopic vision, re oneim ca be age n pre nte to y le se d our ft e , and the othe to the ye r rig e ht ye Anticorre lation pre nts ve thepe ption of rce ste oscopic de re pth

-by either converging or diverging the eyes -three possibilities: -dot patterns presented to the left and right eyes are based on same sequence; correlation = 1; circle defined by depth should be seen -dot patterns based on independent sequences, correlation = 0; no circle is defined -dot patterns are based on same sequence only for every dark circle in one eye, theree is a light circle on the other eye and vice versa; stereogram results in anticorrelation (-1)

Slide 34

Magic Eye

-supposed to be a locomotive

Slide 35

Am biguity in S re atching te om
• Whe looking at two or m ve n ore rtical bars, subje report se ing the e cte num r of cts e xpe d be bars in the sam depth fie e ld
– Each ba ha sam ze dispa r s e ro rity – Corre a ar in sam picture plane ctly ppe e

• So wha is the proble ? t m

Slide 36

• Ea e re ive ide ch ye ce ntical im of ba age rs • Binocula syste sorts which bar a se n by le r m s e ft e be ye longs with which bar se n by the right e e ye • Bra re in gion tha re t sponds to a ma ge ra s tch ne te “false m atche s” • The dispa with re ct fo fixation point is ir rity spe such tha the would signladiffe nt binocular t y re depth in front of or be hind the picture plane • But this doe actually ha n…. sn’t ppe

Slide 37

• V1 ne urons fire just a strongly for “fa s lse ma s” as for ge tche nuine one s • Howe r…. In V2, m ne ve ost urons sfirestronge r for ge nuine m atche s • As a re sult, we don’t actually se the “fa e se lse ma s” tche

Slide 38

Dorsal and Ventral Stre ams
• Old the s re cte orie je d:
– Ste oscopic de e re pth xlusiveto ve l or dorsa ntra l stre am – Sim division of: ple
• Coa stereopsisin dorsa strea rse l m • finestereopsisin ventra strea l m

Slide 39

New theory: the two streams carry out fundamentally different types of stereo communication
– Each streamis specialized and contributes to stereoscopic vision in a unique way – Dorsal: simple computation of both relative and absolute disparity – Ventral: sophisticated communication of relative disparity including 3-D texture

-dorsal: relies on direc5t computation of the bin. Correlation between the left and right eye images’; relative disparity is only shown in processing of spatially extended surfaces and the segregation in depth of one surface to another; suggests that bin. Depth in dorsal stream helps the individual to navigate the world -ventral: full resolution of bin. Matching problem and neurons in this strean appear to ve specifically sensitive to relative deoth between different features located at nearby positions; sensitivity to shape and curvature of three-dimensional figures

Slide 40

Stereovision in dorsal and ventral pathways (monkey cortical areas)

Slide 41

Problems with binocular vision
• Complexity of binocular vision oeaves room for developmental complications • Amblyopia • Squint

Slide 42

• Poor vision as a result of a healthy eye having faulty connection with the rest of the brain • Disrupted transmission of the visual image • Not corrected by lenses • Incorrect linage of one eye cause loss of binocular disparity

Slide 43

• Here!

Slide 44

What we talke about… ’ve d
• • • • Ge stalt The ory Me chanics of Vision Color Pe rception Depth Pe rception

So what we’ve covered today is a little bit of the history of the study of vision, related to Gestalt theory and the first theory that recognized that context is important, and then some mechanics of vision, and tied that to color and depth perception. So before we get into a break and then the next activity, I’d like to reveal to you that next week, the big man, Purves himself, is actually coming to our class to have a conversation with us. And I have no authority to make an assignment, but I’ve heard he loves challenges, he loves questions, and just getting people talking, so I thought it would be cool for everyone to come up with maybe a question for him. I’ll explain the activity after the break!! So we are going to do a little activity to get you guys looking at and playing with some illusions, so since working alone is scary for some but we don’t want to have HUGE group dynamics so some people talk a lot and others no so much, we want you to split into groups of two. We are going to give each group a cool illusion to play with, then you get to present your illusion to the class, and talk about it in terms of some of the ideas we just outlined. And after you do that, in celebration of valentines day, or single awareness day, you get candy! Sound cool? ----------------------------------

EXTRA STUFF When surface illuminated by spectrum similar the intensity is INCREASED and the width of distribution narrows, as in 1 When surface is illuminated by a spectrum that is opposite in its distribution of power, intensity is decreases and width of distribution is broadened and flattened When surface is illuminated by spectrum that is neither the same nor opposite, the spectral return shifts along the x-axis in the direction of the spectral profile of the illuminant And we can measure the contrast effect by placing a stimulus on top of a background and having subjects adjust contrast, hue, and britness until it matches a side color, and I don’t need to get into the specifics, but its always the case that the color contrast theory applies! Empirical explanation color contrast: this effect can be understood as the natural outcome of a visual strategy in which color percepts are generated according to the relative contribution of illumination, reflectance and transmittance to similar stimuli experienced in the past But when we look at a scene there is always always more than one thing, so our visual system has to decide soo am I looking at reflectances under the same illuminant (Fig. 4B), in which case they should appear similarly colored, or different reflectances under different illuminants If perceptions of color are indeed generated in a wholly empirical way, then identical targets presented on differently chromatic backgrounds should give rise to different chromatic sensations, as indeed they do. the chromatic appearance of two identical targets will always follow the probability distribution of the possible sources of the target spectra, given the

constellation spectral returns from the rest of the scene.

Master your semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master your semester with Scribd & The New York Times

Cancel anytime.