1st step in sensory processing involves the nervous system transforming light into neural signals by

receptor cells (i.e. rods/cons)

Sensation doesn't necessarily lead to conscious perception
For each type of sensory processing, there's a mechanism for converting external energy into
neural/electrical signals

Sensory Transduction
All or none
Since action potentials are not graded in intensity, their frequencies must code all neural signaling,
including sensory processing
o These code our experiences
Neural communication relies on firing rates (the # of action potentials a neuron sends in a period of
time)
o Range of possible sensory stimuli is far larger than the possible range of firing rates

Sensory Adaptation: resetting of sensitivity according to ambient condition

Purpose: so sensory processing can occur with maximum efficacy across a range of different
environments
Sensitivity is currently adjusted to yield/adapt to most relevant conditions

Unexpected events carry more information than expected ones

Brain maintains an intention representation of the current external context in its interpretation of
afferent signals

Acuity: degree of precision

Distinguishes two nearby points [Ernst Weber]
o Depending on where on the body surface the stimulation is applied
Varies proportionally with stimulus intensive
Varies dramatically across different regions of sensory organs
Depends on distribution/types of receptor cells across retina
o Limitation of hardware, e.g. bad camera vs. good camera, the same picture on both is going
to look better from the better camera

Blind spot (optic disk) - spot in the retina where the optic nerve leaves, so you have no photoreceptors
there

Saccades: movement of eyes every 3-4 sec in different directions

This is how we overcome the lack of acuity in peripheral vision (by making saccadic eye
movements)

Eccentricity: distance away from line of site

Cones dominate in fovea (central region of retina)

 o Highest level of visual acuity
o Rods sparse there, missing 100% from center

Subcortical visual processing happens before the activity elicited by stimulus reaches the cerebral
cortex

First stage of visual processing occurs in the five layers of retina

This info converges onto retinal ganglion cells
The axons of the ganglion cells leave the retina via the optic nerve
o This is the first component of the primary visual pathway - the major route from eye to
visual cortex in the occipital lobe
Conveys the info in light stimuli that we eventually see as images
Retinal circuitry modulates info sent forth by retinal ganglion cells

Thalamus is a feedforward relay for all incoming sensory signals except olfaction
First pass through here
Thalamic nuclei are specialized for different sense
o LGN: Vision
o MGN: Audition
o Ventral Posterior Nuclear Complex: Somatosensation
Also receives a lot of cortical feedback projections and other thalamic nuclei serve as corto-cortical
relays

After thalamus - next stage of processing are primary sensory cortices, where the analysis of basic
stimulus properties is carried out

V2-V4 is known as the extra striate visual cortex

V4 is important for color vision perception

Major target of retinal ganglion cells = lateral geniculate nucleus in thalamus

o There are two different layers that play different functions
The Magnocellular layer (M retinal ganglion cells) process information about
changes in stimuli that lead to motion perception
The Parvocellular layer (P) process information related to spatial details that
underlies perception
Target of LGN is V1

CORTICAL VISUAL PROCESSING

Eyes collect, filter, and amplify the most relevant aspect of external energy (light)
o Cornea, lens, and pupil filter and focus light/enhance it before it reaches neural retina (back
of eye)
Everything we see is projected onto a 2D surface and then turns it into the 3D world
around us

Temporal lobe is involved in perception

Parietal lobe is involved in attention or doing something about what is perceived
V4 is important for processing info in color vision
MT/MST = detecting motion

Ungerleider + Mishkin are the first to prove and organize two separate pathways that feed
information into temporal and parietal lobes
 o Ventral Stream - "what"
From visual cortex and inferior part of temporal lobe
Form and color
o Dorsal Stream - "where"
From striate cortex into the parietal lobe
Responsible for spatial aspects of vision (analyzing motion and positional
relationships)

As sensory info moves through higher processing, information given by a specific sensation must
be integrated with other systems to improve the efficacy of behavior
o "what we see affects what we hear" --> McGurk Effect

Topography is evident in primary sensory cortices (visual + somatic)

Size of "real estate" in sensory cortices are occupied by a particular input that reflects the
complexity/importance of receptor cells associated with that input, which leads to cortical
magnification
Sensory cortices follow a modular organization, where neurons with similar functional and
response properties can be grouped together in cortical columns
Sensory neurons have limited receptive fields and their receptive field properties (response
properties) can be described through tuning curves

Neurons that share function properties are often grouped together (ocular dominance)
Columns in early visual cortex are grouped together by their preference for orientation of edges
o Take form of "pinwheels" where similar orientation-preferring columns are adjacent to each
other in the brain

Topographical Mapping - stimulate specific locations on retina/body to evaluate how one can see the
location of peripheral stimulation that is reflected in the location of corresponding CNS activity????
Less clear as distance away from primary sensory cortices increases
Topography of retina is re-established in cortex and thalamus

A feature of V1 is that each unit of retinal surfaces are disproportionally represented at the level of
the cortex
o V1 has a lot of space represented to the fovea, which has a small part in the visual field,
because it carries the most importance
Cortical magnification makes sense because the visual detail we get in response to the stimulus of
the fovea requires more work in the cortex by other acutely resolved portions of the visual scene

Receptive field: region of the retina that, when stimulated, elicits a response in the neuron being
examined
At the level of retinal output and thalamus, neurons respond to light

In higher order cortical areas, receptive fields only cover a fraction of a whole visual field
 Topographies apparent in primary cortices are less apparent in higher order areas

In V1, the higher the order of nerve cells, the less they depend on visual input and the more they
depend on info that is not strictly visual

Visual Perception -
Lightness- appearance of a surface (paper)
Brightness- appearance of a light source
o Not subject to direct measurement but can be measured relative to another surface
o Its measurable and physical correlate is luminance
o Simultaneous lightness/brightness contrast is when a patch on background of low luminance
appears lighter/brighter on a background of higher luminance
[some people thought this could be because of a difference in retinal output but they
were wrong - it's determined by illumination of objects, the reflectance of object
surfaces, the transmittance of space between the objects and the observer, previous
experiences etc.]
There's no logical way to determine how many combinations make certain values
appear
Color- distribution of light across the visible spectrum
o Relative amount of light at short, middle, and long wavelengths
o Color Constancy: patches returning different spectra to the eye can appear the same color
o Color Contrast: Two stimulus patches producing the same distribution of light energy at
various wavelengths can appear different from one another based on the surroundings
o Has three perceptual qualities
Hue- perception of relative redness, blueness, greenness, or yellowness of stimulus
Saturation- degree of which percept approaches a neutral gray
Color Brightness- saturation applied to a stimulus that elicits a discernable hue
Form- entails simple geometric characteristics (i.e. length of lines, their orientation, angles)
o Explores how we perceive the distance between two stimuli
We tend to see vertical lines longer than horizontal ones of the same length
Because of how we perceive spatial intervals/orientation of retinal images
o In one study, the active area of V1 varied according to the perceived size instead of the
actual size of an objects

Depth- perception of 3D world from 2D retinal images

o Some developed from one eye, others from both
o 3D development is attributed to both monocular and binocular components
o Binocular vision arises from stereopsis, or the fact that our eyes are separated horizontally
across the face by ~65 mm
Difference in images from both eyes is called retinal disparity
o Signals are separate in the thalamus and in cortical layer 4, they are combined in other layers
of V1
I.e. there are cells in V1 that fire in response to stimulation from both eyes (e.g.
binocular cells)
Motion

In humans, color seeing is based on absorption properties of 3 different cone types with different
photo pigments (opsin)
o Each cone type corresponds best to a different portion of light on the spectrum
Color blindness in humans is unableness to discern red/green hues
Colors we see are strongly influenced by the rest of the background
Color Contrast- stimulus patch generating the exact same distribution of light energy at various
wavelengths can appear different in color depending on its surroundings

Cerebral achromatopsia: patients lose ability to see world in color although light/brightness and form
are fine

Inverse Optics Problem: there is an infinite number of combinations of parameters that can produce the
same output
"different combinations producing the same retinal luminance values"
Interprets likely luminance based on previous experience

Monocular Depth Perception: depends on associations learned from an individual with the arrangement
of objects in space
We gain occlusion from this: when a part of an object is obscure by another, the obstructing object
is perceived to be closer

Inputs from two eyes converge on cortical neurons in V1

Relationship between size and distance is that the same object occupies less space on the retina as
distance to the observer increases (perspective)

Motion Parallax: when the position of the observer changes, the position of the background changes
with respect to an object in the foreground/the observer
One of the monocular acoustics
If you close one eye and move, you see two objects move relative to each other (one is no longer
blocking the other in the same way, so it's a way of infering depth)

Binocular depth perception : stereopsis - each eye has a slightly different view of the same object
Proven by the fact that many neurons in primary and extra striate visual cortex have receptive
fields that are tuned to retinal disparity

Cycloplean fusion- combined image from both eyes at the level of the common target in the visual
cortex

Binocular Rivalry: discordant patterns by different eyes, only one pattern will most likely be perceived
Proof: if both were integrated in the visual cortex, the user should see a mix of both

Motion: the subjective experience when a sequence of different but related images is presented to the
retina over a brief span of time
Relies on perception of speed and perception of direction

Patient L.M. has a bilateral damage to motion-sensitive temporal cortex __ ( get from slides )

Relevant Regions of Motion:

Posterior temporal lobe
Middle Temporal
Middle superior temporal (this and the one above respond to image sequences)

William Newsome: made experiment training monkeys to track movement

Conclusion, the could track an overall sense of direction of motion
 MT plays big role, neurons show selective activity for a particular direction

Aperture Problem: challenge of explaining how visual systems generate definite perceptions of
speed/direction in response to stimuli
You can get the same appearance of motion and just varied the speed in which motions are
presented

Apparent Motion: sense of realistic motion from static images

Motion Aftereffects: (waterfall effect)

When staring at a particular direction of movement, one sees opposite movement or direction when
looking away
o Reason: prolonged exposure causes motion activated neurons to adapt, so when the motion
stimuli are removed, non-adaptive motions are more active, leading to illusion of movement
in the opposite direction
o (optical illusion type things)

OBJECT RECOGNITION -
Detection of oriented lines, colors, size, etc. are the first steps to object recognition
Works by visual processing that triggers association in other regions that leads to outcomes
Relies on information outside of the visual system
o Sensory input must be matched with long-term memory representations
o Triggered by visual processes in ventral stream that travel to temporal lobe/medial temporal
lobe
Fusiform face area
Using fMRI, we've learned subdivisions of it process different information about
recognizing animals, objects, words, etc.

Viewpoint Invariance we are able to recognize the same object from many different viewpoints (object
constancy)
Object constancy must be achieved through a matching process of sensory input with a viewpoint
invariant memory representation (structural description)
View point invariant objects are represented are of "canonical views of objects"
o Stranger, non-conventional viewpoint orientation of the object

Models of object perception assume various stages:

Basic analysis of visual features
Grouping elements into likely objects (figure ground segmentation)
The view-centered percepts is matched against strored "structural descriptions" of objects
(viewpoint-invariant)
The view as you're looking at it
Semantic meaning is attributed and associated information is available (e.g. the object's name)

Memory is important for stages 3 and 4, but might also influence 2 in a top-down manner

-Information passes from V1 through ventral stream, and eventually reaches infratemporal cortex
"simple cell" V1 responses to "complex cell" responses to aggregate, view-tuned representations,
to view-invariant and object-tuned units in anterior IT
FFA CONCLUSIONS
Evidence that the FFA is a module that is special for face processing
Individuals who are experts at identifying other types of objects may recruit the FFA to make these
other kinds of fine-tuned object judgements just like we do for faces

Haxby et. All

Distributed representations
Participants completed a 1-back repetition detection task
Researchers looked at the pattern of activity across the voxels within temporal cortex to determine
whether the pattern associated with any type of object was more similar to other instances that type
of object ___
Multivoxel pattern analaysis

IT cells in rhesus monkeys are also sensitive to the presentation of faces (experiment that
reinforces what we already know about it)
Some regions of the "what" pathway appear to be specialized for processing particular objects that
are of evolutionary importance to us

In non-humans, neurons in inferior temporal cortex respond to faces selectively

They also respond to face-like looking objects or voices

BLINDSIGHT
Complete removal of left striate cortex in 4 monkeys (experimental example)
Connection between brain hemisphere (corpus callosum) was also severed
Researchers tracked eye position while monkeys viewed visual stimuli
The experimental task tested the ability to detect and localize the brief presentation of visual
stimulation
See if the monkey noticed whether a new object appears
All monkeys detected and correctly localized the target in both visual fields more than 90%
of the time, even the lesioned ones
Works because there are other pathways other than LGN and striate cortex
Preserved "islands" of cortex in V1
Superior Colliculus: retinotectal pathways
Retinotopic map
Form a map of space
It is implicated in the generation of saccadic eye movement following the sudden onset
of a stimulus (driver of it)
Geniculo-Extrastriate Pathway

Bilateral destruction of visual cortex

Striate cortex removed from both brain hemisphere
Helen was still able to avoid obstacles with stripes, but not those that were transparent