Human visual perception - topics
n n n n n n
Visual acuity Weber-Fechner Law Lateral inhibition and excitation Transfer functions of the color channels Spatial and temporal masking Eye movements
Bernd Girod: EE368b Image and Video Compression
Human Visual Perception no. 1
Anatomy of the human eye
iris cornea pupil lens
visual axis
fovea
optic axis
optic disk optic nerve
retina sclera
object
retinal image
Bernd Girod: EE368b Image and Video Compression
Human Visual Perception no. 2
Optical properties of the human eye
n
Deviations from ideal perspective projection due to
l l l l
Aperture of the eye Focus errors (spherical aberration) Chromatic aberration Dispersion
Effects can be summarized by a 2D convolution with the optical point-spread function (PSF). Instead of a PSF, an optical line-spread function (LSF) is often given, which can be measured more easily.
Bernd Girod: EE368b Image and Video Compression
Human Visual Perception no. 3
Optical LSF of the human eye
LSF measured for different pupil diameters (Campbell+Gubisch) LSF calculated from eye aperture (due to diffraction)
Bernd Girod: EE368b Image and Video Compression
Human Visual Perception no. 4
Optical modulation transfer function (MTF) of the human eye
The optical modulation transfer function (MTF) can be interpreted as Fourier transform of the optical LSF.
contrast ratio image:object
MTF is measured directly with sinewave gratings.
spatial frequency [cpd]
Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 5
Sine wave grating
contrast ratio =
intensity
L2 L1 L2 + L1
position
Bernd Girod: EE368b Image and Video Compression
Human Visual Perception no. 6
Human retina I
light
Signal propagation in the retina:
optic nerve
ganglion cells (=optic nerve)
fibers ganglion cells
bipolar cells
inner synaptic layer amacrine cells bipolar cells horizontal cells outer synaptic layer
receptors
receptor nuclei receptors pigmented layer (epithelium cells)
Bernd Girod: EE368b Image and Video Compression
Human Visual Perception no. 7
Human retina II
receptors in 1000/mm 2
Rods high sensitivity low light vision monochrome "scotopic vision"
Cones low sensitivity day light vision > 1 cd/m2 color "photopic vision"
blind spot fovea rods
cones
nose
angle [degree]
Video displays
Bernd Girod: EE368b Image and Video Compression
Human Visual Perception no. 8
Visual acuity
n
Spatial resolution in lines/arcmin:
cones
rods
nose
blind spot fovea angle [degree]
Minimum distance of adjacent cones in the central fovea limits spatial resolution. (2 - 2.3 m 25 . . . 29 sec of arc)
Bernd Girod: EE368b Image and Video Compression
Human Visual Perception no. 9
Weber-Fechner law, I
n
Experiment:
Result:
surround luminance L S stimulus area background luminance L B
threshold L in cd /m 2
CONST
L B , background luminance (cd/m 2 )
Bernd Girod: EE368b Image and Video Compression
Human Visual Perception no. 10
Weber-Fechner law, II
n
"Weber-Fechner Law
L = cLB
n
c = 0.01 ... 0.02
Implies logarithmic relationship between physical luminance and subjectively perceived brightness. Other proposed nonlinearities: square-root, cube-root, polynomials
-characteristic of CRT displays is approximate inverse of
nonlinearity of human brightness perception.
Bernd Girod: EE368b Image and Video Compression
Human Visual Perception no. 11
Inhibition and excitation in the retina
n
Receptive field of a ganglion cell (=fiber of the optic nerve) shows center-surround response with both
l l
Lateral inhibition Lateral excitation
receptive field
center surround
activity of neuron
Bernd Girod: EE368b Image and Video Compression
average activity
Human Visual Perception no. 12
Spatial contrast sensitivity
n
Lateral inhibition and excitation together lead to a bandpass characteristic of the contrast sensitivity function of the human visual system
cd/m 2 ) contrast sensitivity (@ 500
spatial frequency (cpd)
Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 13
Color vision: opponent color theory
n
Retina carries out matrix operation to represent colors in the opponent color system (Y, Y-B, R-G)
Opponent color model:
B G R
Opponent color space:
white green
Y YB RG Luminance signal Chromatic channels
blue
yellow
red
B G R
black
Bernd Girod: EE368b Image and Video Compression
Human Visual Perception no. 14
Color vision: contrast sensitivity in opponent color space
n
Spatial frequency response of Y-B and R-G channel (Girod, 1988):
spatial frequency (cpd)
n n
Bandwidth Y:RG:YB approximately 8:5:3. Some researchers have observed bandpass characteristic also for chromaticity channels.
Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 15
Spatial masking, I
dark light distance D dynamic noise bar change in video signal amplitude [0 . . . 255] 80 230 6 dB 80 180 80 130
Experiment:
visibility threshold (w-Modell, Girod, 1987)
visibility threshold [log units] -20
-10 0 10 distance from edge [arcmin]
20
Bernd Girod: EE368b Image and Video Compression
Human Visual Perception no. 16
Spatial masking, II
Visibility threshold for the -predistorted video signal (w-Modell, Girod, 1987):
"-shift"
6 dB visibility threshold [log units] change of video amplitude [0 . . . 255] 80 230 80 180 80 130
-20
-10 0 10 distance from edge [arcmin]
20
Bernd Girod: EE368b Image and Video Compression
Human Visual Perception no. 17
Spatiotemporal contrast sensitivity of luminance perception
n
Spatiotemporal contrast sensitivity of the luminance channel has bandpass characteristic. Contrast sensitivity function separable for high spatial and temporal frequencies only. Plot of contrast sensitivity function (from Kelly):
just perceptable modulation
temporal frequency
Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 18
sp
ati
al
fre
qu
en
cy
Temporal masking
n
Visibility thresholds for -predistorted video signal after luminance discontinuity (w-model, Girod, 1987):
visibility threshold [arbitrary log units]
time after discontinuity [ms]
Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 19
Eye movements
slow drift
800 /s
saccad e; v up to
corrective saccade
SPEM ; v < 30 /s
e cad sac
SPEM: smooth pursuit eye movement
Bernd Girod: EE368b Image and Video Compression
Human Visual Perception no. 20
10
Temporal masking and SPEMs
t=0 t=t 0 t0 < t < t 1 t1 < t
Eye fixates screen
x
Eye tracks moving edge
x
dark
t0 t0
dark
bright
t1 t
trajectory of eye movement
bright
t1 t
trajectory of eye movement
Temporal masking
Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 21
Eye movements and spatiotemporal frequency response of the human visual system, I
n
Assume SPEM of constant velocity:
x' = x - vx t y' = y - v y t t' = t
retina coordinate system display coordinate system
Coordinate transformation in spatiotemporal frequency space (Doppler effect)
x ' = x y ' = y t ' = t + x vx + y vy
Bernd Girod: EE368b Image and Video Compression
Human Visual Perception no. 22
11
Eye movements and spatiotemporal frequency response of the human visual system, II
relative velocity between eye and coordinate system
Bernd Girod: EE368b Image and Video Compression
Human Visual Perception no. 23
Eye movements and spatiotemporal frequency response of the human visual system, III
relative velocity between eye and coordinate system
Bernd Girod: EE368b Image and Video Compression
Human Visual Perception no. 24
12
Perception of a temporally sampled image signal, without movement
spatial frequency
temporal frequency "window of perception"
Bernd Girod: EE368b Image and Video Compression
Human Visual Perception no. 25
Perception of a temporally sampled image signal, translatory movement
spatial frequency 3 pel/frame
temporal frequency
Phosphors with fast decay reproduce more than 20 temporal baseband replicas No spectral overlap -> perfect reconstruction of original signal possible
Human Visual Perception no. 26
"window of perception", with SPEM
Bernd Girod: EE368b Image and Video Compression
13
Anatomy of the human visual system
retina visual cortex
overlapping visual fields
Bernd Girod: EE368b Image and Video Compression
Human Visual Perception no. 27
Human visual perception - summary
n n
Spatial frequency components visible up to 60 cpd Logarithmic relationship between luminance and subjective brightness perception Lateral inhibition -> spatial bandpass characteristic Chromaticity channels have lower bandwidth Visibility threshold often increased in the vicinity of edges, but sometimes decreased (masking). SPEMs shear window of perception in spatiotemporal frequency space.
n n n
Bernd Girod: EE368b Image and Video Compression
Human Visual Perception no. 28
(enlarged)
14
