Human visual perception - topics

n n n n n n

Visual acuity Weber-Fechner Law Lateral inhibition and excitation Transfer functions of the color channels Spatial and temporal masking Eye movements

Bernd Girod: EE368b Image and Video Compression

Human Visual Perception no. 1

Anatomy of the human eye

iris cornea pupil lens

visual axis

fovea

optic axis

optic disk optic nerve

retina sclera

object

retinal image

Bernd Girod: EE368b Image and Video Compression

Human Visual Perception no. 2

Optical properties of the human eye

n

Deviations from ideal perspective projection due to

l l l l

Aperture of the eye Focus errors (spherical aberration) Chromatic aberration Dispersion

Effects can be summarized by a 2D convolution with the optical point-spread function (PSF). Instead of a PSF, an optical line-spread function (LSF) is often given, which can be measured more easily.

Bernd Girod: EE368b Image and Video Compression

Human Visual Perception no. 3

Optical LSF of the human eye

LSF measured for different pupil diameters (Campbell+Gubisch) LSF calculated from eye aperture (due to diffraction)

Bernd Girod: EE368b Image and Video Compression

Human Visual Perception no. 4

Optical modulation transfer function (MTF) of the human eye

The optical modulation transfer function (MTF) can be interpreted as Fourier transform of the optical LSF.

contrast ratio image:object

MTF is measured directly with sinewave gratings.

spatial frequency [cpd]

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 5

Sine wave grating

contrast ratio =
intensity

L2 L1 L2 + L1

position

Bernd Girod: EE368b Image and Video Compression

Human Visual Perception no. 6

Human retina I
light

Signal propagation in the retina:

optic nerve

ganglion cells (=optic nerve)

fibers ganglion cells

bipolar cells

inner synaptic layer amacrine cells bipolar cells horizontal cells outer synaptic layer

receptors

receptor nuclei receptors pigmented layer (epithelium cells)

Bernd Girod: EE368b Image and Video Compression

Human Visual Perception no. 7

Human retina II

receptors in 1000/mm 2

Rods high sensitivity low light vision monochrome "scotopic vision"

Cones low sensitivity day light vision > 1 cd/m2 color "photopic vision"
blind spot fovea rods

cones

nose

angle [degree]

Video displays

Bernd Girod: EE368b Image and Video Compression

Human Visual Perception no. 8

Visual acuity
n

Spatial resolution in lines/arcmin:

cones

rods

nose

blind spot fovea angle [degree]

Minimum distance of adjacent cones in the central fovea limits spatial resolution. (2 - 2.3 m 25 . . . 29 sec of arc)

Bernd Girod: EE368b Image and Video Compression

Human Visual Perception no. 9

Weber-Fechner law, I
n

Experiment:

Result:

surround luminance L S stimulus area background luminance L B

threshold L in cd /m 2

CONST

L B , background luminance (cd/m 2 )

Bernd Girod: EE368b Image and Video Compression

Human Visual Perception no. 10

Weber-Fechner law, II
n

"Weber-Fechner Law

L = cLB
n

c = 0.01 ... 0.02

Implies logarithmic relationship between physical luminance and subjectively perceived brightness. Other proposed nonlinearities: square-root, cube-root, polynomials

-characteristic of CRT displays is approximate inverse of

nonlinearity of human brightness perception.

Bernd Girod: EE368b Image and Video Compression

Human Visual Perception no. 11

Inhibition and excitation in the retina

n

Receptive field of a ganglion cell (=fiber of the optic nerve) shows center-surround response with both
l l

Lateral inhibition Lateral excitation

receptive field

center surround

activity of neuron
Bernd Girod: EE368b Image and Video Compression

average activity
Human Visual Perception no. 12

Spatial contrast sensitivity

n

Lateral inhibition and excitation together lead to a bandpass characteristic of the contrast sensitivity function of the human visual system
cd/m 2 ) contrast sensitivity (@ 500

spatial frequency (cpd)

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 13

Color vision: opponent color theory

n

Retina carries out matrix operation to represent colors in the opponent color system (Y, Y-B, R-G)
Opponent color model:
B G R

Opponent color space:

white green
Y YB RG Luminance signal Chromatic channels

blue

yellow

red
B G R

black

Bernd Girod: EE368b Image and Video Compression

Human Visual Perception no. 14

Color vision: contrast sensitivity in opponent color space

n

Spatial frequency response of Y-B and R-G channel (Girod, 1988):

spatial frequency (cpd)

n n

Bandwidth Y:RG:YB approximately 8:5:3. Some researchers have observed bandpass characteristic also for chromaticity channels.
Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 15

Spatial masking, I
dark light distance D dynamic noise bar change in video signal amplitude [0 . . . 255] 80 230 6 dB 80 180 80 130

Experiment:

visibility threshold (w-Modell, Girod, 1987)

visibility threshold [log units] -20

-10 0 10 distance from edge [arcmin]

20

Bernd Girod: EE368b Image and Video Compression

Human Visual Perception no. 16

Spatial masking, II
Visibility threshold for the -predistorted video signal (w-Modell, Girod, 1987):
"-shift"
6 dB visibility threshold [log units] change of video amplitude [0 . . . 255] 80 230 80 180 80 130

-20

-10 0 10 distance from edge [arcmin]

20

Bernd Girod: EE368b Image and Video Compression

Human Visual Perception no. 17

Spatiotemporal contrast sensitivity of luminance perception

n

Spatiotemporal contrast sensitivity of the luminance channel has bandpass characteristic. Contrast sensitivity function separable for high spatial and temporal frequencies only. Plot of contrast sensitivity function (from Kelly):

just perceptable modulation

temporal frequency
Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 18

sp

ati

al

fre

qu

en

cy

Temporal masking
n

Visibility thresholds for -predistorted video signal after luminance discontinuity (w-model, Girod, 1987):

visibility threshold [arbitrary log units]

time after discontinuity [ms]

Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 19

Eye movements
slow drift
800 /s

saccad e; v up to

corrective saccade

SPEM ; v < 30 /s

e cad sac

SPEM: smooth pursuit eye movement

Bernd Girod: EE368b Image and Video Compression

Human Visual Perception no. 20

10

Temporal masking and SPEMs

t=0 t=t 0 t0 < t < t 1 t1 < t

Eye fixates screen

x

Eye tracks moving edge

x

dark
t0 t0

dark

bright
t1 t
trajectory of eye movement

bright
t1 t
trajectory of eye movement

Temporal masking
Bernd Girod: EE368b Image and Video Compression Human Visual Perception no. 21

Eye movements and spatiotemporal frequency response of the human visual system, I
n

Assume SPEM of constant velocity:

x' = x - vx t y' = y - v y t t' = t
retina coordinate system display coordinate system

Coordinate transformation in spatiotemporal frequency space (Doppler effect)

x ' = x y ' = y t ' = t + x vx + y vy

Bernd Girod: EE368b Image and Video Compression

Human Visual Perception no. 22

11

Eye movements and spatiotemporal frequency response of the human visual system, II
relative velocity between eye and coordinate system

Bernd Girod: EE368b Image and Video Compression

Human Visual Perception no. 23

Eye movements and spatiotemporal frequency response of the human visual system, III
relative velocity between eye and coordinate system

Bernd Girod: EE368b Image and Video Compression

Human Visual Perception no. 24

12

Perception of a temporally sampled image signal, without movement

spatial frequency

temporal frequency "window of perception"

Bernd Girod: EE368b Image and Video Compression

Human Visual Perception no. 25

Perception of a temporally sampled image signal, translatory movement

spatial frequency 3 pel/frame

temporal frequency

Phosphors with fast decay reproduce more than 20 temporal baseband replicas No spectral overlap -> perfect reconstruction of original signal possible
Human Visual Perception no. 26

"window of perception", with SPEM

Bernd Girod: EE368b Image and Video Compression

13

Anatomy of the human visual system

retina visual cortex

overlapping visual fields

Bernd Girod: EE368b Image and Video Compression

Human Visual Perception no. 27

Human visual perception - summary

n n

Spatial frequency components visible up to 60 cpd Logarithmic relationship between luminance and subjective brightness perception Lateral inhibition -> spatial bandpass characteristic Chromaticity channels have lower bandwidth Visibility threshold often increased in the vicinity of edges, but sometimes decreased (masking). SPEMs shear window of perception in spatiotemporal frequency space.

n n n

Bernd Girod: EE368b Image and Video Compression

Human Visual Perception no. 28

(enlarged)

14