PEMBENTUKAN CITRA DAN WARNA:

MANUSIA DAN MESIN

2019/2020 -2 Materi 03 - Visi Komputer

 What picture is it?
1
 Is it easier to look?
2
 What is wrong with this picture?
3
 Capturing Light… in man and machine
4

Many slides by
Alexei A. Efros
 Image Formation
5

Digital Camera

Film

The Eye
 A photon’s life choices
6

 Absorption
 Diffusion light source
 Reflection
λ
 Transparency
 Refraction
 Fluorescence
?
 Subsurface scattering
 Phosphorescence
 Interreflection
 A photon’s life choices
7

 Absorption Absorption of electromagnetic radiation (or

light) is the way in which the energy of a
 Diffusion photon is taken up by matter, typically the
light source
electrons of an atom. The electromagnetic
 Reflection energy is transformed into internal energy of
the absorber, for example thermal energy
λ
 Transparency
 Refraction
 Fluorescence
 Subsurface scattering
 Phosphorescence
 Interreflection
 A photon’s life choices
8
 Absorption
 Diffuse Reflection
Diffuse reflection is the reflection of light or
other waves or particles from a surface such
that a ray incident on the surface is scattered at
many angles rather than at just one angle.

 Reflection An ideal diffuse reflecting surface is said

to exhibit Lambertian reflection, meaning
that there is equal luminance when
λ
 Transparency viewed from all directions lying in the
half-space adjacent to the surface.

 Refraction
 Fluorescence
 Subsurface scattering
 Phosphorescence
 Interreflection
 A photon’s life choices
9

 Absorption light source

 Diffusion
λ
 Specular Reflection
 Transparency
Specular reflection, also known as
regular reflection, is the mirror-like
 Refraction
reflection of waves, such as light,
from a surface. In this process, each
 Fluorescence
incident ray is reflected at the same
angle to the surface normal as the
incident ray, but on the opposing
 Subsurface
side of the scattering
surface normal in the
plane formed by incident and
 Phosphorescence
reflected rays.

 Interreflection
 A photon’s life choices
10

 Absorption Transparency (also called pellucidity or

diaphaneity) is the physical property of allowing light source
 Diffusion light to pass through the material without being
scattered

 Reflection λ

 Transparency
 Refraction
 Fluorescence
 Subsurface scattering
 Phosphorescence
 Interreflection
 A photon’s life choices
11

 Absorption Refraction is the change in direction of light source

wave propagation due to a change in
 Diffusion its transmission medium. The
phenomenon is explained by the
 Reflection conservation of energy and the λ
conservation of momentum. Owing to
 Transparencythe change of medium, the phase
velocity of the wave is changed but its
 Refraction frequency remains constant
 Fluorescence
 Subsurface scattering
 Phosphorescence
 Interreflection
 A photon’s life choices
12

 AbsorptionFluorescence is the light source

emission of light by a
 Diffusion substance that has absorbed
light or other electromagnetic
 Reflection radiation. In most cases, the λ1
emitted light (λ2) has a
 Transparency
longer wavelength, and
therefore lower energy, than λ2
 Refraction the absorbed radiation (λ1).
 Fluorescence
 Subsurface scattering
 Phosphorescence
 Interreflection
 A photon’s life choices
13

 Absorption
Subsurface scattering (or SSS), also
light source
known as subsurface light transport
 Diffusion
(SSLT) is a mechanism of light
transport in which light penetrates the
 Reflection
surface of a translucent object, is
λ
scattered by interacting with the
 Transparency
material, and exits the surface at
some different points.
 Refraction
 Fluorescence
 Subsurface scattering
 Phosphorescence
 Interreflection
 A photon’s life choices
14

 Absorption
Phosphorescence is an effect in which
light source
some chemicals re-emit light that they
 Diffusion
absorb. It is similar to fluorescence, except
that the chemical reactions within
 Reflection
phosphorescent materials happen less often,
t=1
thus they take longer to emit light. The
 Transparency
slower time scales of the re-emission are
associated with "forbidden" energy state t=n
 Refraction
transitions in quantum mechanics

 Fluorescence
 Subsurface scattering
 Phosphorescence
 Interreflection
 A photon’s life choices
15

 Absorption
Diffuse interreflection is a process
light source
whereby light reflected from an object
 Diffusion
strikes other objects in the surrounding
area, illuminating them. Diffuse
 Reflection
interreflection specifically describes light λ
reflected from objects which are not shiny
Transparency
orspecular. In real life terms what this
means is that light is reflected off non-shiny
 Refraction
surfaces such as the ground, walls, or
fabric, to reach areas not directly in view of
a Fluorescence
light source. If the diffuse surface is
colored, the reflected light is also colored,
 Subsurface scattering
resulting in similar coloration of surrounding
objects.
 Phosphorescence (Specular Interreflection)

 Interreflection
 Lambertian Reflectance
16

In computer vision, surfaces are often assumed to be

ideal diffuse reflectors with know dependence on
viewing direction.
Lambertian reflectance is the property that defines
an ideal "matte" or diffusely reflecting surface. The
apparent brightness of a Lambertian surface to an
observer is the same regardless of the observer's
angle of view. More technically, the surface's
luminance is isotropic, and the luminous intensity
obeys Lambert's cosine law. Lambertian
reflectance is named after Johann Heinrich Lambert,
who introduced the concept of perfect diffusion in his
1760 book Photometria.
 Digital camera
17

A digital camera replaces film with a sensor array

 Each cell in the array is light-sensitive diode that converts photons to
electrons Complementary metal–oxide–
 Two common types semiconductor, abbreviated
 Charge Coupled Device (CCD) as CMOS is a technology for
 CMOS constructing integrated circuits
 http://electronics.howstuffworks.com/digital-camera.htm

Slide by Steve Seitz

 Sensor Array
18

CMOS sensor
 Sampling and Quantization
19

Sampling corresponds to a
discretization of the space.
That is, of the domain of the
function, into
f : [1, . . . ,N] × [1, . . . , M]
→ℜm.
discretization is the process of
transferring continuous functions,
models, variables, and equations into
discrete counterparts
The smallest element resulting from
the discretization of the space is
called a pixel (picture element).
For 3-D images, this element is called
a voxel (volumetric pixel).
Quantization corresponds to a discretization
of the intensity values. That is, of the co- a b Generating a digital image. (a) Continuous image. (b) A
domain of the function. c d scaling line from A to B in the continuous image, used to
illustrate the concepts of sampling and quantization. (c)
After sampling and quantization, we get
sampling and quantization. (d) Digital scan line.
f : [1, . . . ,N] × [1, . . . , M] → [0, . . . , L].
 Interlace vs. progressive scan
20

Interlaced scan is a display

signal type in which one-half
of the horizontal pixel rows
are refreshed in one cycle
and the other half in the next,
meaning that two complete
Interlaced video is a technique for doubling the scans are required to display
perceived frame rate of a video display without the screen image.
consuming extra bandwidth. The interlaced signal
contains two fields of a video frame captured at two
different times.

Progressive scan (also known as known

as 480p) is a way of displaying, storing, or
transmitting moving images in which all the
lines of each frame are drawn in one at a
time in sequential order

http://www.axis.com/products/video/camera/progressive_scan.htm Slide by Steve Seitz

 Standar test image
21

https://en.wikipedia.org/?title=
Talk:Interlaced_video/Archive_
1
 Progressive scan
22

http://www.axis.com/products/video/camera/progressive_scan.htm Slide by Steve Seitz

 Interlace scan
23

http://www.axis.com/products/video/camera/progressive_scan.htm Slide by Steve Seitz

 Rolling Shutter
24

Rolling shutter is a method of image capture in which a still picture (in

a still camera) or each frame of a video (in a video camera) is captured
not by taking a snapshot of the entire scene at single instant in time but
rather by scanning across the scene rapidly, either vertically or
horizontally. In other words, not all parts of the image of the scene are
recorded at exactly the same instant.
 The Eye
25

 The human eye is a camera!

 Iris - colored annulus with radial muscles
 Pupil - the hole (aperture) whose size is controlled by the iris
 What’s the “film”?
– photoreceptor cells (rods and cones) in the retina

Slide by Steve Seitz

 The Retina
26

Cross-section of eye Cross section of retina

Pigmented
epithelium
Ganglion axons
Ganglion cell layer
Bipolar cell layer

Receptor layer
 Retina up-close
27

Light
 What humans don’t have:
tapetum lucidum
28

The tapetum lucidum contributes to the

superior night vision of some animals.
Lying immediately behind the retina, it is a
retroreflector. It reflects visible light back
through the retina, increasing the light
available to the photoreceptors (although
slightly blurring the image).
 Two types of light-sensitive receptors
29

Cones
cone-shaped
less sensitive
operate in high light
color vision

Rods
rod-shaped
highly sensitive
operate at night
gray-scale vision
 Rod / Cone sensitivity
30
 Distribution of Rods and Cones
31

Night Sky: why are

there more stars off-
center?
Blind
Answer:

# Receptors/mm2
Fovea Spot
averted vision 150,000 Rods Rods
100,000
Averted vision is a technique for
viewing faint objects which uses 50,000 Cones Cones
peripheral vision. It involves not 0
80 60 40 20 0 20 40 60 80
looking directly at the object, but
looking a little off to the side, while Visual Angle (degrees from fovea)
continuing to concentrate on the
object

Averted vision works because there are virtually no rods (cells which detect dim
light in black and white) in the fovea: a small area in the center of the eye. The
fovea contains primarily cone cells, which serve as bright light and color
detectors and are not as useful during the night. This situation results in a
decrease in visual sensitivity in central vision at night.
 Eye Movements
32

 Saccades
 Can be consciously controlled. Related to perceptual
attention.
 200 ms to initiation, 20 to 200 ms to carry out. Large
amplitude.
 Micro saccades
 Involuntary. Smaller amplitude. Especially evident during
prolonged fixation. Function debated.
 Ocular micro tremor (OMT)
 involuntary. high frequency (up to 80Hz), small
amplitude.
 Electromagnetic Spectrum
33
 Why do we see light of these
34
wavelengths (visible light) ?
• Because that’s where the
Sun’s peak radiation was
• Because the atmosphere is
fairly transparent in that range

© Stephen E. Palmer, 2002

 The Physics of Light
35

# Photons
(per ms.)
# Photons
(per ms.)
400 500 600 700
Wavelength
400 500 600 (nm.)
700
Any patch of light can be completely described
Wavelength
physically by its spectrum: the number of (nm.)
photons
(per time unit) at each wavelength 400 - 700 nm.
 Some examples of the spectra of light
36
sources
A. Ruby Laser B. Gallium Phosphide Crystal
# Photons

# Photons
400 500 600 700 400 500 600 700
Wavelength (nm.) Wavelength (nm.)

C. Tungsten Lightbulb D. Normal Daylight

# Photons

# Photons

400 500 600 700 400 500 600 700

 Some examples of the reflectance
spectra of surfaces
37
% Photons Reflected

Red Yellow Blue Purple

400 700 400 700 400 700 400 700

Wavelength (nm)
 The Psychophysical Correspondence
38

There is no simple functional description for the perceived

color of all lights under all viewing conditions, but …...

A helpful constraint:
Consider only physical spectra with normal distributions
mean

# Photons area
variance

400 500 600 700

Wavelength (nm.)
 The Psychophysical Correspondence
39

Mean Hue
# Photons

blue green yellow

Wavelength
 The Psychophysical Correspondence
40

Variance Saturation

hi. high
# Photons

med. medium

low low

Wavelength
 The Psychophysical Correspondence
41

Area Brightness

B. Area Lightness
# Photons

bright

dark

Wavelength
 Physiology of Color Vision
42

Three kinds of cones:

440 530 560 nm .

RELATIVE ABSORBANCE (%)

100
S M L

50

400 450 500 550 600 650

W AV ELE N G TH (nm .)
 Impossible Colors
43

Can you make the cones respond in ways that typical

light spectra never would?
http://en.wikipedia.org/wiki/Impossible_colors
Some people may be able
to see the color "yellow–
blue" in this image by letting
their eyes cross so that both
+ symbols are on top of
each other
 Tetrachromatism
44

Tetrachromatism is the condition of

Bird cone possessing four independent channels for
responses conveying color information, or possessing
four types of cone cells in the eye.
Organisms with tetrachromacy are called
tetrachromats.

In tetrachromatic organisms, the sensory

color space is four-dimensional, meaning
that to match the sensory effect of
arbitrarily chosen spectra of light within
their visible spectrum requires mixtures of
at least four primary colors.

 Most birds, fish, amphibians, reptiles, insects, and some

mammals (ex. Reindeer), have cones for ultraviolet light.
 Some humans, mostly female, seem to have slight
tetrachromatism.
 More Spectra
45

metamerism is a perceived
matching of the colors with
different (nonmatching) spectral
metamers power distributions. Colors that
match this way are
called metamers.
Color blindness produced well
known instances of metamerism
 Color Sensing in Camera (RGB)
46

 3-chip vs. 1-chip: quality vs.

cost
 Why more green?

A Bayer filter mosaic is a color filter array (CFA) for

arranging RGB color filters on a square grid of
Why 3 colors? photosensors. Its particular arrangement of color filters is
used in most single-chip digital image sensors used in
digital cameras, camcorders, and scanners to create a
color image.
Slide by Steve Seitz
 Practical Color Sensing: Bayer Grid
47

Estimate RGB at
‘G’ cells from
neighboring
values

Slide by Steve Seitz

 Color Image
48

B
 Images in Matlab
49

• Images represented as a matrix

• Suppose we have a N × M RGB image called “im”
– im(1,1,1) = top-left pixel value in R-channel
– im(y, x, b) = y pixels down, x pixels to right in the bth channel
– im(N, M, 3) = bottom-right pixel in B-channel
• imread(filename) returns a uint8 image (values 0 to 255)
– Convert to double format (values 0 to 1) with im2double

column
row 0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.99 R
0.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.95 0.91
0.89 0.72 0.51
0.92
0.55
0.93
0.51
0.94
0.42
0.97
0.57
0.62
0.41
0.37
0.49
0.85
0.91
0.97
0.92
0.93 0.92 0.99 G
0.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.95
0.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.95 0.91
0.71 0.81 0.81
0.89
0.87
0.72
0.57
0.51
0.92
0.37
0.55
0.93
0.80
0.51
0.94
0.88
0.42
0.97
0.89
0.57
0.62
0.79
0.41
0.37
0.85
0.49
0.85
0.91
0.97
0.92
0.93 0.92 0.99
B
0.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.33
0.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.95
0.86 0.84 0.74 0.58 0.95
0.51 0.89
0.39 0.82
0.73 0.89
0.92 0.56
0.91 0.31
0.49 0.75
0.74 0.92 0.81 0.95 0.91
0.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89 0.79 0.85
0.96 0.67 0.54 0.85 0.89
0.48 0.72
0.37 0.51
0.88 0.55
0.90 0.51
0.94 0.42
0.82 0.57
0.93 0.41 0.49 0.91 0.92
0.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.33
0.69 0.49 0.56 0.66 0.96
0.43 0.95
0.42 0.88
0.77 0.94
0.73 0.56
0.71 0.46
0.90 0.91
0.99 0.87 0.90 0.97 0.95
0.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74
0.79 0.73 0.90 0.67 0.71
0.33 0.81
0.61 0.81
0.69 0.87
0.79 0.57
0.73 0.37
0.93 0.80
0.97 0.88 0.89 0.79 0.85
0.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93
0.91 0.94 0.89 0.49 0.49
0.41 0.62
0.78 0.60
0.78 0.58
0.77 0.50
0.89 0.60
0.99 0.58
0.93 0.50 0.61 0.45 0.33
0.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99
0.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74
0.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.97
0.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93
0.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93
0.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99
0.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.97
0.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93
 Color spaces
50

How can we represent color?

http://en.wikipedia.org/wiki/File:RGB_illumination.jpg
 Color spaces: RGB
51

Default color space

0,1,0 R
(G=0,B=0)

G
1,0,0 (R=0,B=0)

0,0,1

Some drawbacks B
(R=0,G=0)
• Strongly correlated channels
• Non-perceptual
Image from: http://en.wikipedia.org/wiki/File:RGB_color_solid_cube.png
 Color spaces: HSV
52

Intuitive color space

H
(S=1,V=1)

S
(H=1,V=1)

V
(H=1,S=0)
 Color spaces: YCbCr
53

Y=0 Y=0.5
Y
(Cb=0.5,Cr=0.5)

Cr

Cb
(Y=0.5,Cr=0.5)
Cb
Y=1
Fast to compute,
good for
compression, Cr
used by TV (Y=0.5,Cb=05)
 Color spaces: L*a*b*
54

“Perceptually uniform”* color space

L
(a=0,b=0)

a
(L=65,b=0)

b
(L=65,a=0)
 A Question
55

If you had to choose, would you rather go

without luminance or chrominance?
 If you chose without luminance
56

If you had to choose, would you rather go

without luminance or chrominance?
 If you chose without luminance
57

Only color shown – constant intensity

 If you chose without chrominance
58

Only intensity shown – constant color

Most information in intensity
 Most information in intensity
59

Original image
 Back to grayscale intensity
60

0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.99
0.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.95 0.91
0.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49 0.91 0.92
0.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.95
0.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89 0.79 0.85
0.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.33
0.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74
0.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93
0.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99
0.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.97
0.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93