EC-364 Computer Vision: Dr. Kamal M. Captain

EC-364 Computer Vision
Dr. Kamal M. Captain
SVNIT, Surat, India.
1 / 184
Cameras
Is an image of the object formed on the screen?
2 / 184
Cameras
Any point on the screen receives light from lot of point on the candle
but we do not see a clear image.
We can argue that an image is being formed but it is a blurry image
and not a clear one.
How can we create a clear image of the candle on screen?
3 / 184
Cameras
A simplest way to do this is by using a pinhole.

Pinhole is an opaque sheet with a tiny hole and it is placed between
the scene and the image plane.
4 / 184
Pinhole Camera: Perspective Projection
Figure: The pinhole imaging model
5 / 184
If the pinhole were really reduced to a point (which is of course

physically impossible), exactly one light ray would pass through each
point in the plane of the plate (or image plane), the pinhole, and
some scene point.
Perspective projection creates inverted images.
It is sometimes convenient to consider a virtual image associated with

a plane lying in front of the pinhole, at the same distance from it as
the actual image plane.
6 / 184
The images B’ and C’ of the posts B and C have the same height,
but A and C are really half the size of B.
Figure: Perspective Effects: far objects appears smaller than the closed ones
7 / 184
A single ray travels from point P on the object to the image plane
′
and projects on to the point P .
′
We want to know the relation between point P and P .
Figure: The perspective projection equations are derived from the colinearity of
the point P, its image P’ and the pinhole O.
8 / 184
A single ray travels from point P on the object to the image plane
′
and projects on to the point P .
′
We want to know the relation between point P and P .
Figure: The perspective projection equations are derived from the colinearity of
the point P, its image P’ and the pinhole O.
9 / 184
Consider a coordinate system (O, i, j, k) attached to a pinhole

camera.
The origin O coincides with the pinhole.
The vectors i and j form a basis for a vector plane parallel to the
′ ′
image plane Π , located at distance f from the pinhole along the
vector k.
′
The line perpendicular to Π and passing through the pinhole is called
the optical axis.
′ ′
The point C where it pierces Π is called the image center.
10 / 184
′
Let P denote a scene point with coordinates (x, y , z) and P denote
′ ′ ′
its image with coordinates (x , y , z ).
′ ′ ′
Since P lies in the image plane, we have z = f
′
−−→′ −→
Since the three points P, O and P are colinear, we have OP = λOP
for some number λ.
Hence,  ′
x = λx

x y
′
f
′ ′
′
y = λy ⇔λ= = = ,
 ′
 x y z
f = λz
Therefore, ( ′ ′
x = f xz ,
′ ′
y = f yz .
11 / 184
Perspective Projection of a Line
Figure: Perspective projection of a line in 3D
12 / 184
Perspective Projection of a Line
Line and a pinhole (i.e., a point) define a plane in 3D.
All the light rays that pass through the pinhole lie on this plane.
All the light rays that pass through the pinhole towards the image
plane also lie on this plane.
So the image of the 3D line on the 2D image must lie at the

intersection of this plane and the image plane.
Hence, the image of a line in 3D has to be a line in 2D.
This is why in the photographs, we see that the straight lines in the
scene maps to straight lines in the photographs.
13 / 184
Image Magnification
We have an object of certain size at a certain distance. What will be

size of the object in an image?
Figure: Image magnification
14 / 184
Image Magnification
We are going to understand this by taking a segment A0 B0 of length

d0 . The coordinates of A0 and B0 are as mentioned in the figure.
This segment lies on the plane in the scene that is parallel to the
image plane.
This segment produces an image which is another segment which is
Ai Bi and its coordinates are as mentioned in the figure.
We want to understand what will be the length of segment Ai Bi be in
an image due to the segment of length d0 in the scene?
The ratio of the length of the segment in the image to the length of
the segment in the scene is called the magnification
q
|di | dxi2 + dyi2
|m|= =q
|d0 | dx02 + dy02
15 / 184
Image Magnification
To simplify the expression, we will apply perspective projection to the

point A0 and B0 .
Using perspective projection for point A0 , we get
xi f xi x0
= ⇔ = (A)
x0 z0 f z0
Similarly,
yi f yi y0
= ⇔ = (B)
y0 z0 f z0
Similarly, for point B0
xi + dxi x0 + dx0 yi + dyi y0 + dy0
= and = (C)
f z0 f z0
16 / 184
Image Magnification
Using A and C, we can write
xi + dxi xi x0 + dx0 x0
− = −
f f z0 z0
which implies
dxi dx0
= (D)
f z0
Similarly, using B and C, we get
dyi dy0
= (E)
f z0
Using D and E in magnification expression, we get
q
|di | dxi2 + dyi2 f
|m|= =q =| |
|d0 | dx 2 + dy 2 z 0
0 0
17 / 184
Image Magnification
The magnitude of m is the absolute value of f , i.e., the effective focal

length, divided by the depth of the object in the scene.
The size or the magnification of an object in an image is inversely

proportional to its distance from the camera.
m is inverted when the image is inverted. In case of pinhole camera m

is negative.
18 / 184
Image Magnification
Image size inversely proportional to the depth.
19 / 184
Image Magnification
Some remarks:
m can be assumed to be constant if the range of scene depth ∆z is
much smaller than the average scene depth z̄.
The ratio is area of the object in the image to the area of object in
the scene is given by
areai
= m2
area0
20 / 184
Vanishing Point
Figure: Parallel straight lines converge at a single image point
Location of vanishing point depends on the orientation of parallel

straight lines.
21 / 184
Finding a Vanishing Point
We want to find vanishing point for these two parallel lines.
Figure: Parallel lines in 3D
All parallel lines in 3D share the same vanishing point.
22 / 184
We need to construct a line parallel to these lines passing through the
pinhole.
The point where this line pierces the image is the vanishing point
corresponding to this set of lines in 3D.
Figure: Location of vanishing point
How do we find this?

23 / 184
We first define the direction of the set of parallel lines. This is given
by the vector I (lx , ly , lz ).
Then, we create a point P in that direction from the pinhole.
Figure: Coordinates of the vanishing point
24 / 184
Now we perspectively project the point onto the image using
perspective projection equations.
Let the coordinates of the vanishing point be (xvp , yvp , f ).
The vanishing point of the line is the projection of point P given by

xvp f lx
= ⇔ xvp = f
lx lz lz
yvp f ly
= ⇔ yvp = f
ly lz lz
Hence, the vanishing point is having coordinates

lx ly
(xvp , yvp ) = f , f
lz lz
25 / 184
Perspective Projection is Non-linear
The perspective projection is a non-linear transformation.
The condition for linearity is
T (αx1 (t) + βx2 (t)) = αT (x1 (t)) + βT (x2 (t))
Let us take two points P1 (x1 , y1 , z1 ) and P2 (x2 , y2 , z2 ).

Now, the perspective projection of P1 is
x1 y1
x1′ = f and y1′ = f
z1 z1
Similarly, for P2 , we have

x2 y2
x2′ = f and y2′ = f
z2 z2
26 / 184
Let P3 = P1 + P2 . We get P3 = (x1 + x2 , y1 + y2 , z1 + z2 ).
Using perspective projection for P3 , we get
x1 + x2 y1 + y2
x3′ = f and y3′ = f
z1 + z2 z1 + z2
But, we have

x1 x2 x1 x2
x1′ + x2′ =f +f =f + ̸= x3′
z1 z2 z1 z2

y1 y2 y1 y2
y1′ + y2′ =f +f =f + ̸= y3′
z1 z2 z1 z2
Thus, for α = β = 1

x1 + x2 x1 x2
T [P1x + P2x ] = f ̸= f + = T [Pix + P2x ]
z1 + z2 z1 z2

y1 + y2 y1 y2
T [P1y + P2y ] = f ̸= f + = T [Piy + P2y ]
z1 + z2 z1 z2
Therefore, perspective projection is non-linear.
27 / 184
Pinhole size
What is the ideal size of a Pinhole?
Figure: Images of some text obtained with shrinking pinholes: large pinholes give
bright but fuzzy images but pinholes that are too small also give blurry images
because of diffraction effects.
28 / 184
Pinhole size
If the pinhole is too large in-place of a single ray of a light a bundle of

rays pass through and creates a blurry image as you can see with
2 mm diameter image.
As you decrease the diameter, the image becomes sharper and it is
very clear with 0.35 mm diameter.
As you further reduce the diameter, the image starts blurring again
due to diffraction effect.
Diffraction is the spreading out of waves as they pass through an
aperture or around objects. It occurs when the size of the aperture or
obstacle is of the same order of magnitude as the wavelength of the
incident wave.
Then, what is the ideal size of a pinhole?
29 / 184
Ideal Pinhole size
The ideal pinhole diameter is given by

√
d ≈2 fλ
where,
f : effective focal length
λ : Wavelength
In case of visible light images, the wavelength lies between 400 nm to
700 nm. You use this wavelength and use the effective focal length to
get the ideal pinhole diameter.
30 / 184
Image Formation using Lenses
The lens performs same projection as pinhole (perspective projection)

except that it gathers a lot more light.
Figure: Projection of point using lens
All the light rays received by the lens from point P0 are refracted or
bent by the lens to converge at a point Pi .
31 / 184
Image Formation using Lenses
The point Pi is the point where point P0 is going to be focused

behind the lens.
The bending power of the lens is defined by its focal length.
In case of pinhole camera, only green ray would have made it through
to the point Pi .
Let us understand the position of point P0 and its image Pi .
32 / 184
Gaussian Lens (Thin Lens) Law
Figure: Projection of point using lens
f : focul length
i : image distance
f : object distance
1 1 1
+ =
i o f
33 / 184
How to Find a Focal Length?
In the Gaussian lens law, if o = ∞, then f = i.

That is, if we use a source or point in a scene that is far away and
then you take image of it, then o is infinity and f = i.
Focal length: Distance at which incoming rays that are parallel to
the optical axis converge.
34 / 184
Finding Focal Length
Showing the lens an object that is far away (such as the sun a very
distant point source) and looking at where the image of that point
source is formed on the sheet of paper, the distance between the
focused image and the lens is the focal length.
The focal length is determined by few factors:

⇒ Material used (refractive index of the material)
⇒ Shape of the lens
⇒ Radius of the curvature
35 / 184
Finding Focal Length
Height of the object is h0 and we want to know the height of an
image hi .
Magnification is defined as: Magnification= hh0i
The highlighted triangles in the image are similar triangles. Using
this, we have
hi i
m= =
h0 o
36 / 184
Two Lens System
We have two lenses L1 and L2 located at d distance.
Figure: Two lens system
The magnification of complete system is a magnification due to lens

L2 times the magnification due to lens L1 .
i2 i1
Magnification:m = ·
o2 o1
37 / 184
Two Lens System
Observation: Without changing the distance between the object and

the image plane, we can move lenses L1 and L2 to change the effect
of magnification.
This is the process of zooming that is moving the lenses to change

the magnification of lens system.
38 / 184
Aperture of the Lens
The aperture of lens is the area of lens that gathers light from the
points in the scene.
It has a diameter D.
39 / 184
Aperture of the Lens
The aperture is implemented using a diaphragm.
A diaphragm is a camera component within a lens comprised of
overlapping metal blades that open and close to change the size of
the opening (they allow different levels of light to pass through to the
sensor - thus controlling the aperture.
40 / 184
f-number of lens
It is convenient to represent aperture as a fraction of focal length.

f
Aperture:D =
N
f
⇒ f-number:N =
D
where N is called the f-number of lens.
Ex: A 50nm focal length, f/1.8 lens implies:

N = 1.8 (D = 27.8nm) when aperture is fully open.
41 / 184
Lens Defocus
What is the price that we pay when we use lens?
There is only one plane in the scene that is perfectly focused onto the
image plane by a lens.
Distance o is focused at the distance i behind the lens.
If we place an image plane at distance i behind the plane, then there

is exactly one plane which is completely in focus. This plane is called
the plane of focus.
42 / 184
Lens Defocus
Any point on the plane of focus will be perfectly focused on to the

image plane.
What happens if you happen to lie outside this plane of focus?
Consider the point at a distance o ′ from the lens.
The point is closer to the lens, so it’s image will be formed behind the
image plane.
43 / 184
Lens Defocus
The light received by the lens from this point will not be focused at a
single point on the image plane but will be distributed over a circular
disc.
So the image of this point will be blurred. This is called a Blur Circle.
Let the diameter of blur circle be b. We want to know the blue circle
diameter for any given position of object in a scene.
44 / 184
Lens Defocus
To derive this we use two similar triangles.
From similar triangles,

b |i ′ − i|
=
D i′
45 / 184
Lens Defocus
The blue circle diameter:

D ′
b= |i − i|
i′
1
⇒b∝D∝
N
If we use a lens which deviates from pinhole camera more and more
(larger and larger aperture), we will get larger degree of blur for the
points that lie outside the plane of focus.
We want to express |i ′ − i| in terms of object distances o ′ and o.
46 / 184
Blue Circle (Defocus)
Using the Gaussian lens law for focused point, we have
1 1 1 of
+ = ⇒i =
i o f o−f
Similarly, for defocused point
1 1 1 ′ o ′f
+ = ⇒ i =
i ′ o′ f o′ − f
Using these two equations, we have
f f
i′ − i = · o − o′

·
(o ′ −f) o −f
Substituting in the expression for blur circle, we have
(o − o ′ ) f 2 (o − o ′ )

b = Df ′ =
o (o − f ) N o ′ (o − f )
47 / 184
Focusing
How do we focus the defocused image?
Figure: Defocused Image Figure: Move the image plane
Figure: Move the lens Figure: Move both lens and image plane
48 / 184
Depth of Filed (DoF)
Summery:
For any given image plane location, that is only one plane in the
scene that is perfectly focused.
All objects that lie outside this plane of focus are going to be out of
focus.
The degree of defocus is going to increase with the distance of the
object from the plane of focus.
Images have finite resolution which are made up of pixels of finite size.
Depth of Field:
The best focused region in the scene (range of depths in the scene
that are best focused) are those points in the scene for which the blur
circle lies within a single pixel.
As long as the blur circle diameter is smaller than the size of a pixel,
all point are going to be equally focused.
49 / 184
Depth of Field (DoF)
Range of object distances over which the image is sufficiently well”
focused, i.e., range over which blur circle diameter b is less than pixel
size.
Let’s find the depth of field.

50 / 184
Let the size of the pixel be c.

We want to see the range of distances of the object o for which the
blur circle is smaller than c.
51 / 184
Consider point o1 whose blur circle is exactly equal to c.

Since the point is closer to the lens, its image is formed behind the
image plane.
52 / 184
Consider point o2 further away from o whose image is formed in front
of the image plane and diverges from there to form a blur circle of
size c.
Depth of filed is given by o2 − o1 .
Let us compute the blur circle. We will make use of the expression for
blur circle.
53 / 184
Let o1 and o2 be the nearest and the furthest distances respectively
for which the blur circle is maximum c.
Then, using the expression for blur circle, we can write
f 2 (o − o1 )
c= (A)
No1 (o − f )
f 2 (o2 − o)
c= (B)
No1 (o − f )
Using A and B, we can get the Depth of Field as
2of 2 cN (o − f )
Depth of Field:o2 − o1 =
f 4 − c 2 N 2 (o − f )2
Note that the distance o2 − o is greater than o − o1 .
Remark: As we go closer and closer to the lens, the defocus is going
to increase more rapidly.
54 / 184
Hyperfocal Distance
The hyperfocal distance is the closest distance that you would focus
a lens at for which all point beyond that distance are going to be in
focus.
All points beyond this distance (all the way to infinity) are going to
produce a blur circle that is smaller than the pixel size.
Consider a point at infinity (o2 = ∞) and the parallel rays that come
to the lens that converge at point i2 and diverge to form a blur circle
that is exactly equal to the pixel size.
55 / 184
Hyperfocal Distance
The closest distance o = h the lens must be focused to keep objects

at infinity o2 = ∞ acceptably sharp (blur circle ≤ c).
The focused distance o for which this is true is the hyperfocal
distance.
We can derive h by using o2 = ∞ in the expression for DoF.
Using this, we get hyperfocal distance as
f2
h= +f
Nc
If we create an imaging system which is focused on the hyperfocal
distance, then all the points beyond that points will be in focus.
Only points that are closer to the lens will be out of focus.
In case of a smart phone camera, we want to set a focus at a
hyperfocal distance.
56 / 184
Aperture Size: DoF vs. Brightless
Trade-off between depth of field and the brightness of an image.
Figure: Focal length = 50 nm, Focus= 1m, Aperture D= 25 mm, f-number N =
57 / 184
Figure: Focal length = 50 nm, Focus= 1m, Aperture D= 12.5 mm, f-number N
=4
58 / 184
=8
59 / 184
=16
60 / 184
Summery:
Large Aperture (Small f-Number):
⇒ Bright Image or Short Exposure Time
⇒ Shallow Depth of Field
Small Aperture (Large f-Number):
⇒ Dark Image or Long Exposure Time
⇒ Large Depth of Field
61 / 184
Tissue Box Camera
Figure: Camera
62 / 184
Tissue Box Camera
Figure: Camera
63 / 184
Blocking the Lens
Figure: Camera
64 / 184
Blocking the Lens
Figure: Image
65 / 184
Tilting the Lens
Let us consider the scenario where we have an image plane and a

tilted lens.
White dotted line is the optical axis which passes through the center
of the lens and pierces the image plane.
66 / 184
Tilting the Lens
If we take the point on the image plane, which is at the intersection

of optical axis and image plane, we can find the point in the scene
which will be focused using the Gaussian lens law.
67 / 184
Tilting the Lens
The plane of focus will not be parallel to the image plane in this
scenario.
What is the use of such system?
68 / 184
Tilting the Lens
You have to have a camera looking out at something, but we are
interested in maintaining in focus not the plane which is parallel to
the image plane.
For example, we want to take a picture of something in front of you
on the street, and want to see the entire ground plane, the road, all
the people on the road in focus but not interested to what happens to
the sky and the things that are further up.
In this case you can tilt the lens and make entire ground plane in
focus. This construction that we just discussed is called Scheimpflug
Condition.
69 / 184
Lens related Issues: Compound Lenses
Why do we need multiple lenses?

It is extremely difficult to create an image even for the plan of focus,
which is completely focused everywhere (same quality all over the
image).
Often lens will produce higher image quality in the center of the
image as compared to the periphery of the image.
So lenses of different shapes are used in order to compensate for
these undesired effects.
70 / 184
Vignetting
We have lenses L1, L2 and L3. They all have different sizes or
openings.
When we place a point on the optical axes, lot of the light make it
way to the image sensor.
However, if we move the point A along the same plane of focus away
from the optical axes, we see that there is greater chance that rays of
light from this point are going to be blocked by the lenses as they
travel through.
71 / 184
Vignetting
For this reason images often tend to be darker towards the periphery
and this effect is called vignetting.
If you know what the vignetting of the lens is, we can correct the
images that are captured for these effects.
one nice thing is given the lens, we can measure these effects,
Figure: Brightness fall-off Figure: Brightness fall-off

(Vignetting) in image of a White (Vignetting) in image of a Natural
wall. Scene.
72 / 184
Chromatic Aberration
Lens is made of a certain material say glass or plastic which has a
certain refractive index and that is what gives lenses the bending
power.
The refractive index is the function of wavelength of light (visible
light goes from 400nm to 700nm). Blue is 400nm and red is 700 nm
and green is somewhere in between.
Since the refractive index depends on wavelength, that means the
focal length also depends on wavelength. That is, the bending power
of lens depends on wavelength.
We can see here, white light

coming in through which has
many wavelengths of light, we
can see that the red light gets
bend the least than the green
and the blue light.
73 / 184
Chromatic Aberration
So the blue image is focused at the different distance from the red
image with respect to the image plane.
This causes some shifts in color in the image.
Below is an image of a printed shit of paper which is black and white
but even so you can see that at edges the colors are shifted.
This is called Chromatic Aberration.
74 / 184
Geometric Distortion
Radial Distortion
As you move away from the center of the image, points tend to get
pushed out more and more. It is also called barrel distortion.
If you know exactly what the barrel distortion is, you can correct for it.
Figure: Radial Distortion
75 / 184
Geometric Distortion
Tangential Distortion
In the case of tangential distortion you have a slight twisting of the
image.
As you go further and further away from the center of the image, the
twisting increases.
These are due to imperfections in the lenses.
Figure: Tangential Distortion
76 / 184
Geometric Distortion Correction
Image is taken with inexpensive wide angle camera.
The straight lines in the scene no longer map to the straight lines
because its not perspective projection. There is a barrel distortion.
If you know what the barrel distortion is, you can apply a software
mapping to get pure perspective image.
All straight lines are maps to straight lines in the corrected image.
The field of view is not rectangular because the field of view of
original image itself due to barrel distortion was not rectangular.
Figure: Radial (Barrel) Distortion Figure: Undistorted Image

77 / 184
Image Sensing
78 / 184
Image Sensing
We now know how to capture an optical image.
Need to convert optical image to digital images (numbers) for

computer representation and use.
79 / 184
Brief History of Imaging
16th Century
Figure: Camera Obscura
80 / 184
17th Century
Figure: A camera obscura box with mirror, with an upright projected image at the
top.
81 / 184
19th Century
Figure: Still Life, Louis Jaques Mande Daguerre, 1837.
82 / 184
19th Century
Figure: Louis Ducos du Hauron, 1887.
83 / 184
Color Film: 19th Century
Figure: Louis Ducos du Hauron, 1887.
84 / 184
Consumer film camera: 1928
Figure: Ernemann camera.
85 / 184
Silicon Image Detector: 1970
Figure
86 / 184
Digital Camera: 1994
Figure
87 / 184
Phones with Cameras: 2007
Figure: iPhone 1
88 / 184
Converting Light into Electric Charge
Photon with sufficient energy incident on a Si atom creates an

electron hole pair.
Figure
89 / 184
Image Sensor
Figure: 18 Megapixels
Types of Image Sensor

1 Charge Coupled Device (CCD)
2 Complementary Metal Oxide Semiconductor (CMOS)
90 / 184
Radiometry
Let say we have an image of a scene and at a particular pixel we

measure the brightness value to be x.
The problem is: What is that value x tells us about the corresponding
point in the scene?
This is the problem of image intensity understanding.

Two concepts will help us understand this:
⇒ Radiometric concepts: The concepts related to light such as
brightness of light sources, scenes, etc.
⇒ Reflectance: The ability of a surface or a material to take light from
one direction and reflect in the other direction.
91 / 184
From 2D to 3D
Figure: Typical vision system
92 / 184
From 2D to 3D
Figure: Typical vision system
93 / 184
Image Intensity
We have an object, a light source and a camera.
Let us consider a point on the object. The light from the source falls
on to this point and reflected or scattered in different directions.
The light that is scattered in the direction of camera is projected by

the lens of the camera on to the image plane where we have an image
intensity value.
What is this image intensity value say about the corresponding point
in the scene?
94 / 184
Image Intensity
Let us take a point on a surface.
A point light source illuminates the point and it reflects the light in
the direction of the camera that produce an image intensity
corresponding to that point.
The factors that influence the image intensity.

Image Intensity =f (Illumination, Surface Orientation,
Surface Reflectance)
95 / 184
Image Intensity
Image intensity understanding is under-constrained.
Under certain setting and assumptions, we can get some information

about the corresponding point in the scene from a single intensity
value.
Radiometry and Reflectance
To interpret image intensities, we need to understand Radiometric

Concepts and Reflectance Properties.
1 Radiometric Concepts (e.g., quantify brightness of light source,
illumination of a surface, brightness of a surface.)
2 Surface Radiance and Image Irradiance
3 BRDF: Bidirectional Reflectance Distribution Function
4 Reflectance Models
5 Dichromatic Model
96 / 184
Concept: Angle in (2D)
We have a circle of radios r . We want to know the radius of the angle
covered by the arc of length dl.
dl
dθ = Unit: radian (rad)
r
The angle here is a dimensionless quantity.

However, radian is used as its unit to distinguish from other
dimensionless quantities.
97 / 184
Concept: Solid Angle in (3D)
Let a point looking at infinitesimally small area dA.
We want to know what is the angle in 3D called solid angle
subtended by this area.
dA′ dA cos θ
dω = 2
= Unit: steradian (sr)
r r2
A solid angle is a measure of the amount of the field of view from
some particular point that a given object covers.
98 / 184
Concept: Light Flux
Light source is illuminating a surface.
Power emitted by the source within the solid angle.
It is denoted by dΦ, and its unit is watts (W).
99 / 184
Concept: Radiant Intensity
Brightness of the source is called the radiant intensity and it is

denoted as J.
The Radiant Intensity is the flux emitted by the source per unit
solid angle.
dϕ
J= Unit: W sr −1
dω
100 / 184
Concept: Surface Irradiance
We can now talk about the illumination of a surface which is at some
distance r from the light source having radiant intensity J.
What is illumination of a surface?
The illumination of a surface is called Surface Irradiance which is
denoted by E .
It is given by the flux falling on the surface per unit area.
dϕ
E= Unit: W m−1
dA
101 / 184
Concept: Surface Irradiance
If we have a source with radiance intensity J, we can now write

surface irrediance as
dω
E =J
dA
We know dω, that is, the solid angle in this case. Hence, we can write
dA cos θ
r2 J cos θ
E =J =
dA r2
Observations:
⇒ Surface irradiance depends on the radiant intensity of the source.
⇒ Surface irradiance decreases with increase in the distance between the
point light source and the surface. (r 2 fall off)
⇒ It is proportional to the cos θ, where θ is the angle between the
direction of the light source and the surface orientation.
⇒ When the source is top on the surface, the irradiance is going to be
maximum as θ = 0.
102 / 184
Concept: Surface Radiance
Light flux emitted per unit foreshortened area per unit solid flux.
d 2Φ
L= unit: W m−2 sr −1
(dAcosθr ) dω
Observations:
⇒ L depends on direction θr : L (θr ).
⇒ Surface can radiate into the whole hemisphere.
⇒ L depends on reflectance property of the surface.
103 / 184
Scene Radiance and Image Irradiance
We have a point on a surface which is illuminated by the light source

that produces a radiance in the direction of a camera.
The radiance of the point is L and this scene radiance is projected by
the lens of camera on the image plane to produce an image irradiance
(E) or image brightness.
We want to understand the relationship between L and E .
104 / 184
We have a surface in the scene that we are observing, a lens and an

image plane.
We assume that the distance between lens and the image plane is f ,
that is, the effective focal length.
105 / 184
We are looking at the intensity captured by the pixel having area dAi .
This pixel is observing through perspective projection an area dAs in
the scene.
Lets assume that the patch has an orientation n̄ which makes an
angle θ with the line of sight shown in yellow line.
The line of sight makes an angle α with the optical axis.
Depth of this patch with respect to the lens is z.
106 / 184
We aim to derive set of equations that provide relationship between

scene radiance and image irradiance.
The solid angle subtended by the patch with respect to the lens is
dωs .
The solid angle subtended by the pixel with respect to the lens is dωi .
107 / 184
We can see that: dωi = dωs .
We can compute dωi and dωs as
dAi cos α dAs cos θ
dωi = 2
and dωs =
(f /cos α) (z/cos α)2
Using dωi = ωs , we get
dAs cos α z 2
= (1)
dAi cos θ f
108 / 184
The solid angle subtended by the lens from the surface point.
Let say the lens has a diameter d.
2
The foreshortened area of the lends is given by πd4 cos α.
Using this, the solid angle is given as
πd 2 cos α
dωL = (2)
4 (z/cos α)2
109 / 184
Energy Conservation:
Flux received by lens from dAs = Flux projected onto dAi
How do we know what is the flux received by the lens?
Let say the radiance of the patch in the scene is L. We know the
equation for L
d 2ϕ
L=
(dAs cos θ) dωL
Using this, the flux received by the lens from dAs is
d 2 ϕ = L (dAs cos θ) dωL (3)
This flux received by the lens is gets bent by the lens and it falls on
the pixel with area dAi .
Therefore, the Image Irradiance is given by
dϕ
E= ⇒ dϕ = E · dAi (flux projected on dAi ) (4)
dAi
110 / 184
We have these four equations:
dAs cos α z 2 πd 2 cos α

= dωL =
dAi cos θ f 4 (z/cos α)2
d 2 ϕ = L (dAs cos θ) dωL dϕ = E · dAi

Using these four equations, we can derive the relationship between
scene radiance L and image irradiance E .
2
π d
E =L cos2 α
4 f
We have d/f term in the equation, which the equal to (1/effective

F-number).
111 / 184
Relationship between scene radiance L and image irradiance E .

2
π d
E =L cos2 α
4 f
Observations:
1 Image Irradiance is proportional to scene radiance.
2 Image brightness falls off from the image center as cos4 α.
3 As we move away from the center of the image, that is the optical axis,
the angle α increases so cos4 α decreases.
4 For small fields of view, effects of cos4 α is small.
112 / 184
Does image brightness vary with scene depth?

2
π d
E =L cos2 α
4 f
If you move the camera away from the scene, does the brightness of
the scene point change?
Intuitively we might think that as we move away from the scene and
therefore collect less light and the brightness should reduce.
If we look at the expression for E , the distance between the point in

the scene and the lens (z) doesn’t show up in the expression.
Therefore, image brightness doesn’t vary with scene depth.
113 / 184
Does image brightness vary with scene depth? No

2
π d
E =L cos2 α
4 f
Let us see why this happens.

We have the set up shown in figure. Let say our camera has a single
pixel.
114 / 184
The area subtended by the scene onto the pixel are shown with yellow
line. If we pull the camera back, that area increases.
That means the same pixel is going to be accumulating light from a
larger area, which seems to imply that the brightness of the pixel is
going to increase.
But we know that is not the case.
115 / 184
Larger the scene depth, larger the area of light accumulation.
On the other hand, we also have the solid angle subtended by the lens
from each point in the scene. That also governs how much light is
being collected.
That is large when the lens is closer to the scene and falls as it moves
away.
These two effects cancel each other as a result of which image
irradiance is independent of scene depth.
116 / 184
Bidirectional Reflectance Distribution Function (BRDF)
The brightness of the point in the scene is dependent on its material

properties or its reflectance properties.
We want to come up with a way to represent the reflectance property

of any material that brings us to BRDF.
117 / 184
Surface Appearance
118 / 184
Surface Reflectance
Two directions are important to us. One is the direction from which
the light arrives and the second is the direction in which the light is
reflected and observed.
To describe the reflectance property of any material, we want to be
able to describe its properties both in terms of the illumination
direction and the viewing direction.
119 / 184
Bidirectional Reflectance Distribution Function
How do we describe the direction?
We can do it using two angles, θ which is the zenith angle and ϕ

which is the azimuth angle which lies in the x-y plane.
The direction of the light source, the direction from which the light
arrives is (θi , ϕi ).
The direction in which the light is being reflected is (θr , ϕr ).
120 / 184
E (θi , ϕi ): Irradiance due to source in direction (θi , ϕi ).
L (θr , ϕr ): Radiance of surface in the direction (θr , ϕr ).
Now, we can define the BRDF as
L (θr , ϕr ) 1
BRDF : f (θi , ϕi , θr , ϕr ) = Unit :
E (θi , ϕi ) sr
The radiance in the direction (θr , ϕr ) due to irrediance from the

direction (θi , ϕi ).
It is a four dimensional function.
121 / 184
The function f fully describes the reflectance property of any point in

the scene.
Properties:
1 f (θi , ϕi , θr , ϕr ) > 0.
2 Helmholtz Reciprocity:
f (θi , ϕi , θr , ϕr ) = f (θr , ϕr , θi , ϕi )
122 / 184
BRDF of Isotropic Surfaces
For rotationally symmetric reflectance (Isotropic Surfaces), BRDF is a

3-D function:
f (θi , θr , ϕi − ϕr )
123 / 184
Isotropic BRDF and Anisotropic BRDF
124 / 184
Reflection Mechanism
Surface Reflection:
Specular Reflection
Glossy Appearance
Smooth Surfaces (e.g., mirror, glass, etc)
125 / 184
Reflection Mechanism
Body Reflection:
Diffuse Reflection
Matte Appearance
Non-homogeneous Medium (e.g., clay, paper, etc)
Image Intensity=Body Reflection+Surface Reflection
126 / 184
Examples
Figure: Body Reflection
Figure: Surface Reflection
Figure: Hybrid Reflection (Body+Surface)
127 / 184
Lambertian Model (Body)
Surface appears equally bright from all directions.
Doesn’t matter which direction you observe it from, it has the same
surface radiance.
It’s BRDF is a constant
ρd
f (θi , θr , ϕi , ϕr ) =
π
where ρd is called the Albedo which is in the range 0 ≤ ρd ≤ 1.
Albedo is 0 for perfectly black surface and 1 for perfectly white
surface.
Perfectly white surface reflects all the light that it receives.
128 / 184
This is a very commonly used model is computer vision and computer
graphics because of its simplicity and also because it also descries at
least approximately a variety of surfaces found in the real world.
Radiance is proportional to irradiance:
▷d
L= E
π
We know that
J cos(θi ) J
E= = 2 (n̄ · s̄)
r2 r
where n̄ is the unit vector in the direction of surface orientation and s̄
is the unit vector in the direction of source.
129 / 184
Hence, we get surface radiance as
ρd J
L= (n̄ · s̄)
π r2
We know that J and ▷d are constants, let us assume that r is also
constant. We want to understand that what happens to Lambertian
surface when we move source at a fixed distance.
Figure
130 / 184
Ideal Specular Model (Surface)
Perfect Mirrors: All incident energy is reflected in a single direction.
All the reflection takes place at the interface itself. So it is surface
reflection not body reflection.
Mirror BRDF:
δ (θi − θr ) δ (ϕi + π − ϕr )
f (θi , θr , ϕi , ϕr ) =
cos θi cos θr
The viewer receives light only when v̄ = r¯.
Figure
131 / 184
Ideal Specular Model (Surface)
Mirror BRDF:
δ (θi − θr ) δ (ϕi + π − ϕr )
f (θi , θr , ϕi , ϕr ) =
cos θi cos θr
The first delta function essentially says that the two zenith angles
must be the same.
The second delta function says that the azimuthal angles must be the
opposite of each other.
The term in the denominator is a factor of proportionality. It is just
making sure that the low of conservation of energy is being satisfied.
That is, the light that is reflected is equal to the light that is reflected.
132 / 184
Examples
Let us assume that we are looking at the sphere which is Lambertian,

and the viewing direction is v̄ and the source direction is s̄.
Source and camera are very far (infinity) from the sphere.
The image that is captured by the camera is shown on right. We can
see that the brightest point in the image is the point p for which the
angle of incidence is 0.
We know that for a Lambertian surface the point is brightest when
illumination comes from top.
We also have the iso-brightness contours on the sphere, which have
equal brightness because of angle of incident for all those points are
same. 133 / 184
Examples
In this case the camera observes a reflection non-zero value only for
one point on the sphere, and that point is q.
The point q has a surface normal which is a bisector of v̄ and s̄ and
that is the point for which the angle of incident and the angle of
reflection are going to be same.
These are very simple but very popular models because we know in
real life we have shiny object and also objects which are matte
Lambertian like objects in real world.
134 / 184
Surface roughness
Let us assume that we are looking at one pixel in the image.

This pixel projects onto some area on the surface.
Irrespective of the reflection mechanism being surface or body, the
patch we are looking at will not be perfectly smooth. It is going to
have some roughness.
We want to come up with a way to model the roughness of a surface
and then see how it changes the reflection model (Lambertian or
Specular).
135 / 184
Modeling Surface roughness
Micro-Facet structure for rough surfaces:
Imagine that the surface is made up of a lot of tiny little facets. The
surface has a mean orientation n̄.
Each facet has its own orientation α.
We can now define the roughness of the surface using a distribution.
Gaussian Micro-Facet Model:
1 α2
p (α, σ) = √ e − 2σ2
σ 2π
where σ is the roughness parameter.
136 / 184
Modeling Surface roughness
Greater the value of σ more is the roughness.
If σ = 0 then all the facets looking up in the direction of the mean
orientation and we get a perfectly flat surface.
This kind of roughness models are used with both the Lambertian
model and the specular model.
That is, the assumptions are made that the facets are made up of
either perfectly mirror like or perfectly Lambertian. Then we look at
what is the aggregate reflection of entire patch we are looking at.
137 / 184
Specular Reflection from Rough Surface
Torrance-Sparrow BRDF Model: (Each Facet is a Perfect Mirror)

ρs
f (s̄, v̄ ) = p (α, σ) G (s̄, n̄, v̄ )
(n̄ · s̄) (n̄ · v̄ )
where p (α, σ) is the surface roughness distribution and G (s̄, n̄, v̄ ) is the
geometric factor (masking, shadowing).
Each one of the facets have its own orientation. When we calculate
the BRDF, we calculate it for the entire ensemble of facets.
ρs is the reflectivity of each facet. Note that, even if it reflects like a

mirror, it is not necessary that it reflects 100 % of the light that it
receives.
138 / 184
The function G takes into account various geometric effects that
happen when you have a surface that is not flat.
Let us consider that we have a V groove in the structure.
If the lighting a coming from the direction of yellow arrows, one facet
could cast a shadow on the other facet.
If we are observing from the direction of red arrows, one facet could
mask the visibility of the other facet.
The function G takes into account all these effects.
If we substitute σ = 0, we get the BRDF of a perfect mirror, a
specular surface.
139 / 184
140 / 184
141 / 184
Body Reflection from Rough Surface
Oren-Nayar BRDF Model: (Each Facet is Limbertian)
ρd
f (θi , θr , ϕi , ϕr ) = (A + B · max (0, cos (ϕr − ϕi )) sin α tan β)
π
σ2
A=1− α = max (θi , θr )
2 (σ 2 + 0.33)
0.45σ 2
B= β = min (θi , θr )
σ 2 + 0.09
where σ is the surface roughness.
Figure
142 / 184
Figure
143 / 184
Figure
144 / 184
Figure
145 / 184
Dichromatic Model
We have discussed two mechanisms of reflection: Body reflection and

surface reflection.
What will be the effect of each one of these mechanisms on the color
of incident light.
That is, when we have the surface radiance or the reflection light by a
surface, what is the relationship between the color of the light that is
reflected by the surface and the color of the light that is incident on
the surface?
That brings us to the dichromatic reflectance model.
146 / 184
Color Reflectance Model
Figure
Color of surface reflection (specular) reflection= color of illumination

Color of body (diffuse) reflection=color of object × color of
illumination
147 / 184
Color Reflectance: Dichromatic Model
Pixel color is a linear combination of the color of body reflection and
the color of surface reflection.
Look at it in the color space.
Any pixel that we measure on this object (assuming object is uniform
in terms of its material properties) is a linear combination of these
two vectors.
148 / 184
Photometric Stereo
We plan to develop a method to recover three dimensional shape

information from image intensity values. This method is photometric
stereo.
The factors that affects image intensity values
Image Intensity I = F (Source, Surface Normal n̄, Surface Reflectance)
149 / 184
Photometric Stereo
We are going to take the following assumption
Image Intensity (known)I = F(Source (known),
Surface Normal n̄ (unknown),
Surface Reflectance) (known)
We are going to assume that the source is at a large distance so that
every point on the surface end up seeing the same source direction.
Photometric stereo is a method for recovering 3D shape information
from image intensities measured using multiple sources.
150 / 184
Surface Gradient and Normal
Let us look at a way to represent surface orientation information that

is called the gradient space.
Let us say we have a surface z = f (x, y ). At each xy, we know z,
which is the depth of the surface.
The surface gradient is defined as

∂z ∂z
− ,− = (p, q)
∂x ∂y
(p,q) is called the gradient of the surface.
151 / 184
Surface Gradient and Normal
The surface normal is now defined as

∂z ∂z
N = − , − , 1 = (p, q, 1)
∂x ∂y
Unit surface normal:
N (p, q, 1)
n= =p
||N|| p2 + q2 + 1
The surface normal is represented with only two parameters (p, q).
152 / 184
Gradient Space
Let us visualize the relationship between the normal vector and the
corresponding (p, q) value.
Let us consider a coordinate system x-y-z and a surface normal unit
vector n̄.
We form a plane at z=1 which is parallel to the x-y plane.
Axes p and q are parallel to axes x and y.
The plane z=1 is called the gradient space or pq plane.
153 / 184
Gradient Space
If we extend the unit vector to intersect the pq plane, where ever it
intersects the plane is the normal vector N(p, q, 1).
That is the relationship between any unit vector and the
corresponding (p, q) value.
Every point (p, q) on the pq space corresponds to a unique
orientation.
Source Direction:
S (ps , qs , 1)
s= =p
||S|| ps2 + qs2 + 1
154 / 184
Reflectance Map R (p, q)
We assume that we are given reflectance property of the surface, i.e.,
BRDF.
We are also given all the information about the source that
illuminating the surface.
We assume that there is a single source of a certain brightness at a
certain distance and in the direction s.
Reflectance map: For a given source direction s and surface
reflectance, image intensity at a point (x, y ).
I = R(p, q)
155 / 184
Reflectance Map R (p, q)
It relates the image intensity that we measure to the surface normal

or the gradient at that point, pq.
Once we have the reflectance map, if you tell me what the surface
orientation is at a point, we can plug that into the reflectance map
and tell what intensity that the surface orientation, that the scene
point would produce.
We want to do the reverse, we want to go from image intensity to the
surface normal.
156 / 184
Lambertian Surface
Image intensity I is independent of viewing direction.
Figure: Lambertian (Diffuse or Matte) Surface
157 / 184
Reflectance Map: Lambertian Surface
The reflectance map of Lambertian surface that is illuminated by a
source in certain direction with a certain brightness is given by
ρ J ρ
Image Intensity :I = c 2
cos θi = c k (n · s)
πr π
where k is the source brightness, ρ is the surface Albedo
(Reflectance) and c is a constant (Camera Gain).
In this case, we know c, ρ, J and r . We can assume c πρ k = 1, then
I = cos θi = n · s.
158 / 184
n and s are unit vectors.
We can express this in gradient space. We know that n is (p, q, 1)
and s is (ps , qs , 1). Using this, we have
pps + qqs + 1
I =n·s = p p = R (p, q)
p 2 + q 2 + 1 ps2 + qs2 + 1
(ps , qs ) is the source direction.
What are contours in the reflectance map that have the same
brightness?
Figure: Reflectance Map 159 / 184

Reflectance Map: Iso-Brightness Contours
What are surface normals or (p, q) values that produce the same
brightness in the image?
We know that, for a Lambertian surface, the surface normals that
subtend the same angle with respect to the source direction. These
surface normals lie of the cone shown in the figure.
All these surface normals would produce the same brightness because
for them the cos θi is the same.
We now extend this cone to intersect the z=1 plane. The ellipse that
we see here corresponds to iso-brightness contour in the reflectance
map.
160 / 184
Figure: Reflectance Map
For what (p, q) values the brightness value go to 0?

We can plug in I=0 in the reflectance map equation. We will get the
equation of line which is shown in Figure.
pps + qqs + 1
I =n·s = p p = R (p, q)
p + q 2 + 1 ps2 + qs2 + 1
2
161 / 184
Shape from Single Image?
Can we estimate the shape of a surface from a single image?
Given Image I , source direction s and surface reflectance, i.e,
reflectance map R(p, q). Can we estimate the surface gradient (p, q)
at any particular point? NO.
For example, as shown in figure, lets take one intensity value. This
intensity value corresponds to an iso-brightness contour in the
reflectance map.
Therefore, there are an infinite number of (p, q) values that would
have produced the same brightness value.
Figure: Reflectance Map

162 / 184
Photometric Stereo
Idea: Use multiple images under different lighting to resolve the
ambiguity in surface orientation.
It uses multiple image of the object taken from the same camera
location, multiple images taken under different lighting conditions,
different sources.
This method was developed by Robert Woodham.
We use the multiple intensity values measured at the same pixel to
resolve the ambiguity in surface orientation.
163 / 184
Photometric Stereo
The setup for photometric stereo is shown in Fig. We are looking at
one point on the surface but the same algorithm is applied to every
point on the surface.
It is assumed that the camera is the same for all the points on the
surface and the source directions are also the same for all the points
on the surface. In other words the camera and sources are far away
from the surface.
We have three light sources S1 , S2 and S3 . Let us assume the point
of interest has an orientation n and the viewing direction is v .
164 / 184
Photometric Stereo
Notations:
Direction of source i: si = (Psi , qsi )
Reflectance map for source i, Ri (p, q)
Image intensity produced by source i : Ii (x, y )
165 / 184
Photometric Stereo: Basic Idea
Figure: Reflectance map for source S1
166 / 184
Figure: Reflectance maps for source S1
167 / 184
Figure: Reflectance map for source S1 and S2
168 / 184
Figure: Reflectance map for source S1 , S2 and S3
169 / 184
Step 1: Acquire K images from K known light sources.
Step 2: Using known source direction and BRDF, construct

reflectance map for each source direction.
Step 3: For each pixel location (x, y ), find (p, q) as the intersection
of K curves. This (p,q) gives the surface normal at pixel (x,y).
Smallest K needed depends on the material properties.

Example: K = 3 for Lambertian surfaces.
The assumption is that each point on the surface can see the three
light sources.
If any of the source is obstructed or outside the visible hemisphere of
any point, then photometric stereo is not possible.
170 / 184
Photometric Stereo: Lambertian Case
Image intensities measured at point (x, y ) under each of three light

sources in case of Lambertian surface:
ρ ρ ρ
I1 = n · s1 I2 = n · s2 I3 = n · s3
π π π
   
nx sxi
where n = ny and s = syi 
  
nz szi
We can write these three equation in the matrix form as
   
I1 sx1 sy1 sz1
ρ
I2  = sx2 sy2 sz2  n
π
I3 sx3 sy3 sz3
Here, I is known , the source matrix is known, the albedo and the
vector n are unknown.
171 / 184
Photometric Stereo: Lambertian Case
The equation can be written as

   
I1 sx1 sy1 s z1
I2  = ρ sx2 sy2 sz2  n ⇒ I = SN
π
I3 sx3 sy3 s z3
where N = πρ n
Solution:
N = (S)−1 I
Surface normal:
N
n=
||N||
Albedo:
ρ
= ||N||
π
172 / 184
When does it nor work?
When S3×3 is not invertible.

When one source direction can be represented as a linear combination
of the other two.
S3 = αS1 + βS2
Figure: All sources and the origin lie on a plane
173 / 184
More sources than minimum needed
Generally, we don’t want to do photometric stereo using minimum
number of sources that we need.
We generally want to use as many sources the application allows us
to. This way we get more robust result.
We get better results by using more than three (K > 3) light sources.
   
I1 sx1 sy1 sz1
 I2  ρ  sx sy sz  ρ
 2 2 2
 ..  =  .. ..  n ⇒ I = SN where N = n
 
..
. π . . .  π
IK sxK syK szK
SK ×3 is not a square matrix and hence invertible.

Solution: Use least square estimation
−1
S T I = S T SN ⇒ N = S T S ST I
174 / 184
”Effective” point source from multiple sources
If we are looking at a Lambertian surface and we have a large number
of light sources or extended or area light source, these light sources if
they are ON at the same time can be replaced by a single point light
source.
This is possible only in case of Lambertian surface. This is what we
call an effective point light source for multiple sources.
Let say we are looking a point on Lambertian surface having
orientation n, and the viewing direction is v . Let say we have a source
in the direction s1 having brightness k1 which is its radient intensity
divided by the square of its distance from the point.
175 / 184
The image intensity measured due to source s1 will be I1 = πρ k1 n · s1 .
Let say we turn on the second source at the same time.
ρ ρ
I1 = k1 n · s1 I2 = k2 n · s2
π π
Since both sources are ON at the same time, the brightness of the
scene point is
ρ ρ
I = n · (k1 s1 + k2 s2 ) = k̂n · ŝ
π π
We are saying the magnitude of resultant source vector is k̂ and the
direction of vector is ŝ.
176 / 184
We are saying that if we turn on the two light sources at the same
time, and its a Lambertian object, as long as all points on the
Lambertian object end up seeing both the light sources, we can
basically replace those two light sources with a single point light
source with brightness k̂ and direction ŝ.
This can be extended to an are light source or extended light source.
ρ X ρ
I = n· ki si = k̂n · ŝ
π π
i

k̂, ŝ : Effective source is in direction of ”Centroid” of the sources.
177 / 184
Results: Lambertian Sphere
In this example, we have a sphere and it is illuminated by five sources.

Using these images, we can compute (p, q) values at each point.
(p,q) values can be used to compute surface normals at each point on
the surface.
178 / 184
Results: Lambertian Sphere
The surface normal map is shown in figure.

The length of the vector that we can see represents the tilt of the
normal with respect to the viewing direction.
The estimated albedo is shown in Figure.
179 / 184
Results: Lamberian Mask
This is the case of pained mask. In this case three light sources are
used (Shallow surface).
180 / 184
Results: Lamberian Mask
We are using color images to represent the albedo.

The photometric stereo algorithm is applied on each color channel
separately, to get the color albedo for each channel.
We can use surface normal computed from any one of those channels
or we can combine them in some way to reduce noise and so on.
181 / 184
Results: Toy
182 / 184
Shape from Surface Normals

∂z ∂z
− ,− = (p, q)
∂x ∂y
183 / 184
References
1 Computer Vision: A Modern Approach, Second Edition, by David A.

Forsyth and Jean Ponce.
2 Video Lectures on ”Radiometry and Reflectance/ Photometric
Stereo” from the lecture series ”First Principles of Computer Vision”
by Prof. Shree Nayar, Computer Science Department, School of
Engineering and Applied Sciences, Columbia University.
184 / 184

EC-364 Computer Vision: Dr. Kamal M. Captain

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

EC-364 Computer Vision: Dr. Kamal M. Captain

Uploaded by

Copyright:

Available Formats

EC-364 Computer Vision

Dr. Kamal M. Captain

SVNIT, Surat, India.

Is an image of the object formed on the screen?

A simplest way to do this is by using a pinhole.

Figure: The pinhole imaging model

If the pinhole were really reduced to a point (which is of course

Perspective projection creates inverted images.

It is sometimes convenient to consider a virtual image associated with

Consider a coordinate system (O, i, j, k) attached to a pinhole

The origin O coincides with the pinhole.

Figure: Perspective projection of a line in 3D

Line and a pinhole (i.e., a point) define a plane in 3D.

So the image of the 3D line on the 2D image must lie at the

We have an object of certain size at a certain distance. What will be

Figure: Image magnification

We are going to understand this by taking a segment A0 B0 of length

To simplify the expression, we will apply perspective projection to the

The magnitude of m is the absolute value of f , i.e., the effective focal

The size or the magnification of an object in an image is inversely

m is inverted when the image is inverted. In case of pinhole camera m

Image size inversely proportional to the depth.

Figure: Parallel straight lines converge at a single image point

Location of vanishing point depends on the orientation of parallel

We want to find vanishing point for these two parallel lines.

Figure: Parallel lines in 3D

All parallel lines in 3D share the same vanishing point.

Figure: Location of vanishing point

How do we find this?

Figure: Coordinates of the vanishing point

Let the coordinates of the vanishing point be (xvp , yvp , f ).

The vanishing point of the line is the projection of point P given by

Hence, the vanishing point is having coordinates

T (αx1 (t) + βx2 (t)) = αT (x1 (t)) + βT (x2 (t))

Let us take two points P1 (x1 , y1 , z1 ) and P2 (x2 , y2 , z2 ).

Similarly, for P2 , we have

If the pinhole is too large in-place of a single ray of a light a bundle of

The ideal pinhole diameter is given by

The lens performs same projection as pinhole (perspective projection)

Figure: Projection of point using lens

The point Pi is the point where point P0 is going to be focused

The bending power of the lens is defined by its focal length.

Let us understand the position of point P0 and its image Pi .

Figure: Projection of point using lens

In the Gaussian lens law, if o = ∞, then f = i.

The focal length is determined by few factors:

Figure: Two lens system

The magnification of complete system is a magnification due to lens

Observation: Without changing the distance between the object and

This is the process of zooming that is moving the lenses to change

It is convenient to represent aperture as a fraction of focal length.

Ex: A 50nm focal length, f/1.8 lens implies:

If we place an image plane at distance i behind the plane, then there

Any point on the plane of focus will be perfectly focused on to the

To derive this we use two similar triangles.

From similar triangles,

The blue circle diameter:

We want to express |i ′ − i| in terms of object distances o ′ and o.

How do we focus the defocused image?

Figure: Defocused Image Figure: Move the image plane

Let’s find the depth of field.

Let the size of the pixel be c.

dAs cos α z 2 πd 2 cos α