P. 1
Robot Vision 15678

Robot Vision 15678

|Views: 496|Likes:
Published by Karan Gala

More info:

Published by: Karan Gala on May 04, 2011
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

01/31/2013

pdf

text

original

Sections

  • 1. Introduction
  • 2. Image Formation & Image sensing
  • 3. Binary images : Geometric properties
  • 4. Binary Image : Topological properties
  • 5. Regions & image segmentation
  • 6. Image Processing : Continuous images
  • 7. Image Processing – Discrete Image
  • 8. Edge & Edge detection
  • 9. Lightness & color
  • 10. Reflectance Map : photometric stereo
  • 11. Reflectance Map : shape from shading
  • 12. Motion field & Optical flow
  • 13. Photogrammetry & Stereo
  • 14. Pattern classification

ROBOT VISION

Lecture Notes




로봇 비젼
강의 노트





Course No. 15678
Credits Hours 3





Instructor:
Professor
Choi, Tae-Sun

Course outline:

The principles of the machine/robot vision are introduced. It covers image formation,
pattern classification, motion and optical effect for object recognition. In addition, the
design technology of the robot vision system with optical device is studied.

Prerequisite: Digital Signal Processing, Image Processing.

Textbook and References: 1. Robot Vision, B. K. P. Horn, MIT Press.
2. Computer Vision, Dana Ballard and Christopher
Brown, Prentice Hall.
Course Schedule
1. Image Formation & Image Sensing
2. Binary Images: Geometrical Properties
3. Binary Images: Topological Properties
4. Regions & Image Segmentation
5. Image Processing: Continuous Images
6. Image Processing: Discrete Images
7. Edge & Edge Finding
8. Lightness & Color
9. Reflectance Map: Photometric Stereo
10. Reflectance: Shape from Shading
11. Motion Field & Optical Flow
12. Photogrammetry & Stereo
13. Pattern Classification
14. Polyhedral Objects
15. Extended Gaussian Images
16. Passive Navigation & Structure from Motion

1
1. Introduction
I. Human vision system

Figure 1—1
• Rod : more sensitive to light
Scotopic vision
Slender receptor

• Cone : shorter & thicker in structure
Photopic vision : R.G.B
6.5 million

• Fovea : Density of cones is greatest
The region of sharpest photopic vision Camera visual system

2

Figure 1—2
i. Photometric information
brightness : (ex: bright green, dark green)
color (ex, R. G. B)
ii. Geometric information
shape

Figure 1—3

2
NTSC : US standard for TV picture and audio coding and transmission
(timing calculation)
For one frame : 512×512 = 256k Byte
For RGB : 3
For real time : 30 frames/sec
256k × 3 × 30 = 22.5M Byte/sec
180 M bit/sec
180 Mbps T1 1.544 Mbps
Compression ratio :
180
120
1.544

Wavelet compression
MPEG I, II, IV
Compression JPEG
Fractal
H.261
II. Robot vision
RV
Computer graphics
Image processing
compressio
3

Figure 1—4


Figure 1—5

4
“Scene description”
Under constrained problems : Inverse problem
for example 2 equations
3 unknown variables are given
1. Image analysis :a detailed but undigested description
2. Scene analysis : more parsimonious, structured descriptions suitable for
decision marking


The apparent brightness of a surface depends on three factors
1. Microstructure
2. Distribution of incident light
3. Orientation of the surface w. r. t light source & observer


5

Figure 1—6
“Lambertian Surface”
One that appears equally bright from all viewing directions and reflects all incident light,
absorbing none

6
2. Image Formation & Image sensing
What determines where(Shape-Geometric) the image of some point will appear?
What determines how(Brightness, Color – Photonetiz) bright the image of some surface
will be?

I. Two Aspects of Image Formation
i. Perspective projection

Figure 2—1



7
ii. Orthographic Projection
2 2
2 2
0
' ' '
(magnification)
x y f
m
z
x y
δ δ
δ δ
+
= =
+

In case the depth range of a scene is small relative to the distance z
0

Figure 2—2

\
|
=
=
my y
mx x
'
'
if m=1

\
|
=
=
y y
x x
'
'

=> This orthographic projection can be modeled by rays parallel to the optical axis
(rather than one passing through the origin)

(a) (b)
Figure 2—3


I
p
E
δ
δ
= [watt/m]
Where δP is the power of the radiant energy falling on the
8

II. Brightness
1. Image Brightness : Irradiance
2
[ / ]
P
E watt m
I
δ
δ
=
2. Scene Brightness :
L=
2
2
[ / ]
P
L watt m
I W
δ
δ δ
= where δ
2
P is the power emitted by the infinitesimal surface patch
of area (δI) into an infinitesimal solid angle (δw)
2
, where is area, and is Distance
A
SolidAngle A D
D

Ex 1 Hemisphere
w δ
R
2
2
4 1
Solid angle = = 2
2
Whole sphere = 4
R
R
π
π
π

Figure 2—4
Ex 2 Small patch
9

Figure 2—5
Lens

Figure 2—6



cos

( / cos )2
Image Irradiance
P
E
I
I
Solid angle
f
δ
δ
δ α
α
=
=

0
2

=
'
cos

( / cos )
Scene Radiance
P
L
O w
Solid angle
z
δ
δ δ
δ θ
α
=

Transfer Function: ( )
L
f
E
δ
δ
= i
10
f(·) : BRDF(Bidirectional Reflectance Distribution Function) in Ch 10
Ex3) Sphere/Radius 2m

Figure 2—7
III. Lenses


Figure 2—8

i. Lens formula
1 1 1
' f z z
= + 2–1
' '
2
s z z
R D

= 2–2
Solve (2-1) & (2-2) to calculate R – blur circle radius
11
1 1 1
( )
2
DS
R
f s z
= − −
R = 0 at focused point P′
ii. Depth of field (DOF)
The range of distances over which objects are focused “sufficiently well” in the sense
that thediameter of the blur circle is less than the resolution of the imaging device.
The larger the lens aperture => the less the DOF

Figure 2—9

iii. Vignetting
In a simple lens, all the rays that enter the front surface of the lens end up being focused in the image

Figure 2—10
12
In a compound lens, some of the rays that pass through the first lens may be occluded
by portions of the second lens, and so on … Vignetting is a reduction in light-gathering
power with increasing inclination of light rays with respect to the optical axis




Figure 2—11
iv. Aberration
Points on the optical axis may be quite well focused, which those in a corner of the
image are smeared out
13
IV. Image sensing
i. Color sensing
a) Human eye

Figure 2—12
b) CCD sense
R, G, B filter
R(λ) : 1 ( at Red color)
14
0 (otherwise)
G(λ) : 1 (at Green color)
0 (otherwise)
B(λ) : 1 (at Blue color)
0 (otherwise)
→ UV filter filtered out UV ray
ii. Randomness and Noise
a) Random variable [R.V]
Consider an experiment H with sample description space Ω the elements on points of
Ω,δ, are the random out comes of H
If to every δ, we assign a real number X(δ) The real establish a correspondence rule
between δ and R the real time, such a rule subject to certain constraints is called a
random variable

15
Figure 2—13
R.V.
Probability density function (p.d.f) p(x)
Where p(x) ≥ 0 for all x, therefore: ( ) 1 p x dx

−∞
=


mean :
( ) xp x dx µ

−∞
=

: first moment
variance :
2 2 2
( ) ( ) ( ) : second moment x p x dx x p x dx δ µ
∞ ∞
−∞ −∞
= − =
∫ ∫


Cumulative probability distribution (CDF) P(x)
( ) ( )
x
P x p x dt
−∞
=


Two R.V.

S V R
X X X
. .
2 1
+ =

P(X
1
) & P(X
2
) what P(x) = ?
Solution

16
Figure 2—14

Given x
2
, x
1
: x-x
2

1 1 1 1 2
( ) ( ) P x x P x x x δ δ ⋅ = −

Now, x
2
can take on a range of values P
2
(x
2
).δ(x
2
)
To find the prob. that x lies between x and x+δx,
1 2 2 2 2
1 2 2 2 2
1 2
1 2
( ) ( ) ( )
( ) ( ) ( )
( ) ( )

P x x P x x xP x dx
P x P x x P x dx
P x t P t dt
P P
δ δ

−∞

−∞

−∞
⋅ = −
∴ = −
= −
= ∗


∫ ∫

For multiple
RV's
:

=
=
N
i
i
x
N
x
1
1

Mean value :
µ
µ
=
N
N

Variance :
2 2
2
N
N N
σ σ
=
Standard Deviation :
N
σ


b) Gaussian (Normal)
p.d.f :
2
1
2
1
( )
2
x
p x e
µ
σ
πσ
− | |

|
\ ¹
=

17


c) Poisson (m>0)
p.d.f : ( )
!
n
m
m
p k e
n

= ⋅
where n arrivals in a time interval T for some m
mean = m variance = σ
2


Ex Let X and Y be independent r.v.'s with ( ) ( )
x
x
p x e u x

= and
1
( ) [ ( 1) ( 1)]
2
y
p y u y u y = + − − and Let Z X Y ≡ + , what is the p.d.f of Z?

(a) (b)
Figure 2—15

18
( )
1
( ) ( ) * ( ) ( ) ( )
(a) 1 ; ( ) 0
1
(b) 1 1 ; ( )
2
(c) z 1 ; (
z x y x y
z
z
z y
z
z
p z p x p y p z y p y dy
z p z
z p z e dy
p z

−∞
− −

= = −
< − =
− ≤ < =



1
( )
1
1
)
2
z y
e dy
− −

=




Figure 2—16
19
3. Binary images : Geometric properties
I. Binary Images

Figure 3—1
1. Area of the object 1 sin cos 0
sin cos
x y
x y θ θ ρ
ρ ρ
θ θ
+ = ⇒ − + =


( , )
I
A b x y dxdy =
∫ ∫
; zero-th Moment
2. Position of the object : center of area
1
( , ) x xb x y dxdy
A
=
∫ ∫
: First moment
1
( , ) y b x y dxdy
A
=
∫ ∫

3. Orientation of the object
(ρ,θ)
2
( , )
I
E r b x y dxdy =
∫ ∫
: second moment

20
II. Simple Geometrical Properties

Figure 3—2
2 2 2
0 0
2 2
2 2 2 2
( ) ( )
( ( sin cos )) ( ( cos sin ))
( ) 2 ( sin cos ) 2 ( cos sin )
r x x y y
x s y s
x y x y s x y s
ρ θ θ ρ θ θ
ρ ρ θ θ θ θ
= − + −
= − − + + − +
= + + + − − + +

To find the shortest distance from (x, y) to (x
0
, y
0
) differentiate with respect to S and set
equal to zero
2
( )
0
d r
ds
=
θ θ sin cos y x S + =
2 2
0
2
(sin cos ) sin ( cos sin ) cos
sin sin sin cos
sin (sin cos )
x x x x y
x y
y
θ θ ρ θ θ θ θ
θ ρ θ θ θ
θ θ θ ρ
− = + + − +
= + −
= − +

) cos sin ( cos
0
ρ θ θ θ + − − = − y x y y
2
0
2
0
2
) ( ) ( y y x x r − + − = ∴

2
) cos sin ( ρ θ θ + − = y x

21

If, r=0 , ; 0 cos sin = + − ρ θ θ y x
; 0 cos sin ), , ( = + − ρ θ θ y x y x At

Finally,
2
( , ) E r b x y dxdy =
∫ ∫


2
( sin cos ) ( , )
I
x y b x y dxdy θ θ ρ = − +
∫ ∫

Differentiating w.r.t ρ and setting the result to zero lead to
2( sin cos ) ( , ) 0
I
dE
x y b x y dxdy
d
θ θ ρ
ρ
= − + =
∫ ∫

0 ) cos sin ( = + − ρ θ θ y x A
Let ' ' x x x x x x − = ⇒ − =
y y y − = '
θ θ ρ θ θ
ρ θ θ
cos ' sin ' cos sin
0 ) cos ) ' ( sin ) ' (
y x y x
y y x x
− = + −
= + − − −

and so

2
2 2
( ' sin ' cos ) ( , )
' sin ( , ) ( 2 ' ') cos sin
I
I I
x y b x y dxdy
x b x y dxdy x y
θ θ
θ θ θ

= + −
∫ ∫
∫ ∫ ∫ ∫

2 2
2 2
( , ) ' cos ( , )
sin sin cos cos
I
b x y dxdy y b x y dxdy
a b c
θ
θ θ θ θ
+
= − +
∫ ∫

where,
22

2
2
( ') ( , ) ' '
2 ( ' ') ( , ) ' '
( ') ( , ) ' '
I
I
I
a x b x y dx dy
b x y b x y dx dy
c y b x y dx dy
=
= ⋅
=
∫ ∫
∫ ∫
∫ ∫

Now
θ θ 2 sin
2
1
2 cos ) (
2
1
) (
2
1
b c a c a E − − − + =

Differentiating w.r.t θ and setting the result to zero, we have
c a
b

= θ
2
tan
unless 0 ; b a c = =

\
|
+ − =

=


θ θ ρ
θ
cos sin
tan
2
1
1
y x
c a
b


III. Projections

Figure 3—3
23
∫ ∫
= = dx y x b x V dy y x b x h ) , ( ) ( , ) , ( ) (
Area :
( , ) ( )
( )
A b x y dxdy h x dx
v y dy
= =
=
∫ ∫ ∫


Position :

∫ ∫ ∫
=
= =
dy y yv
A
y
dx x xh
A
dxdy y x xb
A
x
) (
1
) (
1
) , (
1

Orientation :
c a
b

=
−1
tan
2
1
θ
( , ) ρ θ θ θ ρ cos sin y x + − =
2 2
( , ) ( )
I
x b x y dxdy x h x dx =
∫ ∫ ∫

2 2
( , ) ( )
I
y b x y dxdy y v x dy =
∫ ∫ ∫

∫ ∫ ∫ ∫ ∫
− − = dy y h y x v x dt t d t dxdy y x xyb
I
) (
2
1
) (
2
1
) ( ) , (
2 2 2


Diagonal projection (θ=45°)

Figure 3—4
24
Suppose that θ is 45°
∫ ∫
+ − = = ds s t s t b dxdy y x b t d ) (
2
1
), (
2
1
( ) , ( ) (

Now consider that
2 2
2 2
1
( ) ( , ) ( )
2
1 1
( ) ( , )
2 2
I I
I
x y b x y dxdy t d t dt
x xy y b x y dxdy
+ =
= + +
∫ ∫ ∫ ∫
∫ ∫

so,
2 2 2
1 1
( , ) ( ) ( ) ( )
2 2
I
xyb x y dxdy t d t dt x h x dx y v y dy = − −
∫ ∫ ∫ ∫ ∫


IV. Discrete Binary Images
(reference of book figure 3-8)

area
∑∑
= =
=
N
i
M
j
ij
b A
1 1

position ) , ( j i
∑∑
∑∑
=
=
i j
ij
i j
ij
jb
A
j
ib
A
i
1
1


orientation ( , ) ρ θ
∑∑
∑∑
∑∑
i j
ij
i j
ij
i j
ij
ijb
b j
b i
2
2


25



V. Run-Length coding

Figure 3—5
Where r
ik
is the k-th run of the i-th line and the first run in each row is a run of zeros

2
.......... ,
6
1
2
1
4
1
2 2
i
i i
n
i
m
k
i
n
i
i k i
m
r r r r r A
i
+ + + = =
∑∑ ∑
= = =

position (center of area) : ) , ( j i
26
∑ ∑

= =
=
= ⋅ =
=
n
i
j
n
i
i
k
m
k
i i
jv
A
j h i
A
i
r h
i
1 1
2
2
1
1
..........
1
,

j
v


1. Find the first horizontal differences of the image data


Figure 3—6
2. The first difference of the vertical projection can be computed from the
projection of the first
27
horizontal differences
⇒ count the number of circles subtracted by the number of triangles.
3. The vertical projection v
j
can be found by summing the result from left to right




Orientation ( , ) ρ θ
j
j i
j t
t i j
ij
j
j
i
i
v j h i d t ijb
v j
h i
∑ ∑ ∑ ∑∑


− − =
2 2 2
2
2
2
1
2
1

Where, ) (
2
1
j i t + =
d
t
can be obtained in a way similar to that used to obtain the vertical projection
28
4. Binary Image : Topological properties
Jordan curve theorem
A simple closed curve separate the image into two simply connected regions

Figure 4—1
4 Connectedness – only edge-adjacent cells are considered neighbors.

Figure 4—2


29
Ex 1

Figure 4—3
4 objects
` 2 backgrounds
No closed curve ⇒ contradiction(by Jordan curve theorem)

8 connectedness – Corner-adjacent cells are considered neighbors, too.

Figure 4—4
6 connectedness

30
Figure 4—5


Figure 4—6

Figure 4—7
i. A sequential Labeling Algorithm
Recursive Labeling
1. Choose a point where b
ij
=1 and assign a label to this point and into neighbors
31

Figure 4—8

2. Next, label all the neighbors of these neighbors ⇒ one component will have
been labeled completely

3. Find new places to start a labeling operation whenever an unlabeled point is
found where
b
ij
=1
4. Try every cell in this scan

Sequential Labeling

32

Figure 4—9

1. if A is zero, there is nothing to do.

Figure 4—10
2. if A is one, if D has been labeled, simply copy that label and move on if not [if
one B orc is labeled, copy that label else, choose a new label for A]
(go to step 2)
33
II. Local counting and iterative modification
i. Local counting

Figure 4—11

Horizontal N
Vertical N Total Length = 2N
34

Figure 4—12
The overestimated rate : 1 : 2 2 : 2 = N N
Considering all slopes overestimated average ratio :
4
π

Overestimated average rate :
π
4


-Euler number-
1. No. of Bodies – No. of Holes
2. No. of upstream convexities – No. of upstream concavities

convexities(+1) concavities(-1)
Figure 4—13
35


Figure 4—14
Body hole
1. body - hole : E = 1- 2 = -1 2 - 0 = 2 1 - 0 = 1 1 - 1 = 0 1 - 0 = 1 1 - 0 = 1
2. convexity - concavity : 1- 2 = -1 2 – 0 = 2 1 – 0 = 1 1 – 1 = 0 1 – 0 = 1 1 - 0 = 1




ii. The additive set property

Figure 4—15
36

⇒This permits to split an image up into smaller pieces and obtain an overall answer by
combining the
results of operations performed on the pieces

Figure 4—16
1 2 2 2 1
) ( ) ( ) ( ) ( ) ( ) ( e E d E c E b E a E e d c b a E + + + + = ∪ ∪ ∪ ∪
[ ( ) ( ) ( ) ( )] E a b E b c E c d E d e − + + + ∩ ∩ ∩ ∩
1-0 = 1 2-0 =2 3-0 =3 2-0 =2
(1 2 2 2 1) (1 2 3 2) = 0 = + + + + − + + +
right (body = 1, hole 1) 1-1 =0
convexities =2
1(body) –
37
concavities =2

Ex1) hand

Figure 4—17
Body : 1 Hole : 0 1-0 =1
Convexities : 5 , Concavities : 4 5-4 =1
Euler number = 1


iii. Iterative modification
Instead of adding up the outputs of Local operators in the previous section, the new
binary image determined by the iterative method can be used as input to another cycle
of computation this process, called iterative modification is useful because it allows us
convexit
concavit
38
to incrementally changes an image that is difficult to process into one that might
succumb to the methods already discussed

Euler Differential E*
2. E* = - ( 1 – 1 = 0 ) + ( 1- 0 ) = 1
Part pattern Current pattern
2


3
39

4

5

* *
E E
new
− =
40
5. Regions & image segmentation

I. 5.1 Thresholding Methods

•Image segmentation : An image segmentation is the partition of an image into a set of
non-overlapping regions whose union is the entire image


Figure 5—1
Detection of discontinuities : points, lines, and edges

Figure 5—2
41

Figure 5—3

=
= + ⋅ ⋅ ⋅ ⋅ ⋅ + + =
q
i
i i q q
z w z w z w z w R
1
2 2 1 1


A mark used for detecting isolated points

Figure 5—4
i. Point detection
The detection of isolated points in an image is straightforward
If |R| > T when T is threshold, it’s an isolated point
ii. Line detection
The next level of complexity involves the detection of lines in an image. Consider
the four masks, as shown below


42

Figure 5—5
• If at a certain point in the image, |R
i
|>|R
j
| for all j≠i that point is said to be
more likely associated with a line in the direction of mask i, for example, if
|R
1
|>|R
j
| , for j=2,3,4 the line is more likely associated with a horizontal line.

iii. Edge detection
The idea underlying most edge detection technologies is the computation of a local
derivative operator

Figure 5—6
43
Gradient operator





= ∇
y
f
x
f
f

2
2
|
|
¹
|

\
|


+ |
¹
|

\
|


= ∇
y
f
x
f
f

Figure 5—7
Laplacian Operator
2
2
2
2
2
y
f
x
f
f


+


= ∇
) ( 4
8 6 4 2 5
2
z z z z z f + + + − = ∇ in discrete form

Figure 5—8

44
2
2
5 2 4 6 8
Cross Section of f
f = 4 (z ) z z z z

∇ − + + +


II. Thresholding
• Thresholding is one of the most important approaches to image segmentation

Figure 5—9
1 if ( , )
( , )
0 otherwise
f x y T
g x y
< ¦
=
´
¹


III. Region-oriented segmentation
i. Basic Formulation

Figure 5—10

45
Let R represent the entire image region and R
i
the partitioned region for n=1,2,………n
(a) R Ri
n
i
=
=

1
(entire image region)
(b)
i
R is a connected region, i=1,2,……n
(c)
i j
R R ∩ = φ for all i & j , i ≠j

\
|

=
=
φ
φ
φ
1 1
2 1
5 1
R R
R R
R R




(d) ) (
i
R P = True for i=1,2,…….n

Figure 5—11
ii. Region Growing by Pixel Aggregation
“Region Growing” is a procedure that groups pixels or sub-regions into larger regions.
The simplest of these approaches is pixel aggregation, which starts with a set of “seed”
points and from these grows regions by appending to each seed
Point those neighboring pixels that have similar properties



46
Ex 1

Figure 5—12
iii. Region splitting and Merging
Spitting
Let R represent the entire image region and select a predicate P as discussed in the
previous section For a square image, one approach for segmenting R is to subdivide it
successively into smaller and smaller quadrant regions so that, for any region R
i
,
P(R
i
) = TRUE that is, if P(R
i
or R) = FALSE, divide the image into quadrants

Figure 5—13
Merging
If only splitting were used, the final partition likely would contain adjacent regions
With identical properties, this draw back may be remedied by allowing merging as well
as splitting two adjacent regions R
j
&R
k
are merged only if P(R
j
∪R
k
)=TRUE

Figure 5—14
47
• The preceding discussion may be summarized by the following procedure in
which, at any step, we split into four disjointed quadrants any region R
i
where
P(R
i
)= FALSE

Figure 5—15
a) merge any adjacent regions R
j
& R
k
for which P(R
j
∪R
k
) =TRUE ;
b) stop when no further merging and splitting is possible
Ex 2

Figure 5—16
Merging FALSE R P
TRUE R R R P FALSE R P
FALSE R P FALSE R P
FALSE R P TRUE R P
TRUE R P FALSE R P
→ +
+ +
= +
= =
= =
) (
) ( ) (
) ( ) (
) ( ) (
) ( ) (
4
41 32 23 3
43 2
34 1
21
∪ ∪


Figure 5—17
48
6. Image Processing : Continuous images
I. Linear, Shift-Invariant Systems
Spatial domain → Point Spread Function (PSF)
Frequency domain → Modulation Transfer Function (MTF)

Figure 6—1
• Defocused image ( g ) is a processed version of the focused image ( f )
Double the brightness of ideal image and the defocused image. If the image system
moves, f & g moves the same amount.

Figure 6—2

Linearity : 1 2 1 2 ( , ) ( , ) ( , ) ( , ) f x y f x y g x y g x y α β α β + → +
Shift : ( , ) ( , ) f x a y b g x a y b − − → − −
49
II. Convolution
( , ) ( , ) ( , ) f x y h x y g x y → →
( , ) ( , ) ( , ) g x y f x y h d d
g f h
ε η ε η ε η
∞ ∞
−∞ −∞
= − −
→ = ⊗
∫ ∫

Linear
( ) ( ) 1( , ) 2 ( , ) ? f x y f h g x y α β + → → =
[ ] 1 2 ( , ) ( , ) ( , ) ( , ) g x y f x y f x y h d d α ε η β ε η ε η ε η
∞ ∞
−∞ −∞
= − − + − − ⋅
∫ ∫

1 2 ( , ) ( , ) g x y g x y α β = +
Shift invariant
( , ) ( , ) ( , ) f x a y b h x y g x y − − → →
( , ) ( , ) ( , ) g x y f x a y b h d d ε η ε η
∞ ∞
−∞ −∞
= − −
∫ ∫

Impulse response
( , ) ( , ) ( , ) f x y h x y h x y → →
( , ) ( , ) ( , ) g x y f x y h d d ε η ε η ε η
∞ ∞
−∞ −∞
= − −
∫ ∫

and ( , ) ( , ) g x y h x y =

Only possible if
Origin
( , )
0 otherwise
f x y
∞ ¦
=
´
¹

This function is called unit impulse ( , ) x y δ or Dirac delta function
( , ) 1 x y dxdy δ
∞ ∞
−∞ −∞
=
∫ ∫


50
Shifting Property : ( , ) ( , ) (0, 0) x y h x y dxdy h δ
∞ ∞
−∞ −∞
=
∫ ∫

and ( , ) ( , ) ( , ) x y h d d h x y δ ε η ε η ε η
∞ ∞
−∞ −∞
− − ⋅ =
∫ ∫

=> ( , ) h x y : Impulse Response of the system

Figure 6—3
( , ) ( , ) k x y ε η δ ε η − − : scaled delta function

( , ) ( , ) ( , ) f x y f x y d d ε η δ ε η ε η
∞ ∞
−∞ −∞
= ⋅ − −
∫ ∫

( , ) ( , ) ( , ) g x y f x y h x y d d ε η ε η
∞ ∞
−∞ −∞
= − − ⋅
∫ ∫


b a ⊗ = ( , ) ( , ) a x y b x y d d ε η ε η
∞ ∞
−∞ −∞
− − ⋅
∫ ∫
<= Let , x ε α − = y η β − =
= ( , ) ( , ) a b x y d d b a α β α β α β
∞ ∞
−∞ −∞
⋅ − − = ⊗
∫ ∫


Similarly, ( ) ( ) a b c a b c ⊗ ⊗ = ⊗ ⊗


51
Cascade
1 1 2 ( )
1 2
f h f h h
f h h
⊗ ⊗ ⊗
→ → →
=> 1 2 ( ) f h h ⊗ ⊗

1 2 f h h g → ⊗ →
III. MTF
An Eigen function of a system is a function that is reproduced with at most a change in
amplitude.
( )
jwt jwt
e A w e → →

2 ( )
( , )
i ux vy
f x y e
π +
→ =

{ } 2 ( ) ( )
( , ) ( , )
i u x v y
g x y e h d d
π ε η
ε η ε η
∞ ∞
− + −
−∞ −∞
= ⋅
∫ ∫

=
2 ( ) 2 ( )
( , )
i ux vy i u v
e e h d d
π π ε η
ε η ε η
∞ ∞
+ − +
−∞ −∞

∫ ∫

=
2 ( )
( , )
i ux vy
e A u v
π +


IV. Fourier Transform
Represent the signal as an infinite weighted sum of an infinite number of sinusoids (u: angular frequency)
( ) ( )
iux
F u f x e dx


−∞
=


Note:
cos sin 1
ik
e k i k i = + = −
52
Arbitrary function => Single Analytic Expression
Spatial Domain(x)=> Frequency Domain(u) (Frequency Spectrum F(u))
Inverse Fourier Transform (IFT)
( ) ( )
1
2
iux
f x F u e du
π

−∞
=


( )
2
1
( , ) ( , )
4
i ux vy
f x y F u v e dudv
π
∞ ∞
+ +
−∞ −∞
=
∫ ∫


V. The Fourier Transform of Convolution
Let g f h = ⊗
Now,

( ) ( )
( ) ( )
( ) ( )
( )
( ) ( )
( ) ( )
2
2
2 2
2 2 '


' '

i ux
i ux
i u x i u
i u i ux
G u g x e dx
f h x e d dx
f e d h x e dx
f e d h x e dx
F u H u
π
π
π τ π τ
π τ π
τ τ τ
τ τ τ
τ τ
∞ ∞

−∞ −∞
∞ ∞

−∞ −∞
∞ ∞
− − −
−∞ −∞
∞ ∞
− −
−∞ −∞
=
= −

= −

=

=
∫ ∫
∫ ∫
∫ ∫
∫ ∫

Convolution in spatial domain Multiplication in frequency domain
Spatial Domain (x) Frequency Domain(u)
g f h G FH
g fh G F H
= ⊗ ↔ =
= ↔ = ⊗


53
So, we can find g(x) by Fourier transform

Figure 6—4

Spatial Domain (x) Frequency Domain (u)
Linearity
( ) ( )
1 2
c f x c g x + ( ) ( )
1 2
c F u c G u +
Scaling
( ) f ax 1 u
F
a a
| |
|
\ ¹

Shifting
( )
0
f x x − ( )
0
2 i ux
e F u
π −

Symmetry
( ) F x ( ) f u −
Conjugation
( ) f x

( ) F u


Convolution
( ) ( ) f x g x ⊗ ( ) ( ) F u G u
Differentiation
( )
n
n
d f x
dx

( ) ( ) 2
n
i u F u π



54
VI. Generalized Functions and Unit Impulses
• The integral of the one-dimensional unit impulse is the unit step function,
( ) ( )
x
t dt u x δ
−∞
=

, where ( ) u x = 1, for 0; x ≻
1
2
= , for 0; x =
0 = , for 0 x ≺ .
What is the Fourier transform of the unit impulse?
We have
( )
( , ) 1
i ux vy
x y e dxdy δ
∞ ∞
− +
−∞ −∞
=
∫ ∫
, as can be seen by substituting 0 x = and 0 y =
into
( ) i ux vy
e
− +
, using the sifting property of the unit impulse. Alternatively, we can use
( )
0
lim ( , )
i ux vy
x y e dxdy ε
ε
δ
∞ ∞
− +

−∞ −∞
∫ ∫
, or
0
1 1
lim
2 2
iux ivy
e dx e dy
ε ε
ε
ε ε
ε ε
− −

− −
∫ ∫
,
That is
0
sin sin
lim 1
u v
u v
ε
ε ε
ε ε

= .
VII. Convergence Factors and the Unit Impulse
We want a smoothed function of f(x)
( ) ( ) ( ) g x f x h x = ∗
Let us use a Gaussian kernel

Figure 6—5
55
( )
2
2
1 1
exp
2 2
x
h x
σ πσ

= −



Then
( ) ( )
2
2
1
exp 2
2
H u u π σ

= −



( ) ( ) ( ) G u F u H u =
H(u) attenuates high frequencies in F(u) (Low-pass Filter)!
VIII. Partial Derivatives
What is the F.T. of
( , ) x y f
x


and
f
x


?
2 ( ) i ux vy
f
e dxdy
x
π
∞ ∞
− +
−∞ −∞


∫ ∫
=
2 2 ux vy
f
e dx e dy
x
π π
∞ ∞
− −
−∞ −∞






∫ ∫

2 ux
f
e dx
x
π


−∞



=
2 2
( , ) ( 2 ) ( , )
iux iux
f x y e iu f x y e dx
π π
π


− −
−∞
−∞
− − ⋅


=
2 2
( , ) (2 ) ( , )
iux iux
f x y e iu f x y e dx
π π
π


− −
−∞
−∞
+ ⋅



We can’t proceed unless, lim ( , ) 0
x
f x y
→±∞
=
In that case,
2 ( )
2 ( , )
i ux vy
iu f x y e dxdy
π
π
∞ ∞
− +
−∞ −∞
∫ ∫
=(2 ) ( , ) iu F u v π ⋅

( , )
( , )
2 ( , )
2 ( , )
x y
x y
f
F iu F u v
x
f
F iv F u v
x
π
π
∂ ¦ ¹
∴ = ⋅
´ `

¹ )
∂ ¦ ¹
= ⋅
´ `

¹ )

56
IX. Rotational Symmetry and Isotropic operators
• Rotationally symmetric operators are particularly attractive because the treat
image features in the same way, no matter what their orientation is.
• Circular symmetry

Figure 6—6
Let us introduce polar coordinates in both spatial and frequency domain

Figure 6—7
cos x r φ = and sin y r φ = cos( ) ux vy r ρ φ α + = ⋅ −
cos u ρ α = ⋅ and sin v ρ α =

57
{ }
2 ( )
( , ) ( , )
i ux vy
F f x y f x y e dxdy
π
∞ ∞
− +
−∞ −∞
=
∫ ∫

2 cos( )
0 0
( )
i r
r f r e rdrd
π φ α
φ
∞ ∞
− ⋅ −
= ⋅
∫ ∫

Zeroth order Vessel function
(Hankel Transform) => 0 ( ) 2 ( ) (2 ) r F f r J r rdr ρ ρ π π ρ

−∞
=


0
0
( ) 2 ( ) (2 ) fr r f J r d ρ π ρ π ρ ρ ρ

=


where
2
cos( )
0
0
1
( )
2
i x w
J x e d
π
θ
θ
π
− ⋅ −
=



X. Blurring

2 2
2
1
2
2
1
( , )
2
x y
h x y e
σ
πσ
+

=
. . F T ⇓
2 2
2
1
2 ( ) 2
2
1
( , )
2
x y
i ux vy
H u v e e dxdy
π
σ
πσ
+ ∞ ∞

− +
−∞ −∞
= ⋅
∫ ∫

2 2
1 1
2 2 2 2
1 1
2 2
x y
iux ivy
e e dx e e dy
π π σ σ
πσ πσ
| | | | ∞ ∞
− −
| |
− −
\ ¹ \ ¹
−∞ −∞
= ⋅ ⋅
∫ ∫


2
1
2 2
1
2
x
iux
e e dx
π σ
πσ
| | ∞

|

\ ¹
−∞


2 2
1 1
2 2
cos(2 ) sin(2 )
x x
e ux dx i e ux dx
σ σ
π π
| | | | ∞ ∞
− −
| |
\ ¹ \ ¹
−∞ −∞
= − ⋅ ⋅
∫ ∫


58
The second integral is odd so, integral over symmetric region is zero.

And,
2
2 2
1
2
2
1
cos(2 ) ,
2
x u
a
e ux dx e a
a
π
σ
π
π
σ
| | ∞
− − |
\ ¹
−∞
= ⋅ =


2
2 2 2
1
1
(2 )
2
2
cos(2 ) 2
x
u
e ux dx e
π σ
σ
π π σ
| | ∞

− ⋅ ⋅ |
\ ¹
−∞
= ⋅ ⋅



2 2 2 2 2 2
1 1
(2 ) (2 )
2 2
1 1
( , ) 2 2
2 2
u v
H u v e e
π σ π σ
πσ πσ
π σ π σ
− ⋅ ⋅ − ⋅ ⋅
= ⋅ ⋅ ⋅ ⋅ ⋅
⋅ ⋅


2 2 2 2
1
(2 ) ( )
2
u v
e
π σ − ⋅ + ⋅
=
Which is also rotationally symmetric. { } F G a u s s i a n G a u s s i a n =

Figure 6—8: Low Pass Filtering
XI. Restoration and Enhancement
( , ) f x y → ( , ) '( , ) ( , ) h x y h x y f x y → →
The cascade of the two systems is the identity system.
1
'( , ) min ,
( , )
H u v A
H u v
| |
=
|
|
\ ¹
,
Where, A is the maximum gain. Or more elegantly, we can use something like
59
2 2
( , )
'( , )
( , )
H u v
H u v
H u v B
=
+
,
where
1
(2 ) B
is the maximum gain, if ( , ) H u v is real.

XII. Correlation

Figure 6—9
Cross correlation of ( , ) a x y and ( , ) b x y is defined as
( , ) ( , )* ( , )
( , ) ( , )
ab x y a x y b x y
a x y b d d
φ
ε η ε η ε η
∞ ∞
−∞ −∞
=
= − − ⋅
∫ ∫

If ( , ) a x y = ( , ) b x y , the result is called autocorrelation
( , ) .( ( , ) ( , )) ab ab ab x y Symmetric x y x y φ φ φ → − − =
(0, 0) ( , ) aa aa x y φ φ ≥ for all ( , ) x y
2
(0, 0) ( , ) ( , ) ( , ) aa a a d d a d d φ ε η ε η ε η ε η ε η
∞ ∞ ∞ ∞
−∞ −∞ −∞ −∞
= ⋅ =
∫ ∫ ∫ ∫



60
XIII. 6.13 Optimal Filtering

Figure 6—10
E =
{ }
2
( , ) ( , ) o x y d x y dxdy
∞ ∞
−∞ −∞

∫ ∫

( , ) ( , ) ( , ) i x y b x y n x y = +
( , ) ( , ) ( , ) o x y i x y h x y = ⊗

So,
2 2
( , ) 2 ( , ) ( , ) ( , ) E o x y o x y d x y d x y dxdy
∞ ∞
−∞ −∞
= − ⋅ +
∫ ∫

{ }
2
2
( , ) ( , ) ( , ) o x y i x y h x y = ⊗
( , ) ( , ) ( , ) ( , ) i x y h d d i x y h d d ε η ε η ε η α β α β α β
∞ ∞ ∞ ∞
−∞ −∞ −∞ −∞
= − − ⋅ × − − ⋅
∫ ∫ ∫ ∫


Therefore,
2
( , ) ( , ) ( , ) ( , ) o dxdy h h d d d d i x y i x y dxdy ε η α β ε η α β ε η α β
∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞
−∞ −∞ −∞ −∞ −∞ −∞ −∞ −∞
= ⋅ − − ⋅ − −
∫ ∫ ∫ ∫ ∫ ∫ ∫ ∫


x A x A
y B y B
α α
β β
− = ⇒ = +
− = ⇒ = +

61
⇒ ( ( ), ( ) ( , ) i A B i A B dAdB ε α η β
∞ ∞
−∞ −∞
= − − − − ⋅
∫ ∫

( , ) ii φ ε α η β = − − (Autocorrelation)
62
7. Image Processing – Discrete Image
I. Sampling Theorem
To convert a continuous distribution of brightness to a digital image, it necessary to
extract the brightness at each point arranged at a period. Thus is called sampling.

Assume an image as a one-dimensional function for simplicity.
Let the brightness at position x i.e. pixel value of pixel at x, be fix.

Figure 7—1

Figure 7—2
63
Suppose a function that is composed by infinite numbers of Dirac’s delta functions
assigned at an interval T. This is called comb function defined as follows.
A simple digital image from ( ) f x , denoted ( ) T f x , is expressed as ( ) f x multiplied by
the comb function ( ) T comb x , i.e.

( ) ( ) ( ) T T f x f x comb x = ⋅
{ } { } { } ( ) ( ) ( ) ( ) ( ) ( ) T T FT f x v FT f x v FT comb x v = ⊗
and, { }
1
1
( ) ( ) ( ) T
T
FT comb x v comb V
T
=
{ } { }
1
( ) ( ) ( ) ( ) T FT f x v FT f x v
T
= ⊗
1
( )
T
comb V

That’s a convolution of comb function?
We start from “convolution with delta function”
( ) ( ) ( ) ( ) ( ) ( ) ( ) f t t f y t y dy f t t t dy f t δ δ δ
∞ ∞
−∞ −∞
∗ = ⋅ − = ⋅ − =
∫ ∫

i.e., convolution of a function and delta function is equal to original function itself.
Since a comb function is a sequence of the delta function arranged at constant period,
convolution of a function and comb function is an arrangement of duplication of the
original function at a constant period.

64
The F.T. of ( ) T f x , which is the sampled version of ( ) f x at period T is an infinite
sequence of duplicating of { } ( ) T FT f x which is the F.T. of the original function ( ) f x
at period
1
T
.

Figure 7—3
If the period of comb function in the frequency domain is sufficiently large, adjacent
{ } ( ) FT f x ’s do not overlap. In this case, the F.T. of the original function { } ( ) FT f x ,
can be separated and extracted i.e. no information of the brightness distribution of the
original image is lost by sampling. However, if the internal of the comb function is the
frequency domain is small, adjacent { } ( ) FT f x (v)’s overlap. In this case, the original
{ } ( ) FT f x can’t be separated and a faulty function will be extracted. This effect is
called aliasing.
65
{ } { } { } ( ) ( ) ( ) ( ) ( ) ( ) T T FT f x v FT f x v FT comb x v ⇒ = ⊗

Figure 7—4
Since the support of { } ( ) FT f x is in the range between ~ c c V V − , the period has to be at
least 2 c V to avoid overlapping of { } ( ) FT f x ’s since, T is the sampling period.
1
T

denote the number of sampling per unit length, i.e. sampling rate consequently, the
original brightness distribution can be reconstructed by a sampled digital image of the
sampling rate is more than twice the maximum frequency contained in the original
distribution. This theorem is called sampling theorem.



66
Discrete Fourier Transform (DFT)

1 1
2 ( )
2 ( )
0 0
0 0
1
( , ) ( , ) ( , )
m n
M N
i u v
i ux vy
M N
m n
F u v f x y e dxdy f m n e
MN
π
π
∞ ∞
− −
− +
− +
= =
= = ⋅
∑∑
∫ ∫

(u=0~M-1, v=0~N-1)
Inv. D.F.T.
1 1
2 ( )
0 0
( , )
m n
M N
i u v
M N
m n
F u v e
π
− −
+ +
= =
= ⋅
∑∑
(m=0~M-1, n=0~N-1)

• f(m,n) ≜ Input Signal Amplitude (real or complex) at sample (m, n)
T ≜ Sampling interval
s f ≜ Sampling rate
1
( )
T
= , samples/second
M,N = Numbers of Spatial samples = Number of frequency samples (integer)

• Magnitude
1
2 2
2
( , ) ( , ) ( , ) F u v R u v I u v = +


Phase Angle
1
( , )
( , ) tan
( , )
I u v
u v
R u v
φ

=
Power Spectrum
2
2 2
( , ) ( , ) ( , ) ( , ) P u v F u v R u v I u v = = +


1 1
0 0
1
(0, 0) ( , )
M N
m n
F f m n
MN
− −
= =
=
∑∑
⇒ Average of ( , ) f m n or D.C. component.
If ( , ) f x y is real, its F.T. is conjugate complex
( , ) ( , )
( , ) ( , )
F u v F u v
F u v F u v


= − −
= − −
⇒ Spectrum of D.F.T. is symmetric.

67
• Relationship between samples in the spatial and frequency domains.

1
u
M x
=
⋅ ∆

1
v
N y
=
⋅ ∆


• Translation
0 0
0 0
2 ( )
( , ) ( , )
u v
i x y
M N
f x y e F u u v v
π − ⋅ ⋅ + ⋅
⋅ ⇔ − −
0 0
0 0
2 ( )
( , ) ( , )
ux vy
i
M N
f x x y y F u v e
π − ⋅ +
− − ⇔

Case : when 0 0 ,
2 2
M N
u v = =
{ }
2 ( )
( )
2 2
cos ( ) ( 1)
x y
i
i x y x y
e e x y
π
π
π
⋅ +
+ +
= = + = −
( , )( 1) ( , )
2 2
x y
M N
f x y F u v
+
⇒ − ⇔ − −

And similarly,
( , ) ( , )( 1)
2 2
u v
M N
f x y F u v
+
− − ⇒ −

Rotation
cos
sin
x r
y r
θ
θ
= ⋅
= ⋅

cos
sin
u w
v w
φ
φ
= ⋅
= ⋅


Then, ( , ) f x y and ( , ) F u v become ( , ) f r θ and ( , ) F w φ
68
0 0 ( , ) ( , ) f r F w θ θ φ θ + ⇔ +

⇒ Rotating ( , ) f x y by an angle 0 θ rotates ( , ) F u v by the same angle, and vice versa.

Periodicity ( , ) ( , ) ( , ) ( , ) F u v F u M v F u v N F u M v N = + = + = + +
The inverse transform is also periodic.
( , ) ( , ) ( , ) ( , ) f x y f x M y f x y N f x M y N = + = + = + +

Separability
1 1
2 2
0 0
1 1
( , )
y x
M M
j v j u
N M
x y
F u v e e
M N
π π
− −
− −
= =
= ⋅
∑ ∑

1
2
0
1
( , )
x
M
j u
M
x
F u v e
M
π


=
=

, (where
1
2
0
1
( , )
y
M
j v
N
y
F u v e
N
π


=
=

)
( , ) ( , ) ( , ) f x y F x v F u v → →


• Convolution
1 1
0 0
1
( , ) ( , ) ( , ) ( , )
M N
m n
f x y h x y f m n h x m y n
MN
− −
= =
⊗ = ⋅ − −
∑∑


f h F H
f h F H
⊗ ⇒ ⋅
⋅ ⇒ ⊗
Pr ( . .) ove H W →
1-D
Row transform
1-D
column transform
69
8. Edge & Edge detection
I. Edges in Images
• An Edge is a boundary between two figures with relatively distinct grey-level
properties.
• Edges are curves in the image where rapid changes occur in brightness or in the
spatial derivatives of brightness.
• Change of brightness can be occurred at
• Change of surface orientation
• Different surfaces
• One object occludes another
• A boundary between light and shadow falling on a single surface
II. Differential Operators

Figure 8—1
70

Figure 8—2
sin cos 0 x y θ θ ρ − + =
( ) ( )
1 2 1
E , B B -B u( sin cos ) x y x y θ θ ρ = + − +
If t > 0 B
1
t < 0 B
2

t = 0 ½(B
1
+ B
2
)
( ) ( )
( ) 1
t
u t x dx
x dx
δ
δ
−∞

−∞
=
=




Figure 8—3
) cos sin ( ) ( cos
) cos sin ( ) ( sin
2 1
2 1
ρ θ θ δ θ
ρ θ θ δ θ
+ − − − =


+ − − =


y x B B
y
E
y x B B
x
E


71
squared gradient
2
2 1
2
2
)) cos sin ( ) (( ρ θ θ δ + − − =
|
|
¹
|

\
|


+ |
¹
|

\
|


y x B B
y
E
x
E

Laplacian of E(x,y)
2 2
2
2 2
2
2
1 2 2
2
2
1 2 2
2 '
1 2
sin ( ) '( sin cos )
cos ( ) '( sin cos )
( ) ( sin cos )
xx
yy
E E
E
x y
E
E B B x y
x
E
E B B a y
y
E B B x y
θ δ θ θ ρ
θ δ θ θ ρ
δ θ θ ρ
∂ ∂
∇ = +
∂ ∂

= = − − +


= = − − +

∇ = − − +


III. Discrete Approximations
• Discrete form of Laplacian
E
x
= E( i+1 , j ) – E( i , j )
E
y
= E( i , j+1 ) – E( i , j )

E
xx
= E( i – 1 , j ) – 2E( i , j ) + E( i + 1 , j )
E
xx
+ E
yy
= (E
i-1 j
+ E
ij-1
+ E
i-1 j
+ E
i j+1
) – 4E
ij



72
IV. 8.4 Local operators and noise
bb
nn
H
φ
φ
ρ
+
⋅ − =
1
1
2 '
-------------ⓐ
ρ
2
= u
2
+ v
2


suppose that
2
2
2
& N
S
nn bb
= = φ
ρ
φ
Then
2 2 2
2 2
'
N S
S
H
ρ
ρ
+
− = - ⓐ
at low frequency -ρ
2
: Laplacian operation
at high frequency -
2
2
N
S
<< ∞ small gain Noise can be reduced !
Derivation of ⓐ

Figure 8—4
''
' '' 2

1 1
( )
1
( )
i bb nb
ii bb bn nb nn
bb
bb nn
nn
bb
H
SNR
H H
α
ϕ ϕ ϕ
ϕ ϕ ϕ ϕ ϕ
ϕ
ϕ ϕ
ϕ
ϕ
+
= =
+ + +
=
+
= =
+
= ⋅ ℑ ∇

73
For example, if the optimal filter is a gaussian

Figure 8—5
2
2 2
2
2
2
1
) , (
σ
πσ
y x
e y x h
+

=
) (
2
1
2 2 2
) 0 , (
v u
e u H
+ −
=
σ



74
9. Lightness & color

Figure 9—1
b´(x,y) = e´(x,y) - r´(x,y)
• Note that r´(x,y) is constant within a patch and sharp discontinuities at edges
between patches and e´(x,y) varies smoothly

Figure 9—2
Task : separate r´(x,y) & e´(x,y)
• Take the logarithm of image brightness b´(x,y)
log b´(x,y) = log r´(x,y) +log e´(x,y)
b(x,y) = r(x,y) +e(x,y)

2
b(x,y) =∇
2
r(x,y) +∇
2
e(x,y)

Figure 9—3
75
• cut slow slope by a threshold

Figure 9—4
T(∇
2
b(x,y) = ∇
2
r(x,y)
|∇
2
b(x,y)| > T then t(x,y) = ∇
2
b(x,y) else t(x,y) = 0
• recover r(x,y)

Figure 9—5
∫ ∫
∇ =
= ∇
dxdy y x r y x l
y x t y x l
) , ( ) , ( (
) , ( ) , (
2
2

Fourier Transform :
) (
1
) (
) ( ) (
)] , ( [ )] , ( [
2
2
2
e T L
T L
y x t y x l
ρ
ρ
ρ ρ ρ
− = ⇒
= − ⇒
ℑ = ∇ ℑ

2
1
) ( ..
) ( ) (
ρ
ρ ρ
− =
⋅ =
p G where
T G

ρ
π
+ = ) log(
2
1
) ( r r g
IFT : I(x,y) = g(r) * t(x,y)
76

Figure 9—6
I(x,y) = r(x,y) = reflectance
e(x,y) = b(x,y) – I(x,y)


Figure 9—7



77
. . .
) , ( ) , ( ) , (
) , ( ) , ( ) , (
5 5 5
4 4 4
3 3 3
2 2 2
1 1 1
e r b
e r b
e r b
y x e y x r y x b
y x e y x r y x b
⋅ =
⋅ =
⋅ =
=
=

) (
74
37
:
2 1
2 2
1 1
2
1
1
e e
e r
e r
b
b
R ≅ = = - equation 9.1

52
26
2
1
=
r
r
- equation 9.2
3
2
3
2
5
2
5
2
2
95
38
112
28
:
r
r
b
b
r
r
b
b
R
= =
= =  


Least square error 11 variables 16 equations
0 ≤ reflectance ≤ 1
16
equations
78
10. Reflectance Map : photometric stereo
(using one camera)

I. Radiometry
irradiance
A
p
E
δ
δ
= geometric stereo (using one camera)
radiance
W A
p
L
δ δ
δ
2
=

solid angle = Area of a spherical surface patch / (radius of the sphere)
2


Figure 10—1
2
2
1
4
2
Solid angle 2
R
R
π
π

= =

Figure 10—2
79

Figure 10—3

2 2
1
) 1 , , (
ˆ
q p
q p
n
+ +
− −
=

II. Image Formation

Figure 10—4
80
solid angle =
2
) cos / (
cos
α
θ δ
z
O

solid angle =
2
) cos / (
cos
α
α
S
SI


2
cos
cos
|
¹
|

\
|
=
S
z
I
O
θ
α
δ
δ
--------- equation 10.2.1
The solid angle subtended by the lens

α
α π
α
α
α
π
3
2
2
2
cos
4 ) cos / (
cos
2
|
¹
|

\
|
=
|
¹
|

\
|
= Ω
z z


Thus the power of the light originating on the patch and passing through the lens is
θ δ δ cos Ω ⋅ = O L P
where L is the radiance of the surface in the direction toward the Lens

θ α
α π
δ
δ
δ
δ
θ α
α π
δ δ
cos cos
4
cos cos
4
3
2
3
2
|
¹
|

\
|
= =
|
¹
|

\
|
=
z I
O
L
I
P
E
z
O L p

where E is the irradiance of the image at the patch under consideration

substituting for I O δ δ , we finally obtain
81

α
α π
α
α π
4
2
4
2
cos
4
cos
4
|
¹
|

\
|
⋅ =
∝ ∴
|
¹
|

\
|
=
s
L E
L E
s
L E


III. Bidirectional reflectance distribution function (BRDF)

Figure 10—5

Figure 10—6
82
) , (
i i
E φ θ δ ; irradiance
The amount of light falling on the surface from the direction ) , (
i i
φ θ
) , (
e e L
φ θ δ : radiance

The brightness of the surface as seen from the direction
) , (
e e
φ θ
BRDF :
1
cos sin ) , ; , (
) , (
) , (
) , ; , (
2
0
=
=
∫ ∫

π
π
π
φ θ θ θ φ θ φ θ
φ θ δ
φ θ δ
φ θ φ θ
e e e e e e i i
i i
e e
e e i i
d d
E
L
f


IV. Extended Light sources

Figure 10—7
solid angle w δ
2
) tan (
.
ce dis
area The
= The area =
i i
δφ θδθ sin
83
(proof :
2
0
sin 2
i i i
sw
π
π
π
θ δθ δϕ π

= =
∫ ∫ ∫ ∫

) , (
i i
E φ θ : Radiance per unit solid angle coming from the direction ) , (
i i
φ θ
The radiance from the patch under consideration

i i i i i i i
E E δφ δθ θ φ θ δω φ θ sin ) , ( ) , ( =
Total irradiance of the surface is
2
0
0
( , ) sin cos
i i i i i i
E E d d
π
π
π
θ ϕ θ θ θ ϕ

=
∫ ∫

where
i
θ cos accounts for the foreshortening of the surface as seen from the direction
) , (
i i
φ θ
The radiance of the surface,
∫ ∫
= ) , ( ) , , ( ) , (
i i e e i i e e
E j f L φ θ φ θ φ θ φ θ

i i i i
d d φ θ θ θ cos sin

V. Surface Reflectance properties
• Ideal Lambertian surface
One that appears equally bright from all viewing directions and reflects all
incident light, absorbing none
84
BRDF ) , , (
e e i i
j f φ θ φ θ constant

2
0
2
0
( , , , ) sin cos 1
2 sin cos 1
1
1
i i
e e e e e e
e e e
f j d d
f d
f
f
π
π
π
π
θ φ θ ϕ θ θ θ ϕ
π θ θ θ
π
π

=
=
=
∴ =
∫ ∫



sine BRDF is constant for a Lambertian surface
)
1
(
1
cos sin ) , (
π π
φ θ θ θ φ θ
= ⋅ = ⋅ =
⋅ =
∫ ∫
f E E f
d d E f L
o o
i i i i i i


where the irradiance is E
o

• Ideal specular Reflector

Figure 10—8

85
2
0
( , ; , ) sin cos 1
sin cos 1
1
sin cos
i i e e e e e e
i i
i i
f d d
k
k
π
π
π
θ ϕ θ ϕ θ θ θ ϕ
θ θ
θ θ

=
⇒ =
⇒ =
∫ ∫


In this case,
i i
i e i e e e i i
j f
θ θ
π φ θ δ θ θ δ φ θ φ θ
cos sin
) ( ) ( ) , , ( − − − =


Determine the radiance of a specular reflecting surface under an extended source where
the irradiance is E
o

2
0
( ) ( )
( , )
sin cos
( , ) sin cos
( , )
( , )
e i e i
e e
i i
i i i i i i
e e
i i
L
E d d
E
E
π
π
π
δ θ θ δ ϕ ϕ π
θ ϕ
θ θ
θ ϕ θ θ θ ϕ
θ ϕ π
θ ϕ

− − −
=
⋅ ⋅
= −
=
∫ ∫


VI. Surface Brightness

Figure 10—9
86
i
E L θ
π
cos
1
=

•Light source : a “sky” of uniform radiance E
2
0
2
0
2
0
1
sin cos
sin cos
cos 2
[1 1]
2 2
i i i i
i i i i
i
L E d d
E
d d
E
E E
π
π
π
π
π
π
π
θ θ θ ϕ
π
θ θ θ ϕ
π
θ


= ⋅
=

= = + =


∫ ∫
∫ ∫

The radiance of the patch ≡The radiance of the source

VII. Surface orientation

Figure 10—10

87
T
y y y
T
x x x
y x
q r
p r
q p r r n
) , , 0 (
) , 0 , (
) 1 , , (
δ δ
δ δ
=
=
− − = × =


The unit surface normal nˆ
2 2
1
) 1 , , (
ˆ
q p
q p
n
n
n
T
+ +
− −
= =

Figure 10—11

T
v ) 1 , 0 , 0 ( ˆ ≡
assuming α is very small

2 2 2 2
2 2
1
1
) 1 , 0 , 0 (
1
) 1 , , (
ˆ ˆ cos
1
) 1 , , (
ˆ
q p q p
q p
v n
q p
q p
n
T
+ +
= ⋅
+ +
− −
= =
+ +
− −
=
θ




88
VIII. The reflectance Map

Figure 10—12
) , ( ˆ
) , ( ˆ
2 2
1 1
q p R s
q p R s





i
E L θ
π
cos
1
= for 0 ≥
i
θ

2 2 2 2 2 2 2 2
1 1
1
1
) 1 , , (
1
) 1 , , (
ˆ ˆ cos
s x
s s
s s
T
s s
i
q p q p
q q p p
q p
q p
q p
q p
s n
+ + + +
+ +
=
+ +
− −

+ +
− −
= ⋅ = θ
= R(p,q) ; Reflectance Map
c
q p q p
q q p p
q p R
s s
s s
=
+ + + +
+ +
=
2 2 2 2
1 1
1
) , ( - equation 10.9.1

Figure 10—13
89
2 2 2
1
2
1
2 2
2
2 2 2
1
2
1
1 1
1
1 1
1
) , (
1 1
1
) , (
s s
s s
s s
s s
q p q p
q q p p
q p R
q p q p
q q p p
q p R
+ + + +
+ +
=
+ + + +
+ +
=


IX. Shading in image

Consider a smoothly curved object the image of such an object will have spatial
variations in brightness due to the fact that surface patches with different orientations
appear with different brightness the variation of brightness is called shading

) , ( cos
cos
4
4
2
q p R L E
f
d
L E
i

|
|
¹
|

\
|
⋅ =
θ α α
α
π


after normalizing E(x,y) = R(p,q)
ρ : reflectance factor where 1 0 ≤ ≤ ρ

n s y x E where
n s p y x E
ˆ ˆ ) , ( , 1
) ˆ ˆ ( ) , (
⋅ = =
⋅ ⋅ =
ρ


Ex1) Consider a sphere with a Lambertian surface illuminated by a point source at
essentially the same place as the viewer in the case of cylinder
90

Figure 10—14

In this case,
2 2
1
) 1 , , (
ˆ
) 1 , 0 , 0 ( ˆ
) 1 , 0 , 0 ( ˆ
q p
q p
n
v
s
+ +
− −
=
=
=

From equation 10.9.1 - s n q p R ˆ ˆ ) , ( ⋅ =
2 2
2 2 2 2
1
1
1 1
1
q p
q P q p
q q p p
s s
s s
+ +
=
+ + + +
+ +
=

0
2
0
0
2
1
2 2 2
) (
) 2 ( )) ( (
2
1
z z
z z y
d
d
q
z z
x
x y x r
d
d
p
y
z
x
z


− = =

− = − + − = =


Finally, E(x,y) = R(p,q)
91
2
2 2
2
0
2 2
2 2
1
) (
1
1
1
1
r
y x
z z
y x
q p
+
− =

+
+
=
+ +
=

( x , y ) = ( 0 , 0 )
E( x, y) maximum = 0
x
2
+y
2
=r
2

E(x,y) minimum = 0

X. Photometric stereo

Figure 10—15

92
2
2
2
2
2 2
2 2
2 2
2
1
2
1
2 2
1 1
1 1
1 1
1
) , (
1 1
1
) , (
s s
s s
s s
s s
q P q p
q q p p
q p R E
q P q p
q q p p
q p R E
+ + + +
+ +
= =
+ + + +
+ +
= =


Ex2) R(p, q) : linear and independent
2
2 2
2
1
1 1
1
1
) , (
1
) , (
r
q q p p
q p R
r
q q p p
q p R
+ +
=
+ +
=

then
2 1 2 1
1 2
2
2 2 1
2
1
) 1 ( ) 1 (
p q q p
q r E q r E
p

− − −
=

2 1 2 1
2 1
2
1 1 1
2
2
) 1 ( ) 1 (
p q q p
p r E p r E
q

− − −
=

Ex3) R(p,q) : nonlinear

Figure 10—16
93
XI. Albedo : reflectance factor :
) ˆ ˆ ( n s E
i i
⋅ = ρ
where
2 2
1
) 1 , , (
ˆ
i i
T
i i
i
q p
q p
s
+ +
− −
= for i=1, 2, 3, …………

2 2
1
) 1 , , (
ˆ
q p
q p
n
T
+ +
− −
=

Figure 10—17

Unknown variables p, q, ρ
At least 3 equations



94
11. Reflectance Map : shape from shading
I. Recovering Shape from Shading

Figure 11—1
i. Picture
One more information => strong constraint smoothness neighboring patches of the
surface => similar orientation assumption of smoothness : strong constraint

ii. Linear reflectance map

Figure 11—2
95
2 2
0
2 2
0 0
sin cos tan
b a
b
b a
a
a
b
+
=
+
= = θ θ θ

Figure 11—3

: sin cos ) ( θ θ θ q p m + = directional derivative
)) , ( (
1
sin cos ) (
1
2 2 2 2
0 0 0
y x E f
b a b a
bq ap
q m

+
=
+
+
=
+ = ⇒ θ θ θ


S can be recovered from E(x,y) using above curve

Figure 11—4
96
δξ δ m z = (small change a long the characteristic curve)
)) , ( (
1
1
2 2
y x E f
b a
z
m

+
= =
δξ
δ

θ ξ ξ
θ ξ ξ
sin ) (
cos ) (
0
0
+ =
+ =
y y
x x where

0
0
1
0
2 2
0
( ) ( )
1
( ( , ))
z z m d
z f E x y d
a b
ξ
ξ
ξ θ ξ
ξ

= +
= +
+




iii. Rotationally symmetric reflectance maps

Figure 11—5


97
2 2
2 2
sin
cos
tan
q p
q
q p
p
p
q
s
s
s
+
=
+
=
=
θ
θ
θ


The slope
s s s
q p m θ θ θ sin cos ) ( + =

2 2
q p + =
)) , ( (
1
y x E f

=
The change in Z
δξ δξ θ δ
δξ δξ θ δ
δξ δξ δ
2 2
2 2
1 2 2
sin
cos
)) , ( (
q p
q
y
q p
p
x
y x E f q p m z
s
s
+
= =
+
= =
⋅ + = =


To simplify δz , we could take a step of length δξ
2 2
1
q p +
rather than δξ
δξ
δξ
δ
)) , ( (
) (
1
2 2
2 2 2 2
y x E f
q p
q p q p z

=
+ =
+ × + =

δx = pδξ - equation 11.1.1
δy = qδξ

98
We need to determine p&q at the new point in order to continue the solution
y x t r w y x E ate Differenti
q p f y x E
& , , ) , (
) ( ) , (
2 2
+ =

2 2
'
) (
q p S where
x
S
S f
x
S
S
E
x
E
E
x
+ =


=





=


=

y
z
q
x
z
p
x
q
q
x
p
p
x
s


=


=


+


=


, , 2 2

y x
z
q
x
z
p
∂ ∂

+


=
2
2
2
2 2

2 2
2
2( ) '( )
2( ) '( )
x
y
E pr qs f s
E ps qt f s
p p z z
p x x y
q y x x y
δ δ δ δ
= + |

= +
\
∂ ∂ ∂ ∂
= + = +
∂ ∂ ∂ ∂ ∂
- equation 11.1.2
y s x r δ δ + =
y t x s q δ δ δ + =

in our case δξ δ δξ δ q y p x = &
δξ δ
δξ δ
⋅ + =
⋅ + =
) (
) (
qt ps q
qs pr p

From (2)
δξ δ
δξ δ
' 2
' 2
f
E
q
f
E
p
y
x
=
=


As δξ → 0 , we obtain the differential equations
99

\
|
= =
= + = = =

' 2
,
' 2
)) , ( ( , ,
1 2 2
f
E
q
f
E
p
y x E f q p z q y p x
y
x
ɺ ɺ
ɺ ɺ ɺ


(x, y, z, p & q) solvable
5 Differential Eqs
one more time by differentiating
)) , ( (
' 2
,
' 2
. . , ,
1
y x E f z
f
E
q y
f
E
p x
t r w q y p x
y
x

=
= = = =
= =
ɺ
ɺ ɺ ɺ ɺ ɺ ɺ
ɺ ɺ ξ


iv. General Case

R(p,q) = f(ap + bq)
R(p,q) = f(P
2
+ q
2
)
R(p,q) = arbitarary


y s x r
y
y x
z
x
x
z
p
y q x p z
δ δ
δ δ δ
δ δ δ
+ =
∂ ∂

+


=
+ =
2
2
2
- equation 11.1.3

y t x s
y
y
z
x
y x
z
q
δ δ
δ δ δ
+ =


+
∂ ∂

=
2
2 2


100
|
|
¹
|

\
|
⋅ =
|
|
¹
|

\
|
|
|
¹
|

\
|
=
|
|
¹
|

\
|
y
x
H
y
x
t s
s r
q
p
δ
δ
δ
δ
δ
δ


where H is the Hessian Matrix
2
2
2
2
y
z
x
z
t r


+


= + : Laplacian
Differentiating E(x,y) w.r.t x & y

x
q
q
E
x
p
p
E
x
E
E
x





+





=


=

S R r R
S
y x
z
R
x
z
R
q p
q p
⋅ + ⋅ =
∂ ∂

⋅ +


⋅ =
2
2
2

t R S R E
q p y
⋅ + ⋅ =
|
|
¹
|

\
|
⋅ =
|
|
¹
|

\
|
q
p
y
x
R
R
H
E
E
- equation 11.1.4
from equation 11.1.3 & equation 11.1.4
Let δξ
δ
δ

|
|
¹
|

\
|
=
|
|
¹
|

\
|
q
p
R
R
y
x


q
p
R y
R x
=
=
ɺ
ɺ

Then δξ
δ
δ
δ
δ
|
|
¹
|

\
|
=
|
|
¹
|

\
|
⋅ =
|
|
¹
|

\
|
⋅ =
|
|
¹
|

\
|
y
x
q
p
E
E
R
R
H
y
x
H
q
p

101

\
|
= =
+ = + = ⋅ + ⋅ =
+ = =
=
y x
q p
q p q
p
E q E p
qR pR y q x p z
y
q
x
p
z
qR pR z R y
R x
ɺ ɺ
ɺ ɺ ɺ
ɺ ɺ
ɺ
,
,
δξ
δ
δξ
δ
δξ
δ

5unknown variables : x,y,z, p & q

Figure 11—6

II. 11.3 Singular points

Figure 11—7
) , ( ) , (
0 0
q p R q p R < for all ) , ( ) , (
0 0
q p q p ≠

Figure 11—8
102
0 , 0
0 , 0
= = = =
= =
q p
q p
R y R x
R R
ɺ ɺ

) (
2
1
) , (
2 2
q p q p R + = - equation 11.3.1
(p,q) = (0,0) – singular point
) 2 (
2
1
2 2
0
cy bxy ax z z + + + =

Thus by ax
dx
dz
p + = =
& cy bx
dy
dz
q + = =
substituting p & q in equation 11.3.1
] ) ( ) [(
2
1
) (
2
1
) , (
2 2 2 2
cy bx by ac q p q p R + + + = + =

2 2 2 2 2 2
) (
2
1
) ( ) (
2
1
y c b bxy c a x b a + + + + + =

The brightness gradient is
y c b bx c a E
by c a x b a E
y
x
) ( ) (
) ( ) (
2 2
2 2
+ + + =
+ + + =


Differentiating again,

+ =
+ =
+ =
2 2
2 2
) (
c b E
b c a E
b a E
yy
xy
xx
3 equations, 3 unknowns
103
III. Stereo graphic projection

Figure 11—9

= +
+
=
+
=
+
2 2 2
2 2
2 2
) (
2
R b a
R
q p
a
b
R
q f
a R
b

solve the above equations

2 2 2 2
1 1
2
,
1 1
2
q p
q
g
q p
p
f
+ + +
=
+ + +
=
conversely
2 2 2 2
4
4
,
4
4
g f
g
q
g f
f
p
− −
=
− −
=


104
IV. 11.7 Relaxation methods
characteristic strip expansion


E(x , y) = R(p , q)
= R
s
(f,g)
Minimize

∫ ∫
+ + + = dxdy gy gx fy f e
s
) ( ) (
2 2 2 2

A low, Minimize

∫ ∫
− = dxdy g f R y x E e
s i
2
)) , ( ) , ( (
overall, minimize
i s
e e λ +
if brightness measurement are very accurate,
λis large
if brightness measurement are very noisy,
λis small

105
minimize
∫ ∫
dxdy g g f f g f F
y x y x I
) , , , , , (
= It’s a problem in the calculus of variations.(see appendix)
The corresponding Euler equations are


0
0
=






=






y x
y x
g g g
f f f
F
y
F
x
F
F
y
F
x
F

where F
f
is the partial derivative of F with respect to F
2 2 2 2 2
)) , ( ) , ( ( ) ( ) ( g f R y x E g g f f F
s y x y x
⋅ + + + + = λ
The Euler equations for this problem yield

yy f
xx x f
s
s f
f F
y
f f
x
F
x
f
R
g f R y x E F
y
x
2
2 ) 2 (
)) , ( ) , ( ( 2
=


= ⋅


=




− − = λ


f f f yy xx
F F
y
F
x
f f
y x
=


+


= + ) ( 2

f
R
g f R y x E
s
s


− − = )) , ( ) , ( ( λ

g
R
g f R y x E g
s
s


− − = ∇ )) , ( ) , ( (
2
λ (2 equations, 2 unknowns(f,g))
Application to photometric stereo consider n images,
106

∫ ∫

∫ ∫
=
− + + + − =
n
i
i i I i y x y I
dxdy g f R y x E dxdy g g f fx e
1
2 2 2 2 2
)) , ( ) , ( ( )) ( ) (( λ
where E
i
(x,y) is the brightness measured in the i th image and R
i
is the corresponding
reflectance map
The Euler equations for this problem yield



=
=


− − = ∇


− − = ∇
n
i
i
i i i
i
i i
n
i
i
g
R
g f R y x E g
f
R
g f R y x E f
1
2
1
2
)) , ( ) , ( (
)) , ( ) , ( (
λ
λ


V. 11.8 Recovering Depth from a needle diagram

Figure 11—10



107

Figure 11—11

0 0
( , )
0 0
( , )
( , ) ( , ) ( )
x y
x y
z x y z x y pdx qdy = + +



Minimize the error
dxdy p z p z
I
p x
) ) ( ) ((
2 2
− + −
∫ ∫

where p and q are the given estimates of the components of the gradient while
y x
z z &
are the partial derivatives of the best-fit surface
Minimize
∫ ∫
dxdy z z z F
y x I
) , , (

The Euler equations is
0 =






y x
z z z
F
y
F
x
F

108

so that from
2 2
) ( ) ( q z p z F
y x
− + − =
0
) ( 2 ) ( 2
=



⋅ + −


⋅ =


+


=
q z
y
p z
x
F
y
F
x
F
y x
z z z
y x

) (
0 ) ( 2 ) ( 2
2
y x
y x yy xx
y yy x xx
q p z
q p z z
q z p z
+ = ∇
+ = +
= − + −


109
12. Motion field & Optical flow

Figure 12—1

• Motion field : a purely geometric concept
• Optical flow : Motion of brightness patterns
observed when a camera is moving relative to the objects being imaged

I. Motion Field

Figure 12—2
110

0
0
0
0
, &
ˆ '
i
r r
i
t t
i
v v
r r
f r z
δ δ
δ δ
= =
=

- equation 12.1.1

Differentiate - equation 12.1.1
2
0
0 0 0 0
2
0
0
0 0
0
) ˆ (
) ˆ ( ) ˆ (
) ˆ (
) ˆ ( ) ˆ (
'
1
z r
r z v v z r
z r
r z
dt
dr
dt
dr
z r
dt
r d
f
i

⋅ − ⋅
=

⋅ − ⋅
= ⋅

2
0
0 0
) ˆ (
ˆ ) (
'
1
z r
z v r
v
f
i

× ×
= ⋅ - equation 12.1.2

II. Optical flow

Figure 12—3
111
Ex1)

Figure 12—4

E(x, y, t) ≡ E(x + uδt , y + vδt , t + δt) where u(x, y) & v(x, y) are the x & y components
of the optical flow vector

For a small time interval t δ , if brightness varies smoothly with x, y, &t,
) , , ( ) , , ( ) , , ( t y x E
t
E
t
y
E
y
x
E
x t y x E t t t v y t u x E ≡ + + + + = + + + ρ
α
α
δ
α
α
δ
α
α
δ δ δ δ
where ρ contains second and higher order terms in δx , δy , and δt

0 = ⋅


+ ⋅


+ ⋅


t
t
t
E
t
y
y
E
t
x
x
E
δ
δ
δ
δ
δ
δ

0 = + +
t y x
E v E u E (Optical flow constraint equation)
u, v : two variables
one equation :
⇒ one more constraint needed

III. Smoothness of the optical flow
Usually, the motion field varies smoothly minimize a measure of departure from
smoothness
112
∫ ∫
+ + + = dxdy vy vx uy ux e
s
)) ( ) ((
2 2 2 2

Also, minimize the error in the optical flow constraint eq.
∫ ∫ ∫ ∫
+ + = − =
+ +
y dxd E v E u E dxdy t y x E t y x E e
t y x att t at c
2 2
) ( ] ) , , ( ) , , ( [
δ

Hence, minimize
c s
e e λ + ,
Minimize
c s
e e λ +
Where
∫ ∫
+ + + = dxdy vy vx uy ux e
s
)) ( ) ((
2 2 2 2


∫ ∫
+ + = dxdy E v E u E e
t y x c
2
) (
λ: a parameter that weights the error in the image motion equation relative to the
departure
from smoothness
λ=> Large
=> small
Minimize
∫ ∫
dxdy v v u u v u F
y x y x
) , , , , , (

2 2 2 2 2
) ( ) ( ) (
t y x y x Y X
E v E u E v v u u F + + + + + + = λ
Euler equations are

0
0
=






=






y x
y x
v v v
u u u
F
y
F
x
F
F
y
F
x
F


113
yy y u
xx x u
x t y x u
u u
x
F
y
u u
x
F
x
E E v E u E F
y
x
2 ) 2 (
2 ) 2 (
) ( 2
=


=


=


=


+ + = λ


u u u E E v E u E
yy xx x t y x
2
) ( ∇ = + = + + λ
Similarly,
0
) (
2
= + +
∇ = + +
t y x
y t y x
E v E u E
v E E v E u E λ


IV. Boundary conditions
*Natural boundary conditions (see appendix)
ds
dx
F
ds
dy
F
ds
dx
F
ds
dy
F
y x
y x
v v
u u
=
=

where S denotes arc length along the boundary curve
T
ds
dx
ds
dy
n |
¹
|

\
|
− = , ˆ is a unit vector
perpendicular to the boundary
Rewriting the above conditions.
0 ˆ ) , (
0 ˆ ) , (
= ⋅
= ⋅
n F F
n F F
y x
y x
v v
u u

114
in our case,
2 2 2 2 2
) ( ) ( ) (
t y x y x y x
E v E u E v v u u F + + + + + + = λ
0 ˆ ) , (
0 ˆ ) , (
= ⋅
= ⋅
n v v
n u u
y x
y x


V. Discrete case
From
c s
e e e λ + =
dxdy E v E u E dxdy v v u u
t y x y x y x
2 2 2 2 2
) ( )) ( ) (( + + + + + + =
∫ ∫ ∫ ∫
λ
) (
ij
i j
ij
c s e λ + =
∑∑


) ) ( ) ( ) ( ) ((
4
1
2
, 1 ,
2
, , 1
2
, 1 ,
2
, , 1 , j i j i j i j i j i j i j i j i j i
v v v v u u u u s − + − + − + − =
+ + + +

2
) (
t ij y ij x ij
E v E u E c + + =
Differentiating w.r.t
kl kl
v u & yields

,
) ( 2 ) ( 2
, ) ( 2 ) ( 2
y t kl y kl x kl kl
kl
x t kl y kl x kl kl
kl
E E v E u E v v
v
e
E E v E u E u u
u
e
+ + + − =


+ + + − =


λ
λ

where v u & are average of u & v

115
0
0
=


=


kl
kl
v
e
u
e
solve this

t y kl x kl x y kl y x
t x kl y x kl y kl y x
E E v E u E E v E E
E E v E E u E u E E
λ λ λ λ
λ λ λ λ
− + + − = + +
− − + + = + +
) 1 ( )) ( 1 (
, ) 1 ( )) ( 1 (
2 2 2
2 2 2



Iterative scheme is
.
) ( 1
,
) ( 1
2 2
1
2 2
1
y
y x
t
n
kl y
n
kl x n
kl
n
kl
x
y x
t
n
kl y
n
kl x n
kl
n
kl
E
E E
E v E u E
v v
E
E E
E v E u E
u u
+ +
+ +
− =
+ +
+ +
− =
+
+
λ
λ


Figure 12—5
116
13. Photogrammetry & Stereo
I. Disparity between the two images.

Figure 13—1
Disparity : D x x
r l
= −
' '

) 3 (
) 2 ( ) (
2
) 1 ( ) (
2
' '
'
'
− = =
− ←

=
− ←
+
=
z
y
f
y
f
y
line red
z
b
x
f
x
line blue
z
b
x
f
x
r l
r
l

solve (1) (2) & (3)
f
z
b
x x
r l
= − ) (
' '
: disparity
' ' ' '
' '
' '
' '
,
2 / ) (
,
2 / ) (
r l r l
r l
r l
r l
x x
bf
z
x x
y y
b y
x x
x x
b x

=


=


=
117
D x x
z
r l
1 1
' '
=



II. Photogrammetry

Figure 13—2

• absolute orientation
) , , ( known r r
r l

r
r
r
z
y
x

0
r r R r
l r
+ =

33 32 31
23 22 21
13 12 11
r r r
r r r
r r r
component
rotaional

34
24
14
r
r
r
component
nal translatio


x
r
118
where R is a 3*3 orthonormal matrix
I R R
T
= ⋅

r l l l
r l l l
r l l l
z r z r y r x r
y r z r y r x r
x r z r y r x r
= + + +
= + + +
= + + +
34 33 32 31
24 23 22 21
14 13 12 11


3 equations => 12 unknowns

• Orthonormality
I R R
T
= ⋅
1
1
1
2
33
2
32
2
31
2
23
2
22
2
21
2
13
2
12
2
11
= + +
= + +
= + +
r r r
r r r
r r r


0
0
0
13 33 12 32 11 31
33 23 32 22 31 21
23 13 22 12 21 11
= + +
= + +
= + +
r r r r r r
r r r r r r
r r r r r r



Figure 13—3

119
1 2 1 2 r r l l
r r r r − = −
3+7 = 10 equations
12 unknown variables

P&Q : 10+3 = 13 equations

Figure 13—4

0
r r R r
l r
+ ⋅ = - equation 13.3.1

=

=
34
24
14
0
33 32 31
23 22 21
13 12 11
r
r
r
r
r r r
r r r
r r r
R

III. Relative orientation
r l
r r , ; not known
r l
z z , unknowns
The projection points ) , ( & ) , (
' ' ' '
r r l l
y x y x known determine the ratios of x and y to z
using
120
r
r r
r
r r
l
l l
l
l l
z
y
f
y
z
x
f
x
z
y
f
y
z
x
f
x
= =
= =
' '
' '
&
&
- - equation 13.4.1

12 + 2*(n)
substitute B into A

l
r
r l
l l
l
l
r l l l
z
f
f
z
x r z r
f
z
y r
f
z
x r
z
f
x r z r y r x r
) ( ) ' (
'
14 13 12
'
11
14 13 12 11
= + + +
= + + +



= = + + +
= = + + +
= = + + +
l
r
l
l l
r
l
r
r
l
l l
r
l
r
r
l
l l
z
z
f
z
f
r f r y r x r
s
y
z
z
y
z
f
r f r y r x r
s
x
z
z
x
z
f
r f r y r x r
34 33
'
32
'
31
'
24 23
'
22
'
21
'
14 13
'
12
'
11
- equation 13.3.2

12 + 2n ; # of unknowns for n points
7 + 3n ; # of equations
7 from orthonormality & distance length

121
I R R
T
= ⋅

1 0 0
0 1 0
0 0 1

j i if r r
r
j i
i
≠ =
=
0
1
2

∴12 + 2n = 7 + 3n
n = 5
n ≥5

IV. 13.5 Using a known relative orientation
s
f
z
l
= ; constant s x
f
z
x x
l
l
l l
⋅ = =
/ '
s y y
l l
⋅ =
'
s f z
l
⋅ =
where s is a scaling factor
from equation 13.3.2
w cs r s f r y r x r z
v bs r s f r y r x r y
u as r s f r y r x r x
l l r
l l r
l l r
+ = + + + =
+ = + + + =
+ = + + + =
34 33
'
32
'
31
24 23
'
22
'
21
14 13
'
12
'
11
) (
) (
) (

w cs
v bs
z
y
f
y
w cs
u as
z
x
f
x
r
r r
r
r r
+
+
= =
+
+
= =
'
'





122
V. Computing depth
Once IR , ) , ( ), , ( ,
' ' ' '
0 r r l l
y x y x r known, The depth ) , (
r l
z z can be computed
as follows

Figure 13—5

From equation 13.3.2, multiply by
f
z
l

r l
l l
r
r
l
l l
r
r
l
l l
z r z r
f
y
r
f
x
r
z
f
y
r z r
f
y
r
f
x
r
z
f
x
r z r
f
y
r
f
x
r
= + + +
= + + +
= + + +
34 33
'
32
'
31
'
24 23
'
22
'
21
'
14 13
'
12
'
11
) (
) (
) (


From any two equations
Compute
r l
z z &
Then compute
r
r r T
r r r l
l
l l T
l l l l
z
f
y
f
x
z y x r
z
f
y
f
x
z y x r
|
|
¹
|

\
|
= =
|
|
¹
|

\
|
= =
1 , , ) , , (
1 , , ) , , (
' '
' '


123
VI. 13.7 exterior orientation

Figure 13—6
0
r r R r
a c
+ ⋅ =

c a a a
c a a a
c a a a
z r z r y r x r
y r z r y r x r
x r z r y r x r
= + + +
= + + +
= + + +
34 33 32 31
24 23 22 21
14 13 12 11


I R R
T
= ⋅ (orthonormality)

c
c
c
c
z
y
f
y
z
x
f
x
= =
'
&
'

34 33 32 31
24 23 22 21
34 33 32 31
14 13 12 11
'
'
r z r y r x r
r z r y r x r
z
y
f
y
r z r y r x r
r z r y r x r
z
x
f
x
a a a
a a a
c
c
a a a
a a a
c
c
+ + +
+ + +
= =
+ + +
+ + +
= =
- equation 13.7.1

# of equations = 2n +6
# of unknowns = 12
2n+6 = 12 => n=3 (at least we need three equations)
124
VII. Interior Orientation
• scaling error
• translation error
• skewing error
• shearing error

An affine transformation


23 22 21
13 12 11
) ( ) ( '
) ( ) ( '
a
z
y
a
z
x
a y
a
z
y
a
z
x
a x
c
c
c
c
c
c
c
c
+ + =
+ + =


from equation 13.7.1

34 33 32 31
24 23 22 21
34 33 32 31
14 13 12 11
'
'
s z s y s x s
s z s y s x s
f
y
s z s y s x s
s z s y s x s
f
x
a a a
a a a
a a a
a a a
+ + +
+ + +
=
+ + +
+ + +
=


ij
s
R : not orthonormal
I R R
ij ij
s
T
s
≠ ⋅ 12 unknowns
#of equations = 2n
∴ 2n=12 ; n=6
125
VIII. Finding conjugate points
i. Gray-Level Matching
• (reference figure 3-1)
z
b
x
f
x
z
b
x
f
x
r l 2
&
2
' ' −
=
+
=
at a matched point
) , ( ) , (
' ' ' '
r r r l l l
y x E y x E =
) (
) ' ,
2
( ) ' ,
2
( ~
' '
y y y
y
z
b
x
f E y
z
b
x
f E E
r l
r l
= ≅

=
+


Let
z
bf
y x d
z
x
f
x
= ≅ ) ' , ' ( &
'


) ' ), ' , ' (
2
1
' ( ) ' ), ' , ' (
2
1
' ( y y x d x E y y x d x E
r l
− = +
criterion : minimize
i s
e e e λ + =
where
∫ ∫
∇ ⋅ ∇ = ' ' ) (
2
dy dx d e
s


∫ ∫
− = ' ' ) (
2
dy dx E E e
r l i

(solution) using Euler equation
0
' '
' '
=






y x
d d
F
y
F
x
Fd
where
126
( )
]
' '
[
2
1
()] () [ 2
]
'
()
)
2
1
( )
2
1
(
'
()
()][ () [ 2
)] ' ), ' , ' (
2
1
' ( ) ' ), ' , ' (
2
1
' ( [ ) (
2 2
x
E
x
E
E E
x
E
x
E
E E E Fd
y y x d x E y y x d x E d F
r l
r l
r l
r l
r l


+


− =


− −


− − =
− − + + ∇ ⋅ ∇ =
λ
λ
λ


]
'
()
'
()
[
2
1
()] () [ ) (
) ( 2
' '
2 2
2 2
' '
x
E
x
E
E E d
d F
y
F
x
r l
r l
dy dx


+


− = ∇ ∇
∇ ∇ =


+


λ ∵

127
14. Pattern classification
Reference – “ Pattern Classification & Scene Analysis”
Dudda & Hart (Wiley Interscience pub. C)

1. Bayes Decision Theory
Fundamental statistical approach to the problem of pattern classification
• Maximum likelihood estimation : parameters fixed but unknown
• Baysian estimation : parameters, random variables having some known a prior
distribution

(A) Maximum likelihood estimation

Figure 14—1

Given ; 1. Model of system
2. pata : outcome of some type of probabilistic experiment.
Goal : Estimate unknown model parameters from data
Let n i
i
. ,......... 2 , 1 , = λ be n I.I.D. observations on a random variable X drawn from
) : ( θ x f
x

128
The joint lpdf of
n
X X X ,........ ,
2 1
is
1 x
f

Where ) ; .. ,......... (
1
θ
n
x x L is called the likelihood function (LF)
The maximum likelihood estimation (MLE) θ
ˆ
is that value that maximizes the LF, that
is
) ; .. ,......... ( )
ˆ
; .. ,......... (
1 1
θ θ
n n
x x L x x L ≥

Ex1) Assume ) , ( ;
2
σ µ N X where σis known. Compute the MLE of the mean μ
<solution> The LF for h realizations of X is on a
|
¹
|

\
|
− −
|
|
¹
|

\
|
=

=
h
i
i
n
x X L
0
2
2
2
) (
2
1
exp
2
1
) ; ( µ
σ
πσ
µ

since the log function is monotonic, the maximum of L(x ; μ) is also that of log L(x ; μ)

Hence

=
− − − =
n
i
i
x
n
x L
1
2
2
2
) (
2
1
) 2 log(
2
) ; ( log µ
σ
πσ µ
Set 0
) ; ( log
=


u
x L µ

This yields

=
= −

n
i
i
x
0
2
0 ) (
1
µ

=
=
n
i
i
x
n
1
1
ˆ µ

Ex2) Consider the normal p.d.f.,
129

=
− − =
n
i
x
x x f
1
2
2
2
2
) ) (
2
1
exp(
2
1
) , , ( µ
σ
πσ
σ µ
The log likelihood function is


=
− − − − =
n
i
i
x n
n
j x x L
1
2
2
2
2 1
) (
2
1
log 2 log
2
) , ,....... ( log µ
σ
σ π σ µ
Now set 0 , 0 =


=


σ µ
L L

Obtain the simultaneous equations



=
=
= − + −
= −
n
i
i
n
i
i
x
n
x
1
2
3
1
0 ) (
1
0 ) (
µ
σ σ
µ


Hence

=
=
n
i
i
X
n
1
1
ˆ µ


=
− =
n
i
i
x
n
1
2 2
) ˆ (
1
ˆ µ σ

(B) Baysian Estimation
Suppose that an classifier to separate two kinds of lumber, ash and birch


Figure 14—2

130
A prior : • probability p(w
1
) : the next piece is ash
• probability p(w
2
) : the next piece is birch

Decision Rule : Decide w
1
if p(w
1
) > p(w
2
)
Decide w
2
otherwise
Bay’s Rule ;


) (
) ( ) | (
) | (
x p
w p w x p
X w p
j j
j

=
where

=
⋅ =
r
j
j j
w p w x p x p
1
) ( ) | ( ) (

Bayes Decision Rule for minimizing the probability of error ;
Decide w
1
if p(w
1
|x) > p(w
2
|x)
Decide w
2
otherwise

Decide w
1
if P(x| w
1
)p(w
1
) > p(x| w
2
)p(w
2
)
Decide w
2
otherwise

If equally likely, p(w
1
)=p(w
2
), then
131
Decide w
1
if p(x| w
1
) > p(x| w
2
)
Decide w
2
if otherwise

Figure 14—3

Probability error : p(error)
( ) ( , )
( | ) ( )
p error p error x dx
p error x p x dx

−∞

−∞
=
=



where p(error|x)

\
|
1 2
2 1
) | (
) | (
w decide we if x w p
w decide we if x w p


Figure 14—4
132
(B-1) Generalized Form (for Bayes classifier)
Let } .......... {
1 s
w w = Ω be the finite set of S states of nature and ) ........ {
1 a
A α α = be the
finite set of possible actions.


Let ) | (
j i
w α λ be the loss incurred for taking action
i
α when the state of nature is
j
w
Let the feature vector X be a d-component vector-valued r.v
Then
) (
) ( ) | (
) | (
x p
w p w x p
x w p
j j
j
=
Where

=
=
s
j
j j
w p w x p X p
1
) ( ) | ( ) (
The expected loss associated with taking action
i
α is
) | ( ) | ( ) | (
1

=
⋅ =
s
j
j j i i
X w p w X R α λ α

Target: to find a Bayes decision rule against p(w
j
) that minimizes the overall risk
The overall risk is given by


⋅ = x d x p x x R R ) ( ) | ) ( (α
clearly, if ) (x α is chosen so that ) | ) ( ( x x R α is as small as possible, for every X , then
the overall risk will be minimized Bayes decision rule for the general form; to minimize
the overall risk.
133
1.Compute the conditional risk

=
⋅ =
s
j
j j i i
X w p w X R
1
) | ( ) | ( ) | ( α λ α
for i=1,………..a


2.Select the action
i
α for which ) | ( X R
i
α is minimum
3.Obtain the resulting minimum overall risk called the Bayes risk
B.2 Two category classification
Let ) | (
j i ij
w α λ λ = the loss incurred for deciding w
i
when the true state of nature is w
j
.
) | ( ) | ( ) | (
) | ( ) | ( ) | (
2 22 1 21 2
2 12 1 11 1
X w p X w p X R
X w p X w p X R
λ λ α
λ λ α
+ =
+ =
- equation 14.1

The decision rule :
Decide w
1
if ) | ( ) | (
2 1
X R X R α α < - equation 14.2
Decide w
2
otherwise


If we substitute 1 into 2
We have
) | ( ) ( ) | ( ) (
) | ( ) | ( ) | ( ) | (
1 11 21 2 22 12
2 22 1 21 2 12 1 11
X w p X w p
X w p X w p X w p X w p
λ λ λ λ
λ λ λ λ
− < − ⇒
+ < +

Hence likelyhood ratio
134

Figure 14—5

<Minimum – error – rate classification>
Symmetrical on zero-one loss function
The loss ) | (
j i
w α λ = 0 i=j for i,j=1,……c
1 i≠j
all errors are equally costly
The conditional risk is

=
⋅ =
c
j
j j i i
X w p w X R
1
) | ( ) | ( ) | ( α λ α

) | ( 1
) | (
X w p
X w p
i
c
i j
j
− =
=




To minimize the average prob of error, we should select the i that maximizes the a
posterior prob
) | ( X w p
i
In other words, for minimum error rate j
Decide
i
w if ) | ( ) | ( X w p X w p
j i
> for all , i j ≠
B-3 The multicategory classification
135
A pattern classifier

Figure 14—6

), ( X g
i
for i=1,………c ; discriminant function
The classification is said to assign a feature vector X to class
j
w
If ) ( ) ( X g X g
j i
> for all i j ≠
) | ( ) ( X w p X g
i i
= ; a posterior prob


=
=
c
j
j j
j i
w p w X p
w p w X p
1
) ( ) | (
) ( ) | (
by Bayes rule

) ( log ) | ( log
) ( ) | (
i i
i i
w p w X p
w p w X p
+ ⇒



Decision Rule
1. Divide the feature space into C division regions, R
1
,………..R
c

2. If ) ( ) ( X g X g
j i
> for all j≠i then X is in R
I

136
In other words, assign X to w
I


Ex) three category classification

Probability error calculation
P(error X
o
) ) ( ) (
0 0
X P X error P ⋅ =
) ( ) ( ) ( ) (
3 3 0 2 2 0
w P w x P w P w X P + =
Total error
P(error)


∞ −
= dx x error P ) , (

Course outline:

The principles of the machine/robot vision are introduced. It covers image formation, pattern classification, motion and optical effect for object recognition. In addition, the design technology of the robot vision system with optical device is studied.

Prerequisite:

Digital Signal Processing, Image Processing.

Textbook and References:

1. 2.

Robot Vision, B. K. P. Horn, MIT Press. Computer Vision, Dana Ballard and Christopher Brown, Prentice Hall.

Course Schedule 1. Image Formation & Image Sensing 2. Binary Images: Geometrical Properties 3. Binary Images: Topological Properties 4. Regions & Image Segmentation 5. Image Processing: Continuous Images 6. Image Processing: Discrete Images 7. Edge & Edge Finding 8. Lightness & Color 9. Reflectance Map: Photometric Stereo 10. Reflectance: Shape from Shading 11. Motion Field & Optical Flow 12. Photogrammetry & Stereo 13. Pattern Classification 14. Polyhedral Objects 15. Extended Gaussian Images 16. Passive Navigation & Structure from Motion

1.
I.

Introduction
Human vision system

Figure 1—1

• Rod : more sensitive to light Scotopic vision Slender receptor

• Cone : shorter & thicker in structure Photopic vision : R.G.B 6.5 million

• Fovea : Density of cones is greatest The region of sharpest photopic vision Camera visual system

1

G. Photometric information brightness : (ex: bright green. B) Figure 1—3 2 .Figure 1—2 i. dark green) ii. R. Geometric information shape color (ex.

RV Robot vision Computer graphics Image processing 2 . II.261 II.NTSC : US standard for TV picture and audio coding and transmission (timing calculation) For one frame : For RGB : 512×512 = 256k Byte 3 30 frames/sec For real time : 256k × 3 × 30 = 22.544 Mbps : 180 ≈ 120 1. IV Compression JPEG Fractal H.544 MPEG I.5M Byte/sec 180 M bit/sec 180 Mbps Compression ratio Wavelet compression compressio T1 1.

Figure 1—4 Figure 1—5 3 .

Microstructure 2. r. structured descriptions suitable for The apparent brightness of a surface depends on three factors 1. Distribution of incident light 3. Scene analysis decision marking : :a detailed but undigested description more parsimonious.“Scene description” Under constrained problems : for example 2 equations Inverse problem 3 unknown variables are given 1. Image analysis 2. t light source & observer 4 . Orientation of the surface w.

Figure 1—6

“Lambertian Surface” One that appears equally bright from all viewing directions and reflects all incident light, absorbing none

5

2.

Image Formation & Image sensing

What determines where(Shape-Geometric) the image of some point will appear? What determines how(Brightness, Color – Photonetiz) bright the image of some surface will be?

I.

Two Aspects of Image Formation

i. Perspective projection

Figure 2—1

6

ii. Orthographic Projection
δ x '2 + δ y '2 f' = = m (magnification) z0 δ x2 + δ y 2

In case the depth range of a scene is small relative to the distance z0

Figure 2—2

 x' = mx   y ' = my ⇒  if m=1 =>

 x' = x   y' = y 

This orthographic projection can be modeled by rays parallel to the optical axis

(rather than one passing through the origin)

(a)

(b)

Figure 2—3

Where δP is the power of the radiant energy falling on the

E=

δp [watt/m] δI
7

and D is Distance D2 Ex 1 Hemisphere Solid angle = δw R Figure 2—4 4π R 2 1 ⋅ 2 = 2π 2 R Whole sphere = 4π Ex 2 Small patch 8 . Brightness δI 1.II. Image Brightness : Irradiance E = δ P [watt / m 2 ] 2. where A is area. Scene Brightness : L= L = δ 2P 2 [watt / m 2 ] where δ P is the power emitted by the infinitesimal surface patch δ Iδ W of area (δI) into an infinitesimal solid angle (δw) SolidAngle ≡ A .

Figure 2—5 Lens Figure 2—6 Image Irradiance δP E= δI δ I cos α Solid angle = ( f / cos α )2 Transfer Function: f (i) = Scene Radiance δP L= δ Oδ w ' Solid angle = δ 0 cos θ ( z / cos α )2 δL δE 9 .

Lens formula 1 1 1 = + f z' z 2–1 s − z' z' = 2R D 2–2 Solve (2-1) & (2-2) to calculate R – blur circle radius 10 . Lenses Figure 2—8 i.f(·) : BRDF(Bidirectional Reflectance Distribution Function) in Ch 10 Ex3) Sphere/Radius 2m Figure 2—7 III.

all the rays that enter the front surface of the lens end up being focused in the image Figure 2—10 11 . Vignetting In a simple lens. The larger the lens aperture => the less the DOF Figure 2—9 iii. Depth of field (DOF) The range of distances over which objects are focused “sufficiently well” in the sense that thediameter of the blur circle is less than the resolution of the imaging device.R= DS 1 1 1 ( − − ) 2 f s z R = 0 at focused point P′ ii.

and so on … Vignetting is a reduction in light-gathering power with increasing inclination of light rays with respect to the optical axis Figure 2—11 iv. some of the rays that pass through the first lens may be occluded by portions of the second lens.In a compound lens. which those in a corner of the image are smeared out 12 . Aberration Points on the optical axis may be quite well focused.

B filter R(λ) : 1 ( at Red color) 13 . G.IV. Color sensing a) Human eye Figure 2—12 b) CCD sense R. Image sensing i.

are the random out comes of H If to every δ. such a rule subject to certain constraints is called a random variable 14 . Randomness and Noise a) Random variable [R.0 (otherwise) G(λ) : 1 (at Green color) 0 (otherwise) B(λ) : 1 (at Blue color) 0 (otherwise) → UV filter filtered out UV ray ii.δ.V] Consider an experiment H with sample description space Ω the elements on points of Ω. we assign a real number X(δ) The real establish a correspondence rule between δ and R the real time.

d. therefore: ∞ ∫ p( x)dx = 1 −∞ µ= mean : ∫ xp( x)dx : first moment ∞ −∞ δ2 = variance : 2 ∫ ( x − µ ) p( x)dx = −∞ ∞ ∫x 2 p( x)dx : second moment −∞ Cumulative probability distribution (CDF) P(x) x P( x) = ∫ p( x)dt −∞ X = X1 + X 2 Two R.V. R . S P(X1) & P(X2) what P(x) = ? Solution 15 .V.Figure 2—13 R. Probability density function (p.f) p(x) ∞ Where p(x) ≥ 0 for all x.V .

f : 16 . x1 : x-x2 P1 ( x1 ) ⋅ δ x1 = P ( x − x2 )δ x 1 Now. x2 can take on a range of values P2(x2). ∞ P( x) ⋅ δ x = ∴ P( x) = ∫ P ( x − x )δ xP ( x )dx 1 2 2 2 −∞ ∞ 2 ∫ P ( x − x ) P ( x )dx 1 2 2 2 −∞ ∞ 2 =∫ ∫ P ( x − t ) P (t )dt 1 2 −∞ = P ∗ P2 1 For multiple RV's : x= 1 N ∑x i =1 N i Mean value Variance Standard Deviation : : : Nµ =µ N Nσ 2 σ 2 = N2 N ⇒ σ N b) Gaussian (Normal) −  1 p ( x) = e 2 2πσ 1  x−µ   σ  2 p.δ(x2) To find the prob. that x lies between x and x+δx.d.Figure 2—14 Given x 2 .

v.f of Z? 2 (a) (b) Figure 2—15 17 . what is the p.d.'s with px ( x) = e − x u ( x ) and 1 p y ( y ) = [u ( y + 1) − u ( y − 1)] and Let Z ≡ X + Y .f : p ( k ) = e− m ⋅ mn n! where n arrivals in a time interval T for some m mean = m variance = σ2 Ex Let X and Y be independent r.d.c) Poisson (m>0) p.

.∞ p z ( z ) = px ( x ) * p y ( y ) = (a) (b) (c) z < −1 −1 ≤ z < 1 z ≥1 ∫ p ( z − y) p ( y)dy x y −∞ . . pz ( z ) = 0 1 pz ( z ) = ∫ e− ( z − y ) dy 2 −1 pz ( z ) = 1 −( z− y ) ∫ e dy 2 −1 1 z Figure 2—16 18 .

I. Area of the object ρ sin θ + ρ cos θ = 1 ⇒ x sin θ − y cos θ + ρ = 0 A = ∫ ∫ I b( x. Binary images : Geometric properties Binary Images Figure 3—1 x − y 1. x= 1 A 1 A Position of the object : ∫ ∫ xb( x. (ρ.3. y)dxdy : I second moment 19 . zero-th Moment center of area 2. y)dxdy ∫ ∫ b( x. y )dxdy . y)dxdy : First moment y= 3.θ) Orientation of the object E = ∫ ∫ r 2 b( x.

y0) differentiate with respect to S and set equal to zero d (r 2 ) =0 ds S = x cosθ + y sin θ x − x0 = x (sin 2 θ + cos 2 θ ) + ρ sin θ − ( x cos θ + y sin θ ) cos θ = x sin 2 θ + ρ sin θ − y sin θ cos θ = sin θ (sin θ − y cos θ + ρ ) y − y 0 = − cosθ ( x sin θ − y cosθ + ρ ) ∴ r 2 = ( x − x0 ) 2 + ( y − y 0 ) 2 = ( x sin θ − y cosθ + ρ ) 2 20 .II. y) to (x0. Simple Geometrical Properties Figure 3—2 r 2 = ( x − x0 )2 + ( y − y0 )2 = ( x − (− ρ sin θ + s cos θ )) 2 + ( y − ( ρ cos θ + s sin θ ))2 = ( x 2 + y 2 ) + ρ 2 + 2 ρ ( x sin θ − y cos θ ) − 2 s( x cos θ + y sin θ ) + s 2 To find the shortest distance from (x.

t ρ and setting the result to zero lead to dE = 2( x sin θ − y cos θ + ρ )b( x. At ( x. Finally. y ). y) dxdy I Differentiating w. x sin θ − y cosθ + ρ = 0. y)dxdy = ∫ ∫ x ' sin θ b( x. y )dxdy + ∫ ∫ y '2 cos 2 θ b( x.r. y )dxdy = 0 d ρ ∫ ∫I A( x sin θ − y cosθ + ρ ) = 0 Let x' = x − x ⇒ x = x − x' y' = y − y ( x − x' ) sin θ − ( y − y ' ) cosθ + ρ ) = 0 x sin θ − y cosθ + ρ = x' sin θ − y ' cosθ and so ∫ ∫ ( x 'sin θ − y 'cos θ ) b( x. y )dxdy + ∫ ∫ (−2 x ' y ') cos θ sin θ 2 I 2 2 I I b( x. y)dxdy 2 = ∫ ∫ ( x sin θ − y cos θ + ρ ) 2 b( x. 2 21 .If. r=0 . x sin θ − y cosθ + ρ = 0. E= ∫∫ r b( x. y )dxdy I = a sin θ − b sin θ cos θ + c cos 2 θ where.

Projections Figure 3—3 22 . y )dx ' dy ' I b = 2 ∫ ∫ ( x ' y ') ⋅ b( x. a=c 1 b  −1 θ = tan 2 a−c ∴  ρ = − x sin θ + y cosθ  III.r. we have tan 2 θ = unless b a−c b=0 . y )dx ' dy ' I Now E= 1 1 1 (a + c) − (a − c) cos 2θ − b sin 2θ 2 2 2 Differentiating w. y )dx ' dy ' I c = ∫ ∫ ( y ')2 b( x.t θ and setting the result to zero.a = ∫ ∫ ( x ')2 b ( x.

y )dy.h( x) = ∫ b( x. y )dxdy = ∫ h( x) dx Area : = ∫ v( y )dy x= 1 1 ∫ ∫ xb( x. y)dxdy = A ∫ xh( x)dx A 1 y = ∫ yv( y )dy A 1 b tan −1 2 a−c Position : Orientation ( ρ . y)dxdy = ∫ x h( x)dx I ∫∫ I y 2b( x. V ( x) = ∫ b ( x. y )dx A = ∫ ∫ b( x.θ ) 2 : θ= ρ = − x sin θ + y cosθ 2 ∫ ∫ x b( x. y )dxdy = ∫ t 2 d (t )dt − 1 2 1 2 ∫ x v( x) − 2 ∫ y h( y)dy 2 Diagonal projection (θ=45°) Figure 3—4 23 . y )dxdy = ∫ y 2v ( x)dy ∫∫ I xyb( x.

y )dxdy = ∫ ∫ t 2 d (t )dt I 2 1 1 = ∫ ∫ ( x 2 + xy + y 2 )b( x. y )dxdy = ∫ b( 1 1 (t − s ).θ ) ∑∑ j i j i j bij ij ∑∑ ijb 24 . (t + s )ds 2 2 Now consider that 1 ( x + y )2 b( x. y )dxdy I 2 2 ∫∫ I so. j ) 1 ∑∑ ibij A i j 1 ∑∑ jbij A i j j= ∑∑ i b 2 i j 2 ij orientation ( ρ . ∫ ∫ xyb( x. y)dxdy = ∫ t d (t )dt − 2 ∫ x h( x)dx − 2 ∫ y v( y )dy 2 2 2 I 1 1 IV.Suppose that θ is 45° d (t ) = ∫ b( x. Discrete Binary Images (reference of book figure 3-8) area A = ∑∑ bij i =1 j =1 N M i= position (i .

.. 2 k = ∑ ri 2 + ri 4 + ri 6 + .....V. Run-Length coding Figure 3—5 Where rik is the k-th run of the i-th line and the first run in each row is a run of zeros n mi 2 n A = ∑∑ ri ....ri i =1 k =1 i =1 mi 2 position (center of area) : (i. j ) 25 .

... Find the first horizontal differences of the image data Figure 3—6 2. The first difference of the vertical projection can be computed from the projection of the first 26 ....... 2 k k =1 mi 2 i= vj 1 n 1 n ∑ i ⋅ hi . j = A ∑ jv j A i =1 i =1 1.hi = ∑ ri .

t= 1 2 (i + j ) dt can be obtained in a way similar to that used to obtain the vertical projection 27 .horizontal differences ⇒ count the number of circles subtracted by the number of triangles.θ ) ∑i i j 2 hi vj ij ∑j i 2 ∑∑ ijb j = ∑ t 2dt − t 1 1 ∑ i 2 h j − 2 ∑ j 2v j 2 i j Where. 3. The vertical projection vj can be found by summing the result from left to right Orientation (ρ .

Binary Image : Topological properties Jordan curve theorem A simple closed curve separate the image into two simply connected regions Figure 4—1 4 Connectedness – only edge-adjacent cells are considered neighbors.4. Figure 4—2 28 .

Ex 1 Figure 4—3 4 objects ` 2 backgrounds No closed curve ⇒ contradiction(by Jordan curve theorem) 8 connectedness – Corner-adjacent cells are considered neighbors. too. Figure 4—4 6 connectedness 29 .

Figure 4—5 Figure 4—6 Figure 4—7 i. A sequential Labeling Algorithm Recursive Labeling 1. Choose a point where bij=1 and assign a label to this point and into neighbors 30 .

Find new places to start a labeling operation whenever an unlabeled point is found where bij=1 4. Next.Figure 4—8 2. label all the neighbors of these neighbors ⇒ one component will have been labeled completely 3. Try every cell in this scan Sequential Labeling 31 .

if A is zero. if D has been labeled. if A is one.Figure 4—9 1. choose a new label for A] (go to step 2) 32 . there is nothing to do. simply copy that label and move on if not [if one B orc is labeled. copy that label else. Figure 4—10 2.

Local counting Figure 4—11 Horizontal N Vertical N Total Length = 2N 33 . Local counting and iterative modification i.II.

of Holes No. 2. of upstream concavities convexities(+1) concavities(-1) Figure 4—13 34 . of upstream convexities – No. of Bodies – No. No.Figure 4—12 The overestimated rate : 2 N : 2 N = 2 : 1 Considering all slopes overestimated average ratio Overestimated average rate : : 4 π 4 π -Euler number1.

body .1 = 0 1 .0 = 2 1 .0= 1 ii. The additive set property Figure 4—15 35 .concavity : 1.2 = -1 2 .0 = 1 1 .hole : E = 1.0 = 1 1 . convexity .Figure 4—14 Body hole 1.0 = 1 2.2 = -1 2– 0= 2 1–0 = 1 1– 1=0 1– 0=1 1.

⇒This permits to split an image up into smaller pieces and obtain an overall answer by combining the – results of1(body) operations performed on the pieces Figure 4—16 1 2 2 2 1 E (a ∪ b ∪ c ∪ d ∪ e) = E (a ) + E (b) + E (c) + E (d ) + E (e) −[ E (a ∩ b ) + E (b ∩ c) + E (c ∩ d ) + E (d ∩ e)] 1-0 = 1 2-0 =2 3-0 =3 2-0 =2 = (1 + 2 + 2 + 2 + 1) − (1 + 2 + 3 + 2) = 0 right (body = 1. hole 1) 1-1 =0 convexities =2 36 .

Concavities : 4 Euler number = 1 iii. called iterative modification is useful because it allows us 37 . the new binary image determined by the iterative method can be used as input to another cycle of computation this process. Iterative modification Instead of adding up the outputs of Local operators in the previous section.concavities =2 Ex1) hand convexit concavit Figure 4—17 Body : 1 Hole : 0 1-0 =1 5-4 =1 Convexities : 5 .

( 1 – 1 = 0 ) + ( 1. E* = .0 ) = 1 Part pattern Current pattern 2 3 38 .to incrementally changes an image that is difficult to process into one that might succumb to the methods already discussed Euler Differential E* 2.

4 5 * E new = − E * 39 .

5. and edges Figure 5—2 40 . lines. Regions & image segmentation I.5.1 Thresholding Methods •Image segmentation : An image segmentation is the partition of an image into a set of non-overlapping regions whose union is the entire image Figure 5—1 Detection of discontinuities : points.

as shown below 41 . Line detection The next level of complexity involves the detection of lines in an image. Point detection The detection of isolated points in an image is straightforward If | R| > T when T is threshold. Consider the four masks.Figure 5—3 R = w1 z1 + w2 z 2 + ⋅ ⋅ ⋅ ⋅ ⋅ + wq z q = ∑ wi z i i =1 q A mark used for detecting isolated points Figure 5—4 i. it’s an isolated point ii.

Figure 5—5 • If at a certain point in the image. for j=2. iii. Edge detection The idea underlying most edge detection technologies is the computation of a local derivative operator Figure 5—6 42 .3. for example. if |R1|>| Rj| .4 the line is more likely associated with a horizontal line. |Ri|>|Rj| for all j≠i that point is said to be more likely associated with a line in the direction of mask i.

Gradient operator  ∂f   ∂x  ∇f =    ∂f   ∂y     ∂f   ∂f  ∇f =   +      ∂x   ∂y  2 2 Figure 5—7 Laplacian Operator ∇2 f = ∂2 f ∂2 f + ∂x 2 ∂y 2 ∇ 2 f = 4 z 5 − ( z 2 + z 4 + z 6 + z 8 ) in discrete form Figure 5—8 43 .

Thresholding • Thresholding is one of the most important approaches to image segmentation Figure 5—9 1 g ( x.Cross Section of ∇ 2 f ∇ 2 f = 4z5 − (z 2 + z4 + z6 + z8 ) II. y ) =  0 if f ( x. Basic Formulation Figure 5—10 44 . Region-oriented segmentation i. y ) < T otherwise III.

…….………n (a) (b) (c) ∪ Ri = R i =1 n (entire image region) is a connected region. which starts with a set of “seed” points and from these grows regions by appending to each seed Point those neighboring pixels that have similar properties 45 .2. i ≠j  R1 ∩ R5 = φ   R1 ∩ R2 = φ R ∩ R ≠ φ 1  1 (d) P( Ri ) = True for i=1.Let R represent the entire image region and Ri the partitioned region for n=1.……n Ri Ri ∩ R j = φ for all i & j .2. Region Growing by Pixel Aggregation “Region Growing” is a procedure that groups pixels or sub-regions into larger regions.n Figure 5—11 ii.2. The simplest of these approaches is pixel aggregation. i=1.

P(Ri) = TRUE that is. one approach for segmenting R is to subdivide it successively into smaller and smaller quadrant regions so that. the final partition likely would contain adjacent regions With identical properties. Region splitting and Merging Spitting Let R represent the entire image region and select a predicate P as discussed in the previous section For a square image. if P(Ri or R) = FALSE.Ex 1 Figure 5—12 iii. for any region Ri. this draw back may be remedied by allowing merging as well as splitting two adjacent regions Rj &Rk are merged only if P(Rj∪Rk)=TRUE Figure 5—14 46 . divide the image into quadrants Figure 5—13 Merging If only splitting were used.

b) stop when no further merging and splitting is possible Ex 2 Figure 5—16 P( R) = FALSE P( R1 ) = TRUE P( R2 ) + FALSE P( R3 ) + FALSE P( R4 ) + FALSE P( R21 ) = TRUE P( R34 ) = FALSE P( R43 ) = FALSE P( R23 ∪ R32 ∪ R41 ) + TRUE → Merging Figure 5—17 47 .• The preceding discussion may be summarized by the following procedure in which. we split into four disjointed quadrants any region Ri where P(Ri)= FALSE Figure 5—15 a) merge any adjacent regions Rj & Rk for which P(Rj∪Rk) =TRUE . at any step.

Figure 6—2 Linearity : Shift : α f 1( x. If the image system moves. f & g moves the same amount. y − b) 48 . y − b) → g ( x − a.6. y ) f ( x − a . y ) + β f 2( x. I. y ) → α g1( x. y ) + β g 2( x. Shift-Invariant Systems Spatial domain → Point Spread Function (PSF) Frequency domain → Modulation Transfer Function (MTF) Figure 6—1 • Defocused image ( g ) is a processed version of the focused image ( f ) Double the brightness of ideal image and the defocused image. Image Processing : Continuous images Linear.

y ) ∞ ∞ g ( x. y ) Shift invariant f ( x − a. y) =  0 otherwise This function is called unit impulse δ ( x. y ) ∞ ∞ g ( x.η )d ε dη −∞ −∞ → g = f ⊗h Linear α f 1( x. y ) ∞ ∞ g ( x. y) or Dirac delta function ∞ ∞ ∫ ∫ δ ( x. y − η ) + β f 2( x − ε . y ) Origin ∞ Only possible if f ( x. y ) → h ( x.η )d ε dη f ( x. y − η )h(ε . y − η )h(ε . y)dxdy = 1 −∞ −∞ 49 .η )d ε dη −∞ −∞ = α g1( x. y ) = ? g ( x. y ) → g ( x.II. y ) = and ∫∫ f ( x − ε . y − b)h(ε . y ) → h( x. y ) + β f 2 ( ∞ ∞ ) → h( ) 1 → g ( x. y − b) → h( x. y ) = h( x. y ) = ∫ ∫ [α f ( x − ε .η )d ε dη −∞ −∞ g ( x. y ) = ∫∫ f ( x − ε . y ) = Impulse response ∫∫ −∞ −∞ f ( x − a. y ) → g ( x. y −η )] ⋅ h(ε . y ) + β g 2( x. y ) → h( x. Convolution f ( x .

y ) : Impulse Response of the system Figure 6—3 k (ε . y − η )d ε dη f ( x − ε . y − η ) ⋅ h( x.η )dε dη = h( x. y )dε dη −∞ −∞ ∞ ∞ g ( x. y)dxdy = h(0. (a ⊗ b) ⊗ c = a ⊗ (b ⊗ c) 50 . y)d ε dη <= Let x − ε = α . β ) ⋅ b( x − α . y −η ) ⋅ b( x.η )δ ( x − ε . y) = ∫∫ f (ε . y ) = ∫∫ −∞ −∞ ∞ ∞ b⊗a = =∫ ∫ ∫ a( x − ε . y) −∞ −∞ => h( x. y − η = β −∞ −∞ ∞ ∞ ∫ a(α . y −η ) ⋅ h(ε . y − η ) : scaled delta function ∞ ∞ f ( x.η ) ⋅ δ ( x − ε . 0) −∞ −∞ and ∫ ∫ δ ( x − ε . y − β )dα d β = b ⊗ a −∞ −∞ Similarly.∞ ∞ Shifting Property : ∞ ∞ ∫ ∫ δ ( x. y)h( x.

η )d ε dη −∞ −∞ = e 2π i (ux +vy ) ∫ ∞ ∞ ∫e −2π i ( uε + vη ) −∞ −∞ = e 2π i (ux +vy ) ⋅ A(u. y ) = ∫ ∫e 2π i{u ( x −ε ) + v ( y −η )} ⋅ h(ε . y ) = e 2π i (ux +vy ) ∞ ∞ g ( x. v) IV.Cascade f ⊗ h1 ( f ⊗ h 1) ⊗ h 2 f → h1  h 2  → → => f ⊗ (h1 ⊗ h2) f → h1 ⊗ h2 → g III. Fourier Transform Represent the signal as an infinite weighted sum of an infinite number of sinusoids (u: angular frequency) F (u ) = ∫ Note: ∞ −∞ f ( x ) e−iux dx eik = cos k + i sin k i = −1 51 . MTF An Eigen function of a system is a function that is reproduced with at most a change in amplitude. jwt e → → A( w)e jwt → f ( x.η )d ε dη ⋅ h(ε .

v)e + i ( ux + vy ) dudv −∞ −∞ V.Arbitrary function => Single Analytic Expression Spatial Domain(x)=> Frequency Domain(u) (Frequency Spectrum F(u)) Inverse Fourier Transform (IFT) f (x) = 1 2π ∫ ∞ −∞ F ( u ) eiux du 1 f ( x. The Fourier Transform of Convolution Let Now. y) = 2 4π ∞ ∞ ∫ ∫ F (u. ∞ ∞ g = f ⊗h G (u ) = = = = ∫ ∫ g ( x) e − i 2π ux dx − i 2π ux −∞ −∞ ∞ ∞ ∫ ∫ f (τ ) h ( x − τ ) e ∫ ∫  f (τ ) e  ∫ − i 2π uτ dτ dx −∞ −∞ ∞ ∞ − i 2π u ( x −τ ) dτ   h ( x − τ ) e dx     ∞ −∞ −∞ ∞  f (τ ) e−i 2π uτ dτ    ∫  h ( x ') e −i 2π ux 'dx '   −∞ −∞ = F (u ) H (u ) Convolution in spatial domain Multiplication in frequency domain Spatial Domain (x) Frequency Domain(u) g = f ⊗ h ↔ G = FH g = fh ↔ G = F ⊗ H 52 .

So. we can find g(x) by Fourier transform Figure 6—4 Spatial Domain (x) Linearity Scaling c1 f ( x ) + c2 g ( x ) f ( ax ) Frequency Domain (u) c1 F ( u ) + c2G ( u ) 1 u F  a a e −i 2π ux0 F ( u ) f ( −u ) F ∗ ( −u ) F (u ) G (u ) Shifting Symmetry Conjugation Convolution Differentiation f ( x − x0 ) F ( x) f ∗ ( x) f ( x ) ⊗ g ( x) d n f ( x) dx n ( i 2π u ) n F (u ) 53 .

we can use ∞ ∞ lim ε →0 ∫ − i ( ux + vy ) dxdy . as can be seen by substituting x = 0 and y = 0 into e −i ( ux + vy ) . ε → 0 uε vε VII. ∫ δ (t )dt = u ( x) .VI. Convergence Factors and the Unit Impulse We want a smoothed function of f(x) g ( x) = f ( x) ∗ h ( x) Let us use a Gaussian kernel Figure 6—5 54 . What is the Fourier transform of the unit impulse? ∞ ∞ We have ∫ ∫ δ ( x. = 1 . • Generalized Functions and Unit Impulses The integral of the one-dimensional unit impulse is the unit step function. That is lim sin uε sin vε =1. for x ≺ 0 . 2 = 0 . using the sifting property of the unit impulse. or lim ∫ δε ( x. for x = 0. −∞ x where u ( x) = 1. y)e −∞ −∞ 1 ε → 0 2ε − iux ∫ e dx −ε ε 1 2ε − ∫ε e ε − ivy dy . y ) e −∞ −∞ − i ( ux + vy ) dxdy = 1 . for x ≻ 0. Alternatively.

v)  ∂x  55 . y )  ∴F   = 2π iu ⋅ F (u .T. y ) = 0 x →±∞ In that case. v)  ∂x   ∂f ( x . of ∞ ∞ ∂f ( x . y)e−2π iux ∞ −∞ ∞ ∞ + (2π iu ) ∫ f ( x. y ) ∂f and ? ∂x ∂x ∞  ∞ ∂f −2π ux  −2π vy ∂f −2π i (ux +vy ) e dxdy = ∫  ∫ e dx  ⋅ e dy ∫ ∫ ∂x ∂x −∞ −∞ −∞  −∞  ∂f −2π ux −2π iux ∞ −2π iux ∫ ∂x e dx = f ( x. ∞ ∫ −∞ 2π iu ∫ f ( x. y )  F  = 2π iv ⋅ F (u . y )e−2π i (ux +vy ) dxdy = (2π iu ) ⋅ F (u . lim f ( x. v) −∞ ∞  ∂f ( x . y ) ⋅e−2π iux dx −∞ ∞ We can’t proceed unless. Partial Derivatives What is the F. y)e −∞ − (−2π iu )−∞ f ( x.h (x) =  1 x2  exp  − 2  2πσ  2σ  1 Then 2  1  H ( u ) = exp  − ( 2π u ) σ 2   2  G (u ) = F (u ) H (u ) H(u) attenuates high frequencies in F(u) (Low-pass Filter)! VIII. y) ⋅e dx ∫ −∞ = f ( x.

no matter what their orientation is. • Circular symmetry Figure 6—6 Let us introduce polar coordinates in both spatial and frequency domain Figure 6—7 x = r cos φ u = ρ ⋅ cos α and and y = r sin φ v = ρ sin α ux + vy = r ⋅ ρ cos(φ − α ) 56 . • Rotational Symmetry and Isotropic operators Rotationally symmetric operators are particularly attractive because the treat image features in the same way.IX.

y)e−2π i (ux+ vy ) dxdy −∞ −∞ ∞∞ = ∫ ∫ fr (r )e −2π i ⋅r cos(φ −α ) ⋅ rdrd φ 0 0 Zeroth order Vessel function (Hankel Transform) => F ρ ( ρ ) = 2π ∫ −∞ ∞ ∞ fr (r ) J 0(2π r ρ )rdr fr (r ) = 2π ∫ f ρ ( ρ ) J 0(2π r ρ ) ρ d ρ 0 2π 1 where J 0( x) = 2π ∫e 0 − i ⋅ x cos(θ − w ) dθ X. v) = −∞ −∞ ∫ ∫ 2πσ = 1 1 − 2 e 1 x2 + y2 2 σ2 ⋅ e−2π i (ux +vy ) dxdy 2 2 ∞ 2πσ 2 ∫e −∞ 1 x  −   2 σ  e −2π iux dx ⋅ 2 1 2π σ ∞ ∫e 1 y  −   2 σ  ⋅ e−2π ivy dy 2 −∞ 1 2πσ ∞ ∫e 1 x  −   2 σ  ∞ e −2π iux dx = ∫e 1 x  −   2 σ  cos(2π ux)dx − i ⋅ ∫ e −∞ ∞ 1 x  −   2 σ  ⋅ sin(2π ux)dx −∞ −∞ 57 .T .∞ ∞ F { f ( x. y )} = ∫∫ f ( x. Blurring h ( x. ∞ ∞ H (u . y ) = 1 2πσ 2 − e 1 x2 + y2 2 σ2 ⇓ F .

And.A . y ) The cascade of the two systems is the identity system.   1 H '(u. y ) → h( x. we can use something like 58 . v) = min  . integral over symmetric region is zero. ∫e −∞ ∞ 1 x  −   2 σ  2 cos(2π ux)dx = 2 π a ⋅e − π 2u 2 a . A is the maximum gain.The second integral is odd so. F {G a u s s i a n } = G a u ssia n Figure 6—8: Low Pass Filtering XI. Restoration and Enhancement f ( x.  H (u . v) = =e 1 2π ⋅ σ ⋅ 2πσ ⋅ e 1 − (2π ) 2 ⋅u 2 ⋅σ 2 2 ⋅ 1 2π ⋅ σ ⋅ 2πσ ⋅ e 1 − (2π ) 2 ⋅v 2 ⋅σ 2 2 1 − (2π 2 )⋅( u 2 + v 2 ) ⋅σ 2 2 Which is also rotationally symmetric. Or more elegantly.a = 1 2σ 2 ∫e −∞ ∞ 1 x  −   2 σ  cos(2π ux)dx = 2π ⋅ σ ⋅ e 1 − (2π ) 2 ⋅u 2 ⋅σ 2 2 H (u . v )     Where. y ) → f ( x. y ) → h '( x.

0) ≥ φ aa ( x. y ) ∞ ∞ = ∫ ∫ a(ε − x. − y) = φ ab( x. y ) φ aa (0.η − y ) ⋅ b(ε .η ) ⋅ a(ε .0) = ∫ ∫ a(ε . v)2 + B 2 1 is the maximum gain.H '(u. y ) is defined as φ ab( x. y ) . H (u. y) and b( x. y) = b( x. y) → Symmetric. y )) φ aa (0. y ) = a ( x.η )d ε dη −∞ −∞ If a( x. y ) for all ( x.(φ ab(− x.η )dε dη = ∫ ∫ a −∞ −∞ −∞ −∞ ∞ ∞ ∞ ∞ 2 (ε . v ) is real. if H (u .η )dε dη 59 . v ) . v) = where H (u . Correlation Figure 6—9 Cross correlation of a( x. the result is called autocorrelation φ ab( x. y )* b( x. (2 B) XII.

y ) = {i ( x. y)} dxdy 2 −∞ −∞ i ( x . y ) + n ( x. y −η ) ⋅ i( x − α . y ) = i ( x. y ) = b ( x. y) ⋅ d ( x.η ) ⋅ h(α . y ) ⊗ h ( x.η )d ε dη × ∫ ∫ i( x − α . ∫ ∞ ∞ 2 ∫ o dxdy = ∞ ∞ ∞ ∞ ∫∫∫∫ ∞ ∞ h(ε . E = ∫ ∫ o ( x. y ) o ( x. y) − d ( x. y − β ) ⋅ h(α . y )dxdy o 2 ( x. y)} ∞ ∞ 2 = ∫ ∫ i( x − ε . β )dε dηdα d β ∫ ∫ i( x − ε .13 Optimal Filtering Figure 6—10 ∞ ∞ E= ∫ ∫ {o( x. 6. β )dα d β −∞ −∞ ∞ ∞ −∞ −∞ Therefore. y ) ⊗ h( x. y − β )dxdy x −α = A ⇒ x = A +α y−β = B⇒ y = B+β −∞ −∞ −∞ −∞ −∞ −∞ −∞ −∞ 60 . y ) ∞ ∞ So. y) + d 2 −∞ −∞ 2 ( x. y −η ) ⋅ h(ε . y) − 2o( x.XIII.

B − (η − β ) ⋅ i( A. B)dAdB −∞ −∞ = φ ii (ε − α .η − β ) (Autocorrelation) 61 .∞ ∞ ⇒ = ∫ ∫ i( A − (ε − α ).

Let the brightness at position x i. I. Thus is called sampling.7.e. be fix. Assume an image as a one-dimensional function for simplicity. Image Processing – Discrete Image Sampling Theorem To convert a continuous distribution of brightness to a digital image. Figure 7—1 Figure 7—2 62 . pixel value of pixel at x. it necessary to extract the brightness at each point arranged at a period.

fT ( x) = f ( x) ⋅ combT ( x) FT { fT ( x )} (v ) = FT { f ( x )} (v ) ⊗ FT {combT ( x )} (v ) and.Suppose a function that is composed by infinite numbers of Dirac’s delta functions assigned at an interval T. This is called comb function defined as follows. A simple digital image from f ( x) .e. convolution of a function and delta function is equal to original function itself. convolution of a function and comb function is an arrangement of duplication of the original function at a constant period. 63 .. i. denoted fT ( x) . FT {combT ( x )} (v ) = FT { fT ( x )} (v ) = 1 1 comb T (V ) T 1 1 FT { f ( x )} (v ) ⊗ comb T (V ) T That’s a convolution of comb function? We start from “convolution with delta function” f (t ) ∗ δ (t ) = ∫ ∞ −∞ f ( y ) ⋅ δ (t − y )dy = ∫ ∞ −∞ f (t ) ⋅ δ (t − t )dy = f (t ) i. Since a comb function is a sequence of the delta function arranged at constant period.e. is expressed as f ( x ) multiplied by the comb function combT ( x) .

However. of the original function f ( x ) at period 1 . no information of the brightness distribution of the original image is lost by sampling.T. 64 .The F. adjacent FT { f ( x )} ’s do not overlap.T. of fT ( x) . In this case. the original FT { f ( x )} can’t be separated and a faulty function will be extracted. T Figure 7—3 If the period of comb function in the frequency domain is sufficiently large. adjacent FT { f ( x )} (v)’s overlap.T. of the original function FT { f ( x )} . This effect is called aliasing.e. the F. which is the sampled version of f ( x ) at period T is an infinite sequence of duplicating of FT { fT ( x )} which is the F. if the internal of the comb function is the frequency domain is small. can be separated and extracted i. In this case.

sampling rate consequently.⇒ FT { fT ( x )} (v ) = FT { f ( x )} (v ) ⊗ FT {combT ( x )} (v ) Figure 7—4 Since the support of FT { f ( x )} is in the range between −Vc ~ Vc . 1 T denote the number of sampling per unit length. 65 .e. i. T is the sampling period. This theorem is called sampling theorem. the period has to be at least 2Vc to avoid overlapping of FT { f ( x )} ’s since. the original brightness distribution can be reconstructed by a sampled digital image of the sampling rate is more than twice the maximum frequency contained in the original distribution.

samples/second T M. v)  2   Phase Angle φ (u.n) ≜ Input Signal Amplitude (real or complex) at sample (m. y ) is real. −v) ⇒ Spectrum of D. n) T ≜ Sampling interval 1 fs ≜ Sampling rate (= ) . v ) 2 Power Spectrum P (u .F. 66 . is symmetric. v=0~N-1) Inv. v ) ⋅e m=0 n = 0 M −1 N −1 +2π i ( m n u+ v) M N (m=0~M-1.T. v) =  R 2 (u . v) = F (u . its F. v ) = F ∗ ( −u . v) = ∫ ∫ f ( x. v) + I 2 (u . v) = F ∗ (−u .Discrete Fourier Transform (DFT) F (u. v) R (u . component.T.N = Numbers of Spatial samples = Number of frequency samples (integer) • Magnitude F (u . n) or D. n) ⇒ Average of f (m. is conjugate complex F (u . v) = tan −1 1 I (u. − v ) F (u . v ) + I 2 (u .T. n=0~N-1) • f(m. y)e−2π i (ux +vy ) dxdy = 0 0 ∞∞ 1 MN M −1 N −1 m =0 n =0 ∑ ∑ f (m. If f ( x.C. D. v) • F (0. = ∑ ∑ F (u . n) ⋅e −2π i ( m n u+ v) M N (u=0~M-1. 0) = 1 MN M −1 N −1 m =0 n = 0 ∑ ∑ f (m.F. v) = R 2 (u .

y ) and F (u. v)e ux 0 vy 0 + ) M N Case : when u 0 = e x y i⋅ 2π ( + ) 2 2 M N .θ ) and F ( w. y )( −1) x + y ⇔ F (u − And similarly.v − ) 2 2 ⇒ f ( x. v)( −1)u + v 2 2 Rotation x = r ⋅ cos θ y = r ⋅ sin θ u = w ⋅ cos φ v = w ⋅ sin φ Then. v) become f (r . △u = 1 M ⋅ ∆x △v = 1 N ⋅ ∆y • Translation f ( x.• Relationship between samples in the spatial and frequency domains. y) ⋅ e − i ⋅2 π ( u0 v0 ⋅x + ⋅ y ) M N ⇔ F (u − u 0 . v − v 0 ) − i⋅ 2π ( f ( x − x 0. y − ) ⇒ F (u . f (x − M N . v0 = 2 2 = eiπ ( x + y ) = cos {π ( x + y)} = (−1) x + y M N . f ( x. φ ) 67 . y − y 0) ⇔ F (u.

y − n) f ⊗h ⇒ F ⋅H → Pr ove( H . v + N ) The inverse transform is also periodic. (where F (u. v ) Row transform column transform • Convolution f ( x.f (r . y) → 1-D F (x. n) ⋅ h( x − m. y ) by an angle θ 0 rotates F (u. v) → 1-D F (u . f ( x. y + N ) Separability F (u .) f ⋅h ⇒ F ⊗ H 68 . y) ⊗ h( x. v )e x − j 2π u M . y) = f ( x. v) = F (u + M . θ + θ 0 ) ⇔ F ( w. and vice versa. y ) = 1 MN M −1 N −1 m=0 n =0 ∑ ∑ f (m. φ + θ 0 ) ⇒ Rotating f ( x. y + N ) = f ( x + M . Periodicity F (u. y) = f ( x + M . v + N ) = F (u + M . v) by the same angle. v ) = = 1 M M −1 x =0 1 M M −1 − j 2π u x M ∑e x =0 ⋅ 1 N M −1 − j 2π v y N ∑e y =0 ∑ F (u . v) = 1 N M −1 − j 2π v y N ∑e y =0 ) f (x. v) = F (u.W .

Differential Operators Figure 8—1 69 . • Edges are curves in the image where rapid changes occur in brightness or in the spatial derivatives of brightness. • Edge & Edge detection Edges in Images An Edge is a boundary between two figures with relatively distinct grey-level properties. • Change of brightness can be occurred at • • • • Change of surface orientation Different surfaces One object occludes another A boundary between light and shadow falling on a single surface II. I.8.

y ) = B1 + ( B2 -B1 ) u( xsinθ − ycosθ + ρ ) If t > 0 B1 B2 ½(B1 + B2) t<0 t=0 t −∞ u (t ) = ∫ δ ( x )dx ∫ ∞ −∞ δ ( x)dx = 1 Figure 8—3 ∂E = sin θ ( B1 − B2 )δ ( x sin θ − y cosθ + ρ ) ∂x ∂E = − cosθ ( B1 − B2 )δ ( x sin θ − y cosθ + ρ ) ∂y 70 .Figure 8—2 xsinθ − ycosθ + ρ = 0 E ( x .

j ) – 2E( i . j ) Exx + Eyy = (Ei-1 j + Eij-1 + Ei-1 j + E i j+1 ) – 4Eij 71 . j ) – E( i . j ) + E( i + 1 .y) ∇2 E = Exx = ∂2 E ∂2E + ∂x 2 ∂y 2 ∂2 E = sin 2 θ ( B1 − B2 )δ '( x sin θ − y cos θ + ρ ) ∂x 2 ∂2E E yy = 2 = cos 2 θ ( B1 − B2 )δ '(a sin θ − y cos θ + ρ ) ∂y ∇ 2 E = ( B1 − B2 )δ ' ( x sin θ − y cos θ + ρ ) III.squared gradient  ∂E   ∂E  2   +   = ((B1 − B2 )δ ( x sin θ − y cosθ + ρ ))  ∂y   ∂x    2 2 Laplacian of E(x. j ) Ey = E( i . j+1 ) – E( i . • Discrete Approximations Discrete form of Laplacian Ex = E( i+1 . j ) Exx = E( i – 1 .

8.4 Local operators and noise 1 1+ H ' = −ρ 2 ⋅ φ nn -------------ⓐ φ bb ρ2 = u2 + v2 suppose that φ bb = Then H ' = − S2 ρ2 & φ nn = N 2 ρ 2S 2 -ⓐ S2 + ρ2N 2 at low frequency -ρ2 : Laplacian operation at high frequency Derivation of ⓐ S2 << ∞ small gain Noise can be reduced ! N2 Figure 8—4 H '' = ϕiα ϕbb + ϕ nb = ϕii ϕbb + ϕ bn + ϕ nb + ϕ nn ϕbb = ϕ bb + ϕ nn = 1+ 1 ϕ nn (= ϕbb 1 ) SNR H ' = H '' ⋅ ℑ(∇ 2 ) 72 .IV.

For example.0) = e 1 − σ 2 (u 2 +v 2 ) 2 73 . y ) = 1 2πσ 2 − x 2 + y2 e 2σ 2 H (u. if the optimal filter is a gaussian Figure 8—5 h( x .

y) varies smoothly Figure 9—2 Task • : separate r´(x.y) b(x. Lightness & color Figure 9—1 b´(x.y) • Note that r´(x.y) .y) = e´(x.y) = log r´(x.y) +log e´(x.y) =∇2r(x.y) ∇2b(x.9.y) = r(x.y) +e(x.y) log b´(x.y) is constant within a patch and sharp discontinuities at edges between patches and e´(x.y) Take the logarithm of image brightness b´(x.y) & e´(x.r´(x.y) +∇2e(x.y) Figure 9—3 74 .

y) 75 .y) = 0 • recover r(x.y) = ∇2b(x.y) Figure 9—5 ∇ 2 l ( x.• cut slow slope by a threshold Figure 9—4 T(∇2b(x.y) else t(x..G ( p) = − g (r ) = IFT : 1 ρ2 1 log( r ) + ρ 2π I(x. y )] Fourier Transform : ⇒ − ρ 2 L( ρ ) = T ( ρ ) 1 ⇒ L( ρ ) = − 2 T (e) ρ = G( ρ ) ⋅ T ( ρ ) where.y) = g(r) * t(x. y ) (l ( x. y ) = t ( x. y ) dxdy ℑ[∇ 2 l ( x. y )] = ℑ[t ( x.y) = ∇2r(x.y) |∇2b(x.y)| > T then t(x. y ) = ∫ ∫ ∇ 2 r ( x.

y) = reflectance e(x.y) Figure 9—7 76 .y) – I(x.y) = b(x.y) = r(x.Figure 9—6 I(x.

R1 : b1 re 37 = 1 1 = (e1 ≅ e2 ) . y )e2 ( x.1 b2 r2 e2 74 r1 26 = r2 52 .equation 9. y ) = r1 ( x. ⋅ ⋅ ⋅ e3 e4 e5 .b1 ( x. y ) b2 ( x.equation 9. y ) b3 b4 b5 .2 16 equations R2 : b2 r 28 = = 2  b5 112 r5 b2 38 r2 = = b3 95 r3 Least square error 11 variables 16 equations 0 ≤ reflectance ≤ 1 77 . = r3 = r4 = r5 . y)e1 ( x. y ) = r2 ( x.

10. Radiometry δp geometric stereo (using one camera) δA irradiance E = radiance L = δ2p δAδW solid angle = Area of a spherical surface patch / (radius of the sphere)2 Figure 10—1 1 ⋅ 4π R 2 2 Solid angle = = 2π R2 Figure 10—2 78 . Reflectance (using one camera) Map : photometric stereo I.

1) 1 + p 2 + q2 II. Image Formation Figure 10—4 79 .Figure 10—3 ˆ n= (− p.−q.

1 δI cosθ  S  The solid angle subtended by the lens 2 π α  2 Ω=   =   cos 3 α 2 4 z ( z / cosα ) α  π   cos α 2 2 Thus the power of the light originating on the patch and passing through the lens is δP = L ⋅ δOΩ cos θ where L is the radiance of the surface in the direction toward the Lens δp = LδO   cos 3 α cosθ 4 z π α  2 δP δO π  α  3 E= =L   cos α cosθ δI δI 4  z  where E is the irradiance of the image at the patch under consideration 2 substituting for δO δI . we finally obtain 80 .solid angle = δO cosθ ( z / cos α ) 2 SI cos α solid angle = ( S / cos α ) 2 δO cosα  z  =   --------.2.equation 10.

4   cos α 4 s  ∴E ∝ L

E=L

π α 

2

E = L⋅

π α 

2

4   cos α 4s

III.

Bidirectional reflectance distribution function (BRDF)

Figure 10—5

Figure 10—6

81

δE (θ i , φ i ) ; irradiance
The amount of light falling on the surface from the direction (θ i , φ i )

δ L (θ e , φ e ) : radiance

The brightness of the surface as seen from the direction (θ e , φ e ) f (θ i , φ i ;θ e , φ e ) = BRDF :

δL(θ e , φ e ) δE (θ i , φ i )

∫π∫

π

π

2 0

(θ i ,φ i ;θ e ,φ e ) sin θ e cosθ e dθ e dφ e

=1

IV.

Extended Light sources

Figure 10—7

solid angle δw

The.area = The area = sin θδθ i δφ i (dis tan ce) 2

82

(proof :

∫ ∫ sw = ∫π ∫ sin θ δθ δϕ
i i

π π 2

i

= 2π

0

E (θ i , φ i ) : Radiance per unit solid angle coming from the direction (θ i , φ i )

The radiance from the patch under consideration

E (θ i , φ i )δω = E (θ i , φ i ) sin θ i δθ i δφ i

Total irradiance of the surface is
π π 2

E0 =

∫ ∫ E (θ , ϕ ) sin θ cos θ dθ dϕ
i i i i i

i

−π 0

where cos θ i accounts for the foreshortening of the surface as seen from the direction (θ i , φ i ) The radiance of the surface,
L(θ e , φ e ) = ∫ ∫ f (θ i , φ i jθ e , φ e ) E (θ i , φ i )

sin θ i cosθ i dθ i dφ i

V.

Surface Reflectance properties
Ideal Lambertian surface One that appears equally bright from all viewing directions and reflects all incident light, absorbing none

83

jθ . φ . φ i jθ e .BRDF f (θ i . φ e ) constant π π 2 ∫ ∫ f (θ . φ i ) sin θ i cosθ i dθ i dφ i = f ⋅ Eo = 1 π ⋅ E o (∵ f = 1 π ) where the irradiance is Eo • Ideal specular Reflector Figure 10—8 84 . ϕ ) sin θ i i e e e cos θ e dθ e dϕe = 1 −π 0 π 2π f ∫ sin θ e cos θ e dθ e = 1 0 2 π f =1 ∴f = 1 π sine BRDF is constant for a Lambertian surface L = ∫ ∫ f ⋅ E (θ i .

φ i jθ e . ϕi ) ⋅ sin θ i ⋅ cos θ i dθ i d ϕi = E (θ e . ϕi ) L(θ e . ϕ ) sin θ i i e e e cos θ e dθ e d ϕe = 1 −π 0 ⇒ k sin θ i cos θ i = 1 ⇒k = 1 sin θ i cos θ i In this case. ϕ e − π ) = E (θ i . ϕ . φ e ) = δ (θ e − θ i )δ (θ e − φ i − π ) sin θ i cosθ i Determine the radiance of a specular reflecting surface under an extended source where the irradiance is Eo δ (θ e − θ i )δ (ϕe − ϕi − π ) sin θ i cos θ i − 0 E (θi .θ . f (θ i .π π 2 ∫ ∫ f (θ . Surface Brightness Figure 10—9 85 . ϕ e ) = ∫π ∫ π π 2 VI.

L= 1 π E cosθ i •Light source : a “sky” of uniform radiance E π π 2 L= E − ∫π ∫ π E ⋅ sin θ cosθ dθ dϕ i i i 0 1 i π π 2 = π ∫ ∫ sin θ cosθ dθ dϕ i i i i −π 0  − cos 2θ i  2 E =E  = 2 [1 + 1] = E 2  0 The radiance of the patch ≡The radiance of the source π VII. Surface orientation Figure 10—10 86 .

−q.0. qδ y ) T ˆ The unit surface normal n ˆ n= n (− p.− q.n = rx × ry = (− p.1) 1+ p2 + q2 (− p.1) T assuming α is very small ˆ n= (− p.−q .0.1) rx = (δ x .1) T = 1 1+ p2 + q2 87 .− q.1) 1+ p + q 2 2 ˆˆ cosθ = nv = ⋅ (0. pδ x )T ry = (0.0. δ y .1) T = n 1+ p2 + q2 Figure 10—11 ˆ v ≡ (0.

VIII. q ) ˆ s 2 → R2 ( p .q) .1 Figure 10—13 88 . q ) = 1 + ps p + qs q 1 + p 2 + q 2 1 + p s2 + q s2 = c .1) T 1+ p + q 2 s 2 s = 1 + ps p + qs q 2 1 + p 2 + q 2 1 + p x + q s2 = R(p.1) 1+ p + q 2 2 ⋅ (− p s .equation 10. q ) L= 1 π E cosθ i for θ i ≥ 0 ˆ ˆ cosθ i = n ⋅ s = (− p .−q s .9. Reflectance Map R ( p.−q . The reflectance Map Figure 10—12 ˆ s1 → R1 ( p.

q ) E = L⋅ πd 2 after normalizing E(x.y) = R(p.q) ρ : reflectance factor where 0 ≤ ρ ≤ 1 ˆ ˆ E ( x. y ) = s ⋅ n Ex1) Consider a sphere with a Lambertian surface illuminated by a point source at essentially the same place as the viewer in the case of cylinder 89 . q ) = R2 ( p .R1 ( p. Shading in image Consider a smoothly curved object the image of such an object will have spatial variations in brightness due to the fact that surface patches with different orientations appear with different brightness the variation of brightness is called shading   cos 4 α 4 f    EαLα cosθ i ≡ R( p . E ( x. q ) = 1 + p s p1 + q s q1 1 + p12 + q12 1 + p s2 + q s2 1 + ps p2 + qs q2 1 + p12 + q12 1 + p s2 + q s2 IX. y ) = p ⋅ ( s ⋅ n ) ˆ ˆ where ρ = 1.

q ) = n ⋅ s = = 1 + ps p + qs q 1 + p 2 + q 2 1 + Ps + q s 1 1+ p2 + q2 1 2 2 − d 1 x p = z = (r 2 − ( x 2 + y 2 )) 2 (−2 x) = − z − z0 dx 2 q= y( z − z 0 ) 2 dz =− dy z − z0 Finally.1) ˆ v = (0.−q. ˆ s = (0.y) = R(p. E(x.R( p .1) ˆ n= (− p.Figure 10—14 In this case.9.0.1) 1+ p2 + q2 ˆ ˆ From equation 10.q) 90 .1 .0.

y) maximum = 0 x2+y2 =r2 E(x.0) E( x. Photometric stereo Figure 10—15 91 .y)=(0.= = 1 1+ p2 + q2 1 1+ x2 + y2 ( z − z0 ) 2 x2 + y2 r2 = 1− (x.y) minimum = 0 X.

q) : linear and independent 1 + p1 p + q1 q r1 1 + p2 p + q2 q r2 R1 ( p.q) : nonlinear Figure 10—16 92 . q ) = R2 ( p . q ) = E 2 = R 2 ( p. q ) = 1 + p s1 p + q s1 q 1 + p 2 + q 2 1 + Ps1 + q s1 1 + ps2 p + qs2 q 1 + p 2 + q 2 1 + Ps 2 + q s 2 2 2 2 2 Ex2) R(p. q ) = then p = 2 ( E12 r1 − 1)q 2 − ( E 2 r2 − 1)q1 p1 q 2 − q1 p 2 q= 2 ( E2 r1 − 1) p1 − ( E12 r1 − 1) p 2 p1q 2 − q1 p 2 Ex3) R(p.E1 = R1 ( p.

−q i . Albedo : reflectance factor : ˆ ˆ E i = ρ ( s i ⋅ n) ˆ where si = (− p i .1) T 1 + p i2 + q i2 for i=1.−q. 3.XI. 2. ………… ˆ n= (− p. ρ At least 3 equations 93 .1)T 1+ p2 + q2 Figure 10—17 Unknown variables p. q.

11. I. Linear reflectance map Figure 11—2 94 . Reflectance Map : shape from shading Recovering Shape from Shading Figure 11—1 i. Picture One more information => strong constraint smoothness neighboring patches of the surface => similar orientation assumption of smoothness : strong constraint ii.

y) using above curve Figure 11—4 95 . y )) S can be recovered from E(x.tan θ 0 = b a cosθ 0 = a a +b 2 2 sin θ 0 = b a + b2 2 Figure 11—3 m(θ ) = p cosθ + q sin θ : directional derivative ⇒ m(θ 0 ) = cosθ 0 + q sin θ 0 = ap + bq a +b 2 2 = 1 a +b 2 2 f −1 ( E ( x.

Rotationally symmetric reflectance maps Figure 11—5 96 .δz = mδξ (small change a long the characteristic curve) m= δz 1 = f −1 ( E ( x. y )) 2 2 δξ a +b where x(ξ ) = x0 + ξ cosθ y (ξ ) = y 0 + ξ sin θ ξ z (ξ ) = z0 + ∫ m(θ )d ξ 0 = z0 + 1 ξ 2 a +b 2 ∫f 0 −1 ( E ( x. y ))d ξ iii.

tan θ s = cosθ s = sin θ s = q p p p + q2 q p + q2 2 2 The slope m(θ s ) = p cosθ s + q sin θ s = = p2 + q2 f −1 ( E ( x.1 δy = qδξ 97 . y ))δξ δx = pδξ . y )) The change in Z δz = mδξ = p2 + q2 ⋅ p 2 f −1 ( E ( x.equation 11.1. we could take a step of length 1 p + q2 2 δξ rather than δξ δz = p2 + q2 × p2 + q2 = ( p 2 + q 2 )δξ = f −1 ( E ( x. y))δξ δx = cosθ sδξ = δy = sin θ s δξ = p + q2 q p + q2 2 δξ δξ To simplify δz .

1. q = ∂x ∂x ∂x ∂x ∂y = 2p ∂2 z ∂2z + 2q ∂x∂y ∂x 2  Ex = 2( pr + qs ) f '( s )   E y = 2( ps + qt ) f '( s) . y) = f ( p 2 + q 2 ) Differentiate E ( x.We need to determine p&q at the new point in order to continue the solution E ( x. p = .2 ∂p ∂p ∂ 2 z ∂2 z δ p = δx+ = 2 δx+ δy ∂q ∂y ∂x ∂x∂y = rδx + sδy δq = sδx + tδy in our case δx pδξ & δy = qδξ δp = ( pr + qs) ⋅ δξ δq = ( ps + qt ) ⋅ δξ δp = From (2) Ex δξ 2f' Ey 2f' δq = δξ As δξ → 0 . we obtain the differential equations 98 .equation 11. r . y ) w. t x & y ∂E ∂E ∂S ∂S = ⋅ = f ' (S ) ∂x ∂S ∂x ∂x 2 2 where S = p + q Ex = ∂s ∂p ∂q ∂z ∂z = 2p + 2q .

y = q. z = p 2 + q 2 = f ɺ ɺ ɺ  Ey  Ex ɺ ɺ . y )) iv. w.q) = f(ap + bq) R(p.t ξ ɺɺ = p = ɺ x Ey Ex .q= p = 2f' 2f'  −1 ( E ( x. y)) (x. y.r.equation 11.q) = f(P2 + q2) R(p. ɺɺ = q = y ɺ 2f' 2f' ɺ z = f −1 ( E ( x. z.q) = arbitarary δz = pδx + qδy δp = ∂2 z ∂2 z δx + δy . General Case R(p.1. x = p. p & q) 5 Differential Eqs solvable one more time by differentiating ɺ ɺ x = p. y = q.3 ∂x∂y ∂x 2 = rδx + sδy δq = ∂2 z ∂2z δx + 2 δy ∂x∂y ∂y = sδx + tδy 99 .

4  Rp   δx  Let   =   ⋅ δξ  δy  R     q ɺ x = Rp ɺ y = Rq  R p   Ex   δp   δx  Then   = H ⋅   = H ⋅   =  δξ  δq   δy  R  E       q  y 100 .1.1.y) w. δp   r s  δx   δx    =   δq   s t  δy  = H ⋅  δy             where H is the Hessian Matrix r+t = ∂2 z ∂2 z : Laplacian + ∂x 2 ∂y 2 Differentiating E(x.3 & equation 11.equation 11.1.4 E     y  Rq  from equation 11.t x & y Ex = ∂E ∂E ∂p ∂E ∂q = ⋅ + ⋅ ∂x ∂p ∂x ∂q ∂x = Rp ⋅ ∂2 z ∂2z + Rq ⋅ S ∂x 2 ∂x∂y = R p ⋅ r + Rq ⋅ S E y = R p ⋅ S + Rq ⋅ t  Ex  R    = H ⋅  p  .r.

z.3 Singular points Figure 11—7 R( p . q ) < R( p 0 . q 0 ) Figure 11—8 101 .ɺ  x = Rp  ɺ ɺ  y = Rq .y. z = pR p + qRq   δz = p ⋅ δx + q ⋅ δy ɺ ɺ ɺ z = px + qy = pR p + qRq  δξ δξ δξ  p = E . q = E ɺ x y ɺ 5unknown variables : x. q 0 ) for all ( p. 11. q ) ≠ ( p 0 . p & q Figure 11—6 II.

3. 3 unknowns  2 2   E yy = b + c    102 .Rp = 0. q) = = 1 2 1 ( p + q 2 ) = [(ac + by) 2 + (bx + cy ) 2 ] 2 2 1 2 1 (a + b 2 ) x 2 + (a + c)bxy + (b 2 + c 2 ) y 2 2 2 The brightness gradient is E x = (a 2 + b 2 ) x + (a + c)by E y = (a + c)bx + (b 2 + c 2 ) y Differentiating again.3.1 R( p.q) = (0. q) = 1 2 ( p + q 2 ) . E = a 2 + b 2   xx   E xy = (a + c)b  3 equations.equation 11. Rq = 0 ɺ ɺ x = Rp = 0.0) – singular point z = z0 + 1 (ax 2 + 2bxy + cy 2 ) 2 Thus p = dz = ax + by dx dz = bx + cy & q= dy substituting p & q in equation 11.1 2 (p. y = Rq = 0 R( p.

g = 2q 1+ 1+ p2 + q2 conversely p= 4f 4g . Stereo graphic projection Figure 11—9  b f 2 + q2  = 2R R + a  2 2 b = ( p + q ) a R  2 2 2 a + b = R   solve the above equations f = 2p 1+ 1+ p2 + q2 .q = 2 2 4− f − g 4 − f 2 − g2 103 .III.

y ) − Rs ( f . λis large if brightness measurement are very noisy. λis small 104 .IV.g) Minimize e s = ∫ ∫ ( f 2 + fy 2 ) + ( gx 2 + gy 2 )dxdy A low.7 Relaxation methods characteristic strip expansion E(x . g )) 2 dxdy overall. 11. y) = R(p . minimize es + λei if brightness measurement are very accurate. Minimize ei = ∫ ∫ ( E ( x. q) = Rs(f.

105 . y ) − Rs ( f . g y )dxdy = It’s a problem in the calculus of variations. g )) ∂ ∂ F fx = ⋅ (2 f x ) = 2 f xx ∂x ∂x ∂ F f = 2 f yy ∂y y ∂Rs ∂f 2( f xx + f yy ) = ∂ ∂ Ffx + Ffy = F f ∂x ∂y ∂Rs ∂f ∂Rs ∂g (2 equations.minimize ∫∫ I F ( f . y) − Rs ( f . y) ⋅ Rs ( f .(see appendix) The corresponding Euler equations are ∂ ∂ Ffx − Ff y = 0 ∂y ∂x ∂ ∂ Fg − Fg x − Fg y = 0 ∂x ∂y Ff − where Ff is the partial derivative of F with respect to F F = ( f x + f y ) + ( g x + g y ) + λ ( E ( x. g )) Application to photometric stereo consider n images. y ) − Rs ( f . f x . f y . 2 unknowns(f. g )) ∇ 2 g = −λ ( E ( x. g .g)) = −λ ( E ( x. g )) 2 2 2 2 2 The Euler equations for this problem yield F f = −2λ ( E ( x. g x .

g )) 2 dxdy 2 2 2 i =1 n where Ei(x. g )) i =1 n V.8 Recovering Depth from a needle diagram Figure 11—10 106 . y ) − Ri ( f . 11. g )) i =1 n ∂Ri ∂f ∂Ri ∂g ∇ 2 g = −∑ λi ( Ei ( x.y) is the brightness measured in the i th image and Ri is the corresponding reflectance map The Euler equations for this problem yield ∇ 2 f = −∑ λi ( Ei ( x. y) − Ri ( f . y ) − Ri ( f .e = ∫ ∫ I (( fx 2 − f y ) + ( g x + g y ))dxdy + ∑ λi ∫ ∫ I ( Ei ( x.

y ) = z ( x0 . z y )dxdy The Euler equations is Fz − ∂ ∂ Fz x − Fz y = 0 ∂x ∂y 107 . z x .Figure 11—11 ( x. y 0) ( pdx + qdy ) Minimize the error ∫ ∫ (( z I x − p ) 2 + ( z p − p ) 2 )dxdy where p and q are the given estimates of the components of the gradient while z x & z y are the partial derivatives of the best-fit surface Minimize ∫∫ I F ( z . y ) z ( x. y0 ) + ∫ ( x0.

so that from F = ( z x − p ) 2 + ( z y − q ) 2 Fz = ∂ ∂ Fz x + Fz y ∂y ∂x ∂ ∂ = 2 ⋅ ( z x − p) + 2 ⋅ ( z y − q ) ∂x ∂y =0 2( z xx − p x ) + 2( z yy − q y ) = 0 z xx + z yy = p x + q y (∇ 2 z = p x + q y ) 108 .

Motion Field Figure 12—2 109 .12. Motion field & Optical flow Figure 12—1 • • Motion field : a purely geometric concept Optical flow : Motion of brightness patterns observed when a camera is moving relative to the objects being imaged I.

equation 12.1.1 Differentiate .1.equation 12.2 ˆ f' (r0 ⋅ z ) II. Optical flow Figure 12—3 110 .1 dr dr0 ˆ 0 ˆ d ri (r0 ⋅ z ) dt − ( dt ⋅ z )r 0 1 ⋅ = f ' dt ˆ (r0 ⋅ z ) 2 ˆ ˆ (r ⋅ z )v0 − (v0 ⋅ z )r0 = 0 2 ˆ (r 0 ⋅ z) ˆ (r × v ) × z 1 ⋅ vi = 0 0 2 .equation 12. &v i = δt δt 0 i ri r = 0 ˆ f ' r0 ⋅ z .v0 = δr δr .1.

t) ≡ E(x + uδt . y) & v(x. y. δy . y. y . y) are the x & y components of the optical flow vector For a small time interval δt .Ex1) Figure 12—4 E(x. t + δt ) = E ( x. y + vδt . v : two variables one equation : ⇒ one more constraint needed III. y + vδt . &t. αE αE αE E ( x + uδt . t ) αx αy αt where ρ contains second and higher order terms in δx . and δt ∂E δx ∂E δy ∂E δt ⋅ + ⋅ + ⋅ =0 ∂x δt ∂y δt ∂t δt E x u + E y v + Et = 0 (Optical flow constraint equation) u. Smoothness of the optical flow Usually. t + δt) where u(x. the motion field varies smoothly minimize a measure of departure from smoothness 111 . t ) + δx + δy + δt + ρ ≡ E ( x. if brightness varies smoothly with x. y.

u y . ec = ∫ ∫ [ E ( x. v y )dxdy 2 2 2 2 F = (u X + u Y ) + (v x + v y ) + λ ( E x u + E y v + Et ) 2 Euler equations are ∂ ∂ Fu x − Fu y = 0 ∂x ∂y ∂ ∂ Fv − Fvx − Fv y = 0 ∂x ∂y Fu − 112 . v x . Minimize e s + λec Where e s = ∫ ∫ ((ux 2 + uy 2 ) + (vx 2 + vy 2 ))dxdy ec = ∫ ∫ ( E x u + E y v + Et ) 2 dxdy λ: a parameter that weights the error in the image motion equation relative to the departure from smoothness λ=> Large => small Minimize ∫ ∫ F ( u . u x . y. minimize e s + λec . v. minimize the error in the optical flow constraint eq. t ) at + +δt − E ( x. y. t ) att ]2 dxdy = ∫ ∫ ( E x u + E y v + Et ) 2 dxd y Hence.e s = ∫ ∫ ((ux 2 + uy 2 ) + (vx 2 + vy 2 ))dxdy Also.

λ ( E x u + E y v + Et ) E y = ∇ 2 v E x u + E y v + Et = 0 IV. ˆ ( Fu x . Fv y ) ⋅ n = 0 113 . Fu y ) ⋅ n = 0 ˆ ( Fv x .−  is a unit vector  ds ds  T perpendicular to the boundary Rewriting the above conditions.Fu = 2λ ( E x u + E y v + Et ) E x ∂ ∂ Fu x = (2u x ) = 2u xx ∂x ∂x ∂ ∂ Fu y = (2u y ) = 2u yy ∂y ∂x λ ( E x u + E y v + Et ) E x = u xx + u yy = ∇ 2 u Similarly. Boundary conditions *Natural boundary conditions (see appendix) dy dx = Fu y ds ds dy dx Fv x = Fv y ds ds Fu x where S denotes arc length along the boundary curve  dy dx  ˆ n =  .

j − vi . j − ui . j ) 2 + (u i.r. u y ) ⋅ n = 0 ˆ (v x . j +1 − vi . j ) 2 ) 4 cij = ( E x u ij + E y vij + Et ) 2 Differentiating w. ∂u kl ∂e = 2(v kl − vkl ) + 2λ ( E x u kl + E y v kl + Et ) E y . j +1 − ui . Discrete case From e = es + λec 2 2 = ∫ ∫ ((u x + u 2 ) + (v x + v 2 ))dxdy + λ ∫ ∫ ( E x u + E y v + Et ) 2 dxdy y y e = ∑∑ ( sij + λ cij ) i j si . j = 1 ((u i +1. j ) 2 + (vi +1. F = (u x + u 2 ) + (v x + v 2 ) + λ ( E x u + E y v + Et ) 2 y y ˆ (u x .2 2 in our case.t u kl & v kl yields ∂e = 2(u kl − u kl ) + 2λ ( E x u kl + E y vkl + Et ) E x . ∂v kl where u & v are average of u & v 114 . j ) 2 + (vi . v y ) ⋅ n = 0 V.

∂e =0 ∂u kl ∂e =0 ∂v kl solve this 2 2 (1 + λ ( E x2 + E y ))u kl = +(1 + λE y )u kl − λE x E y vkl − λE x Et . 2 (1 + λ ( E x2 + E y ))v kl = −λE y E x u kl + (1 + λE x2 )vkl − λE y Et Iterative scheme is n +1 kl u =u − n kl n n E x u kl + E y vkl + Et 2 1 + λ ( E x2 + E y ) Ex . n n v kl+1 = vkl − n n E x u kl + E y vkl + Et 2 1 + λ ( E x2 + E y ) Figure 12—5 115 . Ey .

Figure 13—1 Disparity : xl' − xr' = D x+ z x− z ' r xl' = f x r' = f ' l b 2 ← (blue line) − (1) b 2 ← (red line) − (2) − (3) y y y = = f f z solve (1) (2) & (3) ( xl' − xr' ) = b f : disparity z (x' − x' ) / 2 (y' − y' ) / 2 bf x = b l ' r ' .z = ' xl − x r xl − x r xl − x r' 116 .13.y =b l ' r ' . Photogrammetry & Stereo I. Disparity between the two images.

z∝

1 1 = ' x − xr D
' l

II.

Photogrammetry

Figure 13—2

xr

absolute orientation

(rl , rr , known )  xr  y   r  zr   

r r = Rr l

+

r0

rotaional component  r11 r  21 r31  r12 r22 r32 r13  r23   r33  

translational component  r14  r   24   r34   

117

where R is a 3*3 orthonormal matrix RT ⋅ R = I
r11 xl + r12 y l + r13 z l + r14 = x r r21 xl + r22 y l + r23 z l + r24 = y r r31 xl + r32 y l + r33 z l + r34 = z r

3 equations => 12 unknowns

Orthonormality

RT ⋅ R = I
2 2 2 r11 + r12 + r13 = 1 2 2 2 r21 + r22 + r23 = 1 2 2 2 r31 + r32 + r33 = 1

r11r21 + r12 r22 + r13 r23 = 0 r21r31 + r22 r32 + r23 r33 = 0 r31r11 + r32 r12 + r33 r13 = 0

Figure 13—3

118

rl 2 − rl1 = rr 2 − rr1

3+7 = 10 equations 12 unknown variables

P&Q : 10+3 = 13 equations

Figure 13—4

r r = R ⋅ r l + r 0 - equation 13.3.1
 r11 R = r21   r31  r12 r22 r32 r13  r23   r33    r14  r0 = r24     r34   

III.

Relative orientation

r l , r r ; not known z l , z r unknowns The projection points ( xl' , y l' ) & ( x r' , y r' ) known determine the ratios of x and y to z using

119

equation 13.4.3.1 12 + 2*(n) substitute B into A r11 xl + r12 yl + r13 z l + r14 = xr z z z f f (r11 xl' l + r12 y ' l + r13 z l + r14 ) = ( x r' r ) zl f f f zl r11 xl' + r12 y l' + r13 f + r14 ⇒ r21 xl' + r22 y l' + r23 f + r24 z x f = x r' r = r zl zl s z y f = y r' r = r .. # of equations 7 from orthonormality & distance length 120 . # of unknowns for n points 7 + 3n .equation 13.xl' xl y' y = & l = l f zl f zl x r' xr y r' y = & = r f zr f zr .2 zl zl s z f = f r = zl zl r31 xl' + r32 y l' + r33 f + r34 12 + 2n .

constant f where s is a scaling factor from equation 13.2 x r = (r11 xl' + r12 yl' + r 13 f ) s + r14 = as + u y r = (r21 xl' + r22 yl' + r23 f )s + r 24 = bs + v z r = (r31 xl' + r32 yl' + r33 f ) s + r 34 = cs + w x r' x as + u = r = f z r cs + w y r' y bs + v = r = f z r cs + w 121 .5 Using a known relative orientation xl = xl' zl = xl/ ⋅ s yl = yl' ⋅ s z l = f ⋅ s f zl = s .1 0 0  2 r =1 R ⋅ R = I 0 1 0  i   r r = 0 if i ≠ j 0 0 1  i j   T ∴12 + 2n = 7 + 3n n=5 n ≥5 IV.3. 13.

multiply by xl' y' x' + r12 l + r13 ) z l + r14 = r z r f f f xl' y' y' + r22 l + r23 ) z l + r24 = r z r f f f xl' y' + r32 l + r33 ) z l + r34 = z r f f zl f (r11 (r21 (r31 From any two equations Compute z l & z r Then compute  x' y '  r l = ( xl .2. z r ) can be computed as follows Figure 13—5 From equation 13. ( x r' . ( xl' .1 z l  f f     x' y'  r l = ( x r . r .1 z r  f f    122 . y r .V. y l' ). l . yl . Computing depth Once IR . y r' ) known. r 0 . z r ) T =  r . The depth ( z l . z l ) T =  l .3.

7.1 # of equations = 2n +6 # of unknowns = 12 2n+6 = 12 => n=3 (at least we need three equations) 123 . 13.7 exterior orientation Figure 13—6 rc = R ⋅ra + r0 r11 x a + r12 y a + r13 z a + r14 = xc r21 xa + r22 y a + r23 z a + r24 = y c r31 xa + r32 y a + r33 z a + r34 = z c R T ⋅ R = I (orthonormality) x' xc y' y = & = c f zc f zc x' xc r11 x a + r12 y a + r13 z a + r14 = = f z c r31 x a + r32 y a + r33 z a + r34 y ' y c r21 xa + r22 y a + r23 z a + r24 = = f z c r31 x a + r32 y a + r33 z a + r34 .VI.equation 13.

n=6 124 . Interior Orientation • • • • scaling error translation error skewing error shearing error An affine transformation x' = a11 ( y' = a 21 ( y xc ) + a12 ( c ) + a13 zc zc xc y ) + a 22 ( c ) + a 23 zc zc from equation 13.VII.7.1 x' s11 x a + s12 y a + s13 z a + s14 = f s31 xa + s 32 y a + s33 z a + s 34 y ' s 21 x a + s 22 y a + s 23 z a + s 24 = f s 31 x a + s 32 y a + s 33 z a + s34 Rsij : not orthonormal Rsij ⋅ Rsij ≠ I 12 unknowns T #of equations = 2n ∴ 2n=12 .

y' ) E ~ El ( f r z z ' ' (∵ y l ≅ y r = y ) x+ Let x' x bf ≅ & d ( x' . y r' ) b b x− 2 . y l' ) = E r ( xr' . Finding conjugate points i. y' ) = E r ( x'− d ( x' . y' ) = f z z 1 1 El ( x'+ d ( x' . y' ) 2 2 criterion : minimize e = e s + λei where e s = ∫ ∫ (∇ ⋅ ∇d ) 2 dx' dy' ei = ∫ ∫ ( El − Er ) 2 dx' dy ' (solution) using Euler equation Fd − ∂ ∂ Fd 'x − Fd ' = 0 ∂x' ∂y' y where 125 . y ' ). y ' ).VIII. Gray-Level Matching • (reference figure 3-1) xl' = f x+ b b x− x r' 2& 2 = z f z at a matched point El ( xl' . y' ) = E ( f 2 .

y ' )]2 2 2 ∂E () 1 1 ∂E () Fd = 2λ[ El ( ) − E () − E r ()][ l ( ) − (− ) r ] 2 ∂x' ∂x' 2 1 ∂El ∂E r =2λ[ El () − E r ()] [ + ] 2 ∂x' ∂x' ∂ ∂ Fdx ' + Fdy ' = 2∇ 2 (∇ 2 d ) ∂x' ∂y' 1 ∂E () ∂E () ∵ ∇ 2 (∇ 2 d ) = λ[ El () − E r ()] [ l + r ] 2 ∂x' ∂x' 126 . y' ).1 1 F = (∇ ⋅ ∇d ) 2 + λ[ El ( x'+ d ( x' . y ' ). y ' ) − E r ( x'− d ( x' .

.I..14. random variables having some known a prior distribution (A) Maximum likelihood estimation Figure 14—1 Given . C) 1.. Goal : Estimate unknown model parameters from data Let λ i .. observations on a random variable X drawn from f x (x :θ ) 127 .... 1..2. i = 1. Model of system 2..D. pata : outcome of some type of probabilistic experiment. Bayes Decision Theory Fundamental statistical approach to the problem of pattern classification • Maximum likelihood estimation : parameters fixed but unknown • Baysian estimation : parameters..n be n I. Pattern classification Reference – “ Pattern Classification & Scene Analysis” Dudda & Hart (Wiley Interscience pub.

.. Compute the MLE of the mean μ <solution> The LF for h realizations of X is on a  1 L( X .....f...... the maximum of L(x ....d.....θ ) is called the likelihood function (LF) ˆ The maximum likelihood estimation (MLE) θ is that value that maximizes the LF.......θ ) ≥ L( x1 .. µ ) =   2  2πσ   exp − 1 2    2σ  n ∑ (x i =0 h i  − µ)2   since the log function is monotonic.The joint lpdf of X 1 . µ ) = − log(2πσ 2 ) − 2 2σ 2 ∑ (x i =1 n i − µ)2 Set ∂ log L( x. μ) is also that of log L(x ... N ( µ .. µ ) =0 ∂u 1 ∂2 ˆ ∑ ( xi − µ ) = 0 µ = i= 0 n This yields 1 n ∑ xi n i =1 Ex2) Consider the normal p.θ ) Ex1) Assume X ..x n .x n .x n .. σ 2 ) where σ is known...X n is f x1 Where L( x1 ..... μ) n 1 Hence log L( x. that is ˆ L( x1 . 128 ...... X 2 ..

σ 2 ) = − log 2π − n log σ − 2 2σ 2 Now set ∂L ∂L = 0.. =0 ∂µ ∂σ ∑ (x i =1 n i − µ)2 Obtain the simultaneous equations ∑ (x i =1 n i − µ) = 0 1 − n σ + σ3 ∑ (x i =1 n i − µ) 2 = 0 ˆ Hence µ = ˆ σ2 = 1 n ∑ Xi n i =1 1 n ˆ ∑ ( xi − µ ) 2 n i=1 (B) Baysian Estimation Suppose that an classifier to separate two kinds of lumber... σ 2 ) = 1 2πσ 2 exp(− 1 2σ 2 ∑ (x − µ) i =1 n 2 ) The log likelihood function is n 1 log L( x1 . ash and birch Figure 14—2 129 ..... µ .f x ( x.x 2 jµ .

p( w j | X ) = p( x | w j ) ⋅ p(w j ) p ( x) r j =1 where p ( x) = ∑ p ( x | w j ) ⋅ p ( w j ) Bayes Decision Rule for minimizing the probability of error . then 130 .A prior : • probability p(w1) : the next piece is ash • probability p(w2) : the next piece is birch Decision Rule : Decide w1 if p(w1) > p(w2) Decide w2 otherwise Bay’s Rule . p(w1)=p(w2). Decide w1 if p(w1|x) > p(w2|x) Decide w2 otherwise Decide w1 if P(x| w1)p(w1) > p(x| w2)p(w2) Decide w2 otherwise If equally likely.

Decide w1 if p(x| w1) > p(x| w2) Decide w2 if otherwise Figure 14—3 Probability error : p(error) ∞ p (error ) = = ∫ p(error . x)dx −∞ ∞ ∫ p(error | x) p( x)dx −∞  p ( w1 | x) if we decide w2 where p(error|x)   p ( w | x) if we decide w  2 1 Figure 14—4 131 .

.ws } be the finite set of S states of nature and A = {α 1 ...... then the overall risk will be minimized Bayes decision rule for the general form...v Then p ( w j | x) = s p( x | w j ) p( w j ) p( x ) Where p ( X ) = ∑ p ( x | w j ) p (w j ) j =1 The expected loss associated with taking action α i is R(α i | X ) = ∑ λ (α i | w j ) ⋅ p (w j | X ) j =1 s Target: to find a Bayes decision rule against p(wj) that minimizes the overall risk The overall risk is given by R = ∫ R(α ( x) | x) ⋅ p ( x)d x clearly.. 132 .. to minimize the overall risk. for every X . Let λ (α i | w j ) be the loss incurred for taking action α i when the state of nature is w j Let the feature vector X be a d-component vector-valued r...... if α (x) is chosen so that R(α ( x) | x) is as small as possible..α a ) be the finite set of possible actions.(B-1) Generalized Form (for Bayes classifier) Let Ω = {w1 .

2 Decide w2 otherwise If we substitute 1 into 2 We have λ11 p ( w1 | X ) + λ12 p ( w2 | X ) < λ 21 p ( w1 | X ) + λ 22 p (w2 | X ) ⇒ (λ 12 −λ 22 ) p ( w2 | X ) < (λ 21 − λ11 ) p (w1 | X ) Hence likelyhood ratio 133 .1. R(α 1 | X ) = λ11 p ( w1 | X ) + λ12 p ( w2 | X ) R(α 2 | X ) = λ 21 p (w1 | X ) + λ 22 p ( w2 | X ) .Compute the conditional risk R(α i | X ) = ∑ λ (α i | w j ) ⋅ p ( w j | X ) j =1 s for i=1.equation 14.……….a 2.Obtain the resulting minimum overall risk called the Bayes risk B.equation 14.2 Two category classification Let λ ij = λ (α i | w j ) the loss incurred for deciding wi when the true state of nature is wj.Select the action α i for which R(α i | X ) is minimum 3..1 The decision rule : Decide w1 if R(α 1 | X ) < R(α 2 | X ) .

j=1. for minimum error rate j Decide wi if p (wi | X ) > p ( w j | X ) for all j ≠ i. B-3 The multicategory classification 134 .Figure 14—5 <Minimum – error – rate classification> Symmetrical on zero-one loss function The loss λ (α i | w j ) = 0 i=j for i.……c 1 i≠j all errors are equally costly The conditional risk is R(α i | X ) = ∑ λ (α i | w j ) ⋅ p ( w j | X ) j =1 c = ∑ p( w j | X ) j≠i c = 1 − p ( wi | X ) To minimize the average prob of error. we should select the i that maximizes the a posterior prob p ( w i | X ) In other words.

R1. discriminant function The classification is said to assign a feature vector X to class w j If g i ( X ) > g j ( X ) for all j ≠ i g i ( X ) = p (wi | X ) . Divide the feature space into C division regions.………c . a posterior prob = p ( X | wi ) p (w j ) ∑ p( X | w ) p(w j j =1 c by Bayes rule j ) ⇒ p ( X | wi ) p ( wi ) ⇒ log p ( X | wi ) + log p ( wi ) Decision Rule 1.. If g i ( X ) > g j ( X ) for all j≠i then X is in RI 135 .………. for i=1.A pattern classifier Figure 14—6 g i ( X ).Rc 2.

In other words. assign X to wI Ex) three category classification Probability error calculation P(error Xo) = P(error X 0 ) ⋅ P( X 0 ) = P( X 0 w2 ) P( w2 ) + P( x0 w3 ) P( w3 ) Total error P(error) = ∫ P(error . x)dx −∞ ∞ 136 .

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->