Lecture 19

Image Processing,
Retrieval, and Analysis (I)

Prof. Christian Bauckhage
Outline
Lecture 19
Warping and Interpolation

Warping
Interpolation
Summary
goal
I understand and compute image transformations such as
warp function
⇔ a transformation T that alters the geometry of an image
warp function
⇔ a transformation T that alters the geometry of an image
in other words
I if g(u, v) = g(u) denotes the original image and
h(x, y) = h(x) denotes the warped image and if

x T (u, v)
= x or simply x = T(u)
y Ty (u, v)
then

h(x) = h T(u) = g(u)
a picture says a thousand words . . .
y
v
u x

function g(u) warped function h T(u) = h(x)
forward (PUSH) warps

⇔ the following procedure
I iterate over the pixels in the source image g(u)

I map their intensities or colors to corresponding
pixels in the destination image h(x)
forward (PUSH) warps

I iterate over the pixels in the source image g(u)

I map their intensities or colors to corresponding
pixels in the destination image h(x)
I that is
I read intensity or color at source coordinate (u, v)
in the array representing image g
I compute x = T(u)
I set intensity or color at destination coordinate (x, y)
in the array representing image h
example
I horizontal scaling by a factor of 4
for v in range(rmax): # iterate over rows
for u in range(cmax): # iterate over columns
x = 4.0 * u # compute x = T_x(u,v)
y = v # compute y = T_y(u,v)
if x < cmax:
h[y,x] = g[v,u]
T
example (cont.)
source image g(u) destination image h(x)

problem
I method leapfrogs coordinates in the destination image
problem
I for instance, for an arbitrary row r, we have
h(0, r) → g(0, r)
h(4, r) → g(1, r)
h(8, r) → g(2, r)
..
.
I but then what about h(1, r), h(2, r), h(3, r), . . . ?
problem
I for instance, for an arbitrary row r, we have
h(0, r) → g(0, r)
h(4, r) → g(1, r)
h(8, r) → g(2, r)
..
.
I but then what about h(1, r), h(2, r), h(3, r), . . . ?
solution . . .
backward (PULL) warps

I exploit the fact that
u = T −1 (x)
I iterate over the pixels in the destination image and read

corresponding intensity or color values from the source
image
backward (PULL) warps

I exploit the fact that
u = T −1 (x)
I iterate over the pixels in the destination image and read

corresponding intensity or color values from the source
image
I that is
I for each coordinate (x, y) in the array representing h,
compute the corresponding original coordinate (u, v)
I read intensity or color from the array representing g
example
I horizontal scaling by a factor of 4
for y in range(rmax): # iterate over rows
for x in range(cmax): # iterate over columns
u = x / 4.0 # compute u = T_xˆ-1(x,y)
v = y # compute v = T_yˆ-1(x,y)
h[y,x] = resample(g, v, u)
−1
T
example (cont.)
source image g(u) destination image h(x)

note:
I this procedure does not leapfrog coordinates in h
I it guarantees that, for each pixel in h, an intensity

or color is read from the original function g . . .
note:

I . . . however, this value may have to be read from a

location in between the discrete coordinates of g
note:

I . . . however, this value may have to be read from a

location in between the discrete coordinates of g
I this necessitates interpolation

interpolation of 1D functions
I given: a discrete 1D function f that assumes values f [xi ]
at sample points xi
I desired: an approximation f̂ (x) of the values at points x

in between consecutive sample points, i.e.
xi 6 x 6 xi+1
interpolation of 1D functions
I given: a discrete 1D function f that assumes values f [xi ]
at sample points xi
I desired: an approximation f̂ (x) of the values at points x

in between consecutive sample points, i.e.
xi 6 x 6 xi+1
note:
I in general, the sample points need not be equidistant

I for digital images, however, they are!
example
150 150
f (x)
100 f [xi ] 100
50 50
0 0
0 2 4 6 8 10 0 2 4 6 8 10
-50 -50
-100 -100
f (x) = sin(2x) · x2 f [x] = f (x) sampled

nearest-neighbor interpolation
I round x to closest sample point

f [xi ], if | x − xi | 6 | x − xi+1 |
f̂ (x) =
f [xi+1 ], otherwise
⇔ f̂ (x) = f [xk ], where xk = argmin | x − xj |

xj ∈{xi ,xi+1 }
150
f (x)
100 fˆ(x)
f [xi ]
50
0
0 2 4 6 8 10
-50
-100
nn-interpolation
linear interpolation
I assume f̂ to be piecewise linear

I the theorem of intersecting lines
implies that
f̂ (x) = f [xi ] + δ
xi x xi+1
where δ results from

f [xi+1 ] − f [xi ]
δ f [xi+1 ] − f [xi ] δ
= x − xi xi+1 − xi
x − xi xi+1 − xi xi x xi+1 xi x xi+1
f [xi+1 ] − f [xi ]
⇔ δ = (x − xi ) ·
xi+1 − xi
I in python, we may implement this as a function
def linearInterpol1D (a, fa, b, fb, x, eps=0.00001):
if abs(x-a) < eps or abs(a-b) < eps:
return fa
else
return fa + (x-a)*(fb-fa)/(b-a)
150
f (x)
100 fˆ(x)
f [xi ]
50
0
0 2 4 6 8 10
-50
-100
problem
I both methods are rather imprecise
problem
I i.e the interpolation error

Z xi+1

E= f (x) − f̂ (x) dx
xi
between two sample points xi and xi+1 is fairly “large”

problem
I i.e the interpolation error

Z xi+1

E= f (x) − f̂ (x) dx
xi
between two sample points xi and xi+1 is fairly “large”
solution
I interpolation using higher order polynomials
I next , we discuss two methods out of a many
possible approaches
interpolation using Lagrange polynomials

I given: n + 1 sample points xi and function values f [xi ]
I idea: approximate f̂ (x) as a linear combination of

Lagrange polynomials
X
n X
n Y x − xk
f̂ (x) = f [xi ] Li (x) = f [xi ]
xi − xk
i=0 i=0 k6=i
interpolation using Lagrange polynomials

I given two consecutive sample points x0 and x1 , it follows that:
X
1 Y x − xk
f̂ (x) = f [xi ]
i=0 i6=k
xi − xk
x − x1 x − x0
= f [x0 ] + f [x1 ]
x0 − x1 x1 − x0
−1 x1 − x x − x0
= f [x0 ] + f [x1 ]
−1 x1 − x0 x1 − x0

1
= f [x0 ](x1 − x) + f [x1 ](x − x0 )
x1 − x0

1
= f [x0 ](x1 − x0 − x + x0 ) + f [x1 ](x − x0 )
x1 − x0

1
= f [x0 ](x1 − x0 ) + f [x1 ](x − x0 ) − f [x0 ](x − x0 )
x1 − x0
f [x1 ] − f [x0 ]
= f [x0 ] + (x − x0 ) = f [x0 ] + δ
x1 − x0
⇒ linear interpolation is a special case of Lagrange interpolation
problem
I the function f̂ (x) resulting from L-interpolation of
n + 1 sample points is a polynomial of degree n
I therefore, at the upper 150

f (x)
and lower limit of its
100 fˆ(x)
domain, f̂ (x) tends
f [xi ]
to oscillate and 50
overshoot
0
0 2 4 6 8 10
-50
-100
L-interpolation
cubic spline interpolation

I idea: piecewise approximation of f̂ (x) using cubic
polynomials φi (x), i.e.


 φ1 (x), if x1 6 x 6 x2


φ2 (x), if x2 6 x 6 x3
f̂ (x) = .

 ..



φn−1 (x), if xn−1 6 x 6 xn
where
φi (x) = ai (x − xi )3 + bi (x − xi )2 + ci (x − xi ) + di
for i = 1, 2, . . . , n − 1
φi+1(x)
φi (x)
φi−1 (x)
xi−1 xi xi+1 xi+2

150
f (x)
100 fˆ(x)
f [xi ]
50
0
0 2 4 6 8 10
-50
-100
Question:
I how to determine the coefficients in
φi (x) = ai (x − xi )3 + bi (x − xi )2 + ci (x − xi ) + di ?
Answer:
I for the first and second derivative of each φi (x) we have
φi0 (x) = 3ai (x − xi )2 + 2bi (x − xi ) + ci

φi00 (x) = 6ai (x − xi ) + 2bi
..
.
Answer (cont.):
I after some straightforward algebra
6
Mi + 4Mi+1 + Mi+2 = f [xi ] − 2f [xi+1 ] + f [x i+2 ]
h2
for i = 1, 2, . . . , n − 2
I in this system of equations, the Mi only depend on the

known f [x1 ]
note:
I since we only dispose of n − 2 equation but want to

solve for n unknowns, the system is under determined
note:

I there are several possible solutions

note:

I there are several possible solutions
I the simplest: natural splines ⇔ M1 = Mn = 0

Answer (cont.):
I this leads to a solvable system of linear equations that can
be written in form of a matrix equation
4 1 0 ··· 0 0 0 f [x1 ] − 2f [x2 ] + f [x3 ]

    
M2
1 4 1 ··· 0 0 0  M3   f [x2 ] − 2f [x3 ] + f [x4 ] 
···
    
0 1 4 0 0 0  M4   f [x3 ] − 2f [x4 ] + f [x5 ] 
. .. .. .. .. ..   ..  = 6  ..
    
 .. .. 
 . . . . . .
 .  2
 h 
 . 

0
 0 0 ··· 4 1 0 Mn−3 
  
f [xn−4 ] − 2f [xn−3 ] + f [xn−2 ]
 
0 0 0 ··· 1 4 1 Mn−2  f [x ] − 2f [x ] + f [x ]
n−3 n−2 n−1
0 0 0 ··· 0 1 4 Mn−1 f [xn−2 ] − 2f [xn−1 ] + f [xn ]
Answer (cont.):
I this leads to a solvable system of linear equations that can
be written in form of a matrix equation
4 1 0 ··· 0 0 0 f [x1 ] − 2f [x2 ] + f [x3 ]

    
M2
1 4 1 ··· 0 0 0  M3   f [x2 ] − 2f [x3 ] + f [x4 ] 
···
    
0 1 4 0 0 0  M4   f [x3 ] − 2f [x4 ] + f [x5 ] 
. .. .. .. .. ..   ..  = 6  ..
    
 .. .. 
 . . . . . .
 .  2
 h 
 . 

0
 0 0 ··· 4 1 0 Mn−3 
  
f [xn−4 ] − 2f [xn−3 ] + f [xn−2 ]
 
0 0 0 ··· 1 4 1 Mn−2  f [x ] − 2f [x ] + f [x ]
n−3 n−2 n−1
0 0 0 ··· 0 1 4 Mn−1 f [xn−2 ] − 2f [xn−1 ] + f [xn ]
I this can easily be solved using linear algebra libraries

Summary
we now know about

I PUSH and PULL warps
I nearest-neighbor interpolation of 1D functions
I linear interpolation of 1D functions
I cubic spline interpolation of 1D functions

Lecture 19

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lecture 19

Uploaded by

Copyright:

Available Formats

Image Processing,

Retrieval, and Analysis (I)

Warping and Interpolation

a picture says a thousand words . . .

forward (PUSH) warps

I iterate over the pixels in the source image g(u)

forward (PUSH) warps

I iterate over the pixels in the source image g(u)

source image g(u) destination image h(x)

I for instance, for an arbitrary row r, we have

I for instance, for an arbitrary row r, we have

backward (PULL) warps

I exploit the fact that

I iterate over the pixels in the destination image and read

backward (PULL) warps

I exploit the fact that

I iterate over the pixels in the destination image and read

source image g(u) destination image h(x)

I this procedure does not leapfrog coordinates in h

I it guarantees that, for each pixel in h, an intensity

I this procedure does not leapfrog coordinates in h

I it guarantees that, for each pixel in h, an intensity

I . . . however, this value may have to be read from a

I this procedure does not leapfrog coordinates in h

I it guarantees that, for each pixel in h, an intensity

I . . . however, this value may have to be read from a

I this necessitates interpolation

I desired: an approximation f̂ (x) of the values at points x

I desired: an approximation f̂ (x) of the values at points x

I in general, the sample points need not be equidistant

f (x) = sin(2x) · x2 f [x] = f (x) sampled

⇔ f̂ (x) = f [xk ], where xk = argmin | x − xj |

I assume f̂ to be piecewise linear

where δ results from

x − xi xi+1 − xi xi x xi+1 xi x xi+1

I i.e the interpolation error

between two sample points xi and xi+1 is fairly “large”

I i.e the interpolation error

between two sample points xi and xi+1 is fairly “large”

interpolation using Lagrange polynomials

I idea: approximate f̂ (x) as a linear combination of

interpolation using Lagrange polynomials

I therefore, at the upper 150

cubic spline interpolation

cubic spline interpolation

xi−1 xi xi+1 xi+2

cubic spline interpolation

φi0 (x) = 3ai (x − xi )2 + 2bi (x − xi ) + ci

I in this system of equations, the Mi only depend on the

I since we only dispose of n − 2 equation but want to

I since we only dispose of n − 2 equation but want to

I there are several possible solutions

I since we only dispose of n − 2 equation but want to

I there are several possible solutions

I the simplest: natural splines ⇔ M1 = Mn = 0

4 1 0 ··· 0 0 0 f [x1 ] − 2f [x2 ] + f [x3 ]

4 1 0 ··· 0 0 0 f [x1 ] − 2f [x2 ] + f [x3 ]

I this can easily be solved using linear algebra libraries

we now know about

I nearest-neighbor interpolation of 1D functions

I linear interpolation of 1D functions

I cubic spline interpolation of 1D functions