remote sensing

© All Rights Reserved

43 views

remote sensing

© All Rights Reserved

- Lab2
- Sample Assignment Decision Making
- Green Plum
- Linear Gate Assignment: A Fast Statistical Mechanics Approach
- Matlab Tutorial
- All ECE Syllabus Final
- raster
- surat Andres E
- dope
- Lagrange-Newton Interpolation Yashoverdhan
- yuf.uy
- alg1_ch04_extrapractice
- Tutorial
- ShapeOptimisation
- foundations-of-algebra-standards
- zeros of the plant.pdf
- Math HL Calculator Sheet
- Engineering Mathematics Syllabus
- Grid Projector
- 1 Rotational Symmetry

You are on page 1of 48

Warping, Registration,

Morphing

Yao Wang

Polytechnic University, Brooklyn, NY 11201

With contribution from Zhu Liu, Onur Guleryuz, and

Partly based on

A K.

A.

K Jain,

Jain Fundamentals of Digital Image Processing

Lecture Outline

Introduction

Image deformation model

Image warping

Image registration

Image morphing

Geometric Transformation

So far, the image processing operations

we have discussed modify the color values

of pixels in a given image

With geometric transformation, we modify

the positions of pixels in a image, but keep

their colors unchanged

To create special effects

To register two images taken of the same

scene at different times

To morph one image to another

Geometric Transformation

Let (u, v) represent the image coordinate in an original

image, and (x, y) in a deformed (or warped) image. We

use a function pair to relate corresponding pixels in the

two images:

Forward mapping:

x x(u, v)

, or

y

(

u

,

v

)

Inverse mapping:

u u ( x, y )

, or u u (x)

v

(

x

,

y

)

x x(u)

g(x) the deformed image. Then they are related by:

g ( x, y ) f (u ( x, y ), v( x, y ))

g (x) f (u (x))

,

or

f

(

u

,

v

)

g

(

x

(

u

,

v

),

y

(

u

,

v

))

f (u) g ( x(u))

Geometric Transformation

Mapping Functions

y

v

Forward

x(u,v),

( ) y(u,v)

( )

(u, v)

v

u

Geometric Transformation

x

u

Inverse

u(x,y), v(x,y)

(x, y)

Translation

Translation is defined by the following

v

mapping functions:

x u tx

y v ty

and

u x tx

v y ty

In matrix notation

x u t, u x t

where

(tx,ty)

t x

x

u

x , u , t .

y

v

t y

Geometric Transformation

Scaling

Scaling is defined by

x sxu

and

y

s

v

y

u x / s x

v y / s y

v

sx=2,sy=1/2

Matrix notation

x Su, u S 1x

where

sx

S

0

0

s y

If sx < 1 and

d sy < 1,

1 thi

this represents

t a minification

i ifi ti

or shrinking, if sx >1 and sy > 1, it represents a

magnification or zoom.

zoom

Geometric Transformation

Rotation

Rotation by an angle of is defined by

x u cos v sin

i

y u sin v cos

and

i

u x cos y sin

v x sin y cos

In matrix format

v

=/4

x Ru, u R T x

where

cos

R

sin

sin

cos

(x,y)

(u,v)

Geometric Transformation

By Onur Guleyuz

Geometric Transformation

Geometric Transformation

A geometric transformation refers to a

combination of translation,

translation scaling,

scaling and

rotation, with a general form of

x RS(u t ) Au b,

u A 1 (x b) A 1x c,

with A RS, b RSt, c t.

operations will lead to different results.

Geometric Transformation

10

Affine Mapping

All possible geometric transformations are special cases

pp g

of the Affine Mapping:

x a0 a1u a2v

or x Au b

b

u

b

v

0

1

2

a a

A 1 2 ,

b1 b2

a

b 0

b0

matrix it corresponds to a

rotation matrix, and the corresponding affine mapping

reduces to a geometric mapping.

Geometric Transformation

11

Mapping straight lines to straight lines

The mapping between two arbitrary triangles

can be captured by an affine mapping

Affine mapping coefficients can be uniquely

determined from displacements of 3 vertices

v 3

y

2

1

2

x

12

from Vertex Correspondences ?

Go through in class

v 3

y

2

1

2

Geometric Transformation

13

Matlab Functions

two-dimensional affine transformation that maps each row of U

to the corresponding row of X

X. U and X are each 3

3-by-2

by 2 and

define the corners of input and output triangles. The corners

may not be collinear.

E ample

Example

------Create an affine transformation that maps the triangle with vertices

(0 0) (6

(0,0),

(6,3),

3) ((-2,5)

2 5) tto th

the ttriangle

i

l with

ith vertices

ti

((-1,-1),

1 1) (0

(0,-10),

10)

(4,4):

u = [ 0 6 -2]';

2]'

v = [ 0 3 5]';

x = [-1 0 4]';

y = [-1

[ 1 -10

10 4]'

4]';

tform = maketform('affine',[u v],[x y]);

Geometric Transformation

14

G = MAKETFORM('affine',T) builds a TFORM struct G for an Ndimensional affine transformation. T defines a forward

transformation such that TFORMFWD(U,T),

TFORMFWD(U T) where U is a 1

1-by-N

by N

vector, returns a 1-by-N vector X such that X = U * T(1:N,1:N) +

T(N+1,1:N).T has both forward and inverse transformations. N=2 for

2D image transformation

In MATLAB notation

a1

T a2

a0

Geometric

Geometric

Transformation

Transformation

b1

b2

b0

0

T

A

0 T

EL512EL512

ImageImage

Processing

Processing

15

15

according to the 2-D spatial transformation defined by TFORMB;

INTERP specifies the interpolation filter

Example 1

--------Apply a horizontal shear to an intensity image.

I = imread('cameraman.tif');

(

)

tform = maketform('affine',[1 0 0; .5 1 0; 0 0 1]);

J = imtransform(I,tform);

figure imshow(I),

figure,

imshow(I) figure

figure, imshow(J)

Show in class

Geometric Transformation

16

tform = maketform(

maketform('affine'

affine ,[1

[1 0 0; .5

5 1 0; 0 0 1]);

In MATLAB, affine transform is defined by:

[a1,b1,0;a2,b2,0;a0,b0,1]

With notation

t ti usedd in

i this

thi lecture

l t

t

note

1 0.5

A

,

0

1

0

b

0

Note in this example, first coordinate indicates horizontal position, second coordinate indicate vertic

Geometric Transformation

17

Bilinear Mapping

x a0 a1u a2v a3uv

quadrangle

More generally mapping a quadrangle to another

quadrangle.

8

8 parameters.

t

Can be completely determined from how the four

corners moved.

Geometric Transformation

18

coefficients from the vertex

correspondences?

3

v 3

y

2

2

Geometric Transformation

1 u

1

EL512 Image Processing

x

19

Non-chirping models

(Original)

(Affine)

Chirping models

(Bilinear)

(Projective)

(Relativeprojective)

(Pseudoperspective)

(Biquadratic)

Chirping: increasing perceived spatial frequency for far away objects

Converging (Keystone): parallel lines converge in distance

20

Polynomial Warping

The polynomial warping includes all deformations that

can be modeled by polynomial transformations:

x a0 a1u a2 v a3uv a4u 2 a5v 2

b5v 2

y

b

b

u

b

v

b

uv

b

u

0

1

2

3

4

Geometric Transformation

21

x(u, v), y(u, v).

F

Forward

d Mapping

M

i

For each point (u, v) in the original image, find the corresponding

position (x, y) in the deformed image by the forward mapping function,

and let g(x

g(x,y)=f(u,v).

y)=f(u v)

What if the mapped position (x,y) is not an integer sample in the desired

image?

Warping points

are often noninteger samples

Many integer

samples o

are not assigned

Values

Geometric Transformation

22

For each point (x, y) in the image to be obtained, find its

corresponding point (u, v) in the original image using the

inverse mapping function, and let g(x, y) = f(u, v).

What if the mapped point (u,v) is not an integer sample?

Interpolate from nearby integer samples!

P1

P P3

P2

P4

P will be interpolated

from P1, P2, P3, and P4

Geometric Transformation

23

Interpolation Method

Nearest neighbor:

Round (u

(u,v)

v) to the nearest integer samples

Bilinear interpolation:

find four integer samples nearest to (u,v),

apply bilinear interpolation

O

Other

h higher

hi h order

d iinterpolation

l i methods

h d

can also be used

Requiring more than 4 nearest integer

samples!

Geometric Transformation

24

Invert the forward mapping

If x Au b

Then u A 1 x b

correspondence

Geometric Transformation

25

ZI = INTERP2(X,Y,Z,XI,YI, METHOD) interpolates to find

ZI, the values of the underlying 2-D function Z at the

points in matrices XI and YI.

Matrices X and Y specify the points at which the data Z is given

given.

METHOD specifies interpolation filter

'linear'

linear - bilinear interpolation

'spline' - spline interpolation

'cubic' - bicubic interpolation as long as the data is uniformly spaced,

otherwise the same as 'spline'

Geometric Transformation

26

warping

Use inverse mapping

Step

p 1: For all p

possible p

pixels in output

p image

g ((x,y),

,y), find

corresponding points in the input image (u,v)

(X,Y)=meshgrid(1:M,1:N)

Apply inverse mapping function to find corresponding (u

(u,v),

v) for

every (x,y), store in (UI,VI)

Can use tforminv( ) function if you derived the transformation using maketform().

Or write your own code using the specified mapping

image at (UI,VI) from their values at regularly sampled

points (U,V)

(U,V)=meshgrid(1:M,1:N)

Outimg=interp2(U,V,inimg,UI,VI,linear);

Geometric Transformation

27

according to the 2-D spatial transformation defined by TFORM

INTERP specifies

ifi the

th interpolation

i t

l ti filter

filt

Example 1

--------Apply a horizontal shear to an intensity image.

I = imread(

imread('cameraman.tif');

cameraman.tif );

tform = maketform('affine',[1 0 0; .5 1 0; 0 0 1]);

J = imtransform(I,tform);

fi

figure,

i h (I) figure,

imshow(I),

fi

i h (J)

imshow(J)

Geometric Transformation

28

tform = maketform(

maketform('affine'

affine ,[1

[1 0 0; .5

5 1 0; 0 0 1]);

In MATLAB, affine transform is defined by:

[a1,b1,0;a2,b2,0;a0,b0,1]

With notation

t ti usedd in

i this

thi lecture

l t

t

note

1 0.5

A

,

0

1

0

b

0

Geometric Transformation

29

Two-Pass Mapping

The idea is to transform each row of f(u, v) first, to obtain

an intermediate image, f(x,y), and then each column of

this intermediate image is transformed to obtain the final

image g(x, y).

(u,v)

f(u,v)

xx=u

=u, y=y(u

y =y(u,v)

v)

i u

x cos sin

Ex: rotation

y sin cos v

Step 1: Fixed each row x=u, find y=usin+vcos

f( y)=f(u,v)

f(x,

) f( )

Step2: Fix each column y=y find x=ucos-vsin

y 'u sin y ' x' sin

cos

cos

u x'

Geometric Transformation

x x' cos

(u,y(u,v))

f(x,y)

x=x(x,y)=x(u,v)

y=y

(x(u,v),y(u,v))

sin

( y ' x' sin )

cos

cos

30

WAVE1

WAVE2

wave1:x(u,v)=u+20sin(2v/128);y(u,v)=v;

wave1:x(u,v)

u 20sin(2v/128);y(u,v) v;

wave2:x(u,v)=u+20sin(2u/30);y(u,v)=v.

By Onur Guleyuz

Geometric Transformation

31

WARP

SWIRL

WARP

SWIRL

y (u , v) (u x0 ) sin( ) (v y0 ) cos( ) y0 ;

r ((u x0 ) 2 (v y0 ) 2 )1/ 2 , r / 512.

By Onur Guleyuz

Geometric Transformation

32

Image Registration

Suppose we are given two images taken at

different times of the same object. To observe

the changes between these two images, we

need to make sure that they are aligned properly.

To obtain this goal, we need to find the correct

mapping function between the two. The

determination of the mapping functions between

two images is known as the registration problem.

Once the mapping function is determined

determined, the

alignment step can be accomplished using the

warping methods.

Geometric Transformation

33

Assume the mapping function is a polynomial of

order N

Step 1: Identify KN corresponding points

between two images,

g , i.e.

(ui , vi ) ( xi , yi ), i 1,2..., K .

0,,N-1 by solving

x(ui , vi ) a0 a1ui a2 vi xi ,

i 1,2,..., K

y (ui , vi ) b0 b1ui b2 vi yi ,

Geometric Transformation

34

Equations?

Convert to matrix equation:

Aa x, Ab y

where

1 u1

1 u

2

A

1 u K

v1

v2

vK

a0

b0

x1

y1

a

b

x

y

1

1

2

, b

, x , y 2

,a

a

b

x

K

yK

N 1

N 1

a A 1x, b A 1y

If K > N,

N then

th we can use a least

l

t square solution

l ti

a ( A T A) 1 A T x, b ( A T A) 1 A T y

If K < N,

N or A is singular

singular, then more corresponding feature points

must be identified.

Geometric Transformation

35

Examples

If we want to use an affine mapping to

register to images

images, we need to find 3 or

more pairs of corresponding points

If we have only 3 pairs

pairs, we can solve the

mapping parameters exactly as before

If we have more than 3 pairs, these pairs

may not all be related by an affine

mapping. We find the least squares fit by

solving an over-determined system of

equations

Geometric Transformation

36

Example

Geometric Transformation

37

TFORM=CP2TFORM(INPUT_POINTS,BASE_POINTS,TRANSFORM

TYPE)

returns

t

a TFORM structure

t t

containing

t i i a spatial

ti l transformation.

t

f

ti

INPUT_POINTS is an M-by-2 double matrix containing the X and Y

coordinates of control points in the image you want to transform.

BASE_POINTS is an M-by-2 double matrix containing the X and Y

coordinates of control points in the base image.

TRANSFORMTYPE can be 'nonreflective similarity', 'similarity',

'affine', 'projective', 'polynomial', 'piecewise linear' or 'lwm'.

Geometric Transformation

38

two images?

Which points to select in one image (image 1)?

Ideally choose interesting

interesting points:

points : corners, special

features

Can also use points over a regular grid

image (image 2)?

Put a small block around the point in image 1

Find a block in image 2 that matches the block

pattern the best

Exhaustive search within a certain range.

Geometric Transformation

39

Algorithm (EBMA)

Target frame

Rx

Anchor frame

Ry

dm

Bm

Best match

Search region

Bm

Current block

40

MATLAB Example

unregistered = imread('westconcordaerial.png');

figure, imshow(unregistered)

figure,

g

imshow('westconcordorthophoto.png')

(

p

p g)

load westconcordpoints % load some points that were already

picked

t concord = cp2tform(input_points,base_points,

t_concord

cp2tform(input points,base points,'projective');

projective );

info = imfinfo('westconcordorthophoto.png');

registered = imtransform(unregistered,t_concord,...

'XD t ' [1 info.Width],

'XData',[1

i f Width] 'YData',[1

'YD t ' [1 info.Height]);

i f H i ht])

figure, imshow(registered)

Geometric Transformation

41

Image Morphing

Image morphing has been widely used in

movies and commercials to create special visual

effects. For example, changing a beauty

gradually into a monster.

The fundamental techniques behind image

morphing is image warping.

Let the original image be f(u) and the final image

be g(x). In image warping, we create g(x) from

f( ) b

f(u)

by changing

h

i itits shape.

h

IIn iimage morphing,

hi

we use a combination of both f(u) and g(x) to

create a series of intermediate images

images.

Geometric Transformation

42

Cross

Dissolve

I(t) = (1-t)*S+t*T

Mesh

based

George Wolberg,

Wolberg Recent

Recent Advances in Image Morphing

Morphing ,

Computer Graphics Intl. '96, Pohang, Korea, June 1996.

Geometric Transformation

43

Suppose the mapping function between the two

end images is given as x

x=u+d(u).

u d(u). d(u) is the

displacement between corresponding points in

these two images.

In image morphing, we create a series of images,

starting with f(u) at k=0, and ending at g(x) at

k=K. The intermediate images are a linear

combination of the two end images:

hk (u sk d) (1 sk ) f (u) sk g (u d(u)),

)) k 0,1,..., K ,

where sk k / K .

Geometric Transformation

44

control poins

INPUT is the image that needs to be warped to bring it into the

coordinate system of the BASE image

image.

Example

cpselect('westconcordaerial.png','westconcordorthophoto.png')

Geometric Transformation

45

Demo

Show work by senior students

Geometric Transformation

46

Homework

You are given two pictures of the same scene, taken at different

times. In order to align the two pictures, you need to find a

mapping function between the two pictures based on some

common feature

f t

points.

i t Suppose

S

you were able

bl to

t extract

t tN

(N>=3) feature points in both images that correspond to the same

set of object features, with image coordinates given as (u_k,v_k)

and (x_k,y_k),k=1,2,,N. Also, suppose you want to use an affine

mapping

i tto approximate

i t th

the actual

t l unknown

k

mapping

i ffunction.

ti

How would you determine the affine mapping parameters?

Suppose you want make a panoramic picture of a wide landscape

p

y captured

p

p

pictures with a certain overlap.

p

out of two separately

Propose an algorithm for stitching up the two images to make a

panorama. (list the steps involved).

Computer assignment: Write a matlab program that implements

rotation of an image by a certain angle, and apply it on a selected

image. The rotation center should be the image center. Please

note that you should not use the imrotate() or imtransform()

function in MATLAB. You should write your own program, which

can call interp2().

interp2() .

Geometric Transformation

47

Reading

Prof. Yao Wangs Lecture Notes, Chapter

10.

10

R. Gonzalez, Digital Image Processing,

Section 5

5.11

11

George Wolberg, Digital Image Warping,

Wiley-IEEE Computer Society Press, 1990

Geometric Transformation

48

- Lab2Uploaded byWael Bazzi
- Sample Assignment Decision MakingUploaded byscholarsassist
- Green PlumUploaded bysarferazul_haque
- Linear Gate Assignment: A Fast Statistical Mechanics ApproachUploaded byal9028
- Matlab TutorialUploaded bySachin Khandare
- All ECE Syllabus FinalUploaded byKrishna Chaitanya T
- rasterUploaded byAlfredo Alessandrini
- surat Andres EUploaded byMochammad Ramzi
- dopeUploaded byArpan Chopra
- Lagrange-Newton Interpolation YashoverdhanUploaded byYashoverdhan Vyas
- yuf.uyUploaded bySumit Nema
- alg1_ch04_extrapracticeUploaded bywenatchee25
- ShapeOptimisationUploaded byamitavjha
- foundations-of-algebra-standardsUploaded byapi-291483144
- zeros of the plant.pdfUploaded byAnonymous rTRz30f
- Engineering Mathematics SyllabusUploaded byrjpatil19
- TutorialUploaded byendahmama
- Math HL Calculator SheetUploaded byAbdullah Al Sakib
- Grid ProjectorUploaded byArianto Fivers
- 1 Rotational SymmetryUploaded byDonnaHerman
- Assign1Tut1 Stat340 QuesUploaded byjmyim999
- 01583710Uploaded bySrikarPolasanapalli
- Matematik spm 2010Uploaded byDaliya
- Assignment Sheet for Portions of Chapter 5_2016 (1)Uploaded byBenjamin Brandt
- B. Tech II Sem SyllabusUploaded bykhananu
- s0013-4686(02)00677-1Uploaded bypomon666
- hsc-commerce-2014-october-maths1.pdfUploaded byShradha Rohan Bayas
- 10A2018 Kriging OK Lecture HW09 1120Uploaded byMuhammad Dhery Mahendra
- 78758_tocUploaded byKrishna Krrish007
- EE625_2.pdfUploaded byMadhusmita Barik

- Señales y sistemas Sung KimUploaded byBryan Cuervo Arias
- Dyke System ImplementationUploaded byJohn Gkanatsios
- CanThoResilienceactivitiesStratergy-15-30FUploaded byJohn Gkanatsios
- Can Tho Vietnam Resilient Cities ProgramUploaded byJohn Gkanatsios
- C Band Measuremetns of Sea Ice in GreenlandUploaded byJohn Gkanatsios
- Hyperspectral Imaging Carbon IdentificationUploaded byJohn Gkanatsios
- MSc Thesis GuidelinesUploaded byJohn Gkanatsios
- Automated Approach for Sea Ice ThicknessUploaded byJohn Gkanatsios
- sea Ice Classification MethodUploaded byJohn Gkanatsios
- Stp ManualUploaded byJohn Gkanatsios
- Slant to GroundUploaded byJohn Gkanatsios
- Integrated InSAR n LevellingUploaded byJohn Gkanatsios
- FFTandMatLab Wanjun HuangUploaded byDiego Purizaga
- A PhD is not enoughUploaded byAndrew Hall
- inSAR processing for DEM generation.pdfUploaded byJohn Gkanatsios
- Polarimetry for Ice DiscriminationUploaded byJohn Gkanatsios
- LeastSquares Fit in MatlabUploaded byJohn Gkanatsios
- Height Ambiguity_flat Earth_ ExmplesUploaded byJohn Gkanatsios
- GIToolsV2 TutorialUploaded byJohn Gkanatsios
- lecture12_ImageWarpingUploaded byJohn Gkanatsios
- asterUploaded byJohn Gkanatsios
- SarUploaded byJohn Gkanatsios
- SarUploaded byJohn Gkanatsios

- Case 9 Coyote Community CollegeUploaded byJOANNECHU
- jasmin malhotra- folio task 1Uploaded byapi-287665202
- Logica EMO BrochureUploaded bySanjoy Ghosh
- 3. Jay a. Farrell, Aided NavigationUploaded by서영빈
- PAP Statement on AO 2010-0022Uploaded byTinoRepaso
- 123Uploaded byAnonymous QYS50AT
- AbcdUploaded bykumarchaturvedula
- Assignment 1Uploaded byDiganta Hatibaruah
- Precast Manhole for Relocation Work of Digi TelecommunicationUploaded byAuYongTheanSeng
- DC3 trainUploaded bymano121
- Porter Intertextuality and Discourse CommunityUploaded byCjbrhet
- Beam Design Calc DataUploaded byRenjith Rajasekharan Nair
- Mass Effect Age 1.03Uploaded byRyan Adams
- Rolls Royce A Case Study by Wahid311Uploaded byAbdul Wahid Khan
- Radar Systems analasysUploaded byPoduguManikanta
- Engineering Spread Sheet via Oil Offshore MarineUploaded byjimhoops
- ABB Manual Motor ProtectorsUploaded byengazshams
- How to Use Adobe Pagemaker 7Uploaded bySoumya Ganesh
- Current Accounts in Lokhandwala.xlsxUploaded byprateek shah
- FinalUploaded byRashmi Joshi
- Novel Surfactant Technology to Improve Polyurethane SprayUploaded byธนพล กัตติยบุตร
- ucspUploaded byJan Cedric Mogol Hernandez
- pienemann 1989Uploaded byCharles Cornelius
- Lead (II) and Nickel (II) Adsorption Kinetics From Aqueous Metal Solutions PDFUploaded byEdidiong Asuquo
- cgj-2016-0318Uploaded byFrancolt41
- W04 PPT OscilloscopeUploaded bysmmaabz
- StrategicPlanningUPS Decision MakingUploaded byMattheus Biondi
- The Street, Apple, & General Systems TheoryUploaded byDavid Schneider
- NOTA RC 1Uploaded byshada aida
- 2012 IPE Product Resource GuideUploaded byBryan Nicoll

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.