Image

 Registra,on  based  on  

feature  matching  
Iván  F.  Mondragón  B.  
T.A.V.C  
2011  
 Summary  
•  Introduc,on  
•  Projec,on  Models  
•  Projec,on  Model  es,ma,on  based  on  
features  
•  Bad  Correspondences  
•  RANSAC  
•  Image  resampling  
– Applica,ons  
 Introduc,on  
•  We  took  a  set  of  images  of  a  nice  landscape  
and  we  want  to  made  a  Panorama    

1                                                                2                              3           4  

5         6  
 Introduc,on  
•  We  first  select  a  First  Image  and  then  we  
manually  adjust  the  others  to  made  the  
panorama:  

So  we  are  making  a  manual  Image  Registra,on  

 Image  Registra,on  
•  Image   registra,on:   is   the   process   of  
transforming   different   sets   of   data   into   one  
coordinate   system.   Data   may   be   mul,ple  
photographs,   data   from   different   sensors,  
from   different   ,mes,   or   from   different  
viewpoints.  
•  It   geometrically   aligns   two   or   more   images,  
the  reference  and  the  sensed  images.  
 Image  Registra,on  
San  Francisco  Bay.  
bathymetry  data  
•  Used  for:  
•  Change  Detec,on  
•  Image  Mosaicing  
•  Super  Resolu,on  
•  Geographic  
informa,on  Systems  
(GIS)  
•  Cartography  
•  Medicine  
 Image  Registra,on  
•  Classifica,on:  
– Intensity  base  method:    compare  intensity  
paZerns  in  images  via  correla,on  metrics  
registering  en,re  mage  or  subimages.  
– Feature  based  method:  find  correspondence  
between  image  features  such  as  interest  points  
and  descriptors,  establishing  point-­‐by-­‐point  
correspondence  between  the  reference  and  target  
images  
 Feature  based  Image  Registra,on  
•  4  steps  process:  
1.  Feature  Detec,on:  Salient  and  dis,nc,ve  corners  are  
automa,cally  detected.  
2.  Feature  Matching:  correspondences  between  the  
features  detected  in  the  sensed  image  and  the  
reference  image  is  established.  
3.  Transform  model  es,ma,on:  is  the  mapping  
func,on  that  aligns  the  sensed  image  with  the  
reference  
4.  Image  resampling  and  transforma,on:  Sensed  image  
is  transformed  by  means  of  the  mapping  func,on    
 Summary  
•  Introduc,on  
•  Projec,on  Models  
•  Projec,on  Model  es,ma,on  based  on  
features  
•  Bad  Correspondences  
•  RANSAC  
•  Image  resampling  
– Applica,ons  
 Transform  Model  

• Linear   transforma,ons   models     are   used   to   relate   the   target  

image   space   to   the   reference   image   space.   Included   are  
transla,on,   rota,on,   scaling,   general   affine   and   projec,ve  
transforms.    
• Linear   transforma,ons   are   global   in   nature,   thus,   they   cannot  
model  local  geometric  differences  between  images  
 Euclidean  and  Similarity  
Inhomogeneous coordinates (Euclidean)
⎡ xT ⎤ ⎡cos θ − sin θ ⎤⎡ x R ⎤ ⎡ t x ⎤
•  Euclidean  involve  a  rota,on  and   ⎢ y ⎥ = ⎢ sin θ cos θ ⎥⎢ y ⎥ + ⎢ t ⎥
⎣ T⎦ ⎣ ⎦⎣ R ⎦ ⎣ y ⎦
plane  transla,on  
•  Similarity  involve  a  scaled  
rota,on  and  transla,on   Inhomogeneous coordinates (Similarity)
⎡ xT ⎤ ⎡s ⋅ cos θ −s ⋅ sin θ ⎤ ⎡ x R ⎤ ⎡t x ⎤
•  Need  2  points  correspondences     ⎢ y ⎥ = ⎢ s ⋅ sin θ s ⋅ cosθ ⎥ ⎢ y ⎥ + ⎢t ⎥
⎣ T⎦ ⎣ ⎦⎣ R ⎦ ⎣ y ⎦
to  be  es,mated  

Homogeneous coordinates
⎡ xT ⎤ ⎡s ⋅ cos θ −s ⋅ sin θ t x ⎤⎡ x R ⎤
⎢ y ⎥ = ⎢ s ⋅ sin θ s ⋅ cosθ t y ⎥⎢ y R ⎥
⎢ T⎥ ⎢ ⎥⎢ ⎥
⎢⎣ 1 ⎥⎦ ⎢⎣ 0 0 1 ⎥⎦ ⎢⎣ 1 ⎥⎦
 Affine  
•  6  D.O.F.   Inhomogeneous coordinates
⎡ xT ⎤ ⎡ a11 a12 ⎤⎡ x R ⎤ ⎡ t x ⎤
•  Parallel  lines  are  preserved   ⎢ y ⎥ = ⎢a
⎣ T ⎦ ⎣ 21
+
a22 ⎥⎦⎢⎣ y R ⎥⎦ ⎢⎣ t y ⎥⎦

•  Need  3  points  correspondences  

to  be  es,mated     Homogeneous coordinates
⎡ xT ⎤ ⎡ a11 a12 t x ⎤⎡ x R ⎤
⎢ y ⎥ = ⎢a a t y ⎥⎢ y R ⎥
⎢ T ⎥ ⎢ 21 22 ⎥⎢ ⎥
⎢⎣ 1 ⎥⎦ ⎢⎣ 0 0 1 ⎥⎦ ⎢⎣ 1 ⎥⎦
 Homography  
Inhomogeneous coordinates
•  8  D.O.F.   h x + h12 y R + h13
xT = 11 R
h31 x R + h32 y R + h33
•  Straight  lines  are  preserved   h x + h22 y R + h23
yT = 21 R
h31 x R + h32 y R + h33
•  Needs  4  correspondences  to   Homogeneous coordinates
⎡ xT ⎤ ⎡ h11 h12 h13 ⎤⎡ x R ⎤
be  es,mated       ⎢ y ⎥ = ⎢h
⎢ T ⎥ ⎢ 21
h22 h23 ⎥⎢ y R ⎥
⎥⎢ ⎥
⎢⎣ zT ⎥⎦ ⎢⎣h3º h32 h33 ⎥⎦⎢⎣ 1 ⎥⎦
XT = H ⋅ X R
Linear  Transforma,on  summary  
 Summary  
•  Introduc,on  
•  Projec,on  Models  
•  Projec,on  Model  es,ma,on  based  on  
features  
•  Bad  Correspondences  
•  RANSAC  
•  Image  resampling  
– Applica,ons  
 Projec,on  Model  es,ma,on  

If  we  have  a  set  of  matched  points?  How  we  find  the  
transforma,on  H?  
 Projec,on  Model  es,ma,on  
Each  point  to  point  correspondences  give  rise  to  
two  independent  equa,ons    (XT=HXR)  
xT ( h31 x R + h32 y R + h33 ) = h11 x R + h12 y R + h13 ; yT ( h31 x R + h32 y R + h33 ) = h21 x R + h22 y R + h23

• Matrix  H  can  be  wriZen  as  a  vector  h  

H ⇒ h = [ h11 h12 h13 h21 h22 h23 h31 h32 h33 ]
X T = H ⋅ X R Can be expresed as a linear system Ah = 0

⎡ x R1 y R1 1 0 0 0 − x R1 xT1 − y R1 xT1 − xT1 ⎤

⎢ 0 0 0 x R1 y R1 1 − x R1 yT1 − y R1 yT1 − yT1 ⎥
⎢ ⎥
A=⎢          ⎥
⎢x x Rn 1 0 0 0 − x R n xTn − y R n xTn − xTn ⎥
⎢ Rn ⎥
⎢⎣ 0 0 0 x Rn y Rn 1 − x R n yTn − y R n yTn − yTn ⎥⎦
 Projec,on  Model  es,ma,on  
•   Inhomegeneous  solu,on:  One  of  the  nine  matrix  
element  is  fixed  to  unity  value  (h33).    
•  A  new  equa,on  of  the    form  A’h’=b  is  formed  
⎡ h11 ⎤
⎢h ⎥
⎡ x R1 − x R1 xT1 − y R1 xT1 ⎤⎢ ⎥ ⎡ xT1 ⎤
12
y R1 1 0 0 0
⎢ 0 0 0 x R1 y R1 1 − x R1 yT1 − y R1 yT1 ⎥⎢ h13 ⎥ ⎢ y ⎥
⎢ ⎥⎢ h21 ⎥ ⎢ T1 ⎥
⎢         ⎥⎢ ⎥ = ⎢  ⎥
⎢x x Rn 1 0 0 0 − x R n xTn − y R n xTn ⎥⎢ h22 ⎥ ⎢ x ⎥
⎢ Rn ⎥⎢ h23 ⎥ ⎢ Tn ⎥
⎢⎣ 0 0 0 x Rn y Rn 1 − x R n yTn − y R n yTn ⎥⎦⎢ ⎥ ⎢⎣ yTn ⎥⎦
⎢ h31 ⎥
⎢h ⎥
⎣ 32 ⎦

• Solved  using  a  Gaussian  elimina,on  in  the  case  of  a  minimal  

solu,on  or  using  a  pseudo-­‐inverse  method  in  case  of  an  over-­‐
determined  system     h = (A'T ⋅A') −1 A'T ⋅b'
 Projec,on  Model  es,ma,on  
•  Example  
Reference   Target  
(0,0)   (20,100)  
(320,0)   (300,50)  
(0,240)   (70,150)  
⎡ 0 0 1 0 0 0 0 0 ⎤⎡ h11 ⎤ ⎡ 20 ⎤
(320,240)   (270,210)   ⎢ 0 0 0 0 0 1 0 0 ⎥⎢ h ⎥ ⎢100 ⎥
⎢ ⎥⎢ 12 ⎥ ⎢ ⎥
⎢320 0 1 0 0 0 −320 × 300 0 ⎥⎢ h13 ⎥ ⎢300 ⎥
⎢ 0 0 0 320 0 1 −320 × 50 0 ⎥⎢ h ⎥ ⎢ 50 ⎥
⎢ ⎥⎢ 21 ⎥ = ⎢ ⎥
h = (A'T ⋅A') −1 A'T ⋅b' ⎢ 0 240 1 0
⎢ 0
0 0
0 0 0 240 1
0
0
−240 × 70 ⎥⎢ h22 ⎥ ⎢ 70 ⎥
−240 × 150 ⎥⎢ h23 ⎥ ⎢150 ⎥
⎢ ⎥⎢ ⎥ ⎢ ⎥
⎢ 320 240 1 0 0 0 −320 × 270 −240 × 270 ⎥⎢ 31 ⎥ ⎢
h 270 ⎥
⎢ 0 0 0 320 240 1 −320 × 210 −240 × 210 ⎦⎣ h32 ⎦ ⎣210 ⎥⎦
⎥ ⎢ ⎥ ⎢
⎣
⎡ 0.1317 0.2903 20 ⎤
h = A b = [0.1317 −0.0012] ⇒ H = ⎢−0.2801 100 ⎥
−1 T
0.2903 20 −0.2801 0.3840 100 −0.0025 0.3840
⎢ ⎥
⎢⎣−0.0025 −0.0012 1 ⎥⎦
 Summary  
•  Introduc,on  
•  Projec,on  Models  
•  Projec,on  Model  es,ma,on  based  on  
features  
•  Bad  Correspondences  
•  RANSAC  
•  Image  resampling  
– Applica,ons  
 Bad  correspondences  
•  We  want  to  find  the  template  on  target  image  
Templates  

Reference  Image               Target  Image  

 Bad  correspondences  
•  S I F T   f e a t u r e   a r e  
detected   and   matched  
b e t w e e n   t e m p l a t e  
(reference   image)   and  
target  
•  There   are   some   bad  
matched  features    
 Bad  correspondences  
Matched  points  and  the  given  homography  

Template Target
(57,58) (320,406)
(52,67) (318,411)
(14,76) (249,477)
(38,35) (302,384)
(37,12) (301,364)
(7,25) (251,467)
(30,69) (295,414)
(18,70) (284,414)
(15,58) (281,404)
(20,53) (286,400)
(29,55) (294,402)

⎡ −9.1164 −2.4179 293.7651⎤

H SIFT = ⎢−12.2992 −3.8702 439.4023⎥
⎢ ⎥
⎢⎣ −0.0296 −0.0085 1 ⎥⎦
 Bad  correspondences  
Correct  Matching  (hand)  

Reference   Target  
(0,0)   (267,351)  
(93,0)   (352,352)  
(0,91)   (272,433)  
(93,91)   (350,430)  

⎡1.1213 0.3274 267.00 ⎤

H REAL = ⎢0.2181 1.3348 351.00⎥
⎢ ⎥
⎢⎣0.0006 −0.0010 1 ⎥⎦
 Bad  correspondences  

XR XT=HSIFTXR
XT=HSIFTXR
(0,0)   (293,439)   1  
(93,0)   (315,400)   2  
(0,91)   (324,383)   3  
(93,91)   (306,418)   4  

⎡1.1213 0.3274 267.00 ⎤

H REAL = ⎢0.2181 1.3348 351.00⎥
⎢ ⎥
⎢⎣0.0006 −0.0010 1 ⎥⎦
⎡ −9.1164 −2.4179 293.7651⎤
H SIFT = ⎢−12.2992 −3.8702 439.4023⎥
⎢ ⎥
⎢⎣ −0.0296 −0.0085 1 ⎥⎦
 Example  
Object  Tracking  with  bad  correspondences  
 Solu,on?  
•  Use  a  method  that  discard  bad  
correspondences.  
•  Two  commons  methods:  
1.  RANSAC:  RANdom  SAmple  Consensus  
2.  LMedS:  Least  Medians  of  Squares  
•  Both   are   randomized   methods   that   looks   to  
evaluate  a  set  of  data  in  order  to  test  as  many  data  
as   possible   given   a   result   with   a   good   balance  
between  accuracy  and  computa,onal  costs.  
 Summary  
•  Introduc,on  
•  Projec,on  Models  
•  Projec,on  Model  es,ma,on  based  on  
features  
•  Bad  Correspondences  
•  RANSAC  
•  Image  resampling  
– Applica,ons  
 RANSAC  

•  RANSAC   is   an   abbrevia,on   for   "RANdom  

SAmple  Consensus”.    
•  Itera,ve   method   to   es,mate   parameters   of   a  
mathema,cal   model   from   a   set   of   observed   data  
which  contains  “inliers”  and  “outliers”.  
•  Inliers:   data   whose   distribu,on   can   be   explained   by  
some  set  of  model  parameters.  
•  Outliers:   data   that   do   not   fit   the   model,   caused   by  
noise  or  erroneous  measurements  
 RANSAC  overview  
•  Line  adjustment  example:  

Find  the  best  line  that  adjust  to  

the  given  points  
 RANSAC  overview  
•  Line  adjustment  example:  
Li  
• Randomly  select  two  points  
• D efine   the   hypothesized  
model   Li as   the   line   passing  
through  the  two  points  
 RANSAC  overview  
•  Line  adjustment  example:  
Li  
• Randomly  select  two  points  
• D efine   the   hypothesized  
model   Li as   the   line   passing  
through  the  two  points  
• For   each   point   evaluate   the  
distance  to  the  line  Dn  
 RANSAC  overview  
•  Line  adjustment  example:  
Li  
• Randomly  select  two  points  
• D efine   the   hypothesized  
model   Li as   the   line   passing  
through  the  two  points  
• For   each   point   evaluate   the  
distance  to  the  line  Dn
• C alculate   the   number   of  
points   that   have   a   distance  
below  a  defined  threshold  T.  It  
is  the  consensus  Ci  of  trial  i.
• If  Ci < MinConcensus  ,  repeat  
process    
T
 RANSAC  overview  
•  Line  adjustment  example:  

•  Randomly  select  other  two  

points  
•  Define  the  hypothesized  
model  Li+1  
•  Find  consensus  Ci +1  of  trial  
T
i+1  
•  If  Ci+1 > MinConcensus  ,  Li
Li+1   +1 is a candidate to be the
model.  
•  Process  can  be  repeated  a  
fixed  number  of  trials  in  
order  to  find  a  set  Ci+k with  
a  consensus  bigger  than  Ci
+1      
 RANSAC  overview  
•  Line  adjustment  example:  
T

Li+k   • Un,l   fixed   number   of   trials,  

Randomly   selec,ng   other   two  
points   and   evalua,ng     the  
consensus   Ci +k of   the   of   the  
hypothesized  model  Li+k
• If   Ci+k > Ci   ,   Li +k is the new
candidate to be the model.  
• Algorithms   ends   when   fixed  
number   of   trials   are   reached  
returning   the   model   with   the  
large  consensus  Li+k    
 RANSAC  overview  
•  How  to  define  number  of  trials,  distance  
threshold  and  consensus  level?  
 RANSAC  Distance  Threshold  
•  Measurement  error    is  considered  to  have  a  Gaussian  
distribu,ons   with   a   zero   mean   and   standard  
desvia,on  σ.    
•  Threshold   is   defines   as   a   func,on   of   the   error,   a  
probabilis,c   chi-­‐squared   distribu,on   and   the  
probability   that   a   the   point   is   an   inlier   α   (usually  
α=0.95)  .      
Model   t2  

Line,  Fundamental  Matrix   3.84σ2  

Homography,  Camera  Matrix   5.99σ2  
Trifocal  Sensor   7.81σ2  
Distance  threshold  t2  for  a  probability  α=0.95  of  point  to  be  an  inlier  
 RANSAC:  Number  of  samples  
•  Impossible  to  test  all  combina,ons  or  models.  
•  Number  of  samples  choose  to  be  
representa,ve,  ensuring  with  a  probability  (p)    
that  a  least  one  of  the  random  samples    of  (s)  
points  is  free  of  outliers.  Usually  p=0.99.  
•  Defining  
•   w =  probability  of  any  selected  data  is  inlier  
•  And  
•  ε=1-w probability  of  select  and  outlier  
•     
 RANSAC:  Number  of  samples  
•   A  least  N  selec,ons  (of  s  points)  are  requiered  
in  order  to  warranty  that.   (1− w s ) N = 1− p
•        log(1− p)
N=
log(1− (1− ε ) s )

Number  of  Samples  N  to  ensure  with  a  probabily  p=0.99,  that  a  least  one  sample  is  free  
of  outliers  
 RANSAC:  Number  of  samples  
Adap,vely  method    
•  Onen    ε  is  unknown  
•  So  define  the  worst  case  ε=0.5.    
•  Begin  with  a  large  N  and  recalculate  as  soon  as  
we  are  evalua,ng  the  ra,o  of  inliers/outliers.        
 RANSAC:  Min  Consensus  
•  Terminate  when  the  consensus  of  a  trial  set  is  
similar  to  the  size  of  the  number  of  inliers  
believed  to  be  on  the  data  (size  n),  assuming  a  
a  propor,on  of  outliers.  
Consensusmin = (1− ε )n
 Example  with  Homography  
Matched  points  and  the  given  homography  

Template Target
(57,58) (320,406)
(52,67) (318,411)
(14,76) (249,477)
(38,35) (302,384)
(37,12) (301,364)
(7,25) (251,467)
(30,69) (295,414)
(18,70) (284,414)
(15,58) (281,404)
(20,53) (286,400)
(29,55) (294,402)
 Example  with  Homography  
•  n=11        Number  of  sample  
•  s=4   points  needed  to  es,mate  homography    
•  p=0.99 probability  of  a  least  1  sample  free  of  outliers        
•  ε=0.5  Worst  es,ma,on  (50  %  of  points  are  outliers)  
•  σ = 2 standard  devia,on  in  measured  error  (pixels)        
•  t2=5.99σ2=23.96 t=4.89 distance  threshold  
•  MinConsensus=round((1-0.5)*11)=6    
log(1− p) log(1− 0.99)
N= = = 71.35
log(1− (1− ε ) ) log(1− (1− 0.5) )
s 4
Example  with  Homography  
Reprojec,on  error  

d⊥ = d ( x R ,H xT ) + d ( x T ,Hx R )
-1 2 2
 Example  with  Homography  
First      Trial  
Selected  Points   ⎡1.9893 −4.6401 249.9359 ⎤
Template Target Distance H1 = ⎢3.1725 −8.6414 468.9252⎥
⎢ ⎥
(57,58) (320,406) 56,1296   ⎢⎣ 0.0071 −0.0184 1 ⎥⎦
(52,67) (318,411) 0.0000  
(14,76) (249,477) 0.0000  
ε1 = 1− 4 /11 = 0.6364
(38,35) (302,384) 91.7587   log(1− 0.99)
N1 = = 261.1722
(37,12) (301,364) 120.5504   log(1− (1− 0.6364) )
4

(7,25) (251,467) 0.0000  

(30,69) (295,414) 177.1521  
Because ε1 > ε and N1 > N
(18,70) (284,414) 85.5447   ε1 = 0.5
(15,58) (281,404) 0.0000  
N1 = 71.3554
(20,53) (286,400) 54.6576  
(29,55) (294,402) 49.9468  
Consensus 4
 Example  with  Homography  
Second  Trial  
Selected  Points  
Template Target Distance
(57,58) (320,406) 6.8011
⎡3.0075 1.4521 256.0364 ⎤
(52,67) (318,411) H 2 = ⎢2.4468 3.2538 328.6278 ⎥
10.5362
⎢ ⎥
(14,76) (249,477) 106.9644
⎢⎣0.0059 0.0049 1 ⎥⎦
(38,35) (302,384) 0.0000

(37,12) (301,364) 6.6885

ε 2 = 1− 6 /11 = 0.4545
(7,25) (251,467) 152.6005 log(1− 0.99)
N2 = = 49.6694
(30,69) (295,414) 0.0000 log(1− (1− 0.4545) 4 )
(18,70) (284,414) 1.3537

(15,58) (281,404) 0.0000

(20,53) (286,400) 0.7777

(29,55) (294,402) 0.0000

Consensus 6
 Example  with  Homography  
Third  Trial  
Selected  Points  
Template Target Distance ⎡ 0.4820 0.2232 267.5248 ⎤
(57,58) (320,406) 0.0000
H 3 = ⎢−0.5194 1.1472 353.7777⎥
(52,67) (318,411) 5.8306 ⎢ ⎥
(14,76) (249,477) 115.6277
⎢⎣−0.0014 0.0007 1 ⎥⎦
(38,35) (302,384) 1.3215 ε 3 = 1− 8 /11 = 0.2727
(37,12) (301,364) 0.0000
log(1− 0.99)
N 3= = 14.0302
log(1− (1− 0.2727) 4 )
(7,25) (251,467) 155.9488
(30,69) (295,414) 0.0000
(18,70) (284,414) 0.8439
(15,58) (281,404) 11.651
Repeat  the  process  un,l  a  new  
(20,53) (286,400) 0.0000
consensus  is  found  or  the  Number  of  
(29,55) (294,402) 0.1256
trials  is  reach.  
Consensus 8
 Example  
Object  Tracking  with  RANSAC  
 Summary  
•  Introduc,on  
•  Projec,on  Models  
•  Projec,on  Model  es,ma,on  based  on  
features  
•  Bad  Correspondences  
•  RANSAC  
•  Image  resampling  
– Applica,ons  
 Image  Resampling  
•  Image  Resampling  depends  of  the  desired  
applica,on  
– Tracking:  Only  project  the  reference    Frame  on  the  
current  image.   Object I n = H R −n FrameI 0
– Image  Stabiliza,on:  Resampling  the  current  Image  
to  the  reference  frame  by  means  of  
IStable (x, y) = H R−1−n In (x, y)
– Mosaic:  For  each  new  image,  project  it  to  the  
−1
reference  frame     Mosaic
I (x, y) = H R −n I n (x, y)

•  If  pixel  (x,y)  on  mosaic  don’t  have  data.  Put  the  projected  pixel  
•  If  pixe  (x,y)  already  have  informa,on,  make  a  data  interpola,on  
Applica,ons:  Video  Stabiliza,on  

Refrence  

Current  Image           Reprojected  Image  

IStable (x, y) = H R−1−n In (x, y)

Applica,ons:  Video  Stabiliza,on  

Euclidean               Homography  

IStable (x, y) = H R−1−n In (x, y)

Applica,ons:  Mosaics  
Applica,ons:  Mosaics  
Applica,on:  Mosaics  
 Applica,ons:  Mosaics  
•  Define  a  Reference  Image:    
•  For  each  image  
–  Calculate  the  Homography  between  Reference  Image  and  current  
−1
image.   IMosaic (x, y) = H R −n In (x, y)
–  Reproject  pixel  to  pixel  the  each  image  on  the  reference  space.  
–  If  pixel  (x,y)  on  mosaic  don’t  have  data.  Put  the  projected  pixel  
–  If  pixe  (x,y)  already  have  informa,on,  make  a  data  interpola,on  

Reference  
Image  for  
Mosaic  
 Applica,ons:  Mosaics  
•  Define  a  Reference  Image:    
•  For  each  image  
–  Calculate  the  Homography  between  Reference  Image  and  current  
−1
image.   IMosaic (x, y) = H R −n In (x, y)
–  Reproject  pixel  to  pixel  the  each  image  on  the  reference  space.  
–  If  pixel  (x,y)  on  mosaic  don’t  have  data.  Put  the  projected  pixel  
–  If  pixe  (x,y)  already  have  informa,on,  make  a  data  interpola,on  

Reference  
Image  for  
Mosaic  
Applica,ons:  Mosaics  
 Applica,ons:  SuperResolu,on  
•  SuperResolu,on:   Given  
a   s e t   o f   o b s e r v e d  
images,   es,mate   an  
i m a g e   a t   a   h i g h e r-­‐
r e s o l u , o n   t h a n   i s  
present   in   any   of   the  
individual  images  
Applica,ons:  SuperResolu,on  
 References  
•  D.  Lowe,  “Dis,nc,ve  image  features  from  scale-­‐invariant  keypoints,”  
Interna,onal.  Journal  of  Computer  Vision,  vol.  2,  no.  60,  pp.  91–110,  
2004.  
•  H.  Bay,  T.  Tuytelaars,  and  L.  Van  Gool,  “SURF:  Speeded  up  robust  
features,”  
•  R.  Hartley  and  A.  Zisserma.  Mul,ple  View  Geometry.  Second  Edi,on  
•  Barbara  Zitová,  Jan  Flusser:  Image  registra,on  methods:  a  survey.  
Image  Vision  Comput.  21(11):  977-­‐1000  (2003)