Istanbul Medipol University

Electrical-Electronics Engineering And Cyber

Systems

Computer Vision
Problem Set 3 Report
Feature Extraction and Applications

Written By:
Talha YILMAZ (D3210008)

Date: December 22, 2022

Contents
1 Problem 1 2

1
1 Problem 1
In the provided python script for Problem Set 3, three different feature extraction algorithms
(SIFT, SURF, ORB) are implemented. Strength how these matches are evaluated by their
distance property. As a side note mentioned matches are stored in a DMatch data type which
is a class that holds distance property. Distance property gives the difference amount between
descriptors of the keypoints that are matched. This gives us a tool to decide how strong the
matching is between two keypoints. Lower distance signifies strong match while larger distance
means descriptors were much more different and matching strength gets weaker. In my script
top 5 matches are drawn which is shown in Figure 1.1.

Figure 1.1: Top 5 matches obtained by SIFT algorithm.

Next step asked in the assignment is calculation of the repatability rates of the feature de-
tectors/descriptors. In the assignment there is a provided homography matrix which explains
relationship between two images. In my script repeatability calculation is done in two steps.
First find correspondences between matches. Assuming matching is already done, correspon-
dence operation can be shown in (1.1)

Corr(xi ) = ∥xi − xˆj ∥ < ϵ, (1.1)

where xi is the keypoint and xˆj is homography transformed version of its matched keypoint.
For a given ϵ which is a pixel value we can check if our match is a good match or not. For
an example if we choose ϵ = 3 then it means maximum allowed variance around the found
keypoint is 3 pixels. Anything more is classified as a bad match. For my script I chose ϵ = 3.
As we increase ϵ we introduce more slack for possible matches thus correspondence probability
(hence repeatability) increases at the expense of matching quality. From correspondences we
can obtain repeatability as:
!
1 X X
Repeatability = Corr(xi ) + Corr(xj ) (1.2)
N1 + N2 i j

Obtained repeatability scores is shown in Table 1.

2
 Table 1: Comparison of repeatability metric for different feature extraction algorithms.

Repeatability Number of
Method
Score Detected Keypoints
SIFT 56.80% 3674
SURF 59.71% 988
ORB 76.92% 650

Third task asked in the assignment is to calcuate the homography matrix and incorporating
the RANSAC algorithm. Homography matrix can be modelled as

 ′   
wxi h00 h01 h02 xi
wyi′  = h10 h11 h12   yi  (1.3)
w h20 h21 h22 1
From this form we can write the following system of equations
′ h00 xi + h01 yi + h02
xi = (1.4)
h20 xi + h21 yi + h22
′ h10 xi + h11 yi + h12
yi = (1.5)
h20 xi + h21 yi + h22

′
xi (h20 xi + h21 yi + h22 ) = h00 xi + h01 yi + h02 (1.6)
′
yi (h20 xi + h21 yi + h22 ) = h10 xi + h11 yi + h12 (1.7)

 
h00
h01 
 
h02 
 
 ′ ′ ′  h10   
xi yi 1 0 0 0 −xi xi −xi yi −xi  h11  = 0

′ ′ ′ (1.8)
0 0 0 xi yi 1 −yi xi −yi yi −yi  
h12  0
 
h20 
 
h21 
h22
(1.6) can be written in the form Ah = 0. A matrix can be expanded for chosen (x,y) pairs.
Minimum amount of points required to solve this problem is 4 since degree of freedom of vector
h is only 8. Least-squares solution can be obtained by finding the eigenvector which corresponds
to the minimum eigenvalue. For robust estimation of the homography matrix we can utilize
RANSAC algorithm with keypoint matches P given as (x1 ,y1 ,x2 ,y2 ) pairs as follows:
By using Algortihm 1 we can obtain P̂ to have robust estimation of H. In both RANSAC
algorithm and final estimation H least-squares solution is obtained via singular value decom-
position as follows:

Ah = 0 (1.9)
T
A Ah = 0 (1.10)

Best solution that minimizes ∥AT Ah∥ can be obtained by finding the eigenvectors of AT A that
corresponds to the lowest eigenvalue which is equivalent to finding eigenvectors that corresponds
to the lowest singular value of A.

3
Algorithm 1 RANSAC Algorithm
Require: P, ϵ, Nmax
P̂ ← {}
while i < Nmax do
Choose four random points from P and store it as Ptrial .
Find homography matrix H using Ptrial .
Find number of points supported by H PH
if len(PH ) > len(P̂ ) then
P̂ = PH
end if
if len(P̂ ) > ϵlen(P ) then
break
end if
end while

Figure 1.2: Warped Image

Third Task is to warp one image onto the other one using the homography matrix. I have
used bilinear interpolation to achieve this and results are given in Figure 1.2. My code only
works for a single color channel. Thus, warping results only shown on gray-scaled image.
Fourth task was to stitch these two images. For this I need to use my warping function to
warp another image and stitch it to the other one. I used built-in warpPerspective function

4
of cv2 module since it is performing faster and works for all color channels. Stitched image is
shown in Figure 1.3.

Figure 1.3: Stitched Image