GsE 188

1st Semester Academic year 2021-2022

Laboratory Exercise 5
Image Matching

Submitted to:
Engr. Bienvenido G. Carcellar III

Submitted by:
Constancio P. Orolfo III
I. Objective

This laboratory exercise is part of the Image Matching lecture of the course. In this
regard, the techniques and strategies introduced in the lecture were put on display and
demonstrated through various means.

Image matching was performed in two parts of the exercise. With the use of cross-
correlation and the exhaustive search strategy, a template image was matched to an
original image. In the second part, a Python code was used to match a part of an image
to a larger image, even when that part, the slave, was altered in a drawing software.

II. Materials and Instruments used

In performing this exercise, several materials were already provided for, such as:
• A spreadsheet containing a template and image matrix
• A Google Colab Python code
There were also some that were provided by the student, such as:
• An image of the student with their friends, for image matching purposes
• Clip Studio Paint, the editing software of choice of the student.

III. Procedure

As previously mentioned, the exercise had two parts. For the first part of the exercise
the following steps were followed to successfully complete the given task.
1. The given spreadsheet was opened and inspected, the task at hand was to match the
template matrix to the image matrix.
2. Four (4) columns of zeroes (0) were added to both the left and right of the image
matrix, then four (4) rows of zeroes (0) of length equal to number of columns were
added to the top and bottom of the image matrix.
3. The correlation value was obtained from the template matrix and the part of the
image that it overlaps. This starts at the very topmost zero (0) such that the fifth
element of the fifth row of the template matrix overlaps the first non-zero element
of the matrix. The correlation obtained is then recorded in the cross-correlation
matrix.
4. After this, the template moves one (1) column to the right, such that the bottom row
now overlaps with two (2) non-zero elements of the image matrix. This continues
until the last column of the template has overlapped with the last column of zeroes
(0).
5. When the template has no more space to move to the right, it goes down one row
and back to the first column, such that the overlap is now the first two (2) non-zero
elements of the first column of the image matrix to the last two (2) elements of the
last column of the template matrix. Repeat step three (3) to (5) until the first element
 of the template matrix is the only one that is overlapping a non-zero element of the
image matrix.
6. If done right, a 13 by 13 cross-correlation has been produced. The value closest to
one (1) indicates the 5 by 5 part of the image matrix that best matches the template.
The row and column of this value closest to one (1) indicates the row and column
of the first element of the 5 by 5 match.
For the next part of the exercise, a part of a larger image, called the template was
produced, called the slave image. Using Python code that was already provided, this
slave was matched onto the original template image. These were the steps taken to
successfully perform this part of the exercise:
1. An image was chosen, one with several faces including that of the student’s. The
student’s face was cropped out using Clip Studio Paint, producing the slave image.
2. More altered copies of the slave image were produced. One with the brightness
turned all the way up to 100, one with the contrast turned up to 75 and one where
the image was rotated by 45 degrees.
3. The master image and the slaves were uploaded to the provided Google Colab code
and its directory.
4. For the first iteration, the unedited slave was matched onto the master, producing
the first output.
5. The next output was that of the one with the brightness turned up.
6. The next output was that of the one with the contrast increased.
7. The last output was that of the rotated slave image.
8. When the outputs are produced, a correlation value is also produced, this is noted
and saved.

IV. Observations/Data

Table 1. The part of the image matrix that the template matches.

Part of the image that the template matches

199 226 27 177 95
9 64 236 1 31
136 241 13 101 160
162 138 157 237 45
17 222 115 87 6

The table above is the part of the image matrix that closely matches the template. This was found through
finding the correlation of the template with parts of the larger image.
 Table 2. Cross-correlation matrix of the template and the image.
-0.27091 -0.23185 -0.14894 0.098795 -0.18187 -0.27742 -0.27526 -0.0439 -0.00387 0.070593 -0.19986 0.152938 -0.23439
-0.32587 -0.11435 -0.13391 0.029442 -0.15131 0.012198 -0.10283 -0.01011 -0.11317 0.124637 0.086354 0.065389 -0.20477
-0.19774 0.079218 -0.1322 0.093723 0.272696 0.174143 0.172645 -0.00553 0.081348 0.37599 0.211353 0.198973 -0.00296
-0.19944 -0.31968 -0.13767 0.140406 -0.04031 -0.30378 0.04255 -0.07596 0.085378 -0.06691 -0.12048 0.35791 -0.06108
-0.26857 -0.1861 -0.04721 -0.20383 0.041383 0.002572 -0.30761 -0.03805 -0.14426 0.139268 0.039697 0.087739 -0.24893
-0.131 -0.20066 -0.38835 0.269528 0.199634 -0.19209 0.215474 0.164867 -0.08312 0.450154 0.157288 0.359926 0.052229
-0.39463 -0.3455 0.07864 -0.10056 -0.36058 0.15933 -0.04812 -0.10298 0.061164 0.063951 -0.16912 0.518409 -0.1081
-0.11376 -0.04157 -0.30853 -0.12228 0.997566 -0.27564 -0.29663 0.137264 0.172728 0.254426 0.193293 0.016492 -0.00531
-0.22376 -0.43893 -0.11956 0.246372 -0.41952 -0.34395 0.321683 -0.25093 0.098529 0.318059 -0.02273 0.16619 0.069887
-0.25595 -0.07565 0.057293 -0.28045 0.202406 0.332621 -0.04551 -0.00892 0.383733 0.027948 0.309642 0.147894 -0.1539
0.011981 -0.07818 -0.38279 0.118601 0.23014 -0.16737 -0.0828 0.102071 -0.10456 0.215159 0.057572 -0.00061 0.210537
-0.20736 -0.26286 -0.00137 -0.15843 -0.20827 -0.15473 -0.02063 -0.25887 -0.07866 0.024894 0.011714 -0.02835 0.057925
-0.03872 0.057957 -0.04429 0.018929 0.206846 0.125435 0.082881 0.153967 0.237893 0.160584 0.351789 0.248026 0.235223

The table above is the cross-correlation matrix produced by the template and the image. The values range from as low as -
0.43893 to 0.997566. The previously obtained matched matrix was found using this matrix. The cell marked in green has the
value closest to 1, therefore it corresponds to the part of the image that closely matches the template.
 Figure 3. The master image.

The image above was the one used as the master image. The original image was 3032 by
4032 pixels. During the first time the image was put into the code, three hours had passed
and no output had been produced. To expedite the process, the image was cropped, and
then scaled down making the master image only 349 by 289 pixels. This greatly reduced
the time it took for the code to run, with each iteration only taking 11 seconds.

Figures 4.1 to 4.4. The slave images used for the task.

The first image was the original slave image, with no modifications. Second image had its
brightness raised to 100 once. Third image was the first image with the contrast turned to
75 once. The last image was the original slave image rotated 45 degrees with the same
dimensions.
 Figure 5. Output 1

The image above the first output. It is from the master and the first slave image.
Figure 6. Second Output.
The second output is similar to the first one. It was from the master and the second
slave image, the one that had its brightness adjusted.
Figure 7. Third Output.

The third output is also similar to the first. The assumption is that because the slave
isn’t moved in any way, only the values change, the area that matches also doesn’t
change.
Figure 8. Fourth Output.
 The last output is the one where the original slave image was rotated. Its dissimilar to
the other ones because the rotated image covers different pixels that time, moving the
area that matches.

V. Computation

For the first task, the only computation involved was the correlation values. As stated
in the 6th Lecture, the correlation function is the quotient of the covariance of the two
matrices, and the product of the standard deviations of the template and the image. In
the spreadsheet, the functions exist for the necessary values that were needed for the
cross-correlation matrix. It is important to note that the sample formula was used
instead of the population formula.

VI. Analysis and Answer to Questions

The code, as understood, compares the size and dimensions of the slaves and master
images. There is then a portion that crops the master to the same shape of the slave
image and computes the correlation values, probably using the formula that was
described in the computation portion. It is assumed, again, that the code does this over
and over until the last corner of the slave as reached the last corner of the master image.
The code then obtains the highest correlation value and which cropped part of the image
it relates to. This is then displayed in the output along with other values such as the
correlation value of the cropped part and the slave image.

The first result was a correlation value of 0.9997275327460591. It is the highest

correlation, nearest to 1. It might not be exactly 1 because of rounding errors or the fact
that the decimal itself isn’t rounded off.

Changing the brightness did not throw the code off by a lot, although changing the
contrast did, and obviously the rotation. The brightness might be because all the values
are proportionally or equally adjusted, unlike the contrast where some values might
change more than others. The translation offsets the values and changes which group
of pixels will overlap the master image.

The biggest factor against exhaustive search would be the amount of time and resources
spent on larger and larger images. For bigger images, the code would have to overlap
thousands of pixels over an image with over a million pixels, and then do its
computations, and then do this over and over and over again. The pro of this is that its
very thorough.
VII. Conclusions

In the exercise, image matching was performed in two tasks. The first task saw to the
manual performance of image matching through the formula and strategy taught in
class, with the help of software. The next task, with the help of prepared code, sought
to demonstrate the strategy at work with an actual image, at the same time exposing the
strategies strong and weak points. In the same task, the effects of manipulating the
template were also shown, bringing to light that some manipulations can have a larger
impact than others