MEI6900 Experimental Methods

Cellphone Camera Calibration

For MEI6900 – Experimental Methods in Fluid Mechanics and Combustion with Prof. Yiannis
Andreopoulos

April 16th, 2018

Yoribaldis Olivo

Department of Mechanical Engineering, City College of City University of New York, New
York, NY 10031, USA

yolivo01@citymail.cuny.edu

1
 MEI6900

1. Introduction

The primary objective of this study is to determine the calibration parameters from an image
taken of a phone camera. The process of manipulating and analyzing images to improve their
quality or obtain information for further study is known as image processing [1]. Today, image
processing is widely used in different fields for a variety of applications from pattern recognition
to development of machine learning tools [2,3]. For this project we are interested in defining
camera properties that affect the imaging processing procedure and are necessary to calibrate our
image. The image calibration process consists of determining the intrinsic and extrinsic parameters
necessary to find misrepresentations in our images. The intrinsic parameters are the camera’s
internal geometric and optical properties whereas the extrinsic parameters are defined by the
orientation of the camera with respect to a specified world coordinate system [4]. These properties
are necessary to process are images correctly by removing any unwanted characteristics due to the
image discrepancies from real world representation.

Our phone camera introduces significant distortion in the images taken from it. These
distortions are mainly radial and tangential [5]. Due to these distortions, lines on the image may
appear to be curved when in reality they are not. However, these distortions are constant and can
be adjusted for. Knowing the camera calibration parameters, allows us to apply a routine that lets
us eliminate distortion effects from an image. With these calibration parameters, we can determine
the correlation matrix, the relationship between the camera’s units and the real world units along
with rotational and translational matrices. Utilizing a MATLAB image processing algorithm
derived from pinhole camera theory which explains the relationship between a three-dimensional
point and the projection of its image [6]. In this case we will apply the model for a split view discs
for which we will determine the calibration matrix, intrinsic matrix, extrinsic matrix, and camera
calibration parameters from the image taken from our cellphone camera.

2. Methodology

2.1 Pinhole Model Theory

The use of pinhole camera theory allowed people to design and manufacture more affordable
cameras. These cheap cameras come with one big disadvantage: a large distortion on the images
obtained from the camera. However, as mentioned before, these distortions are constant for the
camera model and can be eliminated by processing the image and calibrating it. The idea of the
pinhole camera is that we take a three-dimensional images and project it in a two-dimensional
space ideally without any distortion as observed in Fig. 1. This two-dimensional image is mapped
by the camera matrix into the image plane. The camera matrix is obtained from an algorithm that
uses the intrinsic and extrinsic parameters.

2
 MEI6900

Figure 1. Pinhole camera diagram

The purpose of this calibration is to find the relationship between the camera units in pixels
and the real world units usually in millimeters or inches. By using lensmaker triangles similarity
equation we can obtain the relationship between the focal length and the image location which
provides us with the perspective projection formulas

−𝑓 ̅𝑥 −𝑓 ̅𝑦 −𝑓 ̅𝑧
𝑥′ = ; 𝑦′ = ; 𝑧′ =
𝑧−𝑓 𝑧−𝑓 𝑧−𝑓

2.2 Image Processing

For this study we used an image taken iPhone X designed and manufactured by Apple
Incorporated. This cellphone has a 12 Megapixels wide-angle and telephotos camera. With the
iPhone X camera, we took a photo of a calibration plate with pluses symbols and know spatial
three dimensional coordinates observed in Fig. 2a. The plate has 11 columns with an uneven
number of rows with a total of 97 plus signs. The first step is to determine a calibration pattern
which in our case is the calibration plate observed in Fig. 1a. Then this image was imported into
the MATLAB software algorithm provided named Cal1_Left1_1_NewPlate.m. This program
exports the locations of each of the plus signs in the plate. After importing the image and
determining its size, we manually select the center location for each of the plus signs on our plates

3
 MEI6900

that have different X, Y, and Z coordinates. The program then defines the coordinate system and
the arrangement of the plus signs on the plate. Each of the plus signs on the plate are from their
neighboring signs equally in the X and Y directions where ∆𝑥 and ∆𝑦 equal 25.4 mm. Fig. 2b
below illustrates the calibration plate after being processed by the program with the center location
of the plus signs selected.

Figure 2. Calibration plates with known three dimensional coordinates (a) Before processing (b)
After processing

After determining the three-dimensional location of each plus sign for the first program we
imported the data into the second MATLAB program provided OneViewOnly1.m. The algorithm
determines the calibration coefficients, camera matrix, intrinsic matrix and extrinsic matrix using
pinhole camera theory for one sensor location. These parameters are necessary to calibrate the
image and account for its distortions. After determining the world coordinate system, we start of
by transforming it to the camera coordinate system using the following relationship taking into
account the lens distortion:

𝑥𝑐𝐴 𝑅𝐴11 𝑅𝐴12 𝑅𝐴13 𝑅𝐴14 𝑋𝑤

𝑦𝑐𝐴 𝑅 𝑅𝐴22 𝑅𝐴23 𝑅𝐴24 𝑌𝑤
{ } = [ 𝐴21 ]{ }
𝑧𝑐𝐴 𝑅𝐴31 𝑅𝐴32 𝑅𝐴33 𝑅𝐴34 𝑍𝑤
1 0 0 0 1 1

This R matrix takes into the account the translational and rotational components needed for
the transformation. The 𝑥𝑐𝐴 , 𝑦𝑐𝐴 , and 𝑧𝑐𝐴 represents the camera coordinates while 𝑋𝑤 , 𝑌𝑤 , and 𝑍𝑤
are the real world coordinates. Then the system still has to be converted into the image

4
 MEI6900

coordinate system by relating the camera coordinate transformation matrix with the intrinsic
matrix given by

𝑓𝑆𝑥 0 −𝑐𝑥 𝑆𝑥 0
𝐾 = [ 0 𝑓𝑆𝑦 −𝑐𝑦 𝑆𝑦 0 ]
0 0 1 0

Where 𝑓 is the focal length in pixels, 𝑆 is the skew coefficient (the pixel size in the
respective direction) and 𝑐 is the principal point in pixels. Then the world coordinate
transformation matrix becomes

𝑋𝑤
𝑥𝑠𝐴 𝑓𝑆𝑥 0 −𝑐𝑥 𝑆𝑥 0 𝑅𝐴11 𝑅𝐴12 𝑅𝐴13 𝑅𝐴14
𝑌
𝛼 {𝑦𝑠𝐴 } = [ 0 𝑓𝑆𝑦 −𝑐𝑦 𝑆𝑦 0 ] [𝑅𝐴21 𝑅𝐴22 𝑅𝐴23 𝑅𝐴24 ] { 𝑤 }
𝑍𝑤
1
0 0 1 0 𝑅𝐴21 𝑅𝐴22 𝑅𝐴23 𝑅𝐴24 1

Finally, by simplifying the above matrix we obtain the matrix for the calibration coefficients
below. The matrix below represents the calibration coefficients needed to transform the real world
coordinates into the image coordinates system derived from the intrinsic and extrinsic The results
obtained from the second program provided are illustrated in the results section.

𝑋𝑤
𝛼𝑥𝑠𝐴 𝐿𝐴11 𝐿𝐴12 𝐿𝐴13 𝐿𝐴14
𝑌
{𝛼𝑦𝑠𝐴 } = [𝐿𝐴21 𝐿𝐴22 𝐿𝐴23 𝐿𝐴24 ] { 𝑤 }
𝛼 𝐿𝐴31 𝐿𝐴32 𝐿𝐴33 𝐿𝐴34 𝑍𝑤
1

4. Results

In this section we discuss all the parameters obtained from the study and explain what is the
relationship or effect of each based on the pinhole camera theory. From the calibration coefficients
matrix, the image coordinates are recovered and compared to the original points obtained from the
first MATLAB program using the OneViewOnly1.m. code provided. Fig. 3 below illustrates the
overlay of the points of the calibration plate.

The difference between the original image points and the image coordinates recovered from
the calibration matrix is on average 2.6%. The largest error as expected occur when further away
from the center of the images where the distortion effects are greater. Other parameters calculated

5
 MEI6900

in this study used to calibrate the calibration plate images are tabulated below with a brief
description of their particular contribution to the overall process.

Figure 3. Image location reconstruction from calibration matrix compared to original

Table 1, Table 2, and Table 3 below illustrate the real world coordinates of the calibration
object in mm where the origin of our system is the optical center of the camera for 𝑋𝑤 , 𝑌𝑤 , and 𝑍𝑤 ,
respectively. The world coordinate system is three-dimensional and independent of the camera
position with respect to the object in study.

Table 1. Real World X-Coordinates in mm

C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11
-50.8 -25.4 0 25.4 50.8
-76.2 -50.8 -25.4 0 25.4 50.8 76.2
-101.6 -76.2 -50.8 -25.4 0 25.4 50.8 76.2 101.6
-127 -101.6 -76.2 -50.8 -25.4 0 25.4 50.8 76.2 101.6 127
-127 -101.6 -76.2 -50.8 -25.4 0 25.4 50.8 76.2 101.6 127
-127 -101.6 -76.2 -50.8 -25.4 0 25.4 50.8 76.2 101.6 127
-127 -101.6 -76.2 -50.8 -25.4 0 25.4 50.8 76.2 101.6 127
-127 -101.6 -76.2 -50.8 -25.4 0 25.4 50.8 76.2 101.6 127
-101.6 -76.2 -50.8 -25.4 0 25.4 50.8 76.2 101.6
-76.2 -50.8 -25.4 0 25.4 50.8 76.2
-50.8 -25.4 0 25.4 50.8

6
 MEI6900

Table 2. Real World Y-Coordinates in mm

C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11
-127 -127 -127 -127 -127
-101.6 -101.6 -101.6 -101.6 -101.6 -101.6 -101.6
-76.2 -76.2 -76.2 -76.2 -76.2 -76.2 -76.2 -76.2 -76.2
-50.8 -50.8 -50.8 -50.8 -50.8 -50.8 -50.8 -50.8 -50.8 -50.8 -50.8
-25.4 -25.4 -25.4 -25.4 -25.4 -25.4 -25.4 -25.4 -25.4 -25.4 -25.4
0 0 0 0 0 0 0 0 0 0 0
25.4 25.4 25.4 25.4 25.4 25.4 25.4 25.4 25.4 25.4 25.4
50.8 50.8 50.8 50.8 50.8 50.8 50.8 50.8 50.8 50.8 50.8
76.2 76.2 76.2 76.2 76.2 76.2 76.2 76.2 76.2
101.6 101.6 101.6 101.6 101.6 101.6 101.6
127 127 127 127 127

Table 3. Real World Z-Coordinates in mm

C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11
0 26 0 26 0
26 0 26 0 26 0 26
0 26 0 26 0 26 0 26 0
26 0 26 0 26 0 26 0 26 0 26
26 0 26 0 26 0 26 0 26 0 26
26 0 26 0 26 0 26 0 26 0 26
26 0 26 0 26 0 26 0 26 0 26
26 0 26 0 26 0 26 0 26 0 26
0 26 0 26 0 26 0 26 0
26 0 26 0 26 0 26
0 26 0 26 0

After converting the world coordinates to camera coordinate we finally have to transform them
into image coordinates. Table 4 and Table 5 illustrate the 2D projection of the image in pixel units
for its X and Y coordinates, respectively. This result is obtained after rotating and translating the
world coordinate units since usually the center of the image does not correspond with the center
of the camera coordinates.

7
 MEI6900

Table 4. Image X-Coordinates

C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11
1055 1265 1535 1775 1991
755 1061 1265 1535 1781 1997 2291
593 761 1055 1271 1541 1781 2009 2297 2483
263 605 767 1067 1271 1541 1781 2015 2297 2483 2813
263 599 767 1073 1283 1547 1781 2009 2303 2489 2819
263 599 773 1079 1283 1547 1793 2009 2303 2489 2819
263 599 773 1079 1289 1541 1793 2009 2309 2501 2831
263 605 779 1073 1289 1541 1799 2015 2315 2501 2831
599 779 1079 1289 1541 1811 2027 2321 2513
779 1079 1295 1547 1811 2027 2327
1085 1295 1547 1817 2033

Table 5. Image Y-Coordinates

C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11
355 253 343 247 331
523 601 505 577 493 577 487
829 763 829 751 817 751 817 733 799
1033 1069 1015 1069 1009 1057 1003 1051 997 1033 985
1279 1303 1279 1291 1267 1291 1255 1273 1249 1261 1243
1537 1537 1525 1531 1519 1531 1507 1513 1507 1519 1495
1789 1771 1777 1771 1771 1759 1771 1753 1765 1741 1759
2047 2005 2035 1993 2029 1993 2023 1987 2023 1981 2011
2245 2293 2233 2287 2233 2287 2233 2275 2221
2557 2473 2545 2467 2545 2473 2539
2719 2809 2719 2803 2701

Table 5 illustrates the Camera Matrix (K) which is also referred as the intrinsic matrix. This
matrix contains the intrinsic parameters which includes the focal length, image principal points,
and the camera skew.

Table 5. Camera Matrix (K) for Pinhole model.

K 1 2 3
1 48292.67 -3208.91 -8935.16
2 0 47100.33 -12307.73
3 0 0 1

8
 MEI6900

The extrinsic matrix components are shown below in Table 6 and 7. This matrix represents
the parameters needed to transform the three-dimensional world coordinates into the camera
coordinates by translation (Table 7) and rotation (Table 6).

Table 6. Camera’s Extrinsic Rotation Matrix

𝑹𝒊𝒋 1 2 3
1 0.98 -0.05 0.21
2 -0.08 0.96 0.27
3 0 0 0

Table 7. Camera’s Extrinsic Translation Vector

T 1
1 1.096E+03
2 1.487E+03
3 1

Finally, one we have established the camera internal geometric parameters and three-
dimensional orientation in the real world coordinate system we can determine the calibration
necessary to transform our object into the image coordinate system. The camera calibration matrix
for the pinhole model P is shown below in Table 8. This matrix represents the relationship between
the values necessary to calibrate the object in the real world to the coordinate system of the image.

Table 8. Camera Calibration Matrix for Pinhole Model (P).

P 1 2 3 4
1 9.71 0.03 0.31 1538.68
2 -0.21 9.61 0.25 1519.65
3 0 0 0.00 1

9
 MEI6900

The calibration matrix L is the same as the camera calibration matrix discussed before.

Table 9. Calibration Matrix (L)

L 1 2 3 4
1 9.71 0.034 0.31 1.54E+03
2 -0.21 9.61 0.25 1.52E+03
3 0 0 0 0

5. Conclusion
Image calibration is an essential technique needed to effectively retrieve the correct data
needed for studies in many different field. In order calibrate the image many different internal and
external camera and image parameters have to be taken into account to properly process our
images. The average error from the study was 2.6% error. Since the distortion is constant there
might be some additional parameters that might not be taken into account and in order to determine
that we would need a more in depth understanding of the physical parameters and mathematical
modeling involved.

It would also interesting to follow the same procedure for different calibration patterns with
the same phone camera to observe if there would be any difference in the results observed in this
study. Another study using the same calibration plates for different camera phone and study any
correlation in the results depending a camera specifications and features.

10
 MEI6900

6. References

[1] Šonka, Milan, et al. Image Processing, Analysis, and Machine Vision. Cengage Learning, 2015.
[2] Guo, Yandong, et al. “MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face
Recognition.” Computer Vision – ECCV 2016 Lecture Notes in Computer Science, 2016, pp. 87–
102., doi:10.1007/978-3-319-46487-9_6.
[3] Zhi-Ming, Dong, et al. “Oil-Spills Detection in Net-Sar Radar Images Using Support Vector
Machine.” The Open Automation and Control Systems Journal, vol. 7, no. 1, 2015, pp. 1958–
1962., doi:10.2174/1874444301507011958.
[4] Tsai, R. “A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision
Metrology Using off-the-Shelf TV Cameras and Lenses.” IEEE Journal on Robotics and
Automation, vol. 3, no. 4, 1987, pp. 323–344., doi:10.1109/jra.1987.1087109.
[5] Hartley, Richard, and Andrew Zisserman. Multiple View Geometry in Computer Vision.
Cambridge University Press, 2017.
[6] Zhang, Z. “A Flexible New Technique for Camera Calibration.” IEEE Transactions on Pattern
Analysis and Machine Intelligence. Vol. 22, No. 11, 2000, pp. 1330–1334.
[7] “Tutorial Camera Calibration.” Tutorial Camera Calibration - BoofCV,
boofcv.org/index.php?title=Tutorial_Camera_Calibration.
[8] Bouguet, J. Y. “Camera Calibration Toolbox for Matlab.” Computational Vision at the California
Institute of Technology.

11