Camera Calibration

What is Camera Calibration?

Primarily, finding the quantities internal to
the camera that affect the imaging process
Position of image center in the image
It is typically not at (width/2, height/2) of image
Focal length
Different scaling factors for row pixels
and column pixels
Skew factor
Lens distortion (pin-cushion effect)
Motivation
Good calibration is important when we need
to
Reconstruct a world model: Virtual L.A. project
Interact with the world
Robot, hand-eye coordination
I mage plane
We see a square of
known size
Evaluation of position of a square for 2 focal
lengths (red and blue projection geometry)
Scaling of Rows and Columns
in Image
Camera pixels are not necessarily square
Camera output may be analog (NTSC)
Image may be obtained by digitizing card
A/D converter samples NTSC signal
Camera
NTSC signal Digitizing
Monitor
display
CCD/
CMOS
Compound Lens Imaging
Inexpensive single lens systems distort image at
its periphery
Compound lenses may be used to reduce
chromatic effects and pin-cushion effects
C
Camera
Image plane
Image plane for equivalent
pinhole camera is
not camera image plane
Principal planes
Center of Projection
Nodal Point
f
f
Variety of Techniques
VERY large literature on the subject
Work of Roger Tsai influential
Linear algebra method described here
Can be used as initialization for iterative non
linear methods.
Some interesting methods use vanishing
points
Camera and Calibration Target
Calibration Procedure
Calibration target: 2 planes at right angle with
checkerboard patterns (Tsai grid)
We know positions of pattern corners only with
respect to a coordinate system of the target
We position camera in front of target and find
images of corners
We obtain equations that describe imaging and
contain internal parameters of camera
As a side benefit, we find position and orientation of
camera with respect to target (camera pose)
Image Processing of
Image of Target
Canny edge detection
Straight line fitting to detected linked edges
Intersecting the lines to obtain the image
corners
Matching image corners and 3D target
checkerboard corners
By counting if whole target is visible in image
We get pairs (image point)--(world point)
) , , ( ) , (
i i i i i
Z Y X y x
Central Projection
s
s
i
s
s
i
z
y
f y
z
x
f x
=
=
Scene point
(x
s
, y
s
, z
s
)
Image point
(x
i
, y
i
, f)
x
z
C
f
y
center of
projection
I mage plane

1
0
0
0
1 0 0
0 0
0 0
s
s
s
z
y
x
f
f
w
v
u
w v y w u x
i i
/ , / = =
If world and image points are represented by homogeneous
vectors, central projection is a linear transformation:
Transformation From Lengths to Pixels
s
s
i
s
s
i
z
y
f y
z
x
f x
=
=
Image point
(x
i
, y
i
, f)
x
pix
y
pix
s
s s
y i y pix
s
s s
x i x pix
z
y z y
k f y y k y
z
x z x
k f x x k x
0
0
0
0
+
= + =
+
= + =
y
x

1
0
0
0
1 0 0
0
0
'
'
'
0
0
s
s
s
y
x
z
y
x
y
x
w
v
u

' / '
' / '
w v y
w u x
pix
pix
=
=
then
y y
x x
k f
k f
=
=

with
Image center
x
0
y
0
Transformation uses:
image center (x
0
, y
0
)
scaling factors k
x
and k
y
Internal Camera Parameters

1
0
0
0
1 0 0
0
'
'
'
0
0
s
s
s
y
x
z
y
x
y
x s
w
v
u

y y
x x
k f
k f
=
=

with

x
and
y
focal lengths in pixels
x
0
and y
0
coordinates of image center in pixels
Added parameter s is skew parameter
K is called calibration matrix. Five degrees of freedom.
K is a 3x3 upper triangular matrix
' / '
' / '
w v y
w u x
pix
pix
=
=
[ ]
3 3
0 | I K =

0 1 0 0
0 0 1 0
0 0 0 1
1 0 0
0
0
0
0
1 0 0
0
0
0
0
0
y
x s
y
x s
y
x
y
x

From Camera Coordinates

to World Coordinates
(X
s
, Y
s
, Z
s
)
Image point
(x
i
, y
i
, f)
x
z
C
y
I mage plane
Z
X
Y
(x
s
, y
s
, z
s
)
O
M
From Camera Coordinates
to World Coordinates 2
(X
s
, Y
s
, Z
s
) in world cordinate system
(x
s
, y
s
, z
s
) in image cordinate system
O
M
X
z
x
k
i
I
K
J
j
C
K J I k j i k j i
OM CO CM
S S S z y x S S S
Z Y X T T T z y x + + + + + = + +
+ =
K.i J.i I.i
S S S x S
Z Y X T x + + + =

S
S
S
z
y
x
S
S
S
Z
Y
X
T
T
T
z
y
x
K.k J.k I.k
K.j J.j I.j
K.i J.i I.i
Z
CO = T translation vector
Homogeneous Coordinates

S
S
S
z
y
x
S
S
S
Z
Y
X
T
T
T
z
y
x
K.k J.k I.k
K.j J.j I.j
K.i J.i I.i

1 1 1
S
S
S
z
y
x
S
S
S
Z
Y
X
T
T
T
z
y
x
0 0 0
K.k J.k I.k
K.j J.j I.j
K.i J.i I.i

1
1
1
S
S
S
S
S
S
Z
Y
X
z
y
x
T
3
0
T R
From Camera Coordinates
to World Coordinates 3
(X
s
, Y
s
, Z
s
)
(x
s
, y
s
, z
s
)
O
M
X
z
x
k
i
I
K
J
j
C
K ) - J ) - I ) - k j i
OC - OM CM
C S C S C S S S S
Z Z Y Y X X z y x ( ( ( + + = + +
=
K.i ) - J.i ) - I.i ) -
C S C S C S S
Z Z Y Y X X x ( ( ( + + =
) C - (X R x
cam
~
=
Z
) C -R (T
~
=
system coordinate in world expressed OC vector is C
~
Here we use instead of T
Homogeneous Coordinates 2

1
1
~
1
S
S
S
S
S
S
Z
Y
X
z
y
x
T
3
0
C R - R
C R -
~
Linear Transformation from
World Coordinates to Pixels

1
1
~
1
S
S
S
S
S
S
Z
Y
X
z
y
x
T
3
0
C R - R
[ ]

1
'
'
'
s
s
s
z
y
x
w
v
u
3 3
0 | I K
[ ]

1
1
~
'
'
'
S
S
S
Z
Y
X
w
v
u
T
3
3 3
0
C R - R
0 | I K

1
'
'
'
S
S
S
Z
Y
X
w
v
u
P
X P x =
Combine camera projection and coordinate
transformation matrices into a single matrix P
P has 11 degrees of freedom:
5 from triangular calibration matrix K, 3 from R and 3 from
P is a fairly general 3 x 4 matrix
left 3x3 submatrix KR is non-singular
Properties of Matrix P
[ ] X
0
C R - R
0 | I K x
T
3
3 3

=
1
~
[ ] [ ] [ ] C - | I R C R - R
0
C R - R
0 | I
3
T
3
3 3
~ ~
1
~
= =

[ ]X C - | I R K x
3
~
=
[ ] C - | I R K P
3
~
=
C
~
Further simplification of P:
Calibration
1. Estimate matrix P using scene points and
their images
2. Estimate the interior parameters and the
exterior parameters
Left 3x3 submatrix of P is product of upper-
triangular matrix and orthogonal matrix
[ ] C - | I R K P
3
~
=
Finding Camera Translation
Find homogeneous coordinates of C in the
scene
C is the null vector of matrix P
P C = 0:
Find null vector C of P using SVD
C is the unit singular vector of P corresponding to the
smallest singular value (the last column of V, where P
= U D V
T
is the SVD of P)
[ ] C - | I R K P
3
~
=

0
0
0
0
1
1 0 0
0 1 0
0 0 1
c
c
c
c
c
c
Z
Y
X
Z
Y
X
Finding Camera Orientation and
Internal Parameters
Left 3x3 submatrix Mof P is of form M=K R
K is an upper triangular matrix
R is an orthogonal matrix
Any non-singular square matrix Mcan be
decomposed into the product of an upper-
triangular matrix K and an orthogonal matrix R
using the RQ factorization
Similar to QR factorization but order of 2 matrices is
reversed
RQ Factorization of M
Compute
Multiply Mby R
x
. The resulting term at (3,2) is zero
because of the values selected for c and s
Multiply the resulting matrix by R
y
, after selecting c and
s so that the resulting term at position (3,1) is set to zero
Multiply the resulting matrix by R
z
, after selecting c and
s so that the resulting term at position (2,1) is set to zero

=
1 0 0
0 ' ' ' '
0 ' ' ' '
,
' 0 '
0 1 0
' 0 '
,
0
0
0 0 1
c s
s c
c s
s c
c s
s c
z y x
R R R
2 / 1 2
33
2
32
32
2 / 1 2
33
2
32
33
) (
,
) ( m m
m
s
m m
m
c
+
=
+
=
R K R R R K M K R R R M
T
x
T
y
T
z z y x
= = =
Computing Matrix P
Use corresponding image and scene points
3D points X
i
in world coordinate system
Images x
i
of X
i
in image
Write x
i
= P X
i
for all i
Similar problem to finding projectivity
matrix H (i.e. homography) in homework
Improved Computation of P
x
i
= P X
i
involves homogeneous
coordinates, thus x
i
and P X
i
just have to be
proportional:
Let p
1
T
, p
2
T
, p
3
T
be the 3 row vectors of P

=
i
T
3
i
T
2
i
T
1
i
X p
X p
X p
X P
0 =
i i
X P x

=
i
T
1 i i
T
2 i
i
T
3 i i
T
1 i
i
T
2 i i
T
3 i
i i
X p X p
X p X p
X p X p
X P x
' '
' '
' '
v u
u w
w v
0
' '
' '
' '
=

3
2
1
T
4
T
i i
T
i i
T
i i
T
4
T
i i
T
i i
T
i i
T
4
p
p
p
0 X X
X 0 X
X X 0
u v
u w
v w
vector 1 12 a is
p
p
p
3
2
1

Improved Computation of P,
contd
Third row can be obtained from sum of u
i
times first row - v
i
times second row
So we get 2 independent equations in 11
unknowns (ignoring scale)
With 6 point correspondences, we get
enough equations to compute matrix P
0
' '
' '
' '
=

3
2
1
T
4
T
i i
T
i i
T
i i
T
4
T
i i
T
i i
T
i i
T
4
p
p
p
0 X X
X 0 X
X X 0
u v
u w
v w
A p = 0
Solving A p = 0
Linear system A p = 0
When possible, have at least 5 times as
many equations as unknowns (28 points)
Minimize || A p || with the constraint
|| p || = 1
P is the unit singular vector of A corresponding
to the smallest singular value (the last column
of V, where A = U D V
T
is the SVD of A)
Called Direct Linear Transformation (DLT)
Improving P Solution with
Nonlinear Minimization
Find p using DLT
Use as initialization for nonlinear
minimization of
Use Levenberg-Marquardt iterative minimization
2
) , (
i
i
i
PX x

d
Radial Distortion
We have assumed that lines are imaged as
lines
Not quite true for real lenses
Significant error for cheap optics and for short
focal lengths
Scene
Image
Corrected image
Radial Distortion Modeling
In pixel cordinates the correction is written
Minimize
using lines known to be straight
(x,y) is radial projection of (x,y) on straight line
... 1 ) (
) ( ) (
) ( ) (
) ( ) (
2
2 1
2
0
2
0
2
0 0
0 0
+ + + =
+ =
=
=
r r r L
y y x x r
y y r L y y
x x r L x x
c
c

with
(x
0
, y
0
)
(x, y)
(x
c
, y
c
)

+ =
i
2 2
2 1
) ' ( ) ' ( ) , (
ci i ci i
y y x x f
(x, y)
(x, y)
Distorted line
And ideal line:
References
Multiple View Geometry in Computer Vision, R.
Hartley and A. Zisserman, Cambridge University
Press, 2000, pp. 138-183
Three-Dimensional Computer Vision: A
Geometric Approach, O. Faugeras, MIT Press,
1996, pp. 33-68
A Versatile Camera Calibration Technique for
3D Machine Vision, R. Y. Tsai, IEEE J. Robotics
& Automation, RA-3, No. 4, August 1987, pp.
323-344