Robert Collins

CSE486, Penn State

Lecture 16:
Planar Homographies
Robert Collins
CSE486, Penn State Motivation:
Points on Planar Surface

x
Robert Collins
CSE486, Penn State
Review : Forward Projection
World Camera Film Pixel
Coords Coords Coords Coords
U X x u
V Mext Y Mproj Maff
y v
W Z

U X Mint u
V Y v
W Z

U u
M
V m11 m12 m13 m14 v
W m21 m22 m23 m24
m31 m31 m33 m34
Robert Collins

World to Camera Transformation

CSE486, Penn State

PC PW
W

X
U
R Z V
C
Rotate to Translate by - C
align axes (align origins)

P C = R ( PW - C )
= R PW + T
Robert Collins
CSE486, Penn State

Perspective Matrix Equation

X (Camera Coordinates)
x= f
Z
y= f
Y X 
Z  x'  f 0 0 0  
 y ' =  0  Y
   f 0 0  
Z 
 z '   0 0 1 0  
1
p = M int ⋅ PC
Robert Collins
CSE486, Penn State
Film to Pixel Coords
2D affine transformation from film
coords (x,y) to pixel coordinates (u,v):
X 
u’ a11 a12 xa' 13f 0 0 0  
    Y
v’ a21 a22 ya' 23 = 0
   f 0 0  
w’ Z 
0 0  z 1'   0 0 1 0  
1
Maff Mproj

u = Mint PC = Maff Mproj PC

Robert Collins

Projection of Points on Planar Surface

CSE486, Penn State

Perspective
projection
y
Film coordinates
x

Point
on plane
Rotation +
Translation
Robert Collins
CSE486, Penn State
Projection of Planar Points
Robert Collins

Projection of Planar Points (cont)

CSE486, Penn State

Homography H
(planar projective
transformation)
Robert Collins

Projection of Planar Points (cont)

CSE486, Penn State

Homography H
(planar projective
transformation)

Punchline: For planar surfaces, 3D to 2D perspective

projection reduces to a 2D to 2D transformation.

Punchline2: This transformation is INVERTIBLE!

Robert Collins
CSE486, Penn State
Special Case : Frontal Plane
What if the planar surface is perpendicular to
the optic axis (Z axis of camera coord system)?

Then world rotation matrix simplies:

Robert Collins
CSE486, Penn State
Frontal Plane

So the homography for a frontal plane simplies:

Similarity Transformation!
Robert Collins
CSE486, Penn State
Convert to Pixel Coords
Internal camera a11 a12 a13 v pixels
parameters a21 a22 a23
0 0 1
u
Robert Collins

Planar Projection Diagram

CSE486, Penn State

Here’s where
transformation
groups get useful!
Robert Collins

General Planar Projection

CSE486, Penn State

H1

H
A1
Robert Collins

General Planar Projection

CSE486, Penn State

H1

H-1
A1
Robert Collins

Frontal Plane Projection

CSE486, Penn State

S1

A
A1
Robert Collins

Frontal Plane Projection

CSE486, Penn State

S1

A-1
A1
Robert Collins

General Planar Projection

CSE486, Penn State

H1 H2

H
Robert Collins
CSE486, Penn State
Summary: Planar Projection
Pixel coords
Internal u
params

Perspective v
projection
y
Homography
x

Point on plane
Rotation + Translation
Robert Collins
Applying Homographies to Remove
CSE486, Penn State

Perspective Distortion

from Hartley & Zisserman

4 point correspondences suffice for

the planar building facade
Robert Collins
CSE486, Penn State
Homographies for
Bird’s-eye Views

from Hartley & Zisserman

Robert Collins
CSE486, Penn State
Homographies for Mosaicing

from Hartley & Zisserman

Robert Collins
CSE486, Penn State
Two Practical Issues

How to estimate the homography given

four or more point correspondences
(will derive L.S. solution now)

How to (un)warp image pixel values to

produce a new picture (last class)
Robert Collins
CSE486, Penn State
Estimating a Homography
Matrix Form:

Equations:
Robert Collins
CSE486, Penn State
Degrees of Freedom?

There are 9 numbers h11,…,h33 , so are there 9 DOF?

No. Note that we can multiply all hij by nonzero k
without changing the equations:
Robert Collins
CSE486, Penn State
Enforcing 8 DOF
One approach: Set h33 = 1.

Second approach: Impose unit vector constraint

Subject to the
constraint:
Robert Collins
CSE486, Penn State
L.S. using Algebraic Distance
Setting h33 = 1

Multiplying through by denominator

Rearrange
Robert Collins
CSE486, Penn State
Algebraic Distance, h33=1 (cont)
2N x 8 8x1 2N x 1
Point 1

Point 2

Point 3

Point 4

additional
points
Robert Collins
CSE486, Penn State
Algebraic Distance, h33=1 (cont)
Linear 2Nx8 8x1 2Nx1
equations A h = b

8x2N 2Nx8 8x1 8x2N 2Nx1

Solve:
AT A h = AT b
8x8 8x1 8x1

(AT A) h = (AT b)
-1
h = (AT A) (AT b)

Matlab: h = A\b
Robert Collins
CSE486, Penn State
Caution: Numeric Conditioning

R.Hartley: “In Defense of the Eight Point Algorithm”

Observation: Linear estimation of projective transformation

parameters from point correspondences often suffer from poor
“conditioning” of the matrices involves. This means the solution
is sensitive to noise in the points (even if there are no outliers).

To get better answers, precondition the matrices by performing

a normalization of each point set by:
• translating center of mass to the origin
• scaling so that average distance of points from origin is sqrt(2).
• do this normalization to each point set independently
Robert Collins
CSE486, Penn State
Hartley’s PreConditioning
PointSet1 H = T2-1 S2H?
-1 H
norm S1 T1 PointSet2

Translate
T1 center of mass T2
to origin

Scale so average
S1 point dist is sqrt(2) S2

Estimate Hnorm
Robert Collins
CSE486, Penn State
A More General Approach

What might be wrong with setting h33 = 1?

If h33 actually = 0, we can’t get the right answer.

Robert Collins
CSE486, Penn State
Algebraic Distance, ||h||=1
||h|| = 1

Multiplying through by denominator

Rearrange
=0
=0
Robert Collins

Algebraic Distance, ||h||=1 (cont)

CSE486, Penn State

2N x 9 9x1 2N x 1

4
P
O
I
N
T
S

additional
points
Robert Collins

Algebraic Distance, ||h||=1 (cont)

CSE486, Penn State

Homogeneous 2Nx9 9x1 2Nx1

equations A h = 0
9x2N 2Nx9 9x1 9x2N 2Nx1
Solve:
AT A h = AT 0
9x9 9x1 9x1

(AT A) h = 0
SVD of ATA = U D UT

Let h be the column of U (unit eigenvector)

associated with the smallest eigenvalue in D.
(if only 4 points, that eigenvalue will be 0)