83/183 Computer Vision, Winter 2023

Take-home Quiz 6
Due Date: Thursday Feb 23, 2023 23:59

Question 1 10 points
You will derive the Lucas-Kanade (or forward-additive) image alignment algorithm by repli-
cating the derivation learnt in the classs. Consider first a warp function W (x; p) that maps
coordinate vectors x ∈ R2 to other coordinate vectors in R2 , with the mapping depending
on a set of parameters p ∈ RN . Given an image I(x) and a template T (x), we want to find
the parameters p such that the warp W (x; p) best aligns the image with the template in
terms of sum-of-squared-differences (SSD) error. That is, we want to find the parameters p
that minimize the loss function:
X
min [I (W (x; p)) − T (x)]2 . (1)
p
x

The Lucas-Kanade alignment algorithm minimizes Equation (1) using the Gauss-Newton
algorithm. To this end, given some initial set of parameters p0 , they are updated iteratively
as:
pt+1 = pt + ∆pt , (2)
for t = 0, . . . , T , where the number of iterations T can be selected based on any of the
common convergence criteria. Then, the Gauss-Newton algorithm corresponds to selecting
a specific form for the update vector ∆pt , which you will derive below step-by-step.

1. Use the first-order Taylor expansion to linearize the composite function I (W (x; p))
with respect to p around the value pt . Write out the expression for this Taylor expan-
sion.

2. Combine the Taylor expansion expression with Equation (2), to obtain an approxima-
tion for I (W (x; pt + ∆pt )).

3. Show that, using this approximation, the optimization problem of Equation (1) can be
rewritten in the form: 2
mint A∆pt − b , (3)
∆p

for some matrix A and vector b.

4. Show how to solve the optimization problem of Equation (3) for the parameter update
∆pt , and write out an expression for this solution.

1
 5. Finally, explain how this expression for ∆pt can be evaluated, using image convolutions,
warps, element-wise operations, and matrix-vector operations. You can either explain
this in words or provide pseudocode, but make sure to explain each step clearly.

Question 2 10 points
In this question, you will derive the Baker-Matthews (or inverse compositional) image align-
ment algorithm for a simpler case. Consider first the following warp function that maps
coordinate vectors x ∈ R2 to other coordinate vectors in R2 :
  
p1 0 x
W (x; p) = , (4)
0 p2 y

where the mapping depends on a set of parameters p ∈ R2 . Given an image I(x) and a
template T (x), we want to find the parameters p such that the warp W (x; p) (representing
a scaling operation) best aligns the image with the template in terms of sum-of-squared-
differences (SSD) error.
The idea behind the Baker-Matthews alignment algorithm is to iteratively find incremen-
tal warp parameters ∆p that minimize the following objective:
X
min [T (W (x; ∆p)) − I (W (x; p))]2 . (5)
∆p
x

We will use the Gauss-Newton algorithm to solve this non-linear and non-parametric least
square problem to compute the incremental warp parameters ∆p.

1. The Baker-Matthews alignment algorithm assumes that W (x; 0) is the identity warp,
i.e., W (x; 0) = x. However, the current warp function in Equation (4) does not satisfy
this property. Propose a modified form of this scaling warp function that satisfies the
identity assumption and write out its corresponding Jacobian ∂W ∂p
. (Note that there
are multiple valid solutions for this part of the quiz.)

2. Use the first-order Taylor expansion to linearize the composite function T (W (x; ∆p))
with respect to ∆p around the value 0. Write out the expression for this Taylor
expansion. Plug this Taylor expansion expression back into Equation (5).

3. Show that this approximation to the optimization problem of Equation (5) can be
rewritten in the form:
min kA∆p − bk2 , (6)
∆p

for some matrix A and vector b. Show how to solve the optimization problem of Equa-
tion (6) for the parameter update ∆p, and write out an expression for this solution.

2
 4. Finally, you will need to compute your new warp parameters p0 by using the following
update rule for the inverse compositional algorithm:

W (x; p0 ) ← W (x; p) ◦ W (x; ∆p)−1 . (7)

Given your modified warp function from part 1, write down an expression for the new
parameters p0 in terms of p and ∆p.

Instructions
1. Integrity and collaboration: Students are encouraged to work in groups but each
student must submit their own work. If you work as a group, include the names of
your collaborators in your write up. Plagiarism is strongly prohibited and may lead to
failure of this course.

2. Questions: If you have any questions, please look at slack first. Other students may
have encountered the same problem, and it may be solved already. If not, post your
question on the discussion board. Teaching staff will respond as soon as possible.

3. Write-up: Your write-up should be typeset in LATEXand should consist of your answers
to the theory questions. Please note that we do not accept handwritten scans for your
write-up in quizzes.

4. Submission: Your submission for this take-home quiz should be a PDF file, <DartmouthID.pdf>,
with your write-up. Please do not submit ZIP files.