Attribution Non-Commercial (BY-NC)

40 views

Attribution Non-Commercial (BY-NC)

- Finite Element Method for Professionals
- Image Contrast Enhancement using Combination of DWT & SWT with CLAHE
- Lecture 4
- Back Propagation
- Introduction to the Parareal Algorithm
- Seminar ppt of a maxflow problem
- Assignments
- Iterative rial Auctions
- Chapter 4
- Bode Rules
- Big m Method
- Identification of Humans Using Gait (Synopsis)
- COMPUSOFT, 3(6), 825-830.pdf
- Course SGT 3 - DFT-based Synchrophasor Estimation Algorithms - IpDFT Rev2
- Analysis of Pain Hemodynamic Response Using Near-Infrared Spectroscopy (Nirs)
- 3-2
- Thermal-mechanical Metal-Forming Simulations in LS-DYNA Revisited at the 5th German LS-DYNA Forum in Ulm
- document(12).pdf
- Clustering - Fuzzy C-means
- Week 11

You are on page 1of 4

David Kriegman E-Mail :

(Due date: 3/16/07)

Overview

In this assignment you will implement the Lucas-Kanade optical flow algorithm. Your input will be pairs or sequences

of images and your algorithm will output an optical flow field (u, v). Three sets of test images are available from the

course website. The first contains a synthetic (random) texture, the second a rotating sphere 1 , and the third a corridor

at Oxford university2. Before running your code on the images, you should first convert your images to grayscale and

map intensity values to the range [0, 1].

Implement the single-scale Lucas-Kanade optical flow algorithm. This involves finding the motion (u, v) that min-

imizes the the sum-squared error of the brightness constancy equations for each pixel in a window. As a reference,

you should read pages 191-198 in “Introductory Techniques for 3-D Computer Vision” by Trucco and Verri 3 . Your

algorithm will be implemented as a function with the following inputs,

function [u,v] = optical_flow(I1,I2,windowSize).

Here, u and v are the x and y components of the optical flow, I1 and I2 are two images taken at times t = 1

and t = 2 respectively, and windowSize is a 1 × 2 vector storing the width and height of the window used during

flow computation. In addition to these inputs, you should add a theshold τ such that if τ is larger than the smallest

eigenvalue of AT A, then the the optical flow at that position should not be computed. Recall that the optical flow is

only valid in regions where

P 2 P

AT A = P Ix P Ix2Iy (1)

Iy Ix Iy

has rank 2 (why?), which is what the threshold is checking. A typical value for τ is 0.01; if you use something different

please specify this in your report.

1 Courtesy of http://www.cs.otago.ac.nz/research/vision/Research/OpticalFlow/opticalflow.html

2 Courtesy of the Oxford visual geometry group

3 Available from the course webpage.

1

What to turn in for this section

• Your matlab code.

• Quiver plots of (u, v) for the following image pairs / window sizes (6 quiver plots total),

– SYNTH : I1 = ’synth 000.pgm’, I2 = ’synth 001.pgm’

windowSize = 9x9

windowSize = 15x15

– SPHERE : I0 = ’sphere.0.ppm’, I1 = ’sphere.1.ppm’

windowSize = 15x15

windowSize = 21x21

– CORRIDOR : I0 = ’bt.000.pgm’, I1 = ’bt.001.pgm’

windowSize = 15x15

windowSize = 21x21

• Any modifications and/or thresholds you used.

• Does this algorithm work well for these test images? If not, why?

• Try experimenting with different window sizes. What are the tradeoffs associated with using a small vs. a large

window size? Can you explain what’s happening?

Modify your algorithm to iteratively refine the optical flow estimate from multiple image resolutions. The basic idea is

to initially compute the optical flow at a coarse image resolution. The obtained optical flow can then be upsampled and

refined at successively higher resolutions, up to the resolution of the original input images. By computing the optical

flow in this manner, we can circumvent the aperature problem to some degree – because the window size remains

fixed across all resolutions, the algorithm effectively computes the optical flow using successively smaller aperatures.

This approach also works well on images with large displacements, something the single-scale algorithm is unable to

handle.

In terms of implementation, you should first modify your code from part A so that it accepts and refines parameters

u0 and v0, which correspond to initial estimates of the optical flow,

function [u,v] = optical_flow_refine(I1,I2,windowSize,u0,v0).

This function should refine the optical flow according to,

u = u0 + 4u (2)

v = v0 + 4v (3)

where 4u and 4v represent the offset between the initial estimate and the refined estimate. To compute 4u and

4v we simply shift the window in I2 by (u0 , v0 ) and compute the optical flow between the shifted windows. Once

optical_flow_refine is working you should write another function,

which calls the refinement function. optical_flow_ms should implement the following pseudo-code,

Step 1. Gaussian smooth and scale I1 and I2 by a factor of 2ˆ(1-numLevels)

Step 2. Compute the optical flow at this resolution

Step 3. For each level,

a. Scale I1 and I2 by a factor of 2ˆ(1-level)

b. Upscale the previous level’s optical flow by a factor of 2

c. Compute u and v by calling optical_flow_refine with the previous

level’s optical flow

2

What to turn in for this section

• Your code

• Quiver plots of (u, v) for the following image pairs,

– SPHERE : I0 = ’sphere.0.ppm’, I1 = ’sphere.1.ppm’

windowSize = 15x15, numLevels=3

windowSize = 21x21, numLevels=3

– CORRIDOR : I0 = ’bt.000.pgm’, I1 = ’bt.001.pgm’

windowSize = 7x7, numLevels=5

windowSize = 15x15, numLevels=3

• Any modifications and/or thresholds you used.

• Did this improve results? How and why?

1. Email Neil the code and report (nalldrin AT cs.ucsd.edu)

2. Hand in a report which contains

(a) Hardcopy of code

(b) Report which includes the output described above. Also, include a short description of your conclusions

based on your experience.

Implementation Tips

Useful Matlab Commands

I’m providing the following list of Matlab hints to (hopefully) reduce debug time. Note that some of these commands

require the image processing toolbox.

• Spatial gradient.

The matlab command gradient will compute the gradient of an image for you. For example,

[Ix Iy] = gradient(I);

• Gaussian smoothing.

The following code snippet will smooth an image with a σ = 1.5 Gaussian kernel,

H = fspecial(’gaussian’,[5 5],1.5);

I = conv2(I,H,’same’);

• Quiver plots.

The matlab command quiver can be used to generate quiver plots. However, because Matlab uses a different

coordinate system for images than for other types of data, getting quiver to correctly plot u and v can be tricky.

To save you time, here’s a code snippet that will generate a nice looking quiver plot to display your optical flow

field.

function quiver_uv(u,v)

scalefactor = 50/size(u,2);

u_ = scalefactor*imresize(u,scalefactor,’bilinear’);

v_ = scalefactor*imresize(v,scalefactor,’bilinear’);

% and scaling the magnitude of (u,v) by 2 so it is more visible.

quiver(u_(end:-1:1,:),-v_(end:-1:1,:),2);

axis(’tight’);

3

• Image resizing.

The matlab command imresize will do the trick. To avoid aliasing effects, make sure you specify either

bilinear or bicubic filtering.

• Padding an image with zeros.

You may find it useful at times to pad an image or matrix with zeros on all sides (to avoid boundary problems,

for example). The padarray function greatly simplifies this task.

• In part B, when upsampling u and v, the values also need to be scaled (why?).

• Accidentally switching x and y indices.

• Image boundary issues. There are many ways to handle the image boundary. The simplest methods are either to

pad the input images with zeros or to only compute the optical flow for regions where the neighborhood window

is well defined. A third approach that may yield slightly better results is to only use portions of the neighborhood

window that lie inside the original images. Any of these approaches is perfectly fine for this assignment.

- Finite Element Method for ProfessionalsUploaded byAnonymous lV8E5mEO
- Image Contrast Enhancement using Combination of DWT & SWT with CLAHEUploaded byIJRASETPublications
- Lecture 4Uploaded bysri
- Back PropagationUploaded byModick Basnet
- Introduction to the Parareal AlgorithmUploaded byTina Bargo
- Seminar ppt of a maxflow problemUploaded bynivedi
- AssignmentsUploaded byaddbaz
- Iterative rial AuctionsUploaded bydphvu
- Chapter 4Uploaded bySMBEAUTY
- Bode RulesUploaded bysandertandenborstel
- Big m MethodUploaded byVenkata Dinesh
- Identification of Humans Using Gait (Synopsis)Uploaded byMumbai Academics
- COMPUSOFT, 3(6), 825-830.pdfUploaded byIjact Editor
- Course SGT 3 - DFT-based Synchrophasor Estimation Algorithms - IpDFT Rev2Uploaded byPera Kojot
- Analysis of Pain Hemodynamic Response Using Near-Infrared Spectroscopy (Nirs)Uploaded byIJMAJournal
- 3-2Uploaded byksr131
- Thermal-mechanical Metal-Forming Simulations in LS-DYNA Revisited at the 5th German LS-DYNA Forum in UlmUploaded byIstvan Wunderlich
- document(12).pdfUploaded byhairi zainal
- Clustering - Fuzzy C-meansUploaded bySunilAjmeera
- Week 11Uploaded byZhi Ming Cheah
- Nichols chart_IESL.pdfUploaded byBatuhan Mutlugil 'Duman'
- Finite Difference MethodsUploaded byPeerawatWongsurakul
- Thresholding based R-Peak Detection in ECG SignalsUploaded byIJSTE
- Canonical sUploaded byObada Zezo
- Lecture 5 (Signal-Flow-Graphs).pdfUploaded byMomenNaeem
- SweetFX Settings INSURGENCY ActionMovieMod VibrantDark,Better LightUploaded byNicolaLamanna
- Knapsack algorithm java implementationUploaded byAkash
- 01_IntroUploaded bysandeep_2262
- Project ReportUploaded byshruti
- AalUploaded byDewi Alfiatus

- CSU_MTH182_Summer16_1.2585948.HW04Uploaded byHaardikGarg
- Time HistoryUploaded byshivamomshanti
- Rouche's TheoremUploaded byJeoff Libo-on
- Introduction to Robotics Mechanics and Control 4th Edition Craig Solutions ManualUploaded bya390573370
- La 07 Matrix Decomposition sUploaded byCaley Jousie Amin
- History of ScienceUploaded byguicortei
- NumPyMarkovChainsExamplePartIIIUploaded byivan2
- 11-FJDS-01602-087.pdfUploaded byAngelo Zevallos
- P105A-16-10-syllabusUploaded byChristopherMason
- Rigid Body DynamicsUploaded by29011988
- Matching (Graph Theory)Uploaded byMohit Gupta
- Oc Curve BasicsUploaded byRohit Soni
- vectors2-1Uploaded bykid
- Lectures on Fourier Analysis by S.ThangaveluUploaded byChowdaiah
- Statefb AbiramUploaded byumairnaeem_90
- designul formelorUploaded byCornel Stăvărache
- ROC z TransformUploaded byMohammad Gulam Ahamad
- LSPDE5Uploaded byKevin Dany
- runge-kuttaUploaded byTaimoor Afzal Kiani
- 3SATUploaded byCezar Mocan
- 39_1_norm_distUploaded byEbookcraze
- Alg Linear Axplusbeqcvar AllUploaded byNurul
- c 2 Zigzag MocksUploaded bywizardzx2
- Interior Point MethodsUploaded bySachin
- Artificial Neural Networks an Econometric PerspectiveUploaded byHector Aguilar Torres
- Vector SpacesUploaded byumer
- uq-m1Uploaded byLokesh Darling
- (Bakker & Pekoz,2003)-The FEM for Thin-walled Members-basic PrinciplesUploaded bychristos032
- Biaxial BendingUploaded byJose Manuel
- Master TheoremUploaded byMoamen Said

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.