Attribution Non-Commercial (BY-NC)

22 views

Attribution Non-Commercial (BY-NC)

- Lecture 12
- final report
- aero
- 3D With Glasses
- Math 2280 - Exam 4
- 24_2_properties_fourier_trnsform.pdf
- Cognition
- Discrete Fourier Transform
- Lecture 05 Image Processing
- Matlab_3
- A 168 Final 1
- Final
- A Novel Convergence Control Method for Toed-In Stereo Camera
- Fourier
- 06184167
- DEPTH MAPstereo-Pair Video
- 32LM3400 User Guide
- Exp 8 Discussion
- Ghosh, d. p., 1971, Inverse Filter Coefficients for the Computation of Apparent Resistivity Standard Curves for a Horizontally Stratified Earth
- ME230_F14_Shabany

You are on page 1of 30

Szeliski

Review by: Christopher Rasmussen

Announcements

Homework due by midnight Next homework will be assigned Tuesday, due following Tuesday. It will involve tracking, specifically the Kalman filter, so look at Chap. 18-18.2 and handout for head start Some calendar changes

Project proposal due Oct. 10details soon Second paper presentations pushed back slightly

OS-based screen dump -> Crop Software: SnagIt (Windows), scrot (Linux), etc.

Main Contributions

Automatically construct image mosaics of

Planar surfaces under general camera motion General scene when camera motion is rotation about camera center

Estimating homography without manually selecting point correspondences

Find displacement that best registers one image with the other

Primary Techniques

Nonlinear minimization of SSD error function to estimate homography parameters Fourier phase correlation to estimate large camera motions Multiple planes/cylindrical coordinates to counteract large angle distortion for panoramic mosaicing Projective depth variables estimated for parallax motions

Let u = H u represent homography transformation of a point u (Szeliski uses M for H). As shorthand, write transformation of all image points as H I = I This is a linear system of equations; HW 1 was about solving the minimal form of it for known correspondences If we dont know any correspondences, what can we do? Idea: hypothesize H, transform image (using bilinear interpolation), see how good match is Szeliski uses SSD image similarity measure

Nonlinear Minimization

The SSD error function is nonlinear, so we cant use linear least-squares (e.g., SVD from homework) to solve for H Approach: Nonlinear least-squares to find the parameters of H which minimize the error function Levenberg-Marquardt is a standard technique (see Chap. 15.5 of Numerical Recipes for details) Initial guess: Identity homography Matlab: Pass error function to lsqnonlin after optimset('LevenbergMarquardt','on')

Image Compositing

With homography computed, how to render combined image? Simply putting one image on top of the other, even with bilinear interpolation, may result in a seam due to different brightness levels

Auto-iris can change overall lightness of images Vignetting can make image edges darker

Bilinear blending: Use hat function w indicating weight of contributions of both images to mosaic

w maximal at source image center, falls linearly to 0 at edges

Raw images

Final mosaic

For small overlaps, gradient descent/LM have problem with local minima in SSD error function. Possible solutions:

Hierarchical matching (i.e., image pyramid) Phase correlation (for even less overlap)uses Fourier transform

Also, geometry of panoramic mosaicing invalidates single plane assumption over large angular changes

Fourier Transform

Given a function transform , its Fourier is defined by:

Invertible decomposition of function (image) into waves of different frequencies (sines & cosines) Takes function from spatial domain to (complex) frequency domain

Fourier Theorem

The convolution of two functions is the same as the product of their Fourier transforms

Given that , we have

Helpful way to convolve efficiently (less so for small kernels) Fast Fourier Transform (FFT): Numerical method for computing Fourier transform; computational complexity = n log n.

Matlab: fft2; also, C library at fftw.org

Need to estimate shift between two images. This is like computing cross-correlation and finding peak Cross-power spectrum: Fourier transform of cross-correlation function

Suppose and

. Then:

Estimating Shift

Equivalently:

where the LHS is the cross-power spectrum The inverse Fourier transform of the RHS is , so find the location of the maximum and we are done

Extensions

The preceding analysis can be extended to handle rotation and scaling

See B. Reddy and B. Chatterji, An FFTBased Technique for Translation, Rotation, and Scale-Invariant Image Registration, IEEE Trans. Image Processing, Vol. 5, pp. 1266-1271, August, 1996.

Panoramic Mosaics

Rotation around camera center does not induce parallax Locally, images are coplanar However, just one plane becomes bad approximation over large angles Techniques

Tile sphere with planes, register images to local plane Register images to base frame; choose new base frame periodically Map to cylindrical coordinates before registering

Office

River

Depth Recovery

Want to estimate Z (depth) values for scene points Benefits of 3-D structure

Obstacle detection without recognition More information for recognition

Approaches

Piecewise planar: suggested but not explained Dense (per pixel)

Calculating depths

Known camera motion Stereo reconstruction Unknown camera motion Structure from motion

Projective depth

Definition: Parallax, or image shift, of point between two views after accounting for homography. Proportional to relative depth from scene plane inducing homography Not the same as traditional Euclidean depth

Approach: Decompose motion of scene points into two parts:

2-D homography (as if all points coplanar) Parallax: Discrepancy proportional to distance from plane

Apply nonlinear minimization as before to estimate new variables More next week in Motion lecture

from Hartley & Zisserman

Estimated depths

Synthesized view (wireframe)

Raw image

Estimated depths

Results

Looks good qualitatively, but no quantitative analysis Could mosaic large, known poster to compare pixel by pixel accuracy Projective depth recovery could have imaged known scene to measure accuracy

Limitations

Need scene to be highly-textured for accurate registration, projective depth estimation Sensitive to illumination variations, departures from pinhole model (i.e., radial distortion) Hat function for blending is ad-hoccould we actually calibrate images photometrically (i.e., for exposure, vignetting) and compensate precisely?

More Limitations

What if something in the scene is moving? Images only adjusted pairwise errors propagate. How about a global algorithm estimates alignments of all images simultaneously?

Connections

Techniques have wide applicability

Image registration useful for tracking, recognition Fourier phase correlation can be used for audio localizationi.e., shift is in time of sounds arrival at different microphones, which is proportional to distance, allowing triangulation, etc.

Making panoramic mosaics for recognition (instead of using omnidirectional camera) Mosaicing of sea floor by submersible Building ceiling mosaic for museum robot

Tracking planar patterns (e.g., signs, building facades, the ground from a UAV) over time Super-resolution for higher fidelity picturesCan we make face recognition, OCR, etc. more accurate? Acquiring piecewise-planar 3-D building models automatically

Questions?

- Lecture 12Uploaded byankurkothari
- final reportUploaded bymoonboy
- aeroUploaded bytn8584
- 3D With GlassesUploaded byjack2040
- Math 2280 - Exam 4Uploaded byDeokhyun Ko
- 24_2_properties_fourier_trnsform.pdfUploaded bysandeep
- CognitionUploaded byPradeep Kumar
- Discrete Fourier TransformUploaded byRakesh Yadav
- Lecture 05 Image ProcessingUploaded bykjain_92
- Matlab_3Uploaded bysumitr93
- A 168 Final 1Uploaded byLokesh Ramachandra
- FinalUploaded bysameera0627
- A Novel Convergence Control Method for Toed-In Stereo CameraUploaded byjavalino
- FourierUploaded byFikir Fikir
- 06184167Uploaded byManmatha Krishnan
- DEPTH MAPstereo-Pair VideoUploaded byNk Osuji
- 32LM3400 User GuideUploaded byBrian Stevenson
- Exp 8 DiscussionUploaded byMuhammad Noor Ikmal Zaidan
- Ghosh, d. p., 1971, Inverse Filter Coefficients for the Computation of Apparent Resistivity Standard Curves for a Horizontally Stratified EarthUploaded byCinthia Mtz
- ME230_F14_ShabanyUploaded byAlberto Lopez
- lec40Uploaded bydirgorambo
- ksyajnik ocean warming 2016.pdfUploaded bykisuya
- L51+L52_16BEE0158_AdityaMathur_Task1.pdfUploaded byAnonymous evrwQ8O6
- 2008 Generalized Peierls-Nabarro Model for Curved DislocationsUploaded byMartinAlfons
- optik 16Uploaded byz.umul9031
- 3dUploaded bySimran Saini
- peiper (1).pdfUploaded byJuniorYana
- SE Chem BT M III Notes NewUploaded byshriya
- Civil Engg Syllabus III-VIII sem 14.02.14.docxUploaded byichitha m.k
- latest Syllabus.pdfUploaded bysmit shah

- CN11PR001Rev4 - Electrical Installation WorksUploaded bymasahin
- Gradient of a VectorUploaded bydwarika2006
- Math SlideUploaded byRose Yacob
- Sample MCQs Updated With KeyUploaded byChui Wai
- Core 4 Revision NotesUploaded bybookdoudah
- 2010_Annual_ReportUploaded byGuillaume Ardoise
- bonding basics- ionic bondsUploaded byapi-244705156
- Sulfur FisherUploaded byRizky Ardias Darmawan
- Iodide and Bromide Ions in Brackish Water, Seawater, And Brines D 3869 – 04Uploaded bystevgon
- Computational Materials ScienceUploaded byJohn_Ada
- D5987-Standard Test Method for Total Fluorine in Coal and Coke by Pyrohydrolytic Extraction and Ion Selective Electrode or Ion Chromatograph Methods.pdfUploaded byAya
- winter break holiday home work class-10Uploaded byapi-326046013
- MkeUploaded byHerman Abanda
- Gusset PlatesUploaded byJason Garner
- Manual Panasonic CU C12KKV 7Uploaded byRicardo
- Wind and Ice Loading PresentationUploaded byLava Kumar Addepalli
- SATUAN GUploaded byWoro Indriani
- Computational Studies of Quantum Spin SystemsUploaded byJohn Bird
- SIMINES for ConveyorsUploaded bySlim Kanoun
- CLINIMED Titanium Pin & Wire CutterUploaded byClinimed, Incorporated
- Errata Solutions AcemogluUploaded byYeg Carlos
- Comparative Study On Talahib CompositesUploaded byKirsten Ignacio
- reportdraft 3Uploaded byShubham Jumde
- Satellite CommunicationsUploaded byAdnan Alam Khan
- D402D402M-14 Standard Test Method for Distillation of Cutback Asphalt - AASHTO No. T78Uploaded byAlabbas Fadhel
- Assessing College Students Understanding of Acid Base ChemistryUploaded byElsa Mahardika
- PbS 4Uploaded byapi-3753794
- sintap_BRITISH_STEEL_The_Significance_of_y-t_ratio.pdfUploaded byrahil_sang
- 77727631 or ToUploaded byJavier Farias Vera
- CaplioGX100_EUploaded byDan Foggin