Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Download
Standard view
Full view
of .
Look up keyword
Like this
1Activity
0 of .
Results for:
No results containing your search query
P. 1
Photometric Stereo via Computer Screen Lighting

Photometric Stereo via Computer Screen Lighting

Ratings: (0)|Views: 74|Likes:
Published by Jamie Allen

More info:

Published by: Jamie Allen on Oct 25, 2011
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

10/25/2011

pdf

text

original

 
Photometric Stereo via Computer Screen Lightingfor Real-time Surface Reconstruction
Grant SchindlerGeorgia Institute of TechnologyCollege of ComputingAtlanta, GA, USAschindler@cc.gatech.edu
Abstract
We introduce a method which uses the light emitted bya computer screen to illuminate an object such as a human face from multiple directions, simultaneously capturing im-ages with a webcam in order to perform photometric stereo. Dominant eigenvectors of the captured images provide sur- face normals, which are integrated into a 3D surface usingGauss-Seidel relaxation. The system runs at 10 frames per second on a consumer laptop computer.
1 Introduction
Despite the existence of numerous methods for captur-ing3Dmodels, someofwhichrequireonlyaprojectorandacamera, most home computer users lack the necessary hard-ware setup to make such methods widely practical for any-one but professionals. We introduce a method which usesthe light emitted by a computer screen to illuminate an ob- ject such as a human face from multiple directions, simul-taneously capturing images with a webcam in order to per-form photometric stereo. The significance of this methodis that it relies only on widely adopted consumer hardware,a computer screen and a webcam, both of which are stan-dard components sold with millions of laptop and desktopcomputer systems in the past few years. Thus, with soft-ware alone, we can bring 3D model acquisition systems intohomes across the world, leading to previously unforeseenuses of the technology. Furthermore, the method works inreal-time, enabling applications like 3D video conferenc-ing which previously would have required specialized hard-ware.The two primary contributions of this work are: (1)a software-based photometric stereo solution that exploitscommodity hardware already present in millions of homesacross the world and (2) a real-time photometric stereo
Figure 1. Using only the light from a com-puter screen to illuminate a face from mul-tiple directions (top), we use photometricstereo techniques to recover a textured 3Dsurface (bottom) in real-time from live video.This technique is viable on any laptop ordesktop computer with a screen and web-cam.
method for capturing and displaying 3D models in real-time.
1.1 Related Work
Photometric stereo [18] is a shape-from-shading tech-nique first introduced by Woodham [18] to estimate surfacecurvature from multiple images of an object under varyingillumination. It was later realized that images under vary-ing illumination lie in a low-dimensional subspace [17][10],
 
Figure 2. Here we see binary lighting patterns for
=
4
images (left) and a multispectral lightingpattern for
=
3
images (right). Note that in the standard case on the left, the lighting patternsare separated in time, while in the multispectral case on the right, the patterns are separated inwavelength, allowing three images to be captured simultaneously.
and that therefore surface normals could be extracted di-rectly from input images via Singular Value Decomposition(SVD) [13][9][20]. These surface normals can be used notonly to relight the scene [2] but also to reconstruct the 3Dsurface itself [1].Various lighting hardware setups have been proposedfor photometric stereo systems. Multispectral photomet-ric stereo [19][14] systems have been used to simultane-ously capture multiple illuminations from differently col-ored lights. In addition, real-time photometric stereo sys-tems [15] running on a GPU have been used for inspectingobjects using specialized multi-head lamps. Large outfittedenvironments like the light stage [8] have also been used tocapture extremely high quality reflectance information fora human or other object sitting at the center of a sphere of programmable lights. A number of techniques use projec-tors to illuminate a scene in a structured light [16][3][21]scenario. In a structured light setup, dense 3D models areacquired by projecting known high-resolution light patternsonto the surface an object and observing the surface withone or more cameras. In [3] a simple structured light setupinvolving a lamp and a shadow-casting pencil was used for3D model acquisition on the desktop.Several recent papers [12, 6] explore the idea of us-ing computer screen illumination to perform photometricstereo. In [6], a theoretical justification is provided for us-ing nearby rectangular illuminants rather than distant pointlight sources for photometric stereo. In [12], photometricstereo is performed in an enclosed apparatus using a com-puter screen as a light source. Compared to our work, [12]uses a different strategy for activating screen pixels whichnecessitates long camera exposure times of 10 to 30 sec-onds per image. In addition, the environment matting work of [4] uses video of an object in front of a computer screento model the light-transport paths through the object.Finally, the idea of using a computer screen as a lightsource has several precedents outside of the computer vi-sion and graphics literature. In the field of biochemistry,Filippini et al. [11] developed the computer screen photo-assisted technique (CSPT) which treats a computer screenas a programmable light source in determining the visibleabsorption spectra of samples placed in front of the screen.Commercially, the Photo Booth software bundled with Ap-ple brand computers [7] simulates a camera flash by brieflyturning all pixels on the computer screen completely whitewhen capturing static images.
2 Photometric Stereo Approach
Photometric stereo is a shape-from-shading techniquewhich takes as input a number of images of a static sceneilluminated from various directions by a point light source.Assuming a lambertian reflectance function, the varying re-sponses of individual pixels across the multiple images en-ablestherecoveryofthesurfacenormalateachpixel. Thesesurface normals are then integrated to recover a 3D surface,represented as a depth map or a height field. We discusseach of these steps in detail below.
2.1 Computer Screen Lighting
We capture
images
{
 I 
 j
:
j
=
1
..
 N 
}
of an object, eachunder a different set of lighting conditions
L
 j
. The lightingconditions
L
 j
are defined by the set of illuminated pixels onthe computer screen as follows:
 L
 j
(
 x
,
 y
) =
sign
(
 x
·
cos
(
2
π 
j N 
)+
 y
·
sin
(
2
π 
j N 
))
 
Figure 3. Surface Normals and Depth Map. Above we see, from left to right, the
z
,
x
, and
y
compo-nents of the surface normal vectors, the depth map that results from Gauss-Seidel relaxation, anda perspective view of a uniform mesh with vertices displaced according to the depth map. The fourinput images from which these results were computed can be seen in Figure 2.
where values
1 and 1 map to black and white pixels re-spectively and coordinate
(
0
,
0
)
is at the center of the com-puter screen. This formulation leads to a rotating set of bi-nary images as depicted in Figure 2.
2.2 Recovering Surface Normals
Following a standard approach [2], we treat each im-age
 j
(of width
and height
) as a 1-dimensional col-umn vector of intensity values of size
W
×
1 and placethese
column vectors side by side into a matrix
A
of size
W
×
 N 
. It is well established [20] that the images live in a3-dimensional subspace defined by the eigenvectors corre-sponding to the three largest eigenvalues of the matrix
AA
.These eigenvectors, which can be extracted using SVD, cor-respond to the
z
,
 x
, and
y
components of the surface normalat every pixel in the image (see Figure 3).
2.3 Surface Reconstruction
Though there exist numerous methods of reconstructinga depth map from surface normals [1], we here adopt acoarse-to-fine Gauss-Seidel relaxation approach similar to[14]. In this incremental approach we update the depth esti-mate at each pixel based on measured surface normals andpreviously estimated values for the pixel’s neighbors in thedepth map. Starting from an initially flat depth map, thisapproach converges in less than 20 iterations (see Figure 3).We show in the next section why this method is even morebeneficial in a real-time setting using an incrementally up-dated depth map.
3 Real-time Photometric Stereo
The primary obstacles to real-time photometric stereoare the time it takes to capture images under varying light-ing conditions and the computational costs associated withrecovering surface normals from images, and depth fromsurface normals. We address all three of these issues byadopting an approach that lets us compute continuous in-cremental updates to the model as we go, rather than re-computing the entire model at each step. We address eachof these issues in turn.
3.1 Real-time Image Capture
We do not want to wait for a new set of 
images tobe captured before updating the 3D model during real-timephotometricstereo. Insteadwecaptureimagescontinuouslyusing the following scheme: At time
, we capture image
 j
using light pattern
L
 j
where
j
=
t modulo N 
. Thus, at anytime, we always have the most recently captured
imagesin memory, with each successive image
 j
overwriting thepreviously captured image in position
j
. Every time we geta single new image, we can recompute the 3D model as de-scribed below. Another solution to the capture-speed prob-lem is to use a multi-spectral approach, which we discuss inSection 3.4.
3.2 Real-time Surface Normals
It is well known that the expensive SVD computation of the
W
×
W
matrix
AA
can be avoided by instead com-puting the SVD of the much smaller
×
 N 
matrix
A
 A
.However, the construction of 
A
 A
itself is computationallyexpensive as it involves taking the dot product of every im-age with every other image – an
O
(
WHN 
2
)
operation. Wemake two key observations. First, in our incremental cap-ture scheme, only one of the
images is updated at eachtimestep, meaning that only 2
 N 
1 of the entries need to beupdated. Furthermore, the matrix
A
 A
is of course symmet-ric, so that of its
2
entries, only
entries must be updatedevery time a new image comes in – an
O
(
WHN 
)
operation.Under an assumption of small motion, the resulting sur-face normals will correctly describe the surface under ob-servation. However, even in the presence of temporary largemotion which violates this assumption, the model will havecaught up with reality in a maximum of 
frames when a

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->