You are on page 1of 5

THROUGH WALL IMAGING: HISTORICAL PERSPECTIVE AND FUTURE DIRECTIONS

Dr. Edward J. Baranoski

DARPA/STO
3701 N. Fairfax Drive
Arlington, VA 22203 USA
ed.baranoski@darpa.mil

ABSTRACT multilateration is to correlate range measurements from
multiple sensors to specific points in the image. With
Through wall imaging is highly desirable for police, fire and sufficient spatial diversity from a large set of
rescue, first responder, and military applications. The transmit/receive combinations, specific reflection points will
ultimate desire of such system is to provide detailed start to integrate above the background interference.
information in areas that cannot be seen using conventional However, ambiguities will arise as the number of reflection
measures. Borrowing from successes in geological and points increases. This can provide an overdetermined
medical imaging environments, researchers are applying system relative to the transmit/receive signal pairs which
radio frequency (RF) and other sensing modes to penetrate can detract from the quality of imaging products.
wall materials and make intelligent decisions about the Synthetic aperture radar (SAR) can be thought of as a
contents of rooms and buildings. For this application, they coherent extension of the multilateration concept. Instead
are many propagation differences that provide unique of incoherent combinations of range returns from multiple
challenges that must be addressed to make through wall transmit/receive pairs, coherent algorithms are used to
penetration sensors operationally viable. This paper provide a complex matched filter to specific points in the
outlines the historical context of early research as well as target space. This technique generally assumes free-space
providing new directions for future research in this exciting propagation to each point in the target scene, although
interplay between electromagnetic propagation, signal platform motion compensation and atmospheric effects are
processing, and knowledge-based reasoning algorithms. often removed with autofocusing algorithms. SAR
approaches usually neglect propagation distortions such as
Index Terms— Microwave imaging, Electromagnetic those encountered by signals passing through walls and
propagation, Radar, Electromagnetic tomography objects. These distortions degrade the performance and can
lead to ambiguities in localization and understanding of the
1. INTRODUCTION sensor data.
Free-space assumptions no longer apply after the
Through wall sensing addresses the desire to see inside electromagnetic waves propagate through the first wall.
structure to determine the layout of buildings, where This provides unique challenges and opportunities for
occupants may be, and even identify objects within exploiting building dependent features. Rather than using
buildings. Through wall sensing grew from ground free-space focusing assumptions, propagation effects can be
penetrating radar systems applied to walls, and specific included in the imaging solution. Practical systems will
applications have been increasingly documented in the need to unravel several layers of distortion to unscramble
literature since the late 1990’s showing abilities to sense the waveforms to correctly interpret the physical scene
beyond a single wall from near-range [1]-[7]. causing the waveform degradation. Free-space
These approaches have generally borrowed from approximations may carry imaging systems through to the
traditional optical, radar, and sonar image processing first wall, but propagation effects will then affect further
techniques, which begin with basic wave physics to form imaging results. Shadowing, attenuation, multipath,
matched filters for every point in the imaging target space. refraction, diffraction, and dispersion all play a role in how
In true free space conditions, this represents a the signals will propagate after the first interface. Without
mathematically accurate way to perform imaging. Imaging factoring in these effects, imaging of contents with
of structure features and contents of buildings requires 2-D buildings will be severely impacted. Uncompensated
and preferably 3-D systems. It cannot rely on Doppler refraction through walls can lead to localization or focusing
processing for separation of desired features, so errors, leading to image offsets and blurring [9]. Bragg
multilateration or SAR approaches have been the most scattering off repeating structural elements such as rebar in
common approaches. The general idea behind concrete walls or repetitive voids in concrete block walls

U.S. Government Work Not Protected by U.S. Copyright 5173 ICASSP 2008

Presented at the IEEE International Conference on Acoustics. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. 0704-0188 Public reporting burden for the collection of information is estimated to average 1 hour per response. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release. ABSTRACT see report 15.22203 9. including the time for reviewing instructions. Government or Federal Purpose Rights License. THIS PAGE Same as 4 unclassified unclassified unclassified Report (SAR) Standard Form 298 (Rev. no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. LIMITATION OF 18. Held in Las Vegas. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. U.3701 N. PROGRAM ELEMENT NUMBER 6. SPONSOR/MONITOR’S ACRONYM(S) 11. Directorate for Information Operations and Reports. NAME OF ABSTRACT OF PAGES RESPONSIBLE PERSON a. ABSTRACT c. 1. 8-98) Prescribed by ANSI Std Z39-18 . TASK NUMBER 5f. DATES COVERED 2. SECURITY CLASSIFICATION OF: 17. SUPPLEMENTARY NOTES See also ADM002091. and completing and reviewing the collection of information. Respondents should be aware that notwithstanding any other provision of law. Form Approved Report Documentation Page OMB No. Fairfax Drive. AUTHOR(S) 5d.S.DARPA/STO. 2008. SPONSOR/MONITOR’S REPORT NUMBER(S) 12. CONTRACT NUMBER Through Wall Imaging: Historical Perspective and Future Directions 5b. and Signal Processing (ICASSP 2008).Arlington. including suggestions for reducing this burden. TITLE AND SUBTITLE 5a.VA. gathering and maintaining the data needed. Arlington VA 22202-4302. Suite 1204. SUBJECT TERMS 16. to Washington Headquarters Services. Send comments regarding this burden estimate or any other aspect of this collection of information. REPORT TYPE APR 2008 00-00-2008 to 00-00-2008 4. REPORT DATE 3. distribution unlimited 13. GRANT NUMBER 5c. 14. Speech. searching existing data sources. WORK UNIT NUMBER 7. 1215 Jefferson Davis Highway. Nevada on March 30-April 4. REPORT b. NUMBER 19a. PERFORMING ORGANIZATION REPORT NUMBER Defense Advanced Research Projects Agency. PROJECT NUMBER 5e.

x˾)} 5. All of this must be factored into the propagation continue until acceptable convergence is obtained. for projection data through solid materials. Update model with new features x˾ 4. Predict output due to model and compare with data material 6. reduces resolution.x˾) Gx General approach: Propagation Imaging/ 3-D Building 1. and its contents. the This approach is a major focus of the DARPA VisiBuilding model-based reasoning will require analyzing the expected x y=H(p. sensor positioning and utilization to techniques will perform the proper wavefront corrections maximize information about the building. Tomographic 3-D building deconvolution that operates in a multipath- algorithms are capable of making some of these adjustments rich. A sufficient architecture will therefore be dependent on These effects may be partially corrected using repeated three technical areas: phenomenology of signal penetration application of image focusing techniques [8]. Initial physical components of the structure being examined. Structural details need to be model hypotheses [8]. These into buildings.x) Gp x˾ GH˾(p. air gaps in concrete determine why sensed data differ from predicted data. It requires propagation effects and modeling of the environment so that knowledge of the propagation interaction between the the system can sense deeper within the buildings. wavefronts. then constraints. In this way. Determine dominant features 3. Use baseline p0 to maximize Prediction Inferencing Model initial information 2. tomographic projection approaches are well suited for 2. component by component. important to operational tasks.) may provide less distortion on the RF the model can be updated accordingly. Frequency choices for the system must strike careful loop architecture of Figure 1. dispersion. their presence (or parameters (such as wall position and densities) but are not absence) can be used to determine their potential effects on overly sensitive to nuisance microstructure (such as cavities other components. autofocusing approaches have shown promise for Model-based reasoning is a potential way that propagation identifying some of these parameters by testing various effects can be overcome. This may also require fast Future systems will need to carefully understand the propagation solvers to work through 3-D multipath models. As system design decisions to resolve significant building structural details are hypothesized. Ignoring propagation effects limits DIRECTIONS the degree of understanding of the sensed data. to test multiple hypotheses. the sensed data using all available propagation and Lower frequencies also have the potential benefit that phenomenology information. FUTURE THROUGH WALL IMAGING approaches [10]-[11]. and reflection results. Phenomenology and propagation shadowing and attenuation effects. This approach can be seen in the closed. Sensor architectures must support estimated iteratively. However. etc. This architecture attempts to balances between wall attenuation favoring lower produce the model-based representation that best matches frequencies and resolution favoring higher frequencies. properties Iterate to step 2 Figure 1 Model-based imaging architecture 5174 . Inferences can be made to smaller microstructure (wiring. but do not account for multipath and Bragg scattering.1. and model-based and adjust the imaging focusing algorithms. otherwise. The iterations signal. The assessment and physical modeling. the building model can be in concrete block walls or stud locations) that are less constructed layer by layer.x) Sensing Fusion (state p) GH(p. (optional) Modify sensor state p Building Bootstrap with imaging. As shown in Figure 1. diffractive environment. Through wall propagation diffraction and multipath through walls have been partially addressed in tomographic 2. and reduces the effective depth of penetration Compensation for multipath. Stop if converged. pipes. minp. An imaging architecture must address the physical needs more than just projection information.can cause image ambiguities and modulation of subsequent program. dispersive propagation and develop architectures that and perhaps support fast processing architectures that can provide sufficient diversity to correctly model the building handle these tasks in near-real-time.x)-GH˾(p.x˾ J{GH(p. final product will be the model that provides the best Propagation modeling will also have to be done quickly maximum likelihood match to the sensed data collected. and block walls.

In object-based viewpoint instead of more traditional pixel or the former case. This may require varying fidelity in the propagation model to dig deeper into 2. the presence of strong looks through which the model-based reasoning algorithms scatterers at regular intervals consistent with stud spacing can resolve ambiguities. better than any other model hypothesis. material or all of the following: small distributed hand-held or properties. Model-based 3-D building deconvolution secondary or tertiary effects. Successful key requirement that the derived technologies must be imaging architectures must concentrate heavily on model- operationally useful. allow for any a priori structure information. Multiple hypotheses may be carried very high requirements on real-time processing to provide simultaneously so that uncertainties and ambiguities in the ample time for feedback. Given the near-field ranges for may reinforce a wall hypothesis. systems must make sensor position. Facing dihedrals can Alternatively. These can provide key vehicle-based or airborne platforms. vehicle-borne sensors. weight. and its known provide focal points for autofocusing after penetrating sensitivity to input-output geometric and waveform through one or more walls. Sensor positioning and utilization success to image through walls with image quality quickly degrading through multiple walls or in the presence of The sensor positioning must analyze how to provide the challenging multipath.3. Multi-Output types will have very different signatures. building construction codes. various bistatic or multistatic collection conclusions based upon wall signatures. To determine building structure. Model-based reasoning should information at significantly longer stand-off distances. Such reasoning should be architectures should permit close-range external tightly coupled with physical phenomenology. can further enforce hypotheses. waveform choice. with a has created problems in image understanding. this puts iteration. This is equivalent Airborne systems may also provide useful SAR-based to likelihood estimation. collection time periods can easily local building practices. or even Multi-Input. or temporal characteristics. waveform. Conversely. Sensor that best match the sensed data. stepped structures with many dihedrals and trihedrals that have CW. as well as derived knowledge from the sensor emplaced sensors. They also nature of the deconvolution problem. the diversity can be handled iteratively infer the existence of a wall connecting them that may not using sensor feedback. such as ground-based sensing. Understanding of phenomenology and sensor frequency. sensor configurations should allow for examination able to exploit data sources ranging from optical imagery of of the building structure through some combination of any the building exterior. which can also help to determine configurations. per second since pulse repetition intervals can be very high This highlights the importance of feature extraction to due to the short ranges. noise radars) and multistatic states as well as mobile spatially recognizable signatures. tailor the diversity to resolve ambiguities not other wise An additional goal of building imaging is the detection of possible with an open loop fixed deployment concept-. This diversity can be must be used first to hypothesize the presence and location anticipated and included in the original data collection of a wall. Due to the inherent highly nonlinear be visible itself at unfavorable viewing angles. model state can be probed more deeply. Stand-off distances may be on Figure 1. objects within the building. anchor points in the building model. Resolving these ambiguities may also require careful sensor diversity in position.2. based deconvolution and reasoning to provide 3-D models and logistical tails of sensing architectures. the model should be continuously updated so that the order of street-level access or from neighboring its expected propagation effects represent the sensed data structures to avoid blockage from nearby structures. thus architecture configurations will only be useful if the leading us to the second area of technical inquiry. resultant sensor data can be interpreted correctly. Generally. Single- pass imaging approaches have been used with limited 2. followed by the expected effects on propagation methodology or applied iteratively by providing feedback to through the wall. them using sensed data or sensor diversities. it must also be able to perching sensors. extract information from the sensed data and be able to Various collection geometries may be more sensitive to recognize potential ambiguities in hypotheses and resolve picking up certain features of the building environment. This limits the size. or auxiliary probe hundreds of thousands of diverse collection intervals information. As shown in reconnaissance of buildings. It should be Ideally. Diversity over this period can identify building elements. Buildings have identifiable include waveform (ultrawideband impulse radars. For example. detection can be aided by an the sensors to change their location or operating mode. Different wall angles. Residual ghosting and shadowing data diversity necessary for building reconnaissance. real-time feedback during deployment can dielectric properties of intervening walls. Diversity can be obtained through changes in frequency. Other constraints. However. power. Sensor returns or MIMO radar approaches. building usage. Sensor returns may not be able 5175 .propagation results of the candidate model with each particularly if asset minimization is key. and/or airborne or architecture itself. the collection profile will have to anticipate voxel imaging that attempts to make decisions about each potential areas of ambiguity and provide enough diverse volume in space.

Movement patterns may suggest [5] Lin-Ping Song. 1998. “Through- walkways and openly navigable areas. other [3] S.3. distributed radar sensor network. Aryanfar. 3. M. Building models can be used to make sure that this Doppler 4744. M. shadows from wall imaging (TWI) by radar: 2-D tomographic results and moving objects will also have a Doppler detectable analyses. Mandapati. M. Amin. Dec. 290-292. pp. pp.” and updated with new information. 3375. Tsihirintzis. Devaney and G. Frazier. G. movement through the structure can provide additional data July 1998. Currie. 2005. M. on Image Processing.to isolate building structure from the building contents. on building layouts. Vol 1. New technology advances will be required in three critical areas: phenomenology of signal penetration into buildings. of walls. C. 12. pp. and N. T. Jr. building one interface at a time. Signal Processing. 3577. R. pp. The feedback and iterative model development will Antennas and Propagation Society International hopefully permit the system to probe ambiguities and derive Symposium. Proceedings of SPIE: Radar can readily separate motion from background clutter. In addition. 1785-1795. “Autofocusing of Obviously. Spectroscopy. A. “Three-dimensional In all these cases. 2002. “Methods of improving of the Through building imaging is an important area for first subsurface objects images reconstructed by the tomography responder and military applications. characteristics of motion may make this more “An ultra-wideband through-the-wall radar for detecting the distinguishable than static objects. G. Ahmad. The sensor and Geoscience and Remote Sensing Letters (under review).” closed.” IEEE Transactions on Geoscience and Remote signature and may paint the walls behind the moving Sensing. Past sensing process. pp. Y. F. July 2002. Nag. No. Qing Huo Liu. MA. “A survey of current recognizes objects that do not fit a building hypothesis. D.” Proceedings of the SPIE: These hypotheses might then allow for further iteration on Laser Physics and Photonics. “RADAR flashlight for through-the- including multipath and dispersion that could erroneously wall detection of humans. Vol. 280-285. Volume 5778. 5176 . 2793. Ahmad. model accuracy. July 2007. Boston. G. 2005. but Training Technologies for Law Enforcement. technologies for through-the-wall surveillance (TWS). pp 62-72. Sarabandi. S. Zhang.” Bandwidth and frequency constraints will not provide SPIE Conference on Sensors. observability. A. C31 Information.3066 Vol. Also. and Molecular the building structure itself. J. [8] F. tracking of and Backgrounds: Characterization and Representation IV. [2] L. IEEE National Radar Conference people. Sensor Technology and Data Visualization. Vertiy. 39:672–682. and G. Ferris. Holladay. 20-25 June 2004. [6] F. and Through-the-Wall Radar Imagery under Unknown Wall resolvability are key technical challenges.” International Conference on Mathematical approaches have tried to extrapolate free-space sensing Methods in Electromagnetic Theory.” Proc. Greneker III. Recent advances in propagation modeling and processing power can greatly extend this capability by exploiting model-based building decomposition. Models must be Characteristics. 7. REFERENCES model-based approach to determine the structure hopefully [1] D. crude object models can help include these as part of the November.” IEEE Trans. pp. Payment. However. knowledge of the building. and model-based 3-D building deconvolution that operates in a multipath-rich. waveform diversity can provide extensive knowledge that [7] A. sensor positioning and utilization to maximize information about the building. Hunt. MMET '02. Vol. 1991. signature can be accurately placed within the building by [4] E. K. Amin. 2002. 48-57. Vol. a building model needs to be created wideband beamforming for imaging through a single wall. and sufficient resolution to do full object identification. in effect peeling back the Modeling V. The infinite variety and diversity of people will Electronic Systems.. This is a substantial challenge. 27-31 May 1996. “Through wall imaging at predicted so that the model-based reasoning loop can be microwave frequencies using space-time focusing. 10- algorithms to form images through the dispersive medium 13 Sept. No. “Surveillance through walls and other Another obvious goal is the detection and localization of opaque materials. Tomography. make exact modeling impossible. building contents. Gavrilov.” IEEE Trans. Doppler discrimination motion of people in real time”.2798 personnel. Vol. “Image formation through walls using a can be used to develop hypotheses about the building. Chun Yu. 43.” Proceedings of SPIE Targets position detected signals. representative of the building structure with sufficient detail 16. 3063. Vol. A 4. pp. Barnes. 169-174. pp. SUMMARY [11] A. [10] A. diffractive environment. “Maximum but a necessary technical innovation to probe deeper inside Likelihood Estimation of Object Location in Diffraction the high dispersive and multipath environment of buildings. that the propagation and phenomenology can be adequately [9] F.