Automated Recovery of Structural Drawing Images

Collected from Postdisaster Reconnaissance

Chul Min Yeum 1; Alana Lund 2; Shirley J. Dyke 3; and Julio Ramirez, M.ASCE 4
Downloaded from ascelibrary.org by Universidad Nacional De Ingenieria on 11/05/18. Copyright ASCE. For personal use only; all rights reserved.

Abstract: A large volume of images is collected during postdisaster building reconnaissance. For both older and new buildings, the structural
drawings are an essential record of the structural information needed to extract valuable lessons to improve future performance. With older
construction, these drawings often need to be captured as multiple photographs, herein referred to as partial drawing images (PDIs), taken at a
close distance to ensure critical details are legible. However, the ability to use PDIs is quite limited due to the time-consuming process of
manually classifying such photographs and the challenge of identifying their spatial arrangement. The authors offer a new solution to auto-
matically recover high-quality structural drawing images. First, PDIs are classified from a set of images collected using an image classi-
fication algorithm, called convolutional neural network. Then, using the structure-from-motion algorithm, the geometric relationship between
each set of PDIs and a corresponding physical drawing are computed to identify their arrangement. Finally, high-quality full drawing images
are reconstructed. The capabilities of the technique are demonstrated using real-world images gathered from past reconnaissance missions and
newly collected PDIs. DOI: 10.1061/(ASCE)CP.1943-5487.0000798. © 2018 American Society of Civil Engineers.

Introduction
After a natural event such as an earthquake, windstorm, or associ-
ated events such as tsunami and storm surge inundation, structural
reconnaissance teams are dispatched to collect perishable scientific
data on the conditions of these structures before any damaged struc-
tures are destroyed or repaired. Collecting such data offers valuable
opportunities to learn about the event and its consequences and to
identify potential gaps in existing research or in building codes
through the study of structural performance during the event. From
these studies, important lessons about the performance of the build-
ings, those that exhibit damage as well as those that do not, can be
learned. The high costs associated with such events, in terms of
both the loss of life and the impact on the livelihood of the local
community, make the gathering of such data imperative. Many
engineering organizations and communities around the world have
sponsored and established reconnaissance teams to collect such
valuable information from each disaster and preserve it for future
study (EERI 2009; UC CEISMIC 2011; DataCenterHub 2014;
DesignSafe-CI 2016).
In a typical mission, each team visits between 5 and 10 build-
ings a day and collects various forms of data within a short period.
Some examples of data collection formats appeared in Applied
Technology Council (1989, 2004), and American Concrete Insti-
tute (2017). However, research teams may adopt enhanced forms
of data collection to suit the needs of the effort. Among the diverse
1 when one considers the possible number of buildings involved.
Professor, Dept. of Civil and Environmental Engineering, Univ.
of Waterloo, ON, Canada N2L 3G1 (corresponding author). Email:
cmyeum@uwaterloo.ca
2
Graduate Student, Lyles School of Civil Engineering, Purdue Univ.,
West Lafayette, IN 47907.
3
Professor, School of Mechanical Engineering, Purdue Univ.,
West Lafayette, IN 47907.
4
Professor, Lyles School of Civil Engineering, Purdue Univ.,
West Lafayette, IN 47907.
Note. This manuscript was submitted on November 26, 2017; approved
on June 5, 2018; published online on October 5, 2018. Discussion period
open until March 5, 2019; separate discussions must be submitted for in-
dividual papers. This paper is part of the Journal of Computing in Civil
Engineering, © ASCE, ISSN 0887-3801.
data formats, photographs offer the most convenient and efficient
way of recording and preserving abundant information about the
building (EERI 2009; UC CEISMIC 2011; DataCenterHub 2014;
DesignSafe-CI 2016). Meaningful scenes of damaged and undam-
aged buildings and their components are captured using multiple
images taken from different angles and positions. In addition, the
photographs typically encompass several types of metadata that are
needed for documentation, such as structural drawings, measure-
ment values, or signs. Of all these forms of data, structural drawings
are the focus of this paper. Such drawings provide critical informa-
tion that allows the engineer to infer the behavior of structural and
nonstructural components during extreme events and better under-
stand the cause of the damage shown in the collected images. In
many cases, especially with older buildings, drawings are not likely
to be found in digital format. Since it is difficult to carry several sets
of drawings home or to create complete physical or digital copies
on site, teams often capture structural drawings as segmented pho-
tographs. Multiple photographs of small portions of the structural
drawings are taken at a close distance to ensure the details are leg-
ible (DataCenterHub 2014).
However, in their segmented format, denoted as partial drawing
images (PDIs), the structural drawings are difficult to interpret,
requiring a significant investment of the engineer’s time. These
drawings often represent only a small portion of the large volume
of images collected in the field. Manually categorizing and group-
ing them into sets from the same drawing is a quite challenging task
Moreover, such drawings are often printed on large engineering
paper and contain markers with small-sized contents including
text, digits, or lines. Thus, many images may need to be collected
(e.g., 20–40 images for each drawing) at a close distance in order to
legibly capture all of the details contained in such a drawing. These
manually captured images often are inconsistent in terms of per-
spective distortion and level of detail among the images when they
are captured with different angles or from different distances. Thus,
in addition to the difficulties associated with their segmented forms,
it is laborious to recognize the proper arrangement of the images
and align them. These issues are magnified by the tendency to
continue to store these drawings as PDIs. This means that each

© ASCE 04018056-1 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2019, 33(1): 04018056

 engineer using these drawings will need to tackle the same prob-
lems to view the real content of the drawings, dramatically reducing
the value of the data represented by the ability of the engineer
to reuse it to develop new knowledge. Reconnaissance teams can
save a lot of time and effort if automation is available for dealing
with PDIs in environments that are often hazardous and with lim-
ited resources and time as the community is attempting to return to
normalcy.
To address this challenge, the authors developed an automated
method for constructing high-quality full drawing images (FDIs)
by classifying and stitching PDIs. First, a deep convolutional neu-
Downloaded from ascelibrary.org by Universidad Nacional De Ingenieria on 11/05/18. Copyright ASCE. For personal use only; all rights reserved.
ral network (CNN) algorithm was implemented to extract distinct
visual features for classifying PDIs from the much larger set of
images collected during reconnaissance missions (LeCun et al.
1990; Krizhevsky et al. 2012). Using the authors’ large-scale, real-
world reconnaissance image data set, a robust PDI classifier was
trained and its classification capability is demonstrated. Then,
sets of classified PDIs from the same drawings were grouped by
matching their visual features. To remove perspective distortions in
each of the PDIs and identify their original arrangement, camera
calibration parameters were computed for each PDI using the
structure-from-motion (SfM) technique (Hartley and Zisserman
2003; Snavely et al. 2008). Finally, FDIs for many drawings were
recovered without loss of the visual quality preserved in each of
the PDIs. The performance of the FDI reconstruction was fully
demonstrated using PDIs from multiple large engineering draw-
ings. The authors also examined the implementation of the tech-
nique on a real-world image set containing PDIs, collected after
the Taiwan earthquakes in 2016 (NCREE 2016). The technique
developed herein may also be quite useful in several other appli-
cations in which large volumes of images are collected but where
such automated manipulation can aid in their actual use by human
domain experts.
The balance of this paper is structured as follows. A recom-
mended guideline for collecting PDIs is described in “Collection
of Partial Drawing Images.” “Classification of Partial Drawing
Images” provides the relevant methodology about CNN algo-
rithms for PDI classification. Reconstruction of FDIs from classi-
fied PDIs is illustrated in “Reconstruction of Full Drawing Images.”
In “Experimental Validation,” the capability of PDI classification
and FDI reconstruction is demonstrated using both a simple exam-
ple and real-world image sets collected from past post-earthquake
reconnaissance missions. The final section of this paper presents a
summary and the conclusions from this study.
Collection of Partial Drawing Images
Collection of suitable PDIs is an important step for successfully
yielding high-quality FDIs. Because the tool is designed to bypass
the need for any manual manipulation in classifying the PDIs (see
“Classification of Partial Drawing Images”) and in stitching them
together for the reconstruction of FDIs (see “Reconstruction of
Full Drawing Images”), the quality of PDIs directly affects the
completeness of the final FDIs. Thus, guidelines for image acquis-
ition are suggested here for the best use of the technique developed
in this paper:
1. Physical drawings should be placed on a flat surface after flat-
tening crumpled parts of each drawing. In addition, any object(s)
placed on the drawing should be removed (Mikhail et al. 2001).
2. PDIs need to cover the entire area of each drawing. Also, each
PDI should have large overlapping regions with the next few
images to increase the accuracy of the PDI grouping as well
© ASCE 04018056-2
J. Comput. Civ. Eng., 2019, 33(1): 04018056
as the SfM performance (introduced in “Reconstruction of Full
Drawing Images”).
3. For uniform and fine FDI quality, the camera position should be
roughly parallel to the drawing and maintain a constant distance
as much as possible.
4. Including oblique (angled) images can mitigate incorrect esti-
mation of radial distortion parameters (James and Robinson
2014; Wu 2014). These images are used only for computing
projection matrices and are filtered out when stitchSet is
identified (see the section “Reconstruction of Full Drawing
Images”).
Note that because many PDIs are collected in the field during
just a short time, engineers should carefully consider adopting these
simple guidelines before starting their reconnaissance.
Classification of Partial Drawing Images
In this study, the authors utilized CNNs to classify PDIs from a
much larger set of images collected during a field mission
(Yeum et al. 2017b, 2018). CNNs have been exploited to advance
several computer vision applications, and have enabled the learning
of high-level and robust visual features for image recognition using
large-scale databases (LeCun et al. 1990; Krizhevsky et al. 2012;
Simonyan and Zisserman 2014; Goodfellow et al. 2016; He et al.
2016; Montserrat et al. 2017; Ren et al. 2017). CNNs typically have
one or more convolutional layers tied with weights and pooling
layers to extract scale, translation, and rotation tolerant features.
They also have fully connected layers at the end of the network,
which are associated with these features to classify images or object
categories. Conceptually, CNNs work by finding a set of features
that best describe the given images with a large number of convolu-
tional filters. Several CNN architectures have been introduced in
the literature, but their accuracy varies depending on how one con-
figures the network architecture (Simonyan and Zisserman 2014;
Russakovsky et al. 2015). Optimal network architectures and the
configuration of input images and categories are still a topic of
active research in different domains of applications. However, it
is obvious that CNNs provide exceptional performance in image
classification (Russakovsky et al. 2015).
Here, the authors implemented a two-class (binary) classifica-
tion to train a discriminative boundary between PDIs (positive)
and the remaining images (negative). A large volume of real-world
images was gathered from past reconnaissance events. Then the
gathered images were manually labeled as positive or negative and
used to train a robust classifier. The set of images was selected be-
cause they have similar visual contents and distribution to those that
are expected to be collected in a future event. Depending on the
resolution of the image, it may be resized or stretched, but typ-
ically the training (and testing) of the CNN requires low-resolution
input (e.g., 200–300 pixels on each side depending on algorithms)
(Krizhevsky et al. 2012). Then resized images were augmented to
avoid overfitting with respect to translation, color, or light varia-
tions. The augmentation produced more training images without
changing their original semantic meaning. By training the network
using these additional data sets, the classifier will be invariant to
features that are not relevant to the appearance of the corresponding
class, and thus it will produce more stable and robust outcomes
when the unseen data are tested. For example, in AlexNet, used in
this study, all images were resized to 256 × 256 pixels, and random
square regions of 224 × 224 pixels were extracted (Krizhevsky
et al. 2012). Once the inputs for the training of the CNN were pre-
pared, a large number of parameters at convolutional, pooling, and
fully connected layers were automatically tuned so as to extract
J. Comput. Civ. Eng.
 robust features by minimizing the error in estimating true labels of
the training images. Stochastic gradient descent with a particular
batch size of images was used to optimize the parameters in the
network. At each epoch, a new batch was assigned at each iteration
using randomly ordered postaugmentation images. This training
process, depending on the learning rates, the number of images, the
network configuration, and the hardware specifications, typically
takes from a few hours to many weeks. The details associated with
the implementation are provided in “Training of a Partial Drawing
Classifier.”
Downloaded from ascelibrary.org by Universidad Nacional De Ingenieria on 11/05/18. Copyright ASCE. For personal use only; all rights reserved.
Reconstruction of Full Drawing Images
This section describes how multiple FDIs were constructed from
the entire set of PDIs that were classified using the approach dis-
cussed in the previous section. An overview of the steps in this
procedure is shown in Fig. 1.
For each building, multiple physical drawings associated with
different floors or details were often available. Thus, in Step 1, the
PDIs collected from the same drawing were matched and grouped
across all PDIs. Since each of the PDIs contained only a small part
of a full drawing, and yet had a similar appearance as other draw-
ings across the set, it was challenging to manually group them. In
this step, PDIs collected from the same drawing were classified as
such by automatically matching the common visual features present
in the PDIs. These extracted visual features were also reused for
computing the projection matrix in Step 2. Conventional visual fea-
tures and descriptors [e.g., scale-invariant feature transform (SIFT)
or speeded-up robust features (SURF)] can be used for this process.
Both SIFT and SURF automatically extract visual features (and de-
scriptors) invariant to scaling, rotation, and illumination, allowing
accurate matching of identical local points across images (PDIs in
this study) (Lowe 2004; Bay et al. 2008). Ideally, the PDIs would
only share common features with those in the same group. How-
ever, separate drawings for the same building (e.g., for different
floors) are likely to have some repeated or duplicate regions such
as text descriptions, digits, or graphics (Fig. 4). These will poten-
tially generate spurious correspondences between PDIs even when
they are captured from different drawings.
Fig. 1. Overview of the technique for constructing FDIs. Once sets of PDIs captured from the same drawings are obtained in Step 1, the correspond-
ing FDI in each set is generated using Steps 2–6.
© ASCE 04018056-3
J. Comput. Civ. Eng., 2019, 33(1): 04018056
To identify the PDI groups, the authors adopted a popular
community detection technique called modularity maximization
(Newman and Girvan 2004; Yeum and Dyke 2015). This algorithm
automatically clusters a set of PDIs (network nodes), which are
densely connected internally. Modularity is defined as a measure
of the quality of a particular division of a network into the com-
munity. The optimal community structure is obtained by iteratively
merging communities to maximize modularity in a network. The
adjacent matrix A, which defines the connection (matching) be-
tween nodes (here PDIs), is the only input to this technique. The
matrix A is defined as Aij if nodes i and j are connected, otherwise
0, where Aij is the element at the ith row and jth column. This
matrix can be obtained from the matching results. Further technical
details are provided in Clauset et al. (2004).
In the subsequent steps, processing takes place within individ-
ual groups of PDIs obtained from Step 1. In Step 2 a projection
matrix of each PDI is estimated. The projection matrix describes
the mapping of a pinhole camera from three-dimensional (3D)
points in the scene to two-dimensional (2D) points in an image,
establishing a 2D–3D relationship. The projection matrix contains
intrinsic camera parameters such as focal length, principal points,
and extrinsic orientation including rotation angles and camera
location in 3D. SfM automatically computes the 3D geometry
of the scene (point cloud), projection matrix, and lens distortion
parameters. Recently, SfM has emerged as a technique for
constructing a 3D point cloud from 2D images in conjunction with
the development of unmanned aerial vehicle (UAV) technology,
and some popular software packages such as Pix4D, PhotoScan,
or ContextCapture, and non-commercial software, such as
VisualSFM and OpenMVG (Moulon et al. 2013). Regardless of
the differences in functionalities and algorithms implemented in
these tools or their optimization for specific applications, in gen-
eral the input to SfM is a set of images, and the outputs are the
associated projection matrices (constructed based on several inter-
nal and external camera parameters) and the 3D point cloud gen-
erated from the scene. Nearly all available commercial and
noncommercial SfM software can export these outputs in a read-
able format. In SfM, all of these parameters and data are automati-
cally estimated by matching the features detected in Step 1.
J. Comput. Civ. Eng.
 The advantage of this technique is that it requires neither prior
camera calibration nor networks of physical targets with known
(or even unknown) 3D positions. Once the users collect a set
of suitable photos by following the recommended guidelines,
the projection matrices can be automatically estimated (Hartley
and Zisserman 2003; Snavely et al. 2008; Yeum et al. 2017a).
The projection matrix is a 3 × 4 matrix, denoted as P, and the
following relation holds:
xi ¼ Pi X ð1Þ
Downloaded from ascelibrary.org by Universidad Nacional De Ingenieria on 11/05/18. Copyright ASCE. For personal use only; all rights reserved.
This equation is expressed in homogeneous coordinates
(Hartley and Zisserman 2003). Four-dimensional vector X (4 × 1)
is a 3D point in the scene, defined in the SfM coordinate system;
Pi is the projection matrix of the ith image among a total of n PDIs;
and xi is a point on image i corresponding to X and is typically
defined as pixel values (a three-dimensional vector in homogeneous
coordinates). Note that n will differ depending on the group of PDIs.
Lens distortion should be corrected in advance so that all images
hold the relationship in Eq. (1) based on the pinhole camera model.
Simply, Eq. (1) represents the fact that any arbitrary 3D point can be
mapped to each image if its projection matrix is known.
Step 3 is to estimate the plane on which the drawing is located in
the SfM coordinate system. In this study, a maximum likelihood es-
timation sample consensus (MLESAC) estimator was applied to ac-
curately estimate the drawing surface from the 3D points. MLESAC
is a generalization of a random sample consensus (RANSAC) esti-
mator and introduces scores for inliers, rather than simply counting
them in the RANSAC, to evaluate how well they fit the data
(Torr and Zisserman 2000; MATLAB 2016). The principle of the
RANSAC (as well as MLESAC) estimator is to iteratively search
for the best plane that can be established with the given 3D points.
The 3D points in the previous step are mostly generated from the
drawing surface. The authors could easily identify the drawing plane
in an automated way from this cloud of 3D points (Step 3 in Fig. 1).
The 3D points, X, on the estimated plane π should meet the follow-
ing relation:
πX ¼ 0 ð2Þ
where π ¼ ½ vT 1 . Here, v is a normal vector (1 × 3) of the esti-
mated plane π.
In Step 4, a planar projective transformation between π and
each image is identified. The basic assumption for the technique
developed herein is that the drawings are placed on a planar sur-
face, which is estimated in the previous step. Thus, the authors
could establish a one-to-one relationship between the 2D points
on each image and the points of the drawing, represented as a
homography matrix. The homography matrix is a 3 × 3 matrix
providing a planar projective transformation between two images
(Hartley and Zisserman 2003). Here, the homography matrix of
each image was obtained from its projection matrix and π. How-
ever, because homography is a relationship between 2D points on
two images, the 3D points on the plane π should be represented as
two variables.
Suppose that the 3D point that lies on π in 3D is denoted as Xπ ,
which satisfies Eq. (2). To establish the homography relationship,
one of the dimensions in Xπ should be reduced by transforming the
coordinate system. If the SfM coordinate system is transformed to
align the Z-axis to v, defined in Eq. (2), using a rotation matrix R,
the plane in the new coordinate system, π, ~ becomes parallel to the
XY-plane, and a point on this plane, Xπ~ , has a constant value in the
Z-axis. This relationship can be represented as
© ASCE 04018056-4
J. Comput. Civ. Eng., 2019, 33(1): 04018056
Xπ~ ¼ RXπ ð3Þ
where Xπ~ ¼ ½X π~ Y π~ Zπ~ 1 T ; and π~ ¼ πR−1 ¼ ½ π~ 1 π~ 2 π~ 3 π~ 4 .
The value of R can be computed using the angle between v and
the Z-axis in the SfM coordinate system. The product of Xπ~ and π~
is zero according to Eq. (2). Since the new plane π~ is parallel to the
XY-plane, Zπ~ is not correlated with X π~ or Y π~ and has a constant
value of
Zπ~ ¼ −π~ 4 =π~ 3 ð4Þ
~ which is a
Here, π~ 3 and π~ 4 are the third and fourth element of π,
1 × 4 vector defined in Eq. (3).
If Eq. (1) is represented using Xπ~ and π, ~ in Eq. (3), the homog-
raphy matrix between π~ and each image can be computed as
xi ¼ Hiπ~ × ½ X π~ Y π~ 1 T ð5Þ
where Hiπ~ ¼ ½ p~ 1i p~ 2i Zπ~ p~ 3i þ p~ 4i ; and P~ i ¼ Pi R−1 ¼ ½ p~ 1i p~ 2i p~ 3i p~ 4i .
These matrices are expressed in homogeneous coordinates; Hiπ~
is a 3 × 3 homography matrix that maps the points on the plane π~ to
the ones on the ith image, and p~ ji is the jth column of P~ i . According
to Eq. (1), xi in Eq. (5) is equal to P~ i Xπ~ , which is the product of the
transformed projection matrix and the points on the plane π. ~ Thus,
in Eq. (5), Hiπ~ can be represented as the known values obtained in
the previous steps. This implies that Eq. (5) represents the fact that
any arbitrary 2D point on the plane represented as X π~ and Y π~ can be
mapped to any 2D point on each image.
Step 5 is to project the PDIs onto the drawing plane. Because
oblique images may include large perspective distortion and back-
ground scenes (although they are useful for estimating camera cal-
ibration parameters in the SfM process), they could induce artifacts
on the FDI when the perspective distortion of out-of-the-plane
objects is restored (Brown and Lowe 2007). This process would
result in large unnecessary areas on the FDI, and would also reduce
its resolution when the size of the final FDI is fixed in the begin-
ning. Thus, the authors only used the images that were captured
roughly parallel to the drawing plane (although all of the PDIs were
used for the projection matrix estimation in the previous step). To
find the set of the images that are relatively parallel to the drawing,
the angle between the principal axis (camera direction or optical
axis) of each image and the normal vector v to the drawing is
computed, defined as αi (where i is the image number). The direc-
tion of the principal axis can be obtained from the first three ele-
ments of the last row of the projection matrix in Eq. (1) (Hartley
and Zisserman 2003). The images having αi smaller than an angle
threshold, αt , are selected for FDI reconstruction. The authors de-
note such as group as a stitchSet. For example, in the experimental
study αt was empirically set to 30°, which is smaller than the angle
for the oblique images.
In the previous step, the authors identified Hiπ~ , which is the
relationship between the points on the ith image and the plane
~ However, the homography matrix should define the relationship
π.
between points on each image and points on the FDI, not the plane.
The points on the FDI use the pixel coordinates, meaning that a left-
bottom corner is an origin and the coordinates start from zero along
the X (width) and Y (height) axes. Thus, a proper scaling factor and
translation parameters are needed to update Hiπ~ . A scaling factor
must be computed by setting up the maximum size of the FDIs in
advance. Since the PDIs are captured at a close distance to the
drawing, the FDIs will have extremely high quality if the original
quality of each image is preserved. The procedure to compute the
scaling parameter (denoted as s) and translation parameters (de-
noted as X min and Y min ) was developed by the authors and outlined
as follows:
J. Comput. Civ. Eng.
 function CompParam
X min ¼ ∞
Y min ¼ ∞
X max ¼ −∞
Y max ¼ −∞
for i←1; n do
for αi > αt then
for j←1; 4 do
x ¼ jth column of cornerPointsi
Xh ¼ ðHiπ~ Þ−1 × homðxÞ
Xn ¼ nhomðXhÞ
X max ¼ minðX min ; Xnð1ÞÞ
Downloaded from ascelibrary.org by Universidad Nacional De Ingenieria on 11/05/18. Copyright ASCE. For personal use only; all rights reserved.
Y max ¼ minðY min ; Xnð2ÞÞ
X max ¼ maxðX max ; Xnð1ÞÞ
Y max ¼ maxðY max ; Xnð2ÞÞ
else
continue
s ¼ minðFDIW =ðX max − X min Þ; FDIH =ðY max − Y min ÞÞ
return s; X min ; Y min
Here, cornerPointsi is a 2 × 4 matrix having four corner pixel
locations for the ith image. The function homðxÞ is used to convert
a vector x from Cartesian coordinates to homogeneous coordinates,
and nhomðxÞ is used to implement this transformation in a reverse
way. The maximum dimensions of the FDIs in the width and height
directions are denoted as FDIW and FDIH , respectively. If the for
αi > αt then condition is satisfied, the ith image is included in
stitchSet. Once the scaling factors and translation parameters are
computed, the final homography can be updated as follows (Hartley
and Zisserman 2003):
2 32 3
1 0 X min s 0 0 Representative Problem Demonstrating Full Drawing
6 76 7
Hiπ~ ¼ Hiπ~ 4 0 1 Y min 54 0 s 05
0 0 1 0 0 1
Again, with the homography matrix in Eq. (6), the points on
each image corresponding to a certain point on the FDI can be iden-
tified. By using the homography matrix on each image, the pro-
jection of each image (removing perspective distortion) and its
alignment on the plane of the FDI are completed through this ho-
mography transformation. Only the images included in stitchSet are
used for constructing the FDI.
Step 6 is to stitch and blend the aligned projected images to con-
struct FDIs. For their seamless composition, the authors utilized
gain compensation and multiband blending developed in Brown
and Lowe (2007). Illuminations of images (here, PDIs) vary de-
pending on the angles and camera locations, and thus each image
has different overall gain (intensity), causing artifacts (banding) on
the FDIs without adjusting them. Gain compensation is a process to
normalize such gain variations between images. By minimizing the
intensity difference of overlapping pixels from multiple images, the
proper gain for each image is optimized (Brown and Lowe 2007).
After tuning the gain for each image (here, projected PDIs), they
are seamlessly composed as one single FDI. However, although the
gains of all images are normalized, they still may have inconsis-
tency between images due to vignetting (the intensity varies toward
the edge of the image), out-of-focus portions, or errors in radial
calibration. The multiband blending technique facilitates a smooth
transition between images to create high-quality FDIs. Images are
divided into multiple frequency bands, and weight maps for each
band are calculated to determine the contribution of each PDI to the
final FDI.
© ASCE
J. Comput. Civ. Eng., 2019, 33(1): 04018056
Experimental Validation
To demonstrate the capability of the technique developed, the au-
thors used reconnaissance image collections from past disasters that
include PDIs. Although a large volume of reconnaissance images is
available in the public domain, these images were collected prior
to the development of this technique. Thus, the PDIs used herein
were not collected following the guidelines provided here, and
therefore pose an additional challenge to the technique. For exam-
ple, some sets do not include sufficient overlap or cover only a cer-
tain portion of the entire drawing instead of the whole document.
Though it is still possible to create FDIs from such image sets, the
quality of the resulting image may be greatly reduced. In order to
fully demonstrate the capabilities of this drawing classification and
reconstruction technique, the authors separated this study into two
experiments. First, in “Representative Problem Demonstrating Full
Drawing Image Reconstruction,” reconstruction of a representative
drawing is considered to illustrate the technique on a realistic case
following the guidelines set forth herein. Then “Training of a
Partial Drawing Classifier” shows the capability of the PDI classi-
fication using actual reconnaissance images collected from past
earthquake events. However, due to insufficient quality of the PDIs
for the FDI construction (because they were not captured in the
field using the guidelines), the authors focused on demonstrating
the performance of PDI classification using these real-world PDIs.
In “Implementation of the Developed Technique,” the authors dem-
onstrate how FDIs are obtained from a set of reconnaissance images
taken without using the recommended guidelines. It will be seen
that though the quality of the resulting FDI is impacted by the lack
of a regulated collection technique, the image is successfully recon-
structed and is complete and sufficient for typical uses.
ð6Þ Image Reconstruction
Description of the Experiment
The authors first demonstrated the capability of FDI reconstruction
from a typical set of PDIs. For data preparation, four different dig-
ital drawings were printed on Arch E1 paper 76.2 × 106.7 cm
(30 × 42 in:). Each drawing had fine text and thin lines, for which
the details cannot be captured or included in one photograph. Each
drawing was smoothed out and placed on a table. Fig. 2 shows one
of the drawings used for this demonstration.
A total of 152 images were sequentially collected from the four
different drawings. The number of PDIs for each drawing was ap-
proximately 40 images (not uniform across different drawings).
Fig. 3 shows sample PDIs for each drawing, where the images in
Fig. 2. Structural drawing printed on large engineering paper.
04018056-5 J. Comput. Civ. Eng.
Downloaded from ascelibrary.org by Universidad Nacional De Ingenieria on 11/05/18. Copyright ASCE. For personal use only; all rights reserved.
Fig. 3. Sample PDIs captured from four different drawings.
each column are captured from the same drawing and denoted
as Drawings 1–4. PDIs were taken using a handheld camera, (Nikon
D90, Nikon Corporation, Tokyo) with 18-105 mm lens, and zoom
and flash functions were not utilized. The resolution of each image
was 4,288 × 2,848 pixels. To simulate conditions of data collection
in the field, the images were captured by a human photographer (no
tripod or mechanical device held the camera) while minimizing per-
spective distortion, producing roughly uniform resolution across
each drawing region. As per the guidelines, the images were col-
lected with sufficient overlap in the horizontal and vertical directions
between the images. In Fig. 3, the last image in each column shows
an oblique image, which was captured with a large angle.
The authors implemented the technique using MATLAB and
VisualSFM (MATLAB 2016). VisualSFM is a noncommercial free
software with a graphical user interface (GUI) and is widely used
by many researchers due to its accuracy and speed using NVIDIA
CUDA (Wu 2013). Although its source code is not available in the
public domain, it provides a command line operation without a
GUI. Extraction of SIFT features and descriptors and computation
of projection matrices were conducted with VisualSfM through
command-lines in MATLAB (Wu 2013). To access the processed
data, the authors built a tool to read output files (in NVM format) in
MATLAB. For grouping the PDIs, each image was paired with the
next five images in the sequence of collected PDIs (to replicate pro-
cedures in the field), followed by constructing an adjacency matrix.
The adjacency matrix became an input for the community detection
technique, producing a set of PDIs, which were collected from the
same drawing (Newman and Girvan 2004). Projection matrices
were computed from each set of PDIs using VisualSFM, and then
used for reconstructing the corresponding FDI. For blending the
images, the image blender function in OpenCV, called MultiBand-
Blender, was converted to an executable file to be used in MATLAB
(MATLAB 2016; OpenCV 2017).
Drawing Reconstruction Results
The four constructed FDIs are shown in Fig. 4. The overall perfor-
mance of FDI reconstruction was quite successful. In Fig. 4, the
images on the right are magnified areas corresponding to the boxes
on each FDI, respectively. Small text and digits as well as thin lines
are clearly legible. These FDIs were automatically constructed from
© ASCE 04018056-6
J. Comput. Civ. Eng., 2019, 33(1): 04018056
a mixed set of the collected PDIs without any manual processing.
However, the final four FDIs in Fig. 4 were manually rotated and
cropped to contain only the drawing areas for better visualization.
Training of a Partial Drawing Classifier
Reconnaissance Image Data Set
For demonstration of PDI classification, the authors gathered an
extensive collection of reconnaissance images that were taken by
researchers and practitioners after past earthquake events (Yeum
2016; Yeum et al. 2016, 2017a, c). These images are publicly avail-
able at DataCenterHub. Some of the authors of this study have
taken a significant role in collecting these data in the field, and
the needs of those field missions provide motivation for this work
(Shah et al. 2015; Sim et al. 2015, 2016; NCREE 2016). The
authors utilized images collected from earthquake reconnaissance
missions in Düzce, Turkey, in 1999, Bingöl, Turkey, in 2003, Peru
in 2007, Haiti in 2010, Nepal in 2015, Taiwan in 2016, and Ecuador
in 2016 (Shah et al. 2015; Sim et al. 2015, 2016; NCREE 2016).
Because PDIs are stored apart from the other images on Data-
CenterHub, manual labeling was already performed and no addi-
tional manual labeling was required for this study. A total of 31,173
images were downloaded for these events, including 2,232 PDIs.
Some sample images are shown in Fig. 5.
Configuration
In this study, the authors implemented the established ImageNet
CNN model called AlexNet (TorontoNet in Caffe), framed in the
MatConvNet library (Vedali and Lenc 2015). AlexNet exhibited
superior performance in the 2012 ImageNet image classification
competition and has been widely used as a benchmark test for
newly developed CNN models. The details of its network architec-
ture are explained in LeCun et al. (1990), Krizhevsky et al. (2012),
and Goodfellow et al. (2016). Although several advanced CNN ar-
chitectures have been proposed and customized for specific appli-
cations, the authors chose AlexNet here because it is an established
method. Because the positive and negative images in this study
have an obvious visual boundary, the use of the original architec-
ture is acceptable (Goodfellow et al. 2016; Vedali and Lenc 2015).
J. Comput. Civ. Eng.
Downloaded from ascelibrary.org by Universidad Nacional De Ingenieria on 11/05/18. Copyright ASCE. For personal use only; all rights reserved.
Fig. 4. All four FDIs, automatically constructed using PDIs: (a) through (d) correspond to Drawings 1–4 in Fig. 3, respectively.
For training a classifier, labeled images were first transformed
into inputs for the CNNs. Regardless of the category, all images
were isotopically (preserving their aspect ratio) resized so that
the shorter side of each image became 256 pixels, followed by
cropping the center square region to create 256 × 256 pixel square
images. This is based on the fact that objects of interest (herein,
drawings) do not span the entire image area but rather are often
concentrated at the center of images. The authors further imple-
mented data augmentation, which is a commonly used method that
expands the training set to avoid overfitting by modifying the
existing input data without changing their label. First, 227 × 227
patches were randomly cropped from the 256 × 256 images and
those patches were randomly flipped about the horizontal direction
in each epoch. Additionally, random color shifting was applied to
vary (jitter) the intensities of RGB images to expand the training
images as a part of data augmentation (Krizhevsky et al. 2012). To
train the classifiers and test their capabilities, all labeled images
were divided into groups of 50%, 25%, and 25% for training, val-
idating, and testing, respectively. The validation set was used for
monitoring the network accuracy during training.
The original implementation for the ImageNet image classifica-
tion classified 1,000 output classes using the last 1,000-way soft-
max layer (Krizhevsky et al. 2012). In this study, it was converted
to a logistic loss layer to conduct binary classification. The layers
were initialized as a Gaussian distribution with a zero mean and
variance equal to 0.1 (Krizhevsky et al. 2012; He et al. 2015;
Krähenbühl et al. 2015). The hyperparameters were the same as
those used in AlexNet. The authors trained the models using a sto-
chastic gradient descent with a batch size of 512 images, momen-
tum of 0.9, and weight decay of 0.0005. The authors trained the
network for 120 epochs and the learning rate was logarithmically
decreased from 10−2 to 10−5 during training. The authors chose to
© ASCE 04018056-7
J. Comput. Civ. Eng., 2019, 33(1): 04018056
preset these values based on some preliminary tests conducted with
an initial sample of training images. A PC workstation having a
Xeon E5-2620 (Intel Corporation, US) and Telsa k40c (NVIDIA
Corporation, US) with a 12-GB video memory GPU was used
for training and testing the algorithm. The MatConvNet library in-
stalled on MATLAB, version 2017 was used for this study (Vedali
and Lenc 2015).
Classification Results
In this study, PDI classification successfully attained a relatively
high accuracy. Among a total of 546 PDIs for testing (25% of
all PDIs), the authors obtained rates of 96.15% (525/546 images)
true positive recall (true classification of PDIs) and a precision of
0.941 (defined as the number of true positives over the number of
positives). These high recall and precision values indicate that most
PDIs among the set of images can be accurately detected and the
rate of false positive errors (non-PDI detected as positive) is very
low. Figs. 6(a and b) show samples of false positive and false neg-
ative images. With only a glance, the images in Fig. 6(a) do look
like PDIs. Also, because the images used for training and testing
were collected by many different field engineers and after several
different earthquake events in various countries, the performance of
the classifier was expected to be similar to these results, without
bias toward any real-world future data sets used.
Implementation of the Developed Technique
In this section, the authors demonstrate the capability of the tech-
nique developed herein using real-world images collected during
past building reconnaissance missions. This example is intended to
demonstrate its implementation on an actual image set collected
in the field, with no particular regard for the specific performance.
J. Comput. Civ. Eng.
Downloaded from ascelibrary.org by Universidad Nacional De Ingenieria on 11/05/18. Copyright ASCE. For personal use only; all rights reserved.

Fig. 5. Sample images used for training a binary PDI classifier: (a) PDIs (positive); and (b) the rest of images except PDIs (negative). [Images from
Shah et al. 2015; Sim et al. 2015, 2016; NCREE 2016. License: Creative Commons Attribution License (CC BY-SA 3.0 US, https://creativecommons
.org/licenses/by-sa/3.0/us/).]

Performance was instead discussed in the previous sections.
Engineers collecting these data during past reconnaissance missions
were not aware of the technique developed in this paper. Thus, the
PDIs were collected under less than ideal conditions (without fol-
lowing the image collection guideline suggested in “Collection of
Partial Drawing Images”). If crumpled regions or objects on draw-
ings were not removed, the corresponding areas on the projected
images (images in Step 5) would be shifted due to their different
elevations of the projected plane, producing relief displacements
on the images. Relief displacement is scale variation on the photo-
graphs caused by height variation of the objects or terrain (Mikhail
et al. 2001). Cases with such traits would not yield perfectly aligned
FDIs with high quality, such as those constructed in “Representative
Problem Demonstrating Full Drawing Image Reconstruction.”
As mentioned in the introduction, each reconnaissance team col-
lects a large volume of images from multiple buildings. Here, it was
assumed that the set of images collected from the same building
were already separated. Since modern reconnaissance images con-
tain date and time information (metadata collected by the camera
itself) and are captured in chronological order, images from the
same building can easily be separated using the difference of image
collection times. The image set used for this demonstration was
collected from a single building (Yujing vocational school teaching
building) after the Taiwan earthquake in 2016. Fig. 7 shows all of
the 323 images collected from this building, including 145 PDIs
marked by boxes. This collection of images is publicly available
from DataCenterHub (Purdue University and NCREE 2016). This
image set was selected because a large and relatively complex set of
PDIs was available for testing. The images in this set were not used
for training the PDI classifiers in “Reconstruction of Full Drawing
Images.”
After processing, it was found that PDIs were classified accu-
rately. Images classified as PDIs are marked with solid dots in each
image in Fig. 7. Among 145 images classified as PDIs, 143 PDIs
were correctly classified, yielding a precision of 98.6%. This result
shows that the trained classifier can be readily used for any PDI
data collected from the future events. After classification, the FDIs
were constructed, with two representative samples shown in Fig. 8.
The FDIs in Fig. 8 show that the compiled PDIs were not precisely
aligned with each other. The FDI will thus have irregular resolution
because the number of PDIs was insufficient and because the draw-
ing was not placed on a flat surface at the time the PDIs were col-
lected. However, the visual contents were legible and appropriate
for typical uses. Based on the assumption that the PDIs fully cover

© ASCE 04018056-8 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2019, 33(1): 04018056

Downloaded from ascelibrary.org by Universidad Nacional De Ingenieria on 11/05/18. Copyright ASCE. For personal use only; all rights reserved.

Fig. 6. Samples of (a) false positive; and (b) false negative images. [Images from Shah et al. 2015; Sim et al. 2015, 2016; NCREE 2016. License:
Creative Commons Attribution License (CC BY-SA 3.0 US, https://creativecommons.org/licenses/by-sa/3.0/us/).]

Fig. 7. Original 323 images collected from a single building after 2016 Taiwan earthquake. [Images from NCREE 2016. License: Creative Commons
Attribution License (CC BY-SA 3.0 US, https://creativecommons.org/licenses/by-sa/3.0/us/).]

the entire drawing, an insufficient number of PDIs indicates that the
PDIs do not have much overlap with each other. This causes an
inaccurate estimation of the projection matrix of each image. Also,
capturing PDIs without flattening the drawings produces relief
displacement, mentioned in the beginning of this section. Both
cases cause misalignment of the PDIs on the corresponding FDIs.
Therefore, users should follow the recommended guidelines, using
the suggestions in “Collection of Partial Drawing Images.”
This demonstration also shows the possibility of applying
the technique developed herein to existing legacy data sets to clas-
sify drawing images and recover their full drawing images. Ever
more past and future postdisaster images are being gathered and

© ASCE 04018056-9 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2019, 33(1): 04018056

Downloaded from ascelibrary.org by Universidad Nacional De Ingenieria on 11/05/18. Copyright ASCE. For personal use only; all rights reserved.
Fig. 8. Two samples of constructed FDIs generated from PDIs classified from the images in Fig. 7. [Images from NCREE 2016. License: Creative
Commons Attribution License (CC BY-SA 3.0 US, https://creativecommons.org/licenses/by-sa/3.0/us/).]
published (EERI 2009; UC CEISMIC 2011; DataCenterHub 2014;
DesignSafe-CI 2016). Such automation will have substantial impli-
cations for organizing and reusing those large amounts of data in a
rapid manner to support future research.
Summary and Conclusions
In this study, the authors developed and validated an automated
technique to reconstruct high-quality FDIs using PDIs, which were
automatically classified from a large volume of postdisaster re-
connaissance images. To do so, the authors first built a classifier
for identifying PDIs from a collection of reconnaissance images.
A large volume of real-world images collected from past reconnais-
sance missions was utilized for training the classifier as well as
evaluating its performance. Once the authors classified a set of
PDIs, the tool developed automatically conducted the recovery of
FDIs by grouping and stitching the PDIs after geometrically cor-
recting their distortion and estimating their locations on each FDI.
To demonstrate the ability to perform FDI reconstruction using
PDIs, a set of PDIs were collected from four large drawings having
numerous small details only visible at a close distance. All FDIs
were successfully constructed and all details were clearly legible.
The authors are not yet able to fully demonstrate the technique
using several sets of real-world reconnaissance images because
the existing PDIs were not acquired following the guidelines. How-
ever, the authors plan to further evaluate the proposed technique by
collecting reconnaissance images, including PDIs, following the
suggested image acquisition guidelines, and use those images to
carry out additional demonstrations of the technique.
Acknowledgments
The authors wish to acknowledge partial support from the
National Science Foundation under Grant Nos. NSF 1608762
and DGE-1333468. The authors would like extend to special
thanks to Professor Santiago Pujol at Purdue University for his in-
valuable comments to inform this study. The authors acknowledge
© ASCE 04018056-10
J. Comput. Civ. Eng., 2019, 33(1): 04018056
the valuable image contributions from the Center for Earthquake
Engineering and Disaster Data (CrEEDD) at Purdue University
(DataCenterHub), the EUCentre (Pavia, Italy), the Instituto de
Ingenieria of UNAM (Mexico), FEMA, and the Earthquake Engi-
neering Research Institute (EERI) collections.
References
American Concrete Institute. 2017. Disaster reconnaissance. ACI 133.
Farmington Hills, MI: American Concrete Institute.
Applied Technology Council. 1989. Building safety evaluation forms and
placards. ATC 20. Redwood City, CA: Applied Technology Council.
Applied Technology Council. 2004. Field manual: Safety evaluation of
buildings after windstorms and floods. ATC 45. Redwood City, CA:
Applied Technology Council.
Bay, H., A. Ess, T. Tuytelaars, and L. Van Gool. 2008. “Speeded-up robust
features (SURF).” Comput. Vision Image Understanding 110 (3):
346–359. https://doi.org/10.1016/j.cviu.2007.09.014.
Brown, M., and D. G. Lowe. 2007. “Automatic panoramic image stitching
using invariant features.” Int J. Comput. Vision 74 (1): 59–73. https://
doi.org/10.1007/s11263-006-0002-3.
Clauset, A., M. E. Newman, and C. Moore. 2004. “Finding community
structure in very large networks.” Phys. Rev. E 70 (6): 066111.
DataCenterHub. 2014. “Data sets.” Accessed October 28, 2017. http://
datacenterhub.org.
DesignSafe-CI. 2016. “Rapid experimental facility.” Accessed October 28,
2017. https://rapid.designsafe-ci.org.
EERI (Earthquake Engineering Research Institute). 2009. “Earthquake
clearinghouse.” Accessed October 28, 2017. http://www.eqclearing
house.org.
Goodfellow, I., Y. Bengio, A. Courville, and Y. Bengio. 2016. Deep
learning. 1st ed. Cambridge, MA: MIT Press.
Hartley, R., and A. Zisserman. 2003. Multiple view geometry in computer
vision. 2nd ed. Cambridge, UK: Cambridge University Press.
He, K., X. Zhang, S. Ren, and J. Sun. 2015. “Delving deep into rectifiers:
Surpassing human-level performance on ImageNet classification.” In
Proc., IEEE Int. Conf. on Computer Vision, 1026–1034. New York:
IEEE.
He, K., X. Zhang, S. Ren, and J. Sun. 2016. “Deep residual learning for
image recognition.” Preprint, submitted December 10, 2015. http://
arXiv.org/abs/1512.03385.
J. Comput. Civ. Eng.
 James, M. R., and S. Robson. 2014. “Mitigating systematic error in topo-
graphic models derived from UAV and ground-based image networks.”
Earth Surf. Processes Landforms 39 (10): 1413–1420. https://doi.org
/10.1002/esp.3609.
Krähenbühl, P., C. Doersch, J. Donahue, and T. Darrell. 2015. “Data-
dependent initializations of convolutional neural networks.” Preprint,
submitted November 21, 2015. http://arXiv.org/abs/1511.06856.
Krizhevsky, A., I. Sutskever, and G. E. Hinton. 2012. “ImageNet classifi-
cation with deep convolutional neural networks.” In Proc., 25th Advan-
ces in Neural Information Processing Systems, 1097–1105. Cambridge,
MA: MIT Press.
LeCun, Y., B. E. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. E.
Hubbard, and L. D. Jackel. 1990. “Handwritten digit recognition with a
Downloaded from ascelibrary.org by Universidad Nacional De Ingenieria on 11/05/18. Copyright ASCE. For personal use only; all rights reserved.
back-propagation network.” In Proc., 3rd Advances in Neural Informa-
tion Processing Systems, 396–404. Cambridge, MA: MIT Press.
Lowe, D. G. 2004. “Distinctive image features from scale-invariant
keypoints.” Int. J. Comput. Vison 60 (2): 91–110. https://doi.org/10
.1023/B:VISI.0000029664.99615.94.
MATLAB. 2016. MATLAB Version 7.10.0 (R2016b). Natick, MA:
MathWorks.
Mikhail, E. M., J. S. Bethel, and J. C. McGlone. 2001. Introduction to
modern photogrammetry. New York: Wiley.
Montserrat, D. M., Q. Lin, J. Allebach, and E. Delp. 2017. “Training object
detection and recognition CNN models using data augmentation.” J.
Electron. Imaging 2017 (10): 27–36. https://doi.org/10.2352/ISSN
.2470-1173.2017.10.IMAWM-163.
Moulon, P., P. Monasse, and R. Marlet. 2013. “Global fusion of relative
motions for robust, accurate and scalable structure from motion.” In
Proc., IEEE Int. Conf. on Computer Vision, 3248–3255. New York:
IEEE.
NCREE (National Center for Research on Earthquake Engineering) and
Purdue University. 2016. “Performance of reinforced concrete buildings
in the 2016 Taiwan (Meinong) earthquake.” Accessed October 28,
2017. https://datacenterhub.org/resources/14098.
Newman, M. E., and M. Girvan. 2004. “Finding and evaluating community
structure in networks.” Phys. Rev. E 69 (2): 026113. https://doi.org/10
.1103/PhysRevE.69.026113.
OpenCV. 2017. “Open Source Computer Vision Library.” Accessed
October 28, 2017. http://opencv.org/.
Purdue University and NCREE (National Center for Research on Earth-
quake Engineering). 2016. “Performance of reinforced concrete build-
ings in the 2016 Taiwan (Meinong) earthquake.” Database ID: 94717.
https://datacenterhub.org/resources/14098/.
Ren, S., K. He, R. Girshick, and J. Sun. 2017. “Faster R-CNN: Towards
real-time object detection with region proposal networks.” Preprint,
submitted June 4, 2015. http://arXiv.org/abs/1506.01497.
Russakovsky, O., J. Deng, H. Su, J. Krause, S. Satheesh, and S. Ma. 2015.
“ImageNet large scale visual recognition challenge.” Int. J. Comput.
Vision 115 (3): 211–252. https://doi.org/10.1007/s11263-015-0816-y.
Shah, P., S. Pujol, A. Puranam, and L. Laughery. 2015. “Database on per-
formance of low-rise reinforced concrete buildings in the 2015 Nepal
earthquake.” Accessed October 28, 2017. https://datacenterhub.org
/resources/238.
© ASCE 04018056-11
J. Comput. Civ. Eng., 2019, 33(1): 04018056
Sim, C., C. Song, N. Skok, A. Irfanoglu, S. Pujol, and M. Sozen. 2015.
“Database of low-rise reinforced concrete buildings with earthquake
damage.” Accessed October 28, 2017. https://datacenterhub.org
/resources/123.
Sim, C., E. Villalobos, J. P. Smith, P. Rojas, S. Pujol, A. Y. Puranam, and L.
Laughery. 2016. “Performance of low-rise reinforced concrete buildings
in the 2016 Ecuador earthquake.” Accessed October 28, 2017. https://
datacenterhub.org/resources/14160.
Simonyan, K., and A. Zisserman. 2014. “Very deep convolutional networks
for large-scale image recognition.” Preprint, submitted September 4,
2014. http://arxiv.org/abs/1409.1556.
Snavely, N., S. M. Seitz, and R. Szeliski. 2008. “Modeling the world from
internet photo collections.” Int J Comput Vision 80 (2): 189–210.
https://doi.org/10.1007/s11263-007-0107-3.
Torr, P. H. S., and A. Zisserman. 2000. “MLESAC: A new robust estimator
with application to estimating image geometry.” Comput. Vision Image
Understanding 78 (1): 138–156. https://doi.org/10.1006/cviu.1999.0832.
UC CEISMIC. 2011. “Canterbury earthquake digital archive.” Accessed
October 28, 2017. www.ceismic.org.nz.
Vedali, A., and K. Lenc. 2015. “MatConvNet—Convolutional neural net-
works for MATLAB.” In Proc., 23rd ACM Int. Conf. on Multimedia,
689–692. New York: Association for Computing Machinery.
Wu, C. 2013. “Towards linear-time incremental structure from motion.”
In Proc., IEEE Int. Conf. on 3D Vision, 127–34. New York: IEEE.
Wu, C. 2014. “Critical configurations for radial distortion self-calibration.”
In Proc., IEEE Conf. on Computer Vision and Pattern Recognition,
25–32. New York: IEEE.
Yeum, C. M. 2016. “Computer vision-based structural assessment exploit-
ing large volumes of images.” Ph.D. dissertation, Lyles School of Civil
Engineering, Purdue Univ.
Yeum, C. M., J. Choi, and S. J. Dyke. 2017a. “Autonomous image locali-
zation for visual inspection of civil infrastructure.” Smart Mater Struct
26 (3): 035051. https://doi.org/10.1088/1361-665X/aa510e.
Yeum, C. M., and S. J. Dyke. 2015. “Vision-based automated crack detec-
tion for bridge inspection: Vision-based automated crack detection for
bridge inspection.” Comput-Aided Civ. Inf. 30 (10): 759–770. https://
doi.org/10.1111/mice.12141.
Yeum, C. M., S. J. Dyke, B. Bedrich, T. Hacker, J. Ramirez, A. Lund, and
S. Pujol. 2017b. “Rapid, automated image classification for documen-
tation.” In Proc., 7th Int. Conf. Advances in Experimental Structural
Engineering. Pavia, Italy: EUCENTRE Foundation.
Yeum, C. M., S. J. Dyke, and J. Ramirez. 2018. “Visual data classification
in post-event building reconnaissance.” Eng. Struct. 155: 16–24. https://
doi.org/10.1016/j.engstruct.2017.10.057.
Yeum, C. M., S. J. Dyke, J. Ramirez, and B. Benes. 2016. “Big visual data
analysis for damage evaluation in civil engineering.” In Proc., Int. Conf.
on Smart Infrastructure and Construction, 569–574. London: Institu-
tion of Civil Engineers.
Yeum, C. M., S. J. Dyke, J. Ramirez, T. Hacker, S. Pujol, and C. Sim.
2017c. “Annotation of image data from disaster reconnaissance.”
In Proc., 16th World Conf. Earthquake Engineering. Tokyo:
International Association for Earthquake Engineering.
J. Comput. Civ. Eng.