See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/320935189

Computer vision for SHM of civil infrastructure: From dynamic response

measurement to damage detection – A review

Article  in  Engineering Structures · January 2018

DOI: 10.1016/j.engstruct.2017.11.018

CITATIONS READS

99 2,093

2 authors:

Dongming Feng Maria Qing Feng

Thornton Tomasetti, Inc. Columbia University
42 PUBLICATIONS   974 CITATIONS    221 PUBLICATIONS   5,217 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Design of a MEMS-based, wireless sensor board for vibration monitoring View project

Columns Repair View project

All content following this page was uploaded by Dongming Feng on 27 July 2019.

The user has requested enhancement of the downloaded file.

 Engineering Structures 156 (2018) 105–117

Contents lists available at ScienceDirect

Engineering Structures
journal homepage: www.elsevier.com/locate/engstruct

Review article

Computer vision for SHM of civil infrastructure: From dynamic response MARK
measurement to damage detection – A review
⁎
Dongming Feng , Maria Q. Feng
Department of Civil Engineering and Engineering Mechanics, Columbia University, New York, USA

A R T I C L E I N F O A B S T R A C T

Keywords: To address the limitations of current sensor systems for field applications, the research community has been
Computer vision actively exploring new technologies that can advance the state-of-the-practice in structural health monitoring
Displacement measurement (SHM). Thanks to the rapid advances in computer vision, the camera-based noncontact vision sensor has
Structural health monitoring emerged as a promising alternative to conventional contact sensors for structural dynamic response measure-
Structural dynamics
ment and health monitoring. Significant advantages of the vision sensor include its low cost, ease of setup and
Damage detection
operation, and flexibility to extract displacements of any points on the structure from a single video measure-
Natural frequency
Mode shape ment. This review paper is intended to summarize the collective experience that the research community has
Model updating gained from the recent development and validation of the vision-based sensors for structural dynamic response
measurement and SHM. General principles of the vision sensor systems are firstly presented by reviewing dif-
ferent template matching techniques for tracking targets, coordinate conversion methods for determining cali-
bration factors to convert image pixel displacements to physical displacements, measurements by tracking ar-
tificial targets vs. natural targets, measurements in real time vs. by post-processing, etc. Then the paper reviews
laboratory and filed experimentations carried out to evaluate the performance of the vision sensors, followed by
a discussion on measurement error sources and mitigation methods. Finally, applications of the measured dis-
placement data for SHM are reviewed, including examples of structural modal property identification, structural
model updating, damage detection, and cable force estimation.

1. Introduction
Structures and infrastructure systems including bridges, buildings,
dams, pipelines are complex engineering systems that support a so-
ciety’s economic prosperity and quality of life. As these systems age and
deteriorate, their proper inspection, monitoring and maintenance has
become increasingly important. The conventional practice based on
periodic human visual inspection is inadequate. Nondestructive eva-
luation (NDE) has shown potential for detecting hidden damage but the
structures’ large size presents a significant challenge to implement such
local inspection methods. Over the past two decades, a significant
amount of studies have been conducted in the emerging field of
structural health monitoring, aiming at objective and quantitative
structural damage detection and integrity assessment based on mea-
surements by sensors, mostly accelerometers [1–11]. For example,
Carden and Fanning [7] presented an extensive literature review of the
damage detection techniques based on changes in the frequency-do-
main modal properties, such as natural frequencies, mode shapes and
its curvatures, modal flexibility and its derivatives, modal strain energy,
and frequency response function. Although these studies have produced
SHM methods, frameworks and algorithms validated through numer-
ical, and laboratory and field experimental studies, their wide deploy-
ment in realistic engineering structures are limited by the requirement
of cumbersome and expensive installation and maintenance of sensors
networks and data acquisition (DAQ) systems.
To address these limitations, the research community has been ac-
tively exploring new technologies that can advance the current state-of-
the-practice in SHM, such as wireless sensors [12–14], fiber optic sen-
sors [15–17], and the interferometric radar system [18]. In recent
years, camera and computer vision-based sensors have emerged as a
promising tool for non-contact remote measurement of structural re-
sponses, in which displacements are extracted by tracking the move-
ment of targets from videos images. Compared to the structural accel-
eration response (which most of the SHM studies are based on), the
displacement response directly reflects the structural overall stiffness,
and thus offers a potential for more accurate assessment of structural
conditions [2]. Conventional contact-type displacement sensors such as
the linear variable differential transducer (LVDT) require a stationary

⁎
Corresponding author.
E-mail address: df2465@columbia.edu (D. Feng).

https://doi.org/10.1016/j.engstruct.2017.11.018
Received 6 May 2017; Received in revised form 4 October 2017; Accepted 8 November 2017
0141-0296/ © 2017 Elsevier Ltd. All rights reserved.
D. Feng, M.Q. Feng
reference point, which is often difficult to find in the field. Thanks to
the advances in cameras and computer vision algorithms, the non-
contact vision sensor technologies based displacement measurement
offers significant advantages over the contact-type and other non-
contact-type (e.g., GPS, laser vibrometer) displacement sensors, as
summarized below [19–49]:
(1) In contrast to the conventional contact-type sensor (such as an
LVDT) that requires time-consuming and costly installation of the
sensor on the structure with physical connections to not only a
stationary reference point but also DAQ and power supply, the vi-
sion sensor does not require physical access to the structures, as the
camera can be set up at a remote location. This represents a sig-
nificant time and cost saving. For bridge monitoring, for example,
no traffic control is required.
(2) Compared with the GPS, which still requires installation on the
structure (but not the stationary reference point), the vision sensor
is far more accurate and less expensive. Depending on the cost, the
GPS measurement error is typically in the range of 5 mm–10 mm,
more than an order of magnitude larger than that of the vision
sensor.
(3) Compared with the non-contact laser vibrometer, which needs to be
placed relatively close to the measurement target due to the low
laser power for safety concerns, the vision sensor can be placed
hundreds of meters away (when using a zoom lens) and still achieve
satisfactory measurement accuracy.
(4) In contrast to these conventional sensors, all of which are point-
wise sensors, the vision sensor can be termed as a noncontact dis-
tributed sensing technique as it can simultaneously tracking mul-
tiple points from a long distance. More importantly, one can easily
alter the measurement points after the video images are taken.
The research community has applied vision sensor systems on a
diverse set of structures to measure their displacements in either con-
trolled laboratory or complex and challenging field environments. In
the SHM applications, structural natural frequencies and mode shapes
can be conveniently obtained from displacement measurements using
one or more cameras. The adoption of vision sensors can significantly
reduce test cost and time associated with conventional instrumenta-
tions. For example, Poozesh et al. [50] pointed out that testing of a
typical 50 m utility-scale wind turbine blade requires approximately
200 gages (costing $35 k–$50 k) and about 3 weeks to set up a con-
ventional strain gauge system. By contrast, a multi-camera noncontact
measurement system can significantly reduce the test time and cost.
Data analytics for FE model updating, structure damage detection and
integrity evaluation can be carried out utilizing the measured dis-
placement time histories and corresponding operational modal analysis
results.
In the field of experimental mechanics such as material mechanical
testing and structural stress analysis, the digital image correlation (DIC)
technique has been commonly used as a practical and effective tool. It
can directly provide full-field displacements to sub-pixel accuracy and
full-field strains by comparing the digital images of a test object surface
acquired before and after deformation. Experimental mechanics appli-
cations usually involve specific specimens and the measurements are
made in well controlled environments. To achieve reliable and accurate
DIC analysis, artificial speckle or texture patterns are often applied on
the specimen surface [26]. Pan et al. [51] systematically reviewed and
discussed the methodologies of the 2D DIC technique for displacement
field measurement and strain field estimation, and provided detailed
analyses of the measurement accuracy considering the influences of
both experimental conditions and algorithm details. Based on the
measured strain fields, various material mechanical parameters in-
cluding Young’s modulus, poisson’s ratio, stress intensity factor, re-
sidual stress and thermal expansion coefficient can be further identified
[51,52]. It is noted that the emphasis of this paper is placed on the
106
Engineering Structures 156 (2018) 105–117
vision-based sensor systems for structural dynamic displacement mea-
surement and SHM applications. DIC-based applications for experi-
mental mechanics are not included in this paper.
In summary, this paper aims to provide a review of the collective
experience that the research community has gained from the develop-
ment and application of vision-based displacement sensors, with em-
phasis on structural dynamics and health monitoring applications. The
paper is organized as follows. In Section 2, general principles of vision-
based sensor systems are presented by reviewing various template
matching techniques, coordinate conversion methods, measurement by
tracking artificial targets vs. natural targets, etc. In Section 3, valida-
tions of the measurement capacity and accuracy of vision-based sensors
are reviewed by providing a description of the current state of experi-
mentation in both laboratory and field environments, followed by the
detailed discussion on measurement error sources and error mitigation
methods. In Section 4, current studies on using the measured dis-
placement data for structural SHM are reviewed in detail, including
examples of modal analysis, model updating, damage detection, and
cable force estimation. Finally, Section 5 concludes the paper with a
summary and outlook of future directions of vision-based sensors for
SHM.
2. Basics and principles of vision-based sensor system
2.1. System basics: hardware and software
The vision-based displacement sensor system typically consists of a
video camera (or cameras), a zoom lens (or lenses), and a computer. It
may also require lighting lamps for conducting measurements at night
[27]. Table 1 shows typical hardware components of a vision sensor
system. The camera equipped with the lens is fixed on a tripod and
placed at a remote location away from the structure. The camera is
connected to the computer, which is installed with an image acquisition
and analytics software package. If the software has real-time processing
capability, the measured displacement time histories can be displayed
on the computer screen in real time and automatically saved to the
computer. Otherwise, the images can be saved for post processing.
Oftentimes, it is required to take measurements from a remote distance.
To guarantee the measurement resolution, optical lens with proper
focal length should be selected to zoom in the image to obtain enlarged
tracking target/targets.
In literature [54], an easy-to-use user interface, as shown in Fig. 1, is
built into a real-time image processing and displacement extraction
software package for easy operation by non-technical staff. It sum-
marized that the procedure of the vision-based displacement measure-
ment typically includes:
(1) Vision sensor setup. Fix the camera equipped with the lens on a
tripod and place it at a remote convenient location away from the
structure. The camera is connected to the notebook computer in-
stalled with the image-processing software.
(2) Single- or multiple-target/template registration. Any natural or artifi-
cial texture (summarized in detail in Section 2.7) on the structural
surface can be registered as a tracking target, as long as it has
pattern with a contrast to surrounding background. For each mea-
surement point, a subset with a proper size should be chosen, which
should contain sufficient local texture to allow an accurate pattern
matching [49].
(3) Template matching for displacement extraction. The template
matching algorithm (summarized in detail in Section 2.2), mostly
together with subpixel techniques, is employed to track the targets
registered in the previous step. The motion of the target is tracked
by finding its position in a sequence of video images. It would be
highly time-consuming if the target is searched within the whole
image of each video frame. To reduce computational time, the
searching area could be confined to a predefined region of interest
D. Feng, M.Q. Feng Engineering Structures 156 (2018) 105–117

Table 1
Typical hardware components of vision sensor system.

Component Model Technical specifications

Video camera Maximum resolution: 1280 × 1024

[53] Frame rate: 150 fps
Color: Monochrome
Point Grey/FL3-U3-13Y3M-C Sensor type: CMOS
Pixel size: 4.8 μm
Lens mount: C-mount
Interface: USB3.0
Image sensor: 1/3″ CCD ICX424 AL/AQ
[31] Active picture element: 640 × 480
Maximum frame rate: 86 fps
Nile IMX-5040FT Video output: digital-12 bit camera link
Scanning system: progressive scan
Maximum resolution: 1400 × 1024
[27] Frame rate at maximum resolution: 64 fps
Pixel size: 7.4 μm
Genie/HM1400 Sensor type: CMOS
Color: Monochrome
Lens mount: C-mount + F-mount adaptor
Optical lens Focal length: 9–90 mm
[53]
Maximum Aperture: F1.8
Kowa/LMVZ990 IR Mount: C-mount

Focal length: 12–240 mm

[31] Mount: C-mount

Samsung Techwin (SLA-12240)

Zoom: 80–400 mm
[27] Aperture: F4.5–5.6
Manual zoom and luminosity control
Nikon/80-400 VR ED
Pan-tilt drive and housing Rotation angle: pan 0° –350°, tilt −90° to + 20° ± 5°
Receiver function:
[31] - AUX1 ∼ AUX5 (light, wiper, pump, heater)
- Camera control (zoom, focus)
- Pan tilt/ camera preset
YUSIN system (EPT-6000s)
- Communication method: RS-485/RS-422
Tripod and accessories Laptop computer, Tripod, USB3.0 type-A to micro-B cable, etc.

(ROI) near the template’s location in the previous image. It is noted
that the new ROI of a target must be large enough to cover its
potential position on the next frame. Otherwise, mismatching will
be introduced [41,54].
(4) Coordinate conversion. In order to obtain physical displacements of
the target object from the captured video images, the relationship
between the pixel coordinate and the physical coordinate must be
established. The scaling factor (e.g., with units of mm/pixel) can be
obtained in two ways, as to be discussed in Section 2.5.
2.2. Motion/displacement tracking using template matching techniques
Computer vision-based displacement sensors are primarily enabled
by the template matching technique, one of the most effective image
processing techniques for tracking objects. Template matching is a
computationally intensive process that aims at locating a template
within an image [55]. As illustrated in Fig. 2, the technique involves
two primary components: (1) the template image T; and (2) the source
image I in which a match to the template image is expected to find.
General classifications of template matching approaches are: area or
template-based matching approaches and feature-based matching ap-
proaches [56–58].
2.2.1. Area-based template matching using cross correlation or sum of
squared differences
Area-based template matching methods, sometimes called correla-
tion-like methods, put emphasis rather on the matching step than the
detection of salient objects [59]. Classical area-based methods exploit
for directly matching image intensities using the exhaustive search
strategy. To identify the matching area, the template image is compared
against the source image by moving the template one pixel at a time
(left to right, up to down). At each location, a metric is calculated to
represent the similarity between the template image and the particular
area of the source image. For each location (x, y) of T over I, the match
metric is stored. The position of the template in the source image is
determined through searching the peak position of the distribution of
the match metric. The differences of the positions of the template in
video images yield the in-plane displacement vector, as illustrated in
Fig. 2. The existing correlation criteria or similarity metrics/measures
employed for vision sensors are often categorized into two groups [60]:
namely, (1) the cross-correlation (CC) criterion which includes CC,
normalized cross-correlation (NCC) and zero-normalized cross-correla-
tion (ZNCC), and (2) the sum of squared differences (SSD) correlation
criterion which includes SSD, Normalized sum of squared differences
(NSSD) and zero-normalized sum of squared differences (ZNSSD). Stu-
dies reveal that the ZNCC and ZNSSD correlation criteria offer the most
robust noise-proof performance and are insensitive to the offset and
linear scale in illumination lighting; the NCC and NSSD correlation
criteria are insensitive to the linear scale in illumination lighting but
sensitive to offset of the lighting; and the CC and SSD correlation cri-
teria are sensitive to all lighting fluctuations [51]. All these methods
have been successfully applied for structural displacement/deflection
measurement [31,49,61–63]. For example, the vision-based displace-
ment measurement systems were developed by Ye et al. [62] based on
the NCC criterion, by Dworakowski et al. [63] based on the ZNCC
criterion, and by Pan et al. [49] using the ZNSSD criterion. In addition,
Feng et al. [47] developed a vision sensor based on the upsampled cross
correlation (UCC), which is essentially the CC by means of Fourier
transform.

107
D. Feng, M.Q. Feng
Fig. 1. User interface of a vision sensor software package [54].
Fig. 2. Development schematic of vision-based sensors [47].
2.2.2. Feature-based template matching
Feature-based matching exhibits both geometric (i.e., translation,
rotation and scale) invariance and photometric (i.e., brightness and
exposure) invariance, the main steps of which often include: (1) de-
tecting a set of distinctive key-points and define a region around each
key point, (2) computing and extracting local descriptors from the
normalize regions, and (3) matching local descriptors. Thus, feature
detection, description and matching are three essential components in
feature-based matching applications [64].
108
Engineering Structures 156 (2018) 105–117
Feature detection selects points of interest in an image that have
unique features, such as corners (sharp image features) or blobs
(smooth image features). The key to feature detection is to find features
that remain locally invariant so that they can be detected even in the
presence of illumination and scale changes, rotation, and occlusion. For
example, the features from Harris, accelerated segment test (FAST), and
Shi & Tomasi methods can be adopted for detecting corner features
[65], and the speeded-up robust features (SURF) [66], KAZE [67], and
maximally stable extremal regions (MSER) methods [68] for detecting
blob features. On the other hand, feature extraction involves computing
a compact vector representation of a local regions centered around
detected features. Descriptors, such as scale-invariant feature transform
(SIFT) or SURF, rely on local gradient computations. Binary descriptors,
such as binary robust invariant scalable keypoint (BRISK) [69] or fast
retina keypoint (FREAK) [70], rely on pairs of local intensity differ-
ences, which are then encoded into a binary vector.
Once the local features and their descriptors have been extracted,
the matching of feature points between two images can be performed
by minimizing the Euclidean distance between the descriptors using the
nearest-neighbor matching algorithms. Two algorithms have been
found most efficiently for matching high dimensional features: the
randomized k-d forest and the fast library for approximate nearest
neighbors (FLANN). It is noted that these algorithms are not suitable for
binary features (e.g., FREAK or BRISK), which can be compared using
the Hamming distance calculated via performing a bitwise XOR op-
eration followed by a bit count on the result [71]. In computer vision
applications, in order to remove the false matching points, statistically
robust methods such as the RANdom SAmple Consensus (RANSAC) can
be used to filter outliers in matched feature sets while estimating the
geometric transformation or fundamental matrix [72]. In Ref. [57], Soh
D. Feng, M.Q. Feng
et al. compared several well-known techniques for feature selection and
matching, i.e., Kanade-Lucas-Tomasi (KLT) method, SURF with FLANN,
SURF with brute force matching, and SIFT with RANSAC.
Due to advantages such as geometric and photometric invariances, all
the aforementioned and other available feature detection and matching
techniques have been extensively adopted for the vision-based displacement
sensor development [27,30,32,33,39,46,53,73–81].
2.3. Pixel level vs. subpixel level
In practical applications, one major concern for the vision sensor is
its measurement accuracy. Abovementioned template matching tech-
niques alone usually measure displacements with integer-pixel resolu-
tion since the minimal unit in a video image is one pixel. Although in
many applications the pixel-level accuracy is adequate, a higher re-
solution is often required for measuring small structural vibrations such
as ambient vibration of a short-span concrete bridge. Especially, pixel-
level template matching may result in unacceptable measurement er-
rors if the displacement to be measured is in the same order of mag-
nitude as the scaling factor [47]. To improve the measurement accu-
racy, incorporating the subpixel registration into the template matching
algorithm is regarded as the best practice. The interpolation technique
is most commonly used subpixel approach, examples of which includes
intensity interpolation, correlation coefficient curve-fitting or inter-
polation, phase correlation interpolation and the geometric methods
[82–84].
Subpixel registration can also be formulated as an optimization
problem and solved through heuristic algorithms such as genetic al-
gorithms, artificial neural network algorithms, and particle swarm op-
timization [85,86]. There are also other subpixel techniques that are
based on Newton-Raphson method [87] and gradient-based methods
[88]. Studies have been conducted to investigate their performance for
vision-based displacement measurement. For example, to further im-
prove the accuracy of DIC, Pan et al. [51] reviewed the various subpixel
registration algorithms, including coarse-fine search algorithm, peak-
finding algorithm, iterative spatial domain cross-correlation algorithm,
spatial-gradient-based algorithm, genetic algorithm, finite element
method and B-spline algorithm. Debella-Gilo and Kaab [89] evaluated
two different approaches, namely intensity interpolation and correla-
tion interpolation, to achieve sub-pixel precisions when measuring
surface displacements on mass movements using NCC. Through shaking
table test, Feng et al. [90] demonstrated the improvement of mea-
surement accuracy by applying an upsampling subpixel technique.
It is noted that although in theory subpixel resolution can achieve
maximum accuracy, in practice the resolution is limited as images may
be contaminated with various environmental noises and system noises
arising from the electronics of the imaging digitizer [47]. The subpixel
accuracies reported in many studies vary within orders of magnitude
from 0.5 to 0.01 pixel [82]. Mas et al. [91] demonstrated, through
numerical analysis, a realistic limit for subpixel accuracy, and found
that the maximum achievable resolution enhancement is related to the
dynamic range of the image.
2.4. Single- vs. multi-point measurement
The vision sensor can conduct single-point displacement measure-
ment at its best resolution. By zooming out the lens, multi-point or full-
field displacements in a large field of view (FOV), i.e., area that is
visible in the image, can be measured simultaneously by one camera.
However, tradeoffs between the measurement resolution and the
number of measurement points or FOV is necessary. This is because a
decreased measurement resolution would be expected when measuring
multiple points in a larger FOV for large-scale structures, even though
the sensor accuracy can be significantly improved through sub-pixel
registration techniques. In this case, one possible solution is using
multiple synchronized cameras, with each camera targeting different
109
Engineering Structures 156 (2018) 105–117
sections of a large-scale structure. For example, aiming at measuring
multi-point displacements along structures, Fukuda et al. [92] devel-
oped a time synchronous measurement system using multiple computer
and camera subsystems. The system embeds an algorithm that auto-
matically and periodically performs synchronization through TCP/IP
communication to maintain the time lag between internal clocks of
multiple computers to a range < 5 ms. Lee et al. [93] introduced the
synchronized multi-point vision-based system for real-time displace-
ment measurement of high-rise buildings using a portioning approach,
the accuracy and feasibility of which were verified on a five-story steel
frame tower. Ojio et al. [94] synchronized two cameras by triggering an
interval timer through a relay module driven by solid-state relays. A
pair of high-luminance LEDs were activated by the timer within sight of
each camera and used as a synchronizing timing marker.
2.5. Coordinate conversion and scaling factor
In order to measure structural displacements from the captured
video images, the relationship between image coordinates in the unit of
pixels and physical (or world) coordinates in the units of millimeter or
inch must be established to obtain the scaling factor for converting the
image pixel to the physical length (e.g., with units of mm/pixel). The
scaling factor can be determined (1) based on the intrinsic parameters
of the camera as well as the extrinsic parameters between the camera
and the object structure, which can be obtained through camera cali-
bration; and (2) from the known physical dimension on the object
surface and its corresponding image dimension in pixels.
2.5.1. Camera calibration method
Camera calibration is the process of estimating parameters of the
camera for obtaining the scaling factor using images of a special cali-
bration pattern. The parameters include camera intrinsics, distortion
coefficients, camera extrinsics [63]. Fig. 3 shows the schematic of stereo
calibration to estimate parameters for a pair of cameras, as well as the
relative positions and orientations of the cameras. After obtaining the
camera parameters, the world coordinates of any image point can be
reconstructed from its image coordinates.
The most frequently adopted pinhole model for camera calibration
relates the 3-dimentional (3D) world coordinate (X, Y, Z) of a calibra-
tion target point P with its corresponding location (u, v) in the image
plane using a perspective transformation. The projection equation can
be expressed as:
sm′ = A [R|T] M′ (1)
or
X
u ⎡ fx γ cx ⎤ ⎡ r11 r12 r13 t1 ⎤ ⎡ ⎤
s⎡ v ⎤ = ⎢ 0 f y c y ⎥ ⎢ r21 r22 r23 t2 ⎥ ⎢ Y ⎥
⎢ ⎥ ⎢ ⎥ ⎢r r r t ⎥ ⎢ Z ⎥
⎣1⎦
⎣ 0 0 1 ⎦ ⎣ 21 22 33 3 ⎦ ⎢ ⎣1⎥ ⎦ (2)
where s is a scale factor, fx and fy are the horizontal and vertical focal
lengths expressed in pixel units, (cx, cy) is a principal point that is
usually at the image center, γ is a skew factor. The extrinsic parameters,
R and T, represent a rigid rotation and translation transformation from
3D world coordinate system to the 3D camera's coordinate system. The
intrinsic parameters A denote a projective transformation from the 3D
camera's coordinates into the 2D image coordinates. The intrinsic
parameters do not depend on the scene viewed. Therefore, once esti-
mated, it can be reused as long as the focal length is fixed (if a zoom
lens is used).
Then, given the corresponding locations of the calibration control
points in the 3D world coordinates and in the 2D image coordinates, the
unknown camera parameters can be estimated based on a nonlinear
optimization, e.g., with the Levenberg-Marquardt algorithm. Wang
et al. [26] summarized the total number of unknowns in both single-
and stereo-camera calibrations. It is noted that the camera calibration
D. Feng, M.Q. Feng
can be conveniently conducted using the well-known OpenCV and
MATLAB camera calibration packages. To calibrate the camera, mul-
tiple images of a calibration pattern from different angles are needed.
Popular calibration patterns include the asymmetric checkerboard
(with one side containing an even number of squares, both black and
white, and the other containing an odd number of squares) [95] and the
circular control points [96,97]. There also exist other calibration pat-
terns. For example, Park et al. [98] used a T-shaped wand with multiple
markers attached at pre-determined positions for calibration of multiple
cameras and to set up the origin of the 3D displacement measurement.
After the calibration, the wand is removed and not used during dis-
placement measurement.
Coordinate conversion through camera calibration has been
adopted to many vision-based displacement measurement systems
[63,79,99]. When a projection of 2D world coordinates to 2D image
coordinates is considered, i.e., the objects are only subjected to in-plane
motion, Wu et al. [41] proposed a simplified camera calibration method
on the basis of known world coordinates of at least four points. In lit-
erature [99], the known dimensions from the edges and diagonals of the
installed artificial target are used to establish the transformation be-
tween image and physical coordinates, with the assumption that the
out-of-plane motion is negligible.
2.5.2. Practical calibration method
The camera calibration method requires a one-time access to the
target structure in order to install the calibration panel, which can be
difficult in the field. For this reason, research has been carried out to
develop more practical calibration methods. When the camera optical
axis is perpendicular to the object surface, all points on this surface
have equal depth of fields, meaning that these points can be equally
scaled down into the image plane. In this case, only one identical
scaling factor is needed. In general, the scaling factor can be obtained
from one of the two methods, as expressed in Eq. (3): (1) SF1 based on
the known physical dimension on the object surface and its corre-
sponding image dimension in pixels (i.e., dknown and Iknown ); (2) SF2
110
Engineering Structures 156 (2018) 105–117
Fig. 3. Schematic of stereo calibration.
based on the intrinsic parameters of the camera as well as the extrinsic
parameters between the camera and the object structure (i.e., D, f and
dpixel ) [47,54].
dknown
⎧ SF1 = Iknown
⎨ SF2 = D
d
f pixel
⎩ (3)
where dknown is the known physical length on the object surface, Iknown
are the corresponding pixel length at the image plane, dpixel is per pixel
length (e.g., in μm/pixel), D is the distance between the camera and the
object, and f is the focal length.
However, the prerequisite of SF1 and SF2 is the perpendicularity of
the camera’s optical axis to the object surface. Such a requirement
would impose some difficulties in the practical implementations be-
cause small magnitude of camera misalignment angle can be unnoticed
during the experiment setup but could cause errors especially when the
object distance from the camera is relatively large. Moreover, in out-
door field tests, it is often unavoidable to tilt the camera optical axis by
a small angle in order to track the object surface target. When the
camera optical axis is tilted about the normal directions of the object
surface by an angle θ, the scaling factor can be approximated by SF3 in
Eq. (4) [47].
D
SF3 = dpixel
f cos2 θ (4)
In literatures [47,62,92,100–105], the scale ratio is constructed
using SF1 based on the known physical dimension on the object surface,
such as the size of artificial target panels (if such panels can be installed
on the object) or the sizes of structural members known from design
drawings, and the corresponding image dimensions in pixels. Generally,
the ratio dpixel / f in SF2 and SF3 can be calculated via the camera spe-
cifications provided by the manufacturer; however, this information
can hardly be found for the majority of low cost cameras [106]. Instead,
in literature [106], a unit conversion (e.g., from image pixel to
D. Feng, M.Q. Feng
millimeter) method is developed with the help of a checkerboard.
However, it’s noted that the proposed unit conversion method requires
that the zoom factor be kept constant in the subsequent test measure-
ments. In fact, in cases of non-perpendicular optical axis, the scaling
factor at each measurement point is not a constant. It depends on
various parameters including the image coordinates of the measure-
ment point, the tilt angle θ of the camera, the focal length f as well as
the distance D between a camera and each measurement point [49].
Pan et al. [49] converted image displacements to physical displace-
ments using an improved calibration model of Eq. (4), where the tilt
angle and distance D are measured by a laser rangefinder.
2.6. 2D vs. 3D measurement
Most of vision sensor studies focus on 2D dynamic displacement
measurement. It is known that the in-plane displacement measurement
accuracy using a single camera is sensitive to out-of-plane motion.
Sutton et al. [107] found that the in-plane measurement error due to
out-of-plane translation is proportional to ΔZ/Z, where ΔZ is the out-of-
plane translation displacement and Z the distance from the object to the
camera.
To minimize the effect of the out-of-plane motion, a 3D stereovision
system with a pair of synchronized cameras in stereoscopic configura-
tion can be employed. Park et al. [98] proposed a motion capture
system with multiple cameras to measure 3D structural displacements.
The 2D coordinate data from each camera is used to calculate the 3D
coordinates of the markers attached on structures with respect to the
predetermined origin of the 3D space. The effectiveness was validated
through a free vibration experiment of a 3-story structure. Poozesh
et al. [50] assessed the accuracy of the 3D stereo-vision system to
measure full-field distributed strain and displacement over a large area
of a scaled wind turbine blade. Pan et al. [51] pointed out that for
deformation measurement of a macroscopic object with a curved sur-
face, stereovision-based 3D measurement is more practical and effective
because it can be used for 3D profile and deformation measurement and
is insensitive to out-of-plane displacement.
Three-dimensional measurement based on stereo vision systems is
expected to attract more research and application interests. However,
due to its convenience and efficiency, 2D measurement would still be
sufficient for most civil engineering structural applications such as
measuring the vertical and transverse deformation of bridges and hor-
izontal displacements of buildings or towers.
2.7. Artificial target vs. natural feature
Template matching algorithms rely on the sufficient intensity var-
iations in the reference and target images to ensure reliable point
identification and matching [26]. In order to improve the robustness of
the target tracking and to reduce measurement errors, high-contrast
artificial targets are often attached to the structural surface [49,108],
such as a roundel target [20], concentric rings [99], crosses [42], LEDs
[43], black and white blocks with random sizes [39], and speckle
patterns [61]. For example, in literature [92], a planar target with four
circles is attached to the structure to measure the vibration of bridge,
while in [42] cross-shaped targets are used and the viewing system is
equipped with an additional reference system, which decreases the
sensitivity to vibrations. Ring-shaped and random targets are used in
[30], and multiple targets are simultaneously measured with a single
camera, producing displacements at multiple points. In order to com-
pletely eliminate the need for accessing the structure, efforts have been
made to track natural features on the structural surface without in-
stalling artificial targets. Recent studies on vision-based displacement
sensors have demonstrated the accuracy of vision sensor by tracking
natural structural features [39,79,81,106].
111
Engineering Structures 156 (2018) 105–117
2.8. Real time vs. post-processing
Most existing vision sensor systems post-possess the recorded video
files. In this case, only a consumer-grade commercial video camera is
needed to take videos. This also provides the flexibility to extract
structural displacements at any point from a single recording. The in-
ability to perform real-time displacement measurement, however,
would limit the application for continuous online monitoring.
To achieve high measurement accuracy, many vision-based tech-
niques sacrifice the analysis speed, although real-time and fast-speed
analysis are often necessary in computer vision based applications.
Khuc and Catbas [106] mentioned that the challenges associated with
the video storage requirement and their processing time need to be
addressed. Baqersad et al. [109] also pointed out that real-time cap-
ability is critical to make photogrammetry more practical for dynamic
measurement. The feasibility of real-time measurement depends on the
complexity of the adopted template matching algorithm, programming
language, as well as code efficiency for the developed software. A real-
time vision sensor system contains a notebook computer installed with
a developed image processing software package. The real-time dis-
placement measurement data are saved in the computer, avoiding the
time-consuming and memory-intensive task of saving video files. For
example, Pan et al. [49] developed a real-time displacement tracking
system using subset-based DIC. Feng et al. developed the real-time
video-processing software based on both the UCC and OCM algorithms
[47,110], in which the programming environment for the software is
Visual Studio 2010 using C++ language. During measurement, the
FlyCapture Software Development Kit (SDK) by Point Grey Research is
used to capture video images from Point Grey USB 3.0 cameras using
the same application programming interface (API) under 32- or 64-bit
Windows 7/8 operating system. Then the frame-by-frame image are
processed by the UCC/OCM algorithm and displayed on the screen
using DirectShow library. Meanwhile, the measured displacement his-
tory would be shown on the screen in real time and saved to the
computer.
The real-time displacement measurement capability is particularly
important for long-term continuous monitoring. Data analytics can also
be incorporated in the software to assess the structural health condi-
tions and detect post-event structural damage. However, real-time
measurement may not be feasible depending on the number of mea-
surement points, required video resolution, maximum frame rate per
second, and template and ROI sizes.
3. Validation of measurement capacity and accuracy
Vision sensors are developed to address the challenges of remote
and accurate displacement measurement for both small and large en-
gineering structures. In the literature, a large number of evaluation tests
have been reported on laboratory structures as well as on medium- and
long-span bridges, buildings, wind turbines, among many others.
3.1. Laboratory experimental evaluation
Many recent studies on vision-based displacement sensors by dif-
ferent research groups have experimentally demonstrated that high
accuracy can be achieved for both single-point and multi-point struc-
tural displacement measurements by either tracking high-contrast
predesigned target panels or natural features on the structural surface
[27,29,30,36,37,39,41,43,47–49,53,102,106,111,112]. Note that this
review is not intended to be an exhaustive listing of laboratory ex-
periments of vision sensors. In fact, almost all the vision sensors in
literatures have been validated through laboratory tests. Recently, ef-
forts have also been made to investigate the feasibilities of displacement
measurements utilizing the advanced onboard sensing capabilities of
the smartphone technologies, such as embedded high-resolution/speed
video features, powerful processors and memories, and open-source
D. Feng, M.Q. Feng
computer vision libraries, etc. For example, Min et al. [113] developed
a smartphone software application for real-time displacement mea-
surement and shaking table tests were conducted to study its accuracy.
Based on vibration testing of a small-scale multistory laboratory model,
Ozer et al. [114] demonstrated the dual usage of the ubiquitously
available smartphone for measuring both structural deflections/dis-
placements and accelerations with its embedded camera and accel-
erometer.
Warren et al. [115] experimentally compared the vision-based, laser
and accelerometer measurements for structural dynamics analysis. It is
concluded that vision-based photogrammetry techniques provide ad-
ditional measurement capabilities that complement the current array of
measurement systems by providing an alternative that favors the high-
displacement and low-frequency vibration, which is typically difficult
to measure with accelerometers and laser vibrometers. Additionally,
D’Emilia [108] introduced the concept of synchronization among three
transducers of the cameras, the lasers and the accelerometers.
3.2. Field tests
For civil engineering structural applications, field evaluation of the
vision sensors is of particular importance. The ability of vision sensors
to remotely measure displacements of short- or medium-span bridges
has been reported. For example, field tests, individually conducted by
Feng et al. [47] and Shariati and Schumacher [116], on a pedestrian
bridge located on the Princeton University campus cross-validated the
frequency-domain characteristics of the bridge identified from the
measurements by different vision sensors. Feng et al. [39] carried out
field tests on two railway bridges subjected to freight trainloads tra-
veling in various speeds. Measurements were remotely taken not only
during the daytime but also at night from different distances with and
without artificial target panel. Through comparison with a conven-
tional LVDT reference sensor, the high accuracy of the proposed remote
sensor system that tracks natural targets was demonstrated in realistic
field environments. Similarly, Pan et al. [49] demonstrated the efficacy
and practicality of the proposed video deflectometer through real-time
deflection measurement of a railway bridge. Ribeiro et al. [27] mea-
sured the displacement of a railway bridge’s deck induced by the pas-
sage of trains, yielding a good agreement between the results of dis-
placement measurement obtained with the video system and with an
LVDT, achieving accuracy below 0.1 mm for distances from the camera
to the target up to 15 m and in the order of 0.25 mm for a distance of
25 m. Busca et al. [30] proposed a vison based technique to measure
both the static and dynamic displacement responses of a railway bridge.
It was found that without an artificial target, the reliability is strongly
affected by the structure texture contrast. This study concluded that in
order to measure a large bridge portion with one single camera, a
compromise between field of view and measurement resolution is ne-
cessary.
Studies have also been carried out for long-span bridges and other
large-size structures. As one of the earliest applications, Stephen et al.
[20] employed a visual tracking system in the measurement of deck
displacements at the center of the 1410 m span of the Humber Bridge in
the UK. Whabeh et al. [43] developed a video camera system with
targets consisting of black steel sheets, on which two high resolution
red lights (LED) were mounted to measure displacement of the Vincent
Thomas Bridge located in Sam Pedro, California. Fukuda et al. [100]
conducted vision sensor measurement of the mid-span displacement of
the same bridge from 300 m away and demonstrated the robustness of
their OCM algorithm in dealing with the changing natural lighting
conditions in the field. Field tests on the long-span Manhattan Bridge
(Fig. 4) by Feng and Feng [54], with the camera placed 300 m away
from the bridge, confirmed the remote, real-time and multi-point
measurement capacities of the vision sensor system. Brownjohn et al.
[99] evaluated the performance of a commercial optical system for
tracking mid-span displacement of the Humber Bridge by using both
112
Engineering Structures 156 (2018) 105–117
pre-attached artificial target panel and natural targets on the bridge,
with the camera placed 710 m away from the mid-span targets. The
measurements agree well with data from reference GPS. Ye et al. [48]
demonstrated the robustness of their vision sensor system through field
measurement of the mid-span vertical displacement of Tsing Ma Bridge
in the operational condition, from which a good agreement was ob-
served between the measurement results by the vision-based system
and GPS. In addition, the vision sensor system was used to measure the
vertical mid-span displacement influence lines of the Stonecutters
Bridge in Hong Kong under different loading scenarios. By tracking six
actively illuminated LED targets mounted on the bridge, the video de-
flectometer developed by Tian and Pan [117] was applied for field,
remote, and multipoint displacement measurement of the Wuhan
Yangtze River Bridge in China during its routine safety evaluation tests.
In [106], the measurement accuracy of the vision sensor for displace-
ment responses and modal parameters of a football stadium is com-
pared with those from reference LVDTs and accelerometers, respec-
tively, for various conditions such as changing ambient light and
distance of the camera.
3.3. Measurement error sources
Measurement errors, caused by various sources, cannot be com-
pletely eliminated for vision-based measurement [118]. The application
studies, especially in field tests, have acknowledged the interference
factors that affect the accuracy of vision-based sensor systems. Mea-
surement errors may arise from sources such as the calibration proce-
dure, optical distortion effects, non-linearity of the field view, optical
components, system resolution, data synchronization among cameras,
light intensity, non-uniform air refraction, among others [108]. D’E-
milia et al. [108] evaluated the performance of vision-based vibration
measurements by considering the effect of some peculiar parameters,
i.e., the type of target, the vibration frequency and amplitude, the ex-
posure time and image acquisition frequency. Literatures [119,120]
analyzed the sensitivity of displacement to the image acquisition noise
(e.g. digitization, read-out noise, black current noise and photon noise),
with reference images corrupted by different levels of zero mean
Gaussian noise. It’s demonstrated that the standard deviation of the
measurement errors is proportional to that of the image noise and in-
versely proportional to the subset size and to the average of the squared
grey level gradients. Haddadi [121] investigated the error sources re-
lated to the DIC technique. Based on both numerical and experimental
tests of the rigid-body motion, this study assessed the errors related to
lighting, the optical lens (distortion), the charge coupled devices (CCD)
sensor, the out-of-plane displacement, the speckle pattern, the grid
pitch, the size of the subset and the correlation algorithm. Pan et al.
[51] systematically reviewed the displacement measurement errors of
2D DIC caused by speckle pattern, non-parallel camera sensor and ob-
ject surface, out-of-plane displacement, image distortion, various
noises, subset size, correlation criterion, interpolation scheme, and
shape function, etc. Ferrer et al. [122] performed a parametric study of
the measurement errors introduced by the vision-based method, in
which the influencing factors such as the distance to the target, the
image size, the type of camera and the movement amplitude were
analyzed for four different distances and two types of excitations.
3.3.1. Errors from camera motion
In field measurements, the camera itself is often subjected to am-
bient vibration (wind, traffic, etc.), causing displacement measurement
errors. Studies have highlighted the adverse effect of camera vibration
on the measurement accuracy [39,49,99]. For example, during the field
tests in [39], the camera vibration caused by passing-train-induced
ground motion affected the measurement accuracy, particularly when a
zoom lens was used, which magnified not only the images but also the
camera vibration. This problem becomes more serious for a lightweight
compact camera-tripod system.
D. Feng, M.Q. Feng
Fig. 4. Field measurement of Manhattan Bridge: (a) test setup, and (b) Tracking targets on the bridge and the background building [54].
Yoneyama and Ueda [61] proposed a method for correcting the
effect of camera movement. The relationship between images before
and after the camera movement is described by an equation of per-
spective transformation. The unknown coefficients of the equation are
determined from un-deformed regions of the images. Then the effect of
the camera movement is eliminated by using the perspective transfor-
mation. The effectiveness of the correction method is validated by ap-
plying to the rigid body rotation and translation measurement of a
planar specimen, the deflection measurement of a wide-flange beam,
and the bridge defection measurement during vehicle passage. A more
convenient camera vibration correction method has been proposed in
literatures [54,123,124]. When measuring, for example, a long-span
bridge displacement, a reference object (such as a building) in the
background can be assured as stationary, as shown in Fig. 4. By sub-
tracting the displacement of the reference object from the bridge dis-
placement, the camera motion can be canceled. Kim and Kim [123]
applied this correction method to remove deck-vibration-induced
camera motion when measuring the hanger cable tension forces for
Gwangan Bridge in Korea. However, the need to have a static reference
in the FOV may lessen the advantage of this method. Besides, the tra-
deoff between the FOV and the proportion of tracking targets is ne-
cessary if a static reference need to be in the FOV. Therefore, camera
motion cancellation is still an open problem.
3.3.2. Errors from coordinate conversion
The scaling factor determined by both of the coordinate conversion
methods described earlier can introduce measurement errors. For the
camera calibration method in Section 2.5.1, the extrinsic and intrinsic
matrices for camera calibration do not account for lens distortion,
mostly radial distortion and slight tangential distortion. Lava et al.
[125] and Pan et al. [126] investigated the impact of lens distortion on
the uncertainty of DIC measurement. In order to accurately represent a
real camera for an accurate camera calibration, a general lens distortion
model should also be considered during camera calibration [26]. In
literature [127], a method of lens distortion correction is proposed to
improve the measurement accuracy of DIC for 2D displacement mea-
surement. The lens distortion was first evaluated from displacement
distributions obtained in a rigid body in-plane translation and rotation
tests, and then the measured displacement was corrected using a
coefficient determined by a least squares method. Note that in field
applications of measuring small structural displacement with a rela-
tively long focal-length lens from a remote distance, the errors caused
by the lens distortion may be negligible.
For the practical calibration method described in Section 2.5.2,
there also exist uncertainties in the estimated scaling factor. For the
scaling factors SF2 and SF3, estimation errors would arise from the
uncertainties in the tilt angle estimation, camera distance measurement
and focal length readings from the adjustable-focal-length lens. Using a
fixed zoom lens and an angle measurement device can minimize such
errors. For the scaling factor SF1 based on the known physical dimen-
sion (such as the length of a truss member) on the target surface and the
corresponding image dimension in pixels, errors in the range of ± 2
113
Engineering Structures 156 (2018) 105–117
pixels could occur when selecting the physical members from the image
using a mouse. It is recommended to use the mean value from several
repeated picking operations to average out some of the random errors.
Furthermore, if a dot extraction algorithm is implemented, the error
could be reduced to ± 1/10 th of a pixel. Feng et al. [47] theoretically
studied the effects of the optical axis tilt angle and lens focal length
based on the 1D in-plane translation. It was found that the errors in SF1
and SF3 increase as the tilt angle increases and the error is inversely
related to the focal length. Furthermore, through laboratory tests, Feng
et al. [54] demonstrated that the estimated scaling factors SF1 utilizing
known physical dimensions can yield satisfactory accuracy when the
camera tilt angle is small (e.g., 9°). It is also suggested that for the non-
perpendicular lens optical axis, scaling factors in the horizontal and
vertical directions should be obtained separately. Moreover, when a
series of targets are simultaneously tracked with a single camera to
measure displacements at multiple points along the structure, due to
the projective distortion, different scaling factors should be determined
for each measurement point by utilizing the structural dimension closer
to or encompassing that point.
3.3.3. Errors from hardware limitations
Measurement errors using vision-based system can further arise
from hardware limitations such as the rolling shutter effect and the
temporal aliasing. Two types of image sensors widely used in digital
cameras are the CCD and complementary metal oxide semiconductors
(CMOS). Although CMOS imaging sensors has brought positive im-
provement over CCD, one major distinction between the two sensors is
the readout modes. CCD cameras often use the global shutter mode,
which captures the entire image frame at the same instant. This is
particularly beneficial when the image is changing from frame to frame.
By contrast, most CMOS cameras use the rolling shutter mode, in which
each image frame is recorded by scanning row-by-row or column-by-
column in the pixels. In other words, the rolling shutter method may
produce distortions when recording fast-moving objects, the effect of
which on the measurement accuracy should be rectified when this type
of video cameras are used [128]. Note that not all CMOS sensors have
rolling shutters. For example, in [53], the adopted video camera (Point
Grey/ FL3-U3-13Y3M-C) has a CMOS-type sensor but a global shutter.
The temporal aliasing is another concern for vibration tests. When
structural response has frequency of higher than half of the frame rate,
the measured displacement will contain aliased information from
higher frequencies. Different from the other vibration sensor systems
where the aliasing effect can be removed by using an anti-aliasing filter,
the temporal aliasing cannot be removed in vision-based systems since
images are already aliased [79,129]. According to Nyquist theorem, in
order to avoid aliasing, the sample rate must be at least twice of the
highest frequency component in the measured signal [41]. For field
measurement of a structure, whose natural frequencies are unknown,
preliminary analysis can be conducted to estimate the frequencies and
other sensors (such as laser or accelerometers) can be used to conduct
some preliminary measurements [108]. It is noted that for civil en-
gineering structures, the dominant natural frequencies are usually less
D. Feng, M.Q. Feng
than 50 Hz, and thus a sampling rate of 100 Hz should be sufficient to
avoid the aliasing problem.
3.3.4. Errors from environmental sources
It is well-known that the accuracy of the template matching tech-
niques is largely dependent on the image quality, which is often diffi-
cult to guarantee in outdoor field environmental conditions such as il-
lumination variation, partial target occlusion, partial shading, and
background disturbance, etc. Shadow and low lighting conditions are
the fundamental technical limitation of computer vision systems [110].
For high speed, high magnification or night-time applications, lighting
the area under test is important. Measurement errors can also arise from
the heat haze that occurs when the air is heated non-uniformly by the
high ambient temperature during the field testing [130]. The non-
uniformly heated air causes variation in its optical refraction index,
resulting in image distortion. The measurement errors caused by the
heat haze increase as the measurement distance increases, because the
air volume between the target object and the lens of the camera be-
comes large. Research has been conducted to study measurement errors
from these environmental sources. For example, Ye et al. [62] con-
ducted a series of shaking table experiments to examine the environ-
mental influence factors affecting the accuracy and stability of the vi-
sion-based system. It is demonstrated that the measurement results are
adversely affected by illumination and vapor. Preliminary tests of a
video-based system by Ribeiro et al. [27] have shown that the mea-
surement precision can be affected by the distortion of the field of view,
caused by the flow of heat waves generated by IR incandescent lighting
and, therefore, the operating time of the lamps should be limited. Lee
et al. [131] pointed out that external factors such as precipitation, fog,
variation of natural light and wind action, may influence the perfor-
mance of a vision system. Anantrasirichai et al. [130] proposed novel
method for mitigating the effects of atmospheric distortion using
complex wavelet-based fusion.
4. Applications for SHM
Although study of vision-based sensor applications for SHM is still at
its early stage, extensive efforts have been made towards extracting
quantitative structural condition measures from the low-cost vision-
based displacement data. Some examples are presented as follows.
4.1. Modal property identification
SHM is often based on vibration measurement and structural modal
property identification. Available vibration sensors (such as accel-
erometers, GPS, and laser vibrometers) are mostly point sensors. As a
result, the spatial resolution of the obtained mode shapes is limited by
the number of deployed sensors, which can result in less accurate re-
sults of finite element (FE) model updating and damage localization.
Mas et al. [132] developed a method for simultaneous multipoint
measurement of vibration frequencies through the analysis of a high-
speed video sequence. Wang et al. [133] carried out full-field vibration
measurement on the 3D surface of the bonnet by a 3D DIC system with
random excitation, from which modal parameters of the car bonnet
were successfully identified by using the frequency response functions
(FRFs) of the shape features of the DIC measurements. Through la-
boratory experiments on a scaled simply beam and a 3-story frame
structures, Feng and Feng [54,110] demonstrated that dynamic dis-
placement responses at a series of points can be simultaneously and
accurately measured using one camera, and the identified natural fre-
quencies and mode shapes by the vision sensor match well with those
by multiple accelerometers. Yoon et al. [79] carried out laboratory
vibration experiments on a scaled frame structure. Modal parameters
were identified and agreed with those by the conventional accel-
erometer-based method. Poozesh et al. [50] extracted operating mode
shapes and natural frequencies of a small-scale wind turbine blade by
114
Engineering Structures 156 (2018) 105–117
using two synchronized stereo-vision systems in conjunction with
output-only system identification. The measurements obtained from the
camera pairs are stitched together. The extracted modal properties were
shown to be accurate when compared to those from a validated finite
element model. This study validated the effectiveness of the stitching
approach for developing a multi-camera system for monitoring the
entire surface of large sized blades. Through experiments on scaled
laboratory structures, Yang et al. [134,135] showed the potential for
output-only modal identification using either non-aliased measure-
ments or temporally-aliased video measurements at low frame rates.
4.2. Model updating and damage detection
Finite element models of a structure can be updated by comparing
the analytical and experimental modal properties including the natural
frequencies, mode shapes, and damping ratios based on the vision
sensor measurement [136]. The updated model can then be used for
structural damage detection. For example, based on the measured dis-
placement on a cantilever beam by the phase-based optical flow algo-
rithm, Cha et al. [77] utilized the unscented Kalman filter to detect
structural damage by identifying structural properties such as stiffness
and damping coefficient with an assumption of known structural mass.
Feng and Feng [54] experimentally demonstrated that a smooth mode
shapes from full-field displacement responses measured by a single
camera enabled structural damage detection and location of a simple
beam based on the mode shape curvature index. Feng and Feng [110]
utilized the identified modal parameters from the vision sensor to
successfully update the inter-story stiffness of a laboratory frame
structure. Oh et al. [137] conducted model updating using a multi-
objective optimization algorithm based on displacement responses from
a motion capture system (MCS). Through a free vibration test of a three-
story shear frame model, the performance of the model updating
method is validated by comparing the dynamics properties between the
updated model and the direct MCS measurement. Wang et al. [138,139]
demonstrated that the region-based Zernike moment descriptor (ZMD)
is a robust image processing technique for mode-shape recognition and
finite element model updating of simple plate structures. Furthermore,
FE model updating of nonlinear elasto-plastic material properties was
carried out using the ZMD derived from full-field strain measurements
[140]. Song et al. [33] demonstrated the use of subpixel virtual visual
sensors to acquire modal shapes and frequencies of structures and their
use in a wavelet-based structural damage detection algorithm through
laboratory experiments on steel cantilever beams. Dworakowski et al.
[141] obtained the deflection curve of small-scale laboratory beams by
means of digital image correlation (DIC), and evaluated two deflection
shape-based algorithms for damage detections of the beams. Feng and
Feng [40] proposed a time-domain method to identify the equivalent
stiffness of a railway bridge based on vision-based displacement mea-
surement with the prerequisite of known trainloads. Sensitivity studies
showed that train-induced displacement response is more suited than
acceleration responses to identify the bridge stiffness. Based on ex-
periments on a laboratory scaled beam structure, Feng and Feng [142]
further demonstrated that global stiffness of the beam specimen as well
as external hammer excitation forces can be successfully and accurately
identified from displacement measurement at two points using one
camera.
4.3. Cable force estimation
A cable system is the most important component in cable-supported
bridges and roof structures and their tensions need to be monitored.
Existing methods for cable tension estimation are based on acceleration
response measurement using accelerometers attached on each of the
cables. Such practice is relatively expensive and time-consuming due to
the required installation of the sensors and data acquisition systems
[128]. A few attempts have been made to apply vision-based sensors for
D. Feng, M.Q. Feng
cable force estimates [31,32,123,128]. For example, Ji and Chang [32]
and Kim et al.[31] employed respectively the optical flow and nor-
malized cross correlation temptation methods for cable vibration
measurement and cable force estimates. To ensure that cable forces
reach their design values, a series of field tests have been carried out by
Feng et al. [128] to measure cable forces for the cable-supported roof
structure of the Hard Rock Stadium in Florida. Satisfactory agreements
are observed between measured cable forces by the vision-based sensor
and the reference readings from load cells.
4.4. Other SHM-related applications
In addition to the aforementioned remote measurement of bridge
displacement, computer vision can also be applied to classify traveling
vehicles and measure their weights and positions. For example, Khan
et al. [143] presented a design and application of novel multisensory
testbeds for collection, synchronization, archival, and analysis of mul-
timodal data for health monitoring of transportation infrastructures,
where computer-vision algorithms are used to detect and track vehicles
and extract their properties. Tan et al. [144] used a video images of
passing vehicles to extract information about the vehicle types, arrival
times and speeds to develop a physics-based model of the traffic ex-
citation on bridges. The effectiveness of this video-assisted approach
was validated by the field experimental results on a bridge. Catbas et al.
[145] presented a methodology for bridge load rating by integrating
computer images of passing vehicles with strain gauge measurements.
Ojio et al. [94] proposed a contactless bridge weight-in-motion (WIM)
without the need for any sensors to be attached to the bridge. One
camera is used to track bridge deflections subject to vehicular loadings,
while a second camera is used to monitor traffic and to determine axle
spacing. The static axle weights can be found by minimizing the
squared differences between the measured and theoretical deflection
responses. By developing the capability of simultaneously measuring
traffic loading input and bridge displacement output, vision-based
system can provide more accurate and actionable information to assess
the bridge structural condition and safety.
Yeum and Dyke [146] proposed the vision-based automated crack
detection for bridge inspection. As a pilot study, cracks near bolts on a
steel structure are identified from images. Similarly, Cha et al. [147]
proposed a vision-based method using a deep architecture of convolu-
tional neural networks for detecting concrete cracks without calculating
the defect features. Vision sensor systems can also be used for real-time
tracking of water levels without disturbing the water flow. For example,
in order to characterize the dynamics of tuned liquid column dampers
(TLCDs), vision-based sensing system was developed by Kim et al. [148]
for measurement of the water depth with time during shaking table
dynamic tests. Dynamic characteristics such as frequencies and
damping ratios of the TLCDs are then estimated from measured ex-
perimental data. Additionally, the vision sensors can be used to track
earth mass movements such as glacier flow, rockglacier creep and land
sliding, etc. For example, Debella-Gilo and Kaab [89] evaluated the
accuracy of pixel and subpixel image processing algorithms when
measuring surface displacements on mass movements.
5. Conclusions
This paper presents a comprehensive review on the recent devel-
opment of computer vision-based sensors for structural displacement
response measurement and their applications for SHM. The goal is to
help broaden the application of this emerging low-cost sensor tech-
nology in not only scientific research but also engineering practice such
as field condition assessment of aging civil engineering structures and
infrastructure systems.
The general principles of vision-based sensor systems based on
various template matching algorithms are firstly reviewed. Importation
issues critical to successful measurement are discussed in detail,
115
Engineering Structures 156 (2018) 105–117
including how to convert pixel displacements to physical displace-
ments, how to achieve sub-pixel resolutions, and what to cause mea-
surement errors and how to mitigate the errors. Other subjects of in-
terest include the comparison between measurement using artificial
targets versus natural features on structural surfaces, 2D versus 3D
measurement, and real-time versus post image processing, etc. To
evaluate measurement accuracy and demonstrate the unique features
and merits of vision-based structural monitoring, the research com-
munity has undertaken both laboratory and field experimental studies
on a wide range of structures, including buildings, bridges, wind tur-
bine and mechanical structures. Some of these studies have further
applied the measurement data for SHM, including modal parameter
identification, structural model updating, damage detection, as well as
cable force estimation.
However, in many respects, the vision-based sensor technology is
still in its infancy. The majority studies have still been focused on
measurements of small-scale laboratory structures or field measure-
ments of large structures at a limited number of points for a short period
of time. In the near future, the technology is expected to be deployed in
real structures to fully validate its performance in outdoor field en-
vironments. For large size structures such as long-span bridges, multiple
synchronized cameras targeting different sections of the structure will
be applied to monitor the entire structure. Another direction is to fur-
ther improve measurement accuracy, resolution and robustness, by
addressing the error sources discussed in Section 3.3. Most of the cur-
rent field studies have focused on measurement of relatively large
amplitude displacements such as bridges subjected to moving train-
loads. The various noise sources in complex outdoor conditions such as
heat haze pose challenges for accurate measurement of small-amplitude
displacements, such as response of short- or medium-span concrete
bridges under light-weight vehicles or ambient excitations.
Most existing image processing software programs are for post-
processing of recorded video images. Future applications of vision
sensors for long-term continuous monitoring of structures in the field
will require real-time on-site image processing. Increasingly ubiquitous
traffic and security cameras or cell phone cameras provide opportu-
nities to tap into low-cost sensor resources for real-time SHM and safety
assessment.
Acknowledgement
This work is supported by NCHRP Highway IDEA Project (No. 20-
30/IDEA 189). The authors would also like to acknowledge the anon-
ymous reviewers for their constructive comments which helped in im-
proving the quality of this paper.
References
[1] Wang H, Tao T, Guo T, Li J, Li A. Full-scale measurements and system identifi-
cation on sutong cable-stayed bridge during typhoon Fung-Wong. Sci World J
2014;2014:13.
[2] Feng D, Feng MQ. Model updating of railway bridge using in situ dynamic dis-
placement measurement under trainloads. J Bridge Eng 2015;20(12):04015019.
[3] Wang H, Li A, Guo T, Tao T. Establishment and application of the wind and
structural health monitoring system for the runyang Yangtze River bridge. Shock
Vibration 2014;2014:15.
[4] Guo T, Li A, Wang H. Influence of ambient temperature on the fatigue damage of
welded bridge decks. Int J Fatigue 2008;30:1092–102.
[5] Chen Y, Feng M, Soyoz S. Large-scale shake table test verification of bridge con-
dition assessment methods. J Struct Eng 2008;134:1235–45.
[6] Chen Y, Feng M. Structural health monitoring by recursive Bayesian filtering. J Eng
Mech 2009;135:231–42.
[7] Carden EP, Fanning P. Vibration based condition monitoring: a review. Struct
Health Monit 2004;3:355–77.
[8] Farrar CR, Doebling SW, Nix DA. Vibration–based structural damage identification.
Philos Trans R Soc London, A 2001;359:131–49.
[9] Mita A, Takahira S. Risk control of smart structures using damage index sensors.
Advances in Building Technology. Oxford: Elsevier; 2002. p. 521–8.
[10] Catbas FN, Susoy M, Frangopol DM. Structural health monitoring and reliability
estimation: Long span truss bridge application with environmental monitoring
data. Eng Struct 2008;30:2347–59.
[11] Xing Z, Mita A. A substructure approach to local damage detection of shear
D. Feng, M.Q. Feng
structure. Struct Control Health Monit 2012;19:309–18.
[12] Lynch J, Loh K. A summary review of wireless sensors and sensor networks for
structural health monitoring. Shock Vibration Digest 2006;38:91–128.
[13] Li J, Mechitov KA, Kim RE, Spencer BF. Efficient time synchronization for struc-
tural health monitoring using wireless smart sensor networks. Struct Control
Health Monit 2016;23:470–86.
[14] Sabato A, Niezrecki C, Fortino G. Wireless MEMS-based accelerometer sensor
boards for structural vibration monitoring: a review. IEEE Sens J 2017;17:226–35.
[15] Li S, Wu Z. Development of distributed long-gage fiber optic sensing system for
structural health monitoring. Struct Health Monit 2007;6:133–43.
[16] Kim DH, Feng MQ. Real-time structural health monitoring using a novel fiber-optic
accelerometer system. IEEE Sens J 2007;7:536–43.
[17] Feng MQ, Kim D-H. Novel fiber optic accelerometer system using geometric moiré
fringe. Sens Actuators, A 2006;128:37–42.
[18] Gentile C, Bernardini G. An interferometric radar for non-contact measurement of
deflections on civil engineering structures: laboratory and full-scale tests. Struct
Infrastructure Eng 2010;6:521–34.
[19] Lee JJ, Fukuda Y, Shinozuka M, Cho S, Yun CB. Development and application of a
vision-based displacement measurement system for structural health monitoring of
civil structures. Smart Struct Syst 2007;3:373–84.
[20] Stephen GA, Brownjohn JMW, Taylor CA. Measurements of static and dynamic
displacement from visual monitoring of the Humber Bridge. Eng Struct
1993;15:197–208.
[21] Park J-W, Lee J-J, Jung H-J, Myung H. Vision-based displacement measurement
method for high-rise building structures using partitioning approach. NDT E Int
2010;43:642–7.
[22] Ye XW, Dong CZ, Liu T. A review of machine vision-based structural health
monitoring: methodologies and applications. J Sens 2016;2016:10.
[23] Zhang D, Guo J, Lei X, Zhu C. A high-speed vision-based sensor for dynamic vi-
bration analysis using fast motion extraction algorithms. Sensors 2016;16:572.
[24] Qin J, Gao Z, Wang X, Yang S. Three-dimensional continuous displacement mea-
surement with temporal speckle pattern interferometry. Sensors 2016;16:2020.
[25] Farzad N, Dimitrios K. Evaluation of vision-based measurements for shake-table
testing of nonstructural components.
[26] Wang Z, Kieu H, Nguyen H, Le M. Digital image correlation in experimental me-
chanics and image registration in computer vision: Similarities, differences and
complements. Opt Lasers Eng 2015;65:18–27.
[27] Ribeiro D, Calçada R, Ferreira J, Martins T. Non-contact measurement of the dy-
namic displacement of railway bridges using an advanced video-based system. Eng
Struct 2014;75:164–80.
[28] Kohut P, Holak K, Uhl T, Ortyl Ł, Owerko T, Kuras P, et al. Monitoring of a civil
structure’s state based on noncontact measurements. Struct Health Monit 2013.
[29] Lee JJ, Shinozuka M. A vision-based system for remote sensing of bridge dis-
placement. NDT E Int 2006;39:425–31.
[30] Busca G, Cigada A, Mazzoleni P, Zappa E. Vibration monitoring of multiple bridge
points by means of a unique vision-based measuring system. Exp Mech
2014;54:255–71.
[31] Kim S-W, Jeon B-G, Kim N-S, Park J-C. Vision-based monitoring system for eval-
uating cable tensile forces on a cable-stayed bridge. Struct Health Monit
2013;12:440–56.
[32] Ji Y, Chang C. Nontarget image-based technique for small cable vibration mea-
surement. J Bridge Eng 2008;13:34–42.
[33] Song Y-Z, Bowen CR, Kim AH, Nassehi A, Padget J, Gathercole N. Virtual visual
sensors and their application in structural health monitoring. Struct Health Monit
2014;13:251–64.
[34] Zhang H, Hu S, Zhang X. SIFT flow for large-displacement object tracking. Appl
Opt 2014;53:6194–205.
[35] Gehle RW, Masri SF. Tracking the multi-component motion of a cable using a
television camera. Smart Mater Struct 1998;7:43.
[36] Jeon H, Bang Y, Myung H. A paired visual servoing system for 6-DOF displacement
measurement of structures. Smart Mater Struct 2011;20:045019.
[37] Lee J-H, Ho H-N, Shinozuka M, Lee J-J. An advanced vision-based system for real-
time displacement measurement of high-rise buildings. Smart Mater Struct
2012;21:125019.
[38] Santos CA, Costa CO, Batista JP. Calibration methodology of a vision system for
measuring the displacements of long-deck suspension bridges. Struct Control
Health Monit 2012;19:385–404.
[39] Feng M, Fukuda Y, Feng D, Mizuta M. Nontarget vision sensor for remote mea-
surement of bridge dynamic response. J Bridge Eng 2015;04015023.
[40] Feng D, Feng M. Model updating of railway bridge using in situ dynamic dis-
placement measurement under trainloads. J Bridge Eng 2015;04015019.
[41] Wu L-J, Casciati F, Casciati S. Dynamic testing of a laboratory model via vision-
based sensing. Eng Struct 2014;60:113–25.
[42] Olaszek P. Investigation of the dynamic characteristic of bridge structures using a
computer vision method. Measurement 1999;25:227–36.
[43] Wahbeh AM, John PC, Sami FM. A vision-based approach for the direct mea-
surement of displacements in vibrating systems. Smart Mater Struct 2003;12:785.
[44] Ho H-N, Lee J-H, Park Y-S, Lee J-J. A synchronized multipoint vision-based system
for displacement measurement of civil infrastructures. Sci World J 2012;2012:9.
[45] Myung H, Lee S, Lee B. Paired structured light for structural health monitoring
robot system. Struct Health Monit 2010.
[46] Caetano E, Silva S, Bateira J. A vision system for vibration monitoring of civil
engineering structures. Exp Tech 2011;35:74–82.
[47] Feng D, Feng M, Ozer E, Fukuda Y. A vision-based sensor for noncontact structural
displacement measurement. Sensors 2015;15:16557–75.
[48] Ye XW, Ni YQ, Wai TT, Wong KY, Zhang XM, Xu F. A vision-based system for
dynamic displacement measurement of long-span bridges: algorithm and ver-
ification. Smart Struct Syst 2013;12:363–79.
[49] Pan B, Tian L, Song X. Real-time, non-contact and targetless measurement of
vertical deflection of bridges using off-axis digital image correlation. NDT E Int
116
Engineering Structures 156 (2018) 105–117
2016;79:73–80.
[50] Poozesh P, Baqersad J, Niezrecki C, Avitabile P, Harvey E, Yarala R. Large-area
photogrammetry based testing of wind turbine blades. Mech Syst Signal Process
2017;86(Part B):98–115.
[51] Pan B, Qian K, Xie H, Asundi A. Two-dimensional digital image correlation for in-
plane displacement and strain measurement: a review. Meas Sci Technol
2009;20:062001.
[52] Hild F, Roux S. Digital image correlation: from displacement measurement to
identification of elastic properties – a review. Strain 2006;42:69–80.
[53] Feng D, Feng MQ. Vision-based multipoint displacement measurement for struc-
tural health monitoring. Struct Control Health Monit 2016;23(5):876–90.
[54] Feng D, Feng MQ. Experimental validation of cost-effective vision-based structural
health monitoring. Mech Syst Signal Process 2017;88:199–211.
[55] Mattoccia S, Tombari F, Di Stefano L. Efficient template matching for multi-
channel images. Pattern Recogn Lett 2011;32:694–700.
[56] Mahalakshmi T, Muthaiah R, Swaminathan P. Review article: an overview of
template matching technique in image processing. Res J Appl Sci, Eng Technol
2012.
[57] Soh Y, Qadir M, Mehmood A, Hae Y, Ashraf H, Kim I. A feature area-based image
registration. Int J Comput Theory Eng 2014;6:407–11.
[58] Brunelli R. Template matching techniques in computer vision: theory and practice.
A John Wiley and Sons, Ltd; 2009.
[59] Zitová B, Flusser J. Image registration methods: a survey. Image Vis Comput
2003;21:977–1000.
[60] Pratt WK. Digital image processing: PIKS inside. John Wiley & Sons, Inc.; 2001.
[61] Yoneyama S, Ueda H. Bridge deflection measurement using digital image corre-
lation with camera movement correction. Mater Trans, JIM 2012;53:285–90.
[62] Ye XW, Yi T-H, Dong CZ, Liu T. Vision-based structural displacement measure-
ment: System performance evaluation and influence factor analysis. Measurement
2016;88:372–84.
[63] Dworakowski Z, Kohut P, Gallina A, Holak K, Uhl T. Vision-based algorithms for
damage detection and localization in structural health monitoring. Struct Control
Health Monit 2016;23:35–50.
[64] Hassaballah M, Abdelmgeid AA, Alshazly HA. Image features detection, descrip-
tion and matching. In: Awad AI, Hassaballah M, editors. Image feature detectors
and descriptors: foundations and applications. Cham: Springer International
Publishing; 2016. p. 11–45.
[65] Mair E, Hager GD, Burschka D, Suppa M, Hirzinger G. Adaptive and generic corner
detection based on the accelerated segment test. In: Daniilidis K, Maragos P,
Paragios N, editors. Computer vision – ECCV 2010: 11th European conference on
computer vision, Heraklion, Crete, Greece, September 5–11, 2010, Proceedings,
Part II. Berlin Heidelberg: Springer; 2010. p. 183–96.
[66] Bay H, Tuytelaars T, Van Gool L. SURF: speeded up robust features. In: Leonardis
A, Bischof H, Pinz A, editors. Computer vision – ECCV 2006: 9th European con-
ference on computer vision, Graz, Austria, May 7–13, 2006 Proceedings, Part I.
Berlin Heidelberg: Springer; 2006. p. 404–17.
[67] Alcantarilla PF, Bartoli A, Davison AJ. KAZE features. In: Fitzgibbon A, Lazebnik S,
Perona P, Sato Y, Schmid C, editors. Computer vision – ECCV 2012: 12th European
conference on computer vision, Florence, Italy, October 7–13, 2012, Proceedings,
Part VI. Berlin Heidelberg: Springer; 2012. p. 214–27.
[68] Mikolajczyk K, Tuytelaars T, Schmid C, Zisserman A, Matas J, Schaffalitzky F, et al.
A comparison of affine region detectors. Int J Comput Vision 2005;65:43–72.
[69] Leutenegger S, Chli M, Siegwart RY. BRISK: binary robust invariant scalable
keypoints. In: Proceedings of the 2011 international conference on computer vi-
sion: IEEE computer society; 2011. p. 2548–55.
[70] Ortiz R. FREAK: fast retina keypoint. In: Proceedings of the 2012 IEEE conference
on computer vision and pattern recognition (CVPR): IEEE computer society; 2012.
p. 510–7.
[71] Awad AI, Hassaballah M. Image feature detectors and descriptors: foundations and
applications. Springer International Publishing; 2016.
[72] Peng J, Peng S, Hu Y. Partial least squares and random sample consensus in outlier
detection. Anal Chim Acta 2012;719:24–9.
[73] Son K-S, Jeon H-S, Park J-H, Park JW. Vibration displacement measurement
technology for cylindrical structures using camera images. Nucl Eng Technol
2015;47:488–99.
[74] Feng B, Chen F, Liu G, Xiang Y, Liu B, Lv Z. Image-based displacement and rotation
detection using scale invariant features for 6 degree of freedom ICF target posi-
tioning. Appl Opt 2015;54:4130–4.
[75] Jeong H-J, Choi JD, Ha Y-G. Vision based displacement detection for stabilized
UAV control on cloud server. Mobile Inf Syst 2016;2016:1–11.
[76] Bay H, Ess A, Tuytelaars T, Van Gool L. Speeded-up robust features (SURF).
Comput Vis Image Underst 2008;110:346–59.
[77] Cha YJ, Chen JG, Büyüköztürk O. Output-only computer vision based damage
detection using phase-based optical flow and unscented Kalman filters. Eng Struct
2017;132:300–13.
[78] Guo J, Zhu Ca. Dynamic displacement measurement of large-scale structures based
on the Lucas-Kanade template tracking algorithm. Mech Syst Signal Process
2016;66–67:425–36.
[79] Yoon H, Elanwar H, Choi H, Golparvar-Fard M, Spencer BF. Target-free approach
for vision-based structural system identification using consumer-grade cameras.
Struct Control Health Monit 2016;23:1405–16.
[80] Liu B, Zhang D, Guo J, Zhu Ca. Vision-based displacement measurement sensor
using modified Taylor approximation approach. Opt Eng 2016;55:114103.
[81] Choi I, Kim J, Kim D. A target-less vision-based displacement sensor based on
image convex hull optimization for measuring the dynamic response of building
structures. Sensors 2016;16:2085.
[82] Bing P, Hui-min X, Bo-qin X, Fu-long D. Performance of sub-pixel registration al-
gorithms in digital image correlation. Meas Sci Technol 2006;17:1615.
[83] Foroosh H, Zerubia JB, Berthod M. Extension of phase correlation to subpixel re-
gistration. Image Process, IEEE Trans 2002;11:188–200.
D. Feng, M.Q. Feng
[84] Berenstein CA, Kanal LN, Lavine D, Olson EC. A geometric approach to subpixel
registration accuracy. Comput Vision, Graphics, Image Process 1987;40:334–60.
[85] Pilch A, Mahajan A, Chu T. Measurement of whole-field surface displacements and
strain using a genetic algorithm based intelligent image correlation method. J Dyn
Syst Meas Contr 2004;126:479–88.
[86] Li L, Chen Y, Yu X, Liu R, Huang C. Sub-pixel flood inundation mapping from
multispectral remotely sensed images based on discrete particle swarm optimiza-
tion. ISPRS J Photogrammetry Remote Sens 2015;101:10–21.
[87] Bruck HA, McNeill SR, Sutton MA, Peters III WH. Digital image correlation using
Newton-Raphson method of partial differential correction. Exp Mech
1989;29:261–7.
[88] Davis CQ, Freeman DM. Statistics of subpixel registration algorithms based on
spatiotemporal gradients or block matching. Opt Eng 1998;37:1290–8.
[89] Debella-Gilo M, Kääb A. Sub-pixel precision image matching for measuring surface
displacements on mass movements using normalized cross-correlation. Remote
Sens Environ 2011;115:130–42.
[90] Feng D, Feng M, Ozer E, Fukuda Y. A vision-based sensor for noncontact structural
displacement measurement. Sensors 2015;15:16557.
[91] Mas D, Perez J, Ferrer B, Espinosa J. Realistic limits for subpixel movement de-
tection. Appl Opt 2016;55:4974–9.
[92] Fukuda Y, Feng MQ, Shinozuka M. Cost-effective vision-based system for mon-
itoring dynamic response of civil engineering structures. Struct Control Health
Monit 2010;17:918–36.
[93] Jong-Han L, Hoai-Nam H, Masanobu S, Jong-Jae L. An advanced vision-based
system for real-time displacement measurement of high-rise buildings. Smart
Mater Struct 2012;21:125019.
[94] Doherty C, Carey CH, OBrien EJ, Taylor SE, Ojio T. Contactless bridge weigh-in-
motion; 2016.
[95] Zhang Z. A flexible new technique for camera calibration. IEEE Trans Pattern Anal
Mach Intell 2000;22:1330–4.
[96] Heikkil J. #228. Geometric camera calibration using circular control points. IEEE
Trans Pattern Anal Mach Intell 2000;22:1066–77.
[97] Luo P-F, Wu J. Easy calibration technique for stereo vision using a circle grid. Opt
Eng 2008;47:033607–33610.
[98] Park SW, Park HS, Kim JH, Adeli H. 3D displacement measurement model for
health monitoring of structures using a motion capture system. Measurement
2015;59:352–62.
[99] Brownjohn J, Hester D, Xu Y, Bassitt J, koo KY. Viability of optical tracking sys-
tems for monitoring deformations of a long span bridge. In: EACS 2016–6th
European conference on structural control; 2016.
[100] Fukuda Y, Feng MQ, Narita Y, Kaneko S, Tanaka T. Vision-based displacement
sensor for monitoring dynamic response using robust object search algorithm. Sens
J, IEEE 2013;13:4725–32.
[101] Feng D, Feng MQ. Vision-based multipoint displacement measurement for struc-
tural health monitoring. Struct Control Health Monit 2015;876–90.
[102] Feng DM. Advanced vision-based displacement sensors for structural health
monitoring. Columbia University Academic Commons 2016.
[103] Sładek J, Ostrowska K, Kohut P, Holak K, Gąska A, Uhl T. Development of a vision
based deflection measurement system and its accuracy assessment. Measurement
2013;46:1237–49.
[104] Chen JG, Wadhwa N, Cha YJ, Durand F, Freeman WT, Buyukozturk O. Modal
identification of simple structures with high-speed video using motion magnifi-
cation. J Sound Vib 2015;345:58–71.
[105] Bartilson DT, Wieghaus KT, Hurlebaus S. Target-less computer vision for traffic
signal structure vibration studies. Mech Syst Signal Process 2015;60–61:571–82.
[106] Khuc T, Catbas FN. Completely contactless structural health monitoring of real-life
structures using cameras and computer vision. Struct Control Health Monit 2016.
http://dx.doi.org/10.1002/stc.852.
[107] Sutton M, Yan JH, Tiwari V, Orteu JJ. The effect of out-of-plane motion on 2D and
3D digital image correlation measurements. Opt Lasers Eng 2008;46:746–57.
[108] D’Emilia G, Razzè L, Zappa E. Uncertainty analysis of high frequency image-based
vibration measurements. Measurement 2013;46:2630–7.
[109] Baqersad J, Poozesh P, Niezrecki C, Avitabile P. Photogrammetry and optical
methods in structural dynamics - a review. Mech Syst Signal Process 2016.
[110] Feng D, Feng MQ. Vision-based multipoint displacement measurement for struc-
tural health monitoring. Struct Control Health Monit 2016;23:876–90.
[111] Kohut P, Holak K, Uhl T, Ortyl Ł, Owerko T, Kuras P, et al. Monitoring of a civil
structure’s state based on noncontact measurements. Struct Health Monit
2013;12:411–29.
[112] Nikfar F, Konstantinidis D. Evaluation of vision-based measurements for shake-
table testing of nonstructural components. J Comput Civil Eng 2016;04016050.
[113] Min JH, Gelo NJ, Jo H. Non-contact and real-time dynamic displacement mon-
itoring using smartphone technologies. J Life Cycle Reliability Saf Eng
2015;4:40–51.
[114] Ozer E, Feng D, Feng MQ. Hybrid motion sensing and experimental modal analysis
using collocated smartphone camera and accelerometers. Meas Sci Technol
2017;28:105903.
[115] Warren C, Niezrecki C, Avitabile P, Pingle P. Comparison of FRF measurements
and mode shapes determined using optically image based, laser, and accel-
erometer measurements. Mech Syst Signal Process 2011;25:2191–202.
[116] Shariati A, Schumacher T. Eulerian-based virtual visual sensors to measure dy-
namic displacements of structures. Struct Control Health Monit; 2016:e1977-n/a.
[117] Tian L, Pan B. Remote bridge deflection measurement using an advanced video
deflectometer and actively illuminated LED targets. Sensors (Basel, Switzerland)
2016;16:1344.
117
View publication stats
Engineering Structures 156 (2018) 105–117
[118] Wang D, DiazDelaO FA, Wang W, Lin X, Patterson EA, Mottershead JE.
Uncertainty quantification in DIC with Kriging regression. Opt Lasers Eng
2016;78:182–95.
[119] Roux S, Hild F. Stress intensity factor measurements from digital image correla-
tion: post-processing and integrated approaches. Int J Fract 2006;140:141–57.
[120] Besnard G, Hild F, Roux S. “Finite-element” displacement fields analysis from
digital images: application to Portevin–Le Châtelier bands. Exp Mech
2006;46:789–803.
[121] Haddadi H, Belhabib S. Use of rigid-body motion for the investigation and esti-
mation of the measurement errors related to digital image correlation technique.
Opt Lasers Eng 2008;46:185–96.
[122] Ferrer B, Mas D, García-Santos JI, Luzi G. Parametric study of the errors obtained
from the measurement of the oscillating movement of a bridge using image pro-
cessing. J Nondestruct Eval 2016;35:53.
[123] Kim S-W, Kim N-S. Dynamic characteristics of suspension bridge hanger cables
using digital image processing. NDT E Int 2013;59:25–33.
[124] Neal W, Frédo D, Justin GC, Oral B, William TF, Abe D. Video camera-based vi-
bration measurement for civil infrastructure applications.
[125] Lava P, Van Paepegem W, Coppieters S, De Baere I, Wang Y, Debruyne D. Impact
of lens distortions on strain measurements obtained with 2D digital image corre-
lation. Opt Lasers Eng 2013;51:576–84.
[126] Pan B, Yu L, Wu D, Tang L. Systematic errors in two-dimensional digital image
correlation due to lens distortion. Opt Lasers Eng 2013;51:140–7.
[127] Yoneyama S, Kikuta H, Kitagawa A, Kitamura K. Lens distortion correction for
digital image correlation by measuring rigid body displacement. Opt Eng
2006;45:023602–23609.
[128] Feng D, Scarangello T, Feng MQ, Ye Q. Cable tension force estimate using novel
noncontact vision-based sensor. Measurement 2017;99:44–52.
[129] Choi H-S, Cheung J-H, Kim S-H, Ahn J-H. Structural dynamic displacement vision
system using digital image processing. NDT E Int 2011;44:597–608.
[130] Anantrasirichai N, Achim A, Kingsbury NG, Bull DR. Atmospheric turbulence
mitigation using complex wavelet-based fusion. IEEE Trans Image Process
2013;22:2398–408.
[131] Lee JW, Oh JS, Park MK, Kwon SD, Kwark JW. Bridge displacement measurement
system using image processing. In: Advances in bridge maintenance, safety man-
agement, and life-cycle performance, set of book & CD-ROM: CRC Press; 2006. p.
281–2.
[132] Mas D, Ferrer B, Acevedo P, Espinosa J. Methods and algorithms for video-based
multi-point frequency measuring and mapping. Measurement 2016;85:164–74.
[133] Wang W, Mottershead JE, Siebert T, Pipino A. Frequency response functions of
shape features from full-field vibration measurements using digital image corre-
lation. Mech Syst Signal Process 2012;28:333–47.
[134] Yang Y, Dorn C, Mancini T, Talken Z, Nagarajaiah S, Kenyon G, et al. Blind
identification of full-field vibration modes of output-only structures from uni-
formly-sampled, possibly temporally-aliased (sub-Nyquist), video measurements. J
Sound Vib 2017;390:232–56.
[135] Yang Y, Dorn C, Mancini T, Talken Z, Kenyon G, Farrar C, et al. Blind identification
of full-field vibration modes from video measurements with phase-based video
motion magnification. Mech Syst Signal Process 2017;85:567–90.
[136] Wang W, Mottershead JE, Ihle A, Siebert T, Reinhard Schubach H. Finite element
model updating from full-field vibration measurement using digital image corre-
lation. J Sound Vib 2011;330:1599–620.
[137] Oh BK, Hwang JW, Choi SW, Kim Y, Cho T, Park HS. Dynamic displacements-
based model updating with motion capture system. Struct Control Health Monit;
2016:n/a-n/a.
[138] Wang WZ, Mottershead JE, Mares C. Mode-shape recognition and finite element
model updating using the Zernike moment descriptor. Meas Sci Technol
2009;23:2088–112.
[139] Wang WZ, Mottershead JE, Mares C. Vibration mode shape recognition using
image processing. J Sound Vib 2009;326:909–38.
[140] Wang W, Mottershead JE, Sebastian CM, Patterson EA. Shape features and finite
element model updating from full-field strain data. Int J Solids Struct
2011;48:1644–57.
[141] Dworakowski Z, Kohut P, Gallina A, Holak K, Uhl T. Vision-based algorithms for
damage detection and localization in structural health monitoring. Struct Control
Health Monit 2016;35–50.
[142] Feng D, Feng MQ. Identification of structural stiffness and excitation forces in time
domain using noncontact vision-based displacement measurement. J Sound Vib
2017;406:15–28.
[143] Gandhi T, Chang R, Trivedi MM. Video and seismic sensor-based structural health
monitoring: framework, algorithms, and implementation. IEEE Trans Intell Transp
Syst 2007;8:169–80.
[144] Tan CA, Beyene Ashebo D, Feng MQ, Fukuda Y. Integration of traffic information
in the structural health monitoring of highway bridges; 2007. p. 65291D-D-10.
[145] Catbas FN, Zaurin R, Gul M, Gokce HB. Sensor networks, computer imaging, and
unit influence lines for structural health monitoring: case study for bridge load
rating. J Bridge Eng 2012;17:662–70.
[146] Yeum CM, Dyke SJ. Vision-based automated crack detection for bridge inspection.
Comput-Aided Civil Infrastructure Eng 2015;30:759–70.
[147] Cha Y-J, Choi W, Büyüköztürk O. Deep learning-based crack damage detection
using convolutional neural networks. Comput-Aided Civil Infrastructure Eng
2017;32:361–78.
[148] Kim J, Park C-S, Min K-W. Fast vision-based wave height measurement for dy-
namic characterization of tuned liquid column dampers. Measurement
2016;89:189–96.