Professional Documents
Culture Documents
Engineering Structures
journal homepage: www.elsevier.com/locate/engstruct
A R T I C L E I N F O A B S T R A C T
Keywords: Light and sign poles on elevated highway bridge are secondary structural elements that support functionality of a
Vision-based vibration measurement highway system. Over the time, they may suffer from fatigue damage due to traffic, wind, and earthquake-
Motion magnification induced vibration in a direct or indirect manner. As a secondary structural component, vibrations of light and
Light pole
sign poles are influenced by the bridge vibration, the primary structure. Structural condition assessment of the
Image processing
Dynamic mode decomposition
light and sign poles are normally conducted by visual inspection and/or vibration measurement using contact or
Operational modal analysis noncontact method. In this paper, a vision-based vibration measurement of the light poles on elevated highway
bridges using motion magnification and dynamic mode decomposition methods is proposed. Using the method,
vibrations of the light poles are captured by video camera and the phase-based motion magnification method is
implemented to magnify micro-vibration of the structure. By employing discretized centroid searching method,
spatial displacements of the light poles are extracted. Modal parameters of the structure are then identified from
the displacement responses by dynamic mode decomposition method. The study includes laboratory experiments
for verification, and the full-scale implementation on existing light poles on elevated highway bridges. The
vision-based results are compared with the noncontact vibration measurement using Laser-Doppler vibrometers.
Results of experiments and full-scale tests demonstrate the capability of the proposed method in extracting multi-
mode vibration characteristics of the light poles accurately in a noncontact long-distance measurement.
* Corresponding author at: Institute of Advanced Sciences, Yokohama National University, Japan.
E-mail address: dion@ynu.ac.jp (D.M. Siringoringo).
https://doi.org/10.1016/j.engstruct.2021.112728
Received 12 December 2020; Received in revised form 7 May 2021; Accepted 14 June 2021
Available online 24 June 2021
0141-0296/© 2021 Elsevier Ltd. All rights reserved.
D.M. Siringoringo et al. Engineering Structures 244 (2021) 112728
Vision-based measurement and monitoring systems have been technique. Spatial displacements along the structure are extracted from
viewed as promising tools for vibration measurement of civil structures the amplified vibration using the Discretized Centroid Searching Method
since they offer advantages over the traditional vibration sensors in (DCSM) that basically utilizes the principle of finding the centroid of a
terms of instrumentation cost, efforts, capacity of frequency range, and discretized moving object. Finally, the displacement responses are uti
spatial resolution [8,9]. One of the challenges in applying vision-based lized to obtain modal parameters of the light poles using the Dynamic
measurement system to civil structures is the inability to capture small Mode Decomposition (DMD) [24] method. The paper is organized such
amplitude structural motion during ambient vibration. However, recent that the basics on the methodologies (i.e., PVMM, DCSM and DMD) are
developments in computer vision have made a breakthrough via motion explained first. Then experiment verification is provided to demonstrate
magnification techniques. By employing the motion magnification al feasibility of the method for displacement extraction and estimation of
gorithm, one can extract detailed structural motion even under very modal parameters. Finally, implementation of the method to practical
small amplitude of excitation. Wadha et al. 2013 [10] developed the field measurements on the existing light poles mounted on highway
phase-based video motion magnification (PVMM) technique, which has viaducts is described.
been successfully implemented to obtain the amplified vibration of an
object for various purposes, ranging from the heart rate detection in the 2. Methodologies
medical science to machinery vibration in the mechanical engineering.
Using this method, micro-vibration of an object that was originally The three main methodologies employed in this study to obtain
invisible is amplified within frequency of interest, and the amplified multimode modal parameters of the light pole structure are: 1) Phase-
spatial displacements are extracted using available image processing based Video Motion Magnification (PVMM) to amplify micro-vibration
methods. Various computer vision-based signal processing and machine of the structure, 2) Discretized Centroid Searching Method (DCSM) to
learning techniques have been developed to extract vibration response extract spatial displacement responses of the structure from the video
of the structure from image analysis. Notable methods used for this frames; and 3) Dynamic Mode Decomposition (DMD) to estimate modal
purpose are based on, among others, the principle centroid detection parameters of the structure from spatial displacement responses. Fig. 1
[12], edge detection [13], principal component analysis, and blind illustrates the flowchart of all steps and the detailed processes required
source separation. [14–22]. in each step. In this section, theoretical backgrounds on the methodol
In this paper, a vision-based vibration measurement of the light poles ogies are explained concisely, while detailed explanations of each
on elevated highway bridges using motion magnification and dynamic method are available in the main references.
mode decomposition is proposed. This study is an extension of the
previous study by the authors [23] that employs the phase-based motion
magnification technique. The technique is extended to allows for not 2.1. Phase-based vibration motion magnification (PVMM) method
only single-mode rigid body motion, but also multimode operational
modal analysis of flexible structures. In this study, micro-vibrations of Video measurement of a vibrating object consists of temporally dis
the light poles were recorded by video camera and amplified by PVMM placed frames with image intensity, I(x +δ(x, t)) where x is the pixel
coordinate and δ(x, t) is the spatially and temporally varying local mo
Fig. 1. Flowchart of the proposed operational modal analysis of light pole using noncontact video-based measurement.
2
D.M. Siringoringo et al. Engineering Structures 244 (2021) 112728
tion. The magnitude and the local phase determine the characteristics of A two-dimensional Gaussian smoothing kernel with standard devi
captured video motion, and they can be obtained from two-dimensional ation (σ ) is utilized. The kernel gives a rotationally symmetric Gaussian
Fast Fourier Transform (2D-FFT) by multi-scale and localized filters. The lowpass filter. Finally, one can obtain the small motions without noise
phase information gives a better estimate of the motion than the by combining all phases of the magnified frames to form the final video.
magnitude (amplitude) as it is unaffected by illumination effects and Note that the magnified motion of vibrating object contains the same
surface conditions. The advantages of using phase information to esti dynamics characteristics as the original motion. Therefore, modal pa
mate of the motion is utilized in the local phase-based motion magni rameters of the vibrating object can be extracted from the magnified
fication technique. The PVMM algorithm decomposes the signal in the motion once the displacement responses of the object are obtained.
video into local spatial amplitude and phase using a complex-valued
steerable pyramid filter bank. The local spatial phase signals are
decomposed temporally using the Fourier transform into a series of si 2.2. Extraction of spatial displacements by discretized centroid searching
nusoids representing harmonic motion. Then, the phase signals are method (DCSM)
band-pass filtered, amplified, and recombined back to form a video of
magnified motion. The phase-magnified video provides a four-dimensional (4D) array
In the following section, derivation of the method based on the work as, a-by-b-by-3-by-k and it is divided into k frames of still image. The real
by Wadhwa et al. [11] is briefly described. Readers are advised to refer color images are converted to grayscale by eliminating the hue and
to the original work for the complete derivation. As mentioned previ saturation information while retaining the luminance. The idea of dis
ously, each frame (subset) of a video (set) is spatially multi-scale and cretized centroid method is to obtain the spatial displacement time
temporally local, δ(x,t). To implement PVMM, one needs to first extract history of a vibrating object by dividing the object into numerous dis
the local phase and amplitudes by employing the complex steerable cretized cropped regions. Fig. 2 illustrates schematically the process of
pyramid filters. The filter is a complex Gabor type filter with sinusoids tracing displacement of vibrating object by DCSM. At the first step, the
windowed by the Gaussian envelope [25]. Using Fourier series decom gray-converted still image sampled at time t is divided into several of
position, one can obtain a summation of single frequency (ω) complex grids (xy) containing the vibrating object and the background. In each
sinusoids that represents the displaced frames as: grid, centroid of the discretized object Cxy(t) is computed by employing
the Otsu Threshold Segmentation method (OTSM) [12]. Centroids of the
∑
∞ ∑
∞
I(x + δ(x, t) ) = Bω (x, t) = Pω ei2πω(x+δ(x,t)) (1) discretized cropped regions associated with the vibrating object are
ω=− ∞ ω=− ∞ retained for the consecutive frame t, t + 1, until the last frame. They are
then combined to trace the centroids movement over the length of video.
where Bω (x, t) represents the sinusoid sub-band, and ω(x +δ(x, t)) de The centroid locations represent the movement of the discretized object,
notes the phase that contains motion information. and they can be independently extracted to obtain spatial displacement
The motion can be manipulated by altering the phase as in the response of the object.
Fourier shift theorem. The phase ω(x +δ(x, t)) is temporally filtered to The centroid information of each grid is obtained by Otsu Threshold
isolate motion in a specific temporal frequency range, and then multi Segmentation method (OTSM), whose main idea is described in this
plied by a magnification factor α as I(x +(1 + α)δ(x, t) ). This process will section. Consider the pixels in a grid of a given frame of a still image
magnify the phase of a particular sub-band Bω (x, t) and result in represented in L gray levels [1, 2, ...L]. The number of pixels at level i is
denoted by ni and the total number of pixels by N = n1 , n2 , + ...nL . The
Bω (x, t) = Bω (x, t)eiα2πωδ(x,t) = Pω ei2πω(x+(1+α)δ(x,t)) (2)
gray-level histogram can be normalized and regarded as a probability
where Bω (x, t) is a complex sinusoid that has (1 + α) times magnified
input motion. Consequently, the phase of each coefficient is magnified
for each frame by this factor. The motion magnified video can be
reconstructed by dissolving the pyramid of the complex steerable pyr
amid filters, which is a half octave bandwidth. After computation of the
spatial filters, the phase differences from the reference frame for each
level are computed. A necessary temporal bandpass filtering is applied
prior to applying the amplification factor (α). Introduction of amplifi
cation factor (α) changes the noise characteristics in the image. Let as
sume that a response of a noise-contaminated image for a particular
scale and orientation I +σn n can be written as:
Nω = Bω (x, t) + σn Mω (x, t) (3)
3
D.M. Siringoringo et al. Engineering Structures 244 (2021) 112728
distribution: time-sampled data of the size p and q, where p and q are the user-defined
numbers selected so that matrix X is full rank and has enough amount of
∑
L
pi = ni /N, pi ≥ 0, pi = 1 (6) data.
i=1 ⎡ ⎤
Zωi (0) Zωi (1) ⋯ Zωi (q − 1)
As suggested by Otsu [10], the pixels can be divided into two classes ⎢ Zωi (1) Zωi (2) ⋯ Zωi (q) ⎥
X≜⎢ ⎥ (11)
C0 and C1 (background and objects, or vice versa) by a threshold at level ⎣ ⋮ ⋮ ⋱ ⋮ ⎦
k. In the case of light pole still image, this can be done simply by Zωi (p − 1) Zωi (p) ⋯ Zωi (p + q − 2)
considering sky as the background. The quantity C0 denotes pixels with
Next, a new input matrix Y is constructed using the shifted time
levels [1,2,…,k], and C1 denotes pixels with levels [k + 1, ...L]. A crite
displacement data. Similarly, size of the matrixY is determined by
rion that measures the separability of threshold at level k can be defined
integer p and q as follows.
as the function of probabilities of class occurrence and the class mean
⎡ ⎤
levels as: Zωi (1) Zωi (2) ⋯ Zωi (q)
⎢ Zωi (2) Zωi (3) ⋯ Zωi (q + 1) ⎥
η(k) = σ2B (k)/σ2T (7) Y≜⎢⎣ ⋮
⎥
⎦ (12)
⋮ ⋱ ⋮
Zωi (p) Zωi (p + 1) ⋯ Zωi (p + q)
where
In the dynamic mode decomposition, the procedure for realization of
σ 2B (k) =
[μT ω(k) − μ(k)]2
(8) system matrix starts by taking the singular value decomposition (SVD) of
ω(k)[1 − ω(k)] input matrix X as:
2.3. Extraction of modal parameters from displacement responses The eigenvectors of matrix A that is the dynamic decomposition
mode are given by the columns of matrix Φ (ϕi ). Meanwhile, the ei
In this study, dynamic mode decomposition (DMD) is employed to genvalues λi are expressed in z-domain and they are related to the modal
extract modal parameters from a spatiotemporal displacement of the characteristics using the following transformation:λi = ln(λi )/Δt.
structure. The DMD method was originally used as a low-rank diagnostic Finally, the natural frequency (ωi ) and the damping ratio (ξi ) are ob
tool for decomposing fluid flow data into dominant spatiotemporal tained as:
modes [26] providing a valuable interpretation of coherent structures in √̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅
− Re(λi )
complex systems. The DMD can be considered as the family of subspace ω0i = Re(λi )2 + Im(λi )2 , ξi = (17)
ω0i
identification techniques [24,27] and recent innovations have also made
significant progress in implementing DMD for robust future-state pre
3. Experimental verifications
diction. Since its introduction, DMD was found to be robust and versa
tile. It has been extensively studied and rigorously benchmarked and
To validate the methodologies explained above, experimental veri
compared with other leading techniques in the dimensionality reduction
fication was conducted on a cantilever pole specimen to model the light
and reduced order modeling.
pole. The cantilever pole is made of aluminum with dimensions: height
In this study, the DMD is adopted for modal identification of the
1000 mm, width 40 mm and thickness 2 mm and it was fixed at the
structure using the spatial displacement responses obtained by the
bottom support. Vibration of the cantilever was measured only on the
DCSM. Note that outputs of DCSM are the magnified displacement re
weak axis. The first theoretical modal frequency was calculated using
sponses at specific frequency range ωi . They are expressed as zωi (χ , t) ; √̅̅̅̅̅̅̅̅̅̅̅̅̅̅
formula ω01 = 1.8752 /2π EI/ρL4 ; where the Young’s modulus of
where χ denotes the spatial coordinate of the object χ = 1, 2..., m; and t is
aluminum specimen was E = 69GPa, length of cantilever (L) 1000 mm,
the time. Assuming that the sequence of images can be divided by equal
width of 40 mm, and thickness of 2 mm. The theoretical frequency of the
time interval Δt, the continuous time variable can be represented as a
first mode is calculated as 2.43 Hz.
discrete variable k where k = 0, 1, 2..., (p + q). Therefore, the time-
The specimen was excited by introducing initial deformation on the
history of magnified spatial deformation can be rewritten in a matrix
top of the plate to initiate the free vibration response. Fig. 3 shows the
form as:
setup of experiment verification. The measurements were conducted
Zωi (k) = [ zωi (1, k) zωi (2, k)⋯ zωi (m, k) ]T (10) with the support of ordinary indoor lighting without any external illu
mination enhancement condition. A high-speed stationary video camera
One can construct an input matrix X that consists of consecutive
4
D.M. Siringoringo et al. Engineering Structures 244 (2021) 112728
Fig. 3. Experimental verification of the proposed operational modal analysis of light pole by high-speed video camera system. (a). Image frame of cantilever beam
specimen captured by video camera, (b). Experimental setup of cantilever beam using video camera and Laser Doppler Vibrometer (LDV) as the reference.
5
D.M. Siringoringo et al. Engineering Structures 244 (2021) 112728
Fig. 5. Example of binary image spatial discretization process of the magnified temporal frame into several grids along the vertical line for implementation of DCSM.
Fig. 6. Comparison of cantilever beam displacement responses obtained by LDV and by video camera after implementation of PVMM.
at this experiment only the first cantilever mode was excited. were sampled at 30 Hz and utilized as the input to matrix X as explained
by Eq. (11). The size of input matrix was selected by setting p = 500 and
q = 200, which is equivalent to 21 s of free-vibration displacement data.
3.2. Implementation of DMD system identification The same size of shifted input matrix Y was constructed using the shifted
time displacement data as noted by Eq. (12). The system identification
The free vibration displacement responses derived from video mea process was continued by taking the singular value decomposition of
surement by PVMM and DCSM procedures responses were utilized as
input matrix X and estimating the state matrix A. This involved a
input of the DMD system identification. Fig. 7 shows the displacements
truncation of singular values in the diagonal matrix of singular values Σ
time histories for grid no.1, 3, 5 and 7 together with the corresponding
to the size of n = 2that represents the double of the number of selected
Fourier spectra. The figure clearly demonstrates that the displacement
system order. Afterwards, the eigenvalues of matrix A were estimated,
responses have similar decaying pattern, and they are all characterized
from which the natural frequency (ωi ) and damping ratio (ξi ) were
by the resonant frequency of the first mode at 2.42 Hz. The responses
6
D.M. Siringoringo et al. Engineering Structures 244 (2021) 112728
Fig. 7. Examples of cantilever displacement responses obtained by video camera after implementation of PVMM and DCSM. (a) Time-histories of displacements, and
(b) Fourier spectra of displacements.
obtained. The eigenvectors were estimated by Eq. (16) to obtain the bandwidth used for this process is the half-octave and eight orientations.
corresponding mode shapes. The heavy computation requirement sometimes limits the attainable
The resulted modal parameters identified by DMD method are shown length of displacement response extracted from video measurement.
by Fig. 8. The DMD method successfully identified the first mode at There could be reduction in computational time if alternative or
2.416 Hz and damping ratio 0.415%. The values are very close the re improvement of the phase magnification part is developed especially
sults obtained by LDV which is 2.42 Hz for natural frequency and 0.40% regarding the number of orientations. However, further study is needed
for damping ratio. The mode-shape clearly shows the typical shape of to investigate this effect and to find the optimum balance between
first mode and it is shown that the shape is similar to the operational effective computational time and high accuracy of the results.
deflection shape extracted directly from the camera measurement after
the PVMM using edge detection based on Laplacian of Gaussian method. 4. Implementation on full-scale measurement of light poles on
The experimental verification demonstrates that modal parameters highway viaduct
identified by the proposed methods agree well with reference values
measured by conventional vibration sensor laser Doppler vibrometer. It Full-scale implementation of the proposed measurement methods for
should be mentioned that depending on the pixel size of the frame, the operational modal analysis of light poles on highway viaduct is
PVMM and DCSM procedure require quite extensive computer compu described in this section. The target viaduct is located on the Kinko
tation. For example, the calculation time for one minute video with pixel Junction, Yokohama city, Kanagawa, Japan. It is a double deck curve
resolution 2160 × 1080 and frame rate 30fps for one grid in the DCSM viaduct in a metropolitan expressway network. On the top of bridge
process takes about four hours. The computation was performed using a deck, many light poles of type STB9.1 are mounted. These light-poles
computer with the processor- Intel® Core™ i7-8650U, CPU with pro have the following dimensions: height 9.1 m, diameter 139.8 mm at
cessor base frequency 1.90 GHz and configurable TDP-up frequency the bottom and tapered to 89.1 mm in the middle to the top with a width
2.11 GHz, and 16 GB RAM of memory. To obtain finer resolution of of 60.5 mm. The poles are fixed on bottom by base steel plate with the
modal displacement, more grids are needed, and this consequently will size 35 cm × 40 cm and anchored to the highway curb using M30 high
increase the computational time. The part of algorithm that requires the tension bolts. The poles are made of carbon steel pipe (STK 400 with
most computational time is the phase magnification part because the modulus of elasticity of 206,000 N/mm2) and weighs about 130 kg each.
7
D.M. Siringoringo et al. Engineering Structures 244 (2021) 112728
Fig. 8. (a) Modal parameters of the cantilever identified by DMD method using displacement responses obtained by video camera after applying the PVMM and
DCSM. (b) Operational deflection shape extracted from the PVMM using Laplacian of Gaussian edge detection method (LOG).
Some light poles have signboards weighing 15.2–26 kg attached on Two light poles, P1 and P2 were investigated in this measurement, and
them. These attachments would have effect on the light poles’ dynamic their results are described in detailed in the following section. Fig. 9(b)
characteristics. and (c) show the still image captured by video camera. The figure clearly
Vibration measurement was conducted in an ambient condition, show that the video motion captured on the light pole P1 is the in-plane
where the source of vibration was mainly traffic-induced vibration. motion while that of the light pole P2 is the out-of-plane motion. The
Measurement was performed simultaneously using high-speed video LDV was placed in a location so that the direction of laser beam is
camera system SONY NXCAM and Laser Doppler Vibrometer (Polytec’s perpendicular to the direction of the motion captured by video camera.
RSV-150), where the later device was utilized for validation. It should be Finite element analysis conducted on the typical light poles reveals that
mentioned that the LDV is a single point laser type that provides vi the respective natural frequencies of the pole in the in-plane and out-of-
bration response only from one target point on the pole. Therefore, only plane directions are very close [1]. This is thought to be caused by the
natural frequencies and damping ratios of the light pole were obtained light pole’s geometry and material that are uniform along the height in
by vibration measurement using LDV. The measurement using video both directions except for a small mass attached near the top to model
camera, on the other hand, provided estimations of natural frequencies, the lamp.
damping ratios, and mode-shapes. Ambient vibrations of the light poles
were captured by high-speed video camera system SONY NXCAM with 4.1. Results of measurement by LDV
pixel resolution 3840 × 2160 at frame rate of 30fps. Both devices were
placed on the ground at distance about 100 m away from the target light Fig. 10 shows the examples of 200 s ambient vibration responses of
poles as shown in Fig. 9(a) and the measurement were conducted light pole P1 and P2 obtained by LDV. Note that the light poles vibra
simultaneously in the same condition and measurement settings, tions are ambient vibration with random characteristics of amplitude.
namely, the outdoor setting without any environmental corrections such The displacement amplitudes were in the range of ±2 mm and ±0.5 mm
as lighting and disturbance of wind. for P1 and P2, respectively. The spectra show that the light poles vi
Note that as investigated previously [1], there are two main di brations are largely dominated by the first mode, and they were iden
rections of light pole’s vibration. Direction that coincides with the tified as 1.613 Hz for P1 and 1.563 Hz for P2. Random Decrement
lamp’s hook is considered as the main or strong axis, and the pole vi technique [28,29] was applied to obtain the free vibration responses and
bration in this direction is called the in-plane vibration. Meanwhile di damping ratio was estimated by logarithmic decrement method. Fig. 10
rection perpendicular to the strong axis is considered as the secondary or (a) and (c) show the free-vibration responses obtained after isolating the
weak axis. Vibration in this direction is called the out-of-plane vibration. first mode. Based on the free-vibration responses, damping ratios were
8
D.M. Siringoringo et al. Engineering Structures 244 (2021) 112728
Fig. 9. Implementation on full-scale measurement of the light poles on highway viaduct. (a) Measurement setup using a high-speed video camera system and Laser
Doppler Vibrometer (LDV) as a reference (b) Light pole P1 captured by video camera, (c) Light pole P2 captured by video camera.
Fig. 10. Ambient vibration measurement using Laser Doppler Vibrometer (LDV). Light Pole P1: (a) Time-histories of displacements and free-vibration response
extracted by random decrement, (b) Fourier spectra of displacement and the free-vibration response. Light Pole P2: (c) Time-histories of displacements and free-
vibration response extracted by random decrement, (d) Fourier spectra of displacement and the free-vibration response.
estimated to be 0.44% and 0.56% for P1 and P2, respectively. The the mode shapes information cannot be obtained from laser
spectra of ambient vibration responses (Fig. 10(b) and (d)) reveal two measurement.
other peaks corresponding to the second and the third modes. Similarly,
the Random Decrement technique was applied to the second and third 4.2. Results of measurement by video camera and implementation of
mode to obtain natural frequency and damping ratio. Table 1 lists the PVMM
complete information on modal parameters obtained from LDV mea
surement. Note that since the LDV is a single point measurement system, The video recording was carried out at a frame rate 30fps with pixel
9
D.M. Siringoringo et al. Engineering Structures 244 (2021) 112728
Table 1 Technique (NExT) [30]. In this work, since the objective is to obtain the
Natural frequencies and damping ratios of the light-poles identified from LDV multimode free-vibration responses, the NExT is utilized. In this method,
measurement. the cross-correlation functions between the spatial displacements and a
Pole Natural frequencies (Hz) Damping ratio (%) selected reference displacement is computed to estimate the free-
No.
1st 2nd 3rd 1st 2nd 3rd
vibration responses as follows:
mode mode mode mode mode mode [ ]
Rkir (T) = E yki (t + T) × ykr (t) (19)
P1 1.613 2.197 6.934 0.439 1.821 0.240
P2 1.563 3.662 6.494 0.560 0.889 0.786 where yi denotes the spatial displacement i and yr is the reference
displacement. The reference displacement point is selected as the one
resolution of 2160 × 1080. The original videos of the light pole P1 and that has the maximum modal displacement for each mode. The resulted
P2 were downsampled for further analysis to 1268 × 1268 × 3600-pixel cross-correlation responses Rkir (T) are used as inputs of X and Y matrices
resolution, where 3600 represents the number of frames. The first set of defined in Eqs. (11) and (12).
post-processing is phase amplification. This process was conducted by Examples of the free-vibration responses extracted from random
amplifying the phase for a frequency range corresponding to the range of ambient vibration by the NExT at sampling frequency 30 Hz are shown
the first modal frequency. Amplification factor α = 30 as defined in Eq. in Fig. 13 for grid number 1 to 4 of the light pole P1. The figure clearly
(2) was selected. The magnified frames were kept separated for appli demonstrates that the displacement responses have similar decaying
cation of displacement extraction technique. The true-color (RGB) pattern, and they are all characterized by the resonant frequency of the
image of the magnified image was converted to the grayscale image and first mode at 1.615 Hz. The similar procedure was implemented for the
then to the binary image. In the latter form, the background was sepa light pole P2, and the extracted free-vibration responses at sampling
rated from the target structure image as shown in Figs. 11 and 12 for frequency 30 Hz are shown by Fig. 14 for grid number 1 to 4. The spectra
light pole P1 and P2, respectively. Next, the still image was divided into figure clearly demonstrate identical frequency of all cross-spectra which
several grids along the pole’s height to implement DCSM. The centroids correspond to the first mode at the 1.57 Hz. Note that for both light pole
of the grids were computed in pixel resolution and tracked throughout P1 and P2, the spectra are characterized by sharp peaks and narrow
the video duration. In this case, each still image was divided into nine band under the spectra indicating low damping characteristics of the
grids along the pole’s vertical direction. Afterwards, the pixel coordinate light pole.
was converted to displacement time histories that represent the spatial The cross-correlation responses were then utilized as input for DMD
displacement of the pole. As mentioned previously, the spatial dis system identification and the input matrix X is constructed as explained
placements were the in-plane displacement for light pole P1 and the out- by Eq. (11) using the user-defined values of p = 500 and q = 200. This is
of-plane displacement for light pole P2. Examples of displacement time- equivalent to constructing an input matrix of 21 s of free-vibration
histories of the centroid in the middle grid of pole P1 and P2 are shown displacement data. The same size of shifted input matrix Y was later
by Fig. 11 (d) and Fig. 12(d), respectively. It is evident from the figures constructed using the shifted time displacement data as noted by Eqs.
that displacement responses are very small with the amplitude in the (11) and (12). The system identification procedure was continued by
range of ±2 mm for P1 and ±0.4 mm for P2, which is consistent with the taking the singular value decomposition of input matrix X and esti
amplitude measured by LDV. The procedure was repeated for all fre mating the matrix A. This involved a truncation of singular values in the
quency ranges where the possible light pole natural frequencies can be diagonal matrix Σ to the size of n = 2, that correspond to the double of
found. Three frequency ranges corresponding to the first three natural the number of selected system’s order. Afterwards, the eigenvalues of
frequencies are identified using information of the spectra of displace matrix A were estimated, from which the natural frequency (ωi ) and
ment response for the frequency range 0–10 Hz. This was also supported damping ratio (ξi ) were obtained. The eigenvectors were estimated by
by frequency spectra obtained from laser measurement for the same Eq. (16) to obtain the corresponding mode shapes. The DMD method
frequency range. successfully identified the first three modes of both light poles as shown
Note that the amplitudes of light pole spatial displacement estimated on Fig. 15.
within its natural frequency range are random. One needs to extract the Table 2 lists the three identified modes including the natural fre
free-vibration response from such a random response to obtain modal quencies, damping ratios and modal assurance criteria (MAC) [31]
parameters using DMD method. This can be done by employing the values. Note that the MAC values are calculated by comparing the
random decrement technique (RDT) [28,29] or the Natural Excitation identified mode-shapes with theoretical modes of cantilever. The results
Fig. 11. Temporal frame of magnified motion of Light-pole P1: (a) 100th RGB frame of magnified motion; (b) converted grayscale image; (c) converted binary image
(d) displacement response of the pole’s middle grid extracted from video measurement after amplification with factor α = 30. (Note: the corresponding video are
available as Video 2. Magnified vibration of the 1st Mode and Video 3. Magnified vibration of the 2nd mode).
10
D.M. Siringoringo et al. Engineering Structures 244 (2021) 112728
Fig. 12. Temporal frame of magnified motion of Light pole P2. (a) The 100th RGB frame of magnified motion; (b) converted grayscale image; (c) converted binary
image, and (d) displacement response of the light pole’s middle grid extracted from video measurement after amplification with factor α = 30. (Note: The corre
sponding video are available on Video 4. Magnified vibration of the 1st Mode, and Video 5. Magnified vibration of the 2nd mode).
Fig. 13. Results of NExT implementation to random vibration for mode 1 of the light pole P1: (a). cross-correlation functions of displacements, and (b) cross-spectra
of displacements for grid number 1 until 4.
show that identified natural frequencies are very close to the corre The relatively poor results of the third mode’s MAC value, especially
sponding natural frequencies identified by the LDV measurement. that of light-pole P2 is mainly caused by the condition of light pole and
Whereas the identified damping ratios are slightly different from the limitation of DCSM implementation. Sign boards are attached to both
corresponding values identified by single point laser measurement, but light poles at the location near to the inflection point of the third mode,
still within the reasonable range. The MAC values indicate reasonable especially for the light pole P2. When implementing the DCSM to the
mode-shapes comparison with the values larger than 75%. It is noted binary figure, additional shape was created by the sign board, and this
that the first mode-shape, which is the dominating vibration mode has incorrectly changed the centroid positions of the corresponding grids
the highest MAC value because the mode can be easily identified from out of the pole’s center line. Therefore, these grids are excluded from the
the response. Meanwhile participations of the second and third modes analysis, and the exclusion affected the accuracy of MAC value when the
on the overall response are quite limited. identified mode-shape vector was compared to the analytically derived
11
D.M. Siringoringo et al. Engineering Structures 244 (2021) 112728
Fig. 14. Results of NExT implementation to random vibration for mode 1 of the light pole P2: (a). cross-correlation functions of displacements, and (b) cross-spectra
of displacements for grid number 1 until 4.
one. The results of damping identification from vision and laser mea plane and the camera is not less than 45◦ (positive and negative angle).
surements both show that the light poles have remarkably low damping, The angle less than 45◦ would give unfavorable results since the object
which is consistent with previous study that was based on vibration motion on its two major axis becomes indistinguishable. This can be
testing of the similar light poles [1]. considered as a limitation of the method when using only one video
Note that the LDV measurement was conducted only for verification camera.
purpose. Measurement can also be conducted only by video camera Another important aspect of measurement is weather condition. The
without LDV or other type of contact/noncontact sensor. In such a case, most favorable weather condition for video measurement based on
after recording the object vibration by video camera, the PVMM algo practical application is on a clear day without any clouds and wind gust.
rithm is applied to a wide range of frequency considered to be the most The presence of clouds in the background hinders the process of sepa
possible range of the structure’s dominant frequencies. Prior knowledge rating the target structure from background, and incorrectly results in
from finite element model or analytical model of the structure is helpful multiple centroids which ultimately reduces the accuracy when imple
to determine the more precise range of frequency, but basic knowledge menting the DCSM algorithm. The presence of wind gust would shake
of structural dynamics is normally adequate to predict the possible range the camera and introduces unwanted noise to the actual object vibra
of the structure’s dominant frequencies. The peaks corresponding to the tion. In our experiments on the light poles, vibration of video camera
natural frequencies of the structure would normally appear in the was negligibly small because the measurement was conducted on the
spectra of the motion-magnified vibration within the selected range of ground and the camera was firmly fixed on a tripod. There was no sig
frequency. Using the information of the frequency peaks, further nificant ground vibration from either traffic or wind. However, in a case
filtering and amplification can be conducted to narrow down and obtain where vibration of the camera is substantial and affects quality of the
the more precise range of frequency. The procedure is repeated itera video, an additional countermeasure should be considered. One way to
tively until all the natural frequencies indicated by the frequency peaks solve the problem is by identifying the characteristics of camera vibra
can be identified. tion and removing the frequency content associated with the camera
vibration using digital filtering. For this purpose, characteristics of
camera vibration needs to be identified by recording the camera vibra
4.3. Discussion on the practical aspects of implementation and limitation
tion using small wireless sensor mounted on the tripod.
of the method
The other aspect that needs specially attention during video
recording is the frame rate. The frame rate defines the time interval
There are some practical considerations and important aspects to this
between consecutive image frames captured by video camera. It is
method that require attention especially during video recording. One
normally defined by frame-per-second (fps) unit, and the recommended
important aspect of measurement is the angle and position of the camera
and maximum values are determined by the video camera setting and
with respect to the measured object. The best angle for camera is at the
specification. In this study, the field measurement was conducted using
perpendicular angle to the plane on which the object is vibrating.
frame rate 30fps. After considering the Nyquist principle, this frame rate
However, this is not always possible in practice. Therefore, we need to
should be adequate to capture vibration of an object with the natural
arrange the camera position so that the angle between the vibration’s
12
D.M. Siringoringo et al. Engineering Structures 244 (2021) 112728
Fig. 15. Mode-shapes of the light-poles identified from the proposed vision-based operational modal analysis. Light pole P1: (a) 1st mode, (b) 2nd mode, (c) 3rd
mode. Light pole P2: (d) 1st mode, (e) 2nd mode, (f) 3rd mode.
Table 2
Natural frequencies and damping ratios of the light-poles identified from video measurement using PVMM, DCSM and DMD Methods.
Pole No. Natural frequencies (Hz) Damping ratio (%) MAC
1st mode 2nd mode 3rd mode 1st mode 2nd mode 3rd mode 1st mode 2nd mode 3rd mode
frequency less than 15 Hz accurately. Therefore, one should carefully because the peaks in the spectra that separate the modes become un
consider the appropriate frame rate with respect to the natural fre clear. In such case, the modal selection from frequency spectra using
quency of target object. The dominant natural frequencies of civil classical peak-picking method may not be adequate, and the more robust
structures are normally below 5 Hz. Predominant natural frequencies of identification method such as decomposition of the spectral density
flexible structures such as light pole or stay cable are normally less than function matrix [32] may be necessary. Further research is needed to
3 Hz. Therefore, the current video camera with 30fps sampling rate is deal with the structure with such characteristics using the proposed
adequate to capture vibration of the structure in its predominant vi vision-based method.
bration modes. However, for the stiffer structures with frequency higher
than 15 Hz, the higher sampling rate should be considered. In some 5. Conclusion
cases, even the high-definition high-frequency video camera should be
used when natural frequency of the structure is even higher. Noncontact operational modal analysis of light pole by vision mea
The structure discussed in this paper is a typical structure with low surement is described in this paper. The vision-based modal analysis is
damping and distinct or well-separated modes. These structural char performed by implementing phase-based video motion magnification
acteristics enable modal identification by the proposed method because (PVMM) method to amplify micro-vibration of the structure, the dis
the natural frequencies can be easily located and identified from the cretized centroid searching method (DCSM) to extract spatial displace
spectra. Such characteristics are the preferable condition for imple ment of the pole, and the dynamic mode decomposition (DMD) for
mentation of the proposed method. Implementation on a structure with identification of modal parameters. The proposed method was experi
closely spaced modes with high damping can be more challenging mentally tested in a laboratory and implemented in the field
13
D.M. Siringoringo et al. Engineering Structures 244 (2021) 112728
measurements on highway viaducts. Experimental verifications were [4] Siringoringo DM, Fujino Y. Experimental study of laser Doppler vibrometer and
ambient vibration for vibration-based damage detection. Eng Struct 2006;28(13):
conducted using cantilever beam and the results confirm that the tech
1803–15.
niques can effectively identify modal parameters of the structures [5] Sels S, Vanlanduit S, Bogaerts B, Penne R. Three-dimensional full-field vibration
accurately event under small vibration amplitude. Comparison of the measurements using a handheld single-point laser Doppler vibrometer. Mech Syst
results with measurement using laser doppler vibrometer have demon Sig Process 2019;126:427–38.
[6] Gentile C, Bernardini G. An interferometric radar for noncontact measurement of
strated that displacement can be extracted from vision measurement deflections on civil engineering structures: laboratory and full-scale tests. Struct
with good accuracy. Infrastruct Eng 2010;6(5):521–34.
The full-scale implementation of the method was conducted to [7] Negulescu C, Luzi G, Crosetto M, Raucoules D, Roullé A, Monfort D, et al.
Comparison of seismometer and radar measurements for the modal identification
identify modal parameters of the light poles on highway viaduct. The of civil engineering structures. Eng Struct 2013;51:10–22.
results demonstrate that modal parameters identified from video mea [8] Sun Z, Siringoringo DM, Fujino Y. Load-carrying capacity evaluation of girder
surement are in good agreement with the results from vibration mea bridge using moving vehicle. Eng Struct 2021;229:111645.
[9] Xu Y, Brownjohn JM. Review of machine-vision based methodologies for
surement using laser Doppler vibrometer. In addition to identification of displacement measurement in civil structures. J Civil Struct Health Monit 2018;8
frequencies and damping ratios, the DMD method allows for identifi (1):91–110.
cation of mode-shapes from spatial displacements. The practical [10] Dong CZ, Catbas FN. A review of computer vision–based structural health
monitoring at local and global levels. Struct Health Monit 2020.
importance of the proposed method is that it allows for modal identifi p.1475921720935585.
cation of an object in a distant noncontact manner using a simple video [11] Wadhwa N, Rubinstein M, Durand F, Freeman WT. Phase-based video motion
measurement under small amplitude of ambient excitation. The mea processing. ACM Trans Graphics 2013;32(4):1–10.
[12] Otsu N. A threshold selection method from gray-level histograms. IEEE Trans Sys,
surement system is less expensive and relatively easier to operate
Man, Cyber 1979;9(1):62–6.
compared to other noncontact measurement systems, which is advan [13] Bhowmick S, Nagarajaiah S. Identification of full-field dynamic modes using
tageous for massive and frequent measurements of light poles on the continuous displacement response estimated from vibrating edge video. J Sound
highway viaducts. Full-scale implementations have shown the prom Vib 2020;115657.
[14] Chen JG, Wadhwa N, Cha YJ, Durand F, Freeman WT, Buyukozturk O. Modal
ising results, and application of the method can be extended to other identification of simple structures with high-speed video using motion
structures for automatic noncontact assessment of spatially flexible magnification. J Sound Vib 2015;345:58–71.
structures. [15] Chen JG, Davis A, Wadhwa N, Durand F, Freeman WT, Büyüköztürk O. Video
camera–based vibration measurement for civil infrastructure applications. J Infra
Sys 2017;23(3):B4016013.
CRediT authorship contribution statement [16] Chen JG, Adams TM, Sun H, Bell ES, Büyüköztürk O. Camera-based vibration
measurement of the world war I memorial bridge in Portsmouth, New Hampshire.
J Struct Eng 2018;144(11):04018207.
Dionysius M. Siringoringo: Conceptualization, Investigation, [17] Ye XW, Dong CZ, Liu T. Force monitoring of steel cables using vision-based sensing
Methodology, Formal analysis, Writing - original draft. Samten technology: methodology and experimental verification. Smart Struct Syst, Int J
Wangchuk: Investigation, Formal analysis, Software, Writing - original 2016;18(3):585–99.
[18] Poozesh P, Sarrafi A, Mao Z, Avitabile P, Niezrecki C. Feasibility of extracting
draft. Yozo Fujino: Writing - review & editing, Supervision. operating shapes using phase-based motion magnification technique and stereo-
photogrammetry. J Sound Vib 2017;407:350–66.
Declaration of Competing Interest [19] Yang Y, Dorn C, Mancini T, Talken Z, Kenyon G, Farrar C, et al. Blind identification
of full-field vibration modes from video measurements with phase-based video
motion magnification. Mech Sys Signal Proc 2017;85:567–90.
The authors declare that they have no known competing financial [20] Sarrafi A, Mao Z, Niezrecki C, Poozesh P. Vibration-based damage detection in
interests or personal relationships that could have appeared to influence wind turbine blades using Phase-based Motion Estimation and motion
the work reported in this paper. magnification. J Sound Vib 2018;421:300–18.
[21] Molina-Viedma AJ, Felipe-Sesé L, López-Alba E, Díaz F. High frequency mode
shapes characterization using Digital Image Correlation and phase-based motion
Acknowledgements magnification. Mech Sys Signal Proc 2018;102:245–61.
[22] Yang Y, Dorn C, Farrar C, Mascareñas D. Blind, simultaneous identification of full-
field vibration modes and large rigid-body motion of output-only structures from
This research is supported by Kajima Foundation through a research digital video measurements. Eng Struct 2020;207:110183.
grant to the first author (PI: Dionysius M. Siringoringo). The support is [23] Thiyagarajan JS, Siringoringo DM, Wangchuk S, Fujino Y. Implementation of video
greatly acknowledged. Support of measurement device from Bridge and motion magnification technique for noncontact operational modal analysis of light
poles. Smart Struct Syst 2021;27(2):227–39.
Structure Laboratory, The University of Tokyo is also greatly acknowl [24] Tu JH, Rowley CW, Luchtenburg DM, Brunton SL, Kutz JN. On dynamic mode
edged. We also gratefully acknowledge the technical support and in decomposition: theory and applications. J Comput Dyn 2014;1(2):391–421.
formation of light poles material provided by Metropolitan Expressway [25] Portilla J, Simoncelli EP. A parametric texture model based on joint statistics of
complex wavelet coefficients. Int J Compt Vision 2000;40(1):49–70.
Co. Ltd. Findings, opinions and conclusions presented in this paper are of [26] Schmid PJ. Dynamic mode decomposition of numerical and experimental data.
the authors and not of the abovementioned institutions. J Fluid Mech 2010;656:5–28.
[27] Kutz JN, Brunton SL, Brunton BW, Proctor JL. Dynamic mode decomposition: data-
driven modeling of complex systems. Soc Ind Appl Math 2016.
Appendix A. Supplementary material
[28] Ibrahim SR. Random decrement technique for modal identification of structures.
J Spacecraft Rockets 1977;14(11):696–700.
Supplementary data to this article can be found online at https://doi. [29] Vandiver JK, Dunwoody AB, Campbell RB, Cook MF. A mathematical basis for the
org/10.1016/j.engstruct.2021.112728. random decrement vibration signature analysis technique. J Mech Des 1982;104
(2):307–13.
[30] James GH, Carne TG, Lauffer JP. The natural excitation technique (NExT) for
References modal parameter extraction from operating structures. Modal Anal-Int J Anal Exp
Modal Anal 1995;10(4):260.
[1] Siringoringo DM, Fujino Y, Nagasaki A, Matsubara T. Seismic performance [31] Allemang RJ, Brown DL. A correlation coefficient for modal vector analysis 1982;
evaluation of existing light poles on elevated highway bridges. Struct Infra Eng Vol. 1:110–6.
2021;17(5):649–63. [32] Brincker R, Zhang L, Andersen P. Modal identification from ambient responses
[2] Stanbridge AB, Ewins DJ. Modal testing using a scanning laser Doppler vibrometer. using frequency domain decomposition. In: Proc. of the 18th International Modal
Mech Syst Sig Process 1999;13(2):255–70. Analysis Conference (IMAC), San Antonio, Texas; 2000.
[3] Siringoringo DM, Fujino Y. Noncontact operational modal analysis of structural
members by laser doppler vibrometer. Comput-Aided Civ Infrastruct Eng 2009;24
(4):249–65.
14