applied
sciences
Article
Prediction of Subsidence during TBM Operation in Mixed-Face
Ground Conditions from Realtime Monitoring Data
Hyun-Koo Lee 1 , Myung-Kyu Song 2 and Sean Seungwon Lee 1, *






Citation: Lee, H.-K.; Song, M.-K.;
Lee, S.S. Prediction of Subsidence
during TBM Operation in Mixed-Face
Ground Conditions from Realtime
Monitoring Data. Appl. Sci. 2021, 11,
12130. https://doi.org/10.3390/
app112412130
Academic Editor: Chin Leo
Received: 15 November 2021
Accepted: 12 December 2021
Published: 20 December 2021
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2021 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Appl. Sci. 2021, 11, 12130. https://doi.org/10.3390/app112412130
1 Department of Earth Resources and Environmental Engineering, Hanyang University, 222 Wangsimni-ro,
Seongdong-gu, Seoul 04763, Korea; lhkooo@hanyang.ac.kr
2 R&D Department, Hyundai E&C, 75 Yulgok-ro, Jongno-gu, Seoul 03058, Korea; mk.song@hdec.co.kr
* Correspondence: seanlee@hanyang.ac.kr
Abstract: The prediction of settlement during tunneling presents multiple challenges, as such
settlement is governed by not only the local geology but also construction methods and practices,
such as tunnel boring machine (TBM). To avoid undesirable settlement, engineers must predict the
settlement under given conditions. The widely used methods are analytical solutions, empirical
solutions, and numerical solutions. Analytical or empirical solutions, however, have limitations,
which cannot incorporate the major causes of subsidence, such as unexpected geological conditions
and TBM operational issues, among which cutterhead pressure and thrust force-related factors are
the most influential. In settlement prediction, to utilize the machine data of TBM, two phases of long
short-term memory (LSTM) models are devised. The first LSTM model is designed to capture the
features affecting surface settlement. The second model is for the prediction of subsidence against
the extracted features. One thing to note is that predicted subsidence is the evolution of settlement
along TBM drive rather than its maximum value. The proposed deep-learning models are capable of
predicting the subsidence of training and test sets with excellent accuracy, anticipating that it could
be an effective tool for real-world tunneling and other underground construction projects.
Keywords: tunnel boring machine (TBM) operation; TBM induced ground settlement; deep learning;
long short-term memory (LSTM); machine data
1. Introduction
Underground construction using Tunnel Boring Machines (TBMs) is rapidly increasing
to meet demands for new roads, railways, and electrical and telecommunication infras-
tructure associated with rapid urbanization. The TBM method offers several advantages,
such as closed-mode operations, over other traditional approaches in terms of the safety
measures involved in applying face-support pressures and the instant support provided by
concrete linings, which mitigate the risks posed by the high groundwater pressure under
water table.
Mixed-face ground conditions during TBM driving pose the most challenging risks.
The soft soil at the top of the face and the hard rock at the bottom makes it difficult
to maintain a proper face-support pressure and face stability, and increases the risk of
excessive cutter wear, face collapse, sinkholes, or damage to surrounding structures [1]. To
avoid undesirable settlement and provide appropriate safety measures, engineers must
reliably predict the amount of settlement under given ground conditions. The most widely
accepted analytic solution, proposed by Peck [2], is based on measurements from various
projects and has been modified to apply to TBM excavations in geologically mixed-face
conditions for metropolitan projects in the congested urban area [3].
However, an analytic solution has its own limitations, as it may not be able to incor-
porate the important causes of subsidence, such as unexpected geological conditions and
subsequent ground deformation, over-excavation, untreated tail voids, curvature with
a short radius, and TBM operational issues, including chamber pressure, penetration or
https://www.mdpi.com/journal/applsci
 Appl. Sci. 2021, 11, x FOR PEER REVIEW 2 of 20

Appl. Sci. 2021, 11, 12130 2 of 20

short radius, and TBM operational issues, including chamber pressure, penetration or ad-
vance rate,
advance rate,muck
muckvolume,
volume,torque,
torque,andandthrust
thrust force.
force. Cutterhead face pressure
Cutterhead face pressure andandthrust
thrust
force related factors are most influential among these
force related factors are most influential among these issues [4]. issues [4].
Theproject,
The project,ininthis
thisstudy,
study,involves
involvesaamixed-face
mixed-facecondition
conditionof ofalluvial
alluvialand andweathered
weathered
soil, weathered rock, and soft rock from the top to the bottom of the
soil, weathered rock, and soft rock from the top to the bottom of the tunnel face throughouttunnel face through-
out the length of a tunnel. A previous study found that machine
the length of a tunnel. A previous study found that machine learning with geological learning with geological
informationwithout
information withoutTBM TBMdatadatagenerated
generatedpoorpoorsettlement
settlementestimates
estimates[5].[5].We
Wehypothesized
hypothesized
that the correlation of key parameters, such as thrust force and
that the correlation of key parameters, such as thrust force and advance rate, withadvance rate, withrespect
respect
to the
to the amount
amount of of settlement
settlement would
wouldlead leadtotoananimproved
improved theoretical
theoretical framework,
framework, as as
wewe an-
alyzed more TBM
analyzed more TBM data. data.
In this
In this paper,
paper, aa series
series ofof LSTM
LSTMmodels
modelsare areadopted,
adopted,wherewhere thethefirst LSTM
first LSTM model
modelex-
tracted hidden features that could affect ground movement/subsidence,
extracted hidden features that could affect ground movement/subsidence, and the second and the second
LSTMmodel
LSTM modelwas wastrained
trainedforforthe
theprediction
predictionof ofthe
thesubsidence
subsidenceagainst
againstthe thehidden
hiddenfeatures
features
obtainedfrom
obtained fromthetheearlier
earliermodel.
model.10,660,000
10,660,000TBM TBMmeasurements
measurementsand andsettlement
settlementdata datafrom
from
48settlement
48 settlementmarkers
markers were
were collected
collected andand fed ainto
fed into longa short-term
long short-term
memory memory
(LSTM) (LSTM)
model.
model.
After After several
several weeks of weeks of training
training and testing,
and testing, the LSTM
the LSTM models
models demonstrated
demonstrated excel-
excellent
lent performance
performance of subsidence
of subsidence prediction
prediction for for
bothboth
thethe training
training andand testcases,
test cases,indicating
indicating
significantadvances
significant advancesin inthe
thereliable
reliableprediction
predictionofofsubsidence
subsidenceagainst
againstpurepuremachine
machinedata.data.

2.2.Literature
LiteratureReview
Review
Analytical,
Analytical,empirical,
empirical,andandnumerical
numericalmethods
methodshave
havebeen
beenwidely
widelyused
usedto toestimate
estimatethethe
amount of settlement caused by tunnel excavations. Peck [2] proposed a very
amount of settlement caused by tunnel excavations. Peck [2] proposed a very well-known well-known
empirical
empirical solution
solution using
using anan inverted
inverted Gaussian
Gaussian distribution
distribution curve,
curve, as
as shown
shown in in Figure
Figure 1.
1.
Several
Severalvariations
variationsof ofthe
theempirical
empiricalsolution
solutionhave
havebeen
beenapplied
applied[6,7].
[6,7].Although
Althoughempirical
empirical
solutions
solutions are
are easy
easy to
to apply,
apply,they
theycan
canonly
onlyaccommodate
accommodateaahandful
handfulof offactors
factorsoutoutofofthe
the
various factors shown in Figure 2, and often exclude various temporally and
various factors shown in Figure 2, and often exclude various temporally and spatially rel- spatially
relevant factors,
evant factors, such
such asas localized
localized geological
geological structuresand
structures andananadvance
advanceraterate[8,9].
[8,9].

Figure1.1.A
Figure Atraverse
traversesettlement
settlementtrough
trough(Peck,
(Peck,1969).
1969).

Analytical solutions based on soil and rock mechanics, which are also easy to apply
and enjoy a comparable academic foundation, also suffer from similar limitations. Nu-
merical methods offer advantages over other empirical or analytical methods to build
models that accommodate the constitutive equations, geological layers, localized geolog-
ical structures, and construction sequences. However, they require the elaboration of
time-consuming models in addition to the uncertainty of geotechnical properties [10]. To
tackle these issues and to resolve the non-linear relationship between ground reactions
and subsidence, the various AI models and machine-learning (ML) approaches have been
applied using data from the sensors of a TBM [5,11–13] as summarized in Table 1.
Appl. Sci. 2021, 11, 12130 3 of 20
Appl. Sci. 2021, 11, x FOR PEER REVIEW 3 of 20

Figure 2. Five phases of settlement according to TBM construction.

Figure 2. Five phases of settlement according to TBM construction.

Analytical solutions based on soil and rock mechanics, which are also easy to apply
Table 1. AI models for predicting surface settlement induced by tunneling.
and enjoy a comparable academic foundation, also suffer from similar limitations. Nu-
mericalAImethods
Model offer advantages over other empirical or analytical
Researcher Theme methods to build
Year
models that accommodate the constitutive equations, geological layers,
Ground surface
localized geolog-
ical structures,
PCA/ANFISand construction sequences.
Bouayad, D; Emeriault,However,
F. [12] they require
settlements the elaboration
induced 2017 of
time-consuming models in addition to the uncertainty of geotechnical properties [10]. To
by shield tunneling
tackle these issues and to resolve the non-linear relationship between ground reactions
Predicting shield
Evolutionary
and Hybrid
subsidence, Zhang,
the various AIK.; Lyu, H.
models M.;machine-learning
and Shen, S. L.; (ML) approaches have
tunneling induced 2020been
Neural Network Zhou, A.; Yin, Z. Y. [5]
applied using data from the sensors of a TBM [5,11–13] as ground
summarized in Table 1.
settlement
Predicting shield
Table 1. AI Learning Chen, R.;surface
models for predicting Zhang,settlement
P.; Wu, H.;induced
Wang, by tunneling.
Machine tunneling induced 2019
Z.; Zang, Z. [13]
ground settlement
AI Model Researcher Theme Year
Prediction
Ground surfaceofsettlements
ground in-
PCA/ANFIS Kim, C.; Bae,
Bouayad, G.; Hong, C.;
D; Emeriault, F. Park,
[12] S.; 2017
surface
duced settlement
by shield tunneling 2001
Shin, H. [14]
ANN (Artificial due to tunneling
Evolutionary Hybrid Zhang, K.; Lyu, H. M.; Shen, S. Predicting shield tunneling induced
Neural Network) 2020
Neural Network L.; Zhou, A.; Yin, Z. Y. [5] Prediction the max.
ground settlement
Suwansawat,
Chen, R.; Zhang, S.;Einstein, H. [15]
P.; Wu, H.; surface
Predicting settlement
shield 2006
tunneling induced
Machine Learning caused by EPB Shield 2019
Wang, Z.; Zang, Z. [13] ground settlement
Hasanipanah,
Kim, C.; Bae, G.;M.; Nooria-Bidgoli,
Hong, Predicting
C.; Park, Prediction of groundsurface
surface settle-
PSO-ANN
ANN (Artificial M.;S.;
Jahed
Shin,Armaghani,
H. [14] D.; settlement
ment due to caused by
tunneling 20162001
Neural Network) Khamesi, H. [16] Prediction tunneling
the max. surface settle-
Suwansawat, S.;Einstein, H. [15] 2006
ment causedof
Settlement byshallow
EPB Shield
SVM (Support Vector
Samui, P. Sitharam,
Hasanipanah, T. [17]
M.; Nooria-Bid- foundation on 2008
Machine) Predicting surfacesoil
settlement
PSO-ANN goli, M.; Jahed Armaghani, D.; cohesionless 2016
caused by tunneling
Khamesi, H. [16]
SVM (Support Vector algorithms have become increasingly
Machine-learning Settlement of shallow
popular withfoundation
the advent of
Samui, P. Sitharam, T. [17] 2008
Machine) on
deep-learning (DL) algorithms. Many attempts to apply ML to geotechnical cohesionless soil problems
were made even before the advent of DL. Examples include artificial neural networks
(ANNs) Machine-learning algorithms
[14,15,18], adaptive neurohave become increasingly
fuzzy-based popular[12],
inference (ANFIS) withdecision
the advent of
trees
deep-learning (DL) algorithms. Many attempts to apply ML to geotechnical
(DT) [19], back-propagation neural networks (BPNNs) [15,20,21], support vector minimiza- problems
were(SVM)
tion made[17],
even andbefore
gatedthe advent units
recurrent of DL. Examples
(GRU) [22]. include artificial neural networks
(ANNs) [14,15,18], adaptive neuro fuzzy-based
To estimate subsidence, Samui and Sitharam adopted inference (ANFIS) [12],approach
a hybrid decision trees
using(DT)
an
[19], back-propagation
ANN model reinforced by neural networks
a differential (BPNNs) [15,20,21],
evolutionary algorithmsupport vector minimization
[17]. Similarly, Kim and Lee
(SVM) [17],
adopted ANN andforgated recurrent units
the assessment (GRU)
of risks [22].
of abandoned mine subsidence [23]. Chen et al.
To estimate
and Zhang subsidence,
et al. applied variousSamui
ML andandDL Sitharam adopted
algorithms, a hybrid
including a BPNN,approach
wavelet using an
neural
ANN model
network (WNN),reinforced
logisticby a differential
regression, evolutionary
extreme algorithm
learning machine [17]. Similarly,
(ELM), SVM, and Kim and
random
Lee adopted
forests (RF) toANN for the
estimate theassessment
maximumofsubsidence
risks of abandoned mine excavation,
due to tunnel subsidence [23]. Chen
however,
et. al. and Zhang et al. applied various ML and DL algorithms, including a BPNN, wavelet
Appl. Sci. 2021, 11, 12130 4 of 20

the estimation of maximum settlement with AI is hardly more valuable than analytical,
empirical or numerical methods [13].
Some of the common problems of small datasets are over-fitting and the poor perfor-
mance of models when it comes to applying them to real world problems, as the model is
trained on a subset of real-world conditions. The simple comparison of various methods
does not always provide much value, as each method requires extensive effort, skill, and
expertise for the best performance.
One of the advantages of using the DL, such as LSTM, is self-feature analysis; provided
the volume of the data is sufficiently large and the model is deep enough, then denoise
steps can be omitted without affecting the model’s performance. On the other hand, if the
data is pre-processed too small, because of difficulty of handling a large dataset, and its
features were selected by the engineer, then the chances of overfitting and degradation of
model performance increase.

3. Description of Project and EPB (Earth Pressure Balance) TBM Driving

3.1. TBM Driving
The EPB- (Earth Pressure Balance) type TBM data were obtained from a metropolitan
project in downtown Seoul. The tunnel is 1600 m long and consisted of two stations and
seven ventilation shafts. The lengths of alignment for the lot were 370 m and 373 m for
the northern and southern tunnels, respectively, and a total of 498 segment rings were
installed. One EPB TBM was used for both single-track twin tunnels, for a total excavation
length of 743 m. The tunnel was relatively shallow, at approximately 12 to 15 m, and the
ground at the departure zone was composed of a mix of weathered rock and soft rock,
while the ground at the arrival zone consisted of alluvial soil. The groundwater level
was approximately 7 to 8 m below the surface and 3 to 4 m above the tunnel crown. To
monitor the amount of settlement during EPB TBM excavation, 48 settlement markers were
installed along the alignment. Table 2 provides the dimensions and specifications of the
EPB TBM used in the project.

Table 2. EPB TBM details.

Item Description
Type Earth pressure balance
Supplier Herrenknecht (Germany)
OD/ID 7.71 m/7.69 m
Thrust force 1154 kN/m2 /53,878 kN
Cutter head Dome type, 17-inch cutter, scraper
Torque 10,364–6500 kN-m (α = 22~25)
RPM 3.4 RPM, electric motor type
Segment RC-segment, L1, 500 mm + t300 mm, 7 pieces
Muck handling Muck car + vertical conveyor belt, belt scale
Grouting Upper Section 4 EA, probe drilling (22 holes)

3.2. Geological Site Conditions

Table 3 summarizes the typical geological condition along the alignment of the tunnel.
Figure 3 depicts the geological layout of the alignment. At tunnel depth, the construction
site’s geology is a mixed face condition, consisting of alluvial gravel at the crown, weathered
rock at the middle, and soft to hard rock at the bottom of the tunnel face. This mixed-face
condition is unfavorable in terms of face stability, and the risk of settlement was high due
to the potential for over-excavation at the crown, whereas the bottom section retarded the
advance of the TBM.
Appl.
Appl.Sci. 2021,11,
Sci.2021, 11,12130
x FOR PEER REVIEW 5 of
of2020
Appl. Sci. 2021, 11, x FOR PEER REVIEW 5 of 20

Table 3.Geological
Geological siteconditions.
conditions.
Table 3. Geological site
Table 3. site conditions.
Item Item
Item Characteristics
Characteristics ofRock
Characteristics
of Rock Type
of Rock
Type Type
Residual
Residual soil
soil Sandy gravel,
Sandy gravel, N: 3~50,
N: 3~50, max size: φ500 mm
Residual soil Sandy gravel, N: max
3~50,size: φ500ϕ500
max size: mm mm
Weathered rockrock
Weathered
Weathered rock Siltycore,
Silty core, Cohesion:
SiltyCohesion: 31kPa,
core, Cohesion:
31 kPa, φφ==32°
31 kPa, 32°
ϕ = 32◦
Softrock
Soft rock
Soft rock Gneiss,
Gneiss, RMR:RMR:
Gneiss,
RMR: 30~50
30~50 30~50
Water
LevelLevel
WaterLevel Approx.
Approx.GL-7 GL-7m GL-7
m m
Water Unit weight Approx. 20–21 kN/m 2
Unitweight
Unit weight 20–21kN/m
20–21 kN/m2 2
UCS 20–110 MPa
UCS
UCSPermeability 20–110
20–110
2.5 3MPa
× 10−MPa~2.5 × 10−5 cm/s
Permeability
Permeability 2.5 × 10 −3 ~2.5
2.5 × 10 ~2.5 × 10
−3 × 10−5−5cm/s
cm/s

Figure3.3.Geological
Figure Geologicallayout
layoutalong
alongthe
thealignment.
alignment.

3.3. Occurrence
3.3.Occurrence
OccurrenceofofofSubsidence
Subsidence
3.3. Subsidence
Figure
Figure444provides
provides
providesthe the layout of of the
the4848settlement
settlementmarkers
markersalong
alongthethe alignment.
alignment. As
Asthe
the
Figure layout of the 48 settlement markers along the alignment. As
the tunnel
tunnelwas was a single-track
wasaasingle-track
single-tracktwin twin tunnel,
twintunnel,
tunnel,the the
theTBMTBM departed
TBMdeparted from the launching
departedfrom the launching shaft
launchingshaft and
shaftand
and
tunnel
reached
reached the
the arrival
arrival shaft
shaft along
along the
the southern
southern tunnel
tunnel and
andthen
then returned
returned totothe
thelaunching
launching
reached the arrival shaft along the southern tunnel and then returned to the launching
shaft along
shaftalong the
alongthe northern tunnel.
thenorthern tunnel. As Asthe theTBM
TBMhead headapproached,
approached, some
some markers
markers within
within the
shaft tunnel. As the TBM head approached, some markers within the
the influence
influence zone zone
beganbegan
to to record
record the the initiation
initiation of of movement,
movement, as as shown
shown in in Figure
Figure 2. As 2.
the
influence zone began to record the initiation of movement, as shown in Figure 2. As the
As
TBMthehead
TBMpassed
head passed
through, through,
the the settlement
settlement rate rate and
peaked peaked
then and then converged
converged onto onto
long-term
TBM head passed through, the settlement rate peaked and then converged onto long-term
long-term
settlement. settlement.
settlement.

Figure 4. Layout of settlement markers along the alignment.

Figure 4.
Figure 4. Layout
Layout of
of settlement
settlement markers
markers along
along the
the alignment.
alignment.

Figure 55depicts
Figure depictsthe
themonitored
monitoredsettlements
settlementsfromfromthe
themarkers
markersinstalled
installedalong
alongthethe
southern tunnel. Marker #2 recorded the earliest occurrence
southern tunnel. Marker #2 recorded the earliest occurrence of settlement.
occurrence of settlement. The usual trend
settlement. The usual trend
ofsettlement
of settlementconsists
consistsof
ofan
aninitial
initialheaving
heavingfollowed
followedby byaamaximum
maximumsettlement
settlementrate
ratewithin
within
aashort
shorttime
timeand
andthen
thengradual
gradualconvergence
convergenceon onaafinal
finalvalue,
value,as
asshown
shownin inmarkers
markers#2,#2,#7,
#7,
and #19.
and #19.
 Appl. Sci. 2021, 11, x FOR PEER REVIEW 6 of 20

Appl. Sci. 2021, 11, 12130 6 of 20

Appl. Sci. 2021, 11, x FOR PEER REVIEW 6 of 20

Appl. Sci. 2021, 11, x FOR PEER REVIEW 6 of 20

Figure 5. Evolution of settlement

Figure 5. Evolutionmarkers of a departure
of settlement markers of trip.
a departure trip.
Figure 5. Evolution of settlement markers of a departure trip.
Figure 5. Evolution of settlement markers of a departure trip.
Figure 66 depicts Figure
depicts 6monitored
depicts thesettlements
the monitored monitored settlements
from the from the installed
the markers
markers markers installed
along the along the
the
Figure the settlements from installed along
northern tunnel. northern
Markers tunnel.
#1 and Markers
#5 #1 and the
recorded #5 recorded
earliest the earliest occurrences
occurrences of of settlement.
settlement. Unlike Unlike
northern tunnel. Markers Figure#16 anddepicts the monitored
#5 recorded settlements
the earliest from the
occurrences of markers
settlement.installed
Unlike along the
the trends the
trends associated trends
associated withassociated
theMarkers with
markers the markers
along the along
southern the southern
tunnel, tunnel,
thenorthern the
northern northern tunnel
tunnel Unlike
the northern tunnel. #1along
and #5therecorded the earliest occurrences of settlement.
showedwith the
a long markers
lead between the initialsouthern
movementtunnel,
and thethe
highest rate oftunnel
settlement, be-
showed aa long
showed long lead
the between
leadtrends
between the
theinitial
associated withmovement
initial the markersand
movement thethe
along highest
the rate
southern of settlement,
tunnel, the northernbe- tunnel
cause the settlements monitored along theand southern highest rate
tunnel were of settlement,
influenced by the tunnel
cause the settlements
showed
because the settlements monitored
a long lead
monitored along
betweenthe
along southern
the initial tunnel
movement
the excavation,
southern with were
tunneland influenced
the highest by
ratethe
of tunnel
settlement, be-
driving along the southern tunnel thewere influenced
TBM head returning bytothe
the launch-
driving along cause
the the settlements
southern tunnel monitored along
excavation, with the
the southern
TBM head tunnel were influenced
returning to the by the tunnel
launch-
tunnel driving along theafter
ing shaft southern
a roundtunnel excavation,
trip from with shaft.
the launching the TBM head returning to the
ing shaft after driving along the southern tunnel excavation, with the TBM head returning to the launch-
launching shaftaafter
round trip from
a round tripthe
from launching shaft. shaft.
the launching
ing shaft after a round trip from the launching shaft.

Figure 6. Evolution of settlement markers of a return trip.

Figure 6. Evolution of settlement markers of a return trip.

Figure 6. Evolution
Figure 4. of
LSTM
6. Evolution of Networks
settlement markers
settlement of a return
markers trip. trip.
of a return
The LSTM
4. LSTM Networks network, which was proposed by Hochreiter and Schmidhuber (1997), is
4. LSTM Networks
a special variant of a recurrent
4. LSTM Networks The LSTM network, which was proposed neural network (RNN), in which the connections between
The LSTM network, which was proposed by Hochreiter and Schmidhuber is a(1997), is
nodes from a directed graph are by Hochreiter
depicted along aand Schmidhuber
temporal sequence.(1997),
Any RNN and its
The LSTMof a special
network, variant
which of a recurrent
was network
proposed neural network
by Hochreiter (RNN), in which the connections between
special variant a recurrent
variants have a neural
trained form (RNN),
consisting in whichand
of repetitive theSchmidhuber
units, connections
as shown in between(1997),7.isIn tradi-
Figure
a special variant nodes
of a from a directed
recurrent neural graph
networkare depicted
(RNN), along
in a temporal
which the sequence. Any
connections RNN and its
between
tional RNNs,
nodes from a directed graph the
arerepetitive
depicted unit alone
along has a simple
a temporal structure,Any
sequence. such RNN
as a tanh andlayer.
its When
nodes from variants have a trained form consisting of repetitive units, as shown in Figure 7.itsIn tradi-
variants havea adirected
trained graph
dealing withform are depicted
long-term
consisting dependency
of alongproblems,
repetitive a temporal
units, aas sequence.
traditional
shown RNN
in Any
Figure
tional RNNs, the repetitive unit alone has a simple structure, such as a tanh layer. When
RNN
involves
7. In and
multiple
tradi- matrix
variants
tional havethe
RNNs, a multiplications,
trained
repetitiveform consisting
which canhasof arepetitive
lead to exploding units,
and/oras such
shown
vanishing in Figure
a gradient 7. InWhen
tradi-To solve
problems.
dealing with unit alone
long-term dependency simple structure,
problems, a traditional asRNN tanh layer.
involves multiple matrix
tional RNNs, thesuch
dealing with long-term repetitive
problems, unit
dependency alone
many RNN has a simple
variants have structure,
been such
developed, as a
such tanh
as thelayer.
echo When
matrixnetwork,
state
multiplications, whichproblems,
can lead to aexploding
traditional RNN
and/or involves
vanishing multiple
gradient problems. To solve
the
dealing with long-term gated recurrent
dependency unit, and
problems,LSTM. Among
a traditional theseRNNvariants, LSTM
involves is the
multiple most popular as
multiplications, which can
such problems, leadmany
to exploding
RNN and/or
variants havevanishing gradient
been developed, problems.
such as the echo Tomatrix
solve
state network,
multiplications, itwhich
has the canproven
lead capability
to unit,
exploding of solving
and/or exploding
vanishing and vanishing
gradient gradient To
problems. problems.
solve
such problems, manythe gatedRNN variants
recurrent have andbeen
LSTM. developed,
Among such
these as the echo
variants, LSTM state
is thenetwork,
most popular as
such problems, many
it has RNN
the provenvariants have
capability been
of developed,
solving exploding such
the gated recurrent unit, and LSTM. Among these variants, LSTM is the most popular as itand as the
vanishingecho state
gradient network,
problems.
the the
has gated recurrent
proven unit, and
capability LSTM.exploding
of solving Among these variants, LSTM
and vanishing is the
gradient most popular as
problems.
it has the proven capability of solving exploding and vanishing gradient problems.

Figure 7. A recurrent neural network and its unfolding form.

Figure 7. A recurrent neural network and its unfolding form.

Figure7.7. A
Figure A recurrent
recurrentneural
neuralnetwork
networkand
andits
itsunfolding
unfoldingform.
form.
Appl. Sci. 2021, 11, 12130 7 of 20
Appl. Sci. 2021, 11, x FOR PEER REVIEW 7 of 20

As
AsananRNN
RNNvariant,
variant,LSTM
LSTMisisconnected
connectedby byrepetitive
repetitiveunits.
units.However,
However,unlike
unlikeaatradi-
tradi-
tional
tional RNN, the repetitive units of LSTM are composed of a cell state, an inputgate,
RNN, the repetitive units of LSTM are composed of a cell state, an input gate,anan
output
outputgate,
gate,and
andaaforget
forgetgate
gate(Figure
(Figure8).
8).The
Thecell
cellremembers
remembersvalues
valuesover
overarbitrary
arbitrarytime
time
intervals,
intervals,while
while the
the three
three gates regulate the
gates regulate the flow
flowof ofinformation
informationintointoand
andout
outofofthe
the cell.
cell. In
In the typical LSTM unit depicted in Figure 8, the blue circle represents the network’s
the typical LSTM unit depicted in Figure 8, the blue circle represents the network’s input,
input, the yellow boxes represent the learned neural network layers, and the orange shapes
the yellow boxes represent the learned neural network layers, and the orange shapes rep-
represent element-wise operations, such as addition.
resent element-wise operations, such as addition.

Figure8.8.AAlong
Figure longshort-term
short-termmemory
memoryneural
neuralnetwork.
network.

Thekey
The keytotoLSTM
LSTMisisthe thecell
cellstate
state(C(Ct );t);when
whenititruns
runsstraight
straightdown
downthe
theentire
entirechain,
chain,
onlyminor
only minorinteractions
interactionsoccur.
occur. Information
Information can caneasily
easilyand
andsimply
simplyflow
flowalong
alongthe
thechain
chain
unaffectedand
unaffected andmaintain
maintainits itsintegrity.
integrity.The
Themerging
mergingarrows
arrowsrepresent
representvector
vectorconcatenation,
concatenation,
whilethe
while the forking
forking arrows represent vector vector duplication.
duplication.TheThegates
gatesinin
LSTM
LSTM models
models areare
de-
signed totoremove
designed removeor oradd
addinformation
information to to the the cell state. They are
are composed
composedof ofaasigmoid
sigmoid
layerand
layer andmultiplication
multiplicationoperations.
operations.The Thesigmoid
sigmoidlayer layeroutput
outputisisaavalue
valuebetween
betweenzero
zeroand
and
one,
one,which
whichindicates
indicatesthetheweight
weightofofinformation
informationflow. flow.AnAnLSTM
LSTMmodelmodelachieves
achievescontrol
control
and
andprotection
protectionofofthe thecell
cellstate
statethrough
throughthesethesethreethreegates.
gates.The
Theforget
forgetgate
gatedecides
decideswhat
what
information
informationwill willbe
bedisposed
disposedofoffrom
fromthethecell
cellstate.
state.This
Thistransfer
transfercan
canbebedefined
definedas:
as:

ft =fσ
t= σ f(W
(W f · [ht−1
· [h , xt] + bf)
t−1 , xt ] + bf ) (1)
(1)
whereσσisisthe
where thesigmoid
sigmoidactivation
activationfunction,
function,W Wfisisthetheweight
weightofofthe theconnections
connectionsbetween
between
f
neurons,hht−1 is
neurons, the output of the last neuron, xt is the input of the current neuron, and bf
t−1 is the output of the last neuron, xt is the input of the current neuron, and bf
is the bias of the neuron.The
is the bias of the neuron. Theforget
forgetgate
gatedetermines
determinesthe theeffect
effectofofthe
theinput
inputon onthe
thecurrent
current
cell state, as well as the preservation and discarding of the previous
cell state, as well as the preservation and discarding of the previous cell state. The input cell state. The input
gate determines how much new information will be stored in
gate determines how much new information will be stored in the current cell state. Thisthe current cell state. This
transfer can be defined
transfer can be defined as: as:
it =itσ(W i · [h
= σ(W i · t[h
, x,tx] t+] +
−1t−1 bi )bi) (2)
(2)
C~ t = tanh(WC · [ht−1 , xC ] + bC ) (3)
C˜t = tanh(WC · [ht−1, xC ] + bC) (3)
where tanh is the activation function, and b and b are the biases of the neural network.
i C
The sigmoid
where tanh islayer decides which
the activation value
function, will
and bi be
andupdated, while
bC are the theoftanh
biases layer creates
the neural a
network.
new vector C ~ , which can be added to the cell state. After completing the above steps, the
The sigmoid layer decides which value will be updated, while the tanh layer creates a new
updated cellwhich
vector C˜, state can
can be
be defined
added toas:
the cell state. After completing the above steps, the up-
dated cell state can be defined as:
Ct = ft ∗ Ct − 1 + it ∗ C~ t (4)
Ct = ft ∗ Ct−1 + it ∗ C˜t (4)
where ∗ is an element-wise production operation. In the updating procedure of the cell’s
where
state, ∗ is
the ancell
old element-wise
state Ct−1 isproduction
multipliedoperation. In thesome
by ft to forget updating procedure
information, of the
and the new
cell’s
state, the value
candidate it ∗
old cell state Ct−1 isthe
Ct from multiplied
input gate byisftthen
to forget
added. some information,
After andhas
the cell state thebeen
new
updated,
candidate thevalue it ∗gate
output will output
Ct from the gate
the input current cell state.
is then added. This transfer
After can be
the cell defined
state has beenas:
updated, the output gate will output the current cell state. This transfer can be defined as:
ot = σ (Wo · [ht−1 , xt ] + bo ) (5)
ot = σ (Wo · [ht−1, xt ] + bo) (5)
ht = Ot ∗ tanh(Ct ) (6)
ht = Ot ∗ tanh(Ct)
In summary, the above three gates are composed of sigmoid and tanh neural network (6)
layers, which help in selecting effective information.
Appl. Sci. 2021, 11, 12130 8 of 20

5. Training Model and Prediction for Subsidence

5.1. Challenges
The training of machine data and subsidence measurements are carried out in two
phases. The first phase trains the machine data and generates input for the second phase.
The subsequent phase trains the input from Phase One, which is the weight of the hidden
layer of Phase One neural network model, and eventually predicts subsidence.
The machine data from various TBM sensors consisted of more than 700 items. The
followings were chosen to serve as the input for Phase One training. Penetration is a
key parameter for training as the sequence is organized in such a way that the rest of the
parameters support the penetration prediction. The input sequence was a series of chosen
records: penetration, chainage, thrust cylinder, torque of cutting wheel, current pressure of
real lance one, pressure of excavation chamber one, rotation speed of screw conveyor drive,
calculated excavated material depending on advance one, the actual quantity of excavated
material belt scale one, and stroke-thrust cylinder.
Machine data were recorded every 10 s over a span of several years while TBM was
driving. The amount of machine data from a TBM can be overwhelming, as a human
cannot review and interpret it all to derive tangible outcomes, such as the potential risk of
subsidence around a job site.
The diversity of methods of operating the machine and the resulting range in con-
struction quality make the problem more complicated. For example, it is relatively easy to
infer that subsidence will be correlated with the number of rings installed. However, it is
difficult to estimate how closely subsidence will correlate with the skill and experience of a
TBM operator. While face pressure should be well managed and controlled, because the
balance of the volume of excavation and spoil must be cross-checked repeatedly, it is not so
easy in practice to simultaneously supervise all critical elements.
Machine learning in Phase One is an attempt by artificial intelligence to overlook the
TBM’s entire advance, and capture the features that may be related to adversarial events.
Penetration, revolutions per minute, and chainage may indicate the location and speed of
the excavation, chamber pressure, and thrust force may indicate the earth pressure balance,
and the actual and calculated excavation volume may indicate the material balance. If
the face pressure is well controlled, then subsidence will be minor, and vice versa. We
anticipated the output of Phase One would capture the relevant features in this regard.
Machine data were produced at 10 s intervals while subsidence was measured at
intervals of every few days to weeks, according to the distance from the excavation face to
the settlement marker. For outputs of a single settlement recording, the maximum amount
of machine data as input was 95,040 fields, even if only 11 of 700 fields were recorded. To
train the machine data and subsidence in a single phase would have required a period of
trial-and-error training to produce a useful LSTM model, which is impractical in terms of
both training time and computing cost. We split the process into two phases, for training
the machine data and identifying critical features associated with subsidence. By training
for subsidence with features identified in the previous phase, LSTM models for phases can
be obtained successfully with fewer resources.
Ten records of machine data with the above 11 fields were flattened into a column and
then fed into the LSTM model, which generated one output prediction, which was then
used as the input for the next forward propagation. The loss, which is the mean square
root of the difference between the ground-truth penetration and the predicted penetration,
was defined, and an Adam optimizer with a learning rate of 0.001 steps was used to
minimize errors. We assumed that the machine data encapsulated the overall performance
of the TBM and construction quality so that it had a substantial impact on the occurrence
of subsidence.

5.2. Training Phase One: Feature Extraction from Machine Data

As discussed in the previous section, we used engineering judgment to select some of
the items. Sensitivity analyses of the other items are incomplete and could be served as a
Appl. Sci. 2021, 11, x FOR PEER REVIEW 9 of 20

Appl. Sci. 2021, 11, 12130 9 of 20

5.2. Training Phase One: Feature Extraction from Machine Data
As discussed in the previous section, we used engineering judgment to select some
subjectitems.
of the Sensitivity
for future analyses
research. Eleven ofitems
the other
wereitems are in
chosen incomplete
the training andphase:
could penetration,
be served as
chainage, thrust cylinder (Gr. A), torque cutting wheel, current pressure realpenetration,
a subject for future research. Eleven items were chosen in the training phase: lance one,
chainage,excavation
pressure thrust cylinder
chamber (Gr. one,
A), torque
rotation cutting
speedwheel,
one screwcurrent pressure
conveyor realcalculated
drive, lance one,
pressure excavation
excavated chamber one,
material depending rotationone,
on advance speed
the one
actualscrew conveyor
quantity drive, calculated
of excavated material
excavated
belt scale one,material depending
and stroke thruston advance one, the actual quantity of excavated material
cylinder.
belt scale
Data one,
wereand stroke while
recorded thrustthe
cylinder.
TBM was in operation on each of the approximately
160 working days over the course TBM
Data were recorded while the was in
of a year. operation
Machine dataonrecorded
each of the approximately
when the TBM was 160
working days over the course of a year. Machine data recorded when
idling were excluded. Because the machine data were recorded about every 10 s, there was the TBM was idling
awere excluded.
maximum Because
of 8460 the in
records machine datathe
a day, but were recorded
actual number about everyrecords
of daily 10 s, there
waswas
fewer a max-
due
imum
to ring of 8460 records
building, in a day,
cutter-head but the actual
intervention, number ofbreakdown,
equipment daily records orwas fewer due to ring
maintenance.
building, cutter-head
The machine dataintervention,
were turnedequipment breakdown,
into a sequencing batchor maintenance.
comprising 1000 records taken
everyThe 10 s,machine
producing data were turned
a sequence 2.8 h into
long.aAsequencing
maximum of batch comprising
500 iterations 1000 records
of training was
taken every 10 s, producing a sequence 2.8 h long. A maximum of 500
carried out for each batch. The window size for training a batch was 10, the input size of iterations of training
wasmodel
the carriedwas out1100,
for each
whichbatch.
wasThe
11 window size for training
by 10 flattened, and the aoutputbatch was
size 10,
wasthe input
one, whichsize
of the modelpenetration.
represented was 1100, which was 11 by 10 flattened, and the output size was one, which
represented
Figure 9penetration.
depicts the complicated dynamics taken from the various TBM sensors in
response
Figureto the driving
9 depicts theofcomplicated
the machine. We expected
dynamics takenthat
from thethemachine
variousdata
TBMfrom training
sensors in re-
could
sponsebe to organized
the drivinginto of themeaningful
machine. We features,
expectedsuch
that as
thethe amount
machine dataoffrom
excavation
trainingwork
could
completed,
be organized control of chamberfeatures,
into meaningful pressure, maintenance
such as the amountof muck balance, and
of excavation workspeed of the
completed,
TBM drive.
control of chamber pressure, maintenance of muck balance, and speed of the TBM drive.

Figure 9. Sequence
Figure 9. Sequence of
of chosen
chosen items
items of
of TBM
TBM data
data for
for machine
machinelearning.
learning. network.
network.

In
In order
order to
to analyze
analyze the
the sensitivity
sensitivity of
of chosen
chosen parameters
parameters with
with respect
respect to
to penetration,
penetration,
the Pearson correlation coefficient (PCC) was used as shown in the Equation
the Pearson correlation coefficient (PCC) was used as shown in the Equation (7), (7),

n ∑nni=∑1 xi yxi y− −
∑ni∑ x x∑∑
n
yy
(x q
rx,y (xri ), = )= q=1 i i=1 i (7)
(7)
n ∑nni=∑ 2 x −(∑ n
1 xi − ( ∑i=1 xi )
x2) nn∑∑n yy2 −
i=1 i
− ((∑
∑ni=1 yyi))
2

where rx,y(xi) is the PCC of the effect of i-th influential factor on penetration (yi), xi is the
where rx ,influential
value of y (xi ) is the factors,
PCC of the
andeffect of i-th
n is the totalinfluential
number offactor on penetration (yi ), xi is the
samples.
valueFigure
of influential factors, and n is the total number of samples.
10 shows the time series excavation records in the first box and thr PCCs of
Figure 10 shows the time series excavation records in the first box and thr PCCs of the
the chosen parameters against the penetration of TBM driving in the following boxes on
chosen parameters against the penetration of TBM driving in the following boxes on days 1,
days 1, 178 and 358, which were the first, midst, and last stages of the TBM excavation,
178 and 358, which were the first, midst, and last stages of the TBM excavation, respectively.
respectively. From a few cases of PCC analyses, chainage show low Pearson correlation,
From a few cases of PCC analyses, chainage show low Pearson correlation, which are
which are rather dependent than the influential parameters on penetration. Torque,
rather dependent than the influential parameters on penetration. Torque, thrust, pressure A
(current pressure at real lance one) and screw RPM show mild positive correlation, whereas
pressure B (excavation chamber one pressure), the calculated and actual muck volume, and
Appl. Sci. 2021, 11, x FOR PEER REVIEW 10 of 20

Appl. Sci. 2021, 11, 12130 10 of 20

thrust, pressure A (current pressure at real lance one) and screw RPM show mild positive
correlation, whereas pressure B (excavation chamber one pressure), the calculated and
thrust muck
actual cylinder stroke and
volume, show a negative
thrust correlation.
cylinder Thisaisnegative
stroke show in line with the common
correlation. This intuition
is in line
that driving
with parameters,
the common such
intuition as thrust
that driving and torque, aresuch
parameters, positively influential
as thrust to penetration,
and torque, are posi-
and that
tively ahigh chamber
influential pressure and
to penetration, for the
thatsupport of excavation
ahigh chamber face for
pressure is negatively influential,
the support of exca-
i.e., it maintains a low penetration.
vation face is negatively influential, i.e., it maintains a low penetration.

Figure 10. Pearson correlation coefficients of parameters with respect to penetration at the first, midst, and last stages of
TBM drive.
TBM drive.

Evolution of
Evolution of the Pearson correlation
correlation coefficients with respect to penetration for the
entire duration
entire duration of of TBM
TBM excavation
excavation are are shown
shown in Figure 11. Figure
Figure 11a–d show negative,
negative,
neutral, and positive correlation parameters and the combination of all, respectively.
positive correlation parameters and the combination of all, respectively. Unlike
Un-
Figure
like 10, the
Figure 10,parameter
the parameterof negative PCC is
of negative the is
PCC actual quantity
the actual of excavated
quantity material,
of excavated which
material,
is against
which the common
is against intuition
the common that a that
intuition highapenetration accompanies
high penetration the high
accompanies the quantity
high quan-of
excavated
tity muck. muck.
of excavated However, it is consistent
However, with thewith
it is consistent common intuitionintuition
the common that the that
parameters
the pa-
of positive
rameters PCC arePCC
of positive torque, pressure
are torque, A, andA,
pressure screw RPM,RPM,
and screw as shown commonly
as shown commonly in the
in
Figures 10 and 11. One thing to note is the Pearson coefficients highly fluctuate,
the Figures 10 and 11. One thing to note is the Pearson coefficients highly fluctuate, such such that
oversimplification
that oversimplificationof theofmachine
the machinedata,data,
suchsuch
as average thrust
as average or torque
thrust of ring
or torque of data, may
ring data,
eliminate the critical features related to the characteristics of the machine
may eliminate the critical features related to the characteristics of the machine drive. drive.
Figure 12 shows the graphical representation of hidden features, which are the floating-
point numbers, and are turned into grey scale according to their value from zero to one, for
black to white, respectively. We reiterate that hidden features are part of the model, which
is trained for the prediction of next penetration based on the previous machine parameters,
such as thrust, torque, and chamber pressure. Once the model becomes good at predicting
the penetration, then one of the hidden layers is exported to represent the way the machine
drove for about 2 h.
The Phase One model has three LSTM layers, as well as four layers of fully connected
layers. The third and fourth fully connected layers were devoted to extracting and saving
the weight of the hidden layers, which had 20 and 10 items, respectively. A dropout rate of
0.001 was used for regularization. During Training Phase One for all records, the weights
of the third and fourth hidden layers were saved. Approximately 915,000 machine data
fields were condensed into 915 sequences of hidden features.
Appl. Sci. 2021, 11, 12130 11 of 20
Appl. Sci. 2021, 11, x FOR PEER REVIEW 11 of 20

(a)

(b)

(c)

(d)

Appl. Figure
2021,11.
Figure
Sci. 11,Evolution
11. Evolution ofREVIEW
of
x FOR PEER thePearson
the Pearsoncorrelation
correlationcoefficients
coefficients with
with respect
respect to penetration.
to penetration. They
They are are listed
listed as:Negative
as: (a) (a) Negative
PCC
12 of 20
PCC parameter;
parameter; (b) Neutral
(b) Neutral PCC PCC parameters;
parameters; (c) Positive
(c) Positive PCC PCC parameters;
parameters; (d)parameters
(d) All All parameters combined.
combined.

Figure 12 shows the graphical representation of hidden features, which are the float-
ing-point numbers, and are turned into grey scale according to their value from zero to
one, for black to white, respectively. We reiterate that hidden features are part of the
model, which is trained for the prediction of next penetration based on the previous ma-
chine parameters, such as thrust, torque, and chamber pressure. Once the model becomes
good12.
Figure at Features
predicting the penetration,
extracted then layers
from the hidden one ofofthe hidden
LSTM layers
model 1. is exported to represent
Figure 12. Features extracted from the hidden layers of LSTM model 1.
the way the machine drove for about 2 h.
The Phase One model has three LSTM layers, as well as four layers of fully connected
layers. The third and fourth fully connected layers were devoted to extracting and saving
the weight of the hidden layers, which had 20 and 10 items, respectively. A dropout rate
of 0.001 was used for regularization. During Training Phase One for all records, the
weights of the third and fourth hidden layers were saved. Approximately 915,000 machine
data fields were condensed into 915 sequences of hidden features.
 Figure 12. Features extracted from the hidden layers of LSTM model 1.

The Phase One model has three LSTM layers, as well as four layers of fully connected
layers. The third and fourth fully connected layers were devoted to extracting and saving
Appl. Sci. 2021, 11, 12130 the weight of the hidden layers, which had 20 and 10 items, respectively. A12 dropout
of 20 rate
of 0.001 was used for regularization. During Training Phase One for all records, the
weights of the third and fourth hidden layers were saved. Approximately 915,000 machine
data fields were condensed into 915 sequences of hidden features.
5.3. Results of Phase 1 Training
5.3. Results of Phase 1 Training
Figure 13 shows the evolution of the loss, which represents the progress of the machine
data learning. The lossFigure
at 13 shows
step zerothe
wasevolution of the loss,
small because thewhich represents
training thehad
process progress
begunof the ma-
using the pre-trained model file by accident, after multiple attempts to initiate training. had be-
chine data learning. The loss at step zero was small because the training process
gun using the pre-trained model file by accident, after multiple attempts to initiate train-
However, the performance of the Phase One model was suitable for the generation of an
ing. However, the performance of the Phase One model was suitable for the generation of
input sequence for Phase One training model.
an input sequence for Phase One training model.

Figure 13. Evolution of the loss of training for the first batch of the first day of machine data.
Figure 13. Evolution of the loss of training for the first batch of the first day of machine data.
Figure 14a–c shows the evolution of the prediction versus the ground truth of the
Figure 14a–c shows the evolution of the prediction versus the ground truth of the
machine data for the 500 training iterations. At first there was no capability of prediction,
machine data for the 500 training iterations. At first there was no capability of prediction,
as the initial weights were random. However, after ten iterations, the model began to cap-
as the initial weights were random. However, after ten iterations, the model began to
ture the penetration pattern, and after 100 iterations, the trained prediction sequence
capture the penetration pattern,the
closely matched and after 100 iterations,
ground-truth sequence. the trained prediction sequence
closely matched the ground-truth sequence.
Figure 15 shows the evolution of the loss of training for the first batch of the second-
Figure 15 shows the evolution
day machine data, whichof the loss ofthe
represents training
progressforofthe
the first
machinebatch
dataoflearning
the second-
process. Sim-
day machine data, which
ilar to Figurerepresents
13, the lossthe progress
at the of the
initial step wasmachine databecause
even smaller, learning the process.
training had just
Similar to Figurebegun
13, the lossthe
using at pre-trained
the initial model
step wasfile even smaller, learning
from previous becausesteps.
the training
Periodic had
jumps of the
loss might be due to the dropout layers for the regularization
just begun using the pre-trained model file from previous learning steps. Periodic jumps of and avoiding overfitting.
the loss might be dueBecause the purpose
to the dropout of Phase
layers for theOne training was and
regularization to extract sequences
avoiding of excavation
overfitting.
features to provide input for subsequent Phase Two training,
Because the purpose of Phase One training was to extract sequences of excavation the entire data set was used
for training, but not for the evaluation. However, the effectiveness
features to provide input for subsequent Phase Two training, the entire data set was used of the evaluation was
demonstrated by the fact that the loss on the second-day excavation sequence at the zeroth
for training, but not for the evaluation. However, the effectiveness of the evaluation was
iteration was already low and the prediction well matched with the actual data from the
demonstrated by the fact that the loss on the second-day excavation sequence at the zeroth
beginning of training steps, as shown in Figure 16.
iteration was alreadyFigurelow and the prediction
17 shows wellperformance
the prediction matched with thePhase
of the actual
One data from model
training the at (a)
beginning of training steps,
the first, as shown
(b) midst and (c)in Figure
last day16.of excavation. It shows a great performance, as the
Figure 17 shows the prediction
coefficients of determinantperformance of the
(R2) are 0.99, Phase
0.993 Onerespectively,
and 1.0, training model for theatfirst,
(a) midst,
the first, (b) midst
andand (c) last
last day, which day of excavation.
supplemented It shows
the good a great shown
performance performance, as the
in the Figures 14 and 16.
coefficients of determinant (R2) are 0.99, 0.993 and 1.0, respectively, for the first, midst, and
last day, which supplemented the good performance shown in the Figures 14 and 16.
Appl.
Appl. Sci.
Sci. 2021,
2021, 11,
11, x12130
FOR PEER REVIEW 1313of
of 20
20
Appl. Sci. 2021, 11, x FOR PEER REVIEW 13 of 20

(a)
(a)

(b)
(b)

(c)
(c)
Figure
Figure 14.14. Evolution
Evolution ofof thetrained
the trainedsequences
sequencesaccording
according to
to training
training steps.
steps.(a)
(a)step
step10;
10;(b)
(b)step
step20;20;
(c)(c)
step 100.
step 100.
Figure 14. Evolution of the trained sequences according to training steps. (a) step 10; (b) step 20; (c) step 100.

Figure 15. Evolution of the loss of training for the first batch of the second-day machine data.

Figure
Figure 15.
15. Evolution
Evolution of
of the
the loss
loss of
of training
training for
for the
the first
first batch
batch of
of the
the second-day
second-day machine
machine data.
data.
Appl. Sci. 2021, 11, x FOR PEER REVIEW 14 of 20
Appl. Sci. 2021, 11, 12130 14 of 20
Appl. Sci. 2021, 11, x FOR PEER REVIEW 14 of 20

FigureFigure
16. Prediction
Figure 16. vs. ground
16.Prediction
Prediction vs. truth
vs.ground
ground forfor
truth
truth the
for thesecond
the secondday
second dayat
day atthe
at 0th
the 0th iteration.
0th iteration.
iteration.

(a)

(a)

(b)

(b)

Figure 17. Cont.

 Appl. Sci.
Appl. Sci. 2021, 11, x FOR11,PEER
2021, 12130REVIEW 1515
of 20
of 20

(c)
FigureFigure
17. Prediction performance
17. Prediction of the
performance Phase
of the PhaseOne
Onemodel.
model. (a)
(a) Prediction
Prediction ofofpenetration
penetrationononthethe
1st 1st
day; (b) Prediction
day; of penetration
(b) Prediction on on
of penetration 179th
179thday;
day;(c)
(c)Prediction
Prediction of penetration
penetrationononthethe last
last day.
day.

5.4. Training Phase Two: Subsidence Estimation of Features from Machine Data
5.4. Training Phase Two: Subsidence Estimation of Features from Machine Data
The purpose of Training Phase Two was to train the weights of the hidden layers and
The purpose
to predict of Training
subsidence. Phase Two
Subsidence was to train
is governed by how thethe
weights of thehow
TBM drives, hiddenwelllayers
the faceand
to predict subsidence. Subsidence is governed by how the TBM
pressure is controlled to avoid or minimize ground movement, and the distance of drives, how well the face
the
pressure is controlled
excavation work. to avoid or will
Subsidence minimize ground
be relatively movement,
high and themarker
if the settlement distance of the
is near theex-
cavation work. Subsidence
excavation will be of
face and the quality relatively high ifis the
the excavation settlement
poor. However, marker
subsidenceis near thelow
will be exca-
vationif face
eitherand
the excavation
the qualityquality
of theisexcavation
high, or if theis settlement marker is
poor. However, far from thewill
subsidence excavation
be low if
eitherface. The output from
the excavation Training
quality Phase
is high, or One
if the(subsidence
settlementmonitoring)
marker is and the distance
far from of the
the excavation
settlement markers from the excavation face constituted the input
face. The output from Training Phase One (subsidence monitoring) and the distance of the for Phase Two model.
The structure of the Phase Two model was essentially identical to that of the Phase
settlement markers from the excavation face constituted the input for Phase Two model.
One model. If penetration was the key item for training in Phase One, then subsidence was
The structure
the key item foroftraining
the Phase TwoOne.
in Phase model was
In the essentially
Phase One model, identical to fields,
ten other that ofincluding
the Phase
One model.
chamberIfpressure,
penetration was
torque, the key
thrust, and item for training
muck-out volume, in werePhase
chosenOne, then subsidence
as supplementary
was the key item for training in Phase One. In the Phase One
items for training. In the Phase Two model, four other fields, including chainage model, ten other offields,
exca-in-
cluding chamber
vation pressure,
face, chainage andtorque,
offset ofthrust, and muck-out
settlement markers, and volume, were
date, were thechosen as supple-
supplementary
items
mentary for training.
items Settlement
for training. In themeasurements
Phase Two model, were made at 48 settlement
four other markers.chainage
fields, including Mea-
surement data were assigned to the training and test sets, which
of excavation face, chainage and offset of settlement markers, and date, were the supple-received data from 44 and
4 markers, respectively. To be successful, the Phase Two model
mentary items for training. Settlement measurements were made at 48 settlement mark- should perform well for the
test set.
ers. Measurement data were assigned to the training and test sets, which received data
The Phase Two model has three LSTM layers, as well as three layers of fully con-
from nected
44 andlayers.
4 markers, respectively. To be successful, the Phase Two model should per-
Similar to the model for Phase One, a dropout rate of 0.001 was used for
form regularization.
well for the test Asset.
a summary, Figure 18 displays the flow of Phase One and Phase Two
The Phase
NN models. Two model has three LSTM layers, as well as three layers of fully con-
nected layers. Similar to the model for Phase One, a dropout rate of 0.001 was used for
regularization. As a summary, Figure 18 displays the flow of Phase One and Phase Two
NN models.
Appl. Sci. 2021, 11, 12130 16 of 20
Appl. Sci. 2021, 11, x FOR PEER REVIEW 16 of 20

Figure 18. Phases of machine data and subsidence training.

Figure 18. Phases of machine data and subsidence training.

5.5.Results
5.5. ResultsofofPhase
PhaseTwo
TwoTraining
Training
Asshown
As shownin inFigure
Figure19,
19,the
theloss
lossofofthe
thetest
testand
andtraining
trainingsets
setsrapidly
rapidlydecreased
decreasedtotoaa
negligible level. At the beginning of Phase Two training, the loss of
negligible level. At the beginning of Phase Two training, the loss of the test set the test setwas
waslower
lower
thanthat
than that of
of the training set,
set, but
butthe
theloss
lossininthe
thetest
testsets
setsbecame
became stagnant,
stagnant,while that
while of the
that of
Appl. Sci. 2021, 11, x FOR PEER REVIEW
training
the setset
training steadily decreased.
steadily decreased. InInthe
theend,
end,the
thelosses
lossesin in both
both the training and 17
testofsets
and test 20
sets
converged
convergedtotonegligible
negligiblelevels.
levels.
Figure 20 shows the prediction performance of the Phase Two model. It shows a great
performance, whose coefficient of determinant (R2) is 1.0. As the actual settlements are
measured with the precision of millimeters, points of actual and predicted settlements in
truth-prediction plot are discrete in mm-wise integer numbers.
Figure 21 illustrates the evolution of predictions for the ground truth for one of the
training sets. At the first epoch of training, the prediction of the training set roughly fol-
lowed a timing of the occurrence of subsidence. At the thirteenth epoch, the prediction
was relatively consistent in terms of the timing and maximum value of the settlement. At
the final (49th) epoch, the prediction and ground truth were closely matched, with only
subtle fluctuations in settlement within a brief period. While this may have been a result
of overfitting, Figures 22 and 23 show the satisfactory performance of the LSTM model in
Phase Two: the evolution of the prediction of the test sets, which were not made for train-
ing, were similar or even superior to, those of the training set with respect to ground truth.
Figure 19.
19. Evolution
The superior
Figure of
of loss
loss in
performance
Evolution in the
of training
thethe Phaseand
training Two
and test sets.
testmodel
sets. for the test sets demonstrated that
the weights of the hidden layer of the Phase One model were appropriate features for the
Figure
training 20 shows the
of settlement predictionrecords,
monitoring performance of thethat
suggesting Phase Two
this model.could
approach It shows a great
be success-
performance, whose coefficient of determinant (R 2 ) is 1.0. As the actual settlements are
fully applied to current construction projects.
measured with the precision of millimeters, points of actual and predicted settlements in
truth-prediction plot are discrete in mm-wise integer numbers.
Appl. Sci. 2021, 11, x FOR PEER REVIEW 17 of 20
Appl. Sci. 2021, 11, x FOR PEER REVIEW 17 of 20

Appl. Sci. 2021, 11, 12130 17 of 20

Figure 19. Evolution of loss in the training and test sets.

Figure 19. Evolution of loss in the training and test sets.

Figure 19. Evolution of loss in the training and test sets.

Figure 20.Performance
Figure20. Performanceof
ofsettlement
settlementprediction.
prediction.

Figure 21 illustrates the evolution of predictions for the ground truth for one of the
training sets. At the first epoch of training, the prediction of the training set roughly
followed a timing of the occurrence of subsidence. At the thirteenth epoch, the prediction
was relatively consistent in terms of the timing and maximum value of the settlement. At
the final (49th) epoch, the prediction and ground truth were closely matched, with only
subtle fluctuations in settlement within a brief period. While this may have been a result
of overfitting, Figures 22 and 23 show the satisfactory performance of the LSTM model in
Phase Two: the evolution of the prediction of the test sets, which were not made for training,
were similar or even superior to, those of the training set with respect to ground truth.
Figure 20. Performance of settlement prediction.
Figure 20. Performance of settlement prediction.
Figure 21. Evolution of the trained sequences of settlement (SM#8).

Figure 21.
Figure 21. Evolution
Evolution of
of the
the trained
trained sequences
sequences of
ofsettlement
settlement(SM#8).
(SM#8).
Figure 21. Evolution of the trained sequences of settlement (SM#8).
Figure 22. Evolution of the trained sequences of settlement (SM#2).

Figure 22. Evolution of the trained sequences of settlement (SM#2).

Figure
Figure 22.
22. Evolution
Evolution of
of the
the trained
trained sequences
sequences of
of settlement
settlement (SM#2).
(SM#2).
Appl. Sci. 2021, 11, 12130
x FOR PEER REVIEW 18 of
18 of 20

Figure 23. Evolution of the trained sequences of settlement (SM#3).

Figure 23. Evolution of the trained sequences of settlement (SM#3).
The superior performance of the Phase Two model for the test sets demonstrated
5.6. Discussion
that the weights of the hidden layer of the Phase One model were appropriate features
The
for the input sequence
training for Phase
of settlement 1 training
monitoring was a suggesting
records, series of chosen TBMapproach
that this data: penetration,
could be
chainage, thrust cylinder, torque of cutting
successfully applied to current construction projects. wheel, current pressure of real lance, pressure
of excavation chamber, rotation speed of screw conveyor drive, calculated excavated ma-
5.6. Discussion
terial depending on advance, the actual quantity of excavated material belt scale and
stroke-thrust
The inputcylinder.
sequenceAmongfor Phaseparameters,
1 trainingpenetration
was a seriesisofachosen
key parameter
TBM data: forpenetration,
training in
such a way
chainage, that cylinder,
thrust the rest oftorque
the parameters are supporting
of cutting wheel, the penetration
current pressure prediction.
of real lance, pressure The
of
output
excavationfromchamber,
Trainingrotation
Phase one,speedsubsidence monitoring
of screw conveyor and calculated
drive, the distance of the settlement
excavated material
markers
depending from onthe excavation
advance, face constitute
the actual quantitythe input for Phase
of excavated twobelt
material model.scale and stroke-
thrustThe choice Among
cylinder. of the machine data as
parameters, an input for
penetration is atraining is not by
key parameter forsensitivity
training inanalysis
such a
but
wayby thatengineering
the rest of judgement
the parametersof authors. It was by the
are supporting far more successful
penetration than initially
prediction. antici-
The output
from Training
pated as shown Phase one, subsidence
in Figures 19 and 20, monitoring
in which LSTM and the distance
approach canofbe
theeffectively
settlement markers
applied to
from the
predict the excavation face constitute
ground surface settlement. the input fora Phase
However, very longtwocomputation
model. time was required
The choice
for LSTM modelof the machine
training data
of phase as an
1 due toinput for training
the large amount is ofnot
the by sensitivity
input data. analysis
but byTherefore, next step would be performing the sensitivity analysis to than
engineering judgement of authors. It was by far more successful initially
optimize the
anticipated as shown in Figures 19 and 20, in which LSTM
number of parameters for phase one and applying various AI models to predict the risks approach can be effectively
applied
of to predict
excessive the ground
subsidence surfaceinduced
or sinkhole settlement.by TBMHowever, a veryinlong
excavation mixedcomputation
ground condi- time
was required
tion. It wouldfor beLSTM model training
a meaningful of phase the
to investigate 1 due to the large of
performance amount
the AIofmodels
the input fordata.
the
Therefore,
prediction next step
of ground would as
subsidence bewell
performing the sensitivity
as the machine parameters analysis
of thetoTBM.
optimize the
number of parameters for phase one and applying various AI models to predict the risks of
excessive
6. Conclusions subsidence or sinkhole induced by TBM excavation in mixed ground condition.
It would be a meaningful to investigate the performance of the AI models for the prediction
Due to the high demand for new transportation infrastructure, large numbers of tun-
of ground subsidence as well as the machine parameters of the TBM.
nels and underground spaces are being constructed for urban roads, railways, and elec-
trical and telecommunication infrastructures, which are associated with rapid urbaniza-
6. Conclusions
tion. Among the risks posed by tunnel construction in a congested urban area are the risks
Due to the high demand for new transportation infrastructure, large numbers of
of subsidence or the creation of a sinkhole. The resulting costs to society from such inci-
tunnels and underground spaces are being constructed for urban roads, railways, and elec-
dents can be significant. Tunneling using a TBM, which can apply face pressure to stabi-
trical and telecommunication infrastructures, which are associated with rapid urbanization.
lize the ground, is widely used to address the threat of subsidence. However, to effectively
Among the risks posed by tunnel construction in a congested urban area are the risks of
prevent such incidents, it is crucial to be able to predict where they might happen, so that
subsidence or the creation of a sinkhole. The resulting costs to society from such incidents
it enables the engineer to plan beforehand.
can be significant. Tunneling using a TBM, which can apply face pressure to stabilize the
It is well recognized that subsidence during tunnel construction is closely correlated
ground, is widely used to address the threat of subsidence. However, to effectively prevent
with the driving characteristics of the TBM. However, so far, the use of complete se-
such incidents, it is crucial to be able to predict where they might happen, so that it enables
quences
the engineer of machine data is unprecedented for capturing the features that correlates the
to plan beforehand.
way that TBM drives
It is well recognized withthat
resulting
subsidencesettlement.
duringAtunnelTBM construction
generates data from various
is closely ma-
correlated
chine sensors, including thrust, torque, chamber pressure, and
with the driving characteristics of the TBM. However, so far, the use of complete sequencesmuck discharge volume,
which would
of machine contain
data the TBM driving
is unprecedented characteristics
for capturing including
the features thatthe response
correlates theofway
ground
that
against the thrust force of the boring machine.
TBM drives with resulting settlement. A TBM generates data from various machine sensors,
Throughout
including thrust, this study,
torque, it waspressure,
chamber shown that andmachine data could
muck discharge encapsulate
volume, whichthe fea-
would
tures related to settlement occurrence patterns. The cause and consequence, i.e., TBM
Appl. Sci. 2021, 11, 12130 19 of 20

contain the TBM driving characteristics including the response of ground against the thrust
force of the boring machine.
Throughout this study, it was shown that machine data could encapsulate the features
related to settlement occurrence patterns. The cause and consequence, i.e., TBM driving
and settlement, can be modeled with an LSTM and fully connected layers separately. Data
can be obtained from the TBM every 10 s, whereas settlement is recorded daily or weekly,
according to the distance of the excavation face. The Phase One model for TBM driving,
which is the cause of settlement, can train the TBM data and extract features related to the
consequences. The output of Phase One model is the weights of the hidden layers, which
are extracted features from machine data, and which formed the input for the Phase Two
model, together with additional information, such as the location of the settlement markers,
and tunnel face and settlement records.
The performance of the models for the test sets demonstrates that the weights of the
hidden layer of Phase One model are appropriate for the training of settlement-monitoring
records and, suggests that this two-phase LSTM approach can be effectively applied to
TBM excavation projects.

Author Contributions: Data curation, H.-K.L.; Formal analysis, M.-K.S.; Investigation, H.-K.L.;
Methodology, S.S.L.; Project administration, S.S.L.; Validation, M.-K.S.; Writing—original draft
preparation, H.-K.L.; Conceptualization, S.S.L.; Supervision, S.S.L.; Writing—review & editing,
M.-K.S. All authors have read and agreed to the published version of the manuscript.
Funding: This work is supported by the Korea Agency for Infrastructure Technology Advancement
(KAIA) grant funded by the Ministry of Land, Infrastructure and Transport (Grant 21UUTI-B157786-02).
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Not applicable.
Acknowledgments: This work was supported by the Korea Agency for Infrastructure Technology
Advancement (KAIA) grant funded by the Ministry of Land, Infrastructure and Transport (Grant
21UUTI-B157786-02) and by the National Research Foundation of Korea (NRF) grant funded by the
Korea government (MSIT) (NRF-2020R1A6A3A13077513).
Conflicts of Interest: The authors declare no conflict of interest. The funders had no role in the
design of the study; the collection, analyses, or interpretation of data; the writing of the manuscript;
or the decision to publish the results.

References
1. Ko, T.Y.; Lee, S.W. Effect of Rock Abrasiveness on Wear of Shield Tunneling in Bukit Timah Granite. Appl. Sci. 2020, 10, 3231.
[CrossRef]
2. Peck, R. Deep excavations and tunneling in soft ground. In Proceedings of the 7th International Conference on Soil Mechanics
and Foundation Engineering, Mexico City, Mexico, 29 August 1969; pp. 225–292.
3. You, K.H.; Jung, S.T. A study on the behavior of surface settlement due to the excavation of twin TBM tunnels in the clay grounds.
J. Korean Geo-Environ. Soc. 2019, 20, 29–40. [CrossRef]
4. Kasper, T.; Meschke, G. On the influence of face pressure, grouting pressure and TBM design in soft ground tunneling. Tun-Nelling
Undergr. Space Technol. 2006, 21, 160–171. [CrossRef]
5. Zhang, K.; Lyu, H.M.; Shen, S.L.; Zhou, A.; Yin, Z.Y. Evolutionary hybrid neural network approach to predict shield tunneling
induced ground settlements. Tunn. Undergr. Space Technol. 2020, 106, 103594. [CrossRef]
6. Mair, R.; Taylor, R.; Burland, J. Prediction of ground movements and assessment of risk of building damage due to bored
tunneling. In Geotechnical Aspects of Underground Construction in Soft Ground; Balkema, Brookfield: Rotterdam, The Netherlands,
1996; pp. 713–718.
7. Vorster, T.E.B.; Klar, A.; Soga, K.; Mair, R.J. Estimating the effects of tunneling on existing pipelines. J. Geotech. Geo-Environ. Eng.
2005, 131, 1399–1410. [CrossRef]
8. Franza, A.; Marshall, A.M. Empirical and semi-analytical methods for evaluating tunneling-induced ground movements in sands.
Tunn. Undergr. Space Technol. 2019, 88, 47–62. [CrossRef]
9. Meschke, G. From advance exploration to real time steering of TBMs: A review on pertinent research in the collaborative research
center, “interaction modeling in mechanized tunneling”. Undergr. Space 2018, 3, 1–20. [CrossRef]
Appl. Sci. 2021, 11, 12130 20 of 20

10. Lee, H.; Choi, H.; Choi, S.; Chang, S.; Kang, T.; Lee, C. Numerical Simulation of EPB Shield Tunneling with TBM Operational
Condition Control Using Coupled DEM–FDM. Appl. Sci. 2021, 11, 2551. [CrossRef]
11. Chou, J.S.; Lin, C. Predicting disputes in public-private partnership projects: Classification and ensemble models. J. Comput. Civ.
Eng. 2013, 27, 51–60. [CrossRef]
12. Bouayad, D.; Emeriault, F. Modeling the relationship between ground surface settlements induced by shield tunneling and the
operational and geological parameters based on the hybrid PCA/ANFIS method. Tunn. Undergr. Space Technol. 2017, 68, 142–152.
[CrossRef]
13. Chen, R.; Zhang, P.; Wu, H.; Wang, Z.; Zhong, Z. Prediction of shield tunneling-induced ground settlement using machine
learning techniques. Front. Struct. Civ. Eng. 2019, 13, 1363–1378. [CrossRef]
14. Kim, C.; Bae, G.; Hong, C.; Park, S.; Moon, H.; Shin, H. Neural network-based prediction of ground surface settlements due to
tunneling. Comput. Geotech. 2001, 28, 517–547. [CrossRef]
15. Suwansawat, S.; Einstein, H. Artificial neural networks for predicting the maximum surface settlement caused by EPB shield
tunneling. Tunn. Undergr. Space Technol. 2006, 21, 133–150. [CrossRef]
16. Hasanipanah, M.; Noorian-Bidgoli, M.; Jahed Armaghani, D.; Khamesi, H. Feasibility of PSO-ANN model for predicting surface
settlement caused by tunneling. Eng. Comput. 2016, 32, 705–715. [CrossRef]
17. Samui, P.; Sitharam, T. Least-square support vector machine applied to settlement of shallow foundations on cohesionless soils.
Int. J. Numer. Anal. Meth. Geomech. 2008, 32, 2033–2043. [CrossRef]
18. Choi, Y.-H.; Lee, S.S. Predictive Modelling for Blasting-Induced Vibrations from Open-Pit Excavations. Appl. Sci. 2021, 11, 7487.
[CrossRef]
19. Dindarloo, S.; Siami-Irdemoosa, E. Maximum surface settlement-based classification of shallow tunnels in soft ground. Tunn.
Undergr. Space Technol. 2015, 49, 320–327. [CrossRef]
20. Santos, O.J.J.; Celestino, T.B. Artificial neural networks analysis of Sao Paulo subway tunnel settlement data. Tunn. Undergr. Space
Technol. 2008, 23, 481–491. [CrossRef]
21. Shi, J.; Ortigao, J.A.R.; Bai, J. Modular neural networks for predicting settlements during tunneling. J. Geotech. Geo-Environ. Eng.
1998, 389–395. [CrossRef]
22. Gao, M.; Zhang, N.; Shen, S.; Zhou, A. Real-time dynamic earth pressure regulation model for shield tunneling by integrating
GRU deep learning method with GA optimization. IEEE Access 2020, 8, 64310–64323. [CrossRef]
23. Kim, Y.K.; Lee, S.W. Study on Risk Assessment of Mine Subsidence by Machine Learning. Appl. Sci. 2020, 9, 1302. [CrossRef]