Journal Pre-Proof

Downloaded from https://iranpaper.
ir
https://www.tarjomano.com https://www.tarjomano.com
Journal Pre-proof
Predicting TBM penetration rate in hard rock condition: A

comparative study among six XGB-based metaheuristic
techniques
Jian Zhou, Yingui Qiu, Danial Jahed Armaghani, Wengang

Zhang, Chuanqi Li, Shuangli Zhu, Reza Tarinejad
PII: S1674-9871(20)30223-1
DOI: https://doi.org/10.1016/j.gsf.2020.09.020
Reference: GSF 1091
To appear in:
Received date: 21 May 2020

Revised date: 17 August 2020
Accepted date: 24 September 2020
Please cite this article as: J. Zhou, Y. Qiu, D.J. Armaghani, et al., Predicting TBM
penetration rate in hard rock condition: A comparative study among six XGB-based
metaheuristic techniques, (2020), https://doi.org/10.1016/j.gsf.2020.09.020
This is a PDF file of an article that has undergone enhancements after acceptance, such
as the addition of a cover page and metadata, and formatting for readability, but it is
not yet the definitive version of record. This version will undergo additional copyediting,
typesetting and review before it is published in its final form, but we are providing this
version to give early visibility of the article. Please note that, during the production
process, errors may be discovered which could affect the content, and all legal disclaimers
that apply to the journal pertain.
© 2020 Published by Elsevier.

Downloaded from https://iranpaper.ir
Journal Pre-proof
Predicting TBM penetration rate in hard rock condition: a comparative study among six
XGB-based metaheuristic techniques
Jian Zhoua, Yingui Qiub, Danial Jahed Armaghanic,*, Wengang Zhangd, Chuanqi Lie, Shuangli
Zhuf, Reza Tarinejadg
a
School of Resources and Safety Engineering, Central South University, Changsha 410083,
China
of
b
ro
China
c
-p
Institute of Research and Development, Duy Tan University, Da Nang 550000, Vietnam
re
d
School of Civil Engineering, Chongqing University, Chongqing 400045, China
lP
e
China
na
f
ur
China
Jo
g
Department of Civil Engineering, University of Tabriz, 29 Bahman Blvd, 51666, Tabriz, Iran
* Corresponding author. E-mail address: csujzhou@hotmail.com; j.zhou@csu.edu.cn;
danialjahedarmaghani@duytan.edu.vn
Abstract
A reliable and accurate prediction of the tunnel boring machine (TBM) performance can assist
1
Journal Pre-proof
in minimizing the relevant risks of high capital costs and in scheduling tunnelling projects.
This research aims to develop six hybrid models of extreme gradient boosting (XGB) which
are optimized by gray wolf optimization (GWO), particle swarm optimization (PSO), social
spider optimization (SSO), sine cosine algorithm (SCA), multi verse optimization (MVO) and
moth flame optimization (MFO), for estimation of the TBM penetration rate (PR). To do this,
a comprehensive database with 1286 data samples was established where seven parameters
of
including the rock quality designation, the rock mass rating, Brazilian tensile strength (BTS),
rock mass weathering, the uniaxial compressive strength (UCS), revolution per minute and
ro
trust force per cutter (TFC), were set as inputs and TBM PR was selected as model output.
-p
Together with the mentioned six hybrid models, four single models i.e., artificial neural
re
network, random forest regression, XGB and support vector regression were also built to
lP
estimate TBM PR for comparison purposes. These models were designed conducting several
na
parametric studies on their most important parameters and then, their performance capacities
were assessed through the use of root mean square error, coefficient of determination, mean
ur
absolute percentage error, and a10-index. Results of this study confirmed that the best
Jo
predictive model of PR goes to the PSO-XGB technique with system error of (0.1453, and
0.1325), R2 of (0.951, and 0.951), mean absolute percentage error (4.0689, and 3.8115), and
a10-index of (0.9348, and 0.9496) in training and testing phases, respectively. The developed
hybrid PSO-XGB can be introduced as an accurate, powerful and applicable technique in the
field of TBM performance prediction. By conducting sensitivity analysis, it was found that
UCS, BTS and TFC have the deepest impacts on the TBM PR.
Keywords: TBM penetration rate; Hard rock; XGB-based hybrid model; Predictive model;
2
Journal Pre-proof
Metaheuristic optimization
Abbreviations
TBM Tunnel boring machine RMR Rock mass rating
2
PR Penetration rate R Coefficient of determination
AR Advance rate FPI Field penetration index
XGB Extreme gradient boosting RMSE Root mean square error
ANN Artificial Neural Network α Planes of weakness
GWO Gray wolf optimization and SVR Support vector regression
MFO Moth flame optimization ICA Imperialism competitive algorithm
of
MVO Multi verse optimization GBDT Gradient boosting decision tree
SCA Sine cosine algorithm PSRWT Pahang Selangor Raw Water Transfer
ro
SSO Social spider optimization ANFIS -p Adoptive neuro-fuzzy inference system
PSO Particle swarm optimization CSM Colorado School of Mines
AI NTNU Norwegian University of Science and

Artificial intelligence
re
Technology
ML Machine learning RFR Random forest regression
lP
RQD rock quality designation MAPE Mean absolute percentage error

TFC Trust force per cutter MI Mutual information
na
RPM Revolution per minute

WZ Weathering zone
UCS Uniaxial compressive strength
ur
BTS Brazilian tensile strength

Jo
1. Introduction
Tunnel boring machines (TBMs) have been extensively applied to constructing deep and long
tunnels. Such popularity is due to the fact that these machines are highly economic and highly
efficient. All through the excavation process, TBMs show a high sensitivity to the rock mass
conditions. Indefinite rock mass conditions and uncertain information in this regard can result
in improperly-set operating parameters and, in some cases, even the decrease of both safety
3
Journal Pre-proof
and efficiency level (Armaghani et al., 2017; Liu et al., 2020). As a result, to have a careful
plan for tunneling projects and to make use of the most suitable construction techniques,
engineers need to accurately predict the TBM performance (Zhou et al., 2020b). In addition, a
precise prediction minimizes the frequency of common risks and disadvantages that may take
place during every tunneling project, e.g., high capital costs.
The previously-proposed TBM performance prediction models can be divided into 3 general
of
groups; (i) empirical and theoretical models based on laboratory testing, cutting forces, field
performance of TBMs and rock properties (Graham, 1976; Snowdon et al., 1982; Bamford,
ro
1984; Rostami, 1997; Yagiz, 2002), (ii) statistical models based on mathematical rules (Gong
-p
and Zhao, 2009; Mahdevari et al., 2014), and (iii) computational models based on artificial
re
intelligence, AI, and machine learning, ML, techniques (Benardos and Kaliampakos, 2004;
lP
Simoes and Kim, 2006; Koopialipoor et al., 2020). As example related to the first group,
na
Ozdemir (1977) succeeded to achieve the TBM penetration rate (PR) through taking into
consideration the full-scale laboratory cutting tests and numerous regression analyses. Such
ur
activities finally resulted in creation of a key predictive model of TBM PR termed Colorado
Jo
School of Mines or CSM. This model was then updated by Rostami (1997). Another
extensively-employed predictive model of the TBM performance called NTNU, was proposed
by the Norwegian University of Science and Technology. To create this model, the researchers
carried out regression analyses on both rock mass parameters and driving parameters (Bruland,
1998). Literature consists of some other commonly-used models for prediction of TBM
performance. Hamidi et al. (2010) made an analysis on relationships between the field
penetration index (FPI) of TBM and the five primary parameters of the rock mass rating
4
Journal Pre-proof
(RMR) system. Their findings confirmed the existence of a correlation amongst FPI, uniaxial
compressive strength (UCS), and orientation of discontinuities. The result helps to estimate
the FPI values. In addition, for the purpose of predicting the TBM performance, the three rock
mass classifications i.e., geological strength index, QTBM, and rock mass excavatability were
proposed and applied by some other scholars (Barton, 2000; Bieniawski et al., 2006; Preinl,
2006; Benato and Oreste, 2015; Frough et al., 2015). The proposed models belong to the first
of
group (i.e., empirical and theoretical models), generally take into consideration a limited
number of parameters and they do not consider different significant working conditions and
ro
material features (Bruines, 1998). Therefore, these models fail to offer the accuracy required
-p
for TBMs (Benardos and Kaliampakos, 2004; Yagiz and Karahan, 2015).
re
As the second group (i.e., statistical models based on mathematical rules), several scholars
lP
applied and proposed these techniques in predicting TBM performance. A linear and
na
non-linear multiple regression equations were introduced in the study conducted by Yagiz
(2008) and Yagiz et al. (2009), respectively, to predict TBM PR using 7.5 km data of Queens
ur
Water Tunnel in USA. To do this, five engineering rock properties namely, UCS, Brazilian
Jo
tensile strength (BTS), peak slope index, distance between plane of weakness and angle
between tunnel axis and the planes of weakness (α) were selected as dependent variables and
the measured TBM PR values were considered as an independent variable. Hassanpour et al.
(2011) established several relationships with suitable accuracy between different rock mass
parameters (e.g. rock quality designation, RQD, basic RMR, UCS and joint spacing) and FPI.
From 4 different rock mass properties, they found that a combination of UCS and RQD
obtains the best result for FPI prediction. The data obtained from 6.3 km of Alborz tunnel was
5
Journal Pre-proof
used to predict TBM PR in the study carried out by Rayatdust et al. (2012). In their study,
UCS, α, and volumetric joint count were considered as predictors. Finally, they proposed a
linear multiple regression equation to predict TBM PR based on the mentioned predictors
with a suitable accuracy level. However, Alvarez Grima and Verhoef (1999) mentioned that
statistical models are not always robust enough to describe nonlinear and complex systems
accurately. Moreover, their performance capacity is poor in the presence of outliers and
of
extreme values in the data (Alvarez Grima et al., 2000).
Essentially, finding the connections between the influential factors on TBM performance and
ro
TBM performance parameters, themselves, is known as a common problem that has been
-p
proved to be solved through the use of AI and ML algorithms. As the third and last group (i.e.,
re
computational models based on AI, and ML techniques), Mahdevari et al. (2014), for example,
lP
attempted to construct several models to predict the TBM PR by means of support vector
na
regression (SVR) model. Literature also consists of some other techniques employed
successfully for the same end, such as artificial neural networks (ANNs), particle swarm
ur
optimization (PSO), and fuzzy logic (Okubo et al., 2003; Khandelwal and Singh, 2009; Yagiz
Jo
and Karahan, 2011; Jain et al., 2014; Minh et al., 2017). Armaghani et al. (2017, 2019) made
use of hybrid models including imperialism competitive algorithm (ICA)-ANN and
PSO-ANN for the aim of estimating the TBM PR and TBM advance rate. In another study,
Salimi et al. (2016) utilized SVR and adaptive neuro-fuzzy inference system (ANFIS) models
to predict TBM PR. Their findings confirmed the better performance of SVR compared to
ANFIS in terms of the defined tasks. Fattahi (2016) proposed a hybrid model integrating
ANFIS and fuzzy C–means clustering approach in order to forecast the TBM PR.
6
Journal Pre-proof
Koopialipoor et al. (2019) presented an innovative approach of AI, i.e., group modelling of
data handling aiming at effectively estimating the TBM PR. In another study, a gene
expression programming equation was introduced as a high performance and applicable
model in estimating TBM PR in the study conducted by Armaghani et al. (2018). In another
project, Li et al. (2020a) used and introduced a long-short-term memory neural network for
TBM performance prediction and then, by performing a random forest model, the importance
of
of the input parameters was investigated. Liu et al. (2019) suggested an expert version of SVR,
namely stacked single-target-SVR for prediction of TBM performance and successfully
ro
showed that their proposed model is better than a common SVR predictive model. The models
-p
designed based on AI and ML techniques normally enjoy a desired level of flexibility. This
re
characteristic allows researchers to find more reliable and precise solutions to different
lP
engineering/science problems, particularly in cases where the given problem is highly

na
complex and nonlinear (Yagiz and Karahan, 2011). It should be noted that the AI and ML
techniques have been widely-used in solving science and engineering problems (Khandelwal
ur
and Singh, 2010; Sayadi et al., 2013; Khandelwal and Armaghani, 2016; Khandelwal et al.,
Jo
2017; Pham et al., 2017, 2018, 2020a, 2020b; Khosravi et al., 2018; Bejarbaneh et al., 2020;
Bui et al., 2020; Han et al., 2020; Li et al., 2020; Gao et al., 2020; Ray et al., 2020; Yong et al.,
2020; Zhou et al., 2020b, 2021).
Despite the vast application of AI and ML in predicting the TBM PR, to date, no study has as
yet developed new hybrid predictive models based on concepts of extreme gradient boosting
(XGB) framework with six optimization algorithms i.e., gray wolf optimization (GWO),
particle swarm optimization (PSO), social spider optimization (SSO), sine cosine algorithm
7
Journal Pre-proof
(SCA), multi verse optimization (MVO) and moth flame optimization (MFO). This technique
allows for the proposal of new hybrid predictive models to receive a high level of
performance in estimating TBM PR. For comparison purposes, ANN, SVR, random forest
regression (RFR) and XGB models are also applied and developed for TBM PR prediction.
In the following, first, the backgrounds of the metaheuristic intelligence techniques are given
with more details, then, the procedure of data collection and establishing TBM PR data are
of
described. After explanations of the model development process of the predictive techniques,
the best method will be chosen and introduced. Eventually, a sensitivity analysis will be
ro
conducted to identify the most important parameters on TBM PR.
-p
2. Metaheuristic methods
re
2.1. Extreme Gradient Boosting (XGB)
lP
The core of extreme gradient boosting (XGB) itself is the ensemble algorithm based on the
na
gradient boosting tree (Chen and He, 2015). Gradient boosting is a representative algorithm of
boosting in the ensemble algorithm (Friedman, 2002). XGB algorithm is an efficient

ur
implementation version of gradient boosting algorithm. Because of its excellent efficiency in

Jo
application practice, it is a widely-praised technique in industry and Kaggle machine learning
competitions. XGB is similar to gradient boosting decision tree (GBDT) and is based on the
classification and regression tree theory (Zhou et al., 2015, 2016, 2019a, 2019b; Le et al.,
2019; Ding et al., 2020; Zhang et al., 2020a, 2020b). It is able to build multiple weak
evaluators on the data and then summarizes the modeling results of the weak evaluators. In
parallel, the XGB model can effectively deal with regression and classification problems to
obtain better performance than a single one (Zhou et al., 2019b). In fact, it can symbolise a
8
Journal Pre-proof
soft computing library that combines a new algorithm with the GBDT method.
The XGB optimized objective function introduces regularization terms to prevent overfitting
(Chen and Guestrin, 2016), so that the objective function is composed of two parts. The first
part is used to measure the difference between the predicted value and the actual value
(represents the deviation of the model), and the other part is the regularization term (the
variance of the control model). The prediction accuracy of the model is determined by the
deviation and variance of the model. 𝐷 = *(𝑥𝑖 , 𝑦𝑖 )+ is a data set containing n samples and m
of
features, and the predictor is an addition model composed of 𝑘 base models. Its sample
prediction results can be expressed as:

ro
-p
K
yî   f k  xi , f k   (1)
re
k 1
   f  x   ws  x   s : R m  T , ws  RT 
lP
(2)
where, 𝑥𝑖 is one of the samples, and for a given sample, there is a prediction score of 𝑓𝑘 (𝑥𝑖 ).
na
𝜑 is the set of regression trees, each tree 𝑓(𝑥) has its structural parameters 𝑠 and leaf
ur
weight 𝑤, 𝑇 is the number of leaves in the tree, 𝐾 is the number of trees used to ensemble
Jo
the results, and 𝑦̂𝑖 is the predicted label.
In order to optimize the ensemble tree and obtain the minimum loss function, XGB introduces
model complexity to measure the operation efficiency of the algorithm. Therefore, the
objective function includes the traditional loss function and the model complexity.
 
m t
Obj t    l yi t , yî t 1  f t  xi      f k  (3)
i 1 k 1
  fk    T  1  w
2
(4)
2
where, 𝑖 represents the number of sample in the data set, and 𝑚 represents the total amount
9
Journal Pre-proof
of data imported into the kth tree. The first term in Eq. (3), represents the traditional loss
function, measuring the difference between the actual value and the predicted value. The
second term in Eq. (3), represents the complexity of the model (i.e., the regularization term).
In addition, 𝛾 and 𝜆 are parameters which are able to control the complexity of the tree, and
the regularization term helps to avoid overfitting by smoothing the final learnt weights.
Then in order to further simplify the objective function, Taylor expansion is performed on it:
m T
Obj    ft  xi  gi  1  ft  xi   hi    T  1   w j 2
t 
of
2
(5)
i 1
 2  2 j 1
ro
where gi and hi are the first and second derivatives obtained on the loss function,
respectively.
-p
re
lP
2.2. Intelligent optimization algorithms
The intelligent optimization algorithm based on XGB mainly obtains higher accuracy by
na
adjusting three important parameters of XGB model (i.e., num_boosting_rounds, eta and
ur
lambda). 'num_boosting_rounds' represents the maximum number of trees generated and the
Jo
parameter range during optimization is set to (1–150); 'eta' represents the learning rate and its
parameter range is set to (0.05–1); 'lambda' controls the regularization part and the parameter
range is set to (0.01–5).
2.2.1. Gray Wolf Optimization (GWO)
GWO is a new group of intelligent optimization algorithms in recent years. This algorithm is
an optimized search method inspired by the activity of gray wolf predation. It simulates the
social rank and predatory behavior of gray wolf population in nature (Emary et al., 2016). The
gray wolf will surround the prey during the hunting process, and the behavior of the gray wolf
10
Journal Pre-proof
surrounding the prey can be expressed as:
D  C  X p t   X t  (6)
X  t  1  X p  t   A  D (7)
where D is a vector used to specify the new position of the gray wolf, t is the number of
iterations, X is a vector representing the position of the gray wolf, A and C are the
coefficient vectors, and X p is a vector representing the position of the prey.
of
In the GWO, there are only a few adjustable parameters which are easy to set and the same
time are able to provide a strong level of optimization. Gray wolves belong to canines that
ro
live in groups and are at the top of the food chain. The gray wolf strictly abides by a hierarchy
-p
of social dominance, just like a pyramid hierarchy. Among them, the gray wolf group is
re
divided into four social levels, namely α-wolf, β-wolf, δ-wolf, and ω-wolf, of which the
lP
high-level wolf leads the low-level wolf. The optimization process of GWO includes the
na
social hierarchy of gray wolves and the tracking, enveloping and attacking activities for prey.
According to the relationship between the fitness value and each level of the gray wolf, the
ur
search for the optimal solution is completed when the wolf pack is close to the position of the
Jo
prey (Yildiz and Yildiz, 2018). More explanations regarding GWO algorithm can be found in
literature (Mirjalili et al., 2014; Jaafari et al., 2019; Yu et al., 2020).
2.2.2. Particle Swarm Optimization (PSO)
PSO is an evolutionary computing algorithm derived from the study of bird predation
behavior. It is inspired by the bird foraging process and has the characteristics of heuristics
and random search of evolutionary algorithms (Abido, 2002; Liu et al., 2019). In PSO
algorithm, the bird works as particle, and the entire bird swarm forms a particle swarm. Like
11
Journal Pre-proof
other evolutionary algorithms, there are also "groups" and "individuals" in PSO. In the search
process, each particle can be regarded as a search individual in the N-dimensional search
space. The flight speed of the particles can be dynamically adjusted according to the historical
optimal position of the particles and the historical optimal position of the population. Particles
in PSO have only two properties i.e., speed and position. Speed represents the speed of
movement, and position represents the direction of movement. The equation for updating the
of
velocity and position of each particle can be defined as follows:
V  wV  c1r1  Pbest  X   c2 r2  Gbest  X  (8)
X   X V 
ro (9)
-p
where, Pbest and Gbest are the historical best position of a single particle and the
re
historical best position of the particle swarm, the parameters c1 and c2 are called learning
lP
factors, r1 and r2 are two random probability values distributed in [0, 1], w is the inertial
na
weight, X and V represent the current position and velocity of the particle, respectively,
and the updated position and velocity of the particle are represented by X  and V  ,
ur
respectively.
Jo
The optimal solution searched by each particle, called individual extremum, and the optimal
individual extremum in the particle swarm is taken as the current global optimal solution.
Then, iterate continuously, update the speed and position, and finally get the optimal solution
that meets the termination condition. In the process, each particle cooperates with each other
to better adapt to the environment, and to achieve the optimal search of complex solutions in
complex spaces. For a complete detail of the PSO algorithm, the readers can refer to the study
conducted by Zhou et al. (2012) and Armaghani et al. (2014).
12
Journal Pre-proof
2.2.3. Social Spider Optimization (SSO)
Social spider optimization is based on the cooperative behavior of social spiders. The
optimization algorithm considers the two genders of male and female search spiders (Cuevas
et al., 2013). The social spider community consists of two main parts: its members and its
community network. According to the different genders of spiders, all members are divided
into two different groups, and each agent is conducted by a group of different operators to
of
simulate the cooperative behavior in the group. Among them, male spider populations are
divided into dominant and non-dominant categories. Dominant group spiders have better
ro
adaptability than non-dominant group spiders. They are attracted to the closest female spider
-p
in the public web. On the other hand, non-dominant male spiders tend to be concentrated in
re
the center of the male population in order to utilize the resources wasted by dominant male
lP
spiders. Each spider will bear the weight according to the suitability value of the solution
na
expressed by the social spider：
fitnesst   Worst
wt 
ur
(10)
Best  Worst
Jo
where fitnesst  represents the fitness value obtained by evaluating the position of t-th spider
t = 1, 2, ..., T. Worst and Best respectively mean the worst fitness value and the best fitness
value of the entire population.
SSO assumes that the entire search space is a public web where all social spiders interact with
each other. Each solution in the search space represents the spider location in the public
network. In order to have a better understanding regarding SSO optimization technique, other
studies in literature can be considered (James and Li, 2015).
2.2.4. Sine Cosine Algorithm (SCA)
13
Journal Pre-proof
SCA is an optimization algorithm for mathematical modeling based on sine and cosine
mathematical functions. The most common general optimization algorithm based on random
population is to divide the optimization process into two parts about exploration and
exploitation. It is worth noting that SCA uses the sine and cosine functions to explore and use
the space between the two solutions in the search space, expecting to find a better solution. In
the search space, SCA randomly initialize the position of the current solution ( X i ), and then
of
adjust to the old position as shown in the following formula:
X it  r1sin  r 2 r 3Pit  X it ,r 40.5

t 1
 {X
ro
X (11)
i  r1cos r 2  r 3 Pi  X i ,r 40.5
i t t t
-p
where X it is the position of the current solution in the i-th dimension at the t-th iteration,
re
and Pi is the position of the target point in the i-th dimension.
lP
In addition, in order to avoid local optimization, SCA can effectively explore different regions
of the search space, and then, converge to the global optimal, and use the promising regions of
na
the search space in the optimization process (Mirjalili, 2016). Based on the sine and cosine
ur
functions, SCA searches the global optimal solution with a set of random candidate solutions,
Jo
and then updates their positions outward or toward the optimal solution. At this time, when
the sine and cosine functions return values greater than 1 or less than −1, they will explore
different regions in their search space. When the sine and cosine return values are between −1
and 1, the promising area in the search space will be used. In the following references (Li et
al., 2018; Nenavath et al., 2018), more explanations of the SCA technique can be found.
2.2.5. Moth Flame Optimization (MFO)
MFO algorithm is a group of intelligence optimization inspired by the transverse
orientation-navigation behavior of moths. When moths see artificial light, they try to fly in a
14
Journal Pre-proof
straight line at a similar angle to the light, which is their special navigation method at night
(Frank, 2006; Mirjalili, 2015).
The MFO algorithm has the ability to balance search and development during operation, and
it can reduce the probability of falling into the local optimal solution space. Moth and flame
are two key components of MFO algorithm. To mathematically model the moth's lateral
orientation behavior, the position of each moth relative to the flame can be updated with the
of
following spiral function:
M i  S  M i , Fj   Di ebt COS  2 t   Fj (12)
Di  Fj  M i
ro (13)
-p
where S is a logarithmic spiral function, M i represents the i-th moth, and F j represents
re
the j-th flame, 𝑏 is a random constant of the logarithmic spiral shape, and Di represents the
lP
distance of the i-th moth for the j-th flame.

na
If the candidate solution is assumed to be a moth, the variable of the problem is the position
of the moth in space. In addition, the fitness value of the MFO is the return value of the
ur
fitness (target) function of each moth. The position vector of each moth is passed to the fitness
Jo
function, and the output of the fitness function is assigned to the corresponding moth as its
fitness value. It should be noted that both moths and flames are solutions, and the way of
processing and updating in each iteration, is different. Moths are the actual search agents that
move in the search space, and flames can be considered as the best place for moths to date.
Therefore, each moth will search and update around a flame mark in case it finds a better
solution. For more details, equations and implementation process of the MFO can be referred
to the published studies in literature (Mirjalili, 2015).
15
Journal Pre-proof
2.2.6. Multi Verse Optimization (MVO)
The MVO algorithm originates from the multiverse theory in physics, and builds a
mathematical model based on the three main concepts of the theory i.e., white holes, black
holes, and wormholes. Some people think that multiple universes interact through white holes,
black holes, and wormholes to achieve a stable state, which is the inspiration of the MVO
algorithm (Mirjalili et al., 2016). The MVO algorithm divides the search process into two
of
phases: exploration and development. The concept of white holes and black holes is used to
explore the search space of the algorithm, and the wormhole is used to develop the search
ro
space. And it assumes that the universe selects a white hole according to the expansion rate of
-p
the universe through the roulette wheel selection mechanism：
re
X kj ,r1 NI Ui 
xi  {X j ,r1 NI U
j
 i (14)
lP
j j
where Xi is the j-th parameter of the i-th universe; Xk is the j-th parameter of the k-th
na
universe selected by the roulette mechanism; r1 is the random number extracted from the [0,
ur
1] interval; U i is the i-th universes; NI Ui  is the normal expansion rate of the universe.
Jo
MVO assumes that each solution is analogous to a universe, and that each variable in the
solution is an object in the universe. In addition, MVO assigns an expansion rate to each
solution, which is proportional to the corresponding fitness function value of the solution. In
the optimization process, when the expansion rate is high, the MVO will reach to satisfaction
level. In this situation, the probability of the existence of white holes is large, and the
probability of the existence of black holes is small, and the universe with a high expansion
rate tends to send objects through the white hole. The universe tends to receive objects
through black holes. It is important to know that regardless of the expansion rate, all objects
16
Journal Pre-proof
in the universe may face random movement through the wormhole to the best universe. A
complete version of the MVO optimization algorithm can be seen in previous works (Mirjalili
et al., 2016; Zhao et al., 2018).
3. Materials
3.1. Tunnel site and established database
of
A total of 1286 data samples from the Pahang Selangor Raw Water Transfer (PSRWT) tunnel
project in Malaysia were collected in this study to be used as a database to predict PR of TBM.
ro
This database was used in the construction of multiple swarm intelligence models based on
-p
XGB. Through the PSRWT tunnel, water is transferred from Pahang to Selangor in order to
re
efficiently provide the water shortage problems that may appear in future. The excavation of
lP
the tunnel was done in order to cross the Main Range granite. The height of the mountain
na
forming the Peninsular Malaysia backbone is as high as 100 m to 1400 m. It was planned to
apply TBMs to three sections of the path and to use the commonly-employed drilling and
ur
blasting techniques in four sections.

Jo
According to above discussion, 7 model inputs that have the greatest effect on TBM PR i.e.,
trust force per cutter (TFC), UCS, RPM, BTS, RQD, weathering zone (WZ), and RMR were
set to forecast TBM PR. The 1286 data samples consist of 560 data samples of fresh
rock-mass, 553 data samples of slightly weathered rock-mass, and 173 data samples of
moderately weathered rock-mass. In order to observe/measure the relevant parameters, about
13 km of the tunnel was divided into averagely 10 m panels. In each panel, the relevant
machine factors (such as RPM, stroke speed, boring energy, TFC, cutter head, and cutter head
17
Journal Pre-proof
torque) and rock mass characteristics (such as joint conditions, WZ, water condition, rock
mass strength) were recorded/observed. Additionally, some rock blocks were gathered to
conduct some required experiments in laboratory like UCS, the Schmidt hammer, BTS,
p-wave velocity, density, and point load strength. The experiments were completed in
accordance with the methods suggested by International Society for Rock Mechanics, ISRM
(ISRM, 2007).
of
The minimum, maximum, and average values of the model inputs and model output together
with some other information are presented in Table 1. In Table 1, the ratings of the fresh,
ro
slightly and moderately WZs are considered as 1, 2 and 3, respectively. It should be noted that
-p
the similar procedure was implemented in the study conducted by Benardos and Kaliampakos
re
(2004). Full details of the collected/measured data of PSRWT tunnel can be found in the study
lP
conducted by Armaghani et al. (2017). It can be seen from the matrix analysis chart, the
na
correlation between the input variables in the data set, and the correlation between the input
variables and the output, which is presented in Fig. 1. In addition, the violin plot which shows
ur
the distribution of each input and output, and the analysis of outliers is displayed in Fig. 2.
Jo
The overall analysis process implemented in this study is shown in Fig. 3. According to this
figure, the method of this study is mainly divided into four steps: (1) data set preparation; (2)
model establishment; (3) model verification and evaluation; (4) result analysis.
3.2 Model verification and evaluation
Model verification and evaluation is a vital element of the model development process. After
the model is built, it is necessary to understand whether the undertaken model has evolved
18
Journal Pre-proof
sufficiently accurate results for the goals used and whether the quality of the test model is
excellent enough. In this study, the training set is used to train the predictive models, and the
test set is used to verify the developed models. At the same time, in order to effectively
evaluate the reliability of the hybrid models in this study, the relevant evaluation indicators:
RMSE, R2, MAPE, and a10-index are used to describe the relationship between the predicted
value and the actual value.
of
The RMSE represents the standard deviation of the fitted error between the predicted value
and the actual value. The MAPE indicator is a percentage value (error value), which reflects
the process of comparing with the original data, and 0% indicates a perfect model.
ro
-p
Furthermore, the value of R2 represents the percentage of the square of the correlation
re
between the predicted and actual values. The closer the value of R2 is to 1, the more perfect
lP
the model (Le et al., 2019; Zhang et al., 2019, 2020; Koopialipoor et al., 2020; Li et al.,
na
2020b). It should be noted that a new statistical indicator a10-index with physical engineering
significance is also proposed. And the values of the a10-index is equal to 1.0 indicates a
ur
perfect prediction model. The calculation formulas of the evaluation indicators are as follows:
Jo
N
RMSE    yˆ  yi  / N
2
i
(15)
i 1
  y  yˆ 
2
i i
R2  1 i
(16)
 y  y 
2
i i
i
N
yi  yî
 i 1 yi
MAPE  100 (17)
N
m10
a10  index  (18)
M
19
Journal Pre-proof
where yi represents the observed value, yˆ i is the predicted value of the model, yi
represents the average of the observed values, and N denotes the number of samples in the
training or testing stages. M is the number of samples, and m10 represents the number of
samples with value of rate measurement value/predicted value between 0.90 and 1.10.
4. Results and discussion
of
4.1. Comparison analysis of hybrid models
To predict the TBM penetration rate, the TBM database needs to be prepared. The
ro
training/test set division of this database was divided into two stages according to the most
-p
commonly used division ratio of 80%/20%, based on the Pareto principle (Bunkley, 2008). 80%
re
of the data was randomly selected in the database for training of all models, and 20% of the
lP
data for testing of the models.

na
In order to evaluate the PR prediction model developed, the performance indicators in Eqs.
(15–18) were used, including RMSE, R2, MAPE, and a10-index. Notably, the same test data
ur
set was used for all prediction models.

Jo
The development of all XGB-based models was carried out according to the method in Fig. 3.
At first, the initialization operation of the relevant parameters of the XGB model was
performed. Then, the relevant parameters of each optimization algorithm were set (see Table 2
for relevant parameters). Additionally, a 10-fold cross-validation resampling technique was
used in the XGB-based hybrid model to improve the reliability and performance of the
optimization process. The optimal parameters of the model obtained during the optimization
process were also listed in Table 2.
20
Journal Pre-proof
Next, a variety of XGB-based hybrid intelligent models proposed in this paper were trained
using the training set, and different prediction performances were obtained. As shown in Fig.
4, the correlation between the predicted value and the actual value of the training data set can
be seen. The training effects of these intelligent models are still relatively good, and the
training sample points are basically distributed in near the perfect fit line ("actual PR =
of
predicted PR"). From the point of view of the RMSE, the determination coefficient, the
MAPE, and the a10-index, the MVO-XGB intelligent model has a slightly better training
ro
effect, with RMSE value of 0.1384, R2 value of 0.9555, MAPE value of 3.9882, and
-p
a10-index value of 0.9397. The training effect of the SSO-XGB intelligent model is slightly
re
worse, and its RMSE value is 0.1463, R2 value is 0.9503, MAPE value is 4.0906, and
lP
a10-index value is 0.9309.

na
The R2 value of these mixed models is basically above 0.95, indicating that the six
XGB-based optimization techniques proposed in this paper are able to achieve high training
ur
effects. After the model training is completed, the testing data set is used to verify and
Jo
evaluate these six hybrid intelligent models. As shown in Fig. 5, by analyzing the correlation
and error between the predicted PR value and the actual PR value of the test data set, it can be
seen that the test sample points are also basically distributed in near the perfect fitted line
("actual PR= predicted PR"). The prediction performance of the six hybrid models from high
to low is MFO-XGB (RMSE: 0.1309; R2: 0.9522; MAPE: 3.7589; a10-index: 0.9535),
PSO-XGB (RMSE: 0.1325; R2: 0.951; MAPE: 3.8115; a10-index: 0.9496), GWO-XGB
(RMSE: 0.1345; R2: 0.9496; MAPE: 3.8437; a10-index: 0.9496), SSO-XGB (RMSE: 0.1348;
21
Journal Pre-proof
R2: 0.9493; MAPE: 3.841; a10-index: 0.9535), SCA-XGB (RMSE: 0.1357; R2: 0.9487;
MAPE: 3.8647; a10-index: 0.9535), MVO-XGB (RMSE: 0.1367; R2: 0.9479; MAPE: 3.9106;
a10-index: 0.9496), indicating that the prediction performance of the six hybrid models are all
reaching relatively high prediction accuracy, the MFO-XGB hybrid intelligent model has the
best prediction performance.
In order to further compare and analyze the prediction performance of these six hybrid models,
of
Table 3, Figs. 6 and 7 summarize the model performance of each hybrid model. Table 3 shows
the performance index results and ranking system of six models of GWO-XGB, MFO-XGB,
ro
PSO-XGB, SSO-XGB, SCA-XGB and MVO-XGB in predicting TBM PR. Figure 6 presents
-p
the overall ranking results in a more intuitive stacked graphs way. In Fig. 7, four evaluation
re
indicators of the six mixed models are shown. The comprehensive results show that, the
lP
PSO-XGB hybrid model is not the most accurate prediction model during training and testing
na
compared with the other five hybrid models. But combined with the comprehensive ranking
of training and testing, PSO-XGB is the best among the six intelligent hybrid models.
ur
Moreover, the PSO-XGB hybrid model has the characteristics of fast convergence speed,
Jo
small error and high accuracy in the process of intelligent optimization. In Fig. 8, the target
optimization iteration graph of the six hybrid models is shown, in which it can be seen that the
PSO-XGB hybrid model also shows a good effect. Furthermore, the Taylor diagram of the
developed models in predicting TBM PR is shown in Fig. 9. As it can be seen in this figure,
some other predictive models i.e., SVR, XGB, ANN, and RFR (Zhou et al., 2012, 2017, 2021;
Armaghani et al., 2017, 2019; Le et al., 2019; Li et al., 2020b) were modelled and compared
to predict TBM PR. The results showed that although all models are good in predicting TBM
22
Journal Pre-proof
PR with high level of accuracy, but overall, the PSO-XGB hybrid model provides better
learning and prediction capabilities.
It should be noted that the same data was used in the study conducted by Armaghani et al.
(2017) and actually, it was the original study of TBM performance prediction of PSRWT
tunnel. They applied and developed 2 hybrid models of PSO-ANN and ICA-ANN for
forecasting the TBM PR values. The R2 results of these 2 hybrid models were obtained as
of
(0.897 and 0.905) and (0.919 and 0.912) for training and testing phases of PSO-ANN and
ICA-ANN models, respectively. The results of the original study and the present study
ro
showed that the developed PSO-XGB model in this study with R2 of (0.951 and 0.951 for
-p
train and test phases, respectively) is significantly better than the PSO-ANN and ICA-ANN
re
models in the study conducted by Armaghani et al. (2017). Therefore, this article recommends
lP
using the PSO-XGB hybrid model to predict the TBM PR.

na
4.2. Relative importance of the influenced variables

ur
The prediction of the TBM PR under specific rock mass conditions is the key to the
Jo
mechanical tunneling project. In order to accurately predict the performance of the TBM and
reduce the high cost and the risk of tunneling, the influence of the factors must be considered
and evaluated comprehensively. In summary, all input variables used in this study i.e., RQD,
UCS, RMR, BTS, WZ, TFC, and RPM have an effect level on TBM PR, however, the
sensitivity of each input variable is unclear and needs further study. Meanwhile, in order to
obtain the overall conclusion and optimization plan for predicting TBM PR, this paper
analyzes the importance of the input variables on model output by the mutual information test
23
Journal Pre-proof
(Verron et al., 2008) method. Mutual information method (MI) is a filtering method used to
capture the arbitrary relationship (including linear and nonlinear relationship) between each
feature and the label. It is a measure of the interdependence between variables and indicates
the strength of the relationship between variables. The size of the mutual information between
variables can be calculated by the information gain:
Yv
Ent Y v 
V
Gain Y , X   Ent Y    (19)
v 1 Y
of
where 𝑣 represents the number of all possible values of X , Yv represents the set of Y
ro
corresponding to when 𝑥 takes 𝑥𝑣 , and Ent Y  represents the information entropy.
-p
The larger the value of Gain Y , X  , the higher the correlation between X and Y .
re
Finally, according to the variable score in the mutual information test, the importance level of
lP
the input variables in estimating TBM PR, were computed. As shown in Fig. 10, from the
analysis results, it is observed that UCS, BTS, TFC, RQD, RMR and RPM have a great level
na
of importance on TBM PR. Their importance scores were obtained as 1.4796, 1.0606, 0.9492,
ur
0.8922, 0.8276 and 0.6505, respectively. Therefore, in the prediction of TBM PR, UCS, BTS,
Jo
TFC, RQD, RMR and RPM are important factors to be considered. It is important to mention
that in our database, the WZ received the lowest influence on the TBM PR. The next plan of
the authors is to develop a new model based on these 6 most important factors for predicting
TBM PR. Additionally, in the future, more laboratory tests may be useful to enrich the
database and more number of datasets need to be considered to train and construct the PR
models.
5. Conclusions
24
Journal Pre-proof
This paper systematically verifies and comparatively analyzes the hybrid XGB-based
optimization techniques in predicting TBM PR. The hybrid models were planned by
combining XGB with six intelligent optimization algorithms i.e., GWO, MFO, PSO, SSO,
SCA and MVO. With full consideration of the influencing factors affecting the TBM PR, the
established TBM data set was used to train and test these six XGB hybrid models, and the
performance of them was evaluated by RMSE, R2, and MAPE. Finally, the mutual
of
information test was used to analyze the importance score of each input variable.
In summary, the six hybrid XGB models proposed in this paper have good potential for
ro
predicting TBM PR, and can effectively assist XGB in hyper-parameter adjustment. The
-p
prediction performance of the six hybrid models for test data from high to low is MFO-XGB
re
(RMSE: 0.1309; R2: 0.9522; MAPE: 3.7589), PSO-XGB (RMSE: 0.1325; R2: 0.951; MAPE:
lP
3.8115), GWO-XGB (RMSE: 0.1345; R2: 0.9496; MAPE: 3.8437), SSO-XGB (RMSE:
na
0.1348; R2: 0.9493; MAPE: 3.841), SCA-XGB (RMSE: 0.1357; R2: 0.9487; MAPE: 3.8647),
MVO-XGB (RMSE: 0.1367; R2: 0.9479; MAPE: 3.9106). Among them, the comprehensive
ur
performance of the PSO-XGB hybrid model is superior to other five models.

Jo
Besides, three other predictive methods i.e., SVR, ANN, RFR and XGB were constructed to
predict TBM PR for comparison purposes. The results revealed that the prediction effect of
the XGB-based optimization techniques is better than those non-optimized models. The
optimal model is the PSO-XGB predictive model and it can be introduced as the best one in
field of TBM performance prediction. Finally, the mutual information test was used to obtain
the importance score of each input variable. They were obtained as 1.4796 (UCS), 1.0606
(BTS), 0.9492 (TFC), 0.8922 (RMR), 0.8276 (RQD), 0.6505 (RPM), and 0.0689 (WZ).
25
Journal Pre-proof
Among them, UCS, BTS and TFC are highly sensitive factors compared to others. Notably,
the six hybrid models for predicting PR proposed in this article are only recommended to be
applied under similar conditions, because these models are designed based on the model
inputs selected in this article. In addition, the same procedure introduced in this study can be
implemented for other TBM performance parameters such as AR and FPI. The proposed
models in this study can be used as practical techniques to estimate TBM PR for similar rock
of
mass and material properties in site investigation phase and before tunneling project
construction.
ro
-p
re
Acknowledgements
lP
This research was funded by the National Science Foundation of China (41807259), the
na
Innovation-Driven Project of Central South University (No. 2020CX040) and the Shenghua
Lieying Program of Central South University (Principle Investigator: Dr. Jian Zhou). The
ur
authors also wish to express their appreciation to the Universiti Teknologi Malaysia (UTM)
Jo
for supporting this study during data collection stage.
Reference
Abido, M.A., 2002. Optimal power flow using particle swarm optimization. Int. J. Electr. Power
Energy Syst. 24(7), 563–571.
Alvarez Grima, M., Verhoef, P.N.W. (1999). Forecasting rock trencher performance using fuzzy logic.
International Journal of Rock Mechanics and Mining Sciences 36(4), 413-432.
26
Journal Pre-proof
Alvarez Grima, M., Bruines, P.A., Verhoef, P.N.W. (2000). Modeling tunnel boring machine
performance by neuro-fuzzy methods. Tunnelling and Underground Space Technology 15(3),
259-269.
Armaghani, D.J., Hajihassani, M., Bejarbaneh, B.Y., Marto, A., Mohamad, E.T. (2014). Indirect
measure of shale shear strength parameters by means of rock index tests through an optimized
artificial neural network. Measurement, 55, 487-498.
Armaghani, D.J., Mohamad, E.T., Narayanasamy, M.S., Narita, N., Yagiz, S., 2017. Development of
of
hybrid intelligent models for predicting TBM penetration rate in hard rock condition. Tunnelling
and Underground Space Technology, 63, 29-43.
ro
Armaghani, D.J., Faradonbeh, R.S., Momeni, E., Fahimifar, A., Tahir, M.M., 2018. Performance
-p
prediction of tunnel boring machine through developing a gene expression programming
re
equation. Engineering with Computers, 34(1), 129-141.
lP
Armaghani, D.J., Koopialipoor, M., Marto, A., Yagiz, S., 2019. Application of several optimization
techniques for estimating TBM advance rate in granitic rocks. Journal of Rock Mechanics and
na
Geotechnical Engineering 11(4), 779-789.
Bamford, W.F. (1984). Rock test indices are being successfully correlated with tunnel boring machine
ur
performance. In: Proceedings of the 5th Australian Tunneling Conference, Melbourne, 218.
Jo
Barton, N. (2000). TBM tunnelling in jointed and faulted rock. Balkema, Rotterdam.
Bejarbaneh, E.Y., Masoumnezhad, M., Armaghani, D. J., Pham, B.T. (2020). Design of robust control
based on linear matrix inequality and a novel hybrid PSO search technique for autonomous
underwater vehicle. Applied Ocean Research 101, 102231.
Benardos, A.G., Kaliampakos, D.C., 2004. Modelling TBM performance with artificial neural
networks. Tunnell. Undergr. Space Technol. 19(6), 597-605.
Bieniawski, Z. T., Caleda, B., Galera, J. M., Alvares, M.H. (2006). Rock mass excavability (RME)
index. ITA World Tunnel Congress (Paper no. PITA06-254), April, Seoul, 10p.
27
Journal Pre-proof
Bruland, A. (1998). Hard rock tunnel boring. Ph.D. Thesis, Norwegian University of Science and
Technology, Trondheim.
Bruines, P. (1998). Neuro-fuzzy modeling of TBM performance with emphasis on the penetration rate.
Memoirs of the Centre of Engineering Geology in The Netherlands, Delft, 173, 202.
Bui, X.N., Nguyen, H., Choi, Y., Nguyen-Thoi, T., Zhou, J., Dou, J., 2020. Prediction of slope failure
in open-pit mines using a novel hybrid artificial intelligence model based on decision tree and
evolution algorithm. Scientific reports, 10(1), 1-17.
of
Chen, T., & Guestrin, C. (2016). Xgboost: A scalable tree boosting system. In Proceedings of the 22nd
acm sigkdd international conference on knowledge discovery and data mining (pp. 785-794).
ACM.
ro
-p
Chen, T., & He, T. (2015). XGBoost: Extreme gradient boosting.R package version 0.4-2, 1-4.
re
Cuevas E, Cienfuegos M, Zaldívar D, Pérez-Cisneros M. (2013). A swarm optimization algorithm
lP
inspired in the behavior of the social-spider. Expert Systems with Applications 40(16),
6374-6384.
na
Ding, Z., Nguyen, H., Bui, X.N., Zhou, J., Moayedi, H., 2020. Computational intelligence model for
estimating intensity of blast-induced ground vibration in a mine based on imperialist competitive

ur
and extreme gradient boosting algorithms. Natural Resources Research, 29(2), 751-769.
Jo
Emary, E., Zawbaa, H.M., Hassanien, A.E. (2016). Binary grey wolf optimization approaches for
feature selection. Neurocomputing 172, 371-381.
Farrokh, E., Rostami, J., Laughton, C. (2012). Study of various models for estimation of penetration
rate of hard rock TBMs. Tunnelling and Underground Space Technology 30, 110-123.
Fattahi, H. (2016). Adaptive neuro fuzzy inference system based on fuzzy c–means clustering
algorithm, A technique for estimation of TBM penetration rate. Iran University of Science &
Technology, 6(2), 159-171.
Frank, K.D., 2006. Effects of artificial night lighting on moths. In: Rich, C., Longcore, T. (Eds.),
Ecological consequences of artificial night lighting. Island Press, Washington, DC, pp. 305-344.
28
Journal Pre-proof
Friedman JH (2002) Stochastic gradient boosting. Comput Stat Data Anal. 38(4), 367-378
Friedman, J., Hastie, T., Tibshirani, R. (2000). Additive logistic regression: a statistical view of
boosting (with discussion and a rejoinder by the authors). The Annals of Statistics 28(2),
337-407.
Frough, O., Torabi, S.R., Yagiz, S., 2015. Application of RMR for estimating rockmass–related TBM
utilization and performance parameters: A case study. Rock Mech. Rock Eng. 48 (3), 1305-1312.
Gao, B., Wang, R., Lin, C., Guo, X., Liu, B., Zhang, W., 2020. TBM penetration rate prediction based
of
on the long short-term memory neural network. Underground Space.
https://doi.org/10.1016/j.undsp.2020.01.003
ro
Gong, Q. M., Zhao, J. (2009). Development of a rock mass characteristics model for TBM penetration
-p
rate prediction. International Journal of Rock Mechanics and Mining Science. 46(1), 8-18.
re
Graham, P.C., 1976. Rock exploration for machine manufacturers. In: Bieniawski, Z.T. (Ed.),
lP
Exploration for rock engineering. Johannesburg, Balkema, pp. 173-180.
Hamidi, K.J., Shahriar, K., Rezai, B., Bejari, H. (2010). Application of fuzzy set theory to rock
na
engineering classification systems: an illustration of the rock mass excavability index. Rock
mechanics and rock engineering. 43(3): 335-350.

ur
Han, H., Armaghani, D.J., Tarinejad, R., Zhou, J., Tahir, M.M., 2020. Random forest and bayesian
Jo
network techniques for probabilistic prediction of flyrock induced by blasting in quarry
sites. Natural Resources Research 29, 655–667.
Hassanpour, J., Rostami, J., Zhao, J. (2011). A new hard rock TBM performance prediction model for
project planning. Tunnelling and Underground Space Technology 26(5), 595-603.
ISRM, 2007. The complete ISRM suggested methods for rock characterization, testing and
monitoring: 1974–2006.
Jaafari, A., Panahi, M., Pham, B.T., Shahabi, H., Bui, D.T., Rezaie, F., Lee, S., 2019. Meta
optimization of an adaptive neuro-fuzzy inference system with grey wolf optimizer and
29
Journal Pre-proof
biogeography-based optimization algorithms for spatial prediction of landslide susceptibility.
Catena, 175, 430-445.
Jain, P., Naithani, A.K., Singh, T.N., 2014. Performance characteristics of tunnel boring machine in
basalt and pyroclastic rocks of Deccan traps–A case study. Journal of Rock Mechanics and
Geotechnical Engineering, 6(1), 36-47.
James, J.Q., Li, V.O., 2015. A social spider algorithm for global optimization. Applied Soft
Computing, 30, 614-627.
of
Khandelwal, M., Singh, T.N., 2009. Prediction of blast-induced ground vibration using artificial neural
network. International Journal of Rock Mechanics and Mining Sciences, 46(7), 1214-1222.
ro
Khandelwal, M., Armaghani, D.J., 2016. Prediction of drillability of rocks with strength properties
-p
using a hybrid GA-ANN technique. Geotechnical and Geological Engineering, 34(2): 605-620.
re
Khandelwal, M., Armaghani, D.J., Faradonbeh, R.S., Yellishetty, M., Abd Majid, M.Z., Monjezi, M.,
lP
2017. Classification and regression tree technique in estimating peak particle velocity caused by
blasting. Engineering with Computers, 33(1), 45-53.

na
Khandelwal, M., Singh, T.N., 2010. Prediction of macerals contents of Indian coals from proximate
and ultimate analyses using artificial neural networks. Fuel, 89(5), 1101-1109.
ur
Khosravi, K., Pham, B.T., Chapi, K., Shirzadi, A., Shahabi, H., Revhaug, I., Prakash, I., Bui, D.T.,
Jo
2018. A comparative assessment of decision trees algorithms for flash flood susceptibility
modeling at Haraz watershed, northern Iran. Science of the Total Environment, 627, 744-755.
Koopialipoor, M., Fahimifar, A., Ghaleini, E.N., Momenzadeh, M., Armaghani, D.J., 2020.
Development of a new hybrid ANN for solving a geotechnical problem related to tunnel boring
machine performance. Engineering with Computers, 36(1), 345-357.
Koopialipoor, M., Nikouei, S.S., Marto, A., Fahimifar, A., Armaghani, D.J., Mohamad, E.T., 2019.
Predicting tunnel boring machine performance through a new model based on the group method
of data handling. Bulletin of Engineering Geology and the Environment, 78(5), 3799-3813.
30
Journal Pre-proof
Le, L.T., Nguyen, H., Zhou, J., Dou, J., Moayedi, H., 2019. Estimating the heating load of buildings
for smart city planning using a novel artificial intelligence technique PSO-XGBoost. Applied
Sciences, 9(13), 2714.
Li, C., Zhou, J., Armaghani, D.J., Li, X., 2020b. Stability analysis of underground mine hard rock
pillars via combination of finite difference methods, neural networks, and Monte Carlo
simulation techniques. Underground Space. https://doi.org/10.1016/j.undsp.2020.05.005
Li, E., Zhou, J., Shi, X., Armaghani, D.J., Yu, Z., Chen, X., Huang, P. (2020c). Developing a hybrid
of
model of salp swarm algorithm-based support vector machine to predict the strength of
fiber-reinforced cemented paste backfill. Engineering with Computers, 1-22.
ro
Li, J., Li, P., Guo, D., Li, X., Chen, Z. (2020a). Advanced prediction of tunnel boring machine
-p
performance based on big data. Geoscience Frontiers. https://doi.org/10.1016/j.gsf.2020.02.011
re
Li, S., Fang, H., Liu, X., 2018. Parameter optimization of support vector regression based on sine
lP
cosine algorithm. Expert systems with Applications, 91, 63-77.
Liu, B., Wang, R., Guan, Z., Li, J., Xu, Z., Guo, X., Wang, Y. (2019). Improved support vector
na
regression models for predicting rock mass parameters using tunnel boring machine driving
data. Tunnelling and Underground Space Technology, 91, 102958.

ur
Liu, B., Wang, R., Zhao, G., Guo, X., Wang, Y., Li, J., Wang, S. (2020). Prediction of rock mass
Jo
parameters in the TBM tunnel based on BP neural network integrated simulated annealing
algorithm. Tunnelling and Underground Space Technology, 95, 103103.
Mahdevari, S., Shahriar, K., Yagiz, S., Shirazi, M. A. (2014). A support vector regression model for
predicting tunnel boring machine penetration rates. International Journal of Rock Mechanics and
Mining Sciences 72, 214-229.
Minh, V.T., Katushin, D., Antonov, M., Veinthal, R., 2017. Regression models and fuzzy logic
prediction of TBM penetration rate. Open Eng. 7 (1), 60-68.
Mirjalili, S., Mirjalili, S.M., Lewis, A., 2014. Grey wolf optimizer. Advances in Engineering Software,
31
Journal Pre-proof
69, 46-61.
Mirjalili S (2015) Moth-flame optimization algorithm: a novel nature-inspired heuristic paradigm.
Knowl-Based Syst. 89, 228-249.
Mirjalili, S. (2016). SCA: A sine cosine algorithm for solving optimization problems.
Knowledge-Based Systems, 96, 120-133.
Mirjalili, S., Mirjalili, S.M., Hatamlou, A. (2016). Multi-Verse Optimizer: a nature-inspired algorithm
for global optimization. Neural Comput & Applic 27, 495-513.
of
Mogana, S. N. (2007). The effects of ground conditions on TBM performance in tunnel excavation –
A case history. Proceedings of the 10th Australia New Zealand conference on Geomechanics ,
442-447.
ro
-p
Mogana, S.N., Rafek, A.G., Komoo, I. (1998). The influence of rock mass properties in the assessment
re
of TBM performance. In: Moore, D., Hungr, O. (Eds.), 8th IAEG Congr, Vancouver, Balkema,
lP
Rotterdam, 3553-3559.
Nenavath, H., Jatoth, R.K., Das, S., 2018. A synergy of the sine-cosine algorithm and particle swarm
na
optimizer for improved global optimization and object tracking. Swarm and Evolutionary
Computation, 43, 1-30.

ur
Bunkley, N., 2008. Joseph Juran, 103, pioneer in quality control, dies. New York Times. Retrieved
Jo
from
chrome-extension://ibllepbpahcoppkjjllbabhnigcbffpi/http://www.richardswanson.com/textbookres
ources/wp-content/uploads/2013/08/Ch-8-Joseph-Juran.pdf.
Okubo, S., Kfukie, K., Chen, W. (2003). Expert systems for applicability of tunnel boring machine in
Japan. Rock Mechanics Rock Engineering. 36: 305-22.
Ozdemir, L. (1977). Development of theoretical equations for predicting tunnel borability. Ph.D.
Thesis, T-1969, Colorado School of Mines, Golden, CO, USA.
Pham, B.T., Bui, D.T., Prakash, I., Dholakia, M.B., 2017. Hybrid integration of Multilayer Perceptron
32
Journal Pre-proof
Neural Networks and machine learning ensembles for landslide susceptibility assessment at
Himalayan area (India) using GIS. Catena, 149, 52-63.
Pham, B.T., Prakash, I., Bui, D.T., 2018. Spatial prediction of landslides using a hybrid machine
learning approach based on random subspace and classification and regression trees.
Geomorphology, 303, 256-270.
Pham, B.T., Qi, C., Ho, L.S., Nguyen-Thoi, T., Al-Ansari, N., Nguyen, M.D., Nguyen, H.D., Ly, H.B.,
Le, H.V., Prakash, I., 2020a. A Novel Hybrid Soft Computing Model Using Random Forest and
of
Particle Swarm Optimization for Estimation of Undrained Shear Strength of
Soil. Sustainability, 12(6), 2218.
ro
Pham, B.T., Jaafari, A., Avand, M., Al-Ansari, N., Dinh Du, T., Yen, H.P.H., Prakash, I. (2020b).
-p
Performance evaluation of machine learning methods for forest fire modeling and
re
prediction. Symmetry, 12(6), 1022.
lP
Ray, A., Kumar, V., Kumar, A., Rai, R., Khandelwal, M., Singh, T.N. (2020). Stability prediction of
Himalayan residual soil slope using artificial neural network. Natural Hazards, 103(3):
na
3523-3540.
Rayatdust, H., Shahriar, K., Ahangari, K., Kamali-Bandpey, H. (2012). A Statistical Model for
ur
Prediction TBM Performance using Rock Mass Characteristics in the TBM Driven Alborz
Jo
Tunnel Project. Research Journal of Applied Sciences, Engineering and Technology 4(23),
5048-5054.
Rostami, J. (1997). Development of a Force Estimation Model for Rock Fragmentation with Disc
Cutters through Theoretical Modeling and Physical Measurement of Crushed Zone Pressure.
Ph.D. Thesis, Colorado School of Mines, Golden, Colorado, USA.
Salimi, Alireza, Rostami, J., Moormann, C., Delisio, A. (2016). Application of non-linear regression
analysis and artificial intelligence algorithms for performance prediction of hard rock TBMs.
Tunnelling and Underground Space Technology, 58, 236-246.
33
Journal Pre-proof
Sapigni, M., Berti, M., Behtaz, E., Busillo, A., Cardone, G. (2002). TBM performance estimation
using rock mass classification. International Journal of Rock Mechanics and Mining Sciences and
Geomechanics Abstracts. 39: 771-788.
Sayadi, A., Monjezi, M., Talebi, N., Khandelwal, M. (2013). A comparative study on the application
of various artificial neural networks to simultaneous prediction of rock fragmentation and
backbreak. Journal of Rock Mechanics and Geotechnical Engineering, 5(4), 318-324.
Shi, X.Z., Zhou, J., Wu, B.B., Huang, D., Wei, W., 2012. Support vector machines approach to mean
of
block size of rock fragmentation due to bench blasting prediction. Transactions of Nonferrous
Metals Society of China, 22(2): 432-441.
ro
Simoes, M. G., Kim, T. (2006). Fuzzy modeling approaches for the prediction of machine utilization
-p
in hard rock tunnel boring machines. In: Industry Applications Conference, 2006. 41st IAS
re
Annual Meeting. Conference Record of the 2006 IEEE. IEEE. vol. 2, 947-954.
lP
Snowdon, R. A., Ryley, M. D., Temporal, J. (1982). A study of disc cutting in selected British rocks.
International Journal of Rock Mechanics and Mining Sciences. 19: 107-121.

na
Ulusay R,, Hudson J.A., (2007). Suggested methods prepared by the commission on testing methods.
International Society for Rock Mechanics. 628.

ur
Verron, S., Tiplica, T., Kobi, A., 2008. Fault detection and identification with a new feature selection
Jo
based on mutual information. Journal of Process Control, 18(5), 479-490.
Xu, H., Zhou, J., G Asteris, P., Jahed Armaghani, D., Tahir, M.M., 2019. Supervised machine learning
techniques to the prediction of tunnel boring machine penetration rate. Applied Sciences, 9(18),
3715.
Yagiz, S. (2002). Development of Rock Fracture and Brittleness Indices to Quantifying the Effects of
Rock Mass Features and Toughness in the CSM Model Basic Penetration for Hard Rock
Tunneling Machines. PhD Thesis. T-5605, Colorado School of Mines, CO, USA.
Yagiz, S. (2008). Utilizing rock mass properties for predicting TBM performance in hard rock
34
Journal Pre-proof
conditions. Tunnelling and Underground Space Technology. 23(3), 326-339.
Yagiz, S., Gokceoglu, C., Sezer, E., Iplikci, S. (2009). Application of two non-linear prediction tools
to the estimation of tunnel boring machine performance. Engineering Applications of Artificial
Intelligence, 22(4): 808-814.
Yagiz, S., Karahan, H. (2011). Prediction of hard rock TBM penetration rate using particle swarm
optimization. International Journal of Rock Mechanics and Mining Sciences. 48(3): 427-433.
Yagiz, S., & Karahan, H. (2015). Application of various optimization techniques and comparison of
of
their performances for predicting TBM penetration rate in rock mass. International Journal of
Rock Mechanics and Mining Sciences, 80, 308-315.
ro
Yildiz, B.S., Yildiz, A.R. (2018). Comparison of grey wolf, whale, water cycle, ant lion and
-p
sine-cosine algorithms for the optimization of a vehicle engine connecting rod. Mater. Test. 60,
re
311-315.
lP
Yong, W., Zhou, J., Armaghani, D.J., Tahir, M.M., Tarinejad, R., Pham, B.T., Van Huynh, V., 2020.
A new hybrid simulated annealing-based genetic programming technique to predict the ultimate
na
bearing capacity of piles. Engineering with Computers, 1-17.
https://doi.org/10.1007/s00366-019-00932-9.
ur
Yu, Z., Shi, X., Zhou, J., Chen, X., Miao, X., Teng, B., & Ipangelwa, T. (2020). Prediction of
Jo
blast-induced rock movement during bench blasting: Use of gray wolf optimizer and support
vector regression. Natural Resources Research 29, 843-865.
Zhang, P., Wu, H.N., Chen, R.P., Chan, T.H., 2020. Hybrid meta-heuristic and machine learning
algorithms for tunneling-induced settlement prediction: A comparative study. Tunnelling and
Underground Space Technology, 99, 103383.
Zhang, W., Zhang, R., Wu, C., Goh, A.T.C., Lacasse, S., Liu, Z., Liu, H., 2019. State-of-the-art review
of soft computing applications in underground excavations. Geoscience Frontiers 11(4),
1095-1106.
35
Journal Pre-proof
Zhang, W., Wu, C., Zhong, H., Li, Y., Wang, L., 2020a. Prediction of undrained shear strength using
extreme gradient boosting and random forest based on Bayesian optimization. Geoscience
Frontiers. https://doi.org/10.1016/j.gsf.2020.03.007
Zhang, W., Zhang, R., Wu, C., Goh, A.T., Wang, L., 2020b. Assessment of basal heave stability for
braced excavations in anisotropic clay using extreme gradient boosting and random forest
regression. Underground Space. https://doi.org/10.1016/j.undsp.2020.03.001
Zhao, H., Han, X., Guo, S., 2018. DGM (1, 1) model optimized by MVO (multi-verse optimizer) for
of
annual peak load forecasting. Neural Computing and Applications, 30(6), 1811-1825.
Zhou, J., Bejarbaneh, B.Y., Armaghani, D.J., Tahir, M.M., 2020a. Forecasting of TBM advance rate in
ro
hard rock condition based on artificial neural network and genetic programming techniques.
-p
Bulletin of Engineering Geology and the Environment, 79, 2069-2084.
re
Zhou J., Li E., Wang M., Chen X., Shi X., Jiang L. (2019a). Feasibility of stochastic gradient boosting
lP
approach for evaluating seismic liquefaction potential based on SPT and CPT case histories. J
Perform Constr Facil, 33(3), 04019024

na
Zhou, J., Li, E., Yang, S., Wang, M., Shi, X., Yao, S., Mitri, H.S., 2019b. Slope stability prediction for
circular mode failure using gradient boosting machine approach based on an updated database of
ur
case histories. Safety Science, 118, 505-518.

Jo
Zhou, J., Li, X., Mitri, H.S., 2015. Comparative performance of six supervised learning methods for
the development of models of hard rock pillar stability prediction. Natural Hazards, 79(1),
291-316.
Zhou, J., Li, X., Mitri, H.S. 2016. Classification of rockburst in underground projects: Comparison of
ten supervised learning methods. Journal of Computing in Civil Engineering, 30(5), 04016003.
Zhou, J., Li, X. B., Shi X.Z. 2012. Long-term prediction model of rockburst in underground openings
using heuristic algorithms and support vector machines. Safety Science, 50(4): 629-644.
Zhou, J., Qiu, Y., Zhu, S., Armaghani, D.J., Khandelwal, M., Mohamad, E.T., 2020b. Estimation of
36
Journal Pre-proof
the TBM advance rate under hard rock conditions using XGBoost and Bayesian optimization.
Underground Space. https://doi.org/10.1016/j.undsp.2020.05.008
Zhou, J., Qiu, Y., Zhu, S., Armaghani, D.J., Li, C., Nguyen, H., Yagiz, S., 2021. Optimization of
support vector machine through the use of metaheuristic algorithms in forecasting TBM advance
rate. Engineering Applications of Artificial Intelligence, 97, 104015.
https://doi.org/10.1016/j.engappai.2020.104015
Zhou, J., Shi, X., Du, K., Qiu, X., Li, X., Mitri, H.S. 2017. Feasibility of random-forest approach for
of
prediction of ground settlements induced by the construction of a shield-driven tunnel.
International Journal of Geomechanics, 17(6), 04016129.
ro
-p
re
lP
na
ur
Jo
37
Journal Pre-proof
LIST OF FIGURES
Figure 1. Scatterplot matrix of TBM dataset with correlation.
Figure 2. Violin plots distribution of TBM data.
Figure 3. The overall analysis process of hybrid intelligence models based on XGB.
Figure 4. Correlation analysis between predictive values and actual values of the training
dataset.
Figure 5. Correlation analysis between predictive values and actual values of the testing
of
dataset.
ro
Figure 6. Intuitive display of comprehensive ranking of six mixed models.
-p
Figure 7. Multi-axis graph of model evaluation indicators.
re
Figure 8. The fitness changes with iteration in the optimization process.
lP
Figure 9. Model performance comparison in Taylor diagrams.
Figure 10. Importance score of influencing variables on TBM PR.

na
LIST OF TABLES
ur
Table 1 Summary of variables definition.

Jo
Table 2 The parameters of algorithms and the optimal parameters of models.
Table 3 Comparison of model performance with XGB-based hybrid models.
38
Journal Pre-proof
Declaration of interests
The authors declare that they have no known competing financial interests or personal relationships that
could have appeared to influence the work reported in this paper.
of
ro
-p
re
lP
na
ur
Jo
39
Journal Pre-proof
Highlights:
 Six XGBoost-based hybrid models for predicting TBM penetration rate are proposed.
 GWO, PSO, SSO, SCA, MVO and MFO can assist the hyper-parameters tuning of
XGBoost.
 The prediction performance from high to low is PSO-XGB, MFO-XGB, GWO-XGB,
MVO-XGB, SCA-XGB, SSO-XGB.
 The Mutual information method is applied to demonstrate the relative importance of
each input indicator.
of
ro
-p
re
lP
na
ur
Jo
40

Journal Pre-Proof

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Journal Pre-Proof

Uploaded by

Copyright:

Available Formats

Downloaded from https://iranpaper.

Predicting TBM penetration rate in hard rock condition: A

Jian Zhou, Yingui Qiu, Danial Jahed Armaghani, Wengang

Received date: 21 May 2020

© 2020 Published by Elsevier.

XGB-based metaheuristic techniques

Zhuf, Reza Tarinejadg

* Corresponding author. E-mail address: csujzhou@hotmail.com; j.zhou@csu.edu.cn;

AI NTNU Norwegian University of Science and

RQD rock quality designation MAPE Mean absolute percentage error

RPM Revolution per minute

BTS Brazilian tensile strength

place during every tunneling project, e.g., high capital costs.

use of hybrid models including imperialism competitive algorithm (ICA)-ANN and

expression programming equation was introduced as a high performance and applicable

namely stacked single-target-SVR for prediction of TBM performance and successfully

engineering/science problems, particularly in cases where the given problem is highly

2020; Zhou et al., 2020b, 2021).

boosting in the ensemble algorithm (Friedman, 2002). XGB algorithm is an efficient

implementation version of gradient boosting algorithm. Because of its excellent efficiency in

application practice, it is a widely-praised technique in industry and Kaggle machine learning

prediction results can be expressed as:

the results, and 𝑦̂𝑖 is the predicted label.

2.2. Intelligent optimization algorithms

range is set to (0.01–5).

2.2.1. Gray Wolf Optimization (GWO)

surrounding the prey can be expressed as:

coefficient vectors, and X p is a vector representing the position of the prey.

literature (Mirjalili et al., 2014; Jaafari et al., 2019; Yu et al., 2020).

2.2.2. Particle Swarm Optimization (PSO)

V  wV  c1r1  Pbest  X   c2 r2  Gbest  X  (8)

conducted by Zhou et al. (2012) and Armaghani et al. (2014).

2.2.3. Social Spider Optimization (SSO)

expressed by the social spider：

value of the entire population.

studies in literature can be considered (James and Li, 2015).

2.2.4. Sine Cosine Algorithm (SCA)

X it  r1sin  r 2 r 3Pit  X it ,r 40.5

2.2.5. Moth Flame Optimization (MFO)

MFO algorithm is a group of intelligence optimization inspired by the transverse

(Frank, 2006; Mirjalili, 2015).

M i  S  M i , Fj   Di ebt COS  2 t   Fj (12)

distance of the i-th moth for the j-th flame.

to the published studies in literature (Mirjalili, 2015).

2.2.6. Multi Verse Optimization (MVO)

et al., 2016; Zhao et al., 2018).

3.1. Tunnel site and established database

blasting techniques in four sections.

moderately weathered rock-mass. In order to observe/measure the relevant parameters, about

3.2 Model verification and evaluation

value and the actual value.

4. Results and discussion

data for testing of the models.

set was used for all prediction models.

for relevant parameters). Additionally, a 10-fold cross-validation resampling technique was

process were also listed in Table 2.

a10-index value is 0.9309.

best prediction performance.

learning and prediction capabilities.

using the PSO-XGB hybrid model to predict the TBM PR.

4.2. Relative importance of the influenced variables

variables can be calculated by the information gain:

performance of the PSO-XGB hybrid model is superior to other five models.