Computers and Geotechnics 158 (2023) 105339

Contents lists available at ScienceDirect

Computers and Geotechnics

journal homepage: www.elsevier.com/locate/compgeo

Combining machine learning and physics modelling to determine the

natural cave property with fracturing curves
Xuhai Tang a, b, Di Wu a, Jiangmei Qiao a, *, Fei Gao a, Meizhu Zhang c
a
School of Civil Engineering, Wuhan University, Wuhan 430072, China
b
Wuhan University Shenzhen Research Institute, Shenzhen 518108, China
c
Office of Scientific Research and Development, Wuhan University, Wuhan 430072, China

A R T I C L E I N F O A B S T R A C T

Keywords: Currently, it is still challenging to determine the geological parameters of natural caves in deep reservoirs
ML-physics method accurately. By inheriting the advantages of Machine Learning (ML) method and physics modelling, a novel ML-
Machine learning Physics method is developed to determine the geological parameters of natural caves based on the data mining of
Physics modelling
fracturing curves obtained during Hydraulic Fracturing (HF) operation. The computational time of ML-Physics
Fracturing curve
Real time
method is divided into two stages, preparation-stage and operation-stage. The preparation-stage happens
before HF operation, therefore there is no limitation to the computational time. During this preparation-stage,
the implicit relationship between cave property and fracturing curve is generated using ML, which usually
fails to ensure the accuracy under different geological and operational conditions. The operation-stage happens
during HF operation, in which the computational time is limited because the geological parameters of natural
caves are required to be determined in real time. During this operation-stage, the physical modelling based
inverse analysis method is carried out, in which the initial value is chosen based on the ML results obtained in
preparation-stage. Results show that, with the same target error, the iteration step of ML-Physics method (1
iteration) is much less than that of traditional inverse analysis method (5 iterations). After the same iterations,
the error of fracturing curve using the traditional inverse analysis method is 0.40%, while the error using ML-
Physics method is 0.02%. Meanwhile, the error of permeability using the traditional ML is up to 10.33%,
while the error of ML-Physics method is 0.29%. The present ML-Physics method is potentially useful to optimize
the HF design based on the data mining of fracturing curves in real time.

1. Introduction discontinuities. With the rapid development of Internet of Things (IoT)

The exploitation of oil and gas resources is very important for
maintaining world energy security and keeping economic development.
To exploit reservoirs with a great number of discontinuities, such as the
Tahe carbonate fractured-cavity reservoirs in China (Zhu et al., 2015;
Zhou et al., 2022), it is necessary to determine the geometry and
geological parameters of natural caves to optimize the hydraulic frac­
turing design (Ma et al., 2018; Ali et al., 2019). Since the reservoirs are
often buried deep underground, it is difficult to achieve the geological
properties of natural caves using drilling technology (Li et al., 2016; An
et al., 2021). Currently, seismic tomography is widely used to detect the
distribution of natural fractures and caves (Chalikakis et al., 2011; Xu
et al., 2016). However, there is still a lack of available approaches to
conduct the monitoring data to obtain geological parameters of
technology, a large number of monitoring data, such as fracturing
curves, can be obtained in real time (Wang et al., 2022). Therefore, it is
critical to develop an available method to determine the geological
parameters according to the evolution of fracturing curves.
The geological discontinuities, including natural caves, joints, and
faults have significant impact on the evolution of fracturing curves (Bian
et al., 2016; Liu et al., 2014), which have been investigated by labora­
tory experiments (Li et al., 2019; Zhou et al., 2010), numerical simula­
tions (Zhao et al., 2014; Yan and Zheng, 2017), and field observations
(Gao et al., 2016; Du et al., 2022). Chen et al. (2020) divided the frac­
turing curves into four types according to the fracturing pressure and
morphology for evaluating the relationship between the fracturing
curves and gas productivity. Ni et al. (2020) classified fracturing curves
into five classes based on the pressure fluctuation amplitude and curve

* Corresponding author.
E-mail address: Jiangmei_Qiao@whu.edu.cn (J. Qiao).

https://doi.org/10.1016/j.compgeo.2023.105339

Available online 20 March 2023

0266-352X/© 2023 Elsevier Ltd. All rights reserved.
X. Tang et al.
Fig. 1. Artificial neural network. (a) The signal flow of neuron; (b) The network architecture of network.
shape by investigating 58 coalbed methane wells in the Qinshui Basin.
Zhao et al. (2022) demonstrated that, the pumping pressure curves are
closely correlated to the propagation of fractures. Liu et al. (2020)
classified the connections between hydraulic fractures and preexisting
caves into three main connection modes, which is then used to enhance
the development of carbonate fracture-cavity reservoirs. Qiao et al.
(2022) simulated the impact of the strike angle, geostress, cave internal
pressure and fluid injection pressure on hydraulic fracture propagation
using AiFrac-TOUGH, which showed natural fractures had the greatest
effect on hydraulic fracture propagation, followed by geostress.
In order to determine the geological parameters by the data mining
of fracturing curves, recently several inverse analysis methods have been
developed. Zhang et al. (2021) established the implicit relationship
between the fracturing curves and the natural cave properties using the
Artificial Neural Networks (ANNs), in which the training data of ANNs
was generated by numerical models. As a machine learning method, the
ANNs method is able to get the natural cave geological properties in real
time during hydraulic fracturing operations, but it becomes inaccuracy
when the geological sites and operational conditions are changed. Liu
et al. (2022) combined the physical modelling and gradient descent
inversion algorithm to determine the geological parameters of joints
based on the characteristics of the fracturing curves. As an inverse
analysis method, the gradient descent inversion algorithm has high ac­
curacy but requires large computational amount. Currently, it is still
difficult for inverse analysis method to determine the geological prop­
erties of natural caves with fracturing curves accurately and in real time.
In this work, a combination of machine learning (ML) and physics
modelling named as ML-Physics method is developed to identify the
geological properties of natural caves based on the monitoring data of
fracturing curves. In Section 2, the ML-Physics method is introduced, in
which the procedure is divided into two stages, preparation-stage and
operation-stage. At the preparation-stage, the implicit relationship be­
tween the geological properties and fracturing curves is established
using the artificial neural networks (ANNs). In this work, the dataset of
machine learning is generated by physics modelling. During operation-
stage, the physics modelling based inverse analysis method is used to
identify the geological parameters of natural caves more accurately, in
which the initial value is chosen based on the results of previous ML In
section 3, the ANNs is used to predict geological parameters of natural
caves according to fracturing curves. In Section 4, the traditional inverse
analysis method is used to predict the natural cave properties. The ML-
Physics method is proposed in Section 5 to predict the geological pa­
rameters of natural caves based on fracturing curves. Finally, Section 6
presents the main conclusions.
2. Methodology
2.1. AiFrac-TOUGH for hydraulic fracturing modelling
In this work, the AiFrac-TOUGH simulator (Tang et al. 2019) is used
2
Computers and Geotechnics 158 (2023) 105339
to model three-dimensional fluid-driven propagation of multiple frac­
tures, meanwhile the evolution of fracturing curves are achieved. In the
context of AiFrac-TOUGH simulator, the finite element-mesh-free
method (FEMM) is used to calculate the rock deformations associated
with fracture propagation. Using TOUGH2, the fluid flow in fractures
and porous rock is calculated. With AiFrac-TOUGH, multiple interacting
fractures are able to be simulated at decameter and 100 m scale (Zang &
Stephansson 2019).
2.2. Machine learning method
In this work, the artificial neural networks (ANNs) is used to study
the relationship between the geological parameters of natural caves and
the response characteristic parameters of fracturing curves. The vector
of input signals is x = (x1, x2, …, xp) and the vector of output signals is y
= (y1, y2, …, yq). The subscript p and q are the total numbers of input
signals and output signals, respectively. In the current work, p = 4, and
x1, x2, x3 and x4 are the total pressure drop, pressure drop rate, rising
rate and descending steps of fracturing curves, respectively. q = 1 and y1
is the permeability of the natural cave. The neural network is con­
structed by a set of neurons which are divided into several layers. A
neuron is an information-processing unit that is fundamental to the
functionality of neural network. The Fig. 1 (a) shows the signal flow in
one neuron. In this work, the feed-forward network is used to figure out
the relationship between input signal and output signal. The network
architecture is shown as Fig. 1 (b), which contains an input layer of
source nodes, hidden layers and an output layer. The output signal of the
former layer will be used as the input signal in the latter layer.
The input layer of source nodes requires no computation. The hidden
layer sums all the input signals with the weights and the bias are applied
to make an affine transformation. After that, an activation function is
applied to make a nonlinear transformation. The last layer is output
layer, which is used as the final result of the prediction. In mathematical
terms, the transformation process can be described as the Eq. (1).
⎧ a(l) = x if l = 1
⎨
a(l) = fl (W (l) a(l− (1)
1)
+ b(l) ) if 1 < l < t
⎩
(l) (l− 1) (l)
y = fl (W a +b ) if l = t
where a(l− 1) is the (l-1)th layer’s output signal. fl (⋅) is the activation
function, such as Threshold Function, Logsig Function or Tansig Func­
tion. W (l) and b(l) represent the weights and bias of the lth layer,
respectively.
In this work, Mean Squared Error (MSE) is used as the loss function
when the network is trained:
q
1∑
L= y − yi )2 × 100%
(̂ (2)
q i=1 i
where yi and ̂y i are the true values and predicted values of a single
sample, and q represents the number of output signals for each sample.
X. Tang et al. Computers and Geotechnics 158 (2023) 105339

The training process of artificial neural networks is the updating and

adjusting process of layer weight W and bias b. The parameter update
formulas are expressed as follows:
⎧
⎪
⎪ 1 ∑N ∂L(y(n) , ̂y )
(n)
⎪
⎪ (l) (l)
⎨ W ←W − α(N
⎪ n=1 ∂W (l)
)
(3) fi xk
⎪
⎪
⎪
⎪ (l) (l) 1 ∑N ∂L(y(n) , ̂
(n)
y ) fi xk
⎩ b ←b − α(N
⎪ )
n=1
∂b(l)
where N is the total number of training samples. α is the learning yi-fi xk
yi yi
rate, and L(y(n) , ̂y ) is loss function value of the nth sample.
(n)
yi -fi xk
In order to improve the optimization speed of neural network, the
adaptive moment estimation algorithm (Adam) is used to update the
parameters (Chang et al. 2019). This method uses the momentum as the
direction of the parameter update. The learning rate adjusts adaptively. Fig. 2. The error function for kth iteration step.
The parameter update formulas are expressed as follows:
⎧
1 ∑ N
∂L(y(n) , ̂y (n) ) is defined as the difference between the numerical results simulated by
⎪
⎪
⎪
⎪
g(l)
t (θ) = AiFrac-TOUGH and the monitoring data of fracturing curves collected in
⎪ N n=1 ∂θ(l)
⎪
⎪
⎪
⎪
t
field work. The error function for kth iteration step is defined as (see
⎪
⎪
⎪
⎪
(l)
Mt = β1 Mt− 1 + (1 − β1 )g(l)
(l)
t (θ)
Fig. 2):
⎪
⎪
⎪
⎪ √̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅
⎪
⎪ (l) ( ) ∑m
⎪ Gt = β2 Gt− 1 + (1 − β2 )gt (θ) ⊙ g(l)
(l) (l)
t (θ)
( (k) )
⎪ (7)
(k) (k) (k) (k) 2
⎪
⎪ μ = μ̄ x = μ̄ x1 , x2 , ⋯, xq = [yi − fi (x(k) )]
⎨
Mt(l) (4)
i=1
⎪ M̂ (l)
t =
⎪
⎪
⎪
⎪
(1 − β1 ) where y = (y1, y2, …, ym)T is the vector composed by monitoring
⎪
⎪
⎪
⎪ (l) points value on fracturing curves. m is the total number of monitoring
⎪
⎪ ̂ (l) = Gt
⎪
⎪
⎪
⎪
G t
(1 − β2 ) points. x(k) = (x(k) (k) (k) T
1 , x2 …,xq ) is the unknown parameters for kth iter­
⎪
⎪
⎪
⎪
⎪ ation step and the subscript q is the total number of unknown parame­
⎪ α ̂ (l)
⎪
⎪
⎩ Δθ(l)
t = − √̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅ M t ters. f(x) = (f1 (x), f2 (x), …, fm (x)) is the mapping of unknown parameters
(l)
̂ +ε
G t vector to target vector, which is achieved by AiFrac-TOUGH. In the
present work, the total number of monitoring points m is 50. The un­
where θ(l)
t represents the updated parameters of layer l in step t, known parameter in the present work is the permeability of natural
which is named as W(l) and b(l). gt(l) (θ) represents the gradient associated caves, and the total number of unknowns q is 1.
with the loss function L. M(l)
t and Gt are the mean of the gradient and the
(l)
For each iteration step, the unknown vector x(k+1) is updated as:
variance without subtracting the mean, respectively. β1 = 0.9 and β2 =
x(k+1) = x(k) + V(k+1) (8)
0.99. Δθ(l)
t is the parametric update difference of layer l in step t. α is the
learning rate, which usually sets as 0.001. where V is the velocity matrix for (k + 1)th iteration step, which
(k+1)

In order to adjust the hyperparameters of the neural network, k-fold is calculated as:
cross-validation method is proposed. This method divides training
samples into k folds to train neural network. The mean absolute per­ V(k+1) = αV(k) − λ(k) G(k) (9)
centage error (MAPE) is used to evaluate the validity of the hyper­ (k)
where V is the velocity matrix for the kth iteration step. G is the (k)
parameters in neural network, which is defined as follows: gradient of the error function. λ(k) is the step length. α is the attenuation
coefficient, which varies between [0.5, 0.9] and is set to 0.7 in this work.
1∑ k
MAPE = Lj × 100% (5) The V(k) is a column vector of m rows, and initialized to 0.
k j=1
The gradient of the error function G(k) is calculated as:
in which k represents the number of sample divisions for training set. ⎡
( )
⎤
μ̄ x(k) (k)
1 + ε̄(x1 ) ..., xi(k) ..., x(k) − μ(k)
Lj represents the loss function value of the neural network model, when q
⎢ ⎥
using the jth fold of the training set as validation set. ⎢
⎢
ε̄(x(k)
1 )
⎥
⎥
The prediction accuracy of neural network is evaluated based on ⎢
⎢ ⋮ ⎥
⎥
average absolute percentage error (MAPE) and determination coeffi­ ⎢ (
⎢ (k) (k) (k) (k)
) ⎥
(k) ⎥
cient R2. The smaller the determination coefficient is, the smaller the
⎢ μ̄ x ..., x + ε̄ (x ) ..., x − μ ⎥
(10)
(k) ⎢ 1 i i q ⎥
G =⎢ ⎥
error is. The determination coefficient is calculated as follows: ⎢ ε̄(x (k)
) ⎥
⎢ i ⎥
( )) ⎢ ⎥
N ( ⎢ ⋮ ⎥
1 ∑ y (n)
L y(n) , ̂ q
1 ∑ (n) ⎢ ( ) ⎥
R2 = 1− , ȳ (n)
= y (6) ⎢ (k) (k) (k) ⎥
N n=1 (n)
L(y(n) , ȳ ) q i=1 i ⎣ μ̄ x1 ..., xi (k) (k)
..., xq + ε̄(xq ) − μ ⎦

where yi and ̂
y i represent the actual value and predicted value of the
ε̄(x(k)
q )

ith sample, respectively. q represents the number of output signals for where ε̄(x(k)
q ) is a small variable associated with xq to calculate the
(k)
each sample, and N represents the number of samples.
gradient.
The step length λ(k) is calculated as:
2.3. Inverse analysis based on modified gradient descent method and
physics modelling (11)
k
λ(k) = λ(0) × e− l

The modified gradient descent method is a useful approach to solve where λ(0) is the initial step length. l is the parameter controlling the
nonlinear functions. In this work, the error function (objective function) reduction of λ(k) and set to 3.

3
X. Tang et al.
Fig. 3. The flowchart of ML-Physics method.
2.4. The combination of machine learning and physics modelling (ML-
Physics method)
As shown in Fig. 3, the computational time of ML-Physics method is
divided into two stages: preparation-stage and operation-stage. The
preparation-stage happens before Hydraulic Fracturing (HF) operation,
in which the execution time, tbe, can be as long as possible. Meanwhile,
the operation-stage happens during HF operation, in which, the execu­
tion time, tin, should be very short to meet the requirement of HF opti­
mization in real time. By inheriting the advantages of Machine Learning
(ML) and physics modelling, a novel ML-Physics method based on the
data mining of fracturing curves to determine the natural cave property
is proposed as:
Preparation-stage with machine learning method: as shown in
Fig. 3, before the hydraulic fracturing operation in field work, the im­
plicit relationship between natural cave properties and fracturing curves
is generated, using physics modelling and machine learning methods,
4
Computers and Geotechnics 158 (2023) 105339
such as the Artificial Neural Networks (ANNs). First, the geometries of
natural caves can be detected using seismic tomography (see Fig. 3 (a)),
which assumed to be accurate. Then, a set of numerical models with
different natural cave properties are investigated using AiFrac-TOUGH
modelling, to get the database of ANNs which contains the informa­
tion of fracturing curves and the properties of natural caves. The HF
simulation is extremely time-consuming, which would spend a lot of
time to generate enough data. Because preparation-stage is completed
before hydraulic fracturing operation in field work, there is enough time
to run HF simulation and ANNs analysis. However, since the database is
generated before hydraulic fracturing operation, it is difficult to ensure
the accuracy under different geological and operational conditions.
Operation-stage with physics modelling based inverse analysis:
as shown in Fig. 3, during the hydraulic fracturing operation, fracturing
curves can be easily collected in field work, which reflect the property of
natural caves. At this stage, the modified gradient descent method based
inverse analysis is used to determine the geological parameters of
X. Tang et al.
Fig. 4. The geometry of the numerical model. (a) Seismic profiles of natural caves; (b) 3D computational model.
Table 1
The mechanical and hydraulic parameters for simulation.
Parameters Values
Elastic modulus, E (GPa) 20
Poisson’s, λ 0.25
Pore pressure of reservoir, Pr (MPa) 10
Permeability of reservoir, kr (m2) 3 × 10-18
Porosity of reservoir, ϕr 0.001
Permeability of hydraulic fracture, k (m2) 3 × 10-10
natural caves by data mining of fracturing curves, which can be used to
optimize the HF operation in real time. The initial value of unknown
parameters for inverse analysis is chosen according to the results of
ANNs achieved at preparation-stage. The forward problem of inverse
analysis is solved using AiFrac-TOUGH based physics modelling, and the
difference between the results of physics modelling and measured
fracturing curves is calculated. The unknown parameters are updated
using Eq. (8) for iteration. The inverse analysis method is carried out to
search for the exact solution until convergence. Because the initial value
of unknown parameters is chosen according to the results of ANNs and is
already close to the accurate solution, this ML-Physics method is able to
significantly reduce the iteration steps. While solving of forward prob­
lem based on physics modelling is extremely time-consuming, the pre­
sent ML-Physics method is able to reduce the computation cost at
operation-stage, meanwhile ensuring the accuracy.
3. Performance of Artificial Neural Networks (ANNs)
3.1. Numerical model of hydraulic fracturing with a natural cave
As shown in Fig. 4 (a), the geometry of natural caves can be accu­
rately detected using seismic tomography. A 50 × 50 × 1 m three-
dimensional (3D) numerical model with a natural cave and a hydrau­
lic fracture is established as Fig. 4 (b). The natural cave maximum height
and width are 18 m and 21 m, respectively. The initial length of the
hydraulic fracture is 2.4 m, and the initial distance between the natural
cave and fracture tip is 4.8 m. The stress applied in the x-direction and y-
direction are 25 MPa and 15 MPa, respectively. The fluid injection point
is located in the center of the model. The initial hydraulic fracture is set
with a constant flow rate of 0.09 kg/s. The matrix initial porous pressure
and the constant boundary pressure are set to 10 MPa. The porosity of
the natural cave ϕc = 0.5. In order to study the evolution patterns of
fracturing curves with different permeability of natural caves, a nu­
merical model is established. Total 81 groups of natural cave perme­
ability vary from kc = 1.0 × 10− 16 m2 to 9.0 × 10− 8 m2. The mechanical
and hydraulic parameters for simulation are shown in Table 1. The
hydraulic fracture initiates under the fluid flow and connects with the
5
Computers and Geotechnics 158 (2023) 105339
Table 2
Parameters used in training ANNs model.
Input data: Response parameters Output data: Geological parameters of natural
of fracturing curves caves
Total pressure drop The logarithm of natural cave permeability to
Pressure drop rate reservoir permeability (log (kc/kr))
Descending step
Rising rate
natural cave at the 18th step.
3.2. Establish database of ANNs
The training of ANNs requires a great number of sample data. In field
work, the monitoring data of fracturing curves is easily obtained.
However, it is difficult to obtain the propagation information of hy­
draulic fractures and natural caves in deep reservoirs. Simulating by
AiFrac-TOUGH, 81 groups of natural cave numerical models under
different permeability are obtained as shown in supplementary
information.
As shown in Table 2, the response features of fracturing curve
including four parameters are used as input data, and the geological
parameters of natural caves are used as output data. The logarithm of the
ratio that natural cave permeability to reservoir permeability (rkc = log
(kc/kr)) is defined to represent the influence of natural cave permeability
on fracturing curve. Four parameters related to the response features of
the fracturing curve are the pressure drop rate, the total pressure drop,
the descending steps and the rising rate of the fracturing curve.
3.3. The training and validation of ANNs
The database of neural network is divided into two sets, the training
set and the test set. There are 81 groups of data in the training set and 6
groups of data in the test set. Since the magnitude of parameters varies
greatly, it is necessary to normalize each parameter. The error back
propagation algorithm (BP algorithm) is used to adjust the learning
parameters (weight and bias) and iterate the whole process until the stop
condition is met. In the k-fold cross-validation method, mean absolute
percentage error (MAPE) is used to evaluate the validity of the hyper­
parameters, including the number of hidden layers and nodes in each
hidden layer. This method also can evaluate the validity of the activation
function and optimization algorithm of the neural network.
The first step is to determine the number of hidden layers and the
number of neurons in each hidden layer. In the process of model
training, the number of hidden layers and neuron nodes in each hidden
layer are changed to obtain the relationship between the error and the
X. Tang et al. Computers and Geotechnics 158 (2023) 105339

Fig. 5. The training error and cross-validation error under different number of hidden layers.

Fig. 6. The network model errors under different activation functions and optimization algorithms.

structure of the hidden layer. Fig. 5 shows the MAPE results with
different number of hidden layers and nodes of each hidden layer. When
the number of hidden layers is 3 and the number of nodes in each hidden
layer is greater than or equal to 67, the MAPE of training and cross-
validation data sets tends to be stable, which indicates the model ach­
ieves a good balance between the fitting ability and complexity. More­
over, the error of fitting results meets the requirements.
The second step is to determine the activation function and optimi­
zation function. In order to choose from three activation functions and
three optimization algorithms, MAPE of neural network models with
different activation function and optimization algorithm are used as the
evaluation criteria. Based on the result of the first step, the model con­
tains 67 neuron nodes in each of three hidden layers. To select the
activation function, select the Adam algorithm as the optimization al­
gorithm, MAPE of models with different activation functions are
compared for the evaluation of validity. As shown in Fig. 6 (a), MAPE
reaches the minimum when Purelin function is used, therefore Purelin
function is selected as the activation function. As shown in Fig. 6 (b), to

Table 3
The error of natural cave geological parameters obtained from fracturing curve.
Serial Response features from fracturing curve Actual geological Predicted geological Error
number parameters parameters

Total pressure drop Pressure drop Rate Rising rate (MPa/ Descending rkc rkc rkc
(MPa) (MPa/step) step) steps

0 15.60614 0.52020 0.00040 30 4.00000 3.95263 0.04737

1 16.04719 0.84459 0.00031 19 4.42600 4.38120 0.04480
2 16.57308 0.97489 0.00113 17 5.00000 5.01870 0.01870
3 16.73536 0.98443 0.00207 17 5.42600 5.42540 0.00060
4 16.68008 2.08501 0.00343 8 6.00000 5.97300 0.02700
5 16.87256 4.21814 0.00365 4 6.4260 6.50010 0.07410

6
X. Tang et al.
Table 4
The artificial neural network accuracy coefficients.
Coefficient Value
Mean absolute percentage error 0.03543
Maximum absolute percentage error 0.07410
Determination coefficient R2 0.99747
select the optimization algorithm, MAPE of models with different opti­
mization algorithm are calculated. When using Adam algorithm, MAPE
reaches the minimum, therefore Adam algorithm is selected as the
optimization algorithm.
In summary, after optimizing the hyperparameters, the final struc­
ture of the artificial neural network model is selected as: three hidden
layers and each of them contains 67 neuron nodes. The Purelin function
is chosen as the activation function and the Adam algorithm is chosen as
the optimization algorithm.
3.4. Predicting the geological parameters of natural caves by fracturing
curves
In field work, it is important to determine the geological parameters
of natural cave to optimize hydraulic fracturing. Based on the optimi­
zation results given above, geological parameters of natural cave can be
predicted using the neural network model proposed in previous section.
As shown in Table 3, the results of natural cave geological parameters
predicted by neural network are compared with the actual parameters
based on the response features of fracturing curve. It shows that the
prediction of the neural network model are in good agreement with the
actual geological parameters of the natural cave.
Table 4 shows the mean absolute percentage error, maximum ab­
solute percentage error and determination coefficient between the pre­
dicted value by the neural network model and the actual value. The
results show the MAPE of natural cave permeability is equal to 0.035.
The determination coefficients R2 is equal to 0.99, close to 1. The pre­
dicted permeability is 2.69 × 10-14 m2, and the error using this
Fig. 7. The pressure distribution before hydraulic fracture connecting with natural cave.
7
Computers and Geotechnics 158 (2023) 105339
traditional ML is up to 10.33%. These results show the artificial neural
network model can accurately predict the geological parameters of
natural caves according to the response features of the fracturing curves.
3.5. Effect of natural cave permeability on fracturing curve
As shown in Fig. 7, with the continuous injection of fluid, the hy­
draulic facture pressure increases continuously, and begins to propa­
gate. Before the hydraulic fracture connecting with the natural cave, the
pressure results with different permeability are almost the same. In the
process of hydraulic fracture connecting with natural cave, the pressure
results with different permeability are shown in Fig. 8. According to
simulation results, when hydraulic fracture connecting with natural
cave, the pressure results with different natural cave permeability are
significantly different (the 18th simulation step). When the permeability
of natural cave is high, its conductivity is high. After the hydraulic
fracture connecting with natural cave, the internal pressure of natural
cave spreads rapidly, resulting in a rapid pressure drop of fracturing
curve. To conclude, the pressure in hydraulic fracture decreases with the
increase of the permeability of natural cave.
As shown in Fig. 9, the fracturing curves related to natural caves
under different permeability can be obtained by monitoring the injec­
tion pressure of the wellbore. The fracturing curve of hydraulic fracture
connecting with natural cave can be divided into two stages: pre-
connection stage (stage I), and connection stage (stage II). For natural
cave with different permeability, the fracturing curve at stage I is
consistent. In the stage II, as the hydraulic fracture connects with the
natural cave, the fracturing curve drops sharply. With the continuous
injection of fluid, the fracturing curve rises slowly. For the fracturing
curve of the stage II, the permeability of natural cave has a significant
impact on the evolution of fracturing curves.
Fig. 10 shows the total pressure drop and the pressure drop rate of
the fracturing curve with different natural cave permeability in the stage
II. With the increase of natural cave permeability, the total pressure drop
of fracturing curve increases. This phenomena may because more fluid
injection is required to compensate for the diffusing fluid. When rkc ≥
X. Tang et al.
Fig. 8. The pressure distribution of the 18th simulation step under different natural cave permeability.
5.43, the total pressure drop trends to be stable. Meanwhile, with the
increase of natural cave permeability, the pressure drop rate of frac­
turing curve increases. This phenomenon may because the stronger the
conductivity is, the faster the whole cave can be filled, and the bigger the
pressure drop rates is. The black curve in Fig. 11 shows the relationship
between the descending steps of the fracturing curves and the perme­
ability of the natural cave. With the permeability of natural cave
increasing, the descending step of fracturing curves decreases. This
phenomenon may because the bigger the permeability is, the stronger
the conductivity is, and the fewer the steps is required when the internal
pressure drops to be stable. Meanwhile, with the increase of natural cave
permeability, the rising rate of fracturing curves increases slightly. This
may because the rising rate of fracturing curves is mainly related to the
permeability of rock, not the permeability of natural cave.
4. The performance of traditional inversion method
The initial value of the traditional inversion methods is set randomly
with a certain range. Then, the unknown parameters of model is
adjusted until the error between the inverse analysis results and the
actual value is small enough. Because the initial value of inverse analysis
is chosen randomly, the convergence speed is relatively low which leads
8
Computers and Geotechnics 158 (2023) 105339
to long computation time. In this section, with the traditional inversion
method, the actual permeability of natural cave (3 × 10-14 m2) is taken
as the target of inverse analysis. The difference between the actual
fracturing curve and the calculated fracturing curve is defined as the
error function. The errors of each iteration step are shown in Fig. 12.
Through 5 iteration steps, the error is decreased from 23.45% to 0.40%,
and the accurate natural cave permeability is obtained.
Fig. 13 shows the comparison between the fracturing curves calcu­
lated using inverse analysis method and the actual fracturing curve
under each iteration step. After five iteration steps, the two fracturing
curves almost coincide. The natural cave permeability finally obtained
with inverse analysis is 2.82 × 10-14 m2, which is in good agreement
with the actual value 3 × 10-14 m2. Therefore, this traditional inversion
method has adequate accuracy.
5. The performance of ML-physics method
Different from the traditional inverse analysis in Section 4, the initial
inversion value of ML-Physics method is chosen by the natural cave
permeability of 2.69 × 10-14 m2 achieved using artificial neural net­
works in Section 3.4. Then the inverse analysis method is used to
calculate more accurate permeability by iteration. After five iteration
X. Tang et al. Computers and Geotechnics 158 (2023) 105339

kc kr

kc kr

Fig. 11. Descending step and rising rate of fracturing curves with different
permeability in stage II.

Fig. 9. The fracturing curves of natural cave with different permeability.

Fig. 12. The error of fracturing curves using traditional inversion method.

accuracy. The evolution of permeability error is shown in Fig. 14 (b).

With traditional inverse analysis method, the error of permeability de­
creases from the 90.00% to 6.05%. Meanwhile, with the ML-Physics
method, the error decreases from the 10.33% to 0.29%.The computa­
tional time of the present method spent is 102 min when reached the
same target error (0.50%), which is much shorter than the computa­
tional time of the traditional method (more than 374 min). The results
show the ML-Physics method has higher accuracy and faster conver­
gence, compared to the traditional inverse analysis method. This is
particularly important for real-time analysis of field data.
kc kr
6. Conclusions
Fig. 10. Total pressure drop and the pressure drop rate of fracturing curves
with different permeability in stage II.
By inheriting the advantages of Machine Learning (ML) method and
physics modelling, a novel ML-Physics method is proposed. This method
steps, the permeability of natural cave of 2.99 × 10-14 m2 is achieved,
is developed to determine the geological parameters of natural caves
which is more accurate than the results obtained by the traditional in­
and optimize the operation of hydraulic fracturing in real time, based on
verse analysis method (2.82 × 10-14 m2). Meanwhile, the error of
the data mining of fracturing curves. Some useful conclusions draw
permeability using the traditional ML method is up to 10.33%, while the
below:
error of ML-Physics method is 0.29%, which shows that the ML-Physics
method is more accurate.
(1) The computational time of ML-Physics method is divided into two
Fig. 14 shows the comparison of errors between the traditional in­
stages, preparation-stage and operation-stage. The preparation-
verse analysis method and the ML-Physics method with different itera­
stage and operation-stage happen before and during HF pro­
tion steps. As shown in Fig. 14 (a), with traditional inverse analysis
cess, respectively. There is no limitation to the computational
method, the fracturing curve error decreases from the initial 23.45% to
time of preparation-stage, which is allowed to last for several
0.40% after 5 iteration steps. Meanwhile, with the ML-Physics method,
months. To determine the geological parameters of natural caves
the error decreases from 0.72% to 0.02% after the same iteration steps.
and optimize the operation of hydraulic fracturing in real time,
This shows the ML-Physics method can obtain higher inversion

9
X. Tang et al. Computers and Geotechnics 158 (2023) 105339

Fig. 13. The comparison between inversion fracturing curve and actual fracturing curve.

Fig. 14. The error with different inverse analysis methods.

the computational time of operation-stage should be extremely
short.
(2) In preparation-stage, the implicit relationship between natural
cave properties and fracturing curves is generated using the
Artificial Neural Networks (ANNs), which is a kind of machine
learning method. During operation-stage, the modified gradient
descent method is used for inverse analysis, in which the initial
value is chosen based on the ML results obtained on preparation-
stage. This method is able to ensure the computational speed and
accuracy during operation-stage, which is potentially useful to
optimize the HF design based on the data mining of fracturing
curves in real time. In this paper, when the fracturing curve error

10
X. Tang et al. Computers and Geotechnics 158 (2023) 105339

Table A1
Sample of neural network model.
Serial number Geological parameters of natural cave Response features of fracturing curve

rkc Total pressure drop (MPa) Pressure drop rate (MPa/step) Rising rate (MPa/step) Descending steps

1 1.523 6.64838391 2.21612797 0.01945317 3

2 1.824 6.26395659 2.08798553 0.03815930 3
3 2 6.38071882 1.59517971 0.02085370 4
4 2.125 6.99477581 0.49962684 0.02833875 14
5 2.222 6.66313010 0.66631301 0.01981265 10
6 2.301 6.98344968 0.58195414 0.01735759 12
7 2.368 7.25747140 0.48383143 0.01874018 15
8 2.426 8.09007696 0.44944872 0.01259431 18
9 2.477 8.34347977 0.39730856 0.01027412 21
10 2.523 7.83277832 0.39163892 0.00983806 20
11 2.824 9.89677491 0.30927422 0 32
12 3 11.27545461 0.35235796 0 32
13 3.125 11.48979036 0.35905595 0 32
14 3.222 12.12310608 0.37884707 0 32
15 3.301 12.67372441 0.39605389 0 32
16 3.368 13.11905347 0.40997042 0 32
17 3.426 13.51031458 0.42219733 0 32
18 3.477 13.82328635 0.43197770 0 32
19 3.523 14.06352778 0.43948524 0 32
20 3.824 15.21343331 0.47541979 0 32
21 4 15.60614076 0.52020469 0.00040186 30
22 4.125 15.62965861 0.60114072 0.00071560 26
23 4.222 15.79274724 0.65803114 0.00212365 24
24 4.301 15.90368627 0.72289483 0.00270596 22
25 4.368 15.98483617 0.79924181 0.00288147 20
26 4.426 16.04718989 0.84458894 0.00305532 19
27 4.477 16.09652339 0.94685432 0.00286417 17
28 4.523 16.13652985 0.94920764 0.00297925 17
29 4.824 16.33263327 1.25635641 0.00168335 13
30 5 16.57307569 0.97488681 0.00112658 17
31 5.125 16.46887392 0.91493744 0.00112484 18
32 5.222 16.66719216 0.92595512 0.00124358 18
33 5.301 16.69609121 0.87874164 0.00165089 19
34 5.368 16.71807318 0.928781843 0.00186870 18
35 5.426 16.73536171 0.98443304 0.00206812 17
36 5.477 16.74953664 1.04684604 0.00223648 16
37 5.523 16.76125319 1.11741688 0.00236944 15
38 5.824 16.81938109 1.68193811 0.00314929 10
39 6 16.68008369 2.08501046 0.00343783 8
40 6.125 16.85314499 2.40759214 0.00358119 7
41 6.222 16.69922257 2.78320376 0.00363260 6
42 6.301 16.86608057 3.37321611 0.00362802 5
43 6.368 16.86954964 3.37390993 0.00370864 5
44 6.426 16.87256252 4.21814063 0.00364921 4
45 6.477 16.87463643 4.21865911 0.00370628 4
46 6.523 16.87654430 4.21913609 0.00374285 4
47 6.824 16.72393853 8.36196927 0.00370127 2
48 7 16.72789517 8.36394759 0.00380293 2
49 7.125 16.72882743 8.36441372 0.00381903 2
50 7.222 16.73183101 8.36591551 0.00382347 2
51 7.301 16.73091885 16.73091885 0.00374906 1
52 7.368 16.89339853 16.89339853 0.00377459 1
53 7.426 16.73254131 16.73254131 0.00379400 1
54 7.477 16.73297267 16.73297267 0.00380468 1
55 7.523 16.73322444 16.73322444 0.00381173 1
56 7.824 16.73403141 16.73403141 0.00382915 1
57 8 16.73417843 16.73417843 0.00383111 1
58 8.125 16.83221806 16.83221806 0.00383092 1
59 8.222 16.73405570 16.73405570 0.00383201 1
60 8.301 16.73431394 16.73431394 0.00383215 1
61 8.368 16.90343421 16.90343421 0.00383248 1
62 8.426 16.73732768 16.73732768 0.00383243 1
63 8.477 16.73411711 16.73411711 0.00383244 1
64 8.523 16.90235332 16.90235332 0.00383271 1
65 8.824 16.74172588 16.74172588 0.00383281 1
66 9 16.74324393 16.74324393 0.00383289 1
67 9.125 16.73955502 16.73955502 0.00383290 1
68 9.222 16.74250484 16.74250484 0.00383295 1
69 9.301 16.74467368 16.74467368 0.00383297 1
70 9.368 16.62992135 16.62992135 0.00383277 1
71 9.426 16.74471980 16.74471980 0.00383295 1
72 9.477 16.74755272 16.74755272 0.00383304 1
73 9.523 16.79513338 16.79513338 0.00383196 1
(continued on next page)

11
X. Tang et al. Computers and Geotechnics 158 (2023) 105339

Table A1 (continued )
Serial number Geological parameters of natural cave Response features of fracturing curve

rkc Total pressure drop (MPa) Pressure drop rate (MPa/step) Rising rate (MPa/step) Descending steps

74 9.824 16.90995175 16.90995175 0.00383344 1

75 10 16.91033178 16.91033178 0.00383237 1
76 10.125 16.74571836 16.74571836 0.00383301 1
77 10.222 16.75135658 16.75135658 0.00383349 1
78 10.301 16.67885501 16.67885501 0.00383183 1
79 10.368 16.84319051 16.84319051 0.00383259 1
80 10.426 16.79580585 16.79580585 0.00383243 1
81 10.477 16.90838725 16.90838725 0.00383103 1

convergence criterion is 0.50%, iteration step of the present
method spent during operation-stage (one iteration step) is much
less than the iteration step of the traditional method (five itera­
tion steps). The computational time of the ML-Physics method
spent (102 min) is much less than the computational time of the
traditional inverse analysis method (more than 374 min). After
the same 5 iteration steps, the fracturing curve error with the
traditional inverse analysis method is 0.40% during operation-
stage, while the error of ML-Physics method is 0.02%. Mean­
while, the error of permeability using the traditional ML is up to
10.33%, while the error of ML-Physics method is 0.29%, which
shows that the ML-Physics method is more accurate.
Declaration of Competing Interest
The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influence
the work reported in this paper.
Data availability
Data will be made available on request.
Acknowledgements
This work was supported by the National Natural Science Foundation
of China No.12172264 and Basic Research Program of Central Gov­
ernment Funds for Shenzhen Science and Technology Development
(2021Szvup103).
Appendix
See Table A1.
References
Ali, M.T., Ezzat, A.A., Nasr-EI-Din, H.H., 2019. A model to simulate matrix-acid
stimulation for wells in dolomite reservoirs with vugs and natural fractures. SPE J.
25 (02), 609–631. https://doi.org/10.2118/199341-PA.
An, M.K., Zhang, F.S., Dontsov, E., Elsworth, D., Zhu, H.H., Zhao, L.X., 2021. Stress
perturbation caused by multistage hydraulic fracturing: Implications for deep fault
reactivation. Int. J. Rock Mech. Min. Sci. 141, 104704 https://doi.org/10.1016/j.
ijrmms.2021.104704.
Bian, X.B., Jiang, T.X., Jia, C.G., Wang, H.T., Li, S.M., Su, Y., Wei, R., 2016. A new post-
frac evaluation method for shale gas wells based on fracturing curves. Nat. Gas Ind.
B. 3 (2), 146–151. https://doi.org/10.1016/j.ngib.2016.03.010.
Chalikakis, K., Plagnes, V., Guerin, R., Valois, R., Bosch, F.P., 2011. Contribution of
geophysical methods to karst-system exploration: an overview. Hydrogeol. J. 19 (6),
1169–1180. https://doi.org/10.1007/s10040-011-0746-x.
Chang, Z.H., Zhang, Y., Chen, W.B., 2019. Electricity price prediction based on hybrid
model of adam optimized LSTM neural network and wavelet transform. Energy 187,
115804. https://doi.org/10.1007/s10040-011-0746-x.
Chen, Y., Liu, D.M., Cai, Y.D., Yao, J.J., 2020. Fracturing curve and its corresponding gas
productivity of coalbed methane wells in the Zhengzhuang block, southern Qinshui
Basin, North China. Energy Explor. Exploit. 38 (5), 1387–1408. https://doi.org/
10.1177/0144598720923820.
Du, X., Li, Q., Li, P.C., Xian, Y.X., Zheng, Y., Lu, D.T., 2022. A novel pressure and rate
transient analysis model for fracture-caved carbonate reservoirs. J. Pet. Sci. Eng.
208, 109609 https://doi.org/10.1016/j.petrol.2021.109609.
Gao, B., Huang, Z.Q., Yao, J., Lv, X.R., Wu, Y.S., 2016. Pressure transient analysis of a
well penetrating a filled cavity in naturally fractured carbonate reservoirs. J. Pet. Sci.
Eng. 145, 392–403. https://doi.org/10.1016/j.petrol.2016.05.037.
Li, B.Q., Gonçalves, B., Einstein, H., 2019. Laboratory hydraulic fracturing of granite:
acoustic emission observations and interpretation. Eng. Fract. Mech. 209, 200–220.
https://doi.org/10.1016/j.engfracmech.2019.01.034.
Li, Y.Q., Hou, J.G., Ma, X.Q., 2016. Data integration in characterizing a fracture cavity
reservoir, Tahe oilfield, Tarim basin. China. Arab. J. Geosci. 9 (8), 1–12. https://doi.
org/10.1007/s12517-016-2562-z.
Liu, Z.Y., Chen, M., Zhang, G.Q., 2014. Analysis of the influence of a natural fracture
network on hydraulic fracture propagation in carbonate formations. Rock Mech.
Rock Eng. 47, 575–587. https://doi.org/10.1007/s00603-013-0414-7.
Liu, Z.Y., Tang, X.H., Tao, S.J., Zhang, G.Q., Chen, M., 2020. Mechanism of connecting
natural caves and wells through hydraulic fracturing in fracture-cavity reservoirs.
Rock Mech. Rock Eng. 53, 5511–5530. https://doi.org/10.1007/s00603-020-02225-
w.
Liu, Z.Y., Zhang, M.Z., Sun, L., Ye, S., Chen, Z.T., Tang, X.H., 2022. The influence of
natural joints on the evolution of fracturing curves: from a numerical perspective.
Arabian J. Sci. Eng. 1–14 https://doi.org/10.1007/s13369-022-07097-6.
Ma, G.W., Chen, Y., Jin, Y., Wang, H.D., 2018. Modelling temperature-influenced
acidizing process in fractured carbonate rocks. Int. J. Rock Mech. Min. Sci. 105,
73–84. https://doi.org/10.1016/j.ijrmms.2018.03.019.
Ni, X.M., Zhao, Z., Liu, X., Liu, D., Li, Z., 2020. Classes of fracturing curves and their
formation mechanism under different combinations of coal structures and geostress.
Energy Sour. Part A Recov. Util. Environ. Effects. 1–18. doi: 10.1080/15567036.
2020.1810176.
Qiao, J.M., Tang, X.H., Hu, M.S., Rutqvist, J., Liu, Z.Y., 2022. The hydraulic fracturing
with multiple influencing factors in carbonate fracture-cavity reservoirs. Comput.
Geotech. 147, 104773 https://doi.org/10.1016/j.compgeo.2022.104773.
Tang, X.H., Rutqvist, J., Hu, M.S., Rayudu, N.M., 2019. Modeling three-dimensional
fluid-driven propagation of multiple fractures using TOUGH-FEMM. Rock Mech.
Rock Eng. 52 (2), 611–627. https://doi.org/10.1007/s00603-018-1715-7.
Wang, S.Y., Si, G.Y., Wang, C.B., Cai, W., Li, B.L., Oh, J., Canbulat, I., 2022. Quantitative
assessment of the spatio-temporal correlations of seismic events induced by longwall
coal mining. J. Rock Mech. Geotech. Eng. 14 (5), 1406–1420. https://doi.org/
10.1016/j.jrmge. 2022.04.002.
Xu, C., Di, B.R., Wei, J.X., 2016. A physical modeling study of seismic features of karst
cave reservoirs in the Tarim Basin China. Geophysics. 81(1), B31–B41. <https://doi.
org/10.1190/geo2014-0548.1>.
Yan, C.Z., Zheng, H., 2017. FDEM-flow3D: a 3D hydro-mechanical coupled model
considering the pore seepage of a rock matrix for simulating three-dimensional
hydraulic fracturing. Comput. Geotech. 81, 212–228. https://doi.org/10.1016/j.
compgeo.2016.08. 014.
Zang, A., Stephansson, O, 2019. Special Issue Hydraulic Fracturing in Hard Rock. Rock
Mech. Rock Eng. 52, 471-473. doi: 10.1007/s00603-019-1740-1.
Zhang, M.Z., Liu, Z.Y., Jiang, Q., He, B.G., 2021. Influence of natural caves on hydraulic
fracturing fracturing curves: numerical modelling and ANNs. Arabian J. Geosci. 14
(20), 1–22. https://doi.org/10.1007/s12517-021-08437-w.
Zhao, Q., Lisjak, A., Mahabadi, O., Liu, Q.Y., Grasselli, G., 2014. Numerical simulation of
hydraulic fracturing and associated microseismicity using finite-discrete element
method. J. Rock Mech. Geotech. Eng. 6 (006), 574–581. https://doi.org/10.1016/j.
jrmge. 2014.10.003.
Zhao, Y., Zhang, Y.F., Yang, H.Q., Liu, Q., Tian, G.D., 2022. Experimental study on
relationship between fracture propagation and pumping parameters under constant
pressure injection conditions. Fuel. 307, 121789 https://doi.org/10.1016/j.fuel.
2021.121789.
Zhou, J., Jin, Y., Chen, M., 2010. Experimental investigation of hydraulic fracturing in
random naturally fractured blocks. Int. J. Rock Mech. Min. Sci. 47(7), 1193–1199.
<https://doi.org/10.1016/j.ijrmms.2010.07.005>.
Zhou, L.B., Guo, A.B., Wang, X.G., Qiao, J.M., Tang, X.H., 2022. The effect of
temperature, natural fractures and vugs on the acidizing process in fractured-vuggy
reservoirs with hydro-thermal-chemical coupled modeling. J. Pet. Sci. Eng. 213,
110416 https://doi.org/10.1016/j.petrol.2022.110416.
Zhu, H.Y., Deng, J.G., Jin, X.C., Hu, L.B., Luo, B., 2015. Hydraulic fracture initiation and
propagation from wellbore with oriented perforation. Rock Mech. Rock Eng. 45,
585–601. https://doi.org/10.1007/s00603-014-0608-7.

12