Engineering Applications of Artificial Intelligence 24 (2011) 813–821

Contents lists available at ScienceDirect

Engineering Applications of Artificial Intelligence

journal homepage: www.elsevier.com/locate/engappai

Modelling load–settlement behaviour of piles using high-order neural

network (HON-PILE model)
A. Ismail b, D-S. Jeng a,b,n
a
Center for Marine Geotechnical Engineering, Department of Civil Engineering, Shanghai Jiao Tong University, Shanghai 210040, China
b
Division of Civil Engineering, University of Dundee, DD1 4HN, UK

a r t i c l e i n f o abstract

Article history: An accurate estimation of pile response to loading is a challenging task due to the complexity of the
Received 30 August 2010 soil–pile interactions and uncertainties in the soil properties. Conventional methods of predicting pile
Received in revised form load–settlement relationship either oversimplify the problem or require the parameters that are
5 February 2011
difficult to determine in the laboratory. In this study, a high-order neural network (HON) is developed
Accepted 8 February 2011
to simulate the pile load–settlement curve using properties of the pile and SPT data along the depth of
Available online 7 April 2011
pile embedment as inputs. The results indicated a significant improvement in the quality of HON
Keywords: predictions over that of BPN, RBF and GRNN models. Based on the comparisons with the predictions of
Artificial neural network elastic and hyperbolic models, the proposed HON model provides better predictions than existing
High-order neural network
theoretical models.
Soil–pile interaction
& 2011 Elsevier Ltd. All rights reserved.
SPT
Load–settlement behaviour

1. Introduction analysis. However, the assumption of linear deformation beha-

Piles have been widely used as foundation for various impor-
tant infrastructures such as tall buildings, bridges, wind farms,
etc. One of major functions of pile foundation is to transmit
loading from the superstructure to the subsurface through shaft
friction and end bearing. As a serviceability requirement, the pile
head settlement shall not exceed the design level. Therefore, a
reliable estimation of pile head settlement is essential for the
design of pile foundations. The problems posing a considerable
challenge to accurate prediction of load–settlement behaviour of
piles are the complicated nature of load transfer mechanism,
involving the inter-related displacement of pile and surrounding
soil, the uncertainty with regard to soil properties due to sample
disturbance during material testing and the inevitable modifica-
tion of soil properties around the pile as a result of installation.
This constitutes a serious handicap particularly to more rigorous
computational models like non-linear finite element analysis.
Poulos (1977) observed that the suitably accurate information
about the soil and soil–pile interface for FEM analysis may be
difficult to determine due to sampling and pile installation related
problems. Relatively simple methods such as elastic solutions and
hyperbolic models can rely on correlations with in-situ tests for
estimating the relevant soil parameters used in settlement
n
Corresponding author at: Division of Civil Engineering, University of Dundee,
DD1 4HN, UK.
E-mail address: d.jeng@dundee.ac.uk (DS. Jeng).
viour of soil in the case of elastic method is overly simplistic
(Desai, 1977) and is limited to very low strain levels where the
soil can be safely regarded as such (Tomlinson, 1995). In the case
of hyperbolic models, though in agreement with general con-
sensus that soil is a non-linear geo-material, the constants of a
single hyperbolic equation are too few to adequately represent
the complex behaviour of load transfer along the pile shaft for
various types of soil (Poskitt, 1993).
Pile loading tests have been the most reliable means of
assessing not only the serviceability but also the stability of pile
foundations. However, the cost makes it infeasible to be carried
out at every design stage apart from very important construc-
tions. Development of a reliable correlation between pile loading
test and in-situ soil tests such as SPT will be of utmost benefit to
practicing engineers, because it will provide them with a model
based on simple tests and serving dual purpose of stability and
serviceability analysis.
Unlike the conventional methods such as elastic approxima-
tion (Vesic, 1977; Vijayvergiya, 1977; Poulos and Davis, 1980;
Randolph and Wroth, 1978; Fleming et al., 1992), hyperbolic
models (Chin, 1970; Fleming, 1992) and FE analysis (Desai, 1974),
neural networks (NN) are non-mechanistic models (Jaksa et al.,
2008) which are solely based on the observed experimental data.
Their ability to acquire knowledge through interaction (learning),
makes them quite suitable for exploring correlations from such a
very noisy and imprecise information as in-situ soil test and pile
loading test data. Artificial neural networks (ANNs) have been

0952-1976/$ - see front matter & 2011 Elsevier Ltd. All rights reserved.
doi:10.1016/j.engappai.2011.02.008
814 A. Ismail, D.-S. Jeng / Engineering Applications of Artificial Intelligence 24 (2011) 813–821
successfully applied to a wide variety of geotechnical problems
such as geo-material modelling (Ellis et al., 1995; Penumadu and
Jean-Lou, 1997; Basheer and Najjar, 1998), soil liquefaction
problems (Agrawal et al., 1997; Ural and Saka, 1998), prediction
of pile capacity (Goh, 1995, 1996; Chan et al., 1995; Teh et al.,
1997; Lee and Lee, 1996; Nawari et al., 1999; Das and Basudhar,
2006). Shahin et al. (2002) developed a BPN model for predicting
shallow foundation settlement using SPT blow counts as repre-
sentative of soil property in the input space. Recently, Nejad et al.
(2009) used a BPN network to correlate between pile head
settlement and SPT blow counts of soil around the pile.
The successes of the aforementioned investigations strengthen
the position of neural networks as alternative numerical methods of
solving intrinsically non-linear geotechnical problems such as soil–
pile interaction. Among the various types of neural networks, back-
propagation network (BPN) is the most widely used model for
geotechnical engineering problems. This is due to the simplicity of
learning rule used in training the network and the stability of
sigmoid transfer function, which is used as activation function.
Despite their popularity, BPN networks are not devoid of problems;
the sigmoid transfer function is a bi-asymptotic one and becomes
insensitive to the variation of inputs as it approaches either 1 or 0.
This severely limits the ability of the network to make reasonable
extrapolations. One of the solutions to the said problem is to use
non-asymptotic processing elements such as high-order neurons.
High-order neural networks use polynomial functions to map inputs
into output and can be trained through error back-propagation
algorithm. They have successfully been applied in pattern recogni-
tion (Foresti and Dolso, 2004; Artyomov and Yadid-Pecht, 2005) and
function approximation (Rovithakis et al., 2004), but have not yet
been applied in geotechnical modelling.
In this study, HON-PILE, a high-order neural network model
for the simulation of load–settlement behaviour is developed. The
model predicts the relationship between the pile head settlement
and the magnitude of axial loading exerted on the pile given the
geometric parameters and elastic modulus of pile and the proper-
ties of foundation soil, represented by the standard penetration
test blow counts (SPT-N). A comprehensive comparison between
the proposed HON model, BPN and the elastic approximation and
hyperbolic model will be performed. The relative performance of
HON will also be determined in comparison with other poly-
nomial regression, RBF and GRNN networks.
2. Existing pile load–settlement models
In this section, the currently used methods of predicting pile
settlement, including elastic hyperbolic, step integration and FEM
methods will be outlined.
2.1. Elastic models
Elastic models are based on Mindlin equations for the dis-
placement due a point load on a linearly elastic, semi-infinite soil
layer. They are also based on the assumption that the pile is acted
upon by a system of uniform shear stress around its shaft, while
the base is acted upon by a uniform pressure. Elastic solutions are
severely limited to circumstances where the simple linear soil
behaviour can be considered adequate. The most popular elastic
solution was developed by Poulos and Davis (1980) and can be
expressed as follows:
P
s¼ I ð1Þ
Es Db
where I¼IoRkRhRvRb: Io is the displacement influence factor for a
rigid pile in a deep layer, for a Poisson’s ratio of 0.5; Rk the
correction factor for pile compressibility; Rh the depth correction
factor; Rv the correction for Poisson’s ratio; Rb the correction for
end bearing; s is the immediate settlement; Es the elastic modulus
of soil along the pile shaft and Db stands for the diameter of the
pile base.
Other methods based on elastic solution were developed
by Vesic (1977), Randolph and Wroth (1978) and Fleming et al.
(1992). Elastic solutions are severely limited to circumstances
where the simple linear soil behaviour can be considered
adequate.
2.2. Hyperbolic models
In this approach, the pile head settlement is related to the pile
head load using a hyperbolic model. It is an attempt to represent
the non-linear behaviour of soil-shaft and soil-base interaction
using relatively simple model. The method was first described
by Chin (1970) and later improved by Fleming (1992) to treat the
components of shaft and base resistances separately. According
to Fleming (1992), the pile load and the pile head settlement are
related as follows:
aDt bDt
Pt ¼ þ ð2Þ
c þ Dt d þ eDt
in which Pt and Dt represent the pile load and pile head settle-
ments, respectively, while a ¼Us, b¼DBEBUB, c¼MsDB, d ¼0.6UB
and e¼ DBEB. Us, UB, DB, EB and Ms, are the ultimate shaft
resistance, ultimate base resistance, pile base diameter, soil
deformation soil modulus at the pile base and flexibility factor,
respectively. The method tends to yield better results than elastic
methods. However, the assumption of hyperbolic stress–strain
relationship is not applicable to all circumstances. Furthermore,
there is no established correlation between the hyperbolic para-
meters and in-situ (SPT) tests, and one has to rely on correlations
between SPT and soil parameters, which are unreliable.
2.3. Step integration methods
Step integration approach was developed by Coyle and Reese
(1966). In this approach, shear-deformation relation for the soil–
pile interface is modelled using series of independent non-linear
curves known as t–z curves. The base pressure versus deformation
is represented by another non-linear model known as q–y curve.
The soil–pile system is simulated in this method as series of
incompressible weights connected by weightless springs to
represent the pile and side springs attached to the weights to
represent soil resistance described by t–z curves. At the bottom of
the bottom most weight, another spring is attached to represent
base resistance described by q–y curve. The problem is then
solved either using finite difference or one-dimensional finite
element method. The major challenge in step integration method
is the determination of parameters defining t–z and q–y curves.
Various attempts have been made to develop an empirical
correlation between the curve parameters and in-situ tests
(Chang and Broms, 1991, Tan et al., 1998; Balakrishnan et al,
1999). However, high scatter undermines the reliability of the
correlations. Their inability to develop a good correlation is not
unconnected to the data size, the noise associated with the data
and, possibly due to limitations of the regression models used.
2.4. Finite element method
Finite element method is the most rigorous technique used in
the prediction of pile head settlement. Although more computa-
tionally intensive, than other methods discussed earlier, it is more
 A. Ismail, D.-S. Jeng / Engineering Applications of Artificial Intelligence 24 (2011) 813–821 815

versatile as it can handle complex geometries such as spatial x1

variation of soil continuum. It also can conveniently handle non- p1,j
linear material behaviour. The analysis involves discrediting both
the pile and the supporting soil into finite elements. In compli- x2 p2,j
ance with equilibrium, compatibility and boundary conditions,
Oj
displacements can be determined by solving a system of equa-
tions derived from the geometry and material behaviour of finite
elements. The accuracy and reliability of FEM model depends on pn,j
the selection of material constitutive model to describe the Node j
behaviour of the soil elements and the soil–pile interface. Simple xn
models such linearly elastic or elastic–plastic models, which
Fig. 1. High-order neuron.
require little detail, could lead to a gross prediction error, while
complicated more accurate models such as require many para-
meters that are difficult to determine (Scott, 1988). Table 1
Even more difficult to predict is the behaviour of the soil–pile Sources of database.
interface. Lee and Salgado (1999), in the absence of simple and
Source Location of the test Number of tests
reliable means of measuring the properties of the soil–pile inter-
face, used linear model to represent pre-failure regime and Federal highway administration, USA 78
coulomb failure model to account for slip at the interface and US Department of transportation
assumed the friction and adhesion factors of unity. The uncer- Trevi foundation nig. Ltd. Nigeria 2
Federal Ministry of works Nigeria Nigeria 4
tainty with regards to parameter estimation and the effects of pile
Julius Berger Nigeria limited Nigeria 9
installation, which are difficult to take into consideration in FE Balakrishnan et al. (1999) Kuala Lumpur, Malaysia 5
analysis, tend to undermine performance of the powerful method
in comparison with less rigorous methods.

Unlike summation units where the net input signal is a weighted

3. Neural network model (HON-PILE model)
Neural networks are computational models capable of carrying
out complex input–output mapping. They are based on the
information processing infrastructure of the biological brain.
The structure of ANN is a highly simplified model of biological
neural architecture. The computational simplicity of the proces-
sing elements makes the network much easier to handle than
rigorous mechanistic computational methods. The ability of
neural networks to perform accurate modelling of complex
systems lies in the interconnection between various neurons in
the network. The model of artificial neuron was first proposed
by McCulloch and Pitts (1943). However, artificial neural net-
works did not attract lots of attention until the development of
back-propagation algorithm for artificial neurons (Rumelhart
et al., 1986). Since then, a significant amount of research has
been done on their improvement and their application to various
aspects across many disciplines. The most widely used neural
network is the back-propagation neural network (BPN), which
uses sigmoid function in processing input signals and back-
propagation algorithm in prediction error correction. BPN has a
superior function approximation capability than radial basis
function network (RBN), which has a superior capability in
pattern classification (Haykin, 1999). The performance of BPN
was found to be comparable to more complicated neuro-fuzzy
network (ANFIS) as reported in the literature (Shahin et al., 2003;
Bateni et al., 2007). However, the downside of BPN is the limited
dynamic range of sigmoid function, which, at times, makes it
necessary to use large number of neurons to achieve the desired
accuracy. High-order networks are alternative neural network
types with a trainable activation function and a capability of
achieving desired accuracy with a smaller network size (Ismail
and Engelbrecht, 2000). In this study, the capability of high-order
neural networks in simulating the load–settlement behaviour of
pile is investigated.
3.1. High-order networks
High-order neural network uses high-order neurons instead of
summation units like sigmoid neurons as processing elements.
sum of the inputs fed into that unit, high-order units process the
products of the inputs they receive. HON provides an alternative
to conventional networks, with the advantages of increased
information capacity and reduced number of processing elements
ability to form higher-order combinations of inputs (Durbin and
Rumelhart, 1989). The type of high-order neuron used here is
based on the HONEST model proposed by Abdelbar and Tagliarini
(1996) with slight modification.
The mathematical model of the neuron is:
iY
¼n
p
Oj ¼ wj xi i,j ð3Þ
i¼1
where Oj, wj and pi,j are the output and nodal weight of neuron j
(Fig. 1), and the synaptic weight of i–j link, respectively. xi is the
input from node i.
The advantage of high-order neurons over sigmoid neurons is
that the input and output parameters do not have to be normal-
ized within certain range. This is due to the fact that HON models
are not asymptotic and do not have a limited dynamic range.
Another advantage of HON is transparency. While other types of
network are criticized of operating on the principles of black box,
the strength of relationship between various inputs and the
output in HON is easier to appreciate. Due to its polynomial
nature, HON bears some similarities with group method of data
handling (GMDH) models, which, in theory, can approximate any
continuous function (Ivakhnenko, 1971). GMDH algorithm
involves determining the coefficients of the Kolmogorov–Gabor
polynomial expressed by
X
n XX XXX
yn ¼ ao þ ai x i þ aij xi xj þ aijk xi xj xk þ    ð4Þ
i¼1
where {xi, xj, xk, y} is the vector of inputs variables and {ao, ai, aij,
aijk, y} is the vector of coefficients. The advantage of HON
network over GMDH models is that HON can assume real valued
exponents and is capable of undergoing a gradual training like
BPN and RBF networks. In this study, the HON network is trained
using gradient decent algorithm.
816 A. Ismail, D.-S. Jeng / Engineering Applications of Artificial Intelligence 24 (2011) 813–821

Table 2 and that of the pile. The second and third hidden layers consist of
Input and output parameters for various models. high-order networks with three neurons in each layer, while the
output layer consists of single linear neuron. Alongside the type I
Model Input parameters Output
HON model, a BPN with 4-8-3-1 configuration was developed for
BPN Ep Ab Pile head settlement the sake of comparison. Type II HON model consists of 14 input
P, ,N s Ai ,Nb Ab
L nodes, six hidden nodes in the first hidden layer and one output
HON (Type I) Ep Ab Pile head settlement
P, ,N s Ai ,Nb Ab layer (14-6-1) at the initial stage. Second hidden layer is then
L
HON (Type II) Ep Ab h  i Pile load introduced in the course of training and the number of nodes
s, , N s ,Nb Ab , % sand,
L thereof is gradually increased from 0 to 5 the final configuration
% silt, % clay, pile Type
at the end of training session is 14-6-5-1. RBF (with 15 Gaussian
nodes) and GRNN networks are trained together with HON type II
model for assessment of relative performance.
Table 3 The database used for training and testing the models consists
Summary of training and testing results (BPN and HON (Type I) models). of results of static load tests carried out on 60 driven piles and 38
bored piles with over 900 data points. The pile data is drawn from
Network type R2-value RMSE (mm)
the sources given in Table 1. The detailed information of database
HON Type I (training) 0.91 6.7 are provided in a spread sheet that is available from the authors
HON Type I (testing) 0.88 7.1 and journal, upon request. All piles in the database are concrete
BPN (training) 0.87 6.5 cylindrical piles. The methods of installation include impact
BPN (testing) 0.78 8.9
driving in the case of driven piles and wet and dry excavations
in the case of bored piles. The range of pile diameters and lengths
are from 0.3 to 0.8 and 6 to 30 m, respectively. The elastic moduli
Table 4 of the piles range from 2.19  107 to 3.44  107 kN/m2. The soil
Summary of training and testing results (polynomial regression, RBF, GRNN and test results in the database consist of soil classification and
HON (Type II) models).
uncorrected SPT-N values determined at 1.5–3 m intervals and
Model R2-value RMSE (kN) soil classification. The rationale behind the use of uncorrected
values in this study is that they reflect the variation of over-
HON Type II (training) 0.91 554.4 burden stress, which in turn affects shaft and base resistance in
HON type II (testing) 0.89 607.6 coarse soils. The soil classification serves as additional informa-
Non-linear regression (training) 0.756 896.05
Non-linear regression (testing) 0.659 1090.04
tion for a better characterisation of the foundation soil. The soil
RBF network (training) 0.769 753.41 types in the database range from coarse soils (sand) to fine soils
RBF network (testing) 0.828 738.2 (silt and clay), with coarse soil being predominant. BPN and HON
GRNN model (training) 0.851 712.15 type I model were developed based on driven piles only, while
GRNN model (testing) 0.676 1124.28
type II HON was trained to predict the behaviour of both driven
Hyperbolic 0.562 1614.14
and bored piles.

3.2.1. Input variables

3.2. Development of HON-PILE
Development of neural networks involves not only optimizing
the training but also the architecture. The most challenging aspect
of topology development is the control of the size of the network.
Too many neurons will lead to redundancy of some connections
slow training process and lack of generalization. On the other
hand, using insufficient neurons will lead to inability of the
network to extract the necessary information from the network.
There are various methods of developing best architecture
(Haykin, 1999). In this study, bottom-to-top approach is used,
based on incremental learning algorithm proposed by Bahi et al.
(2009). Starting with a small network, the size is gradually
increased as the training progresses. When a hidden layer is
added it starts with a single linear node. High-order nodes are
then monotonically added to the layer as the training progresses
until the convergence is obtained.
Based on the number of inputs and predicted parameters, two
types of high-order networks were developed. Type I model
predicts the pile head settlement, while type II model predicts
the pile head load. The initial topology of type I model was
initialized with four input nodes, one hidden layer with four
nodes and an output layer with one node (4-4-1). In the course of
training, the two hidden layers, with three neurons in each, are
added into the network and the resulting configuration becomes
4-4-3-3-1. At each increment, the network is trained until no
further improvement in error reduction. The first hidden layer of
the optimized network consists of linear neurons for the purpose
of summing up the stifnessses of soil along the shaft and the base,
The ability of ANN to make a reasonable prediction depends
largely on the choice of input parameters. A good understanding
of the factors that affect the prediction of pile head settlement is
therefore crucial to development of a reliable network. Based on
traditional methods of predicting pile settlement such as Poulos
and Davis (1980) and Vesic (1977), the parameters the controlling
load–settlement behaviour of piles are the stiffness of the pile and
its geometry, and the stiffness of the soil of the of the soil around
the shaft and at the pile base. The pile stiffness is a function of
pile geometry and material property of the pile, while the soil
resistance is a function of soil deformation modulus, which is
estimated from in-situ or laboratory tests. In this study, SPT-N is
chosen to represent the soil stiffness. SPT is the most widely used
in-situ test due to its lack of sophistication and low cost. Although
less accurate compared to other in-situ tests, it gives a good idea
about strength ad deformability of geo-materials (Prevost and
Popescu, 1996). A number of empirical correlations between
deformation modulus of soil and SPT-N values are reported in
the literature (i.e. Poulos and Davis, 1980). The pile head settle-
ment as a non-linear function of related variables can be written
as
s ¼ f ðP,ks ,Ep ,D,LÞ ð5Þ
where P, ks Ep, D and L are pile load, soil stiffness, pile modulus,
pile diameter and pile length, respectively. The unknown function
f( ) relates the pile and pre-installation soil parameters to the post
installation response of the soil–pile system to loading. The load–
settlement curve can also be modelled with pile load as the
dependant parameter. Eq. (12) below expresses pile head load as
 A. Ismail, D.-S. Jeng / Engineering Applications of Artificial Intelligence 24 (2011) 813–821 817

180 120

160 R2 = 0.780
R2 = 0.870 100
140

Predicted settlement (mm)

Predicted settlemet (mm)

120 80

100
60
80

60 40

40
20
20

0 0
0 20 40 60 80 100 120 140 160 180 0 20 40 60 80 100 120
Measured settlement (mm) Measured settlement (mm)

180 120

160
R2 = 0.87
100
R2 = 0.910
140
Predicted settlement (mm)
Predicted settlement (mm)

120 80

100
60
80

60 40

40
20
20

0 0
0 20 40 60 80 100 120 140 160 180 0 20 40 60 80 100 120
Measured settlement (mm) Measured settlement (mm)

Fig. 2. Comparison of measured versus predicted settlements: (a) BPN versus training data; (b) BPN versus testing data; (c) HON type I versus training data; and (d) HON
type I versus testing data.

a function of settlement, and soil and pile properties: Other factors considered are the soil type and method of
installation. The settlement function in Eq. (11) is approximated
P ¼ f 0 ðs,ks ,Ep ,D,LÞ ð6Þ
using BPN and HON type I as
0
The function f ( ) is the inverse of the settlement function. The  
Ep Ab
soil stiffness (ks) in Eqs. (11) and (12) is a function of the soil sNN ¼ NN P,Ns As ,Nb Ab , ð9Þ
L
resistance around the pile, which can be expressed in terms of SPT
as follows: The NN approximated load function by HON type II is
expressed by
k ¼ f ðNs ,Nb Þ ð7Þ
 h i 
 Ep Ab
in which Ns and Nb are the SPT-N along the shaft and at the base, PNN ¼ NN s, Ns ,Nb Ab , ,styp,ptyp ð10Þ
L
respectively.
To account for variation of N-value along the shaft, the pile is where Ns is the average value of N over the shaft length, denoted
divided into five segments. The weighted average value of N over by
the length of each segment is determined by
Pn Pi ¼ 5
Nj l j Ns,i i¼1
Ns,i ¼ 1 ð8Þ Ns ¼ ð11Þ
li 5

The term EpAb/L stands for the pile stiffness. ½Ns  is the vector
where n is the number of sub-layers along the shaft of pile segment. of the product of average N-values along the shaft segments and
lj is the thickness of sub-layer j. li is the thickness of pile segment. the corresponding shaft area, respectively. It is expressed as
818 A. Ismail, D.-S. Jeng / Engineering Applications of Artificial Intelligence 24 (2011) 813–821

12000 10000

10000 R2 = 0.910
8000

Predicted pile load (kN)

Predicted load (kN)

8000
6000
6000

4000
4000

2000
2000

0 0
0 2000 4000 6000 8000 10000 12000 0 2000 4000 6000 8000 10000
Actual pile load (kN) Actual pile load (kN)
Fig. 3. Comparison of actual versus predicted loads: (a) HON type II versus training data and (b) HON type II versus testing data.

15000 follows:
  T h i
R2 = 0.562 Ns ¼ N s,1 As,1 N s,2 As,2 N s,3 As,3 Ns,4 As,4 Ns,5 As,5 ð12Þ

where As,1 to As,1 are the areas of shaft segments. The parameters
Predicted pile load (kN)

styp and ptyp stand for soil type and method of pile installation.
10000
Soil type is represented by the percentage particle size, which are
approximately deduced from soil classification. The approximate
values so generated will then serve as inputs to the network four
nodes are dedicated for the approximate values of sand, silt and
clay percentages. The fourth node is assigned to plasticity index.
5000
Method of installation is represented by two nodes. For a
particular type of pile the appropriate node (or nodes) will fire a
value of unity. For example if a pile type is driven then both nodes
are disabled, when it is bored then one node is switched on. The
second node is activated to fire a value of 1.0 when the bored
0
pile is installed by dry excavation. The input parameters to the
0 5000 10000 15000
network include the pile head settlement; s, pile stiffness; EpAb/L.
Actual pile load (kN) 
Others include six parameters to represent soil resistance; ½Ns 
Fig. 4. Comparison of measured versus settlements predicted by hyperbolic model. and NbAb, percentage of sand, silt and clay and three input
parameters to represent method of pile installation (driving,
wet excavation and dry excavation). The input parameters used
in different models (BPN and HON) is summarized in Table 2.
10000

9000 3.2.2. Networks training and validation

The pile load test data was partitioned into training and
8000 validation data, with the former being used to develop the
network. The latter was used to verify the predictive quality of
7000
the trained model. 79% of the database was used for training,
Load (kN)

while the remaining 21% of the data was used for testing the
6000
network prediction. The training data is, as much as possible,
5000 made to capture the widest variations in input and output
patterns in the database. This is to avoid having extreme data in
4000 the testing set, which could make it impossible to assess the true
Measured data
HON type II simulation
generalization capability of the model within the domain of the
3000 training data as in the case of completely randomized selection
(Shahin et al., 2004).
2000
The synaptic weights were initialized uniformly distributed
random numbers in both networks, while learning parameters of
1000
0 5 10 15 20 25 30 0.01 and 0.1 were used in the training of HON and BPN,
respectively. Offline training was used in the case of HON while
Settlement (mm)
the BPN was trained using online method. The exponents of HON
Fig. 5. Comparison of HON type II simulations and training data. model are restricted to a range of  3.0 to 3.0. This is to prevent
 A. Ismail, D.-S. Jeng / Engineering Applications of Artificial Intelligence 24 (2011) 813–821 819

1000 3.3. Comparisons

900 The ability of ANN to adapt to learning is measured by the

level of agreement between the training data and the network
800
output, while the quality of the network’s prediction is gauged by
700 the comparing the testing data and the network predictions. In
Fig. 2(a) and (b), the BPN network outputs are compared with
600 training and testing data, respectively. The HON simulations are
Load (kN)

compared with the training and testing data in Fig. 2(c) and (d),
500 respectively. The high-order network, compared to the BPN,
correlate better with the training data (R2 ¼0.91 for HON and
400 0.88 for BPN). This implies the level of learning in the case of HON
type I model exceeds that of BPN. The R2 values of 0.87 and 0.78
300
Measured data for type I HON and BPN, respectively, point to the better predic-
HON type II simulation tion of HON compared to BPN. It is however noted that at
200
settlement levels of 40 mm or less, the correlation is rather poor
100 in both HON type I and BPN.
The predictions of type II HON model are compared with
0 training and testing data in Fig. 3(a) and (b), respectively. The
0 1 2 3 4 5 6 7 strong correlation between HON predictions and duo of training
Settlement (mm) and testing data is indicative of the high level of learning achieved
by the network and the high quality of prediction the network is
Fig. 6. Comparison of HON type II simulations and testing data.
capable of doing. Also, by comparing the scattergrams of HON
type II (Fig. 3(b)) and Fleming’s hyperbolic model (Fig. 4), it could
be rightly said that the former remarkably out performs the
latter. Results of HON type II model (R2 and RMSE) are compared
Table 5
with four different models in Table 4. The models used in the
Summary of sensitivity analysis.
comparison include Fleming’s hyperbolic model, multivariate
Parameter Network input RMSE (kN) polynomial regression model, RBF network and GRNN model.
From the summary of results, the HON type II has the best
All Ep Ab h  i 554.40 performance (R2 ¼ 0.890), followed by RBF network (R2 ¼0.828).
s, , N s ,Nb Ab , % sand, % silt, % clay, pile Type
L
Settlement Ep Ab h  i 2961.09 The polynomial regression model and hyperbolic model are at the
, N s ,Nb Ab , % sand, % silt, % clay, pile Type
hL  i
bottom of the performance list with the later having the least
Ep Ab s, N s ,Nb Ab , % sand, % silt, % clay, pile Type 717.00 prediction quality among the other models based on the testing
h L i data (R2 ¼0.562). To assess the quality of load–settlement curves
Ns Ep Ab 1275.91
s, ,Nb Ab , % sand, % silt, % clay, pile Type
L predicted by HON type II, the simulations are compared with a
NbAb Ep Ab h  i 767.47 selected pile loading test from training data in Fig. 5 and with a
s, , N s ,Nb Ab , % sand, % silt, % clay, pile type
L
Pile type Ep Ab h  i 684.73 loading test from testing data in Fig. 6. Both Figs. 5 and 6
s, , N s ,Nb Ab , % sand, % silt, % clay, pile Type
L indicated that HON simulations correlate well with field mea-
Soil type Ep Ab h  i 778.30 surements. A further comparison of type II HON, elastic and
s, , N s ,Nb Ab , pile Type
L
hyperbolic models were also made against the measured values
(Fig. 7). It is clearly observed from the figure that HON gives the
the network from diverging during training. The training was closest estimate of settlement and therefore most accurate.
terminated when no further improvement in the quality of
prediction in comparison with the test data is observed. Termina-
tion of training at this juncture is necessary in order to prevent 10000
over-learning and loss of generalization. After termination of the
training, the network simulations were compared with the test 9000
data, which is not part of the training. The assessment parameters 8000
in both training and testing are the roots mean square error
(RMSE) and the coefficient of determination (R2). 7000
The root mean square error is defined as follows:
6000
Load (kN)

sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
Pn 2
i ðyp ym Þ 5000
RMSE ¼ ð13Þ
n
4000
The coefficient of determination R is a measure of scatter or
lack of it between two sets of data. It is given by 3000
Measured data
 P P P 2 2000 HON type II prediction
n ym yp ð ym Þð yp Þ
R2 ¼ P P
P P
ð14Þ Elastic solution (Poulos and Davis, 1980)
nð y2m Þð ym Þ2 nð y2p Þð yp Þ2 1000 Hyperbolic model (Fleming, 1992)

0
in which ym and yp are measured and predicted parameters, 0 5 10 15 20 25 30 35 40
respectively.
Settlement (mm)
Tables 3 and 4 summarize the training and testing results for
settlement and load functions, respectively. Fig. 7. Performance comparison of various settlement prediction methods.
820
Settlement per unit load (mm/kN) A. Ismail, D.-S. Jeng / Engineering Applications of Artificial Intelligence 24 (2011) 813–821

1.2 0.6

Settlement per unit load (kN)

1 0.58
0.8
0.56
0.6
0.54
0.4

0.2 0.52

0 0.5
0 7 14 21 28 35 10 20 30 40 50
Pile length (m) Slenderness ratio, L/D ( mm)

Settlement per unit load (mm/kN)

Settlement per unit load (mm/kN)

0.5 1.2

0.45 1

0.8
0.4
0.6
0.35
0.4
0.3 0.2

0.25 0
0 0.05 0.1 0.15 0.2 0.25 0 20 40 60 80
Pile stiffness (GN/m) SPT Number of blows (shaft)

0.7
Settlement per unit load (mm/kN)

0.6
0.5
0.4
0.3
0.2
0.1
0
0 30 60 90 120 150
SPT Number of blows (base)

Fig. 8. Sensitivity of HON type II model to: (a) pile embedded length, (b) pile stiffness, (c) average SPT blow counts along the pile shaft, and (d) SPT blow counts at the
pile base.

3.4. Parametric study and sensitivity analysis indicative of the good agreement between the HON model and
actual pile behaviour, where the capacity of pile increases with the
The results of parametric study carried out to assess the shaft area (a function of length and perimeter) and the resistance
generalization ability of HON type II model are presented in to compression is proportional to stiffness. The settlement is
Fig. 8(a)–(e). The approach used in the analysis is similar to the observed (in Fig. 8(c)) to vary proportionally with slenderness
one used by Shahin et al. (2004), where all input variables, except ratio (with diameter varied while length is kept constant). This is in
the subject of the analysis are re-fixed to the mean of training agreement with the study carried out by Poulos (1977). With
values while the subject parameter is allowed to vary between regard to the behaviour of foundation soil, the average number of
the minimum and maximum values used for model training. The blows along the shaft varies inversely with settlement; although
response of the model to the variation of the single parameter is the curve is not very smooth (Fig. 8(d)). Inverse relationship is also
examined in the light of the current knowledge about the observed in the case SPT-N for the pile base (Fig. 8(e)).
phenomenon. The analysis is then repeated with different con- Sensitivity analysis has also been carried out to see how
trolling parameters. strongly various input parameters affect the HON output. To test
Based on the results obtained, it is noticeable that there is a a particular parameter, its values are taken away from the input
good agreement with the HON type II response and expected vector and the performance of the model is evaluated. The results
behaviour of piles when subjected to axial loading. The consistent of the analysis are summarized in Table 5. It can be seen that all
decrease in the predicted pile head settlement (per unit load) with input parameters considered in this model significantly affect the
increment in pile length and stiffness in Fig. 8(a) and (b) is quality of HON prediction.
 A. Ismail, D.-S. Jeng / Engineering Applications of Artificial Intelligence 24 (2011) 813–821
4. Conclusions
Due to the complexity of soil–pile interaction and the soil
disturbance during sampling and pile installation, the conven-
tional methods of predicting pile load–settlement relationship
suffer from lack of reliability. In this paper, a high-order network
to simulate the pile load–settlement curve using simple input
data is proposed. The input data consist of the average value of
SPT along the pile shaft, the SPT value at the pile base, the pile
stiffness, the shaft and base area, and the pile load. Other
parameters include soil type and installation method. Based on
the coefficient of determination and root mean squared error,
as well as the quality of load–settlement curves, a significant
improvement was observed from the comparison of HON results
with BPN, elastic and hyperbolic models. The proposed model was
also found to be more accurate in prediction than RBF, GRNN and
polynomial regression. Also, based on the parametric study
carried out, the HON model was found to respond reasonably
well to various input parameters in a manner consistent with the
anticipated behaviour of an axially loaded pile.
Appendix A. Supporting information
Supplementary data associated with this article can be found
in the online version at doi:10.1016/j.engappai.2011.02.008.
References
Abdelbar, A.G., Tagliarini, G., 1996. HONEST: a new high order feed forward neural
network. In: Proceedings of the IEEE International Conference on Neural
Networks, vol. 2, pp. 1257–1262.
Agrawal, G., Chameau, J.A., Bourdeau, P.L., 1997. Assessing the liquefaction
susceptibility at a site based on information from penetration testing. In:
Kartam, N., Flood, I., Garrett, J.H. (Eds.), Artificial Neural Networks for Civil
Engineers: Fundamentals and Applications, New York, USA, pp. 185–214.
Artyomov, E., Yadid-Pecht, O., 2005. Modified high-order neural network for
invariant pattern recognition. Pattern Recognition Letters 26 (6), 843–851.
Bahi, J.M., Contassot-Vivier, S., Sauget, M., 2009. An incremental learning algo-
rithm for function approximation. Advances in Engineering Software 40,
725–730.
Balakrishnan, E.G., Balasubramaniam, A.S., Phien-wej, N., 1999. Load deformation
analysis of bored piles in residual weathered formation. Journal of Geotechni-
cal & Geoenvironmental Engineering, ASCE 125 (2), 122–131.
Bateni, S.M., Borghei, S.M., Jeng, D.-S., 2007. Neural network and neuro-fuzzy
assessments for scour depth around bridge piers. Engineering Applications of
Artificial Intelligence 20, 401–414.
Basheer, Najjar, Y.M., 1998. Modeling cyclic constitutive behavior by neural
networks: theoritical and real data. In: Proceedings of the 12th Engineering
Mechanics Conference, La Jolla, California, 17–20 May, pp. 952–955.
Chan, W.T., Chow, Y.K., Liu, L.F., 1995. Neural network: an alternative to pile
driving formulas. Computers and Geotechnics 17, 135–156.
Chang, M.F., Broms, B.B., 1991. Design of bored piles in residual soils based on
field-performance data. Canadian Geotechnical Journal 28, 200–209.
Chin, F.K., 1970. Estimation of the ultimate load of piles from tests not carried to
failure. In: Proceedings of the Second SE Asian Conference on Soil Engineering,
Singapore, pp. 81–92.
Coyle, H.M., Reese, L.C., 1966. Load transfer for axially loaded piles in clay. Journal
of Soil Mechanics and Foundations Division, ASCE 92 (SM2), 1–26.
Das, S.K., Basudhar, P.K., 2006. Undrained lateral load capacity of piles in clay using
artificial neural network. Computers and Geotechnics 33 (8), 454–459.
Desai, C.S., 1977. Deep Foundation. Numerical Methods in Geotechnical Engi-
neerng. McGraw-Hill, New York, pp. 235–271.
Desai, C.S., 1974. Numerical design – analysis for piles in sands. Journal of
Geotechnical Engineering Division, ASCE 100 (GT6), 613–635.
Durbin, R., Rumelhart, R., 1989. Product units: a computationally powerful and
biologically plausible extension to backpropagation networks. Neural Com-
puting 1, 133–142.
Ellis, G.W., Yao, C., Zhao, R., Penumadu, D., 1995. Stress–strain modelling of sands
using artificial neural networks. Journal of Geotechnical Engineering, ASCE
121 (5), 429–435.
821
Fleming, W., 1992. A new method of single pile settlement and analysis.
Geotechnique 42 (3), 411–425.
Fleming, W.G.K., Weltman, A.J., Randolf, M.F., Elson, W.K., 1992. A New Method of
Single Piling Engineering. John Wiley & Sons, New York.
Foresti, G.L., Dolso, T., 2004. An adaptive high-order neural tree for pattern
recognition. IEEE Transactions on Systems, Man, and Cybernetics, Part B:
Cybernetics 34 (2), 988–996.
Goh, A.T.C., 1995. Modeling soil correlations using neural networks. Journal of
Computing in Civil Engineering, ASCE 9 (4), 275–278.
Goh, A.T.C., 1996. Pile driving records reanalyzed using neural networks. Journal of
Geotechnical Engineering, ASCE 122 (6), 492–495.
Haykin, S., 1999. Neural networks. A comprehensive foundation, 2nd ed. Prentice-
Hall, Upper Saddle River.
Ismail, A., Engelbrecht, A.P., 2000. Global optimization algorithms for training
product-unit neural networks. In: Proceedings of the IEEE-INNS-ENNS Inter-
national Joint Conference on Neural Networks (IJCNN 2000), Como Italy, vol. 1,
pp. 132–137.
Ivakhnenko, A.G., 1971. Polynomial theory of complex systems. IEEE Transactions
on Systems, Man, and Cybernetics SMC-1, 364–378.
Jaksa, M.B., Maier, H.R., Shahin, M.A., 2008. Future challenges for artificial neural
network modelling. In: The 12th International Conference of International
Association for Computer Methods and Advances in Geomechanics. pp.
1710–1719.
Lee, I.M., Lee, J.H., 1996. Prediction of pile bearing capacity using artificial neural
networks. Computers and Geotechnics 18 (3), 189–200.
Lee, I.M., Salgado, R., 1999. Determination of pile base resistance n sands. Journal
of Geotechnical and Geoevironmental Engineering, ASCE 125 (8), 122–131.
McCulloch, W.S., Pitts, W., 1943. A logical Calculus of the ideas Immanent in
nervous activity. Bulletin of Mathematical Biophysics 5, 115–133.
Nawari, N.O., Liang, R., Nusairat, J., 1999. Artificial intelligence techniques for the
design and analysis of deep foundations. Electronic Journal of Geotechnical
Engineering /http://geotech.civeng.okstate.edu/ejge/ppr9909/index.htmlS.
Nejad, F.P., Jaksa, M.B., Kakhi, M., McCabe, B.A., 2009. Prediction of pile settlement
using artificial neural networks based on standard penetration test data.
Computers and Geotechnics 36 (2009), 1125–1133.
Penumadu, D., Jean-Lou, C., 1997. Geomaterial modeling using artificial neural
networks. Artificial Neural Networks for Civil Engineers: Fundamentals and
Applications, ASCE, 160–184.
Prevost, J.H., Popescu, R., 1996. Constitutive relations for soil materials. Electronic
Journal of Geotechnical Engineering /http://www.ejge.com/1996/Ppr9609/
Abs9609.htmS.
Poskitt, T.J., 1993. A new method for single pile settlement prediction and
analysis—discussion. Geotechnique 43 (4), 615–619.
Poulos, H.G., Davis, E.H., 1980. Pile Foundation Analysis and Design. Wiley.
Poulos, H.G., 1977. Settlement of Pile Foundations. Numerical Methods in
Geotechnical Engineerng. McGraw-Hill, New York, pp. 326–363.
Randolph, M.F., Wroth, C.P., 1978. Analysis of deformation of vertically loaded
piles. Journal of Geotechnical Engineering Division 104, 1465–1488.
Rovithakis, G.A., Chalkiadakis, I., Zervakis, M.E., 2004. High-order neural network
structure selection for function approximation applications using genetic
algorithms. IEEE Transactions on Systems, Man, and Cybernetics, Part B:
Cybernetics 34 (1), 150–158.
Rumelhart, D.E., Hinton, G.E., Williams, R.J., 1986. Learning internal representation
by error propagation. In: Rumelhart, D.E., McClelland, J.L. (Eds.), Parallel
Distributed Processing, vol. 1. MIT Press, Cambridge, MA (Chapter 8).
Scott, R.F. (1988). Constitutive relations for soil: present and future. In: Saada and
Bianchini (Eds.), Constitutive Equations for Granular Non-cohesive Soils,
Rotterdam.
Shahin, M.A., Jaksa, M.B., Maier, H.R., 2002. Predicting settlement of shallow
foundations using neural networks. Journal of Geotechnical and Geoenviron-
mental Engineering 128 (9), 785–793.
Shahin, M.A., Jaksa, M.B., Maier, H.R., 2003. Settlement prediction of shallow
foundations on granular soils using B-spline neurofuzzy models. Computers
and Geotechnics 30 (8), 637–647.
Shahin, M.A., Maier, H.R., Jaksa, M.B., 2004. Data division for developing neural
networks applied to geotechnical engineering. Journal of Computing in Civil
Engineering 18 (2), 105–114.
Tan, Y.C., Chen, C.S., Liew, S.S., 1998. Load transfer behaviour of cast-inplace bored
piles in tropical residuals soils. In: Proceedings of the 13th Southeast Asian
Geotechnical Conferences, Taipei, pp. 563–571.
Teh, C.I., Wong, K.S., Goh, A.T.C., Jaritngam, S., 1997. Prediction of pile capacity
using neural networks. Journal of Computing in Civil Engineering, ASCE 11 (2),
129–138.
Tomlinson, M.J., 1995. Pile Design and Construction Practice, 6th edition Longman.
Ural, D., Saka, H., 1998. Liquefaction prediction by neural networks. Electronic
Journal of Geotechnical Engineering /http://geotech.civen.okstate.edu/ejge/
ppr9803/index.htmlS.
Vesic, A.S., 1977. Design of pile foundations, National Cooperative Highway
Research Program, Synthesis of Practice No. 42. Transportation Research
Board, Washington, DC.
Vijayvergiya, V.N., 1977. Load movement characteristics of piles. In: Proceedings
of the Ports’77 Conference, Long Beach, California.