Construction and Building Materials 270 (2021) 121424

Contents lists available at ScienceDirect

Construction and Building Materials

journal homepage: www.elsevier.com/locate/conbuildmat

A Bayesian machine learning approach for inverse prediction of

high-performance concrete ingredients with targeted performance
Xinyuan Ke a,⇑, Yu Duan b,⇑
a
Department of Architecture and Civil Engineering, The University of Bath, Bath BA2 7AY, UK
b
Department of Mechanical Engineering, Imperial College London, London, UK

h i g h l i g h t s g r a p h i c a l a b s t r a c t

An accurate Gaussian process (GP)

strength predictive model for HPC.

Global sensitivity analysis of GP can
reflect the physicochemical nature of
HPC.

A novel performance-based concrete
design framework based-on Bayesian
inference.

a r t i c l e i n f o a b s t r a c t

Article history: High-performance concrete (HPC) plays an important role in improving the sustainability and reliability
Received 22 August 2020 of buildings and infrastructures. Machine learning predictive models have been used for predicting con-
Received in revised form 9 October 2020 crete performance from ingredients, however it remains a challenge to achieve inverse prediction of
Accepted 20 October 2020
ingredients from targeted performances. This study proposes an in-house coded informatics-based mate-
Available online 6 November 2020
rials analysis framework to enable computational design of HPC with targeted strength performance. The
Gaussian processes (GP) emulator is used to construct the surrogate predictive model based-on 453
Keywords:
experimental measurements. The validity of the GP emulator is assessed using the leave-one-out
Informatics design
Global sensitivity analysis
cross-validation (LOO-CV) and also a separate validation dataset. The variance-based global sensitivity
Sobol indices analysis, Sobol indices, is applied to understand the impact of physical ingredients on the HPC perfor-
Bayesian inference mances. The results suggest that the trained GP emulator can provide sufficiently accurate and reliable
Markov chain Monte Carlo predictions, as well as reflect the real-world physicochemical nature of HPC materials. The inverse mate-
High-performance concrete rial design is achieved by the Bayesian inference method with a Markov chain Monte Carlo stochastic
sampling method, the Metropolis-Hastings (MH) algorithm. Combining with the Bayesian inference
method, the proposed design framework can infer a list of potential HPC formulae of a targeted perfor-
mance, each evaluated by the likelihood of resulting in the targeted strength. The data-driven material
analysis and design framework proposed in this study provides a novel approach to achieve
performance-based design of HPC, with the potential to maximise resource efficiency and reduce eco-
nomical cost. The methodology presented in this study can also be extended to be applied to a wide range
of construction materials, targeting difference service performances including durability.
Ó 2020 Elsevier Ltd. All rights reserved.

⇑ Corresponding authors.
E-mail addresses: x.ke@bath.ac.uk (X. Ke), y.duan@imperial.ac.uk (Y. Duan).

https://doi.org/10.1016/j.conbuildmat.2020.121424
0950-0618/Ó 2020 Elsevier Ltd. All rights reserved.
X. Ke and Y. Duan Construction and Building Materials 270 (2021) 121424

Nomenclature

Acronyms si ; s i main effect sensitivity indices
t
cdf cumulative distribution functions sti ; si total effect sensitivity indices
pdf probability density functions
Ti total variance
due
 to xi
BFS blast furnace slag 
! !
V; V variance of f x due to varying x
CA coarse aggregate
CS HPC strength Vi partial variance due toxi
FA fine aggregate W2 integral of squared difference between the cdfs of two
HPC high-performance concrete datasets
n on
! !
GP Gaussian processes x ;X ¼ x i input(s) of a process
i¼1
LHS Latin hypercube sampling
x unknown input of a process
LOO-CV leave-one-out cross-validation !
xt inputs causingyt
NRMSE normalised root mean square error !
PFA fly ash y; y ¼ fyi gni¼1 observed outcome(s) of a process
RMSE root mean square error yt target value
SE squared exponential y output of a process atx
SP superplasticizer y mean of GP prediction atx
v arðy Þ variance of GP prediction atx
Nomenclature
! Greek Letters
F ðÞ cdf of y

F ðÞ cdf of y
!  observational error
rf signal variance of SE covariance function
K ð; Þ; K  ð; Þ; K  ð; Þ covariance matrix in the Gaussian processes rn standard deviation of
kð; Þ covariance function !
! h vector of hyperparameters
l length scales vector of SE covariance function
mðX Þ mean function in the GP

1. Introduction concrete using an adaptive probabilistic neural network (APNN),

The high-performance concretes (HPCs) play important roles in
civil engineering applications. Due to the increasing concern of CO2
emission from the cement industry, supplementary cementitious
materials (SCMs) with very low embodied carbon (e.g. slag, fly
ash, calcined clays) have been widely used in the cement industry
[1,2]. These blended cement binder systems are not only more
environmentally friendly, but also exhibit superior engineering
performances, such as higher strength and better durability [3].
Predicting the strength of these HPCs is not a straightforward
task. The inclusion of different types of SCMs could have distinct
effects on the hydration kinetics of cement [4,5], one of the domi-
nant driving factors controlling the strength of an HPC. Besides, the
addition of superplasticiser and water reducer would have non-
linear and non-uniform effects on strength development [6,7].
Other key factors, such as water content, aggregate content, and
curing regimes also play important roles. The variance of one or
multiple parameters can result in significant changes in the final
strength performance.
Predictive models have been developed for predicting the
strength of HPCs containing supplementary cementitious materi-
als. De Larrard [8] proposed one of the early predictive model for
concrete containing pozzolanic admixtures and lime fillers. How-
ever, this somewhat physical-based model relies on manually tun-
ing of parameters and weighting factors. Also, it cannot be used for
concrete containing superplasticiser and/or water reducer [8]. To
tackle this multi-variance-controlled problem, many researchers
applied machine learning methods to predict the strength of HPCs.
The Artificial neural network (ANN) was first used as the non-
physical model for predicting the strength of HPC [9]. Since then,
machine learning has been widely used to perform concrete
strength prediction [10–18]. Fazel Zarandi, et al. [12] proposed a
hybrid neural network model combing fuzzy logic with polynomial
neural networks to improve the performance of a simple ANN
model. Lee et al. [19] predicted the strength of conventional
which improved the training efficiency of basic neural network
and has the ability to calculate the smoothing parameter automat-
ically. Khan [13] proposed a new ANN model for predicting HPC
containing fly ash and silica fume, and extend the performance
assessment to durability related properties such as chloride and
gas permeability. Chou and Tsai [14] used a hierarchical machine
learning model, combining classifier support vector machine
(SVM) with ANNs, in order to improve the adaptability and flexibil-
ity of the training model for HPC strength prediction. Madandoust
et al. [20] proposed the use of cored samples to assess the in-situ
concrete strength by combining the group method of data handling
(GMDH) type neural networks with adaptive neuro-fuzzy infer-
ence systems (ANFIS).
Despite that a large number of research efforts have been put
into investigating HPC with different concrete ingredients and
improving the prediction accuracy, it remains as a general chal-
lenge to design concrete with targeted performance through these
data-driven predictive models. The design of a concrete structure is
based on the choice of concrete with known strength grade [21].
Therefore, it would significantly expedite the material selection
process if the trained informatic model can also suggest the most
plausible concrete designs based on the targeted strength grade.
Recent efforts have been made using machine learning-powered
multi-objective optimisation algorithms to suggest concrete
designs with optimised performances. Yeh [22] proposed a cost-
based concrete design optimisation framework using the genetic
algorithm. Huang et al. [23] proposed a hybrid intelligent system
combining different machine learning models with multi-
objective optimisation methods to optimise the use of recycled
aggregate in concrete with minimal cost. Zhang et al. [24] com-
bined the support vector regression method with a firefly algo-
rithm to determine the mix design of steel fibre reinforced
concrete for optimised compressive and flexural strength. How-
ever, the current methods are not able to perform probabilistic-
based concrete design inference of any given performances. In this

2
X. Ke and Y. Duan Construction and Building Materials 270 (2021) 121424

study, a Bayesian machine learning approach is investigated to 2.1. Gaussian processes (GP)
achieve probabilistic-based inverse prediction of HPC concrete
design with targeted performance. !
Given observationsy ¼ fyi gni¼1 with observational error
Bayes’ theorem allows one to infer the posterior probability of n on
an unknown event based on prior probability density function
 Nð0; r2n Þ at n inputs X ¼ !xi
!
, where x i 2 RD , the emulator
i¼1
!
related to the event. In this work, Bayes’ theorem is used to infer is designed to estimate the y at a new input x  . This problem
the plausible concrete ingredients with respect to the targeted can be expressed as finding the most plausible y with the knowl-
strength by exploring the posterior probability of ingredients in !
edge of y , which can be expressed as
the design domain. The Gaussian process (GP), also known as the

!
Kriging originated from geostatistics [25], is a supervised non- argmax P y j y ð2Þ
!
parametric Bayesian machine learning method. It can also be x 2A
viewed as a prior on one-layer feed-forward Bayesian neural net-

!
works with an infinite number of hidden units [26]. It provides According to the Bayes’ theorem, p y j y in Eq. (2) can be writ-
the interpolation of the unknown based on prior knowledge sup- ten as
plied by observed data. Comparing to parametric regression meth-

!
P y jy P ðy Þ
ods, such as least-squares linear regression and polynomial !
Pðy j y Þ ¼
 ð3Þ
regression, GP is a more rigorous method for the treatment of com- !
P y
plex noisy non-linear functions [27] and normally applied to create
the surrogate model for the computational expensive modelling In the Gaussian processes, it is assumed that any finite number
[28–31]. The output of the GP model is the posterior probability of a collection of random variables follow a multivariate Gaussian


of the quantify of interest (QoI) with the input data as prior knowl- !
distribution [35–37]. Hence P y is expressed as
edge. Because of this unique feature, the GP model can be easily


implemented with the Bayesian inference method. The Bayesian ! !!
P y ¼ NðmðX Þ; Kð x ; x ÞÞ ð4Þ
inference method has attracted increasing attention from the
materials research community for its ability to assist with the

!
design and search of new materials [32]. Kim et al. [33] used the Furthermore, P y ; y can be written as
Bayesian inference method to study the phase transformation in 2
T 3
  !
alloy from macro-scale characterisation. Balachandran [34] combi-

! 
 mðX Þ 6 KðX; XÞ K  x ; X 7
nes Bayes’ theorem with density functional theory for designing P y ; y ¼ Nð ;4

 5Þ ð5Þ
mðx Þ ! ! !
functional materials with targeted performances. However, it has K  x ; X K  x  ; x 
not been applied to ingredient-related materials prediction, such !
as HPC. The conditional distribution of y given condition y is



In this study, a GP-powered framework is proposed to analyse ! !
P y j y ¼ Nðmðx Þ þ K  K 1 y  mðX Þ ; K 
the effects of different ingredients in the HPC strength and to pro-
 K  K 1 ðK  Þ Þ
T
pose potential HPC formulae with targeted HPC strength. The rest ð6Þ
of this paper is organised as follows: the Gaussian processes
! !
together with the proposed framework for the ingredient analysis In Eqs. (4)–(6), mðÞ is the mean function and Kð x ; x Þ,



and the ingredient inference are introduced in Section 2. Section 3 ! !  ! !
K x; x , K x ; x are covariance matrixes of which ele-
describes the data adopted to train and validate the GP model. The
ments are calculated using the covariance function kð; Þ, which
details about the model training and validation are included in Sec-
provides the covariance between two arbitrary sample locations,
tion 4, followed by the global sensitivity analysis of material ingre- ! !
x i and x j .T denotes the transpose of the vector or matrix.
dients in Section 5, and formula inference in Section 6. The
perspectives in application and further extension of the proposed To simplify the calculation, it is common to assume that the
model is discussed in Section 7. mean of the joint Gaussian distribution is zero. Therefore the Eq.
(6) can be simplified as


! !
P y j y ¼ NðK  K 1 y ; K   K  K 1 ðK  Þ Þ
T
ð7Þ
2. Mathematic background and proposed framework
The mean of the estimations of y is
To describe a given process, the relation between the inputs and y ¼ K  K 1 y ð8Þ
the outcome can be generalised as Eq. (1),
while the variance of the estimations is


!
y¼f x þ ð1Þ v arðy Þ ¼ K   K  K 1 ðK  ÞT ð9Þ

Instead of giving the form of the latent function as prior, the

!
where y is the observed outcome of the process f with inputs x ,
and  is the error describing the difference between y and the true
process of f . In many cases, the true expression of f is unobservable.
However, it is possible to mimic the behaviour of the latent function
f using observed data and the machine learning method. To distin-
guish between the true process and the machine learning model, we
refer the later as the emulator in the rest context. In this section, a
brief introduction on Gaussian processes (GP) is given, followed by
a description about the performance-based design framework. For
readers who are interested in more details about the Gaussian pro-
cesses, please refers to [35].
3
prior in the GP is the covariance function, which essentially defines
the data structure and allows the data to determine the form of the
latent function. The reliability of a GP emulator depends on the
form and the hyperparameters ofkð; Þ. The design of the covariance
function is an active area of research in the GP research community.
More about covariance functions can be found in [35]. In this study,
the squared exponential (SE) covariance function is chosen as it
generally performances well in many problems [38]. It is defined as
h i
  ! ! !
ð x i  x j Þ=2 l 2
k xi ; xj ¼ rf e
2
þ r2n dðxi ; xj Þ ð10Þ
X. Ke and Y. Duan Construction and Building Materials 270 (2021) 121424

in which dðxi ; xj Þ is the Kronecker delta function. The hyperparame- dataset. In LOO-CV, the training dataset is divided into two groups,
! ! ! 
ters in the kernel function, denoted h ¼ frf ; l ; rn g, are the signal namely the LOO-CV training data y i ¼ y1 ;    yi1 ; yiþ1 ;    yn and
! !
variance, the length scale and the white noise level, respectively. the LOO-CV validation data yi . The y i and h are used to estimate
The signal variance, rf , is a scaling factor which determines the !
the process outcome at x i . The outcome is then compared to yi .
variation of the values from the mean of the prediction. The length The integrity of the emulator is qualitatively checked using the
! !
scale l describes how smooth the function f is. Smaller l means a graphical comparisons (the quantity-to-quantity plot and the error
! histogram) and quantitatively assessed using following metrics:
more rapid change in the function, while larger l will cause a
Root Mean Square Error (RMSE):
smoother prediction. The white noise level, rn , specifies how much
noise is expected to be in the observation.
  1X n
 2
With the form of k xi ; xj determined, the problem then RMSE ¼ y  yi ð13Þ
n i¼1 i
! !
becomes finding the most plausible h with the knowledge of y
and X. According to the Bayes’ theorem, the posterior over the Normalised Root Mean Square Error (NRMSE):
hyperparameters is defined as RMSE


! NRMSE ¼ 1 Pn ð14Þ

!  ! ! i¼1 yi
p y jX; h p h n
!
p h j y ;X ¼ R

! ! ð11Þ
! ! Pearson Correlation Coefficient (q)
p y jX; h p h d h


!!

!

! !
 cov y ; y 
where p h is the prior of hyperparameters and p y jX; h is the q¼ ð15Þ
R
! !
! ! ry ry
marginal likelihood. The fraction p y jX; h p h d h is a normal-

!
!
 where cov ð; Þ is the covariance of two vectors, ry and ry stand for
isation constant. Therefore the maximum of p h j y ; X , is effec- ! !

 the standard deviations of y and y  , respectively.
! ! ! Integral of squared difference between cumulative distribution
tively determined by p y jX; h when pð h Þ is sufficiently flat
! functions (cdf) of two datasets (W 2 )
[38]. Hereby, determination of the robust h becomes


 Z
! ! ! ! 1
argmax p y jX; h . In practice, the logarithm of p y jX; h ,
ðF ðyÞF  ðyÞÞ dy
2
! Dþ2 W2 ¼ ð16Þ
h 2R 0
Eq. (12), is usually maximised
where F ðÞ and F  ðÞ stand for the cdf of y and y , respectively.


! ! 1! ! 1 n
log p y jX; h ¼  y T K 1 y  ln jK j  ln2p ð12Þ
2 2 2
3. Input data
where n is the number of observations. Optimisation algorithm is
! The ingredients in the HPC formulae considered in the GP emu-
needed in order to determine the robust set of h . It should be noted
that optimisation algorithm may lead to a local maximum of lator are cement, blast furnace slag (BFS), fly ash (PFA), water,


! ! ! superplasticizer (SP), coarse aggregate (CA), and fine aggregate
ln p y jX; h , hence the suitability of h must be scrutinised. More
(FA). The database used in this study contains the data collected
details will be given in the next section. by Yeh [9] and the experimental data from [40], in total 453 data
points pairing formula with the HPC strength (CS). Concrete curing
2.2. Performance-based inference design framework age is not considered as input in this work and only data corre-
sponding to 28 days concrete strength are included in this study.
Fig. 1 shows the flowchart of the performance-based materials The proposed design and analysis framework is targeted for pro-
analysis and design framework proposed in this study. This GP viding guidance for concrete design practice, where the 28 days
and Bayesian inference-powered analysis and design framework strength is used as the reference in the concrete structure design
can be summarised as the following: code [21]. The ingredients of HPCs are normalised by the total
weight (kg/m3) of all ingredients. This is justified as the total
1. Normalising experimental observations. weight of all ingredients represents the density of the concrete.
2. Randomly dividing the data into the training dataset and valida- HPC strength (CS) is normalised using the maximum in the training
tion dataset. data set. Table 1 shows the ranges of the ingredients before and
! after normalisation.
3. Finding the robust set of hyperparameters h using the training
dataset. A total of 363 experimental measurements are randomly
4. Validating the GP emulator. assigned to the training dataset and the other 90 experimental
5. Analysing sensitivity of each ingredient and/or inferring the for- measurements are included in the validation dataset. The ingredi-
mula regarding to the targeted HPC strength. ents and CS in both training and validation datasets are plotted
against each other in Fig. 2, including the histograms of probability
In this study, 20% of the data is randomly assigned to the vali- density functions (pdf) of each variable and CS. There is no obvious
dation dataset while the rest of the 80% data is treated as training relationship between each variable and CS, except for the cement
dataset. The adaptive Nelder-Mead algorithm [39] is applied to find and CS. CS increases with higher cement content in the formula.
! Meanwhile, the histogram of ingredients, except for FA, in the
the optimal h . The search stops when the difference between the
training dataset and validation dataset are similar. And the range
maximum and minimum in the search is smaller than 103 . Two of each variable in the validation dataset is within the range of that
!
methods are adopted to check the validity of h , the leave-one- in the training dataset. Fig. 3 shows the box-and-whisker diagram
out cross-validation (LOO-CV) test and the validation test. LOO- of each variable in the two datasets. The variables in both datasets
CV uses the training data to validate the emulator itself, whilst possess similar statistical features. For instance, the 25% quantile
the other is to check the emulator against a separate validation and 75% quantile of each variable are similar, seeing Fig. 3.
4
X. Ke and Y. Duan Construction and Building Materials 270 (2021) 121424

Fig. 1. Flow chart of the proposed Bayesian theory-powered design and analysis framework.

Table 1 each other, with W 2 ¼ 3:6807  104 . This indicates that the GP
Ranges of input components before and after data normalisation.
emulator and the real process have very similar statistical proper-
Ingredients Original data from After normalisation (Volumetric ties. Therefore, it is reasonable to use the GP emulator to explore
source (Kg/m3) weight fraction) the statistical features of the true process. This is again approved
Cement 102.0–540.0 0.04482–0.2254 by the close alignment of the correlation coefficient between each
BFS 0.0–359.4 0.0–0.1503 HPC ingredient and CS shown in Fig. 5. The correlation coefficients
PFA 0.0–204.8 0.0–0.08884
between each ingredient and CS predicted using the LOO-CV are
Water 115.0–247.0 0.04996–0.1122
SP 0.0–35.8 0.0–0.01549 almost the same as those calculated using the training data. This
CA 801.0–1247.0 0.3459–0.5264 suggests that the GP emulator presented in this study has the abil-
FA 500.0–992.6 0.2164–0.4141 ity to represent the real-world physical features controlling the
strength development of the HPCs.
In order to ensure that the GP emulator performs non-
4. GP emulator validation preferential predictions, for example, no large prediction error to
always appear in certain areas of the domain, the training dataset
The adaptive Nelder-Mead simplex optimisation is chosen to is separated into four error range groups and shown in the cobweb
optimise the hyperparameters of the GP emulator. It is known that plots, Fig. 6. ‘Group-1’ contains the training data with the absolute
its outcome is dependent on the predefined initial searching error ratio of LOO-CV smaller than 0.1. The data with absolute error
points. In order to obtain a robust GP emulator, we perform several ratio within (0.1, 0.25] is assigned to ‘Group-2’. ‘Group-3’ consists
searches with randomised initial searching points in each attempt. of data with absolute error ratio in (0.25, 0.5]. The rest is included
The LOO-CV and the validation test of the best performing GP emu- in ‘Group-4’. In order to improve the visibility of the patterns, val-
lator are shown in this section. ues of each ingredient are normalised by their maximum values.
Fig. 4 shows the evolution of log marginal likelihood log ðP Þ in Clearly, no centration pattern is shown in the figure. Hence, the
the optimisation procedure as well as the results of LOO-CV. GP emulator does not have biased predictions within certain sam-
Although nine hyperparameters are considered in the optimisation ple domains.
procedure, the adaptive Nelder-Mead search converged in less The performance of the GP emulator is further checked against
than 200 steps, as shown in Fig. 4(a). The LOO-CV predictions are the validation dataset. The results are shown in Fig. 7. In general,
plotted against the training data in Fig. 4(b). As shown in the figure, the results of validation test are akin to the LOO-CV test. The
most of the LOO-CV predictions are landed within the 25% range quantity-to-quantity plot (Fig. 7(a)) demonstrates that the points
of training data. The RMSE, NRMSE and q are 0.07754, 0.16857 and are distributed at both sides of the equilibrium line, suggesting
0.90595, respectively. The histogram of the error ratio, defined as there is no preferred overestimation or underestimation of the val-
fLOO-CV prediction - training datag=training data, indicates that idation data. Most of the points land between the 25% error
around 60% of the LOO-CV predicted values are within 10% of range. RMSE, NRMSE and q are 0.072548, 0.16094 and 0.91893,
training data and nearly 90% of the LOO-CV results are within respectively. These values are close to those obtained from the
25% of the training data, referring Fig. 4(c). As presented in LOO-CV test. Nearly 60% of the GP predictions are within the
Fig. 4(d), the cumulative distribution functions (cdf) of CS in the 10% accuracy range of the validation data and around 90% of
training data and predicted in the LOO-CV test almost overlap with the GP predictions are within the 25% accuracy range. Together

5
X. Ke and Y. Duan Construction and Building Materials 270 (2021) 121424

Fig. 2. A view of the training dataset and verification dataset.

Fig. 3. Box-and-whisker diagrams of each variable and CS in training and validation datasets.

with the results of the LOO-CV test, it is reasonable to claim that
the trained GP emulator is valid and can provide sufficiently satis-
factory predictions.
Table 2 compares the accuracy of the trained GP emulator in
this study with the results of other predictive models using the
same HPC training data set. Since different validation methods
were used, here the RMSE for the training and validation data sets
are used as indicators of the general prediction accuracy. Both
RMSEs provided by the GP emulator, shown in the table, are con-
verted to equivalent values in MPa based on the data normalisation
procedure. Table 2 demonstrates three factors. Firstly, the GP emu-
lator in this study provides the closest RMSE value for both data-
sets (the difference is just 0.04 MPa) which indicates the trained
GP emulator is less likely to overfit the data. Secondly, RMSE values
of the GP emulator are smaller than most of the predictive models
reported in the literature, except for the back-propagation neural
networks and the firefly colony algorithm-based support vector
regression. Finally, it should be noted that the difference of RMSE

6
X. Ke and Y. Duan Construction and Building Materials 270 (2021) 121424

Fig. 4. (a) Evolution of logðPÞ in the optimisation process of hyperparameters; (b) Quantity-quantity plot (LOO-CV predictions vs training data); (c) histogram of error ratio
({LOO-CV predictions – Training data}/ Training data), (d) cumulative distribution function (cdf) of HPC strength in training data and LOO-CV predictions.

Fig. 5. Correlation coefficient between ingredients and CS obtained using training data and LOO-CV predictions.

values generated by GP emulator back-propagation neural net-
 X

d X  
!
works and the firefly colony algorithm-based support vector f x ¼ f0 þ f i ð xi Þ þ f ij xi ; xj þ   
i¼1 1 i j d
regression is marginal (<0.8 MPa).
þ f 1;2;;n ðx1 ;    xd Þ ð17Þ

5. Sensitivity analysis where

The Sobol sensitivity indices [42] are used here to quantitatively R
! Q
d  
f0 ¼ f x hXj xj dxj
analyse the sensitivity of CS due to varying ingredients. Sobol sen- j¼1
sitivity analysis attributes the total variance of a process output y R
! Q
d  
to each input variable with the consideration of interaction effects f i ðxi Þ ¼ f x hXj xj dxj  f 0
j¼1;j–i
among input variables. It is a very useful tool to study the impor- ..
tance of inputs/variables in a complex system [43,44]. .


 Pd P   P
! !
Assuming f x is integrable, it can be decomposed into 2n f 1;2;;n ðx1 ;    xn Þ ¼ f x  f 0  i¼1 f i ðxi Þ  1 i j d f ij xi ; xj     ni¼1 f i ðxi Þ

orthogonal functional terms as ð18Þ

7
X. Ke and Y. Duan Construction and Building Materials 270 (2021) 121424

Fig. 6. Cobweb plots of data in different accuracy groups (Group-1: absolute error ratio 0.1; Group-2: 0.1 < absolute error ratio 0.25; Group-3: 0.25 < absolute error
ratio 0.5; Group-4: absolute error ratio > 0.5). The grey web is the total training data, and the coloured web is the training data assigned to each group. The values shown in
y-axis is the normalised (to the maximum) values.

Fig. 7. (a) Verification data vs GP predictions; (b) histogram of error ratio ({GP predictions – Verification data}/Verification data).

Table 2
Comparison of prediction accuracy, root mean square error (RMSE) for HPC design using different predictive models from literature.

Num Predictive model RMSE (MPa) Training RMSE (MPa) Validation Ref
1 Neural networks 3.96 8.82 [11]
2 Genetic Operation Tree 9.50 9.3 [41]
3 Back-propagation neural networks 5.51 5.7 [41]
4 Time-Weighted Evolutionary Fuzzy Support Vector Machines 8.85 10.40 [16]
5 Firefly colony algorithm-based support vector regression 5.63 – [15]
6 Gaussian processes regression 6.33 6.29 This study

8
X. Ke and Y. Duan Construction and Building Materials 270 (2021) 121424

In Eq.(18), hX j ðÞ indicates the probability density function of xj . also shown in the figure. The total interactive effect represents the
xi ¼ fx1 ;    xi1 ; xiþ1 ;    xd g and d is the number of elements in the portion of variation due to interactions between one ingredient
input vector. Thanks to the orthogonality of the components in Eq. and others.


! As shown in Fig. 8, among all ingredients, the variation of CS is
(17), the variance of f x can be written as follows:
mostly due to the variation of water. This is matching with the
X
n X chemistry of HPC as the hydraulic reactions between water and
V¼ Vi þ V i;j þ    þ V 1;2;;d ð19Þ other binders (cement, BFS, and PFA) form the main strength giving
i¼1 1 i j n chemical phase, aluminium-substituted calcium silicate hydrate
where V i is the partial variance due to xi whilst others are interac- [4,5]. The impact of superplasticizer (SP), which is added in order
tive variance contributions due to different combination of inputs. to reduce the total amount of water required for reaching desirable
The Sobol indices (su ) are defined as the ratio between each workability [6], is much smaller than water itself, especially the
total interactive effect. Between the three binders assessed in this
component and V, namely su ¼ VVu ; u # f1; 2;    ; dg, among which
study, cement, blast furnace slag (BFS) and fly ash (PFA), the main
the Sobol main effect sensitivity indices and Sobol total effect sen-
and total effect of these three components follow the order of
sitivity indices for each inputs are of particular interest. The main
cement < BFS < PFA. This means that, comparing with the equiva-
effect sensitivity index is defined as
lent amount of cement, changes in the quantity of BFS and PFA are
Vi more likely to cause changes in the HPC strength. Combining with
si ¼ ð20Þ
V the correlation coefficient shown in Fig. 5, this suggests that
replacement of cement with BFS in HPC can result in better
where si measures the effect of varying xi alone. Larger si means the
strength performance, while replacement of cement with PFA
varying xi contributes more to V. While the total effect sensitivity
may result in lower strength performance. This is also consistent
index is
P with experimental observations, as HPC with up to 55 wt% BFS
Ti fu:i2ug V u replacement showed better 28 days strength than the ordinary
sti ¼ ¼ ð21Þ
V V Portland cement [46], while HPC containing a higher amount of
PFA showed the latent hydraulic property and lower 28 days
where fu : i 2 ug means any subset of f1; 2;    ; dg, which includes i
strength [47]. In comparison, the Sobol indices of CA (coarse aggre-
in it. As suggested in the definition sti
si . The total effect indices
gate) and FA (fine aggregate) are much smaller than the other
measure the contribution to the output variance of xi , including
ingredients. Aggregates are used in HPC as solid fillers and do
all variance caused by its interactions with any other variables. Both
not have hydraulic properties, thus not having strong an influence
the main and the total effective indices are portions of the output
on the strength development of HPC when varying within the sam-
variance that can be attributed to the influence of a single input

 pling ranges. In summary, the Sobol indices of the ingredients, cal-
!
alone. si ¼ sti ¼ 0 suggests that f x is independent of the ith ele- culated with the data simulated from the GP emulator, show good


! agreement with the physicochemical nature of HPC materials. This
ment of the input vector, while si ¼ sti ¼ 1 means f x is only
then further proves that the GP emulator proposed in this study
dependent on the element. can sufficiently represent the real-world HPC performance.
According to the definition, the calculations of exact indices
require computing several integral, which can sometimes be
impractical. For this reason, the Monte Carlo estimation is used
6. Inference of formula
in this framework. Furthermore, indices defined in Eqs. (20) and
(21) are approximated using the estimation proposed by Owen
As the GP model is established, it becomes possible to infer the
[45]. !
n o potential ingredients of a HPC ( x t ) with targeted CS (yt ). According
! ! !
Let define two independent sets of input X ¼ x 1 ; x 2 ;    x n !
n 0 o n ðiÞ ðiÞ o to the Bayes’ theorem, the plausibility of x t can be expressed as
0 ! !0 !0 ! ! !ðiÞ
and X ¼ x 1 ; x 2 ;    ; x n . Let W ðiÞ ¼ w 1 ; w 2 ;    ; w n , in




 !! !
!ðiÞ 0 0 0 0 p yt j y ; x t p x t
which w j ¼ xj;1 ;    xj;i1 ; xj;i; xj;iþ1 ;    ; xj;d ; j ¼ 1;    ; n. The main ! !
p x t j y ; yt ¼ R

 ð24Þ
!! ! !
effect is then calculated using p yt j y ; x t p x t d x t



ðjÞ 
0  2 
Pn ! ! 0
The ingredient search problem then becomes
2n
2n1
1
n i¼1 f x i f w i  E þE
2
þV
þ V 4n

! !
si ¼ ð22Þ argmax p x t j y ; yt . Similar to the logics in determining the GP
V !
x t 2A


And the total effect is calculated using ! !
hyperparameters, argmax p x t j y ; yt is effectively determined
!
Pn

!0 
ðjÞ 2
!

x t 2A
i¼1 f x i  f w i ! !
1
t 2n by p yt j y ; x t . In this study, a Markov chain Monte Carlo sam-
si ¼ ð23Þ
V pling method, the Metropolis-Hastings (MH) algorithm, is applied



  ! ! ! !
where E and V are the mean and the variance estimated using X, to explore the profile of p x t j y ; yt with respect to p yt j y ; x t .
0 0 To illustrate the inference process, a desired CS (37.5 MPa, typ-
whilst the E and V are estimated using X 0 .
ical C30/37 grade concrete) is chosen from the verification dataset.
To estimate the Sobol indices of each ingredient in the HPC, two !
sets of inputs each containing 50,000 pairs of ingredients are gen- Each element in the x t is assumed to be uniformly distributed in
erated. The element in each set of inputs is sampled uniformly, the range of each ingredient in the original database (documented
!
using the latin hypercube sampling (LHS) method, in the range of in Table 1). Meanwhile, the sum of elements in x t always equal to
related ingredient ratio available in the database (Table 1). The unity. 100,000 MH searches are carried out to explore the shape of


! !
results are shown in Fig. 8. Except for the main effect and the total p x t j y ; yt . Such 100,000 MH search cost ~5 min with our in-
t 
effect Sobol indices, the total interactive effect, defined as s i  s i , is house Matlab code using a single core in IntelÒ i7-7600U CPU.
9
X. Ke and Y. Duan
Fig. 8. Sobol indices of the ingredients.
Fig. 9 presents the prior and posterior distributions of the ele-
! !
ments in x t as well as x t , which lead to the maximum


! ! !
p yt j y ; x t in the MH search. Clearly, the elements in x t are
located close to the peak of each pdf, seeing Fig. 9. A factor worth
knowing is that the level of differences between the prior distribu-
tion and the posterior distribution follows the order of Sobol
indices, where higher Sobol indices correlate with larger differ-
ences. For instance, the most obvious difference between the prior
and the posterior distribution is observed in the pdfs of water, of
which Sobol indices are the highest. The prior and posterior distri-
butions of CA almost overlap with each other and the Sobol indices
of CA are much smaller than others and close to zero. The four most
plausible formulae and the original are listed in Table 3. All four
formulae are nearly halved in the usage of cement with increasing
the BSF and PFA usage. It is known that reducing the cement con-
tent can decrease the carbon footprint of the HPC [5]. This suggests
that the Bayesian inference method can be used for searching HPC
formulae with a lower carbon footprint but equivalent strength
performance, improving the sustainability of the final product.
Also, since the posterior distribution of the searched mix design
is generated using the MH stochastic sampling method and the
previously validated GP emulator, the reliability of the proposed
mix design should be the same as the GP surrogate predictive
Fig. 9. Prior and posterior distribution of ingredient. The kernel fit of the posterior distribution and ingredients x t with maximised p yt j y ; x t are also shown.
10
Construction and Building Materials 270 (2021) 121424
model. After obtaining the inferred mix designs, small scale exper-
imental test can be performed using the most plausible mix
designs to further verify the results.
7. Perspectives in model applications
Fig. 10 shows the schematic structure of the informatics-based
materials analysis framework proposed in this study to enable the
computational design of HPC with targeted performance. The
trained and validated predictive surrogate model based on user-
input data will build a robust ingredient-property correlation.
The predictive model is further assessed via the variance-based
Sobol global sensitivity analysis, in order to evaluate the impact
of inputs. The trained ingredient-property correlations that are vio-
lating the physical laws of the materials systems can also be iden-
tified at this stage, where the training of a new predictive model
would be required. Following the sensitivity analysis, the Bayesian
inference method will be applied to infer the plausible material
ingredient formulae, ranked based on the likelihood of reaching
the targeted strength. Before verifying the inferred concrete design
with experimental measurements, an additional assessment and
selection procedure might apply. Unrealistic mix designs that
might be generated through the random sampling process, such
as very low water to binder ratio, can be removed from the inferred


! ! !
X. Ke and Y. Duan Construction and Building Materials 270 (2021) 121424

Table 3
Original ingredient (from verification dataset, experimental data) and first four most plausible ingredient set obtained, results shown in kg/m3, assuming a total concrete density
of 2300 kg/m3, for the targeted strength of 37.5 MPa.


Formula Cement BSF PFA Water SP CA FA ! !
p yt j y ; x t

Original 518.4 0 0 162.8 8.3 804.8 805.7 6.751

INGD-1 270.2 103.3 82.8 188.1 8.0 900.4 746.8 6.376
INGD-2 246.3 92.7 77.1 185.1 6.7 912.6 779.5 5.707
INGD-3 278.8 120.3 84.2 186.5 7.8 886.6 735.8 5.624
INGD-4 269.8 137.3 84.9 192.0 6.2 876.7 733.0 5.615

Fig. 10. Schematic structure of the informatics-based materials analysis framework to enable computational design of HPC with targeted performance.

mix designs. Additional constraints, such as local availability and
cost of specific concrete ingredients can also be taken into consid-
eration and implemented in a weighted decision-making process.
As a data-driven approach, the availability, completeness and qual-
ity of data are vital to the practicality and reliability of this pro-
posed material analysis framework. Future improvement of this
framework might be achieved with a larger concrete design data-
base, including a wider range of alternative supplementary cemen-
titious materials, recycled aggregates, functional fibre
reinforcement. By connecting this proposed framework with addi-
tional database containing the geological distribution of resources
(related concrete ingredients) and their associated cost, the further
development of this framework will be able to expedite the mate-
rial selection process in construction and improve resource effi-
ciency in buildings and infrastructures [22–24].
8. Conclusion
In this work, a novel Bayes’ theorem-powered material analysis
and design framework is proposed for the performance-based
design of high-performance concrete (HPC). This robust design
framework is performed based on Gaussian processes (GP). The
GP emulator, trained on the widely dispersed database, is used to
predict the strength of HPC taking into consideration of seven vari-
ables. The suitability of the GP emulator is rigorously tested using
the LOO-CV test and the validation test. Results of both tests sug-
gest that the GP emulator generated in this study can perform suf-
ficiently satisfactory predictions. The global sensitivity analysis
using the Sobol indices suggests that the GP emulator is not just
capable of catching the trend and statistical features of the exper-
imental data but can also reflect the physicochemical nature of the
HPC materials. The proposed performance-based design method
11
combines the GP emulator with the Bayesian inference method.
It can be used to infer a list of plausible HPC formulae of a given
targeted strength through the stochastic sampling process, the
Metropolis-Hastings (MH) algorithm. To assist the decision-
making among the inferred outcomes, each suggested formula is


! !
evaluated by the likelihood (p yt j y ; x t ) of resulting in the tar-
geted strength. The design framework proposed in this study
enables a faster and more efficient design of HPC; in the mean-
while, significantly reduces the amount of experimental work
required to search for a feasible design. This proposed data-
driven material design framework can also be applied to a wide
range of construction materials, targeting different service
performances.
CRediT authorship contribution statement
Xinyuan Ke: Conceptualization, Funding acquisition, Investiga-
tion, Methodology. Yu Duan: Investigation, Methodology.
Declaration of Competing Interest
The authors declare that they have no known competing finan-
cial interests or personal relationships that could have appeared
to influence the work reported in this paper.
Acknowledgement
The participation of XKe is sponsored by the University of Bath
Prize Fellowship. The authors would like to thank Professor I-
Cheng Yeh for providing the database.
X. Ke and Y. Duan
Replication of Results
The data required to reproduce these findings will be available
upon request.
References
[1] C. Shi, Z. Wu, J. Xiao, D. Wang, Z. Huang, Z. Fang, A review on ultra high
performance concrete: part I. Raw materials and mixture design, Constr. Build.
Mater. 101 (2015) 741–751.
[2] M.C.G. Juenger, F. Winnefeld, J.L. Provis, J.H. Ideker, Advances in alternative
cementitious binders, Cem. Concr. Res. 41 (2011) 1232–1243.
[3] P.C. Aı̈tcin, The durability characteristics of high performance concrete: a
review, Cem. Concr. Compos. 25 (2003) 409–420.
[4] J. Skibsted, R. Snellings, Reactivity of supplementary cementitious materials
(SCMs) in cement blends, Cem. Concr. Res. 124 (2019) 105799.
[5] M.C.G. Juenger, R. Siddique, Recent advances in understanding the role of
supplementary cementitious materials in concrete, Cem. Concr. Res. 78 (2015)
71–80.
[6] V.M. Malhotra, D. Malanka, Performance of Superplasticizers in Concrete:
Laboratory Investigation-Part I, ACI Symposium Publication, 1979.
[7] V.M. Malhotra, Effect of repeated dosages of superplasticizers on slump,
strength and freeze-thaw resistance of concrete, Matériaux et Construction 14
(1981) 79–89.
[8] F. De Larrard, Concrete Mixture Proportioning: A Scientific Approach, CRC
Press, 1999.
[9] I.C. Yeh, Modeling of strength of high-performance concrete using artificial
neural networks, Cem. Concr. Res. 28 (1998) 1797–1808.
[10] A. Öztasß, M. Pala, E.A. Özbay, E.A. Kanca, N. Çaglar, M.A. Bhatti, Predicting the
compressive strength and slump of high strength concrete using neural
network, Constr. Build. Mater. 20 (2006) 769–775.
[11] I.-C. Yeh, Analysis of strength of concrete using design of experiments and
neural networks, J. Mater. Civ. Eng. (2006).
[12] M.H. Fazel Zarandi, I.B. Türksen, J. Sobhani, A.A. Ramezanianpour, Fuzzy
polynomial neural networks for approximation of the compressive strength of
concrete, Appl. Soft Comput. 8 (2008) 488–498.
[13] M.I. Khan, Predicting properties of High Performance Concrete containing
composite cementitious materials using Artificial Neural Networks, Autom.
Constr. 22 (2012) 516–524.
[14] J.-S. Chou, C.-F. Tsai, Concrete compressive strength analysis using a combined
classification and regression technique, Autom. Constr. 24 (2012) 52–60.
[15] J.-S. Chou, A.-D. Pham, Smart artificial firefly colony algorithm-based support
vector regression for enhanced forecasting in civil engineering, Comput.-Aided
Civ. Infrastruct. Eng. 30 (2015) 715–732.
[16] M.-Y. Cheng, J.-S. Chou, A.F.V. Roy, Y.-W. Wu, High-performance concrete
compressive strength prediction using time-weighted evolutionary fuzzy
support vector machines inference model, Autom. Constr. 28 (2012) 106–115.
[17] Z. Yuan, L.-N. Wang, X. Ji, Prediction of concrete compressive strength:
research on hybrid models genetic based algorithms and ANFIS, Adv. Eng.
Software 67 (2014) 156–163.
[18] M. Jalal, Z. Grasley, C. Gurganus, J.W. Bullard, Experimental investigation and
comparative machine-learning prediction of strength behavior of optimized
recycled rubber concrete, Constr. Build. Mater. 256 (2020) 119478.
[19] J.J. Lee, D. Kim, S.K. Chang, C.F.M. Nocete, An improved application technique of
the adaptive probabilistic neural network for predicting concrete strength,
Comput. Mater. Sci. 44 (2009) 988–998.
[20] R. Madandoust, J.H. Bungey, R. Ghavidel, Prediction of the concrete
compressive strength by means of core testing using GMDH-type neural
network and ANFIS models, Comput. Mater. Sci. 51 (2012) 261–272.
[21] Eurocode 2: Design of concrete structures - Part 1-1: General rules and rules
for buildings. European Standard EN 1992-1-1:2004.
12
Construction and Building Materials 270 (2021) 121424
[22] I.C. Yeh, Computer-aided design for optimum concrete mixtures, Cem. Concr.
Compos. 29 (2007) 193–202.
[23] Y. Huang, J. Zhang, F. Tze Ann, G. Ma, Intelligent mixture design of steel fibre
reinforced concrete using a support vector regression and firefly algorithm
based multi-objective optimization model, Constr. Build. Mater. 260 (2020)
120457.
[24] J. Zhang, Y. Huang, F. Aslani, G. Ma, B. Nener, A hybrid intelligent system for
designing optimal proportions of recycled aggregate concrete, J. Cleaner Prod.
273 (2020) 122922.
[25] D.G. Krige, A statistical approach to some basic mine valuation problems on
the Witwatersrand, J. South Afr. Inst. Min. Metall. 52 (1951) 201–203.
[26] R.M. Neal, Priors for infinite networks, in: R.M. Neal (Ed.), Bayesian Learning
for Neural Networks, Springer, New York, New York, NY, 1996, pp. 29–53.
[27] M.A. Chilenski, M. Greenwald, Y. Marzouk, N.T. Howard, A.E. White, J.E. Rice, J.
R. Walk, Improved profile fitting and quantification of uncertainty in
experimental measurements of impurity transport coefficients using
Gaussian process regression, Nucl. Fusion 55 (2015) 023012.
[28] A. O’Hagan, Bayesian analysis of computer code outputs: a tutorial, Reliab. Eng.
Syst. Saf. 91 (2006) 1290–1300.
[29] Y. Duan, C. Cooling, J.S. Ahn, C. Jackson, A. Flint, M.D. Eaton, M.J. Bluck, Using a
Gaussian process regression inspired method to measure agreement between
the experiment and CFD simulations, Int. J. Heat Fluid Flow 80 (2019) 108497.
[30] D.D. Cox, J.-S. Park, C.E. Singer, A statistical method for tuning a computer code
to a data base, Comput. Stat. Data Anal. 37 (2001) 77–92.
[31] O.T. Kajero, R.B. Thorpe, T. Chen, B. Wang, Y. Yao, Kriging meta-model assisted
calibration of computational fluid dynamics models, AIChE J. 62 (2016) 4308–
4320.
[32] T. Lookman, P.V. Balachandran, D. Xue, J. Hogden, J. Theiler, Statistical
inference and adaptive design for materials discovery, Curr. Opin. Solid State
Mater. Sci. 21 (2017) 121–128.
[33] H. Kim, J. Inoue, T. Kasuya, M. Okada, K. Nagata, Bayesian inference of ferrite
transformation kinetics from dilatometric measurement, Comput. Mater. Sci.
184 (2020) 109837.
[34] P.V. Balachandran, Machine learning guided design of functional materials
with targeted properties, Comput. Mater. Sci. 164 (2019) 82–90.
[35] C.E. Rasmussen, C.K.I. Williams, Gaussian Processes for Machine Learning, MIT
Press, 2006.
[36] V.R. Joseph, Y. Hung, A. Sudjianto, Blind kriging: a new method for developing
metamodels, J. Mech. Des. 130 (2008).
[37] I. Couckuyt, A. Forrester, D. Gorissen, F. De Turck, T. Dhaene, Blind Kriging:
implementation and performance analysis, Adv. Eng. Software 49 (2012) 1–13.
[38] S. Roberts, M. Osborne, M. Ebden, S. Reece, N. Gibson, S. Aigrain, Gaussian
processes for time-series modelling, Philos. Trans. R. Soc. London, Ser. A 371
(2013) 20110550.
[39] F. Gao, L. Han, Implementing the Nelder-Mead simplex algorithm with
adaptive parameters, Comput. Optim. Appl. 51 (2012) 259–277.
[40] M. Tokyay, Strength prediction of fly ash concretes by accelerated testing,
Cem. Concr. Res. 29 (1999) 1737–1741.
[41] I.C. Yeh, L.-C. Lien, Knowledge discovery of concrete material using Genetic
Operation Trees, Expert Syst. Appl. 36 (2009) 5807–5812.
[42] I.M. Sobol0 , Global sensitivity indices for nonlinear mathematical models and
their Monte Carlo estimates, Math. Comput. Simul. 55 (2001) 271–280.
[43] J. Nossent, P. Elsen, W. Bauwens, Sobol’ sensitivity analysis of a complex
environmental model, Environ. Modell. Software 26 (2011) 1515–1525.
[44] D. Prieto, M.I. Asensio, L. Ferragut, J.M. Cascón, Sensitivity analysis and
parameter adjustment in a simplified physical wildland fire model, Adv. Eng.
Software 90 (2015) 98–106.
[45] A.B. Owen, Variance Components and Generalized Sobol’ Indices, SIAM/ASA J.
Uncertainty Quantification, 1 (2013) 19-41.
[46] A. Oner, S. Akyuz, An experimental study on optimum usage of GGBS for the
compressive strength of concrete, Cem. Concr. Compos. 29 (2007) 505–514.
[47] D. Zhang, X. Cai, Y. Shao, Carbonation curing of precast fly ash concrete, J.
Mater. Civ. Eng. 28 (2016) 04016127.