PROOF OF CONCEPT FOR THE BAYESIAN ANALYSIS OF COMPUTER CODE

FOR BUILDING ENERGY MODELLING

Michael Wood, Matthew Eames, and Peter Challenor
College of Engineering, Mathematics and Physical Sciences, University of Exeter, Exeter, UK




ABSTRACT
Building energy simulations are computationally
expensive processes. One method of reducing the
computation time is to create meta-models or
emulators, but these simplified models often sacrifice
precision for speed. Recently, regression based
emulators have proven popular (Jenkins, Patidar,
Banfill, & Gibson, 2011), however, this paper
presents an alternative approach to emulation based
on the Bayesian analysis of computer code output
(BACCO).

The Bayesian building emulator (BBE) detailed in
this paper is based on knowledge of 30 training
simulations, which vary the buildings glazing ratio,
orientation and insulation thickness. Validation
shows that the emulator represents the uncertainty
surrounding the untested simulator inputs well.
INTRODUCTION
Dynamic models of buildings using an electrical
analogy originated in the 1950s (Burnand, 1952), the
ideas of which are still being used in today's
computational building models. Building models
vary in both complexity and accuracy, and are
generally used in industry to comply with building
regulations and to assist in make decisions about the
design of the building (King, 2010). Early computer
models were, by necessity, simplified versions of the
thermal behaviour of real buildings (Achterbosch,
Jong, Meulen, & Verberne, 1985), but as
computational power has increased, so has the
complexity of the models themselves. Modern
simulators such as Energy Plus and IES Virtual
Environment are based on physical principles making
them credible to engineers and building scientists.
However, the added computational power can restrict
the number of iterations of a model in an
optimisation process. Such complexity is therefore
likely to prohibit an exhaustive search of all possible
variations of the input parameter space.

One method for speeding up the optimisation process
is to reduce the number of simulations needed to
reach an optimal solution. This can be achieved by
using genetic algorithms or other optimal search
techniques (Siddharth, Ramakrishna, Geetha, &
Sivasubramaniam, 2011). Another method is to
create a meta-model which can complete simulations
faster than the simulator. Meta-models can be created
by using neural networks (Kumar, Aggarwal, &
Sharma, 2013), with regression based meta-models
also having been shown to be successful (Jenkins et
al., 2011). More recently, supercomputing techniques
have been used to speed up the simulation process by
brute force (Greenberg et al., 2013).

In this paper we present a meta-model (emulator)
which was created by using Bayesian analysis
(OHagan, 2006). The proposed emulator is a
statistical model of the building simulator and has
several advantages over traditional meta-modelling
techniques. Like all meta-modelling techniques, the
Bayesian emulator requires a number of training runs
to learn about the simulator it is modelling. The
emulator produced is fast, accurate and has an
advantage over regression methods in that it includes
a measure of its own uncertainty. However, the
creation of the emulator is based on the assumption
that the output of the simulator can be modelled as a
Gaussian Process (GP).

Examples of previous work using Bayesian
emulators to model building performance have
focussed on particular building elements, such as
glazing (Kim, Ahn, Park, & Kim, 2013), cooling
systems (Kang, Kim, Ahn, & Park, 2013) or applied
to specific complex problems such as computational
fluid dynamics (CFD) (Tagade, Jeong, & Choi,
2013). The work detailed in this paper demonstrates
a Bayesian emulator which accounts for the effect of
several non-related building parameters (i.e. glazed
area, insulation thickness and orientation).
METHOD
The Bayesian meta-model of a building simulator is
created using a four-stage process:

Run 'training simulations' of the building
model to 'learn' about the relations between
the input and the output.
Use the results of the training simulations to
train the emulator.
Validate the emulator by completing
additional runs of the building model to
compare to the output of the emulator.
Use the emulator to make inferences
regarding the output of the simulator.

To avoid confusion, the original building model is
always referred to as the simulator and the newly
created model of the simulator is always referred to
as the emulator. The building simulator is based on a
three-zoned thermal network model of a two-storey
building with a roof space, which is glazed to one of
the faades (Figure 1). Thermal network models
allow any building to be represented by an analogy
with an electrical model and then solved as a series
of first order differential equations. Such models
have been extensively used to solve such thermal
modelling problems in the past as the calculations are
completed analytically rather than numerically. The
system consists of a series of nodes, which are
coupled by conduction and convection and the
thermal storage is represented by a capacitor
(Mathews, Richards, & Lombard, 1994). The heat
flow by radiation is represented by a current source
on the node where the radiation is incident and the
node voltage represents its temperature. Each layer of
the construction contributes to the dynamic response
of the building and is modelled as an independent
element when creating the thermal model. The nodes
and boundary conditions of the thermal elements are
determined from the diffusivity of the material and
time step of the dynamic model (Tabares-Velascoa &
Griffith, 2012).


Figure 1: The building model
The building model is shown in Figure 1. Each
building story is a heated zoned of dimensions 8 m !
8 m ! 2.4 m and has a constant air exchange rate of
0.2 air changes per hour. The roof is pitched at an
angle of 30 with no overhang. The roof space is
unheated and has a constant air exchange rate of
three air changes per hour. For simplicity, at each
time step, the heating and cooling loads are
calculated for each heating zone to maintain the
internal temperature at 21 C using an ideal plant
with an unlimited capacity. The purpose of this work
is to demonstrate the effectiveness of the emulation
procedure to represent the simulation of the building
model as such this simplification is appropriate. The
thermal properties for the materials used for the
construction for each surface are shown in Table 2
and the glazing system is shown in Table 1. The
building is located in Plymouth and the standard
CIBSE Test reference year is used. No other internal
gains are considered.

The building model created allows the glazed area,
insulation thickness (of the external wall only) and
orientation of the building to be varied. The glazed
area, insulation thickness and the orientation form the
input to the simulator.

CREATING A SIMPLE BAYESIAN
EMULATOR
The Bayesian Building Emulator (BBE) treats the
building simulator as a multi-input function with one
output (in this case the annual energy use). The
emulator is based on the assumption that we can
model the output of the simulator as a Gaussian
Process. Representing the output in this way is a pre-
requisite of the Bayesian method. This is because
complete probability distributions are required to
specify uncertainty, and Gaussian Processes are by
far the simplest way of achieving this. The emulator
outputs are therefore represented as multivariate
normal (Gaussian) distributions.

To create the BBE, the building simulator is
represented by a function, !!!!, where ! is the input
vector of a particular input configuration and !!!! is
the output of the simulator (in this case the annual
energy use). If we run a number of training
simulations, this gives us a number of values for !
for which we know the output (!!!!). These training
inputs (!, where ! ! !), each have an associated
output (!!!!).

GLAZING LAYER GLASS 1
ARGON
CAVITY
GLASS 2
Thickness / mm 4 16 4
Thermal
Conductivity /
Wm
-1
K
-1

1.06 0.016 1.06
Transmittance 0.899 - 0.672
Forward Reflectance 0.08 - 0.188
Backward
Reflectance
0.08 - 0.163
External emissivity 0.837 - 0.059
Internal emissivity 0.837 - 0.837
Table 1: Properties of the glazing system used in the
building model. The frame makes up 20 % of the
glazing area and has a U-value of 2 Wm
-2
K
-1
and a
linear transmittance of 0.08 Wm
-1
K
-1
.


S
U
R
F
A
C
E

/

M
A
T
E
R
I
A
L

T
H
I
C
K
N
E
S
S

/

M
M

T
H
E
R
M
A
L

C
O
N
D
U
C
T
I
V
I
T
Y

/

W
M
-
1
K
-
1

D
E
N
S
I
T
Y

/

K
G
M
-
3

H
E
A
T

C
A
P
A
C
I
T
Y

/

J
K
G
-
1
K
-
1

External Wall
Cast concrete 220 1.13 2000 1000
Insulation 10-100 0.038 25 1030
Cast concrete 220 1.13 2000 1000
Ground floor
London Clay 750 1.41 1900 1000
Brick 220 0.77 1750 1000
Concrete 100 1.13 2000 1000
Insulation 80 0.025 30 1400
Chipboard 25 0.15 800 2093
Carpet 10 0.06 160 2500
Internal Floor/ceiling
Carpet 10 0.06 160 2500
Chipboard 25 0.15 800 2093
Cavity 100 - - -
Insulation 10 0.04 15 1300
Plaster Board 13 0.16 600 1000
External Roof
Clay tile 5 0.84 1900 800
Glass fibre 25 0.04 12 840
Roofing felt 5 0.19 960 837
Insulation 180 0.043 12 840
Plaster Board 13 0.16 600 1000
Table 2: Thermal properties of the constructions
used in the building model.
The BBE represents the output as a mean function
(!!!!) and a covariance function (!!!! !!!)
(Kennedy & OHagan, 2001). The mean function is
the expected value of !!!! and the covariance
function provides a measure of the uncertainty
around the mean function. The form of both these
functions is set prior to running the simulator and are
then modified by hyper-parameters
1
following the
emulator training phase. Note that the posterior mean
will always evolve so that it passes exactly through
the training set where the emulator uncertainty goes
to zero.

The mean function takes the form ! ! ! !!!!
!
!
where ! ! ! !! !
!
! !
!
! !
!
!
and ! is a vector of
four hyper-parameters, each associated with a
column in !!!!. The covariance function is defined
as ! !! !! ! !
!
!!!! !
!
!, where !!!! !!! is the
correlation function. For the purposes of the simple
one-dimensional BBE, a Matrn function (with the
shape parameter set to 3/2) is used as the correlation
function:

! !! !
!
! ! ! ! !
!!!!
, where

1
Note that in Bayesian statistics, the term hyper-parameter
describes a parameter in the prior distribution rather than a
parameter of the computer model itself. It is therefore
important to note that these hyper-parameters have no
direct association to the physical parameters in the model.
! ! ! ! !
! !
!!! ! !
!
!
!
!


The matrix ! is a diagonal matrix of the correlation
lengths !
!
, where !
!!
! !
!
!!
.

Each dimension of the input will have an associated
!
!
,

which defines the 'roughness' of the output. A low
value of ! implies a more rough output than a higher
value, which in turn implies the output is smoother
for variation in this input dimension. Like the other
hyper-parameters of ! and !, values for ! must all be
estimated based on the results of the training
simulations.
Estimating the Hyper-Parameters
For the BBE, ! and ! are determined analytically for
a given !, and only ! needs to be changed to estimate
the other parameters. The most likely values of ! for
a given training set (and hence ! and !) are
determined by maximising the log likelihood
function (Bastos & OHagan, 2009).

!" !
!
!
!!! ! !!!! ! ! ! !" !
!
! !!! !" ! !
!!!!" ! !
!
!
!!
! !, (2)

where ! ! !!!" !!!, ! ! !!!
!
! (note ! ! !
!! !
!
in our model), ! is the number of training
inputs, ! is the number of elements in ! and
!
!!!
! !!!
!
! !
!
!. The function !
!
!
!!! is the posterior
distribution function of ! (and hence !) with !
!

calculated from

!
!
!
!! ! ! ! !!
!!
! !
!
!
!!
!
!
!!
! !
!
!
!!
!
!!
!
!
!
!!
!!!!. (3)

The log likelihood function was maximised
numerically by calculating !" !!
!
!
!!!! for different
trial combinations of !. To do this it was assumed
that each element of delta would lie between 0.1 and
20 respectively and would be equally likely to fall
between these two values (thereby assuming a
uniform prior between these two values). Due to the
computational cost of calculating the log likelihood
function for each delta, a two-stage search was used.
First, a coarse-grid search was used, followed by a
fine-grid search around the maximum of the initial
search. The resolution of both these grid searches is
10x10x10.

The value of ! that yields that largest value for
!" !!
!
!
!!!! was carried forward (!). Using !! ! was
calculated from equation 3 and ! was calculated
from ! ! !
!
!
!!
!
!!
!
!
!
!!
!!!!.

The Posterior Mean and Covariance Function
The posterior mean !
!
!!! and covariance function
!
!
!!! !!! are defined by the following equations (J.
Oakley, 2004).

!
!
! ! !!!!
!
! ! ! !! !
!
!
!!
!! ! ! !!!, (4)

!
!
!! !
!
! !
!
! !! !
!
! ! !
!
!
!!
! !
!
!
!! !
!
! ! !
!
!
!!
!!!!
!
!
!!
!!
!!
!! !!
!
!
! !!
!
!
!!
!!
!
, (5)

Detailing the derivation of equations 4 and 5 would
take up too much space here, but interested readers
will find O'Hagan's tutorial (OHagan, 2006) a good
general introduction to Bayesian emulation. Further
information on how the log likelihood functions and
posterior mean and variance functions are linked to
Bayes Theorem can be found in Gaussian Processes
for Machine Learning appendix A1 and A2 by CE
Rasmussen and CKI Williams (Rasmussen &
Williams, 2006)



Figure 2: An emulator trained from five simulations
points (orientation = 0 radians, glazed area = 50%).
The Matrn (!
!
!
!
!
) correlation function is used to
evaluate the mean function and the covariance
function.
Simple One Dimensional Emulator
Figure 2 shows the results of a simple one input / one
output emulator which models variation in the
external wall insulation thickness. The original
output of the simulator is shown by the blue circles
(training points), with the true range of the simulator
output shown by the blue line (solid). The red (close-
dotted) line shows the posterior mean function and
the two red (long-dotted) lines show the 95%
confidence intervals.

Table 3 shows the hyper-parameters and the training
simulation results associated with this emulator.
These values provide the information to be able to
evaluate the posterior mean and covariance functions
that constitute the Bayesian Emulator.
Table 3: Variables and values for the simple
emulator
VARIABLE VALUE
! (scaled
between 0
and 1)
[0.0, 0.11, 0.22, 0.33, 0.44, 0.56, 0.67, 0.78,
0.89, 1.0]
!!!! (kWh)
(scaled
between 0
and 1)
[23,390, 16,153, 13,915, 12,859, 12,249,
11,855, 11,579, 11,376, 11,220, 11,097]
T

!
0.95
!
0.149 (note that this value of sigma is based on
FD being scaled by dividing it by its maximum)
!
[1.12, 0.67]
T


Although the building emulator shown in Figure 2
provides good emulation, it can be improved by
adding more training simulation points. Figure 3
shows improved accuracy with the addition of five
additional training points.


Figure 3: An emulator trained from twelve
simulations points (orientation = 0 radians, glazed
area = 50%)

Training a Bayesian emulator therefore requires a
balance between number and location of training
simulations and emulator accuracy. The more
training simulations undertaken, the more accurate
the emulator is likely to be, but the more time the
emulator will take to create.
Creating a Three Dimensional Bayesian Emulator
A three-dimensional emulator Bayesian emulator was
created to examine the effect of changing three
inputs:

Percentage glazed area (between 50-90%)
on one surface.
External wall insulation thickness (between
10 100 mm)
Orientation of glazed surface (between -!/2
and +!/2 radians, where 0 radians is due
South)

The three input parameters create a three-
dimensional input space. In order to train the
emulator well, the input configurations need to be
chosen carefully. Given that we would like the
simulator to be run as few times as possible, it is
important that the input space is 'filled' efficiently.

When training the one dimensional simulator given
in the previous example, providing a 'good fill' of
training variables was trivial, since the most efficient
fill is achieved by equally spaced inputs. However,
for the three dimensional input space a more
sophisticated Latin Hypercube design process
approach is required. For the Latin Hypercube design
process, it has been shown that 10 training
simulations per input dimension provides a good
balance between minimising the number of training
simulations and providing enough information for the
emulator (Loeppky, Sacks, Welch, & Welch, 2009).
Since the model in question has three input
dimensions (glazed area, insulation thickness and
orientation), 30 training simulations were used. To
generate the input configurations, 100 random, 30-
point Latin Hypercubes and the hypercube with the
maximum minimum distance between the design
points was selected.

The input variables are scaled to range between 0 and
1 to simplify both the creation of the Latin
Hypercube and the mathematics of the emulator. The
simulator was then run for each of the 30 input
configurations. Maximising the log likelihood
function yielded the hyper-parameters shown in
Table 4.
Table 4: Parameters for the 3D emulator
HYPER-
PARAMETER
VALUE
! [0.95, 0.12, 0.02, -0.42]
T

! [3.42, 19.78, 0.76]
! 0.0073

RESULTS
To check that the emulator was performing as
expected, it was subjected to a number of validation
procedures (Bastos & OHagan, 2009).
Individual Standardised Errors
The individual standardised errors provide a measure
of the accuracy of the emulator. However, they
require additional validation simulations to calculate
them. The standardised errors (!) are given by,

!
!
!
! !
!
!
!!
!
!!
!
!
!
!
!
!!
!
!
!!
!
!
!
, (6)

where !
!
are the individual standardised errors, !
!
!
are
the individual simulator inputs for the validation
samples, and ! is the index of each sample (where
! ! !! !! ! !!!).

A set of 15 validation simulations (!!) were used to
calculate the individual standardised errors. The
validation inputs in !! were generated using the same
Latin Hypercube design process that was used for the
original training set.

Since the emulator provides both a mean (the
expected value) and variance for each input, it is
important to note that we do not expect all the
simulation points to lie on the mean function, but to
be distributed about the mean function according to
the emulators covariance function. Therefore, if the
building emulator is accurate, then on average 95%
of the standardised errors would be within 2 standard
deviations of the mean. For the validation samples,
93.4% of the errors lay within two standard
deviations of the mean function. This result
demonstrates that there is a high probability that the
emulator is representing the simulator output well by
providing a good estimation of its own uncertainty
(for a normal distribution, 95% of the residuals
would be within two standard deviations of the
mean).

Calculating the Mahalanobis Distance
The Mahalanobis distance (!) is a scalar value that
provides an alternative means of measuring the
validity of the emulator (Bastos & OHagan, 2009).
! is calculated using the simulation and emulation
results for the validation sample (!!).

! ! ! !
!
! !
!
!!!!
!
!
!
!
!
! !!
!!
! !
!
!
!
!
!!!! , (7)

For a valid emulator, ! will have a scaled F-
distribution with degrees of freedom given by !! and
!! ! !!, and an expected value of !!. For the
validations points considered, the expected value of
! is 15 with a variance of 53.2. The Mahalanobis
distance for the validation samples was 15.7, which
falls close to the mean of the expected distribution of
! (Figure 4).

Figure 4: The Mahalanobis Distance of the
validation sample (vertical line) in relation to its
predicted probability density function
Comparing the Simulator to the Emulator over a
Larger Number of Points
A !!!!! grid of equally spaced input points (!
!"#
)
was simulated to further test the validity of the
building emulator. The simulator results were then
compared to those of the emulator. Figure 5 shows a
histogram of the standardised errors in the emulation
over the 729 points, and Figure 6 shows the location
of emulation inputs the simulation output is more
than two standard deviations from the mean function.
Figure 7 and Figure 8 compare the results of the
emulator and simulation.


Figure 5: Histogram of the standardised errors
between the emulator and the simulator based on 729
simulations and emulations.
Figure 6 shows that most of the points where the
simulator outputs are more than two standard
deviations from the mean function are at the edges
of the input space and represent 5.5% of the input
values tested. It is expected that around 5% of
emulator outputs would be greater than two standard
deviations from the true value (given the probabilistic
nature of the emulator).

Figures 7 and 8 show how closely the trends in the
output of the emulator match those of the simulator.


Figure 6: Configurations of the input points were the
true output of the simulator is more than 2 standard
deviations from the mean function (as predicted by
the covariance function of the emulator)
DISCUSSION AND CONCLUSION
This paper has shown that a Bayesian Emulator can
represent the modelled energy use of a simple
building. Based on a training set of 30 simulations,
the validation diagnostics show that the emulator's
mean and variance predictions fit the modelled data
well. Further interrogation of the emulator by using a
brute-force method showed that 94.5% percent of
729 simulation values were within two standard
deviations of the mean function, a finding that in
agreement with the uncertainty specified by the
emulator.

The validation diagnostics show that the emulator is
valid, however, some areas of the emulator creation
need improvement. Such improvements include an
improved method for estimating the hyper-
parameters, particularly !.

One of the major advantages of Bayesian emulation
is the ability to identify key trends and patterns in the
output of the simulator with only a limited number of
simulator runs. Figure 7 and Figure 8 show that
trends in the output of the emulator closely match
those of the simulator. Patterns identified by the
building emulator include:

The orientation of the building is not as
important as might have been expected in
determining the total heating and cooling
load (i.e. there is not much variation
observed in the orientation axis).
An increase in glazed area leads to a linear
increase in the total amount of energy used
(combined heating and cooling).
Increasing the amount of insulation used in
the building provides a very steep reduction
in the amount of energy used from around
10 mm to 40 mm, but between 40 mm and
100 mm the increasing thickness of the
insulation appears to lessen in effect.

The Bayesian methodology also allows a variety of
analyses, which are usually too computationally
expensive to perform with the building simulator.
These analyses include applying MC analysis to the
emulator mean and MC uncertainty analysis (i.e.
assessing the effect of uncertainties in input values
on the output) as well as sensitivity analyses (J. E.
Oakley & OHagan, 2004) and model calibration
(Kennedy & OHagan, 2001).

MC analysis using the emulator rather than the
simulator can achieve high levels of accuracy in
significantly quicker time than could be achieved
with the computationally expensive simulator. For
example, using the analysis of 729 simulation points
discussed earlier, a MC evaluation of the emulation
mean for these points reveals a value of 16,018 kWh
for the annual energy use. For the same set of
simulation samples, the simulator has a mean of
16,039 kWh, which represents a difference of 0.13%
from the output of the emulator. Following the 30
training simulations, the emulator is !!!!!"
!
times
quicker than the simulator in producing this output.

The research has demonstrated that a Bayesian
Building emulator can be produced with a limited
number of training runs, allowing a statistical
analysis of the building to be completed quickly.
Further work will include the investigation of the
effects of varying other input parameters to the
building model, and improvements to the accuracy of
the emulator. In addition to this, we will expand the
emulator to include dynamic inputs, along with
improvements to the ! search function. Dynamic
inputs will add significant additional capabilities to
the emulator, since we can consider then consider
time-varying effects.

NOMENCLATURE
!, simulator / emulator input;
!!!!, function representing the simulator;
!, training data;
!!, validation data;
!, Mahalanobis distance;
!", Monte Carlo;
!
!
, standardised errors;
ACKNOWLEDGEMENTS
This work was supported by the Engineering and
Physical Sciences Research Council [EPSRC grant
number EP/J002380/1].


Figure 7: Comparison between the simulator (left) and the emulator (right) for a fixed glazing ratio of 50%

Figure 8: Comparison between the simulator (left) and the emulator (right) for a fixed orientation of
!!
!
radians
REFERENCES
Achterbosch, G. G. J., Jong, P. P. G. De, Meulen,
S. F. Van Der, & Verberne, J. (1985). The
Development of a Convenient Thermal
Dynamic Building Model, 8, 183196.
Bastos, L. S., & OHagan, A. (2009). Diagnostics
for Gaussian Process Emulators.
Technometrics, 51(4), 425438.
doi:10.1198/TECH.2009.08019
Burnand, G. (1952). The study of the thermal
behaviour of structures by electrical analogy.
British Journal of Applied Physics, 50(3), 50
53.
Greenberg, D., Pratt, K., Hencey, B., Jones, N.,
Schumann, L., Dobbs, J., Walter, B.
(2013). Sustain: An experimental test bed for
building energy simulation. Energy and
Buildings, 58, 4457.
doi:10.1016/j.enbuild.2012.11.026
Jenkins, D. P., Patidar, S., Banfill, P. F. G., &
Gibson, G. J. (2011). Probabilistic climate
projections with dynamic building
simulation: Predicting overheating in
dwellings. Energy and Buildings, 43(7),
17231731.
doi:10.1016/j.enbuild.2011.03.016
Kang, J.-E., Kim, Y., Ahn, K.-U., & Park, C.-S.
(2013). Gaussian Proces Emulator for
Optimal Operation of a High Rise Office
Building. In 13th Conference of International
Building Performance Simulation Association
(pp. 22252231).
Kennedy, M. C., & OHagan, A. (2001). Bayesian
Calibration of Computer Models. Journal of
the Royal Statistical Society: Series B
(Statistical Methodology), 63(3), 425464.
Kim, Y.-J., Ahn, K.-U., Park, C.-S., & Kim, I.-H.
(2013). Gaussian Emulator for Stochastic
Optimal Design of a Double Glazing System.
In 13th Conference of International Building
Performance Simulation Association (pp.
22172224).
King, D. (2010). Engineering a low carbon built
environment: The discipline of Building
Engineering Physics.
Kumar, R., Aggarwal, R. K., & Sharma, J. D.
(2013). Energy analysis of a building using
artificial neural network: A review. Energy
and Buildings, 65, 352358.
doi:10.1016/j.enbuild.2013.06.007
Loeppky, J. L., Sacks, J., Welch, W., & Welch, W.
J. (2009). Choosing the Sample Size of a
Computer Experiment: a Practical Guide.
Technometrics, 51(4), 366376.
doi:10.1198/TECH.2009.08040
Mathews, E., Richards, P., & Lombard, C. (1994).
A first-order thermal model for building
design. Energy and Buildings, 21, 133145.
OHagan, A. (2006). Bayesian analysis of computer
code outputs: A tutorial. Reliability
Engineering & System Safety, 91(10-11),
12901300. doi:10.1016/j.ress.2005.11.025
Oakley, J. (2004). Estimating percentiles of
uncertain computer code outputs. Journal of
the Royal Statistical Society: Series C
(Applied Statistics), 53(1), 8393.
doi:10.1046/j.0035-9254.2003.05044.x
Oakley, J. E., & OHagan, A. (2004). Probabilistic
sensitivity analysis of complex models: a
Bayesian approach. Journal of the Royal
Statistical Society: Series B (Statistical
Methodology), 66(3), 751769.
doi:10.1111/j.1467-9868.2004.05304.x
Rasmussen, C. E., & Williams, C. K. I. (2006).
Gaussian processes for machine learning.
International journal of neural systems (Vol.
14, pp. 199201).
Siddharth, V., Ramakrishna, P. V., Geetha, T., &
Sivasubramaniam, A. (2011). Automatic
generation of energy conservation measures
in buildings using genetic algorithms. Energy
and Buildings, 43(10), 27182726.
doi:10.1016/j.enbuild.2011.06.028
Tabares-Velascoa, P. C., & Griffith, B. (2012).
Diagnostic test cases for verifying surface
heat transfer algorithms and boundary
conditions in building energy simulation
programs. Journal of Building Performance
Simulation, 5(5), 329346.
Tagade, P. M., Jeong, B.-M., & Choi, H.-L. (2013).
A Gaussian process emulator approach for
rapid contaminant characterization with an
integrated multizone-CFD model. Building
and Environment, 70, 232244.
doi:10.1016/j.buildenv.2013.08.023