Applied Energy 214 (2018) 219–238

Contents lists available at ScienceDirect

Applied Energy
journal homepage: www.elsevier.com/locate/apenergy

Uncertainty and global sensitivity analysis for the optimal design of T

distributed energy systems
⁎
Georgios Mavromatidisa,b, , Kristina Orehouniga,b, Jan Carmelieta,c
a
Chair of Building Physics, ETH Zurich, Switzerland
b
Laboratory for Urban Energy Systems, Empa Duebendorf, Switzerland
c
Laboratory for Multiscale Studies in Building Physics, Empa Duebendorf, Switzerland

H I G H L I G H T S

• AA framework for uncertainty investigation in model-based DES design is presented.

• Probabilistic
deterministic DES design model based on the energy hub concept is formulated.
• Monte Carlo uncertainty
descriptions of all the relevant uncertain parameters are developed.
• Two-step sensitivity analysis
analysis examines the variation of the design model outputs.
• identifies the most influential uncertain parameters.

A R T I C L E I N F O A B S T R A C T

Keywords: The effective design of Distributed Energy Systems (DES) is subject to multiple uncertainties related to aspects
Distributed energy system like the availability of renewable energy, the building energy demands, and the energy carrier prices.
Energy hub optimization Nevertheless, current practices involve the use of deterministic design models, which overlook uncertainty and
Monte Carlo uncertainty analysis can lead to suboptimal DES configurations that fail to deliver the desired performance.
Global sensitivity analysis
A necessary condition in order to obtain robust DES designs against uncertainty is the understanding of
Morris screening
uncertainty’s impacts and main drivers. Therefore, this paper presents a novel methodological framework for the
Sobol indices
investigation of uncertainty in the context of DES design, which combines optimisation-based DES models and
techniques from Uncertainty Analysis (UA) and Global Sensitivity Analysis (GSA). Moreover, the application of
the framework is illustrated with a case study for the optimal DES design of a Swiss urban neighbourhood.
Embarking from a deterministic DES design model, first, all sources of uncertainty are identified and ap-
propriate probabilistic descriptions are assigned to all uncertain model parameters. UA is then performed using
Monte Carlo (MC) simulations to quantify the impacts of uncertainty. Results reveal substantial variations in
terms of economic and carbon performance of the optimal DES, but also in terms of optimal DES configurations.
Moreover, the UA results indicate that the optimal system costs are mostly higher than the deterministic cost
estimates, while the inverse is observed for the case of carbon emissions. Additionally, in many of the MC
simulations, the resulting optimal DES configurations deviate significantly from the deterministically obtained
designs, which confirms the risk of suboptimal decisions in deterministic design processes. Moreover, the results
of UA can function as decision support by identifying the DES configurations that are optimal for most rea-
lisations of uncertainty.
Finally, to identify the uncertain parameters that are mostly responsible for the variation of the economic
performance of the DES, a two-step GSA is launched, combining the Morris method and the variance-based Sobol
method. Results of the GSA indicate the energy demand patterns and the energy carrier prices as primarily
responsible for the variability of the optimal system cost, while parameters like the investment costs and the
technical characteristics of the technologies exert only minimal influence. The results of GSA, besides offering a
better understanding of uncertainty to DES designers, also identify the parameters for which additional effort
needs to be invested to reduce their uncertainty and, as a result, the uncertainty associated with DES design.

⁎
Corresponding author at: Chair of Building Physics, ETH Zurich, Stefano-Franscini-Platz 1, 8093 Zurich, Switzerland.
E-mail address: gmavroma@ethz.ch (G. Mavromatidis).

https://doi.org/10.1016/j.apenergy.2018.01.062
Received 7 October 2017; Received in revised form 18 December 2017; Accepted 22 January 2018
0306-2619/ © 2018 Elsevier Ltd. All rights reserved.
G. Mavromatidis et al.
1. Introduction
1.1. Background and previous work
One goal of the United Nations’ ‘2030 Agenda for Sustainable
Development’ [1] is the transition to a more efficient, sustainable energy
future that will include high shares of renewable energy in the global
energy mix. A promising pathway towards this vision lies in the
adoption of Distributed Energy Systems (DES). DES are placed in
proximity to the energy end-use sector they serve, hence, minimising
energy transmission losses [2] and incorporating locally available en-
ergy resources. Moreover, they typically incorporate multiple energy
carriers and renewable and other efficient technologies that convert,
store and deliver energy in the form of heating, cooling, and electricity
or as other energy carriers (e.g. hydrogen) [3], which allows them to
increase operational flexibility Overall, DES deliver a series of eco-
nomic, environmental and technical benefits as discussed in [3–5].
The effective design of DES is critical for the realisation of their
benefits. As a result, mathematical optimisation models are commonly
developed to assist with their design [6–26]. For instance, Merkel et al.
[14] developed an optimisation model for the design of cost-optimal
micro-CHP systems for a set of residential buildings. Carvalho et al.
[20] developed an optimisation model to determine the optimal
synthesis of a hospital’s trigeneration plant investigating the trade-off
between system costs and carbon emissions. Finally, Di Somma et al.
[26] presented a model for the design of a DES for 30 residential
buildings with the major innovation of their study being the integration
of exergetic objectives in the design. All these DES design models,
though, are characterised as deterministic, because they assume perfect
knowledge of all model parameters. However, the DES design process
combines multiple economic, technical, and policy aspects, which can
render model input parameters uncertain. Such aspects include the
stochastic nature of renewables and the unknown future energy carrier
prices, among others.
Uncertainty in DES design introduces additional complexity along-
side the risk of suboptimal decisions in cases when the actual parameter
values deviate from the ones assumed during the design. Under such
circumstances, it is beneficial to understand how uncertainty influences
the optimal DES designs and to identify the most influential uncertain
parameters. Uncertainty Analysis (UA) and Sensitivity Analysis (SA) are
methods that are exactly aimed to address such concerns.
UA investigates the variability of a model’s output given uncertain
input parameters; hence, it allows modellers to answer the question
“How uncertain is my model output?” UA is mostly performed via Monte
Carlo (MC) simulations in which a deterministic model is evaluated re-
peatedly for different uncertain parameter samples. UA then seeks to
statistically characterise the variable model output by calculating sta-
tistical measures, identifying patterns etc.
A number of studies have performed UA in the context of DES. MC
analysis of the optimal dispatch of an energy hub system is performed
in [27] considering uncertain energy carrier prices. Mavrotas et al. [28]
performed UA for the design of a DES considering uncertain discount
rate and energy carrier prices. In [29], UA is performed for the design of
a multi-generation system considering uncertainty pertaining to in-
vestment costs, the price of ethanol, and the CO2 emissions displaced by
ethanol. Besides economic parameters, UA was used in [30] to in-
vestigate the impact of wind power uncertainty on the design of a mi-
crogrid. Similarly, Li et al. [31] performed MC simulations to account
for uncertain energy demands in the design of a trigeneration system.
Finally, Dufo-López et al. [32] investigated the uncertainties of solar
radiation and energy demands for the design of an off-grid energy
system using UA.
Overall, though, the number of studies performing UA remains
limited, while the focus is commonly placed only on a few uncertain
parameters with some studies considering economic parameters (e.g.
energy prices) and others renewable energy availability and building
220
Applied Energy 214 (2018) 219–238
energy demands. A final shortcoming pertains to the treatment of un-
certain time series parameters like the energy demands and solar/wind
patterns. Typically, each time interval is assigned a probability dis-
tribution (e.g. the Weibull in [30] and the Normal in [31,32]) and is
treated independently from the rest. Variable profiles are generated by
sampling random values from the specified distributions for each time
step. However, this approach neglects the autocorrelation of these time
series. Moreover, correlations and interactions between parameters are
not considered (e.g. between a building’s heat demand and solar ra-
diation/solar gains). Finally, specific aspects that cause the uncertainty
of these parameters like occupant behaviours cannot be explicitly
considered.
Sensitivity Analysis (SA) is complementary to UA and aims to
quantify the importance of uncertain parameters regarding their con-
tribution to model output variability [33]. The most common SA
method is Local Sensitivity Analysis (LSA), which investigates parameter
importance by varying one parameter at a time, while keeping all other
parameters fixed. The popularity of LSA stems primarily from its sim-
plicity and ease of understanding [34]. However, LSA is an inadequate
practice as it does not effectively cover the input parameter space. Ad-
ditionally, parameter interactions cannot be studied as this would re-
quire simultaneous variation of the input parameters [34]. The draw-
backs of LSA are alleviated by using Global Sensitivity Analysis (GSA),
which varies all uncertain parameters simultaneously, hence, offering
better coverage of the uncertain parameter space and allowing for the
study of parameter interactions. A wide range of GSA techniques is
available [35] and up-to-date reviews of GSA developments are given in
[36,37].
Despite the maturity of GSA methods and their great diffusion in
relative fields to DES, such as Building Performance Simulation (BPS)
[38], the majority of the studies in DES design use LSA to investigate
parameter importance (e.g. [13,14,39–49]) and only a limited number
of studies have applied GSA. For instance, Moret et al. [50] have used
the GSA Morris method [51], on the problem of optimal design of a
residential energy system, considering multiple technical, economic
and energy demand uncertainties. Lythcke-Jørgensen [29] also applied
the Morris method to assess the relative importance of each input
parameter on output variability.
1.2. This paper
In summary, previous efforts that used UA to investigate uncertainty
in DES design considered only a subset of the relevant uncertain model
parameters instead of the whole set, while the probabilistic descriptions
assigned to time series parameters exhibit certain drawbacks.
Moreover, studies that aimed to investigate parameter importance with
SA mostly chose to perform LSA instead of the more accurate GSA.
This paper aims to address these shortcomings via the following
contributions:
i. The development of a complete methodological framework that
integrates state-of-the-art methods for UA and GSA with optimisa-
tion-based DES design models.
ii. The consideration of the full set of uncertain model parameters in a
DES design model.
iii. The development of a modelling scheme for the generation of
variable building energy demand and solar radiation profiles, which
better reflect their stochastic nature and correlations. The scheme
incorporates aspects like stochastic occupant behaviours and cli-
mate change projections and is coupled with a Building
Performance Simulation (BPS) tool.
iv. The application of the framework to a case study to illustrate the
insight that can be generated and to highlight the importance of
uncertainty considerations.
Hence, the main novelty of this paper is that it is the first study in
G. Mavromatidis et al.
Fig. 1. General workflow for uncertainty investigation in DES (illustration inspired by Sudret et al. [52]).
the DES literature to integrate DES design models with methods of UA &
GSA and to showcase their application and full potential. Moreover,
another novelty of this paper is that by employing BPS tools to char-
acterise the uncertainty of building energy demands, it is the first effort
to bridge the disciplines of uncertainty in BPS and DES design.
The paper is structured as follows: Section 2 details the metho-
dology for uncertainty investigation including a deterministic DES de-
sign model, Uncertainty Characterisation (UC), Uncertainty Analysis
(UA) and Global Sensitivity Analysis (GSA). Section 3 introduces the
paper’s case study for DES design. Section 4 presents and discusses the
paper’s results regarding both the deterministic model and the UA and
SA. Finally, Section 5 provides concluding remarks.
2. Methodology for uncertainty investigation
This paper’s methodology for uncertainty investigation in DES de-
sign is illustrated in Fig. 1. Step A includes the development of a de-
terministic optimisation model for optimal DES design. Step B includes
Uncertainty Characterisation (UC) in which the uncertain parameters of
the deterministic model are identified and attributed probabilistic de-
scriptions. In Step C, Uncertainty Analysis (UA) is performed using
Monte Carlo (MC) simulations. Finally, in Step D, a two-step Global
Sensitivity Analysis (GSA) identifies the most important uncertain
parameters by calculating Sobol indices as importance measures. Steps
A to D are described in Sections 2.1–2.4 and a discussion of the
methodology’s strengths and criticisms is given in Section 2.5.
2.1. Deterministic DES design model
In the methodology’s first step, a deterministic optimisation model
is developed for the task of optimal DES design. The candidate tech-
nologies for the DES are presented in Fig. 2 and include: a biomass
boiler (BIO), a gas boiler (GAS), a gas-fired CHP engine (CHP), a
ground-source heat pump (GSHP), a thermal storage tank (TS), a bat-
tery (BAT) and photovoltaic panels (PV). All technologies are con-
sidered for installation in the DES except for PV panels, which are to be
installed in a spatially distributed manner on the individual building
roofs. The generated electricity is delivered to the buildings via the
local distribution grid, while for thermal energy a district heating
221
Applied Energy 214 (2018) 219–238
network is used.
This section presents the main model concepts; the complete
mathematical formulation is given in Appendix A. The model identifies
simultaneously the optimal DES design (technology selection and
sizing) and its optimal operating strategies. It is based on the energy
hub modelling framework [53] and it is a mixed-integer linear program
(MILP) following the work done in previous papers [54–57].
The minimisation of the total system cost and of the total CO2
emissions are the objectives considered for the DES design. Following
the ε-constraint method [58], the former objective forms the model’s
objective function, while the latter is expressed in the form of a con-
straint setting an upper limit to the CO2 emissions, as shown in Eqs. (1)
and (2).
min Total system cost: EAC = Invest + Op (1)
s. t . CO2 emissions ⩽ CO2 limit (2)
The total system cost (Eq. (1)) is expressed as the Equivalent Annual
Cost (EAC) and is calculated for a typical representative year. The EAC
consists of an annualised investment cost, Invest , which includes the
investment for the generation, storage and distribution technologies of
the DES. The term Op represents the total annual operating expenditure
and includes the cost for the consumption of energy carriers and the
income stream associated with electricity exports. The CO2 emissions
(Eq. (2)) are also calculated over the period of a year and account for
the system’s operating emissions and not for its life-cycle emissions. The
CO2 limit can be selected by the DES designer and varied to obtain
multiple designs that achieve different carbon performance. The defi-
nitions of the terms in Eqs. (1) and (2) are given in Appendix A.
Besides Eq. (2), additional model constraints are required to describe
the system’s energy balances and other operational and technical con-
straints. The formulation of all these constraints is found in Appendix A.
As inputs to the model, parameters pertaining to the technical (e.g.
conversion efficiencies etc.), economic (e.g. investment costs, energy
carrier prices etc.) and environmental dimensions of the energy tech-
nologies (e.g. emission factors) are required, in addition to time series
profiles of building energy demands and solar radiation availability.
The deterministic parameter values are given in Section A.2 of
Appendix A.
Finally, model decision variables can be split into design variables that
G. Mavromatidis et al.
Inputs Distributed energy system
Biomass Biomass
boiler
Gas boiler
Natural gas
CHP
engine
Grid
electricity
Solar
PV panels
radiaƟon
Ba eries
Fig. 2. Superstructure representation of candidate set of energy technologies for the DES.
represent the selection and the capacity of a generation/storage tech-
nology, and operating variables that represent the utilisation of those
technologies and the energy imports/exports for each model time step.
2.2. Uncertainty characterisation (UC)
Uncertainty Characterisation (UC) entails the identification of a
model’s uncertain parameters and the assignment of a mathematical
description to their uncertainty. Probabilistic methods, which treat
parameters as random variables following Probability Density
Functions (PDF), are the most common to describe parameter un-
certainty [59,60]. A review of UC approaches for uncertain parameters
in DES design models can be found in [61]. In the following sections,
the UC approaches for all the parameter categories of the DES design
model are discussed in detail. In total, 27 individual uncertain para-
meters are identified for the model of Section 2.1 and a summary table
is given in Appendix B.
2.2.1. Energy carrier prices
For the energy carrier prices, deterministic models commonly use
values that correspond to current prices at the time of design (see e.g.
[10,13,48]). Nevertheless, during the long lifetime of a DES, prices are
expected to vary driven by a multitude of factors like energy markets,
energy policy etc. [62]. Their exact evolution is difficult to predict,
hence, they should be treated as uncertain. In this work, to characterise
the uncertainty of energy carrier prices information is sourced from the
Swiss Energy Strategy 2050 (SES2050) [63]. Projections of future nat-
ural gas, biomass, and electricity prices are taken for the period
2021–2040, which is selected as the system’s operational horizon.
Three scenario projections are available (Weiter Wie Bisher – WWB,
Politische Massnahmen – POM, Neue Energiepolitik – NEP),1 which
differ in terms of the Swiss energy landscape they describe [63,64]. The
electricity price projections are illustrated in Fig. 3a.
However, these scenarios represent only two possible futures and
not the full array of possibilities (see e.g. analogous projections in [64]).
To capture a wider range of possible prices, a “cone of uncertainty” is
assigned to each SES2050 price projection. The cone is constructed as
follows: Taking e.g. the electricity prices, each scenario’s price for 2021
is assigned a ±1% uncertainty range. Similarly, for 2022, a ±2% range
is assigned and so on until a ±20% range is assigned to the prices of
2040 as can be seen in Fig. 3b. The same process is applied for biomass
prices too, while due to the higher price volatility of gas prices, the
1
Note that the price projections for the WWB and the POM scenarios in the SES2050
are the same.
222
Applied Energy 214 (2018) 219–238
Outputs
Thermal
storage
Thermal HeaƟng
network demand
GSHP
Electrical
Electricity
network demand
Electricity
export
uncertainty range for 2021 is set to ±2%, increasing at a 2% rate per
year.2 Overall, the cone reflects how uncertainty increases as the pro-
jection horizon moves further into the future. Moreover, as can be
observed in Fig. 3, the prices are mostly projected to increase compared
to current prices (see Appendix A).
As a final step, a uniform distribution is constructed per energy
carrier price by taking the 20-year means of the most and the least
expensive price pathways (see Fig. 3b), similarly to Carpentiero et al.
[66]. The distributions for all energy carriers are given in Table 1.
Regarding the feed-in tariff for exported electricity, FiT, due to the lack
of any projections, a uniform distribution shown in Table 1 is assigned
to it allowing a ± 30% variation around the nominal value.
2.2.2. Emission factors
Additional uncertain parameters pertain to the energy carrier
emission factors. The natural gas emission factor, however, is con-
sidered constant due to negligible variations in the fuel’s carbon con-
tent. Additionally, the factors of renewable energy carriers are taken as
zero. Hence, only the emission factor associated with grid electricity is
considered uncertain due to the dynamic character of a country’s
electricity mix that evolves driven by environmental regulations, en-
ergy targets etc.
Currently, the Swiss grid emission factor for is very low (9.5 gCO2/
kWh, see Appendix A) due to the high shares of nuclear and hydro
energy in its generation mix [67]. However, Switzerland’s decision to
phase-out nuclear energy by 2033 [67] is expected to alter the country’s
electricity generation map. The degree and the direction of change
cannot be known in advance and, thus, the SES2050 offers multiple
projections for the country’s electricity demands and electricity supply
[63].
Thus, to characterise the uncertainty of the Swiss grid emission
factor, a similar approach as with the energy carrier prices is adopted.
For all electricity demand and supply scenarios of SES2050, the evo-
lution of the grid emission factor is calculated and used to construct a
uniform distribution by taking the minimum and the maximum values.
The resulting range of the distribution is [10.1–85.7] gCO2/kWh.
2.2.3. Investment costs
The investment costs for the generation, storage and distribution
2
The choice to attribute the 2% increase rate value for the uncertainty ranges of the
fossil fuel carrier prices was also made by examining the evolution of the consumer prices
over the period between 2000 and 2015, as reported by the Swiss Federal Office of Energy
(SFOE) [65], which reveal much higher fluctuations for the natural gas prices compared
to electricity.
G. Mavromatidis et al.
40
a b
Electricity price (Rp./kWh)
35
30
25
2021 2025
Fig. 3. Electricity price evolution according to SES2050 and associated uncertainty cones.
Table 1
Probability distributions for energy carrier prices.
Energy carrier price
Grid electricity, OCgrid
Natural gas, OCgas
Biomass, OCbio
Feed-in tariff, FiT
technologies are considered uncertain since “depending on the market,
the final investment cost might be different than the costs assumed in the
design stage” [68]. In this work, the approach in [68] is adopted and
each investment cost parameter is assigned a normal distribution with a
mean equal to their nominal value (see Appendix A) and a standard
deviation equal to 7% of the mean.
2.2.4. Technical characteristics
The technical performance of DES technologies under ‘real-world’
conditions might differ from their lab measured performance.
Moreover, factors like poor commissioning and ineffective maintenance
can also contribute to performance deterioration. As a result, the
technical characteristics of DES technologies are also considered un-
certain (see Table B.1 for a full list of the uncertain technical char-
acteristics). In this work, the approach in [69] is adopted and a half-
normal distribution is assigned to the technical characteristics of each
technology to represent the equipment’s potential fainting perfor-
mance. The distribution’s location parameter, μ, is taken as equal to the
characteristic’s nominal value (given in Appendix A), while the scale
parameter, σ2, is assumed equal to 5%.
2.2.5. Energy demand and solar radiation patterns
Deterministic design procedures commonly also employ determi-
nistic Building Performance Simulation (BPS) to calculate building
energy demands and solar radiation patterns. This practice assigns
identical values to multiple buildings regarding occupant densities and
behaviours, while weather data that only reflect the current conditions
and do not take into account extreme weather and future climatic
changes are used.
As a result, the stochastic nature of building energy demands is not
captured and the resulting demand profiles exhibit an unnatural tem-
poral match, which would not occur in reality due to occupant beha-
viour diversity. Moreover, the year-to-year variability of solar radiation
patterns cannot be represented. In order to design robust DES, it is
crucial to account for these aspects when calculating building energy
demands and solar radiation patterns.
Applied Energy 214 (2018) 219–238
40
Electricity price (Rp./kWh)
35
Most
expensive
pathway
30
25 Least
expensive
pathway
2030 2035 2040 2021 2025 2030 2035 2040
Years Years
WWB/POM Uncertainty Cones of
NEP ranges uncertainty
As discussed in the introduction, using probability distributions to
generate variable profiles for these uncertain parameters is an in-
adequate practice. Therefore, in this work, a Monte Carlo BPS (MC-BPS)
Continuous distribution
approach is selected, which is considered a best-practice approach to
U[24.8, 33.7] Rp./kWh generate variable energy demand profiles (see e.g. [70–72]). First,
U[10.4, 17.8] Rp./kWh probabilistic descriptions are assigned to the inputs of a BPS tool (in this
U[12.1, 18.3] Rp./kWh paper, EnergyPlus [73] is used) and then variable profiles are generated
U[5.6, 10.4] Rp./kWh by running multiple BPS simulations with random uncertain parameter
samples.
The use of a BPS tool ensures the high accuracy of the generated
profiles. Moreover, since the profiles are generated in the same simu-
lation environment, their interactions and correlations are taken into
account (e.g. the effects that internal gains due to electricity usage have
on a building’s heat demand). A discussion of the uncertain BPS input
parameters is as follows:
Normal distributions are assigned to material properties, infiltration
and ventilation rates and thermostat settings3 [74,75]. Triangular dis-
tributions are assigned to occupant density (m2/person), installed
lighting and electrical equipment capacity (W/m2), and hot water de-
mands (W/m2)3 [79]. Moreover, stochastic schedules are generated to
represent occupant presence and activities. First, the values of the de-
terministic schedules in the Swiss norm SIA 2024 [78] (occupancy,
lighting, appliance and hot water usage) are randomly varied
by ± 15%. Then, the authors’ approach in [56] is applied, which ran-
domly permutes the hourly schedule values within specified hourly
blocks to represent temporal diversity of occupant actions (e.g. when
the lights are turned on). Finally, regarding weather and solar radiation
uncertainty, multiple future climate projections are sourced from the
CORDEX initiative4 [81] for the period 2021–2040 and are transformed
into weather files that can be used by a BPS tool using the ‘morphing’
approach [82].
The probabilistic scheme for the BPS inputs is created in MATLAB
[83], which is used to generate samples of the uncertain parameters and
to execute the MC-BPS simulations. Overall, by running N simulations
of a BPS model for a building with different parameter samples N sets of
energy demand and solar radiation patterns are obtained. These are
treated as a discrete uniform distribution U{1,N} from which random
profile sets can be sampled.
3
Normal distributions are constructed using the nominal values from [76–78] as the
mean and a standard deviation equal to 10% of the mean, with the exception of ther-
mostat settings where the standard deviation is taken as 1 °C. Triangular distributions are
constructed using the nominal value as the distribution’s mode and the range in SIA 2024
for the distributions minimum and maximum values.
4
The CORDEX initiative is a program sponsored by the World Climate Research
Program (WCRP) with the task to produce regional climate change projections for all land
regions world-wide [80].
223
G. Mavromatidis et al.
2.3. Uncertainty Analysis (UA)
To perform UA, a deterministic model y = g (x ) of the examined
system (Section 2.1) and a probabilistic framework for its uncertain
parameters (Section 2.2) are required. The first step then involves the
generation of N samples for each of the k uncertain input parameters
using their probability distributions. These samples are stored in a
matrix X (Eq. (3)), in which x i,j represents the value of the jth sample of
a parameter Xi . The sampling from parameter distributions is per-
formed using the Sobol sequence for its better space filling properties
compared to random sampling [84].
x x ⋯ xk,1
⎡ 1,1 2,1 ⎤ parameter to the output’s variance by calculating two metrics per para-
x1,2 x2,2 ⋯ xk,2 ⎥
X=⎢
⎢ ⋮ ⋮ ⋱ ⋮ ⎥
⎢ x1,N x2,N ⋯ xk,N ⎥
⎣ ⎦
An important aspect during sample generation is the treatment of
possible parameter correlations. In this paper, no correlations are taken
between investment costs, since the uncertainty in estimating one
technology’s price (e.g. a heat pump) does not influence the estimate for
another technology’s price (e.g. a boiler). The same argument applies to
the devices’ technical characteristics; hence, no correlations are taken.
Regarding the energy demand and solar radiation patterns, their cor-
relations are treated inherently by the approach in Section 2.2.5, which
calculates them with a BPS tool and compounds them in a discrete
random variable. Finally, regarding energy carrier prices, the prices of
biomass, natural gas and electricity over the last two decades were
analysed and no significant correlations were found. The diagrams
showing the price variations and their weak correlations are given in
the paper’s Supplementary Material.
Once the sample matrix is generated, the last step involves N Monte
Carlo evaluations of the deterministic model - one per row of the matrix
- to obtain the variable model response, y , (see Eq. (4)). By evaluating
the variability and the statistical characteristics of the model’s variable
output the issue of quantifying uncertainty’s impacts is addressed.
y g (x1,1,x2,1,…,xk,1) ⎤
⎡ 1⎤ ⎡
⎢ y2 ⎥ ⎢ g (x1,2,x2,2,…,xk,2) ⎥
y= =
⎢⋮⎥ ⎢ ⋮
⎥ However, UA & GSA are decision-support techniques, rather than
⎢ yN ⎥ ⎢ ⎥
⎣ ⎦ ⎢ ⎣ g (x1,N ,x2,N ,…,xk,N ) ⎥
⎦ (4)
2.4. Sensitivity Analysis (SA)
The final step of the methodology is to perform SA to identify the
most influential uncertain parameters regarding the observed output
variations. Given the drawbacks of LSA, a two-step GSA approach is
used, as shown in Fig. 4. The technical details of the methods are given
in Appendix C, but their main concepts are briefly described here as
well. More extensive information on the methods used can be found in
[35].
In the first step, the Morris method [51] is applied on the full set of k
uncertain parameters to screen out unimportant ones at a low compu-
tational cost as it requires r(k + 1) MC model evaluations (re-
commended values for r are between 4 and 10) [35]. The method of
Morris ranks parameter importance by calculating a metric μ∗ per un-
certain parameter (Eq. (C.2)), which quantifies its overall influence on
the output. However, the ranking is only qualitative i.e. it cannot
Fig. 4. Two-step GSA procedure consisting of a screening first step using the Morris
method and a second step using the variance-based Sobol method on a smaller subset of
the most influential uncertain parameters.
Applied Energy 214 (2018) 219–238
quantify how much more important one parameter is over another.
Nevertheless, this does not set a limitation, as another GSA technique is
applied in the second step on the set of parameters that are not elimi-
nated by the Morris method to provide more quantitative information
about parameter importance.
This GSA method is the variance-based Sobol method [85]. Ad-
vantages of this method include that no assumptions on the model form
are required (e.g. linearity or monotonicity), it covers the full range of
input parameters considering their distributions, and it provides quan-
titative information for parameter importance. In the Sobol method, the
model output variance, V (Y ) , is used as a measure of its uncertainty.
The method’s goal is to quantify the contribution of each uncertain input
meter named first-order Sobol index, Si , and total-order Sobol index, STi
(3) (see Eqs. (C.3) and (C.4)). These indices quantify the portion of the output
variance V (Y ) that is explained by a parameter alone (first-order index)
and by a parameter and its interaction effects5 with other parameters
(total-order index). Hence, high Sobol index values indicate that a
parameter is highly significant regarding its influence on the output
variations.
Methods for calculating these indices are based on MC simulations,
which is the main weakness of this technique [34] as state-of-the-art
methods require n(k + 2) model evaluations, with k being the number
of uncertain parameters and n a parameter with a suggested value of
500–1000 [35]. Thus, while the Sobol method can be applied directly to
the full parameter set, if the number of uncertain parameters is large,
the computational cost can be prohibitive. Hence, the use of the Morris
method as a first screening step is recommended [51].
2.5. Method discussion and critical reflections
UA and GSA offer quantitative insights about uncertainty’s effects
on a model’s output and they indicate the parameters from which un-
certainty mainly stems. Additionally, since they rely on Monte Carlo
simulations of a deterministic model, they do not require any mod-
ification to the models that are already used in deterministic DES design
making their adoption easier.
decision-making techniques. While UA results can assist in deciding on
an optimal DES configuration (e.g. the one that emerges most often as
optimal), they cannot point towards a single decision that is optimal for a
specific criterion against uncertainty. This is instead the task of tech-
niques for Optimisation under Uncertainty (OU2) [84] (e.g. Stochastic
Programming (SP) [85]), which incorporate uncertainty directly in the
model and allow decisions to be made before uncertainty is resolved.
Nevertheless, UA, GSA and OU2 are not competing but rather sy-
nergetic techniques. It is useful to confirm whether uncertainty has a
significant impact on DES design via performing UA, before a more
complex model based on SP is developed. Moreover, GSA can benefit
model development by identifying the most important uncertain para-
meters and the ones whose uncertainty can be safely ignored.
3. Case study
3.1. Description
As a case study for this paper, the design of a DES for a fictitious
urban neighbourhood in Zurich, Switzerland, is investigated. The
neighbourhood, presented in Fig. 5a and b in 2D and 3D representa-
tions, is composed of ten buildings of different types and ages. Multi-
family houses (MFH) and offices are included, which are also combined
with other commercial premises to form the MFH-mixed and Office-
5
Two parameters are interacting when their combined effect is different than the sum
of their single effects on the output.
224
G. Mavromatidis et al. Applied Energy 214 (2018) 219–238

Fig. 5. a. Building age and type of the Swiss urban neighbourhood case study, and neighbourhood DES location, b. 3D illustration of the urban neighbourhood case study.

mixed categories. A centralised neighbourhood DES is envisioned that range. Hence, when random values are sampled from the parameters’
generates, stores and distributes energy to the individual buildings and distributions in the stochastic case, it is highly likely these will be
PV panels are to be installed on the buildings’ roofs. The location of the smaller than the deterministic ones, which in turn lead to smaller
neighbourhood DES and the district heating network’s path are also electricity demands. Overall, though, significant energy demand var-
illustrated in Fig. 5a. iations are observed, which the deterministic profiles cannot reflect. For
a better understanding of the seasonal distribution of heat demands,
Fig. 6b presents the monthly energy demands for the deterministic and
3.2. Deterministic and variable energy demands
all the variable profiles.
Finally, as was mentioned in Section 2.2.5, in order to be able to
In this section, the energy demands of the case study buildings are
sample from the generated set of profiles, a proxy variable Ltotal is de-
presented – both deterministic and stochastic. The former are generated
fined that follows a discrete uniform distribution, defined as U {1,1440} .
by running an EnergyPlus simulation per building using nominal values
for all input parameters and a weather file reflecting today’s climate
4. Results
(more information is given in Section A.2). For the variable profiles, the
process from Section 2.2.5 is followed and 1440 energy demand and
4.1. Deterministic results
solar radiation profiles are generated per building. The demands in both
cases are aggregated to form the neighbourhood energy demands.
Initially, the results of the deterministic analysis are presented. In
The results are presented in terms of annual heat and electricity
total, five scenarios are considered in which cost-optimal DES are
demands with the violin plots in Fig. 6a. Since the future, warmer cli-
sought with different emission requirements set according to Eq. (2).
mate is considered for the stochastic case, the neighbourhood’s heat
The scenarios are:
demands are projected to mostly decrease compared to the determi-

• Scenario 1: No CO limit.
nistic value. Additionally, the electricity demands are also mostly
2

• Scenarios 2 to 5: 20%, 40%, 60%, and 80% CO

smaller than the deterministic values. This is because the deterministic
2 reduction against the
values for the lighting and appliances capacities from the Swiss norm
1st scenario, respectively.
SIA 2024 are in many cases equal to the highest values in their possible

120 500
a b
Deterministic profile
Variable profile
100 400
demand (kWh/m2 )
Annual energy

demand (MWh)
Monthly heat

80 300

200
60 Determ. point

100
40

Heat Electricity 0
Jan Mar May Jul Sep Nov
demand demand
Fig. 6. a. Violin plots for the annual neighbourhood heat and electricity demands, b. Variable monthly neighbourhood heat demands.

225
G. Mavromatidis et al. Applied Energy 214 (2018) 219–238

30
a Scenarios c

0
100
No carbon limit

10
Carbon emissions (kgCO 2 /m2 ) 90
25 20 kgCO2 /m 2

20
15 kgCO2 /m 2 80

30
10 kgCO2 /m 2 70

%)
40
20

Gr
2 60
5 kgCO 2 /m

il (

id
50
ss

(%
50

Fo

)
60
15 40

70
30

80
20
10

90
10

10
0
0

0
10
20
30
40
50
60
70
80
90

0
5

10
25.5 26 26.5 27 27.5 Renewable (%)
2
Total cost (CHF/m )

1200
b
Technologies
1000 Oil boiler
Bio boiler
Capacity (kW | kWh | m 2 )

CHP
800 GSHP
Th. Storage
PV
600

400

200

0
No carbon limit 20 kgCO2 /m2 15 kgCO2 /m2 10 kgCO2 /m2 5 kgCO2 /m2
Scenarios
Fig. 7. a. Total system cost and carbon emissions of the deterministic optimal configurations at the different scenarios, b. Technology selection and capacities for the deterministic optimal
system configurations for the different scenarios, c. Energy consumption shares for the deterministic optimal configurations of the different scenarios.

The optimal systems’ cost and carbon performance are given in is eliminated, while in Scenario 5, a biomass boiler is added to achieve
Fig. 7a normalised per m2 of total building floor area. In the first sce- emissions below 5 kgCO2/m2. Finally, the thermal storage capacity is
nario, the system’s total cost is approx. 25.5 CHF/m2, while its emis- the same across all scenarios indicating its high value.
sions are equal to 25 kgCO2/m2. Using the latter value, the CO2 limit Besides design aspects, the model of Section 2 also calculates the
values of Eq. (2) for Scenarios 2 to 5 become 20, 15, 10, and 5 kgCO2/ optimal operating patterns of each technology. Thus, the percentage
m2, respectively. In these scenarios, as more stringent emission con- shares of grid electricity, fossil fuels (natural gas) and renewable energy
straints are imposed, the total cost is increased. However, the rate of (biomass and PV electricity) in the system’s total energy consumption
increase remains small, with the total cost in Scenario 5 (27.1 CHF/m2) (TEC) can be calculated. The mathematical definitions of those terms
being only 6.3% higher compared to the cost of Scenario 1. are given in Appendix A. The energy shares for the different scenarios
The corresponding optimal DES configurations are given in Fig. 7b. are plotted in Fig. 7c using ternary plots.
For Scenario 1, a three-generation-system configuration emerges com- In Scenario 1, the system relies heavily on natural gas, which cor-
posed of a GSHP, a gas boiler and a CHP engine, while the PV area is responds to almost 90% of the TEC, with the grid and renewable shares
equal to 816 m2 (65% of the total roof area). Due to emission con- being below 10% each. This is attributed to the high CHP engine uti-
straints in Scenarios 2 to 5, the CHP capacity is gradually reduced and lisation that covers large parts of the neighbourhood’s heat and elec-
the gas boiler and GSHP capacities are increased. In Scenario 4, the CHP tricity demands. From Scenario 2 to Scenario 4 the fossil share is

226
G. Mavromatidis et al. Applied Energy 214 (2018) 219–238

No carbon limit

20 kgCO2 /m 2
Scenarios

15 kgCO2 /m 2

10 kgCO2 /m 2

5 kgCO 2 /m 2
Determ. value

25 30 35 0 10 20 30
2
Total cost (CHF/m ) Carbon emissions (kgCO 2 /m 2 )

Fig. 8. Violin plots for the variation of the total system cost and the carbon emissions for the different scenarios.

progressively reduced, while the grid share is increased indicating emerge. Hence, gas boilers, GSHPs, thermal storage and photovoltaics
higher GSHP utilisation and, hence, lower emissions. This also explains are popular technologies for the configurations of all scenarios. On the
why the total emissions are reduced despite the increasing gas boiler other hand, CHP engines are included in many configurations of
capacities in Fig. 7b. Finally, while the renewable share remains rela- Scenarios 1 and 2, but from Scenario 3 onwards their popularity is
tively constant in Scenarios 2 to 4, the addition of the biomass boiler in decreasing until Scenarios 4 and 5 where they are hardly ever installed.
Scenario 5 pushes it to almost 30%, while the fossil share is limited to The same visualisation also illustrates the gradual increase of biomass
approx. 20%. boiler selection for the optimal configurations. Unlike the deterministic
case when a biomass boiler was only included in Scenario 5, when
uncertainty is considered, these devices become “popular” already from
4.2. Uncertainty Analysis results
Scenario 4. Such results can offer decision support to decision-makers
regarding the selection of an optimal DES configuration by selecting,
For the UA step of the methodology, 5000 MC evaluations of the
for instance, the one that emerges most often as the optimal one.
deterministic model of Section 2.1 are performed. The five scenarios of
Another important dimension pertains to the fact that many optimal
the deterministic case and their CO2 limit values are maintained to
configurations from the MC simulations deviate from the deterministic
contrast the UA results against the deterministic results.
solutions. To get a quantitative perspective on this matter, Fig. 10
Fig. 8 illustrates the variation of the systems’ cost and carbon per-
presents the percentage of MC runs per scenario for which such de-
formance using violin plots. Starting from the cost distributions, sig-
viations are observed in terms of technology selection (technology ca-
nificant variations are observed across all scenarios along with a gen-
pacity differences are not considered). This percentage essentially then
eral shift to higher values for lower emission requirements. Moreover,
represents the probability that the deterministic configurations are ac-
in most cases the costs are considerably higher than the deterministic
tually suboptimal decisions.
values. Therefore, deterministic cost estimates can be considered un-
For Scenarios 1 to 3, significant portions of the MC runs, ranging
reliable, which is an important issue for stakeholders in DES decision-
from 47% to 75%, result in different solutions. These deviations are
making.
primarily due to many MC runs not including a CHP engine in their
Regarding CO2 emissions, in Scenario 1, the distribution’s median is
configuration unlike their deterministic solutions. Additionally, the
lower than the deterministic value, mainly due to lower natural gas
tighter the emission constraint, the more likely it is for a CHP engine
consumption, which in turn is due to its projected increasing prices
not to be included in the configurations; hence, the MC run percentage
(Section 2.2.1). Additionally, the violin’s form reveals a bimodal
that deviate from the deterministic configuration increases. For
emission distribution, which is due to the large disparities between the
Scenario 4, the percentage remains high and the discrepancy between
carriers’ emission factors. Systems that rely more on fossil fuels cluster
the MC and the deterministic solutions is mainly driven by biomass
at the higher emissions side and systems that rely more on grid elec-
boilers. In the deterministic case, such a boiler type was not included,
tricity and renewables cluster at the lower part. Additionally, despite
while for many of the MC runs, as Fig. 9 shows, a biomass boiler is
the lack of an emission constraint in Scenario 1, there are cases that
necessary to achieve the required carbon performance. Finally, for
result in lower emissions than the limits in Scenarios 2 to 5. In Scenarios
scenario 5, the deterministic system selection seems rather robust, since
2 to 5, in which, emission limits are set, the emission distributions show
as it is also evidenced by Fig. 9, most configurations relying on a GSHP
most points being concentrated at each scenario’s emission limit, while
and a biomass boiler are aligned with the deterministic design.
there also instances where the actual emissions are below the limit.
Finally, Fig. 11 presents the variation of the systems’ energy shares
Additional insight is gained by investigating the variations of the
and contrasts them against their corresponding deterministic points. A
optimal system configurations with the “parallel coordinate” plots of
first observation pertains to the considerable variation that the shares
Fig. 9. In this plot, the optimal design from each MC run is visualised
exhibit across all scenarios. For Scenario 1, for instance, there are cases
with a polyline intersecting vertical axes corresponding to energy
where the grid share is almost 0% and others where it reaches 70%, the
conversion and storage devices. The intersection points between the
renewable share can be above 40% and almost 70%, and there are other
axes and a polyline indicate the device’s capacity (if a device is not
cases where the fossil shares exceed 90%. In Scenarios 2 and 3, similar
selected in any MC run, it is completely omitted from the plot).
patterns emerge along with a general shift of the points towards
This visualization allows the technology selection patterns to

227
G. Mavromatidis et al. Applied Energy 214 (2018) 219–238

No carbon limit
1257 kW 250 kW 370 kWth 1024 kW 600 kWh 1246 m2

Capacity range

10 kgCO2 /m 2

1229 kW 250 kW 152 kWth 1067 kW 600 kWh 1260 m2

0 kW 0 kW 0 kWth 0 kW 483 kWh 2

0m

Capacity range
Gas B. GSHP CHP Bio B. Th. Stor. PV
System capacities

20 kgCO2 /m 2

1257 kW 250 kW 341 kWth 1024 kW 600 kWh 1260 m2

0 kW 0 kW 0 kWth 0 kW 483 kWh 0 m2
Gas B. GSHP CHP Bio B. Th. Stor. PV
Capacity range

System capacities

5 kgCO2 /m 2

1075 kW 250 kW 28 kWth 1024 kW 600 kWh 1260 m2

0 kWth 2
0 kW 44 kW 0 kW 483 kWh 0m

Capacity range
Gas B. GSHP CHP Bio B. Th. Stor. PV
System capacities

15 kgCO2 /m 2

1262 kW 250 kW 332 kWth 1024 kW 600 kWh 1260 m2

0 kW 0 kW 0 kWth 0 kW 483 kWh 0 m2
Gas B. GSHP CHP Bio B. Th. Stor. PV
Capacity range

System capacities

0 kW 50 kW 0 kWth 0 kW 497 kWh 0 m2

Gas B. GSHP CHP Bio B. Th. Stor. PV
System capacities
Fig. 9. Parallel coordinate plots for the different scenarios illustrating the optimal system selection for the MC runs. The black dashed line in each scenario corresponds to the de-
terministic system configuration.

to the fossil fuel usage. However, in the uncertainty domain and given
the unknown future Swiss grid carbon factor, a higher reliance on grid
electricity and renewable energy might be required to meet the re-
quired emission levels.

4.3. Global Sensitivity Analysis results

The UA results illustrated the significant impacts of uncertainty on

Fig. 10. Percentage of MC runs that correspond to optimal system configurations that are
different than the deterministic solutions.
reduced fossil shares to achieve the required carbon performance. For
Scenarios 4 and 5, the fossil fuel shares never exceed 50% and 30%,
respectively, and considerably higher grid and renewable shares are
observed. It is also worth mentioning that significant variations are
observed even for Scenario 5 despite the overwhelming emergence of a
GSHP-Biomass boiler configuration for the DES shown in Fig. 9.
Valuable insight is also drawn by comparing the shares’ variations
against the deterministic point. For all scenarios, the deterministic
point is placed at high fossil shares and lower renewable shares com-
pared to most of the MC results. That is because in the deterministic
case, due to the low current Swiss grid carbon factor, the emission
limits could be met even with low grid and renewable shares compared
multiple model output dimensions – economic and carbon performance,
technology selection, and energy shares. GSA can generate additional
insight by identifying the uncertain parameters that are mostly re-
sponsible for these output variations. GSA can be applied to investigate
the main drivers behind the variability of every model output. In this
paper, it is used to identify the most important parameters regarding
the system cost variations as shown in Fig. 8.
For the two-step GSA, given the total number of uncertain para-
meters is k = 27 (see Table B.1), r is selected as 100 for the Morris
method, leading to r(k + 1) = 2800 model evaluations. Then, for the
ten most influential parameters, the Sobol method is applied with
n = 1000, leading to n(10 + 2) = 12,000 additional model evaluations.
If the Sobol method was directly applied to the full uncertain parameter
set, then 29,000 evaluations would be needed, instead. Hence, the two-
step GSA reduces the total number of required model runs by almost
50%.
The GSA results for all scenarios are presented in Fig. 12. In each
plot, the ten most influential parameters from the Morris method are
presented and ranked according to their μ∗ value. The Sobol indices

228
G. Mavromatidis et al. Applied Energy 214 (2018) 219–238

No carbon limit 20 kgCO2 /m2

0
100 100

10

10
90 90

20

20
80 80

30

30

Gr
70 70

)
l (%

l (%
40

40

id
Gr
60 60

(%
id
ssi

ssi
50

50
(%

)
Fo

Fo
50 50

)
60

60
40 40
70

70
30 30
80

80
20 20
90

90
MC point MC point
10 10
10

10
Determ. point Determ. point
0

0
0 0
0
10
20
30
40
50
60
70
80
90

0
0

10
20
30
40
50
60
70
80
90

0
10

10
Renewable (%) Renewable (%)

15 kgCO2 /m2 10 kgCO2 /m2

0

0
100 100
10

10
90 90
20

20
80 80
30

30
70 70
)

)
Gr

Gr
l (%

l (%
40

40
id

id
60 60
ssi

ssi
50

50
(%

(%
50 50
Fo

Fo
)

)
60

60

40 40
70

70

30 30
80

80

20 20
90

90

MC point MC point
10 10
10

10

Determ. point Determ. point

0

0 0
0
10
20
30
40
50
60
70
80
90

0
0

10
20
30
40
50
60
70
80
90

0
10

10
Renewable (%) Renewable (%)

5 kgCO2 /m2
0

100
10

90
20

80
30

70
Gr
)
l (%
40

id

60
(%
ssi
50

50
)
Fo
60

40
70

30
80

20
90

MC point
10
10

Determ. point
0

0
0
10
20
30
40
50
60
70
80
90

0
10

Renewable (%)
Fig. 11. Grid electricity, fossil fuel and renewable energy share variations for the different scenarios.

calculated by applying the Sobol method on this screened parameter set
are also reported in the figure. A first observation pertains to the fact
that the Morris results are aligned with the Sobol results in terms of
parameter ranking as the Sobol indices decrease in the same order as
the value for μ∗. Therefore, if only a quantitative ranking was required,
GSA could be performed using only the Morris method. This could also
limit the number of MC model runs required.
Even though the Morris method has identified the most important
parameters, the Sobol indices reveal that rather a small set of these
parameters is responsible for the greatest portion of the output’s var-
iance, indicated by the sum of the first- and total-order indices. In all
cases, this set is composed by the energy demands/solar radiation
(Ltotal) and the energy carrier prices (OCm), but at different relative
importance. In Scenario 1, due to the absence of any emission con-
straints, systems that rely heavily on natural gas emerge; hence, the gas
price, OCgas, is the most important parameter. Second most important

229
G. Mavromatidis et al. Applied Energy 214 (2018) 219–238

No carbon limit 20 kgCO 2/m 2 15 kgCO 2/m 2

105 104 105
2 2
0.6 10 0.5 0.5
0.5 8 0.4 0.4

Sobol indices

Sobol indices

Sobol indices
0.4
6 0.3 0.3
*

*
1 0.3 1
4 0.2 0.2
0.2

0.1 2 0.1 0.1

0 0 0 0 0 0
nnet

nnet

nnet
nCHP

Inv CHP
nGAS

nCHP

nGAS
Inv CHP

nGAS

Inv GAS
Ltotal

Inv GSHP

Inv PV

COP GSHP

Ltotal

Inv GSHP

Inv PV
COP GSHP

Ltotal

Inv GSHP
COP GSHP
Inv PV

nPV
OCgrid

OCgrid

OCgrid
OCgas

OCgas

OCgas
10 kgCO 2/m 2 5 kgCO 2/m 2
105 105

2 0.6 2 0.6

0.5 0.5

Sobol indices

Sobol indices
0.4 0.4
*

*
1 0.3 1 0.3

0.2 0.2

0.1 0.1

0 0 0 0
nnet

nnet
nGAS

nBIO
Ltotal

Inv GSHP

COP GSHP

Inv PV
OCbio

Ltotal

OCbio
Inv GSHP

COP GSHP

Inv PV
OCgrid

cgrid

OCgrid

cgrid
OCgas

OCgas
Fig. 12. Results of the method of Morris and of the Variance-based Sobol method indicating the most influencing uncertain parameters with regards to system costs for the different
scenarios (Ltotal refers to the energy demand and solar radiation patterns, OC refers to energy carrier prices, Inv refers to the investment cost of an energy technology, n and COP refer to
the efficiencies of an energy technology).

are the energy demands/solar radiation, Ltotal, followed by the elec- 1

tricity price, OCgrid. For Scenarios 2 and 3, the same parameters are
identified as most influential; however, in this case Ltotal is deemed as
0.8
Cumulative probability

the most important one with OCgas falling at the second and third place
in the two scenarios, respectively. Finally, in Scenarios 4 and 5, OCgas
becomes relatively insignificant, while the price of the biomass energy 0.6
carrier, OCbio, increases in importance. Overall, the energy demands/
solar radiation Ltotal are identified as universally important, while de-
pending on the emission limits, different relative importance is assigned 0.4
to the energy carrier prices OCgrid, OCgas, OCbio.
Besides the most important parameters, GSA results also indicate
that the uncertainty of parameters like investment costs and technical 0.2 Full uncertain parameter set
characteristics can be safely neglected. Hence, these parameters can be Screened uncertain parameter set
fixed at their deterministic values without significant loss of output
0
variance. 20 25 30 35 40 45
As mentioned earlier, the two-step GSA approach reduces the
number of required model runs by almost 50%. To investigate whether Total cost (CHF/m2 )
the screening of the ten most important parameters leads to any loss of Fig. 13. Comparison of Scenario 3 Total cost Cumulative Density Functions (CDF) from
output variability, Fig. 13 compares the Cumulative Distribution Uncertainty Analysis (UA) using the full uncertain parameter set and from Sobol Analysis
Functions of the system cost for Scenario 3 resulting from the MC model using the screened parameter set by the method of Morris.
evaluations of UA and the Sobol method. For the former, all uncertain
parameters are included, while for the latter only the ten most im- parameters and the rest are ignored, no significant loss of accuracy
portant parameters identified by the Morris method. The good agree- would occur.
ment shown between the two curves indicates that indeed the majority
of output uncertainty is accounted for by the ten parameters screened
with the Morris method. 5. Conclusions
Overall, GSA results offer valuable information to DES designers
about the drivers of uncertainty. For the case study examined, for in- This paper presents a four-step framework for the investigation of
stance, results indicate that the DES designer should aim for a reduction uncertainty in optimal DES design. In the first step, a deterministic
of the uncertainty of the energy demands and the energy carrier prices. model is developed for optimal DES design and operation. The second
This would, in return, greatly reduce the uncertainty of the model’s step pertains to UC, in which probabilistic descriptions are assigned to
output i.e. the uncertainty over which design to select as the optimal the model’s uncertain parameters, which are energy carrier prices and
one. Finally, GSA can also inform further modelling efforts. For in- emission factors, investment costs and technical characteristics, and
stance, in the case that a model based on Stochastic Programming (SP) energy demand and solar radiation patterns. In the third step, UA is
for decision-making under uncertainty were to be developed for this performed using Monte Carlo simulations to investigate the variability
paper’s case study, the results would indicate that if only the energy of model outputs. Finally, the fourth step includes a two-stage GSA
carrier prices and the energy demands are considered as uncertain approach combining the Morris and the variance-based Sobol methods

230
G. Mavromatidis et al. Applied Energy 214 (2018) 219–238

to determine the most influencing parameters driving the total cost of
the DES.
The framework is applied to an optimal DES design task for a Swiss
urban neighbourhood, in which cost-optimal systems are sought that
achieve progressively better emission performance. The deterministic
results reveal that while a GSHP-CHP-gas boiler configuration achieves
the lowest cost, a gradual phase-out of the CHP engine is required to
reduce emissions and biomass is only required for CO2 emission per-
formance below 5 kgCO2/m2. UA, though, reveals significant variability
in the model outputs and deviations from the deterministic results.
More specifically, in many MC simulations, the total system cost is
higher than the deterministic cost estimates. Moreover, the determi-
nistic configurations are shown to be frequently suboptimal choices,
confirming, thus, the risks of deterministic design. Having established
the significant impacts of uncertainty, GSA reveals that the main drivers
behind the variability of the optimal systems’ cost are the energy de-
mand patterns and the energy carrier prices with the rest of the un-
certain parameters having only minimal influence.
Overall, given the multi-faceted nature of uncertainty in DES design,
maintaining deterministic practices is insufficient. The framework
presented in this paper, combining a deterministic DES design model
with UA and GSA techniques can be adopted by DES researchers and
practitioners to help them understand the range of optimal designs and
their performance, and the parameters with the greatest effect on the
design outcome. Moreover, the use of these techniques, especially in
practice, can lead to more robust energy systems against uncertainty.
DES designers can use UA to test the “robustness” of their designs by
examining whether they remain optimal for different realisations of the
uncertain parameters. Such investigations of uncertainty can also assist
with helping interested parties to invest in DES. This could in turn ac-
celerate the adoption of DES and contribute to real-world cost, emission
and energy savings.
Regarding future work, UA and GSA could be used to investigate
uncertainty in different settings, such as for the design of multi-site
DES. Additionally, models based on techniques for Optimisation under
Uncertainty (e.g. Stochastic Programming) can be developed for op-
timal decision-making under uncertainty. The results of the GSA can be
directly fed into those models as they indicate the most influencing
uncertain parameters that need to be considered.
Acknowledgements
The research in this paper is supported in part by funds from the
Competence Center Energy and Mobility (CCEM) project IDEAS4cities
(CCEM 801), and the Swiss Commission for Technology and Innovation
within the Swiss Competence Centres for Energy Research (SCCER)
Future Energy Efficient Buildings & Districts (FEEB&D) (CTI
1155002539).

Appendix A

A.1. Deterministic DES design model formulation

In this appendix, the formulation of the deterministic DES design model is provided. The model data and equations are indexed over the sets
presented in Table A.1.

A.1.1. Objective function

The Equivalent Annual Cost (EAC) of the system is used as the model’s objective function, calculated for a typical year represented with a set of
typical days consisting of 24 hourly time steps each. The selection process for these typical days is discussed in Section A.2. The EAC is composed of an
annualised investment cost of all generation, storage and distribution technologies, Invest , and the total annual operating cost, Op , which are
expressed as follows:
minInvest + Op (A.1)

Invest = ∑ (yi ·FCi + capi ·LCi )·CRFi + DH

i∈I (A.2)

DH = ∑ (peakb ·LCHX )·CRFHX + lnet ·LCnet ·CRFnet

b∈B (A.3)

Op = ∑ (Im,t ·OCm·wt )− ∑ (Ltexp ·FiT ·wt )

t ∈ T ,m ∈ M t∈T (A.4)
In Eq. (A.2), yi is a binary variable for the installation of technology i and capi is a continuous nonnegative variable that represents the capacity of
technology i [kW for conversion technologies, kWh for storage technologies, m2 for PV]. FCi [CHF] and LCi [CHF/(kW, kWh, m2)] are fixed and
capacity-dependent cost parameters for the installation of technology i, respectively, and CRFi is the capital recovery factor (CRF) parameter of
technology i, defined as:
r (r + 1)lifetimei
CRFi =
(r + 1)lifetimei−1 (A.5)

Table A.1
Model sets and indices used for equation formulations.

Index Set Description

i I All technologies: CHP, GSHP, BIO, GAS, PV, TS, BAT

j J⊆I
k K⊆I
l L⊆ I
m M
t T
b B
Dispatchable technologies: CHP, GSHP, BIO, GAS
Solar technologies: PV
Energy storage technologies: TS, BAT
Energy carriers: grid, gas, bio
Time
Building(s) connected at the output of the DES: b ∈ {1,…,10}

231
G. Mavromatidis et al. Applied Energy 214 (2018) 219–238

where r is the discount rate, and lifetimei corresponds to the lifetime of technology i in years.
The term DH (Eq. (A.3)) represents the district heating network cost composed of the network piping cost and of the cost of the heat exchanger
sub-stations in the buildings. peakb is a parameter for the peak heat load of building b [kW], which dictates the size of its heat exchanger, LCHX is a
linear, capacity-dependent cost parameter for the installation of a heat exchanger HX [CHF/kW], CRFHX is the capital recovery factor (CRF) parameter
for the heat exchanger HX, lnet is a parameter for the length of the piping network [m], and LCnet is the linear cost parameter per m of piping network
[CHF/m].
In Eq. (A.4), Im,t is a continuous nonnegative variable representing the consumption of energy carrier m at time step t [kWh], OCm is a parameter for
the price of energy carrier m [CHF/kWh], Ltexp is a continuous nonnegative variable for electricity exported to the grid at time step t [kWh], and FiT is a
parameter for the feed-in-tariffs of exported electricity [CHF/kWh]. The term wt corresponds to the number of time steps that every typical time step
represents and is calculated by the clustering procedure used to define the set of typical days considered for the design, as explained in Section A.2.

A.1.2. Constraints
In this section, the model constraints are presented defining the feasible region of each decision variable.
Carbon emissions constraint: A constraint setting an upper limit to the system’s CO2 emissions is defined in Eq. (A.6), where Cm is the carbon
emission factor parameter for energy carrier m [gCO2/kWh].

CO2 emissions: ∑ (Im,t ·Cm·wt )− ∑ (Ltexp ·Cgrid·wt ) ⩽ CO2 limit

t ∈ T ,m ∈ M t∈T (A.6)
Heat and electricity balances: Additional constraints are required to describe energy balances for each energy demand type, which are heat and
electricity in this case. These are expressed as:
dis ch heat
nGAS ·PGAS,t + nBIO ·PBIO,t + COPGSHP ·PGSHP,t + nCHP,th ·PCHP,t + QTS ,t −QTS ,t = Lt / nnet , ∀ t ∈ T (A.7)
Nb
exp
PGRID,t −PGSHP,t + nCHP,el ·PCHP,t + ∑ (nPV ·capPV ,b ·SbPV
,t )−Lt
dis
+ QBAT ch
,t −QBAT ,t = Lt
elec
, ∀t∈T
b=1 (A.8)
where Ltheat
and Ltelec
are the heat and electricity demands of the neighbourhood at time step t [kWh], nnet , is the efficiency of the thermal distribution
network, Pi,t , is a continuous nonnegative variable for the input energy stream to converter i at time step t [kWh], SbPV
,t , is the incoming solar radiation
patterns at the roof of building b at time step t [kWh/m2], ni , COPi , are the conversion efficiency or the coefficient of performance (COP) of device i,
and Qlch dis
,t , Ql,t , are continuous nonnegative variables for the charging and discharging rates for the storage module l at time step t [kWh].
An additional set of constraints is required to connect the input energy carrier streams to the DES Im,t with the inputs to each individual energy
conversion device Pi,t :

⎧ PGRID,t = Igrid,t ⎫
PBIO,t = Ibio,t , ∀t∈T
⎨ ⎬
P + PGAS,t = Igas,t (A.9)
⎩ CHP,t ⎭
Energy storage balances: The energy storage technologies are assumed to only deal with short-term, daily fluctuations and cannot act as seasonal
storages. To describe the energy balances for the storage devices, two constraints are needed, which are expressed as follows:

El,t = nlself ·El,t − 1 + nlch ·Qlch dis dis

,t −(1/ nl )· Ql,t , ∀ l,t ∈ T ⧹T (A.10)

El,t = nlself ·El,t + 23 + nlch ·Qlch dis dis

,t −(1/ nl )· Ql,t , ∀ l,t ∈ T (A.11)
where El,t is a continuous nonnegative variable representing the amount of energy stored at module l at time step t [kWh] and nlself ,nlch,nldis
are the self-
discharge losses, the charging and the discharging efficiency, respectively, of storage element l.
The sets that define the domain of each constraint are: T , which is the set of every time step in the model, and T , which includes the first hour of
each typical day. The first equation applies for the difference between T and T sets and states that the total amount of energy stored is equal to the
stored amount at the previous time step and the difference between the charging and discharging flows. The second equation ensures that the energy
stored at the beginning of the day is equal to the energy stored at the end of the day plus the change of the energy content in the first interval. The
two constraints follow similar formulations used by other studies that also considered daily storage utilisation [8,43].
Fixed cost constraint: To impose the fixed investment cost FCi connected with the selection of a technology i in the objective function (Eqs. (A.1)
and (A.2)), the constraint in Eq. (A.12) is used, in which M is a sufficiently large value following the “Big M” convention [86].
capi ⩽ M·yi , ∀ i ∈ I (A.12)
Technical constraints: Additional constraints are added to impose operational constraints on the equipment considered, such as the non-violation
of capacities of energy conversion (Eq. (A.13)) and energy storage devices (Eq. (A.14)).
Pj,t ·nj ⩽ capj , ∀ j ∈ J ,t ∈ T (A.13)

El,t ⩽ capl , ∀ l ∈ L,t ∈ T (A.14)

Supplementary constraints are introduced with Eqs. (A.15) and (A.16) to limit the maximum possible charge/discharge rates of energy storage
technologies, where Qlch,max and Qldis,max are parameters expressed in percentage terms that limit the maximum charge/discharge rates for storage
module l.
Qlch
,t ⩽ Ql
ch,max
·capl , ∀ l ∈ L,t ∈ T (A.15)

Qldis
,t ⩽ Qldis,max ·capl , ∀ l ∈ L,t ∈ T (A.16)
Apart from technology related constraints, other limitations are imposed due to “real world” restrictions. In this model, the capacity of PV panels

232
G. Mavromatidis et al. Applied Energy 214 (2018) 219–238

is restricted by the maximum allowable roof surface utilisation, the capacity of storage technologies is constrained due to assumed space limitations,
while the GSHP capacity is also constrained due to limited urban space availability for the required boreholes. The constraints are correspondingly
expressed as follows:

∑ capb,k ⩽ Abroof , ∀ b ∈ B
k∈K (A.17)

capl ⩽ Elmax , ∀l∈L (A.18)

max
capGSHP ⩽ capGSHP (A.19)
where Abroof
is a parameter representing the total available roof area [m ] on building b’s roof, 2
is the maximum allowable capacity [kWh] for Elmax
max
storage of type l , and capGSHP is the maximum allowable GSHP capacity [kW].
Energy shares: The energy system’s total energy consumption (TEC) and energy consumption shares among the different energy carriers (grid
share (GS), fossil share (FS) and renewable share (RS)) are defined as follows:

Total energy consumption: TEC = ∑ (Im,t ·wt ) + ∑ (SbPV

,t · capPV ,b · nPV · wt )
t ∈ T ,m ∈ M t ∈ T ,b ∈ B (A.20)

GS = ∑ (Igrid,t ·wt )/ TEC·100%

t∈T (A.21)

FS = ∑ (Igas,t ·wt )/ TEC·100%

t∈T (A.22)

⎡ ⎤
RS = ⎢ ∑ (Ibio,t ·wt ) + ∑ (SbPV
,t · capPV ,b · nPV · wt ) ⎥ / TEC ·100%
⎣t∈T t ∈ T ,b ∈ B ⎦ (A.23)

A.2. Deterministic model parameters

The investment costs for the energy technologies are given in Tables A.2 and A.3 reflecting current prices in the Swiss market. Regarding data for
the energy carrier prices and the electricity feed-in tariffs, it is common practice in deterministic models to use values that correspond to the current
conditions at the time of the design (see e.g. [10,13,48,87]). In this work, values for these parameters are sourced from the Swiss Energy Strategy

Table A.2
Investment cost parameters corresponding to the fixed cost (FCi) and the linear cost (LCi) components of heating and cogeneration technologies [92].

Heating & cogeneration technologies

Gas boiler Biomass boiler GSHP Gas CHP

Capacity (kW) FCGAS (CHF) LCGAS (CHF/kW) FCBIO (CHF) LCBIO (CHF/kW) FCGSHP (CHF) LCGSHP (CHF/kW) FCCHP (CHF) LCCHP (CHF/kWth)

< 50 11,920 640 24,940 1150 9070 2170 43,450 3100

50–200 32,170 330 32,520 950 25,220 1950 124,860 1530
> 200 66,100 200 146,610 440 94,120 1670 299,140 790

Table A.3
Investment cost parameters corresponding to the fixed cost (FCi) and the linear cost (LCi) components of solar, energy storage and district heating technologies [92–94].

Solar technologies Energy storage technologies District heating technologies

Photovoltaic Thermal storage Battery District District

heating heating heat
network exchanger

FCPV LCPV FCTS LCTS FCBAT LCBAT LCnet LCHX (CHF/

(CHF) (CHF/ (CHF) (CHF/ (CHF) (CHF/ (CHF/m) kW)
m2) kWh) kWh)

5 750 290 1 685 12.5 – 2 000 800 100

2050 (SES2050) [63] to reflect current rates in Switzerland and are shown in Table A4.6 The emission factors used consider direct carbon emissions
from sources within the DES (e.g. due to fuel combustion) as well as indirect emissions associated with the consumption of purchased electricity and
do not reflect the overall life-cycle of the energy carrier and the system that utilises it. The values for these factors are also sourced from SES2050 and
are reported in Table A.4.7

6
Prices for 2016 are not given directly in the SES2050 and their values are linearly interpolated between 2015 and 2020 values.
7
The emission factor for electricity is not directly reported in SES2050 and is calculated by dividing the total carbon emissions due to electricity generation in 2016 by the total
electricity demand of Switzerland for the year 2016. Therefore, a country average emission factor in gCO2 per kWh of electricity consumed in Switzerland can be obtained. The same
approach was also used in [88].

233
G. Mavromatidis et al. Applied Energy 214 (2018) 219–238

The values of the technical parameters of the model are given in Tables A.5–A.7. The district heating network length, lnet , is equal to 196 m. The
total building roof area for PV is equal to 1260 m2. The maximum capacity for thermal and electrical storage are taken as 600 kWh (Eq. (A.18)),
while for the GSHP it is taken as equal to 250 kW (Eq. (A.19)). The discount rate r is taken as 8%.
The building energy requirements for heating and electricity and the incoming solar radiation patterns are calculated using the Building
Performance Simulation software EnergyPlus [73]. Building construction and infiltration information are taken from [76,77]. Information regarding
occupancy densities, capacities for lighting and electrical equipment, thermostat and ventilation settings, and hot water demands, and their cor-
responding schedules are taken from the Swiss norm SIA 2024 [78]. Finally, as weather conditions a Typical Meteorological Year (TMY) from [89] is
used. Having information about energy demands and solar radiation for a full calendar year, a set of typical days is generated using the clustering-
based (k-medoids) method in [90]. In total, 12 + 2 days are selected to form the typical set following literature recommendations [90,91]. The
12 days are representative days of the whole year and the 2 additional days correspond to the days when the peak heat and electricity demands are
observed and are maintained for accurate system sizing.

Table A.4
Energy prices and emission factors for the different energy carriers [63].

Energy carrier Energy price (Rp.1/ Carbon emission factor

kWh) (gCO2/kWh)

Grid electricity, OCgrid 25.6 9.5

Natural gas, OCgas 11.3 198
Biomass, OCbio 10.0 0

Electricity feed-in tariff, FiT 7.98 –

1
The Rp. is the hundredth denomination of the Swiss franc currency (CHF).

Table A.5
Conversion efficiency, lifetime and minimum available capacities of energy technologies.

Conversion technologies

Gas boiler Biomass boiler GSHP Gas CHP Photovoltaic

Conversion efficiency < 50 kW nGAS = 85% nBIO = 80% COPGSHP = 3.5 nCHP,th/ nCHP,el = 52%/33% nPV = 15%
50–200 kW nGAS = 87.5% nBIO = 82.5% COPGSHP = 3.5 nCHP,th/ nCHP,el = 53.5%/34%
> 200 kW nGAS = 87.5% nBIO = 85% COPGSHP = 3.5 nCHP,th/ nCHP,el = 55%/35%

Lifetime, lifetimei 25 years 20 years 20 years 20 years 20 years

Min. capacity, capimin 5 kW 5 kW 5 kW 5 kW 4 m2

Table A.6
Technical characteristics and lifetime of energy storage modules [95].

Energy storage technologies

Technical characteristic Thermal storage Battery

Charge/discharge efficiency, nlch/ nldis 90% 90%

Maximum charge/discharge rate, 40% 40%
Qlch,max / Qldis,max
Storage standing losses, nlself 99% 99.9%
Lifetime, lifetimel 20 years 20 years
Minimum capacity, caplmin 20 kWh 1 kWh

Table A.7
Efficiency and lifetime of district heating network components [94,96].

District heating technologies

District heating District heating heat Total network

network exchanger

Efficiency npipe = 92% nHX = 98% nnet = npipe ·nHX = 90%

Lifetime 40 years 20 years –

234
G. Mavromatidis et al. Applied Energy 214 (2018) 219–238

Appendix B

See Table B.1.

Table B.1
Summary of uncertain parameters and their characterisation approaches for this paper (The parameter μ for the normal (N) and the half-normal (HN) distributions refer to the nominal,
deterministic value).

Uncertain parameter Parameter description Representation of uncertainty

Energy carrier prices (Section 2.2.1)

OCgrid Cost of grid electricity imports (CHF/kWh) Uniform distribution (see Table 1)
OCgas Cost of natural gas (CHF/kWh) Uniform distribution (see Table 1)
OCbio Cost of biomass (CHF/kWh) Uniform distribution (see Table 1)
FiT Electricity feed-in tariff (CHF/kWh) Uniform distribution (see Table 1)

Emission factors (Section 2.2.2)

Cgrid The emission factor of grid imported electricity (gCO2/kWh) Uniform distribution (see Section 2.2.2)

Investment costs (Section 2.2.3)

InvGAS Investment cost of the gas boiler Normal distribution: N(μ, 0.07μ)
InvBIO Investment cost of the biomass boiler Normal distribution: N(μ, 0.07μ)
InvGSHP Investment cost of the GSHP Normal distribution: N(μ, 0.07μ)
InvCHP Investment cost of the CHP engine Normal distribution: N(μ, 0.07μ)
InvPV Investment cost of the PV panels Normal distribution: N(μ, 0.07μ)
InvTS Investment cost of the thermal storage Normal distribution: N(μ, 0.07μ)
InvBAT Investment cost of the battery Normal distribution: N(μ, 0.07μ)
Invnet Investment cost of the thermal network Normal distribution: N(μ, 0.07μ)
InvHX Investment cost of the heat exchanger Normal distribution: N(μ, 0.07μ)

Technical characteristics (Section 2.2.4)

nGAS Conversion efficiency of the gas boiler Half-normal distribution: HN(μ, 0.05μ)
nBIO Conversion efficiency of the biomass boiler Half-normal distribution: HN(μ, 0.05μ)
COPGSHP Conversion efficiency of the GSHP Half-normal distribution: HN(μ, 0.05μ)
nCHP,th/ nCHP,el Conversion efficiencies of the CHP engine Half-normal distribution: HN(μ, 0.05μ)
nPV Conversion efficiency of the PV panels Half-normal distribution: HN(μ, 0.05μ)
ch dis
nTS / nTS Charge/discharge efficiency of thermal storage Half-normal distribution: HN(μ, 0.05μ)
ch,max
QTS dis,max
/ QTS Max charge/discharge rate of thermal storage Half-normal distribution: HN(μ, 0.05μ)
self
nTS Standing losses of thermal storage Half-normal distribution: HN(μ, 0.05μ)
ch
nBAT dis
/ nBAT Charge/discharge efficiency of battery Half-normal distribution: HN(μ, 0.05μ)
ch,max
QBAT dis,max
/ QBAT Max charge/discharge rate of battery Half-normal distribution: HN(μ, 0.05μ)
self
nBAT Standing losses of battery Half-normal distribution: HN(μ, 0.05μ)
nnet Efficiency of thermal distribution network Half-normal distribution: HN(μ, 0.05μ)

Energy demands and solar radiation (Section 2.2.5)

Ltotal Energy demand patterns and incoming solar radiation per building Discrete uniform distribution: U{1–1440}

Appendix C

In this Appendix, the technical details of the GSA method of Morris and the Variance-based Sobol method are given. Extensive information on
these methods can be found in [35].

C.1. Morris method

The method of Morris [51] includes the Monte Carlo evaluation of a model for points along a multi-step trajectory in the input parameter space.
This trajectory is created by modifying one randomly selected input parameter per step by an amount Δ, while keeping the rest constant. The process
is repeated until all k uncertain parameters have moved exactly one step each, bringing the total number of trajectory points to k + 1. As in each step
of the trajectory only a single parameter value xi is altered, its Elementary Effect (EE) can be calculated as follows:
g (x1,⋯,x i − 1,x i ± Δ,⋯,xk )−g (x1,⋯,xk )
EEi =
Δ (C.1)
By defining r different trajectories and performing the same procedure, r elementary effects can be calculated per parameter. The computational
cost for the complete procedure then becomes equal to r(k + 1) model evaluations with a suggested value for r being 4–10 [35].
The original method by Morris then includes the calculation of the mean, μi and the standard deviation, σi, of the EEi as measures of parameter
importance. However, in cases when the elementary effects of one parameter have opposite signs and cancel each other out, small μi values might
emerge, which would falsely indicate a non-influential parameter. As a result, Campolongo et al. [97] have suggested a modified mean measure μi∗,
which considers the absolute values of the elementary effects:
r
1
μi∗ = ∑ |EEi |
r j=1 (C.2)

235
G. Mavromatidis et al. Applied Energy 214 (2018) 219–238

Campolongo et al. [97] have demonstrated that the measure μi∗ can be used very effectively to rank parameters. However, it is a qualitative
measure and cannot indicate how much more important one parameter is over another. Nevertheless, calculating the measure μi∗ for each parameter
allows for uninfluential parameters to be identified and removed in order for the most computationally intensive Variance-based Sobol method to be
performed.
All the calculations for the Morris screening have been performed using the morris() function from the R package sensitivity [98] that
incorporates all of the method aspects discussed.

C.2. Variance-based Sobol method

The variance-based Sobol method is based on variance decomposition and its aim is the quantification of the contribution of each parameter to
the total variance of the output. For that, the most common practice involves the calculation of two different metrics named first-order Sobol index, Si ,
and total-order Sobol index, STi , respectively:
VXi (E X∼ i (Y |Xi ))
Si =
V (Y ) (C.3)

E X∼ i (VXi (Y |X∼ i ))
STi =
V (Y ) (C.4)
In Eq. (C.3), the term E X∼ i (Y |Xi ) takes the mean of Y overall all the possible values of X∼ i when parameter Xi is fixed. The term X∼ i represents all
input parameters but Xi . The outer variance term VXi is then considered over all values of Xi , thus, giving the variance of the inner mean term as Xi
varies. A more intuitive interpretation of the Si index is that it represents the “expected reduction in the output variance that would be obtained if Xi could
be fixed” [34]. As a result, this index is used to identify the uncertain parameters that have significant contribution to the output’s variance. However,
the first-order indices do not consider any interactions between parameters and only quantify the effect of a parameter by itself. Therefore, a small
first-order index Si cannot indicate non-significance for the parameter Xi as there might exist higher-order interactions.
The total-order Sobol index aims to quantify the total effect of a parameter to the output considering interactions with other parameters. In Eq.
(C.4), the term VXi (Y |X∼ i ) is the variance of the output Y over all possible values for Xi when X∼ i is fixed. The outer term E X∼ i considers all the
possible values of X∼ i , hence, giving the expected variance of the output when X∼ i is fixed. Similarly to the case of the first-order index, Saltelli and
Annoni [34] offer a more intuitive explanation writing that the term E X∼ i (VXi (Y |X∼ i )) “would represent the expected variance that would remain if all
factors but Xi could be fixed”. If for a parameter Xi , STi is equal to zero, this would mean that no variance in the output would remain; hence, despite
the fact that Xi remains uncertain, it has no influence on the output. As a result, the condition STi = 0 is necessary and sufficient for a parameter to be
deemed non-influential.
By definition, the sum of the first-order indices cannot exceed unity and it represents the percentage of the output variance that can be explained
due to inputs taken independently. The difference between the sum of the first-order indices and unity indicates the contribution of interactions
between the parameters.
In order to estimate the first- and total-order Sobol indices, methods based on MC simulations have been proposed [99-101]. All calculations in
this paper are performed using the R package sensitivity [98] that implements all estimation formulae in the functions sobol2007() and so-
boljansen().

Appendix D. Supplementary material

Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.apenergy.2018.01.062.

References Energy Convers Manag 2015;99:252–63. http://dx.doi.org/10.1016/j.enconman.

[1] United Nations. The 2030 Agenda for Sustainable Development – Goal 7: Ensure
access to affordable, reliable, sustainable and modern energy for all; 2015.
< http://www.un.org/sustainabledevelopment/energy/ > .
[2] Di Somma M, Yan B, Bianco N, Graditi G, Luh PB, Mongibello L, et al. Operation
optimization of a distributed energy system considering energy costs and exergy
efficiency. Energy Convers Manag 2015;103:739–51. http://dx.doi.org/10.1016/j.
enconman.2015.07.009.
[3] Mancarella P. MES (multi-energy systems): an overview of concepts and evaluation
models. Energy 2014;65:1–17. http://dx.doi.org/10.1016/j.energy.2013.10.041.
[4] Alanne K, Saari A. Distributed energy generation and sustainable development.
Renew Sustain Energy Rev 2006;10:539–58. http://dx.doi.org/10.1016/j.rser.
2004.11.004.
[5] Akorede MF, Hizam H, Pouresmaeil E. Distributed energy resources and benefits to
the environment. Renew Sustain Energy Rev 2010;14:724–34. http://dx.doi.org/
10.1016/j.rser.2009.10.025.
[6] Arcuri P, Beraldi P, Florio G, Fragiacomo P. Optimal design of a small size tri-
generation plant in civil users: a MINLP (Mixed Integer Non Linear Programming
Model). Energy 2015;80:628–41. http://dx.doi.org/10.1016/j.energy.2014.12.
018.
[7] Ashouri A, Fux SS, Benz MJ, Guzzella L. Optimal design and operation of building
services using mixed-integer linear programming techniques. Energy
2013;59:365–76. http://dx.doi.org/10.1016/j.energy.2013.06.053.
[8] Harb H, Reinhardt J, Streblow R, Müller D. MIP approach for designing heating
systems in residential buildings and neighbourhoods. J Build Perform Simul
2015:1–15. http://dx.doi.org/10.1080/19401493.2015.1051113.
[9] Fuentes-Cortés LF, Ponce-Ortega JM, Nápoles-Rivera F, Serna-González M, El-
Halwagi MM. Optimal design of integrated CHP systems for housing complexes.
2015.04.036.
[10] Omu A, Choudhary R, Boies A. Distributed energy resource system optimisation
using mixed integer linear programming. Energy Policy 2013;61:249–66. http://
dx.doi.org/10.1016/j.enpol.2013.05.009.
[11] Zhou Z, Liu P, Li Z, Ni W. An engineering approach to the optimal design of dis-
tributed energy systems in China. Appl Therm Eng 2013;53:387–96. http://dx.doi.
org/10.1016/j.applthermaleng.2012.01.067.
[12] Pruitt KA, Braun RJ, Newman AM. Evaluating shortfalls in mixed-integer pro-
gramming approaches for the optimal design and dispatch of distributed genera-
tion systems. Appl Energy 2013;102:386–98. http://dx.doi.org/10.1016/j.
apenergy.2012.07.030.
[13] Wouters C, Fraga ES, James AM. An energy integrated, multi-microgrid, MILP
(mixed-integer linear programming) approach for residential distributed energy
system planning – a South Australian case-study. Energy 2015;85:30–44. http://
dx.doi.org/10.1016/j.energy.2015.03.051.
[14] Merkel E, McKenna R, Fichtner W. Optimisation of the capacity and the dispatch of
decentralised micro-CHP systems: a case study for the UK. Appl Energy
2015;140:120–34. http://dx.doi.org/10.1016/j.apenergy.2014.11.036.
[15] Braslavsky JH, Wall JR, Reedman LJ. Optimal distributed energy resources and the
cost of reduced greenhouse gas emissions in a large retail shopping centre. Appl
Energy 2015;155:120–30. http://dx.doi.org/10.1016/j.apenergy.2015.05.085.
[16] Zhang D, Evangelisti S, Lettieri P, Papageorgiou LG. Optimal design of CHP-based
microgrids: multiobjective optimisation and life cycle assessment. Energy
2015;85:181–93. http://dx.doi.org/10.1016/j.energy.2015.03.036.
[17] McKenna R, Merkel E, Fichtner W. Energy autonomy in residential buildings: a
techno-economic model-based analysis of the scale effects. Appl Energy
2017;189:800–15. http://dx.doi.org/10.1016/j.apenergy.2016.03.062.
[18] Liu P, Georgiadis MC, Pistikopoulos EN. An energy systems engineering approach
for the design and operation of microgrids in residential applications. Chem Eng

236
G. Mavromatidis et al.
Res Des 2013;91:2054–69. http://dx.doi.org/10.1016/j.cherd.2013.08.016.
[19] Allegrini J, Orehounig K, Mavromatidis G, Ruesch F, Dorer V, Evins R. A review of
modelling approaches and tools for the simulation of district-scale energy systems.
Renew Sustain Energy Rev 2015;52:1391–404. http://dx.doi.org/10.1016/j.rser.
2015.07.123.
[20] Carvalho M, Lozano MA, Serra LM. Multicriteria synthesis of trigeneration systems
considering economic and environmental aspects. Appl Energy 2012;91:245–54.
http://dx.doi.org/10.1016/j.apenergy.2011.09.029.
[21] Marquant JF, Evins R, Bollinger LA, Carmeliet J. A holarchic approach for multi-
scale distributed energy system optimisation. Appl Energy 2017;208:935–53.
http://dx.doi.org/10.1016/j.apenergy.2017.09.057.
[22] Gabrielli P, Gazzani M, Martelli E, Mazzotti M. Optimal design of multi-energy
systems with seasonal storage. Appl Energy 2017. http://dx.doi.org/10.1016/j.
apenergy.2017.07.142.
[23] Wu R, Mavromatidis G, Orehounig K, Carmeliet J. Multiobjective optimisation of
energy systems and building envelope retrofit in a residential community. Appl
Energy 2017;190:634–49. http://dx.doi.org/10.1016/j.apenergy.2016.12.161.
[24] Stadler M, Groissböck M, Cardoso G, Marnay C. Optimizing Distributed Energy
Resources and building retrofits with the strategic DER-CAModel. Appl Energy
2014;132:557–67. http://dx.doi.org/10.1016/j.apenergy.2014.07.041.
[25] Ren H, Zhou W, Gao W. Optimal option of distributed energy systems for building
complexes in different climate zones in China. Appl Energy 2012;91:156–65.
http://dx.doi.org/10.1016/j.apenergy.2011.08.044.
[26] Di Somma M, Yan B, Bianco N, Graditi G, Luh PB, Mongibello L, et al. Multi-
objective design optimization of distributed energy systems through cost and ex-
ergy assessments. Appl Energy 2017;204:1299–316. http://dx.doi.org/10.1016/j.
apenergy.2017.03.105.
[27] Kienzle F, Ahcin P, Andersson G. Valuing investments in multi-energy conversion,
storage, and demand-side management systems under uncertainty. IEEE Trans
Sustain Energy 2011;2:194–202. http://dx.doi.org/10.1109/TSTE.2011.2106228.
[28] Mavrotas G, Florios K, Vlachou D. Energy planning of a hospital using
Mathematical Programming and Monte Carlo simulation for dealing with un-
certainty in the economic parameters. Energy Convers Manag 2010;51:722–31.
http://dx.doi.org/10.1016/j.enconman.2009.10.029.
[29] Lythcke-Jørgensen C, Ensinas AV, Münster M, Haglind F. A methodology for de-
signing flexible multi-generation systems. Energy 2016;110:34–54. http://dx.doi.
org/10.1016/j.energy.2016.01.084.
[30] Hawkes AD, Leach MA. Modelling high level system design and unit commitment
for a microgrid. Appl Energy 2009;86:1253–65. http://dx.doi.org/10.1016/j.
apenergy.2008.09.006.
[31] Li C-Z, Shi Y-M, Liu S, Zheng Z, Liu Y. Uncertain programming of building cooling
heating and power (BCHP) system based on Monte-Carlo method. Energy Build
2010;42:1369–75. http://dx.doi.org/10.1016/j.enbuild.2010.03.005.
[32] Dufo-López R, Pérez-Cebollada E, Bernal-Agustín JL, Martínez-Ruiz I. Optimisation
of energy supply at off-grid healthcare facilities using Monte Carlo simulation.
Energy Convers Manag 2016;113:321–30. http://dx.doi.org/10.1016/j.enconman.
2016.01.057.
[33] Saltelli A, editor. Sensitivity analysis in practice: a guide to assessing scientific
models. Hoboken, NJ: Wiley; 2004.
[34] Saltelli A, Annoni P. How to avoid a perfunctory sensitivity analysis. Environ
Model Softw 2010;25:1508–17. http://dx.doi.org/10.1016/j.envsoft.2010.04.012.
[35] Saltelli A, editor. Global sensitivity analysis: the primer. Chichester, England;
Hoboken, NJ: John Wiley; 2008.
[36] Borgonovo E, Plischke E. Sensitivity analysis: a review of recent advances. Eur J
Oper Res 2016;248:869–87. http://dx.doi.org/10.1016/j.ejor.2015.06.032.
[37] Iooss B, Lemaître P. A review on global sensitivity analysis methods. In: Dellino G,
Meloni C, editors. Uncertain. Manag. Simul.-Optim. Complex Syst., vol. 59, Boston,
MA: Springer US; 2015, p. 101–22.
[38] Tian W. A review of sensitivity analysis methods in building energy analysis.
Renew Sustain Energy Rev 2013;20:411–9. http://dx.doi.org/10.1016/j.rser.
2012.12.014.
[39] Ashouri A, Petrini F, Bornatico R, Benz MJ. Sensitivity analysis for robust design of
building energy systems. Energy 2014;76:264–75. http://dx.doi.org/10.1016/j.
energy.2014.07.095.
[40] Bracco S, Dentici G, Siri S. DESOD: a mathematical programming tool to optimally
design a distributed energy system. Energy 2016;100:298–309. http://dx.doi.org/
10.1016/j.energy.2016.01.050.
[41] Bustos F, Toledo A, Contreras J, Fuentes A. Sensitivity analysis of a photovoltaic
solar plant in Chile. Renew Energy 2016;87(Part 1):145–53. http://dx.doi.org/10.
1016/j.renene.2015.09.070.
[42] Lindberg KB, Doorman G, Fischer D, Korpås M, Ånestad A, Sartori I. Methodology
for optimal energy system design of Zero Energy Buildings using mixed-integer
linear programming. Energy Build 2016;127:194–205. http://dx.doi.org/10.
1016/j.enbuild.2016.05.039.
[43] Mehleri ED, Sarimveis H, Markatos NC, Papageorgiou LG. Optimal design and
operation of distributed energy systems: application to Greek residential sector.
Renew Energy 2013;51:331–42. http://dx.doi.org/10.1016/j.renene.2012.09.009.
[44] Ren H, Gao W, Ruan Y. Optimal sizing for residential CHP system. Appl Therm Eng
2008;28:514–23. http://dx.doi.org/10.1016/j.applthermaleng.2007.05.001.
[45] Ren H, Gao W. A MILP model for integrated plan and evaluation of distributed
energy systems. Appl Energy 2010;87:1001–14. http://dx.doi.org/10.1016/j.
apenergy.2009.09.023.
[46] Rubio-Maya C, Uche-Marcuello J, Martínez-Gracia A, Bayod-Rújula AA. Design
optimization of a polygeneration plant fuelled by natural gas and renewable en-
ergy sources. Appl Energy 2011;88:449–57. http://dx.doi.org/10.1016/j.
apenergy.2010.07.009.
237
Applied Energy 214 (2018) 219–238
[47] Yoshida S, Ito K, Yokoyama R. Sensitivity analysis in structure optimization of
energy supply systems for a hospital. Energy Convers Manag 2007;48:2836–43.
http://dx.doi.org/10.1016/j.enconman.2007.06.045.
[48] Zhou Z, Liu P, Li Z, Ni W. Economic assessment of a distributed energy system in a
new residential area with existing grid coverage in China. Comput Chem Eng
2013;48:165–74. http://dx.doi.org/10.1016/j.compchemeng.2012.08.013.
[49] Li L, Mu H, Gao W, Li M. Optimization and analysis of CCHP system based on
energy loads coupling of residential and office buildings. Appl Energy
2014;136:206–16. http://dx.doi.org/10.1016/j.apenergy.2014.09.020.
[50] Moret S, Bierlaire M, Maréchal F. Robust optimization for strategic energy plan-
ning. Lausanne: Transport and Mobility Laboratory; 2014.
[51] Morris MD. Factorial sampling plans for preliminary computational experiments.
Technometrics 1991;33:161–74. http://dx.doi.org/10.2307/1269043.
[52] Sudret B. Uncertainty propagation and sensitivity analysis in mechanical models.
Contributions to structural reliability and stochastic spectral methods. Clermont-
Ferrand: Université Blaise-Pascal; 2007.
[53] Geidl M, Koeppel G, Favre-Perrod P, Klockl B, Andersson G, Frohlich K. Energy
hubs for the future. IEEE Power Energy Mag 2007;5:24–30. http://dx.doi.org/10.
1109/MPAE.2007.264850.
[54] Evins R, Orehounig K, Dorer V, Carmeliet J. New formulations of the ‘energy hub’
model to address operational constraints. Energy 2014;73:387–98. http://dx.doi.
org/10.1016/j.energy.2014.06.029.
[55] Orehounig K, Evins R, Dorer V. Integration of decentralized energy systems in
neighbourhoods using the energy hub approach. Appl Energy 2015;154:277–89.
http://dx.doi.org/10.1016/j.apenergy.2015.04.114.
[56] Mavromatidis G, Orehounig K, Carmeliet J. Evaluation of photovoltaic integration
potential in a village. Sol Energy 2015;121:152–68. http://dx.doi.org/10.1016/j.
solener.2015.03.044.
[57] Mavromatidis G, Evins R, Orehounig K, Dorer V, Carmeliet J. Multi-objective
optimization to simultaneously address energy hub layout, sizing and scheduling
using a linear formulation. In: Araujo A, editor. Eng. Optim. 2014, CRC Press;
2014, p. 603–8.
[58] Haimes Y, Ladson LS, Wismer DA. On a bicriterion formulation of the problems of
integrated system identification and system optimization. IEEE Trans Syst Man
Cybern 1971;1:296–7. http://dx.doi.org/10.1109/TSMC.1971.4308298.
[59] Aven T. Interpretations of alternative uncertainty representations in a reliability
and risk analysis context. Reliab Eng Syst Saf 2011;96:353–60. http://dx.doi.org/
10.1016/j.ress.2010.11.004.
[60] Chowdhury S, Champagne P, McLellan PJ. Uncertainty characterization ap-
proaches for risk assessment of DBPs in drinking water: a review. J Environ
Manage 2009;90:1680–91. http://dx.doi.org/10.1016/j.jenvman.2008.12.014.
[61] Mavromatidis G, Orehounig K, Carmeliet J. A review of uncertainty character-
isation approaches for the optimal design of distributed energy systems. Renew
Sustain Energy Rev, submitted for publication.
[62] Mirkhani S, Saboohi Y. Stochastic modeling of the energy supply system with
uncertain fuel price – a case of emerging technologies for distributed power gen-
eration. Appl Energy 2012;93:668–74. http://dx.doi.org/10.1016/j.apenergy.
2011.12.099.
[63] Prognos AG. Die Energieperspektiven fur die Schweiz bis 2050. Swiss Federal
Office of Energy SFOE (Bundesamt für Energie BFE); 2012.
[64] Densing M, Hirschberg S, Turton H. Review of Swiss electricity scenarios 2050.
PSI–Report 14-05 Paul Scherrer Institute, Villigen PSI, Switzerland (2014) ISSN
1019-0643; 2014. Available from: < http://www.psi.ch/eem/
PublicationsTabelle/PSI-Bericht_14-05.pdf > .
[65] SFOE. Schweizerische Gesamtenergiestatistik 2015. Swiss Federal Office of Energy
(SFOE); 2015. Available from: < http://www.bfe.admin.ch/themen/00526/
00541/00542/00631/index.html?lang=en&dossier_id=00763 > .
[66] Carpentiero V, Langella R, Testa A. Hybrid wind-diesel stand-alone system sizing
accounting for component expected life and fuel price uncertainty. Electr Power
Syst Res 2012;88:69–77. http://dx.doi.org/10.1016/j.epsr.2012.02.003.
[67] IEA. Energy Policies of IEA Countries – Switzerland. International Energy Agency
(IEA), Paris, France; 2012.
[68] Burhenne S, Tsvetkova O, Jacob D, Henze GP, Wagner A. Uncertainty quantifi-
cation for combined building performance and cost-benefit analyses. Build Environ
2013;62:143–54. http://dx.doi.org/10.1016/j.buildenv.2013.01.013.
[69] Giannakoudis G, Papadopoulos AI, Seferlis P, Voutetakis S. Optimum design and
operation under uncertainty of power systems using renewable energy sources and
hydrogen storage. Int J Hydrog Energy 2010;35:872–91. http://dx.doi.org/10.
1016/j.ijhydene.2009.11.044.
[70] de Wit S. Uncertainty in building simulation. In: Augenbroe G, Malkawi A, editors.
Adv. Build. Simul., Oxford: Taylor & Francis; 2003.
[71] Gang W, Augenbroe G, Wang S, Fan C, Xiao F. An uncertainty-based design op-
timization method for district cooling systems. Energy 2016;102:516–27. http://
dx.doi.org/10.1016/j.energy.2016.02.107.
[72] Hopfe CJ, Hensen JLM. Uncertainty analysis in building performance simulation
for design support. Energy Build 2011;43:2798–805. http://dx.doi.org/10.1016/j.
enbuild.2011.06.034.
[73] EnergyPlus. EnergyPlus Simulation Software; 2016. Available from: < https://
energyplus.net/ > .
[74] Macdonald IA. Quantifying the effects of uncertainty in building simulation.
University of Strathclyde; 2002.
[75] Ioannou A, Itard LCM. Energy performance and comfort in residential buildings:
sensitivity for building parameters and occupancy. Energy Build 2015;92:216–33.
http://dx.doi.org/10.1016/j.enbuild.2015.01.055.
[76] Landolt J. A bottom-up modelling approach to address sustainable transformation
strategies for the Swiss building stock. ETH Zurich; 2016.
G. Mavromatidis et al.
[77] Wang D, Landolt J, Orehounig K, Carmeliet J. Dynamic urban energy demand
modelling to address building retrofitting alternatives in Switzerland. Status-
Semin. 2016 Zurich Switz.; 2016.
[78] SIA. SIA 2024 Standard-Nutzungsbedingungen fuer die Energie- und
Gebaeudetechnik. Swiss society of Engineers and Architects; 2006.
[79] Gang W, Wang S, Shan K, Gao D. Impacts of cooling load calculation uncertainties
on the design optimization of building cooling systems. Energy Build 2015;94:1–9.
http://dx.doi.org/10.1016/j.enbuild.2015.02.032.
[80] IPCC. Climate Change 2014: Synthesis Report. Contribution of Working Groups I,
II and III to the Fifth Assessment Report of the Intergovernmental Panel on Climate
Change [Core Writing Team, R.K. Pachauri and L.A. Meyer (eds.)]. IPCC, Geneva,
Switzerland; 2014. 151 pp.
[81] Giorgi F, Jones C, Asrar GR. Addressing climate information needs at the regional
level: the CORDEX framework. World Meteorol Organ WMO Bull 2009;58:175.
[82] Belcher S, Hacker J, Powell D. Constructing design weather data for future cli-
mates. Build Serv Eng Res Technol 2005;26:49–61. http://dx.doi.org/10.1191/
0143624405bt112oa.
[83] Mathworks. MATLAB 2016b. The MathWorks, Inc., Natick, Massachusetts, USA;
2016.
[84] Kucherenko S, Albrecht D, Saltelli A. Exploring multi-dimensional spaces: a
Comparison of Latin Hypercube and Quasi Monte Carlo Sampling Techniques.
ArXiv150502350 Stat; 2015.
[85] Homma T, Saltelli A. Importance measures in global sensitivity analysis of non-
linear models. Reliab Eng Syst Saf 1996;52:1–17. http://dx.doi.org/10.1016/
0951-8320(96)00002-6.
[86] Williams HP. Model building in mathematical programming. 5th ed. Hoboken,
N.J: Wiley; 2013.
[87] Morvaj B, Evins R, Carmeliet J. Optimization framework for distributed energy
systems with integrated electrical grid constraints. Appl Energy
2016;171:296–313. http://dx.doi.org/10.1016/j.apenergy.2016.03.090.
[88] Mavromatidis G, Orehounig K, Richner P, Carmeliet J. A strategy for reducing CO2
emissions from buildings with the Kaya identity – a Swiss energy system analysis
and a case study. Energy Policy 2016;88:343–54. http://dx.doi.org/10.1016/j.
enpol.2015.10.037.
[89] Meteotest. Meteonorm. Bern, Switzerland: Meteotest; 2015.
[90] Domínguez-Muñoz F, Cejudo-López JM, Carrillo-Andrés A, Gallardo-Salazar M.
238
Applied Energy 214 (2018) 219–238
Selection of typical demand days for CHP optimization. Energy Build
2011;43:3036–43. http://dx.doi.org/10.1016/j.enbuild.2011.07.024.
[91] Schütz T, Schraven H, Harb H, Fuchs M, Müller D. Clustering algorithms for the
selection of typical demand days for the optimal design of building energy systems.
Proc. ECOS 2016; 2016.
[92] Jakob M, Kallio S, Nägeli, C, Ott W, Bolliger, R, von Grünigen S. INSPIRE-Tool.
TEP Energy GmbH & econcept AG. Supported by the Swiss Federal Office of Energy
(SFOE) – EnergieSchweiz; 2015. Available from: < http://www.inspire-tool.ch > .
[93] Jenni Energietechnik. Hot or Cold Storage Systems; 2016. Available from: <
http://www.jenni.ch/files/jenni/inhalte/pdf/Produkte/Preise_Solartank.pdf > .
[94] Zernez. Zernez Energia 2020; 2015. Available from: < http://zernezenergia2020.
ch/2015 > .
[95] Marnay C, Venkataramanan G, Stadler M, Siddiqui AS, Firestone R, Chandran B.
Optimal technology selection and operation of commercial-building microgrids.
IEEE Trans Power Syst 2008;23:975–82. http://dx.doi.org/10.1109/TPWRS.2008.
922654.
[96] Gudmundsson O, Thorsen JE, Zhang L. Cost analysis of district heating compared
to its competing technologies; 2013, p. 3–13. 10.2495/ESUS130091.
[97] Campolongo F, Cariboni J, Saltelli A. An effective screening design for sensitivity
analysis of large models. Environ Model Softw 2007;22:1509–18. http://dx.doi.
org/10.1016/j.envsoft.2006.10.004.
[98] Pujol G, Iooss B, Boumhaout AJ with contributions from K, Veiga SD, Fruth J,
Gilquin L, et al. sensitivity: Global Sensitivity Analysis of Model Outputs. R
package version 1.12; 2016. Available from: < http://CRAN.R-project.org/
package=sensitivity > .
[99] Saltelli A, Annoni P, Azzini I, Campolongo F, Ratto M, Tarantola S. Variance based
sensitivity analysis of model output. Design and estimator for the total sensitivity
index. Comput Phys Commun 2010;181:259–70. http://dx.doi.org/10.1016/j.cpc.
2009.09.018.
[100] Sobol’ IM, Tarantola S, Gatelli D, Kucherenko SS, Mauntz W. Estimating the ap-
proximation error when fixing unessential factors in global sensitivity analysis.
Reliab Eng Syst Saf 2007;92:957–60. http://dx.doi.org/10.1016/j.ress.2006.07.
001.
[101] Jansen MJW. Analysis of variance designs for model output. Comput Phys
Commun 1999;117:35–43. http://dx.doi.org/10.1016/S0010-4655(98)00154-4.