Applied Energy 195 (2017) 616–633

Contents lists available at ScienceDirect

Applied Energy
journal homepage: www.elsevier.com/locate/apenergy

Operational planning and optimal sizing of microgrid considering

multi-scale wind uncertainty
Joohyun Shin a, Jay H. Lee a,⇑, Matthew J. Realff b
a
Chemical and Biomolecular Engineering Department, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea
b
Chemical and Biomolecular Engineering Department, Georgia Institute of Technology, Atlanta, GA, USA

h i g h l i g h t s

A microgrid is managed by considering dynamic and uncertain nature of the system.

2SSP for a day operation is combined with a day-to-day MDP for temporal connection.

Size of the system component is optimized based on the value function for the MDP.

A multi-scale wind model is developed for integration of the decision hierarchy.

The proposed method is examined for a benchmark case study with real wind data.

a r t i c l e i n f o a b s t r a c t

Article history: Distributed and on-site energy generation and distribution systems employing renewable energy sources
Received 22 November 2016 and energy storage devices (referred to as microgrids) have been proposed as a new design approach to
Received in revised form 16 March 2017 meet our energy needs more reliably and with lower carbon footprint. Management of such a system is a
Accepted 17 March 2017
multi-scale decision-making problem encompassing hourly dispatch, daily unit commitment (UC), and
yearly sizing for which efficient formulations and solution algorithms are lacking thus far. Its dynamic
nature and high uncertainty are additional factors in limiting efficient and reliable operation. In this
Keywords:
study, two-stage stochastic programming (2SSP) for day-ahead UC and dispatch decisions is combined
Microgrid operation and design
Multi-scale decision making
with a Markov decision process (MDP) evolving at a daily timescale. The one-day operation model is inte-
Wind uncertainty grated with the MDP by using the value of a state of commitment and battery at the end of a day to
Stochastic optimization ensure longer term implications of the decisions within the day are considered. In the MDP formulation,
Value function approximation capturing daily evolving exogenous information, the value function is recursively approximated with
sampled observations estimated from the daily 2SSP model. With this value function capturing all future
operating costs, optimal sizing of the wind farm and battery devices is determined based on a surrogate
function optimization. Meanwhile, a multi-scale wind model consistent from seasonal to hourly is devel-
oped for the connection of the decision hierarchy across the scales. The results of the proposed integrated
approach are compared to those of the daily independent 2SSP model through a case study and real wind
data.
Ó 2017 Elsevier Ltd. All rights reserved.

1. Introduction
An energy system (referred to as ‘grid’) has to be managed for
stable and efficient generation and distribution. A new concept
has emerged recently, the smart grid, which is an intelligent energy
grid system including a variety of advanced energy supplier/
customer such as renewable energy resources, smart meters, and
⇑ Corresponding author.
E-mail addresses: sinnis379@kaist.ac.kr (J. Shin), jayhlee@kaist.ac.kr (J.H. Lee),
matthew.realff@chbe.gatech.edu (M.J. Realff).
electric cars with improved efficiency by integrating with ICT along
the entire energy supply networks. As the infrastructure of grid
becomes more advanced in terms of energy generation, informa-
tion sharing/management, and communication with complex and
fully integrated network, smart management applications and ser-
vices should keep pace with it in order to achieve the objectives
related to supply and demand balance, operation cost reduction,
and utility maximization [1,2]. However, the smart grid manage-
ment in its essence is a stochastic dynamic optimization problem
having a multi-time scale, multi-period decision horizon and high
uncertainty, for which efficient mathematical problem formula-
tions and solution algorithms are lacking thus far.

http://dx.doi.org/10.1016/j.apenergy.2017.03.081
0306-2619/Ó 2017 Elsevier Ltd. All rights reserved.
 J. Shin et al. / Applied Energy 195 (2017) 616–633 617

Nomenclature

Indices/Sets bdm;h wind model parameter denoting the effect of daily-

d index for days average for month m hour h
Dm set of days in month m bh1
m;h wind model parameter denoting the effect of previous
g generators hour for month m hour h
G set of generators bSR
m; vector of parameters in the surrogate value function for
Gs set of slow-start generators month m
h hours c discount factor for the value function approximation
H set of daily hours gch charging efficiency of battery
Huc set of unit commitment decision epochs hnm n-th iterated coefficient vector of the value function for
k energy dispatch sources month m
K set of energy dispatch sources containing producing lem;h hourly mean value of random noise in the intraday wind
from generators, charging/dis-charging battery, buy- model for month m
ing/selling, which is defined by {G, ch, disch, buy, sell} pam;d probability of intraday wind scenario realized in time
n iterative numbers for the value function approximation (m, d)
N the number of iterations for the value function approx- pem state transition probability of daily average wind value
imation for month m
m months rb self-discharging rate of battery
M set of months rem;h hourly standard deviation of random noise in the intra-
t multi-scale time index which is defined by (m, d, h) (or day wind model for month m
just simplified h in the one-day operation model)
Am;h set of the wind model parameters related to the hourly Variables
x
variation for month m bt battery level in time t given scenario x
Em set of the wind model parameters related to the inter- cw capacity of wind generators
day variation for month m cb capacity of battery
x intraday wind scenarios Dtg shut-down cost of generator g in time t
Xam;d set of intraday wind scenarios realized in time (m, d) x
dt demand in time t given scenario x
Xem discrete space of daily average wind values for month m em;h hourly random noise in the intraday wind model for
month m
Parameters px
k;t dispatch decision from source k in time t given scenario
Bmin lower limit of battery state x
Bmax upper limit of battery state px wind;t wind power output in time t given scenario x
C Ug unit start-up cost of generator g sm;d day-to-day state vector in time (m, d)
C Dg unit shut-down cost of generator g sxt lost-demand in time t given scenario x
utg unit commitment of generator g in time t
C Sg unit setup cost of generator g
U tg start-up cost of generator g in time t
C disp;k unit dispatch cost of source k
v^ nm n-th iterated sampled observation for the value function
C lost unit penalty cost of lost-demand approximation for month m
Cw capa unit investment cost for capacity acquisition of wind wt wind speed in time t
plant
wday daily average wind value in time (m, d)
C bcapa unit investment cost for capacity acquisition of battery m;d
^ day
w exogenous information variable of daily average wind
emin
m;h lower bound of hourly random noise in the intraday m;d
value realized in time (m, d)
wind model for month m
xw capacity acquisition of wind generator
emax
m;h upper bound of hourly random noise in the intraday capa

wind model for month m xbcapa capacity acquisition of battery

MT minimum up/down time limitation of fuel-based gener- xm;d vector of daily operational decisions in time (m, d)
ators x1;m;d vector of 1st stage decisions in the one-day operation
Pkmin lower limit of dispatch source k model in time (m, d)
xx vector of 2nd stage decisions in the one-day operation
Pkmax upper limit of dispatch source k 2;m;d
model in time (m, d) given scenario x
Pwr rated electrical power in the wind conversion model
g
Rdown hourly ramp-up limit of generator g
g
Functions
Rup hourly ramp-down limit of generator g C daily operational cost function
wc cut-in wind speed in the wind conversion model f 1st stage cost function in the one-day operation model
wr rated wind speed in the wind conversion model Qx 2nd stage cost function in the one-day operation model
wf cut-off wind speed in the wind conversion model given scenario x
bWIND
m;h vector of parameters in the intraday wind model for m
V approximated value function for month m
month m hour h ^ SR
V surrogated function of the value function for month m
m
bhm;h wind model parameter denoting the bias of daily-hour
/VF basis function for the value function approximation
for month m hour h
/WIND basis function for the wind model

One of the most promising new grid paradigms is the microgrid with conventional fuel-based generators, and energy storage
(MG) [3], which is for providing energy in a small and localized devices to make up for the intermittent nature of the renewable
area with its own distribution grid. A MG has generally multiple energy sources. The choice of a suitable generation mix is entirely
distributed generators including renewable energy sources along site-specific depending on the availability of the various renewable
618 J. Shin et al. / Applied Energy 195 (2017) 616–633
energy sources. Among the renewable sources of power, wind is
one of the fastest growing. As a motivating example of this study,
Jeju, which is a very windy island in Korea, contains the largest
wind farm in Korea and it is expected to gain considerably in total
capacity. As the capacity of wind generation increases, those of
energy storage and bi-directional HVDC (which is used to transmit
electricity) are also expected to be increased [4]. In such a hybrid
renewable energy system, the unpredictability of wind power
and dynamic nature of storage devices combined with certain con-
straints regarding energy generation or interactions with the main
grid can make the reliable and sustainable operation a real chal-
lenge. In this study, thus, an efficient decision strategy for the
design and operation of a hybrid renewable MG system is proposed
based on a rigorous forecasting model for wind uncertainty and
state information on the storage; Fig. 1 shows the MG and manage-
ment system studied in this paper.
Operational planning of such an energy system is a multi-scale,
multi-stage decision-making problem encompassing unit commit-
ment (UC) (from one day to one week ahead) and economic dis-
patch (from a few minutes to an hour ahead) [5,6]. UC plans of
conventional slow-start generators should be pre-determined at
least one day ahead because their flexibilities are limited by oper-
ational constraints such as ramping up/down limit and minimum
up/down time [7]. Generation dispatch of various generating units
and other providing sources (e.g., battery or HVDC) is then decided
on a much faster timescale to ensure timely response to faster-
changing wind or load. The dispatch decision is made to minimize
the operational costs while respecting the load balance and prede-
termined UC decision. In addition, the combination, design and siz-
ing of micro-sources should be optimally decided by capturing
trade-off between investment and operation; introducing renew-
able generators and storage requires high investment cost but at
the same time, they can reduce the lifecycle energy generation
cost. Also in order to assure the maximum utilization of wind
energy resources, it is very important to match the optimum bal-
ance between the sizing of wind power generation and a battery
bank [8]. The overall decision hierarchy of design and operation
is shown in Fig. 2 (left). As the temporal scale increases, a fully
(or approximately) integrated model is required for system
management.
For optimal design and operation of MG systems, mathematical
optimization problems have been formulated and solved. The gen-
erality of such formulations allowed for consideration of the speci-
fic condition and circumstance of each case [9–11]. Especially in
MG management, forecasting the uncertainty arising from the
intermittent and non-dispatchable nature of most renewable
energy sources and incorporating it into the decision-making prob-
lem is a significant issue. Specifically, the influence of the wind
power uncertainty on the generation cost and operational strate-
gies to address it have been investigated by several researchers
[12,13]. For the efficient consideration of system dynamics and
Fig. 1. Studied micro-grid and management system.
future disturbances, Parisio et al. [14] applied the technique of
model predictive control (MPC) to the problem of MG operation.
The resulting long-term look-ahead decision policy optimizes the
current and future costs within a chosen prediction horizon in
deciding each current action. Employing stochastic optimization
instead of deterministic optimization in the formulation can pro-
vide advantages as it can explicitly incorporate uncertainty models
into the decision making. Many published works on the UC prob-
lem construct a stochastic look-ahead policy for a finite horizon
[6,7].
One of the most general approaches for the one-day operation
of the MG is two-stage stochastic programming (2SSP), which
relies on discrete scenarios of the uncertainty realizations
[15,16]. In the first stage, day-ahead commitment decisions of tra-
ditional slow-start generators are made, while in the second stage
dispatch decisions are made as the recourse variables based on the
realized uncertainty. Niknam et al. [17] constructed a stochastic
multi-objective optimal policy for operating a MG system based
on rigorous scenario generation and reduction techniques for var-
ious uncertain sources. In [18,19], stochastic security constrained
unit commitment (SCUC) models were developed for a hybrid
renewable energy system considering transmission constraints as
well as operational constraints. Typically, these types of formula-
tions allow for the sequential revision of the decision as more
and more information on the uncertainty is observed as the time
horizon proceeds. Multi-stage models beyond the typical two-
stage formulation can be found in [20,21]. Another alternative for
stochastic optimization is robust optimization which minimizes
the worst-case cost with respect to a deterministic set of uncer-
tainty outcomes [22–24]. In [23], an integrated framework based
on multi-agent modeling and robust optimization was proposed,
and the work was later extended [24] by addressing the uncertain-
ties in the wind generation and demand which were described in
the form of prediction intervals.
These look-ahead policies are obtained with a short decision
horizon as accurate forecasting over a longer time is difficult and
computationally prohibitive. However, the use of a short horizon
in multi-period decision making can be limiting, since what is an
optimal solution within a short horizon may be highly suboptimal
over the long run beyond the considered time horizon, due to what
is known as end-effects [25]. For example, in the one-day operation
model, an optimal solution results zero level of battery and off-line
status of generators at the end of the day because there is no need
to maintain a certain level of battery and keeping the generator on
in the finite horizon operation. The problem of battery depletion in
a smart grid system is also described in [26]. To resolve this prob-
lem, the value of the system state (e.g., UC or battery level) at the
end-of-horizon should be estimated from an aggregate model with
a longer time span [27]. As an alternative, more systematic valua-
tion of being in a state at a point in time can be obtained by
stochastic dynamic programming, and thus a trade-off between
 J. Shin et al. / Applied Energy 195 (2017) 616–633
Fig. 2. Decision hierarchy for micro-gird management (left side) and the corresponding multi-scale wind characteristics (right side).
the current-stage costs and the future costs can be explicitly cap-
tured in the decision making policy [7].
In dynamic programming (DP), a decision policy is constructed
by estimating the long-term consequences of action (which is
called value function), and consequently, it can be applied to con-
trol problems involving temporally extended behavior [28].
Approximate dynamic programming (ADP) uses some sort of
stochastic simulation and function approximation methods for
learning a policy or value function to resolve the computational
challenges of DP. Taking advantages of both computation and algo-
rithmic structure, ADP based adaptive stochastic control for the
smart grid holds the promise for providing an autonomous intelli-
gent management in complicated and highly uncertain environ-
ments [29]. Xie and Ilic [30] formulated the UC problem through
policy function approximation by repeatedly solving a finite hori-
zon problem with a rolling horizon manner. Recently, there have
been several studies on the resource allocation problem for manag-
ing dispatchable energy sources using the concept of value func-
tion [31–33]. Powell et al. [31] developed a stochastic, multiscale
energy policy model that can handle high-dimensional energy dis-
patch and storage over a large network and hundreds of thousands
of time periods using the ADP algorithm. More sophisticated value
function approximation techniques were introduced for grid-level
energy storage in [32,33].
Meanwhile, for optimal design of an energy system, one should
be able to evaluate the operational performance of a fixed config-
uration. In most size optimization studies, general performance
indicators such as loss of power supply probability (for assessment
of system reliability) and annualized/levelized cost of energy (for
economic evaluation) are used [8,34]. However, sizing methods
with such performance indicators based on average values (e.g.
monthly wind speed) or worst scenarios show a tendency of
undersizing or oversizing system components [34]. The fluctuating
characteristics of uncertainty over the faster time scale (e.g., wind
speed) must be addressed in the design stage. With a computa-
tional model of the energy system, the performance of a set of pos-
sible system configurations can be assessed through simulation.
619
Ekren and Ekren [35] pointed out the conventional sizing
approaches only considered particular design scenarios; they char-
acterized how the total system cost changes with respect to the
size of design parameters using a quadratic response surface
method. In their study, hourly random input data on the renewable
sources and load are fitted to probability distributions but the total
cost is computed by a rule-based simulation without conducting
an optimization for the hourly or daily operational decisions.
Another significant challenge for managing hybrid renewable
energy systems is how to manage the uncertainty on the intermit-
tent renewable sources or energy demand for supply-demand bal-
ance. In this study, we focus on modeling the wind uncertainty of
which variations have both deterministic and stochastic character-
istics across time scales [36], and thus its forecasting can be highly
challenging. The seasonal and daily periodicities of wind time ser-
ies make them nonstationary, as well as large stochastic ramping
behaviors between the consecutive hours or days exist, as shown
in Fig. 2 (right). This variability of wind production affects the deci-
sion hierarchy of the hybrid renewable energy system across all of
the time frames: from hourly/daily variations directly related to
the operation for hourly dispatch and daily UC [12] to yearly vari-
ations affecting the long-term sizing problem performed on a
yearly basis [37]. That is, for integrating the decision hierarchy, a
multi-scale wind model consistent across the different time scales
is required reflecting all of the characteristics demonstrated above.
In general, wind model can be classified into two categories
based on the methodology used [38]. Physical approaches such
as numeric weather predictions (NWPs) use meteorological data
and they can provide accurate predictions when the weather con-
ditions are stable. However, it is difficult to gain information in
short-time with these methods due to the heavy computational
load and the associated high cost [39]. Statistical methods are
based on the analysis of historical time-series data to predict wind
speed or directly power generation [40]; it is easy and inexpensive
to model, and effective for relatively short-term predictions (few
minutes to an hour). Recently, hybrid approaches combining the
physical and statistical methods have begun to appear. A number
620 J. Shin et al. / Applied Energy 195 (2017) 616–633
of accurate wind models have been developed, but most of them
are valid only for a specific time scale considered.
In this research, a two-stage stochastic programming (2SSP)
model is formulated for one-day operation: UC (first stage) and dis-
patch (second stage) with a number of intra-day scenarios of wind
uncertainty capturing the hourly change. Here, the difference from
the conventional 2SSP model is that the value of being in a state of
current commitment and battery at the end-of-day (called value
function) is incorporated into the objective function to ensure
longer term implications of the decisions are duly considered.
Meanwhile, in the daily-evolving Markov decision process (MDP)
formulation, inter-day transition information on wind is captured,
and the value function is approximated with a linear model with a
basis function of system state vector (e.g. wind, battery, and com-
mitment). The coefficient vector of the linear model is recursively
updated based on the error between the previous approximation
and a sampled observation estimated from simulating the daily
2SSP model.
The value function which is approximated on the daily opera-
tional level can be used for the yearly sizing problem. Sample data
of the value function is computed for a given capacity of energy
battery and wind farm, and a surrogate model is constructed with
these sample data within a design space of battery and wind farm
parameters. Optimal sizing problem can thus be formulated based
on the capital costs as well as the surrogate function of the opera-
tional costs. Since the value function contains all future operational
costs, an extended horizon solution can be obtained. The main dis-
tinguishing feature of our approach from those in the previous
studies on size optimization is that a surrogate model adopted
for design phase is built with data from stochastic simulations in
which optimizations for the operational decisions are embedded.
For the abovementioned temporal integration of the decision
hierarchy, a general wind model for time-scales from seasonal to
hourly is developed to enable seamless connection of the decision
making across the scales. A monthly-based model is constructed to
remove the seasonality, and inter-day exogenous information on
wind is captured by modeling a daily average time series as a sta-
tionary Markov chain for each distinct month. At the same time, to
represent intra-day scenarios of wind uncertainty, daily periodicity
is extracted from the data, and hourly ramping up and down move-
ments are added to the deterministic profile.
The rest of the paper is organized as follows. Section 2 provides
theoretical backgrounds on multi-period decision under uncer-
tainty, especially 2SSP and MDP, and surrogate models. In Section 3,
the studied energy system is introduced and the multi-scale wind
model is developed and validated through real wind data. In Sec-
tion 4, the proposed decision-making strategy for design and oper-
ation of the hybrid renewable MG is proposed, and of which
usefulness is proved through a simple benchmark case study in
Section 5.
2. Methods
There are mainly two modeling approaches to address the prob-
lem of multi-period decision-making under uncertainty [41].
Stochastic programming is to find a sequence of decisions over
time based on the realization of random outcomes (commonly
expressed as discrete scenarios) up to the time of each decision.
On the other hand, MDP is based on a state-driven model decom-
posed by each decision time and probabilistic state transitions for
capturing both physical system dynamics and the flow of informa-
tion. Each of the methods is reviewed in Section 2.1 and Section 2.2,
respectively. Lastly, parametric surrogate models for estimating
interest output within a design space are reviewed in Section 2.3.
2.1. Two-stage stochastic programming
In two-stage stochastic programming (2SSP), the decision vari-
ables are categorized by whether they are resolved before or after
the uncertainty realization. The first stage is to find ‘here-and-now’
decisions x1 to minimize the objective function composed of the
first stage cost f(x1) and expectation of the second recourse func-
tion Q(x1, x) over all possible scenarios x. Since the first stage
decisions are made before random outcomes are realized, x1 can
consider only the prior knowledge of uncertainty. After informa-
tion on the realization of x is given, in the second stage, the
‘wait-and-see’ (or recourse) variables x2 are to be decided based
on the realized outcome by solving the second equation in Eq. (1).
minff ðx1 Þ þ Ex ½Q ðx1 ; xÞg
x1 2X 1
ð1Þ
Qðx1 ; xÞ ¼ minfqðx2 ; xÞjT x x1 þ W x x2 6 hx g
x2
Note that the recourse nature of the second stage decision vari-
ables x2 affects the optimal decision of the first stage decision x1,
thus the need for the two-stage formulation. With a discrete and
finite scenario space X and all linear functions, the stochastic
model Eq. (1) can be reformulated as the full deterministic equiv-
alent linear form, as Eq. (2). Here, the expectation of the recourse
function is calculated through the prior knowledge of the uncer-
tainty and its probability distribution (x, px). By taking advantage
of the block structure of the two-stage extensive form, efficient
solution algorithms using decomposition techniques have been
developed [42].
(  )
X 
x x
min f ðx1 Þ þ px qðx2 ; xÞx1 2 X 1 ; T x x1 þ W x x2 6 hx ð2Þ
x1 ;xx
2 x2X

For a dynamic system where the recourse actions are taken in a
sequential fashion based on a time sequence of random outcomes,
a multi-stage formulation is needed. In such problems, a decision
policy allows sequential revisions of the decision as more informa-
tion on the uncertainty is observed. However, the number of sce-
narios increases exponentially as the number of decision stages
increases, and consequently, the problem size can become quickly
intractable. Without taking advantage of the special structure of
SP, computation is inefficient. Lee [43] shows the difficulty associ-
ated with addressing the multi-scale, multi-period energy supply
chain design and operation problem by using the SP formulation.
2.2. Value function approximation
A formal Markov decision process (MDP) formulation requires
following specifications with time index t: (1) state variable, st to
compute all future dynamics of the system, (2) decision variable,
xt, (3) exogenous information variable, xt, which is expressed in
the form of random variables governed by probability distribu-
tions, (4) stage-wise cost function, Ct (st, xt), and (5) state transition
function, st+1 = St (st, xt, xt+1). Decisions are represented by a deci-
sion policy p, which is a map indicating which action to take for
any given state [44]. The problem of finding the best policy is to
minimize all future costs,
(  )
X1 
c C t ðst ; pt ðst ÞÞs0
V 0 ðs0 Þ ¼ min E t
ð3Þ
pt 2Pt
t¼1

where a discount factor c < 1 is used to ensure the convergence of
the infinite horizon cost. The stochastic optimization problem of
the form Eq. (3) can be solved by recursively computing the follow-
ing optimality equation which is called the Bellman equation:
V t ðst Þ ¼ minfC t ðst ; xt Þ þ cE½V tþ1 ðstþ1 Þjst g ð4Þ
xt 2X t
 J. Shin et al. / Applied Energy 195 (2017) 616–633
where Vt(st) is defined by value function (or value-to-go) containing
all future costs from time t at state st. Since the formulation of DP
allows us to derive an optimal feedback control policy off-line
through the Bellman equation, it can yield a significantly improved
policy over the approach of solving an open-loop control problem
formulated on a finite moving prediction horizon [45], i.e., DP can
reduce the performance loss by the mismatch between the mathe-
matical formulation and the reality of uncertainty and information
propagation in multi-stage decision making.
The infinite horizon problem is often formulated as a stationary
problem where the cost function, transition function, and the pro-
cess governing the exogenous information process do not vary over
time [46]; consequently, value function V in Eq. (4) can be replaced
with a time invariant one (without index t). Stationary value func-
tion is thus iteratively computed until it converges, and this proce-
dure is called value iteration. However, finding an exact solution by
solving the optimality equation is computationally infeasible in
most practical problems due to the huge dimensions of the state
and action spaces, which is termed the curse of dimensionality. A
common strategy for alleviating this computational bottleneck is
to replace the exact value function with some sort of approxima-
tion. One of the simplest and most popular approximation strate-
gies in energy system problems is to use a linear model with
univariate basis functions /f parameterized by state variables
[6,47],
X

VðsjhÞ ¼ hf /f ðsÞ ð5Þ
f 2F
Then value function is iteratively approximated with the itera-
tion counter n, which proceeds as the stochastic system is simu-
lated with the policy based on the most current value function
approximation. The sampled observation of the value function at
the visited state sn is estimated by (6). The function’s coefficients
are then recursively updated with the error between the previous
prediction and this observation. This method is generally referred
to as approximate dynamic programming (ADP).
( " #)
X n1
v
^n ¼ min
n
Cðs ; xÞ þ cE
n
hf /f ðs Þ
nþ1
ð6Þ
x2A
f 2F
2.3. Surrogate model
Surrogate function, which is a map indicating how response val-
ues change with respect to design variable, is established based on
a pair of N sampled data sets, S = (x1, . . . , xN) and fs = (f1, . . . , fN). The
output f(x) for any untried design option x can be predicted by the
surrogate model: f ðxÞ ¼ ^f ðxÞ þ eðxÞ. This surrogate model is useful
when it is computationally infeasible to try to estimate the output
values in the continuous design space with the original model.
Parametric approaches for constructing surrogate models choose
a global functional form of the direct relationship between the
response variable and the design variables; popular methods of
such parametric approaches are polynomial response surface and
kriging [48,49].
In polynomial regression model (PRG), the interesting output is
estimated with polynomial basis functions zj depending on the
sample data xi, and the errors are assumed to be independent ran-
dom variable with zero mean. Linear formulation for each observa-
tion is shown in Eq. (7) with a set of basis functions, NPRG. The
vector of the estimated parameters then can be found by least
squares.
X
NPRG
f i ðxÞ ¼ bj zj ðxi Þ þ ei ; ei  Nð0; r2 Þ ð7Þ
j¼1
621
Kriging model (KRG) is developed in the field of spatial statistics
and the general formulation of the model is the sum of two compo-
nents: the linear model (e.g., polynomial trend) and a residual rep-
resenting the fluctuations around the trend, with the basic
assumption being that the latter are correlated and the correlation
depends on the distance between the sample data points [49].
f ðxÞ ¼ l þ eðxÞ; where EðeÞ ¼ 0; covðeðxi Þ; eðx j ÞÞ – 0 ð8Þ
PRG has seen considerable use in recent engineering optimiza-
tion studies due to its computational simplicity and ease of use, but
low-ordered polynomial models may be of limited accuracy when
the response data to be modeled have multiple local extrema. On
the other hand, KRG has the flexibility to model response data with
multiple local extrema but the much higher computational
expense of KRG limits its use [50].
3. Model
3.1. System model
The studied small and isolated power system has its own grid
but connected with the main grid through HVDC (grid-tied sys-
tem). Two types of generators are considered: Slow-start genera-
tors have some operational limitations and thus their
commitment plans should be decided at least a day ahead, and
fast-start generators (such as gas turbine) provide much more flex-
ible energy sources. The hybrid renewable system contains a wind
farm with significant generation capacity and a battery storage
device. System models for the different micro-grid components
are constructed by referring to previous studies [8,51] and revising
them appropriately for this work.
Wind power conversion model: In this study, the wind power
output pwind is approximated as a function of wind speed distribu-
tion, described by
8 w
< Pr
> ðwr 6 w 6 wf Þ
ðwwc Þ
pwind ðwÞ ¼ Pw ðwc 6 w 6 wr Þ ð9Þ
>
:
r ðwr wc Þ
0 ðw < wc or w > wf Þ
where Pw r is the rated electrical power, wc is the cut-in wind speed,
wr is the rated wind speed, and wf is the cut-off wind speed.
Battery model: In most previous studies, charging or discharging
of the battery is automatically computed by the difference
between the energy production and required load. However in this
study, these are considered as decision variables since the system
is connected to the main grid; the extra or deficit of energy can
be sold to or bought from the main grid, respectively. The state
of a battery is thus decided by charging or discharging variables
bh ¼ bh1 ð1  rb Þ þ pch;h gch  pdisch;h ð10Þ
where bh is the battery state at time h, pch,h is the charging amount
to the battery, pdisch,h is the discharging amount from the battery, rb
is the self-discharging rate, and gch is the charging efficiency,
respectively.
Daily demand: General daily demand profile is characterized by
two peaks; the transition from relatively lower loads to higher
loads in the morning which is called the morning ramp, and the
highest peak in the evening. The equality constraint of load satis-
faction is relaxed by using the variable ‘lost-demand’ which is
the unsatisfied load for problem feasibility.
Operational cost model: All dispatch costs follow a linear model.
The operation of a slow-start generator requires startup, shut-
down, and setup cost, decided by commitment decision; and the
variable cost linearly increases with the production amount. In
the case of a fast-start generator, the only linearly variable cost is
622 J. Shin et al. / Applied Energy 195 (2017) 616–633

imposed but the unit production costs of the fast-start generators during the summer, and higher average wind output during the
are much more expensive compared to those of the slow-start gen- winter. The higher the wind output, the more variability is
erators. Generation from the wind plant and discharging from the observed, and it makes forecasting more difficult. The third column
battery are assumed to be free. Electricity prices for buying from from the left in Table 1 presents the autocorrelation coefficient
and selling to the main grid are fixed. Penalty costs are imposed with time-lag of an hour. The current wind value has a large posi-
for lost-demand. tive correlation with the value of an hour before, especially in the
winter and spring season.
Wind time series has both deterministic and stochastic charac-
3.2. Wind uncertainty model
teristics. The seasonal and daily periodicities of wind time series
make them nonstationary, and by eliminating these periodicities,
Hourly average wind speed data used are given from the
we can obtain stationary (or approximately stationary) stochastic
National Wind Technology Center, located in Boulder, Colorado,
time series [36]. On the other hand, intra-day variations feature
USA. Datasets of twelve years (2002–2014, except 2011) are used
deterministic daily hours profiles as well as stochastic hourly
for the wind data analysis and wind model identification. A
ramping [53,54]. In this study, multi-scale wind uncertainty model
stochastic wind model is developed based on the data of wind data
is constructed to combine all these characteristics. To remove the
converted into wind power output data through Eq. (9). The reason
seasonality, a monthly-based model is constructed, and then for
for this is to direct the data usage to the relevant region and to pre-
each distinct month, the detailed model is constructed in two
vent over-fitting with irrelevant data (e.g. one below the cut-in
parts: Daily average time series is modeled as a stationary Markov
speed or over the cut-off speed). The downside is that the wind
chain for capturing inter-day variation, and hourly value is esti-
model should be re-identified whenever the conversion model
mated by combining the deterministic effects by daily hours profile
changes, but by directly predicting the wind power output, inaccu-
and the stochastic ramping up/down movements to generate intra-
racies in the decision-making process can be lowered. Here, cut-in,
day wind scenarios. Details of the overall model are described with
rated, cut-off wind speed and rated electrical power are assumed
month, day, and hour time index (m, d, h) where m 2 M = {1, . . . ,
to be 2, 15, 25 m/s, and 2 mW, respectively. These parameters
12}, d 2 Dm = {1, . . . , dm}, and h 2 H = {1, . . . , 24}.
are within the ranges of typical wind turbine specs [52], and specif-
Inter-day variation: The autocorrelation function (ACF) with the
ically with reference to ENERCON E70 wind turbine.
lag p, is useful for identifying stationarity of a time series: For a sta-
Wind variability occurs across all of the time scales from hourly
tionary time series, the ACF will drop to zero relatively quickly
to yearly and exhibits different characteristics corresponding
with the lag, while the ACF of non-stationary data decreases
scales. The seasonal periodicity of the real wind data is presented
slowly. The partial autocorrelation function (PACF) is similarly
in Fig. 3 and Table 1, which consistently show low wind output

Fig. 3. Monthly average boxplot of real wind power data (left) and generated wind power data (right).

Table 1
Monthly statistics of both real and generated wind power data.

Actual data (mW) Generated data (mW)

a
Mean Std ACF(1) Mean Std ACF(1)a
1 0.494 0.555 0.869 0.527 0.480 0.834
2 0.383 0.473 0.851 0.440 0.414 0.817
3 0.411 0.462 0.847 0.442 0.395 0.803
4 0.362 0.409 0.839 0.390 0.359 0.801
5 0.278 0.327 0.788 0.304 0.293 0.760
6 0.264 0.298 0.724 0.289 0.270 0.688
7 0.232 0.267 0.689 0.248 0.243 0.655
8 0.220 0.258 0.690 0.234 0.229 0.632
9 0.240 0.306 0.786 0.264 0.265 0.751
10 0.293 0.395 0.854 0.329 0.339 0.818
11 0.378 0.466 0.842 0.419 0.407 0.805
12 0.463 0.531 0.858 0.511 0.458 0.815
a
ACF(1): autocorrelation coefficient with an hour time-lag.
 J. Shin et al. / Applied Energy 195 (2017) 616–633 623

measured the correlation between yt and yt-k after removing the what we would expect when the daily-average time series is sta-
dependence of yt on all other time lags yt+1 through yt+k-1. These tionary within a month, and the daily transition is only affected
measured values, ACF and PACF, play an important role in data the subsequent day. The ACF and PACF plots for the other seasonal
analysis aimed at identifying the extent of the lag in an autoregres- months are shown in Fig. 5, and they have similar characteristics.
sive (AR) model that was introduced as part of the Box-Jenkins Accordingly, daily average value wday is modeled as a first-order
method [55]. A data series can be represented as an AR(p) model Markov chain (MC) as shown in Eq. (11): it evolves daily with an
if the ACF plot decay in an exponential or sinusoidal fashion and exogenous information variable. A discrete space Xe defined by
PACF plot cuts off after lag p [56]. Fig. 4 shows the daily average the minimum and maximum values and state transition probabil-
wind data series of February through 2002 to 2014 (above), and ities from state i to j are derived based on the real data (Eq. (12)):
its ACF and PACF plots with rejection region bands for the null
wday day
^ day
m;dþ1 ¼ wm;d þ wm;dþ1 ð11Þ
hypothesis (below). In the ACF plot, the early spikes exponentially
decrease with the lag and all the spikes are within the critical   n
bounds thereafter. At the same time, there is only one significant pem ði; jÞ ¼ P wday day
m;dþ1 ¼ jjwm;d ¼ i ¼
Xij for i; j 2 Xem ð12Þ
spike in the PACF and then no significant spikes outside the limits nij
j
thereafter. Therefore these patterns in the ACF and the PACF is

Fig. 4. Daily average wind data series of February over years (above), and ACF and PACF plots of the time series (below).

Fig. 5. ACF and PACF plots of daily average wind data series by months.
624 J. Shin et al. / Applied Energy 195 (2017) 616–633

where nij represents the number of transitions from state i to state j
during one period. Daily average wind data series can thus be
generated by Markov chain Monte Carlo (MCMC) simulation [57].
In order to test the validity of the MC model, general statistical
parameters of those time series are presented in Table 2 by
representative month of each season. The quantitative comparison
of the descriptive statistics and the Weibull distribution parameters
of simulations indicate that the results are very close to the
realizations.
Intra-day scenario generation: Left graphs in Figs. 6 and 7 show
the day-ahead profiles and hourly ramping rates, respectively, of
real data differed by month. Day-ahead profiles are more consis-
tent over the years during the summer, whereas the uncertainty
on the deterministic intra-day profile is much higher in the winter.
In Fig. 7, stochastic ramping rates which are the differences of
hourly average wind values between two consecutive hours are
presented in terms of the percent with respect to the maximum
wind power output.

Table 2
The descriptive statistics of daily average time series of real and generated data.

Actual data (mW) Generated data (mW)

Jan. Apr. Jul. Oct. Jan. Apr. Jul. Oct.
Mean 0.495 0.363 0.232 0.294 0.521 0.352 0.230 0.316
Standard deviation 0.431 0.276 0.121 0.297 0.442 0.263 0.124 0.313
Minimum 0.001 0.020 0.018 0.001 0.001 0.001 0.001 0.004
1st quartile 0.124 0.164 0.146 0.099 0.140 0.154 0.135 0.122
Median 0.346 0.297 0.220 0.202 0.371 0.293 0.223 0.214
3rd quartile 0.792 0.485 0.296 0.370 0.836 0.488 0.296 0.403
Maximum 1.811 1.428 0.795 1.797 1.795 1.332 0.725 1.796
Weibull parameter scale 0.505 0.401 0.263 0.302 0.535 0.383 0.257 0.326
shape 1.053 1.400 2.031 1.066 1.072 1.336 1.844 1.078

Fig. 6. Intra-day variation: daily hour-average wind profile by month.

Fig. 7. Hourly ramping rate distribution of real data (left) and generated data (right).
 J. Shin et al. / Applied Energy 195 (2017) 616–633 625

Combined to these, hourly wind data model is constructed as transition probability of the Markov chain in Eqs. (11) and (12);
follow: Αm is for generating intra-day wind scenarios, which are the esti-
 T mated parameter vector and statistics of the random error noise
wm;d;h ¼ bhm;h þ bdm;h wday h1 WIND
m;d þ bm;h wm;d;h1 þ em;h ¼ bm;h /WIND
m;d;h þ em;h
term in Eq. (13). This distinction is made for using them separately
h iT h iT in the intra-day unit commitment model and inter-day value func-
where bWIND
m:h ¼ bm;h bm;h bm;h
h d h1
; and /WIND day
m;d;h ¼ 1 wm;d wm;d;h1 tion approximation which are explained in the next section (Fig. 8).
0   1
ð13Þ Em ¼ Xem ; pem for 8m
WM ¼ @   A ð16Þ
m;h ; lm;h ;
Am;h ¼ bWIND rem;h ; emin
m;h ; em;h for 8m; h
where the first bias term denotes the deterministic daily hour pro- e max

file; the second one considers the effect of daily-average value; and
the third one is the dependency of consecutive hours. In the case
that random noise is normally distributed, least square provides
an optimal solution for the parameter estimation. However, the
error noise term is not normally distributed due to the boundedness
and skewness of the studied wind data. Robertson and Lee [58]
introduced several parameter estimation methods dealing with a
bounded or skewed error distribution. Here, bounded quadratic
programming and truncated normal distribution whose density is
defined by Eq. (15) are used to estimate the parameter:
(  ) Table 1). Inter-day variations are well captured by the first-order
X ei  le 2 T i
min je ¼ w  b / ; e
i i min
6e 6e
i max
ð14Þ
b
i
e r
8    
< c exp  1 ele 2 emin 6 e 6 emax
pðeÞ ¼ 2 r e
; ð15Þ
: hourly to enable seamless connection of the decision making
0 otherwise;
where c is a normalization constant for integrating to one and i is
indexing for sample data used for the parameter estimation. Statis-
tical parameters of truncated normal distribution are numerically
estimated using the moment equations [59]. Due to a lack of a
closed-form solution for the moment equations, the parameters
must be found using a numerical approach such as Newton’s
method. The estimated parameters of the intraday wind model for
January and July are listed as shown examples in Tables 3 and 4,
respectively.
As a result, the overall wind model for each month m is defined
by model parameters Eq. (16), which can be derived from matching
real wind data; Em is a parameter set related to the inter-day
variation of wind data including discrete state space and state
The model parameters defined by Eq. (16) are estimated using
the training data, and then hourly wind data for one year can be
generated from the fitted model. The results of sixty synthesized
datasets from the developed wind model are compared to those
of real wind data through Figs. 3, 6, 7 and Tables 1, 2, in terms of
monthly statistics, daily average time series, deterministic daily
profile, and stochastic hourly ramping. The developed model tends
to follow the overall seasonal characteristics in terms of mean,
standard deviation, and ACF with one hour time-lag (Fig. 3 and
MC model (Table 2) as well as intra-day profiles and hourly ramp-
ing changes are quite well followed by the autoregressive model
identified through constrained parameter estimation (Figs. 6 and
7). As a result, the proposed wind model is judged to be adequate
for representing the uncertainty for timescales from seasonal to
across the scales.
4. Decision-making
The overall decision-making framework is presented in Fig. 8.
Two-stage stochastic programming (2SSP) is implemented for the
one-day operation planning: unit commitment (UC) decisions
and dispatch decisions are made based on a number of intraday
wind scenarios. The long-term value of being in a state at the
end of day is contained in the objective function through the value
function of the day to day MDP to ensure that longer term implica-
tions of the decisions are taken into account. For the value function
approximation (VFA), a day to day Markov decision process (MDP)
is formulated and inter-day exogenous information on wind power

Table 3
Parameters of intraday wind model: January (Am¼1 ).

h bhm;h bdm;h bh1

m;h
lem;h rem;h emin
m;h
emax
m;h

1 0.019 0.263 0.766 0.000 0.285 0.933 1.611

2 0.019 0.214 0.852 0.000 0.254 0.917 1.399
3 0.052 0.409 0.691 0.000 0.265 1.065 1.418
4 0.014 0.273 0.720 0.002 0.270 0.794 1.485
5 0.004 0.318 0.690 0.000 0.282 0.984 1.253
6 0.026 0.333 0.650 0.000 0.286 1.136 1.138
7 0.025 0.370 0.681 0.002 0.286 0.892 1.714
8 0.012 0.270 0.785 0.000 0.260 0.824 1.119
9 0.044 0.414 0.642 0.000 0.246 0.993 1.085
10 0.008 0.405 0.625 0.000 0.293 0.944 1.387
11 0.031 0.362 0.735 0.000 0.266 1.268 1.051
12 0.013 0.406 0.679 0.000 0.265 0.912 1.235
13 0.031 0.279 0.731 0.000 0.251 1.060 1.428
14 0.017 0.240 0.767 0.002 0.223 0.630 1.036
15 0.006 0.433 0.597 0.000 0.261 1.672 1.438
16 0.003 0.441 0.583 0.000 0.271 1.189 1.263
17 0.001 0.115 0.831 0.000 0.232 0.878 1.285
18 0.027 0.302 0.715 0.000 0.260 1.045 1.691
19 0.024 0.289 0.668 0.000 0.246 1.254 0.786
20 0.019 0.173 0.713 0.000 0.281 1.238 1.505
21 0.020 0.125 0.821 0.000 0.236 0.988 1.597
22 0.019 0.129 0.824 0.000 0.261 0.853 1.284
23 0.019 0.108 0.846 0.000 0.263 0.828 1.426
24 0.036 0.210 0.787 0.000 0.295 1.872 1.014
626 J. Shin et al. / Applied Energy 195 (2017) 616–633

Table 4
Parameters of intraday wind model: July (Am¼7 ).

h bhm;h bdm;h bh1

m;h
lem;h rem;h emin
m;h
emax
m;h

1 0.047 0.217 0.521 0.017 0.185 0.486 0.745

2 0.001 0.388 0.602 0.020 0.176 0.434 0.726
3 0.006 0.364 0.540 0.000 0.174 0.523 1.587
4 0.018 0.426 0.563 0.000 0.168 0.583 1.297
5 0.008 0.289 0.619 0.005 0.170 0.468 0.778
6 0.008 0.256 0.584 0.009 0.152 0.401 0.718
7 0.012 0.249 0.568 0.000 0.133 0.446 0.533
8 0.033 0.412 0.615 0.027 0.154 0.376 0.983
9 0.016 0.286 0.684 0.000 0.138 0.455 0.889
10 0.001 0.227 0.577 0.000 0.114 0.380 0.441
11 0.032 0.180 0.702 0.000 0.117 0.422 0.510
12 0.034 0.247 0.644 0.000 0.120 0.392 0.510
13 0.032 0.215 0.743 0.000 0.122 0.459 0.774
14 0.043 0.466 0.554 0.018 0.177 0.443 0.974
15 0.007 0.756 0.478 0.000 0.189 0.657 0.865
16 0.027 0.683 0.612 0.020 0.210 0.563 0.772
17 0.025 0.786 0.450 0.000 0.238 0.754 1.295
18 0.032 1.059 0.449 0.013 0.220 0.597 0.917
19 0.008 0.877 0.440 0.018 0.258 0.682 0.978
20 0.040 0.819 0.520 0.007 0.241 0.667 0.854
21 0.026 0.542 0.606 0.009 0.225 0.627 0.864
22 0.047 0.659 0.472 0.062 0.238 0.560 1.146
23 0.048 0.171 0.596 0.000 0.216 0.674 1.029
24 0.054 0.072 0.584 0.015 0.202 0.546 0.890

Fig. 8. The overall integrated decision-making framework and multi-scale wind model.

is represented. Furthermore, the yearly operating cost is estimated
from the surrogate model of the value function given the capacity
of wind farm and battery for all distinct months, and the optimal
sizing problem is then formulated based on the surrogate model
as well as investment cost.
4.1. 2SSP: One-day operation
For one day operation of the energy system, a two-stage model
is used whose decision variables are defined by Eqs. (17) and (18).
The monthly and daily time indices are omitted for convenience.
The first stage decision x is unit commitment ug of slow-start gen-
erators g 2 GS; start-up cost Ug, and shut-down cost Dg for all daily
hours. In the second recourse stage, all dispatch decisions pk
including production from generators g 2 G, discharging/charging
amount of the battery, and buying/selling of electricity from/to
the main grid are made, i.e., k 2 K = {G, disch, ch, buy, sell}. Battery
level b, and lost-demand s for all daily hours are decided as well.
A number of intraday wind scenarios x are considered; the first
stage decision is fixed for whatever wind scenario is realized (as
they are made in the absence of such information) while the sec-
ond stage decision is differentiated by the scenarios. A set of intra-
day wind scenario Xa is derived by daily average wind value wday,
and the parameters given from the wind uncertainty model, Α in
Eq. (16).
x1 ¼ uhg ; U hg ; Dhg for 8g 2 GS ; h 2 H ð17Þ
h i
x
xx x x
2 ¼ pk;h ; bh ; sh for 8 k 2 K; h 2 H; x 2 Xa ðwday ; AÞ ð18Þ
The objective function to find an optimal solution is specified in
Eqs. (19)–(21): The first stage cost function includes start-up, shut-
down, and set-up cost which is imposed by turning on/off the
slow-start generators; and the second stage cost is the sum of all
dispatch costs and penalty costs of lost-demand. Here, the second
stage function is an expected value over possible intraday wind
scenarios x 2 Xa, and the probability of each scenario realized is
 J. Shin et al. / Applied Energy 195 (2017) 616–633 627

approximated by pa = 1/|Xa|. For the overall operational model, the 4.2. 2SSP+VFA: multi-day operation
initial state of UC, battery level, and the daily average wind power
value, which is not yet realized, are required as input values. The one-day operation model should be temporally extended
X beyond a day due to the physical dynamics of energy storage and
f ðx1 ; u0 Þ ¼ U hg þ Dhg þ C sg uhg ð19Þ the operational constraints caused by commitment decisions. With
h;g2Gs the decision horizon of one day, an optimal solution always gives a
( ) zero level of battery and off-line status of generator at the end of
  X X day because there is no benefit from maintaining a nonzero level
x x x
Q x x2 ; b 0 ¼ C disp;k pk;h þ C lost sh ð20Þ of battery and keeping the generator on. In addition, daily average
h k
wind value of the current day affects that of the next day and this
X inter-day variation of wind uncertainty should be explicitly consid-
min
x
f ðx1 ; u0 Þ þ pa Q x ðxx2 ; b0 Þ ð21Þ ered in the operational decision. To resolve this problem, long-
x1 ;x2
x2Xa ðwday Þ term value of being in a system state (e.g. wind, battery and com-
Operational limitations and balance equations should be con- mitment) at the end of day is linearly approximated and the coef-
sidered as inequality and equality constraints of the optimization ficient vector of the linear model is offline learned. Thus generated
model, specified by Eqs. (22)–(31). The first two equations are for value function is incorporated into the 2SSP model. Fig. 9 shows
deciding start-up and shut-down cost, respectively. The fuel- the overall learning-based temporal integration framework of daily
based generator is generally required minimum time (MT) that operation.
should hold online/offline once it has been turned on/off. The UC In the day to day MDP formulation to approximate a value func-
decision is thus made every minimum up/down time period tion at the end of the day h = H, temporal integrity is maintained by
as shown in Eq. (24), with a set of decision epochs defined by (m, d, H) = (m, d + 1, 0), i.e., state at the end of the day is the initial
HUC = {Hj|Hj = (j – 1)MT + 1, j 2 N, Hj  H}. state of the next day. State variable sm,d which is the available sys-
tem information at the corresponding time (m, d, 0) contains the
U hg  C Ug uhg  uh1
g
P 0 for 8h; g 2 GS ð22Þ initial state of UC of slow-start generators and the battery level.
In addition to this, daily average wind value of the previous day
is considered as another state variable summarizing the daily
Dhg  C Dg uh1
g
 uhg P 0 for 8h; g 2 GS ð23Þ
evolving exogenous information on wind which is modeled as a
first-order Markov chain (MC):
Hjþ1 1 
X h iT
uhg  uHg j ¼ 0 for 8g 2 GS ; Hj 2 HUC ð24Þ g
sm;d ¼ um;d;0 bm;d;0 wday : ð32Þ
m;d1
h¼Hj

The overall one-day cost function is evaluated by the initial

Eqs. (25)–(27) shows the upper and lower bounds of the second state, decision for the daily operation (both the first and second
stage decisions. Ramp up and down limitation of generators, stage, x = (x1, x2)), and the exogenous information on wind realized
inequality constraints for lost-demand variables (where d is at the corresponding day, as Eq. (33). Here, monthly and daily nota-
demand that should be satisfied), and battery balance equation tions are added in the first and second stage decision variables
are specified through Eqs. (28)–(30). Since these constraints are which are defined by Eqs. (17) and (18), the first and second stage
related to the second stage, they should be repeated for all uncer- costs are stated in Eqs. (19) and (20), respectively. Here, the prob-
tainty scenarios x. ability of each scenario realization is equally distributed.
  X
uhg Pmin
g
6 px g g
g;h 6 uh P max for 8h; x; g 2 GS ð25Þ C sm;d ; xm;d ; wday pam;d Q x ðxx2;m;d ; bm;d;0 Þ
m;d ¼ f ðx1;m;d ; um;d;0 Þ þ
x2Xam;d
 
Pkmin 6 px k
k;h 6 P max for 8h; x; k 2 K n GS ð26Þ pam;d ¼ 1=jXam;d j; and Xam;d ¼ Xa wday
where m;d ; Am

x ð33Þ
Bmin 6 bh 6 Bmax for 8h; x ð27Þ
The state transition probability from sm,d to s = (u , b , w0 ) is 0 0 0

x x
g
Rdown 6 pg;h  pg;h1 6 g
Rup for 8h; x; g ð28Þ defined by Eq. (34). The transition probability of daily average
wind state is given by the MC model, given by Eq. (12). The states
X x of UC and battery level are determined by daily operation deci-
px x x x x
g;h þ pdisch;h þ pbuy;h þ sh P dh  pwind;h for 8 h; x; g ð29Þ sions, the specific overall state takes the value 1 if (u0 , b0 ) is the
g2G
optimal solution of 2SSP, and 0 otherwise.
 
h 
X  pðs0 jsm;d ; xm;d Þ ¼ p w0 jwday 0 0
x hh0 m;d pðu ; b jsm;d ; xm;d Þ ð34Þ
bh  x
gch pch;h0  gdisch pdisch;h0 ð1  rb Þ
x

h0 ¼0 ( 0 
b h
1 if u0 ¼ um;d;H and b ¼ bm;d;H
¼ b0 ð1  r Þ
0
for 8 h; x ð30Þ 0
pðu ; b jsm;d ; xm;d Þ ¼ ð35Þ
0 otherwise

uhg 2 f0; 1g; x1 ; xx

2 P 0 for 8h; x
The size of the optimization problem is defined as follow: the
number of the 1st stage decision variables and constraints is 3|
H||GS| and 2|H||GS|, respectively, and the number of the 2nd stage
decision variables and constraints with one uncertainty scenario is
|H|(|K|+2) and |H|(|K|+4|GS|+3), respectively. This 2SSP problem is
solved by being reformulated as the full deterministic equivalent
linear form and the resulting optimization problem is an MILP.
ð31Þ The value function is linearly approximated with a basis func-
tion of these state variables as well as a constant term, as Eq. (5).
Since the daily transition of wind follows an MC model for each
distinct month, the value function can be expressed in a stationary
form for each month with an infinite horizon setting. For approxi-
mating the value function for month m, random sample path of
 
daily-averaged wind value wday;1
m ; . . . ; wday;N
m is generated through
MCMC simulation using the wind model parameters Em in Eq. (16),
628 J. Shin et al. / Applied Energy 195 (2017) 616–633

Fig. 9. Learning-based temporal integration of daily operation.

and for each iteration n, intraday wind scenarios x are generated capacity of battery and wind generator, c = (cw, cb), is established
using wday;n
m and Αm in Eq. (16). At the nth iteration, a sampled based on a pair of sampled data sets, S = (c1, . . . , cN) and Vs =
observation is then computed by the 2SSP model with the value (V1, . . . , VN). A value of the value function at the average of the vis-
function at the previous iteration n  1, giving us Eq. (36). The ited state during the simulation for VFA is used as an output
expectation should be computed over both inter-day and intraday response corresponding to an input point. Thus the output V(c)
variations. Then the coefficient vector of the linear model is recur- for any untried design option c can be predicted by this surrogate
sively updated based on the error between the previous prediction model:
and the estimated observation [46].
V ^ SR ðcw ; cb Þ þ e
 m ðcw ; cb Þ ¼ V ð39Þ
" # m
X  nþ1 
v
^ nm ðsnm Þ ¼ min
n
Ex
xm
Cðsnm ; xnm ; wday;n
m Þ þ c hn1
m;f /f sm ðxÞ ð36Þ where the error vector e is composed of independent random vari-
f 2F
ables with zero mean.
    The value function is approximated by uniformly exploring a
hnm ðsÞ U ^en ; sn1  hn1  v^ n ðsn Þ
where ^en ¼ V ð37Þ
m m m m feasible design space: Fig. 10 shows 36 sampled data points with
respect to the capacity of battery (left) and wind generator (right)
By offline approximating the value function until n = N with a num-
for January. The results demonstrate the value function containing
ber of random samples, final decisions can be made from the 2SSP
all operating costs, is monotonically changed by the size of battery
model with the resulting value function coefficient as shown in
and wind generator, and its non-linearity is not severe within the
Eq. (38). Here, pe and pa is the probability of inter-day variation
design range. Therefore, a polynomial regression model is used in
and intraday uncertainty realization, respectively.
which the sampled data is fitted by least-squares:
arg minf ðx1;m;d ; um;d;0 Þ  
xm;d ^ SR ðcw ; cb Þ ¼ bSR þ bSR cb þ bSR cw
V
( ) m m;00 m;01 m;10
X     X N  
þ pem wday
m;d1 ; w pm;d Q x x2;m;d ; bm;d;0 þ c
0 a x
hm;f /f ðsx
m;d Þ þ bSR 2 SR 2 SR
m;02 cb þ bm;20 cw þ bm;11 c w cb þ . . .
w0 2Xe
m f 2F XX SR
x2Xa
m;d ¼ bm;ij ciw cbj ð40Þ
h i i j
x
where sx
g
m;d ¼ um;d;H bm;d;H w0
where each number of subscript in the surrogate model parameter
ð38Þ
bSR indicates the degree of cw and cb respectively. To select an appro-
priate model structure considering a trade-off between the model
4.3. Surrogate model based optimal sizing accuracy and complexity, model quality metrics such as Akaike
information criteria (AIC), normalized AIC, and Bayesian informa-
Since the value function evaluates all future operating costs by tion criteria (BIC) are measured; model accuracy is measured by
definition, response surface method (RSM) for the value function is residual sum of square (RSS) which are the sum of the differences
used for the size optimization of the studied wind/battery inte- between the observed values and the regressed values, and model
grated hybrid energy system. Surrogate function which is a map complexity is described by the number of regression parameters
indicating how much operating costs are to be incurred for a given and the number of values in the estimation data set. The preferred
 J. Shin et al. / Applied Energy 195 (2017) 616–633 629

Fig. 10. Value function by the capacity of battery (left) and wind generator (right) for January.

model is one with the minimum AIC/BIC values, and thus the next
term to be added is determined by reducing the AIC/BIC values
the most. In addition to these criteria, convexity of the regression
function is confirmed assuring the existence of global optimum of
the surrogate-based optimization, which is formulated for optimal
sizing problem considering both capital costs and estimated operat-
ing costs, as follow
X XX
min C icapa xicapa þ V^ SR
m ðcw ; c b Þ
xcapa
i2fw;bg m2M d2Dm
s:t: ci ¼ ci;0 þ xicapa i 2 fw; bg
where xcapa is a decision vector of acquisition of battery and wind
generator capacity, and Ccapa is unit investment cost.
5. Illustrative example and discussion
In order to illustrate the application of the proposed decision
strategy, the energy scenario is defined to reflect the energy supply
situation in the future [60]; the overall capacity of the wind farm
will increase rapidly and surpass those of fuel-based generators,
and energy storage capability corresponding to a certain portion
(10–20%) of the installed wind capacity is required to mitigate
the wind generators’ output variability. The capacity of each
micro-grid component and load is defined by reflecting these cir-
cumstances and all the system parameters used in the case study
are listed in Table 5. Through the case study, scaled electricity
and cost units are used for simplicity, but they reflect the real
ratios in various costs. Thus all data in Table 5, Figs. 11–13 are
scaled values and general units are not required. The operating
conditions and unit costs for three different slow-start generators
(SGs), a fast-start generator (FG), battery, and HVDC are specified:
Slow-start generators require start-up, shut-down, and setup costs
but unit production cost of SG is much lower than those of the
others. Also, the battery is assumed to have 0.005% self-
discharging rate and 0.8 charging efficiency. Unit penalty cost for
the lost-demand is 5 that is much higher value than the dispatch
costs of main sources such as SGs and battery, and daily load sce-
narios are generated by adding stochastic random noise to the
deterministic daily profile.
Twelve years of wind data through 2002–2014 (except 2011)
are used for the model identification as shown in Section 3.2, and
the most recent 2015 data are then used in the case study for com-
ð41Þ paring the decision policies. Two different approaches are exam-
ined for the unit commitment operation of the energy system:
The first one, Method 1, uses the 2SSP model without the value
function (corresponding to daily independent decision making)
and for which inter-day wind variation correlation is not
accounted; second, Method 2, a day to day MDP model is con-
structed considering the inter-day wind variation, and the 2SSP
model with a linearly approximated value function is used for daily
operation. The formulated 2SSP problem is solved by being refor-
mulated as the full deterministic equivalent MILP form. With ten
uncertainty scenarios, the number of decision variables is 2616
among which 72 are binary; and the number of both equality
and inequality constraints is 5736. This optimization problem is
solved by MATLAB solver intlinprog.
Fig. 11 illustrates an example case showcasing what role the
value function plays in decision-making phase. In the first day,
the operating policy with the value function provides the same
unit commitment decisions with the ones from the daily indepen-
dent policy, but the wind production increases towards midnight
and a higher level of battery is maintained. That’s because a higher
battery level at the end of day gives a lower value of the value func-
tion, whereas there is no reason to store extra energy in the finite
horizon problem. In the next day, even the second generator is
turned off during the early day by Method 2, and the charged
battery can satisfy the off-peak load. As a result, 8.65% of operating

Table 5
Parameters for case study.

SG1 SG2 SG3 FG Battery HVDC System Others

Out In Buy Sell

P kmax 8 6 4 3 2 2 5 3 Pw
r
20 Bmin 0

P kmin 4 3 2 0 0 0 0 0 wc 2 Bmax 5
g
Rup 4 3 2 3 – – – – wr 15 C lost 5
g
Rdown 4 3 2 3 – – – – wf 25 MT 4
C Ug 3 2.5 2.5 0 – – – – gch 0.8 Demand
C Dg 2.5 2 2 0 – – – – rb 0.005 Peak 25
C Sg 1 0.8 0.5 0 – – – – OFF-peak 13
C disp;k 1.5 2 2.5 3 0 0 4.5 2
630 J. Shin et al. / Applied Energy 195 (2017) 616–633

Fig. 13. Optimal balance between the capacity of wind generators and battery
devices where a is the logarithm value of the ratio of the unit investment cost to the
average one-day operating cost.

Fig. 11. Illustrative example of maintaining high level of battery and commitment.
costs are reduced for this day. Meanwhile, daily average wind
power of the second day is 12.26 while it drops to 5.22 the next
day. Since day-ahead unit commitment decisions are made before
realizing the wind uncertainty of the corresponding day, too opti-
mistic a unit commitment decision is made without consideration
of inter-day wind variation, that is, the second generator is turned
off for the entire third day. Consequently, the overall load will be
satisfied using more expensive energy sources such as the fast-
start generator or purchase from the national grid. However, in
the proposed method, the value function accounting daily transi-
tion probability of wind enables one to prepare for the sudden
decrease of wind production by maintaining a high level of battery
and turning on the second generator during the morning ramp and
the evening peak of the load. As a result, 29.45% of improvement is
acquired in terms of operating costs for the third day.
The overall yearly simulation results are specified in Table 6. For
124 days among the 365 days (33.97%), daily decisions made by
the 2SSP and VFA combined method differ from those of the orig-
inal 2SSP (that is defined by effected day). For these effected days,
the overall improvements of the suggested method over the con-

Fig. 12. Average results of daily dispatch decisions in winter (upper) and summer (lower).
 J. Shin et al. / Applied Energy 195 (2017) 616–633
Table 6
Overall improvement.
Month Improvement of Method 2b Month Improvement of Method 2b
over Method 1a (%) over Method 1a (%)
1 7.28 7 2.98
2 1.58 8 5.62
3 7.03 9 3.04
4 0.87 10 10.08
5 3.60 11 17.02
6 1.23 12 9.09
a
Method 1: daily independent 2SSP model.
b types of generation systems as components, including CHPs [64],
Method 2: 2SSP model combined with VFA.
ventional method in terms of monthly mean cost are given in
Table 6. Especially in the winter seasons which have high wind
production and high variation, decisions considering the value of
commitment and battery state at the end-of-day as well as the
inter-day correlation of wind power can provide much benefits
over daily independent decisions. The improved decisions result
because the value function captures trade-offs between the current
day costs and the future costs in the decision-making model, and
then one can ensure better optimality for the longer term.
Fig. 12 shows the average results of daily dispatch for the
effected days in the winter (upper) and the summer (lower): pro-
duction from slow-start generators (left), and battery levels (right).
It is common sense that more energy is produced from SGs during
the morning ramp and evening peak time of demand while the bat-
tery is charged in the off-peak time and discharged at peak time.
When the decision is made by the 2SSP and VFA combined model,
more energy is produced from SGs given the high uncertainty of
wind transition, and it is possible to operate a more flexible battery
usage according to the daily hours. In the winter, since wind power
production is relatively higher all through the morning (Fig. 6),
charged energy is used to back up wind uncertainty instead
employing SGs. It leads a lower first peak in battery level, and at
the same time, a higher battery level at the end of the day is
attained given an incentive of lower value of value function. On
contrary to the winter, wind production is low from midnight
through the morning in the summer (Fig. 6). It requires energy pro-
curement from SGs for the corresponding hours, and it is more
beneficial to produce a large amount of energy from SGs once it
turned on due to the start-up and setup costs. Thus the extra
energy is charged to battery in the morning, and when wind power
production decreases in the evening the stored energy is dis-
charged to minimize the usage of the expensive energy sources.
Here there is little benefit in storing the energy at the last hour
of the day, unlike the winter, due to the lower wind production
and lower variation.
In the decision-making phase at a much faster time scale (con-
trol or real-time optimization) for the real execution of the dis-
patch plan, recourse actions should be calculated in response to
the newly realized uncertainty in real-time. For such a correction,
rescheduling is implemented using either a moving or a shrinking
horizon approach [61]. Shrinking horizon control (SHC) with a
look-ahead window from current time to the end of day is benefi-
cial to deal with the constraints related to pre-determined UC and
daily periodicity of wind uncertainty. For efficient use of SHC, it is
necessary to maintain a boundary condition (e.g. battery level) at a
fixed moment [62,63]. Instead of heuristically determining the
boundary conditions at the end of the horizon, the proposed
method combining mathematical programming and value function
approximation can systemically capture a trade-off between the
cost within a control/decision horizon and the one beyond the
horizon. That is, the SHC strategy with the value evaluation at
the last time can be expected to assure long-term optimality as
well as to react to unexpected short-term changes.
631
In this case study, a benchmark energy system with a simple
production model for micro-generator, which considers the start-
up, shut-down and set-up cost (related to commitment variable)
and the linearly increased variable cost, is assumed to demonstrate
the performance of the proposed approach. On the other hand, the
introduced method combining a one-day operation model with a
day-to-day MDP formulation has no restriction on the size and
type of one-day model used. Though we did not specify specific
types of generators, other than fast- and slow-starting (FG and
SG), our approach should be capable of accommodating various
when the appropriate model information is available. In addition,
it should be possible to adopt a more complex production cost
model for conventional generators (e.g. thermal, diesel) expressed
as a quadratic function or a piecewise linear model [65] within the
overall method. For handling energy networks employing demand-
side management [66], inclusion of additional state variables such
as electricity price may be needed.
By uniformly probing a feasible design space with respect to the
capacity of battery and wind generator, 144 sampled data points
on the value function are obtained, and thus the value function
containing all operating costs can be regressed as a function of siz-
ing variables of both battery and wind farm. Since the capacity of
battery and wind affects the operation of each other, interactive
terms are required as well. Various surrogate models starting from
a linear one to quartic polynomials by adding new terms are tested,
of which parameters are defined in Table 7. The results of quality
information criterion of all the specified surrogate models are
shown in Table 7: the degree of increases of AIC, nAIC, and BIC val-
ues of all the examined models over the minimum value of the cor-
responding information criterion. After a certain number of
regression parameters used, the model quality improvement is
not significant. Rather, the model containing thirteen numbers of
parameters (numbered 10) has the smallest BIC value; over-
estimated models are penalized in terms of model complexity.
Thus the following regression model structure is chosen:
^ SR ðcw ; cb Þ ¼ bSR þ bSR cb þ bSR cw þ bSR c2 þ bSR c2
V m m;00 m;01 m;10 m;02 b m;20 w
þ bSR SR SR SR
m;11 c w c b þ bm;03 c b þ bm;30 c w þ bm;21 cw c b
3 3 2
þ bSR 2 SR 4 SR 3 SR
m;12 c w c b þ bm;04 c b þ bm;13 c w c b þ bm;22 c w c b
2 2
ð42Þ
where cw and cb are the capacity of wind generator and battery,
respectively.
Table 8 shows the estimated regression parameters of the
selected surrogate model for the operational value function by rep-
resentative month of each season and yearly average. In overall,
the absolute values of the model parameters are greater in the win-
ter month than in the summer month, i.e., wind and battery size
give much larger impacts on the overall operation in winter. The
marginal impact of wind generator capacity is much greater than
that of battery size, but a negative value of correlation term b11
means that the two elements (capacity of battery and wind gener-
ator) are complementary to each other. Since Hessian matrix of the
objective function in Eq. (42) is positive definite over the entire
design space, the optimal sizing problem is convex optimization
assuring the existence of global optimum. The resulting optimal
balance of battery and wind capacity is shown in Fig. 13. Here,
an optimal sizing curve is obtained by varying the unit investment
cost of battery, where a is defined by logarithm value of the ratio of
the unit investment cost to the average one-day operating cost.
As an extension of the current study, the sizing problem can be
formulated as a multi-stage decision problem, in which case the
sizing decision is made incrementally, say, once a year over a time
span of several decades. This will necessitate building and using a
632 J. Shin et al. / Applied Energy 195 (2017) 616–633

Table 7
Parameter vector and results of tested model structures of the surrogate value function.

Model number Parameter vector # of parameters RSS ( 105) Value increase (%) over the minimum Convexity
value of
AIC nAIC BIC
1 bSR;1 ¼ bSR
00 bSR
01 bSR
10
3 135.59 88.90 141.30 82.82 Satisfied
2 bSR;2 ¼ bSR;1 bSR bSR 5 10.35 55.69 88.51 51.39 Satisfied
02 20
3 bSR;3 ¼ bSR;2 bSR 6 4.85 45.99 73.10 42.33 Satisfied
11
4 bSR;4 ¼ bSR;3 bSR 7 2.08 35.10 55.79 32.12 Unsatisfied
03
5 bSR;5 ¼ bSR;4 bSR 8 1.13 27.32 43.42 24.90 Unsatisfied
12
6 bSR;6 ¼ bSR;5 bSR 9 0.98 25.74 40.92 23.67 Satisfied
04
7 bSR;7 ¼ bSR;6 bSR
22
10 0.90 24.82 39.45 23.06 Satisfied
8 bSR;8 ¼ bSR;7 bSR 11 0.33 12.00 19.07 10.99 Satisfied
30
9 bSR;9 ¼ bSR;8 bSR 12 0.30 10.95 17.41 10.26 Satisfied
21
10 bSR;10 ¼ bSR;9 bSR 13 0.13 0.01 0.02 0 Satisfied
13
11 bSR;11 ¼ bSR;10 bSR 14 0.13 0 0 0.27 Satisfied
31
12 bSR;12 ¼ bSR;11 bSR 15 0.13 0.20 0.32 0.74 Unsatisfied
40

Table 8
The estimated parameters of the surrogate model for value function.

Jan. Apr. Jul. Oct. Yearly average

bSR
m;01
89.930 137.761 234.091 139.537 150.690
bSR
m;10
1811.315 1197.037 474.408 624.825 1023.700
bSR
m;02
18.328 18.863 28.256 10.334 18.972
bSR
m;20
43.277 18.363 6.346 2.687 13.030
bSR
m;11
21.042 8.017 0.314 6.459 8.898
bSR
m;03
1.333 1.375 2.012 0.700 1.357
bSR
m;30
0.419 0.130 0.177 0.172 0.048
bSR
m;12
1.087 0.788 0.566 0.824 0.815
bSR
m;21
0.227 0.124 0.217 0.095 0.054
bSR
m;04
0.036 0.046 0.060 0.027 0.042
bSR
m;13
0.022 0.036 0.029 0.032 0.030
bSR
m;22
0.007 0.009 0.008 0.004 0.004

value function with a yearly time increment, in a similar fashion as Furthermore, since the value function evaluates all future operat-
done for the daily unit commitment decision. ing costs by definition, a polynomial surrogate model for the value
function was constructed for the size optimization of the studied
6. Conclusion wind/battery integrated hybrid energy system.

For stable and efficient management of a microgrid system Acknowledgement

employing a large wind farm and energy storage devices, multi-
scale decision hierarchy including daily operational planning and
yearly sizing problem is studied. To integrate decisions across
timescales, a general wind model consistent multi-time scales
from yearly to hourly was developed and validated through real
wind data. Two-stage stochastic programming (2SSP) was imple-
mented for day-ahead unit commitment and dispatch decisions
considering a number of intraday wind uncertainty scenarios. At
the same time, a day to day Markov decision process (MDP) was
formulated and inter-day exogenous information on wind was cap-
tured for approximating the value of the end state of daily unit
commitment and energy storage with respect to the future. By
adding this value function into the 2SSP model, the finite-horizon
operational planning represents information beyond the single
day. The performance gain from employing the daily integrated
model, considering the inter-day transition of wind uncertainty
as well as intraday variations, was compared to the daily indepen-
dent 2SSP model through a case study. The results showed that the
proposed method can reduce the overall operational costs by
capturing the trade-off between the current-day costs and future
costs through evaluation of the state at the end of decision horizon.
This work was supported by the National Research Foundation
of Korea (NRF) grant funded by the Korea government (MEST) (No.
NRF-2015R1A2A2A01007102).
References
[1] Fang X, Misra S, Xue G, Yang D. Smart grid—the new and improved power grid:
a survey. Commun Surveys Tutorials, IEEE 2012;14(4):944–80.
[2] Manfren M, Caputo P, Costa G. Paradigm shift in urban energy systems through
distributed generation: methods and models. Appl Energy 2011;88
(4):1032–48.
[3] Ustun TS, Ozansoy C, Zayegh A. Recent developments in microgrids and
example cases around the world—a review. Renew Sustain Energy Rev 2011;15
(8):4030–41.
[4] Kim Y-H, Kim S-H. Increasing effect analysis of the wind power limit using
energy storage system in Jeju-Korea. J Kor Sol Energy Soc 2014;34(1):81–90.
[5] Liang H, Zhuang W. Stochastic modeling and optimization in a microgrid: a
survey. Energies 2014;7(4):2027–50.
[6] Powell WB, Meisel S. Tutorial on stochastic optimization in energy—Part I:
modeling and policies. IEEE Trans Power Syst 2016;31(2):1459–67.
[7] Zheng QP, Wang J, Liu AL. Stochastic optimization for unit commitment—a
review. IEEE Trans Power Syst 2015;30(4):1913–24.
[8] Yang H, Lu L, Zhou W. A novel optimization sizing model for hybrid solar-wind
power generation system. Sol energy 2007;81(1):76–84.
 J. Shin et al. / Applied Energy 195 (2017) 616–633
[9] Chen Y-H, Lu S-Y, Chang Y-R, Lee T-T, Hu M-C. Economic analysis and optimal
energy management models for microgrid systems: a case study in Taiwan.
Appl Energy 2013;103(145–154).
[10] Ren H, Gao W. A MILP model for integrated plan and evaluation of distributed
energy systems. Appl Energy 2010;87(3):1001–14.
[11] Hawkes A, Leach M. Modelling high level system design and unit commitment
for a microgrid. Appl Energy 2009;86(7):1253–65.
[12] Wang J, Botterud A, Miranda V, Monteiro C, Sheble G. Impact of wind power
forecasting on unit commitment and dispatch. In: Proc. 8th Int. workshop
large-scale integration of wind power into power systems; 2009. p. 1–8.
[13] Wang J, Botterud A, Bessa R, Keko H, Carvalho L, Issicaba D, et al. Wind power
forecasting uncertainty and unit commitment. Appl Energy 2011;88
(11):4014–23.
[14] Parisio A, Rikos E, Tzamalis G, Glielmo L. Use of model predictive control for
experimental microgrid optimization. Appl Energy 2014;115(37–46).
[15] Ruiz P, Philbrick CR, Zak E, Cheung KW, Sauer PW. Uncertainty management in
the unit commitment problem. IEEE Trans Power Syst 2009;24(2):642–51.
[16] Zheng QP, Wang J, Pardalos PM, Guan Y. A decomposition approach to the two-
stage stochastic unit commitment problem. Ann Oper Res 2013;210
(1):387–410.
[17] Niknam T, Azizipanah-Abarghooee R, Narimani MR. An efficient scenario-
based stochastic programming framework for multi-objective optimal micro-
grid operation. Appl Energy 2012;99(455–470).
[18] Quan H, Srinivasan D, Khambadkone AM, Khosravi A. A computational
framework for uncertainty integration in stochastic unit commitment with
intermittent renewable energy sources. Appl Energy 2015;152(71–82).
[19] Wang J, Shahidehpour M, Li Z. Security-constrained unit commitment
with volatile wind power generation. IEEE Trans Power Syst 2008;23(3):
1319–27.
[20] Barth R, Brand H, Meibom P, Weber C. A stochastic unit-commitment model
for the evaluation of the impacts of integration of large amounts of
intermittent wind power. In: Probabilistic methods applied to power
systems, 2006. PMAPS 2006. International Conference on. IEEE; 2006. p. 1–8.
[21] Tuohy A, Meibom P, Denny E, O’Malley M. Unit commitment for systems with
significant wind penetration. IEEE Trans Power Syst 2009;24(2):592–601.
[22] Bertsimas D, Litvinov E, Sun XA, Zhao J, Zheng T. Adaptive robust optimization
for the security constrained unit commitment problem. IEEE Trans Power Syst
2013;28(1):52–63.
[23] Kuznetsova E, Li Y-F, Ruiz C, Zio E. An integrated framework of agent-based
modelling and robust optimization for microgrid energy management. Appl
Energy 2014;129(70–88).
[24] Kuznetsova E, Ruiz C, Li Y-F, Zio E. Analysis of robust optimization for
decentralized microgrid energy management under uncertainty. Int J Electr
Power Energy Syst 2015;64(815–832).
[25] Fisher M, Ramdas K, Zheng Y-S. Ending inventory valuation in multiperiod
production scheduling. Manage Sci 2001;47(5):679–92.
[26] Sousa T, Morais H, Soares J, Vale Z. Day-ahead resource scheduling in smart
grids considering Vehicle-to-Grid and network constraints. Appl Energy
2012;96(183–193).
[27] Wallace SW, Fleten S-E. Stochastic programming models in energy. In:
Handbooks in operations research and management science, vol. 10, No.
637-677; 2003.
[28] Sutton RS, Barto AG, Williams RJ. Reinforcement learning is direct adaptive
optimal control. Control Systems, IEEE 1992;12(2):19–22.
[29] Anderson RN, Boulanger A, Powell WB, Scott W. Adaptive stochastic control for
the smart grid. Proc IEEE 2011;99(6):1098–115.
[30] Xie L, Ilic MD. Model predictive economic/environmental dispatch of power
systems with intermittent resources. In: Power & energy society general
meeting. PES’09. IEEE. IEEE; 2009. p. 1–6.
[31] Powell WB, George A, Simao H, Scott W, Lamont A, Stewart J. SMART: a
stochastic multiscale model for the analysis of energy resources, technology,
and policy. Informs J Comput 2012;24(4):665–82.
[32] Asamov T, Powell WB. Regularized decomposition of high-dimensional
multistage stochastic programs with markov uncertainty, vol. 2015.
Available from: arXiv preprint 1505.02227.
[33] Asamov T, Salas DF, Powell WB. SDDP vs. ADP: the effect of dimensionality in
multistage stochastic optimization for grid level energy storage, vol. 2016.
Available from: arXiv preprint 1605.01521.
[34] Luna-Rubio R, Trejo-Perea M, Vargas-Vázquez D, Ríos-Moreno G. Optimal
sizing of renewable hybrids energy systems: a review of methodologies. Sol
Energy 2012;86(4):1077–88.
[35] Ekren O, Ekren BY. Size optimization of a PV/wind hybrid energy conversion
system with battery storage using response surface methodology. Appl Energy
2008;85(11):1086–101.
633
[36] Pesch T, Schröders S, Allelein H, Hake J. A new Markov-chain-related statistical
approach for modelling synthetic wind power time series. New J Phys 2015;17
(5):055001.
[37] Potter CW, Archambault A, Westrick K. Building a smarter smart grid through
better renewable energy information. In: Power systems conference and
exposition, 2009. PSCE’09. IEEE/PES. IEEE; 2009. p. 1–5.
[38] Wang X, Guo P, Huang X. A review of wind power forecasting models. Energy
Proc 2011;12(770–778).
[39] Soman SS, Zareipour H, Malik O, Mandal P. A review of wind power and wind
speed forecasting methods with different time horizons. In: North American
Power Symposium (NAPS), 2010. IEEE; 2010. p. 1–8.
[40] Foley AM, Leahy PG, Marvuglia A, McKeogh EJ. Current methods and advances
in forecasting of wind power generation. Renew Energy 2012;37(1):1–8.
[41] Cheng L, Subrahmanian E, Westerberg A. Design and planning under
uncertainty: issues on problem formulation and solution. Comput Chem Eng
2003;27(6):781–801.
[42] Birge JR, Louveaux F. Introduction to stochastic programming. Springer Science
& Business Media; 2011.
[43] Lee JH. Energy supply planning and supply chain optimization under
uncertainty. J Process Control 2014;24(2):323–31.
[44] Puterman ML. Markov decision processes: discrete stochastic dynamic
programming. John Wiley & Sons; 2014.
[45] Lee JM, Lee JH. Approximate dynamic programming strategies and their
applicability for process control: a review and future directions. Int J Control
Autom Syst 2004;2(263–278).
[46] Powell WB. Approximate dynamic programming: solving the curses of
dimensionality. John Wiley & Sons; 2007.
[47] Powell WB, Meisel S. Tutorial on stochastic optimization in energy—Part II: an
energy storage illustration. IEEE Trans Power Syst 2016;31(2):1468–75.
[48] Han Z-H, Zhang K-S. Surrogate-based optimization. INTECH Open Access
Publisher Rijeka; 2012.
[49] Queipo NV, Haftka RT, Shyy W, Goel T, Vaidyanathan R, Tucker PK. Surrogate-
based analysis and optimization. Prog Aerospace Sci 2005;41(1):1–28.
[50] Giunta A, Watson L. A comparison of approximation modeling techniques-
Polynomial versus interpolating models. In: 7th AIAA/USAF/NASA/ISSMO
symposium on multidisciplinary analysis and optimization; 1998. p. 4758.
[51] Bhandari B, Lee K-T, Lee G-Y, Cho Y-M, Ahn S-H. Optimization of hybrid
renewable energy power systems: a review. Int J Prec Eng Manuf-Green
Technol 2015;2(1):99–112.
[52] Gipe P. Wind power: renewable energy for home, farm, and business. Chelsea
Green Pub. Co; 2005.
[53] Wan Y. Long-term wind power variability. Contract 2012;303(275–3000).
[54] Wan Y-h. Analysis of wind power ramping behavior in ERCOT. Contract
2011;303(275-3000).
[55] Box GE, Jenkins GM, Reinsel GC, Ljung GM. Time series analysis: forecasting
and control. John Wiley & Sons; 2015.
[56] Liu L-M, Hudak GB, Box GE, Muller ME, Tiao GC. Forecasting and time series
analysis using the SCA statistical system. IL: Scientific Computing Associates
DeKalb; 1992.
[57] Karatepe S, Corscadden KW. Wind speed estimation: incorporating seasonal
data using Markov chain models. ISRN Renew Energy 2013;2013.
[58] Robertson DG, Lee JH. On the use of constraints in least squares estimation and
control. Automatica 2002;38(7):1113–23.
[59] Pender J. The truncated normal distribution: applications to queues with
impatient customers. Oper Res Lett 2015;43(1):40–5.
[60] Kim Y-H, Kim S-H. Increasing effect analysis of the wind power limit using
energy storage system in Jeju-Korea; 2014.
[61] Olivares DE, Lara JD, Cañizares CA, Kazerani M. Stochastic-predictive energy
management system for isolated microgrids. IEEE Trans Smart Grid 2015;6
(6):2681–93.
[62] Thomas MM, Kardos J, Joseph B. Shrinking horizon model predictive control
applied to autoclave curing of composite laminate materials. In: American
control conference; 1994. IEEE; 1994. p. 505–9.
[63] Almassalkhi M, Hiskens I. Impact of energy storage on cascade mitigation in
multi-energy systems. In: Power and energy society general meeting, 2012
IEEE. IEEE; 2012. p. 1–8.
[64] Kopanos GM, Georgiadis MC, Pistikopoulos EN. Energy production planning of
a network of micro combined heat and power generators. Appl Energy
2013;102(1522–1534).
[65] Šumbera J. Modelling generator constraints for the self-scheduling problem.
Vedecký seminár doktorandu FIS–únor 2012.
[66] Wu C, Mohsenian-Rad H, Huang J, Wang AY. Demand side management for
wind power integration in microgrid using dynamic potential game theory.
GLOBECOM Workshops (GC Wkshps), 2011 IEEE. IEEE; 2011. p. 1199–204.