This article has been accepted for inclusion in a future issue of this journal.
Content is final as presented, with the exception of pagination.
IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY
Spatiotemporal Optimization Through Gaussian Process-Based Model
Predictive Control: A Case Study in Airborne Wind Energy
Shamir Bin-Karim , Alireza Bafandeh , Ali Baheri, and Christopher Vermillion
Abstract— This brief presents a model predictive control
(MPC)-based spatiotemporal optimization strategy that is applied
to the problem of optimizing the altitude of a type of airborne
wind energy (AWE) system, specifically a buoyant airborne
turbine. Altitude optimization for AWE systems represents a
challenging problem under which the wind speed varies with
both time and altitude, is only instantaneously observable at the
altitude where the AWE system is operating, and dictates the
net power produced by the system. The proposed MPC strategy
avoids the need for a computationally expensive Markov process
model for characterizing the wind speed and is structured in a
way that the need for instantaneous power maximization (termed
exploitation) is balanced with the need to maintain an accurate
map of wind speed versus altitude (termed exploration). The
MPC strategy is calibrated through a Gaussian process regression
framework. Real wind speed versus altitude data have been used
to validate the strategy.
Index Terms— Airborne wind energy (AWE) system, explo-
ration versus exploitation, Gaussian process (GP), model predic-
tive control (MPC), partially observable system, spatiotemporal
optimization, uncertainty modeling.
I. I NTRODUCTION
I N RECENT years, many commercial and research insti-
tutions have been working in the area of airborne wind
energy (AWE) systems (see [1]). These systems replace con-
ventional towers with tethers and a lifting body to reach
altitudes that are inaccessible with traditional wind turbines,
and avoid high tower and foundation costs. A majority
of these systems use crosswind flight to produce energy
(see [2]–[5]). However, only a few studies have examined
the use of altitude adjustment to optimize the wind speed
presented to the airborne system (see [6]–[9]).
In [6], the wind shear profile (i.e., wind speed vs. altitude) is
assumed to follow a known monotonic power law profile. This
assumption is often violated in practice, as illustrated through
data presented in this brief. Bafandeh and Vermillion [7], [8]
Manuscript received September 24, 2017; accepted November 18, 2017.
Manuscript received in final form November 28, 2017. This work was
supported by the National Science Foundation under Grant 1437296.
Recommended by Associate Editor L. Fagiano. (Corresponding author:
Shamir Bin-Karim.)
S. Bin-Karim, A. Bafandeh, and A. Baheri are with the Mechanical
Engineering and Engineering Science Department, University of North
Carolina at Charlotte, Charlotte, NC 28223 USA (e-mail: sbinkari@uncc.edu;
abafande@uncc.edu; akhayatb@uncc.edu).
C. Vermillion is with the Engineering Science Department, Faculty of the
Mechanical Engineering, University of North Carolina at Charlotte, Charlotte,
NC 28223 USA, and also with Altaeros Energies, Boston, MA 02143 USA
(e-mail: cvermill@uncc.edu).
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TCST.2017.2779428
1063-6536 © 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
1
lifted this monotonicity assumption by using a switched
extremum seeking technique to hunt for the optimal altitude in
real time. However, this technique is prone to get stuck in local
optima, particularly in the presence of complex, nonmonotonic
wind shear profiles.
The altitude optimization problem is in fact one of many
optimization applications that involve a spatiotemporally vary-
ing random variable (wind speed), such as [10] and [11].
In optimal control applications that only involve a tempo-
rally varying random variable (such as power scheduling for
conventional wind energy systems in [12]–[16] and power
split management for hybrid electric vehicles in [17]–[19]),
it is common to characterize the evolution of the random
variable with a Markov model. However, when a scalar random
variable is varying both spatially and temporally, an exponen-
tial dependence of computational complexity on the number
of states renders Markov-based approaches computationally
infeasible. Furthermore, the random variable (wind, in our
case) is partially observable, as we only measure wind speed
at the operating altitude. In the case of partial observability,
balancing exploration with exploitation is very important.
However, this is not considered through the application of
traditional stochastic model predictive control (SMPC).
This brief presents an MPC-based spatiotemporal opti-
mization that does not require computationally expensive
Markov models and does balance exploration with exploita-
tion. This brief replaces the heuristic statistical modeling tech-
nique discussed in [20] with established tools from Gaussian
process (GP) modeling [21]. The fusion of GP-based modeling
with SMPC has been discussed in [22]–[24]. Diwale et al. [25]
used GP for learning system dynamics of an AWE system.
Nevertheless, the aforementioned publications do not address
the exploration–exploitation balance. This brief also presents
a comprehensive analysis of the proposed MPC strategy
under varying exploration weights (which dictate the balance
between exploration and exploration) and varying rate limits
(which dictates the maximum allowable altitude change from
one time step to the next), and provides a comparison between
the performance under the local Lyapunov-based switched
extremum seeking (LSES) altitude control strategy of [8] and
the MPC strategy pursued in this brief. The specific case study
in this brief focuses on a 100-kW AWE system with airborne
energy generation. This conforms to the design of the Altaeros
buoyant airborne turbine (BAT), which uses a lighter-than-air
shroud to lift a horizontal-axis turbine.
This brief is organized as follows. Section II provides a
data-based discussion of the random nature of the wind shear
profile, along with a simple model for the net power output of
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
2 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY
Fig. 1. Spatial variation in wind speed. Distance to BAT versus wind
speed deviation, with respect to wind speed at a base altitude (z 0 ) of 546 m
(at 00.00 h, i.e., midnight), computed for 15 days during January 2014 (starting
from January 1), based on actual data from [26].
the BAT. Section III provides a review of GP modeling and
discusses kernel selection (for capturing the characteristics of
the conditional mean and variance of the wind shear profile)
and hyperparameter identification (for fitting parameters to
these models of conditional mean and variance). Section IV
details the cost function and constraints for the MPC strat-
egy used in this brief. Section V illustrates the resulting
altitude profiles and energy output that emanate from the
MPC strategy, demonstrating that this strategy outperforms the
LSES control algorithm presented in [8] and offline-optimized
constant altitude operation. This section also presents a
detailed assessment of the effect of different exploration
weights and rate limits.
II. W IND P ROFILE AND E NERGY G ENERATION M ODEL
A. Wind Profile
The wind data used in this brief was obtained from
a 915-MHz Doppler RADAR-based wind profiler in Cape
Henlopen State Park [26] (located in Lewes, Delaware). This
data set includes average wind speeds at 30 min time intervals
with a consensus averaging time of 24 min, at 48 altitudes
from 146 up to 3000 m. The quantized data was interpolated
in space and time for the purposes of control system validation.
Considering practical flight altitude limitations of the BAT, the
portion of data involving tabulated altitudes up to 947 m was
considered for this research.
To maximize energy generation, the BAT should operate at
altitudes where the wind speed is as close as possible to the
turbine’s rated wind speed, v rated . Clearly, operation in winds
below v rated will result in reduced power output, but operation
in winds above v rated will also reduce the net power output,
due to the increased power required to control the system in
high winds. Because the wind speed varies significantly with
time and altitude, determining which altitude will yield a wind
speed closest to v rated is a challenging task.
Fig. 1 shows the spatial variation in wind speed at
31 separate instances during January 2014. All measure-
ments were taken at a specific time, t0 , during each day
Fig. 2. Temporal variation of wind speed, with respect to base data taken at
time t0 , at an altitude of 146 m, for 31 days during January 2014, based on
actual data from [26].
(which happened to be midnight). To illustrate the spatial
variability of wind shear profiles taken at significantly different
times (all at least 24 h apart from each other), Fig. 1 plots the
difference between the wind speed at a particular altitude, z,
and the wind speed at a reference altitude, z 0 (which is taken
as 546 m in Fig. 1), denoted by v wind,spatial(z, t0 , z 0 )
v wind,spatial(z, t0 , z 0 ) = v wind (t0 , z 0 + z) − v wind (t0 , z 0 ).
(1)
v wind,spatial(z, t0 , z 0 ) increases with increasing z. From
an altitude optimization perspective, this implies, unsurpris-
ingly, that one’s ability to predict wind speed at a particular
altitude depends on the proximity of that altitude to ones where
data have already been gathered.
Fig. 2 shows the temporal nature of wind speed over the
31 days in January 2014, at a constant altitude z 0 = 146 m.
As with Figs. 1 and 2, the difference in wind speed between
time t and a reference time t0 is denoted by v wind,temporal
(t, t0 , z 0 )
v wind,temporal(t, t0 , z 0 ) = v wind (t0 + t, z 0 )−v wind (t0 , z 0 ).
(2)
The qualitative conclusions in Fig. 2 are again unsurprising;
given a measurement of wind speed at time t0 , the proximity
of wind speed to v wind (t0 , z 0 ) diminishes with increasing time.
From an altitude optimization perspective, Figs. 1 and 2 show
that in order to maintain accurate knowledge of the wind shear
profile, it is necessary to continually explore different altitudes;
otherwise, the uncertainty in wind speed grows to be very
large. A large-scale analysis of the data from Cape Henlopen
was used to quantify the statistical variability of wind speed
with time and altitude, thereby providing the appropriate
mathematical mechanism for incentivizing exploration, which
is detailed in Section III.
B. Energy Generation Model
A portion of the total energy produced by the turbine
in an AWE system is consumed to adjust or maintain the
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
BIN-KARIM et al.: SPATIOTEMPORAL OPTIMIZATION THROUGH GP-BASED MPC
operating altitude. This energy is required to reject the distur-
bances resulting from changes in wind velocity and direction.
In light of this, the instantaneous net power generation can be
approximated as follows:
Ptotal = Ptur (v wind (z i , ti ), v rated )
− Pmaint (v wind (z i , ti ))
+ Pz (v wind (z i , ti ), z i , z i−1 )
where Ptur is the power generated by the turbine, Pmaint is the
power required to maintain altitude, Pz is the instantaneous
power required for or regenerated by adjusting altitude, z i is
the operating altitude at time ti , v wind (z i , ti ) is the wind speed
at the operating altitude at time ti , and v rated is the turbine’s
rated wind speed. Because any upward altitude adjustment
of the system must later be compensated with a downward
altitude adjustment (the system must land eventually), and
because it is not the explicit objective of a system with
airborne energy generation to strategically leverage additional
energy through regeneration on the ground, it is possible to
simplify the objective function for the system, which should
be maximized, as follows:
P = C1 min(v wind (z i , ti ), v rated )3
− C2 [v wind (z i , ti )]2
|z i − z i−1 |
− C̄3 [v wind (z i , ti )]2 (4)
t
where C1 and C2 are the constant parameters that depend
upon the turbine’s power curve and flight control system
(comprising both the controller and the aerodynamics of the
lifting body), respectively. C̄3 is a lumped parameter that is
dependent on overall winch efficiency and aerodynamics of
the lifting body. (See [8], for a detailed discussion about
the parameters C1 , C1 , C̄3 and their derivation from actual
aerodynamic data from the BAT.) t is the elapsed time
between time step ti−1 and ti .
III. S TATISTICAL W IND S HEAR P ROFILE
C HARACTERIZATION
For spatiotemporal optimizations where the random
variable is partially observable in space, an appropriate bal-
ance between exploitation and exploration must be achieved.
Exploitation involves making the best decision given the
current information. Exploration, on the other hand, involves
information gathering. In the case of AWE systems, exploita-
tion involves using past and present wind data to determine
and go to the optimal altitude for maximum energy generation.
However, if the AWE system sits at a fixed altitude for a long
time, the knowledge of the wind velocity at altitudes it visited
earlier on the same day becomes less effective as earlier wind
velocity measurements could be very different from the wind
velocities later in the day (see Fig. 2). To quantify the need
for exploration, the statistical characteristics of the wind shear
profile are very important.
In striking the appropriate balance between exploration and
exploitation, we are interested in understanding the value of
new data in terms of predicting wind speed. As such, we
are interested in the conditional probability characteristics of
3
wind speed, conditioned upon past and present measurements.
These can be characterized through a conditional mean and
conditional variance, denoted by
 
μc  E v wind (z, t)|D (5)
and
(3) (σ 2 )c  E[(v wind (z, t) − E[v wind (z, t)])2 |D] (6)
respectively. Here, D represents a matrix of m previously
measured wind speeds and the corresponding measurement
points. To precisely characterize the aforementioned condi-
tional statistics, D should include all previously acquired data
points; however, for practical purposes, considering a finite set
of m recent points is sufficient to accurately approximate the
quantities, as spatiotemporally distant measurements provide
a little predictive values. D is specified by
⎡ ⎤
z0 ... z m−1
D=⎣ t0 ... tm−1 ⎦. (7)
v wind (z 0 , t0 ) . . . v wind (z m−1 , tm−1 )
In this brief, we use tools from the machine learning commu-
nity to automatically characterize the aforementioned statisti-
cal properties.
A. Preliminaries for Gaussian Process Modeling
Let Z denotes the domain of allowable altitude choices and
T denotes the temporal space. Our goal is to model wind
speed at altitude z ∈ Z and time t ∈ T . In order to do that,
we construct a function f : Z × T → R, whose output is
a random variable y. In this application, since wind speed
can be reasonably modeled by a log-normal distribution [27],
y is taken as the natural log of the ratio of the wind velocity
v wind (z, t) to a reference wind velocity, v wind (z 0 , t0 )

v wind (z, t)
y = log . (8)
v wind (z 0 , t0 )
This transformation results in a random variable that can
be reasonably modeled by a GP [21]. The GP is completely
specified by its mean function and covariance function, and is
written as
f (z, t) ∼ GP(μ(z, t), k((z, t), (z  , t  ))) (9)
where μ(z, t) is the mean function and k((z, t), (z  , t  )) is the
covariance function of the GP prior. Let Y be the recently
collected observations at X, where
X = [(z 0 , t0 ), . . . , (z m−1 , tm−1 )] (10)
Y = [y0 , . . . , ym−1 ] . T
(11)
Given the aforementioned mean and covariance functions,
the conditional mean and variance (which we are ultimately
interested in for this application) can be written as
μ̂c = μ(z, t|Y, X) (12)
= μ(z, t) + k̂ X (z, t)T ( K̂ X + σ 2 I )−1 (δY )
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

4 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY

and Similar to the spatial covariance kernel in (18), the structure

 of k2 in (19) describes a stationary kernel for which the
(σ̂ 2 )c = k (z, t), (z, t)|X covariance is a function of the temporal separation of two


= k (z, t), (z, t) − k̂ X (z, t)T ( K̂ X + σ 2 I )−1 k̂ X (z, t) points, relative to a length-scale factor.
(13) Based on these spatial and temporal kernel structures, (17)
can be written as
respectively. Here, δY is the deviation of Y from its prior mean 
  (z − z  )2 (t − t  )2
δY = [y0 − μ(z 0 , t0 ), . . . , ym−1 − μ(z m−1 , tm−1 )]T . (14) k((z, t), (z , t )) = σ f exp −
2
− (20)
2l12 2l22
k̂ X (z, t) is a column vector of the covariance values between
(z, t) and every observed location in X where σ 2f = σ12 σ22 . The structure of the SE kernel ensures
that if two data points are close in spatiotemporal space, they
k̂ X (z, t) = [k((z 0 , t0 ), (z, t)), . . . , k((z m−1 , tm−1 ), (z, t))]T ∈ Rm. have higher correlation than points those are further away
(15) spatiotemporally.

K̂ X is the Gram matrix that encodes the pairwise covariance

values between all locations in X
K̂ X = [k((z i , ti ), (z j , t j ))]i, j ∈[0,1,...,m−1] ∈ Rm×m
and σ 2 is the noise variance (assuming independent identically
distributed Gaussian noise).
B. Kernel Functions
In order to use a GP model in the spatiotemporal optimiza-
tion problem at hand, it is necessary to generate relatively
simple parametric approximations for the conditional mean
and variance of (12) and (13). Because the conditional mean
and variance depend heavily on the covariance matrix, it is
consequently important to be able to characterize the covari-
ance
between any  pair of spatiotemporal points, denoted by
k (z, t), (z  , t  ) , through a relatively simple parameterization.
This parameterization is obtained through the choice of a
covariance kernel, which provides a simple structure for the
covariance between two spatiotemporal points [28], [29]. For
the problem at hand, the covariance kernel is obtained through
the multiplication of two components—a spatial component
(k1 (z, z  )) and a temporal component (k2 (t, t  ))
k (z, t), (z  , t  ) = k1 (z, z  )k2 (t, t  ). (17)
1) Form of the Spatial Covariance Kernel (k1 ): For k1 , a
C. Hyperparameter Estimation
Incorporation of a GP model into an MPC strategy requires
(16)
not only a parametric model for conditional mean and variance
but also a mechanism for identifying the parameters of the
kernel, termed hyperparameters. In the case of the SE kernel
structure proposed in this brief, these hyperparameters include
l1 , l2 , and σ f . In practice, we identify these parameters
through the maximization of the log marginal likelihood over
the domain of candidate hyperparameters (for more details,
see [31]). The log marginal likelihood, which reflects the log-
arithm of the likelihood of achieving a particular observation
Y based on past observations X and hyperparameter set θ , is
given by
1
L = log p(Y |X, θ ) = − Y T ( K̂ X + σ 2 I )−1 Y
2
1 m
− log |( K̂ X + σ 2 I )| − log 2π. (21)
2 2
For the application at hand, θ represents the set of kernel
hyperparameters
θ = {l1 , l2, σ f }. (22)
To determine the hyperparameters by maximizing the mar-
ginal likelihood, partial derivatives of the marginal likelihood
with respect to the hyperparameters are calculated
squared exponential (SE) kernel, also known as the radial basis 

function kernel, is used. It has the form ∂L 1 T −1 ∂ K̂ X −1 1 −1 ∂ K̂ X

= Y K̂ X
1 (z − z  )2
k1 (z, z  ) = σ12 exp − (18)
2 l12
where l1 and σ12 are hyperparameters. The parameter l1 , known
as the characteristic length scale, determines the length of the
spatial fluctuations in the function f . The hyperparameter σ12
is called the signal variance, which correlates with the average
distance of the function f away from its mean [30].
2) Form of the Temporal Covariance Kernel (k2): For k2 ,
we have also used an SE kernel with hyperparameters l2
and σ22 as the characteristic length scale and signal variance,
respectively,
K̂ X Y − trace K̂ X . (23)
∂θ 2 ∂θ 2 ∂θ
For the AWE system, available wind shear data were used
to identify the hyperparameters in (22) that minimize the
parameter estimation marginal likelihood, using a gradient-
based optimizer.
IV. M ODEL P REDICTIVE C ONTROL F ORMULATION
The basic idea of MPC is to exploit a model of the process to
predict the future evolution of the system and compute control
actions by optimizing a cost function that depends on these
predictions [16]. In performing this optimization, MPC is one

of a select few control strategies that can explicitly consider
 1 (t − t  )2
k2 (t, t ) = σ2 exp −
2
. (19) hard constraints on both the control signal and the system
2 l22 states.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
BIN-KARIM et al.: SPATIOTEMPORAL OPTIMIZATION THROUGH GP-BASED MPC
For the spatiotemporal optimization problem at hand, the
MPC optimization minimizes a cost function, J
−1
k+T
J (zsp (k)) = [ Jexploit (z sp (i |k)) + w Jexplore (z sp (i |k))]
i=k
subject to constraints, as follows:
z∗ (k) = arg min J (z(k))
z(k)
subject to z(k + 1) = z sp (k)
|z(i |k) − z(i − 1|k)| ≤ rlim t,
z(i |k) ∈ , i = k . . . k + T − 1
Equation(12)
Equation(13)
where T is the MPC prediction horizon,  is the set of
allowable altitudes, rlim is a rate limitation (corresponding to
the fact that tethers can only be spooled out/in so fast), and
t is the time step. The vector z(k) corresponds to the altitude
trajectory vector over the receding horizon, given as
zsp (k) = [ z sp (k|k) . . . z sp (k + T − 1|k) ]T .
The notation z(i |k) denotes the value of z(i ) for the optimiza-
tion taking place at step k. w is a scalar weighting parameter
that sets the balance between exploration and exploitation in
the cost function. As is typical with MPC, the first term in the
optimized control input trajectory represents the commanded
control input at step k. In this case, the commanded control
input is the altitude setpoint, given as
z sp (k) = z ∗ (k|k).
It is assumed in this brief that an underlying lower level flight
control system regulates the system’s operating altitude (z)
to the setpoint (z sp ). Such an altitude control system has been
validated numerically and experimentally in both [3] and [32].
The exploitation term in the cost function is given as
Jexploit (z(i |k)) = Prated − E(P(z(i |k))|D).
Here, E(P(z(i |k))|D) is the expected power output for the
AWE system at a given altitude, conditioned upon m previous
measurement data (denoted by D). Prated is the (constant) rated
power of the AWE system, which is the maximum power that
can be produced at any instant.
The exploration term is given as
z  =z
 on the potential of any real-time strategy.
max
Jexplore (z(i |k)) = (σ̂ 2 )c (t (i ), z  )|D, D
z  =z min
where z max and z min are the upper and lower bounds of
explorable altitudes. D is given as
 
 z(k|k) . . . z(i |k)
D = . (30)
t (k) . . . t (i )
Thus, at each time step, the exploration term penalizes the
one-norm over altitude of the conditional variance of the wind
speed at each step in the MPC horizon, conditioned upon
5
TABLE I
D ESIGN PARAMETERS
(24)
all measurements taken up to step i . D is included in the
calculation of the exploitation term expressly for the purpose
of considering the contribution of future measurements on
the conditional variance of the wind speed prediction, taken
throughout the MPC prediction horizon. Thus, the one-norm
that is penalized in (29) is kept small by continually exploring
(25)
the range of available altitudes throughout the MPC horizon.
For the altitude optimization problem, design parameters
were selected as shown in Table I. The maximum and min-
imum allowable altitudes are dictated by available data. The
interpolation of wind data can be justified given the gradual
nature of wind speed change, as the hyperparameter values
suggest.
(26)
V. R ESULTS
A. MPC With Exploration
To demonstrate the merits of the proposed MPC-based
altitude optimization, the system performance was compared
under five different control “scenarios.” An exploration rate
limit of 2 m/s has been considered for all scenarios.
Scenario 1 (MPC With Exploration): This scenario uses the
MPC control scheme of (25) with an exploration weight of
(27)
w = 300 (i.e., the exploration term is active). The reason for
using this particular weight is discussed in Section V-B.
Scenario 2 (Off-Line Optimized Trajectory): This scenario
involves flying the BAT at its optimal altitude, based on an off-
line optimization of net energy generation. The optimization
is as follows:
τ −1

(28) minimize Joffline = −P(v wind (z(t), t))
z(t )
t =0
subject to z(t) ∈ , t = 0 . . . τ − 1
|z(t) − z(t − 1)| ≤ rlim t (31)
where τ is the final time step. It should be noted that this calcu-
lation requires knowledge of future and is not implementable
in real time. However, this scenario represents an upper bound
(29) Scenario 3 (Off-Line Optimized Constant Altitude): This
scenario involves off-line determination of the optimal con-
stant altitude during the time period over which performance
is being evaluated
τ −1

minimize Jconst.alt. = −P(v wind (z, t)). (32)
z=constant
t =0
This scenario requires future knowledge and also, therefore,
cannot be implemented in real time. However, it represents an
upper bound on the potential of any constant altitude strategy.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

6 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY

Fig. 3. Comparison of different control strategies for a duration of one week. Data acquired from January 12, 2014 to January 18, 2014. (a) Comparison of
altitude setpoints. (b) Comparison of wind speed during different altitude setpoints. (c) Comparison of net energy generation.

Scenario 4 (Off-Line Optimized Constant Altitude Using
Historical Data): This scenario uses the historical data
from [26] to determine the optimal constant altitude over
a lengthy period of time. This historical data comes from
a different time window than the time window over which
performance is being evaluated. Therefore, the control strat-
egy is implementable in real time. The offline-optimized
constant altitude is determined by performing the following
optimization:

tn
minimize Jhist. = −P(v wind (z, t))ti . . . tn ∈ D.
z=constant
t =ti
Here, ti is the initial time in the data set, and tn represents the
last time instance that is used in training this constant altitude
control strategy.
Scenario 5 (LSES): In this scenario, we consider the
extremum seeking-based altitude optimization algorithm, a
nonmodel-based control scheme, proposed in [7] and [8].
Here, extremum seeking applies a sinusoidal perturbation to
the altitude setpoint that allows the BAT to explore the wind
profile locally, and then move in the direction of increasing
net power output. A Lyapunov-based switch curtails the size
of this perturbation (and, therefore, reduces control energy
expenditure) when convergence upon an optimal altitude is
detected. For a detailed discussion of LSES theory, readers
(33) are referred to [33]; for details on the application to the AWE
system at hand, readers are referred to [8].
The objective of the altitude optimization in this brief is
to maximize the net energy output of the BAT. The control
strategies have been simulated on a desktop computer with
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

BIN-KARIM et al.: SPATIOTEMPORAL OPTIMIZATION THROUGH GP-BASED MPC 7

Fig. 4. Energy produced over a week in January 12–18, 2014 using different
exploration weights. Maximum energy is produced using w = 300.

an Intel core-i7-4790 3.60-GHz processor and 16.0-GB RAM.

The optimization problem was solved with a commercial
nonlinear programming solver, using sequential quadratic pro-
gramming. For training the hyperparameters, one week of data
(January 5–11, 2014) was used. Fig. 3 shows the altitude
setpoint, wind speed at operating altitude and cumulative net
energy production under the five aforementioned scenarios,
for a period of one week. Actual wind data from [26] were
used. It is clear from Fig. 3 that MPC with exploration
generates the best performance in terms of net energy gener-
ation among control strategies that are implementable in real
time.

B. Impact of Exploration Weight

The scalar weighting parameter w plays an important role
in the altitude optimization strategy. Selecting too high a value
for w causes overexploration. A large w keeps the uncertainty
measure (σ̂ 2 )c small in the cost function, but requires the
BAT to lose energy due to unnecessary exploration. A low
value of w, on the other hand, does not provide enough
initiative for the system to explore the available spatial domain.
Fig. 4 illustrates the energy production under the pro-
posed MPC strategy (Scenario 1) using different exploration
weights (w). For the data set used in this brief, the optimal off-
line value for the exploration weight was found as w = 300.
Using a value lower or higher than this heuristic optimal off- Fig. 5. Comparison of different rate constraints for a duration of one
week. Data acquired from January 12, 2014 to January 18, 2014. (a) Energy
line value for the weighting parameter w results in lower production with 2 m/s rate limit. (b) Energy production with 1.5 m/s rate
net energy production, as shown in Fig. 4. In fact, the MPC limit. (c) Energy production with 1 m/s rate limit.
strategy with w = 0 completely fails to follow the off-
line optimized trajectory and misses out on significant energy the proposed MPC control strategy significantly outperforms
generation potential, as manifested in Fig. 4. offline-optimized altitude based on historical data.

C. Exploration Rate Limitation VI. C ONCLUSION AND F UTURE W ORK

Constraints over the exploration rate of the BAT represent a A. Conclusion

practical reality that is dictated by the system dynamics and the
corresponding winch and tether system. In this brief, we have
studied the impact of different rate limitations (rlim ) on net
energy production. Fig. 5 shows the comparison among three
different rate limits. As expected, the BAT produces the most
net energy under the least severe rate limit of 2 m/s, as shown
in Fig. 5(a). However, even with rate limits as strict as 1 m/s,
This brief proposes an MPC strategy that balances explo-
ration and exploitation in optimizing the altitude setpoint for
a buoyant AWE system. Ultimately, the goal is to find the
operating altitude that results in optimal power output while
maintaining an accurate characterization of the wind shear
profile (wind speed vs. altitude). Simulation results, which are
based on real wind data, show that MPC with exploration can
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
8 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY
outperform constant altitude operation and MPC strategies that
do not include exploration. MPC with exploration has been
investigated further for different exploration weights.
B. Future Work
Future work will include the following.
1) Implementation of a real-time controller that dynami-
cally adjusts the exploration weight as a function of
recent statistical properties of the wind.
2) Experimental validation of altitude optimization algo-
rithms on the Altaeros BAT, where the optimization will
be performed on a high-performance ground-based PC.
ACKNOWLEDGMENT
The authors would like to thank C. Archer of the University
of Delaware for providing the wind shear data used in this
brief [26].
R EFERENCES
[1] A. Cherubini, A. Papini, R. Vertechy, and M. Fontana, “Airborne wind
energy systems: A review of the technologies,” Renew. Sustain. Energy
Rev., vol. 51, pp. 1461–1476, Nov. 2015.
[2] M. Canale, L. Fagiano, and M. Milanese, “High altitude wind energy
generation using controlled power kites,” IEEE Trans. Control Syst.
Technol., vol. 18, no. 2, pp. 279–293, Mar. 2010.
[3] C. Vermillion, T. Grunnagle, and I. Kolmanovsky, “Modeling and control
design for a prototype lighter-than-air wind energy system,” in Proc.
Amer. Control Conf., Montreal, QC, Canada, Jun. 2012, pp. 5813–5818.
[4] C. Vermillion and L. Fagiano, “Tethered wind energy systems,” Mech.
Eng., vol. 135, no. 9, pp. S13–S21, 2013.
[5] A. U. Zgraggen, L. Fagiano, and M. Morari, “On real-time optimization
of airborne wind energy generators,” in Proc. 52nd IEEE Conf. Decision
Control, Florence, Italy, Dec. 2013, pp. 385–390.
[6] C. Vermillion, “Altitude and crosswind motion control for optimal
power-point tracking in tethered wind energy systems with airborne
power generation,” in Proc. ASME Dyn. Syst. Control Conf., Stanford,
CA, USA, 2013, p. V003T49A001.
[7] A. Bafandeh and C. Vermillion, “Real-time altitude optimization of
airborne wind energy systems using Lyapunov-based switched extremum
seeking control,” in Proc. Amer. Control Conf., Boston, MA, USA,
Jul. 2016, pp. 4990–4995.
[8] A. Bafandeh and C. Vermillion, “Altitude optimization of airborne wind
energy systems via switched extremum seeking—Design, analysis, and
economic assessment,” IEEE Trans. Control Syst. Technol., vol. 25,
no. 6, pp. 2022–2033, Nov. 2017.
[9] A. Baheri, S. Bin-Karim, A. Bafandeh, and C. Vermillion, “Real-time
control using Bayesian optimization: A case study in airborne wind
energy systems,” Control Eng. Pract., vol. 69, pp. 131–140, Dec. 2017.
[10] E. Frazzoli and F. Bullo, “Decentralized algorithms for vehi-
cle routing in a stochastic time-varying environment,” in Proc.
43rd IEEE Conf. Decision Control, Nassau, Bahamas, Dec. 2004,
pp. 3357–3363.
[11] A. Lecchini Visintini, W. Glover, J. Lygeros, and J. Maciejowski,
“Monte Carlo optimization for conflict resolution in air traffic con-
trol,” IEEE Trans. Intell. Transp. Syst., vol. 7, no. 4, pp. 470–482,
Dec. 2006.
[12] C. Li, H. Peng, and J. Sun, “Predictive control and sizing of energy
storage to mitigate wind power intermittency,” Wind Energy, vol. 19,
no. 3, pp. 437–451, 2016.
[13] P. Panagiotis, S. Trimboli, and A. Bemporad, “Stochastic MPC for
real-time market-based optimal power dispatch,” in Proc. Joint 50th
IEEE Conf. Decision Control Eur. Control Conf., Orlando, FL, USA,
Dec. 2011, pp. 7111–7116.
[14] W. Qi, J. Liu, X. Chen, and P. D. Christofides, “Supervisory predictive
control of standalone wind/solar energy generation systems,” IEEE
Trans. Control Syst. Technol., vol. 19, no. 1, pp. 199–207, Jan. 2011.
[15] C. Sun, F. Sun, and S. J. Moura, “Data enabled predictive energy
management of a PV-battery smart home nanogrid,” in Proc. Amer.
Control Conf., Chicago, IL, USA, Jul. 2015, pp. 1023–1028.
[16] F. Oldewurtel et al., “Use of model predictive control and weather
forecasts for energy efficient building climate control,” Energy Buildings,
vol. 45, pp. 15–27, Feb. 2012.
[17] G. Rippaccioli, D. Bernardini, S. Di Cairano, A. Bemporad, and
I. Kolmanovsky, “A stochastic model predictive control approach for
series hybrid electric vehicle power management,” in Proc. Amer.
Control Conf., Baltimore, MD, USA, Jun. 2010, pp. 5844–5849.
[18] S. J. Moura, H. K. Fathy, D. S. Callaway, and J. L. Stein, “A stochastic
optimal control approach for power management in plug-in hybrid
electric vehicles,” IEEE Trans. Control Syst. Technol., vol. 19, no. 3,
pp. 545–555, May 2011.
[19] E. D. Tate, Jr., J. W. Grizzle, and H. Peng, “Shortest path stochastic
control for hybrid electric vehicles,” Int. J. Robust Nonlinear Control,
vol. 18, no. 14, pp. 1409–1429, 2008.
[20] S. Bin-Karim, A. Bafandeh, and C. Vermillion, “Spatio-temporal opti-
mization through model predictive control: A case study in airborne
wind energy,” in Proc. 55th IEEE Conf. Decision Control, Las Vegas,
NV, USA, Dec. 2016, pp. 4239–4244.
[21] C. Rasmussen and C. Williams, Gaussian Processes for Machine Learn-
ing. Cambridge, MA, USA: MIT Press, 2006.
[22] J. Kocijan, R. Murray-Smith, C. E. Rasmussen, and A. Girard, “Gaussian
process model based predictive control,” in Proc. Amer. Control Conf.,
vol. 3. Boston, MA, USA, Jun. 2004, pp. 2214–2219.
[23] A. Grancharova, J. Kocijan, and T. A. Johansen, “Explicit stochastic
predictive control of combustion plants based on Gaussian process
models,” Automatica, vol. 44, no. 6, pp. 1621–1631, 2008.
[24] G. Cao, E. M.-K. Lai, and F. Alam, “Gaussian process based model
predictive control for linear time varying systems,” in Proc. IEEE 14th
Int. Workshop Adv. Motion Control (AMC), Apr. 2016, pp. 251–256.
[25] S. S. Diwale, I. Lymperopoulos, and C. Jones, “Optimization of an
airborne wind energy system using constrained Gaussian processes with
transient measurements,” in Proc. 1st Indian Control Conf., Chennai,
India, 2015.
[26] C. Archer. (2014). “Wind profiler at cape henlopen.” Dept. Phys. Ocean
Sci. Eng., Univ. Delaware, Lewes, DE, USA, Tech. Rep. [Online].
Available: http://www.ceoe.udel.edu/our-people/profiles/carcher/fsmw
[27] A. Garcia, J. L. Torres, E. Prieto, and A. De Francisco, “Fitting
wind speed distributions: A case study,” Solar Energy, vol. 62, no. 2,
pp. 139–144, 1998.
[28] A. Wilson and R. Adams. (2013). “Gaussian process kernels for
pattern discovery and extrapolation.” [Online]. Available: https://
arxiv.org/abs/1302.4245
[29] D. Duvenaud, J. Lloyd, R. Grosse, J. Tenenbaum, and Z. Ghahramani,
“Structure discovery in nonparametric regression through compositional
kernel search,” in Proc. ICML, vol. 3. 2013, pp. 1166–1174.
[30] D. J. MacKay, Information Theory, Inference and Learning Algorithms.
Cambridge, U.K.: Cambridge Univ. Press, 2003.
[31] C. Rasmussen, “Gaussian processes in machine learning,” in Advanced
Lectures on Machine Learning. Berlin, Germany: Springer-Verlag, 2004,
pp. 63–71.
[32] C. Vermillion, T. Grunnagle, R. Lim, and I. Kolmanovsky, “Model-
based plant design and hierarchical control of a prototype lighter-than-air
wind energy system, with experimental flight test results,” IEEE Trans.
Control Syst. Technol., vol. 22, no. 2, pp. 531–542, Mar. 2014.
[33] S. J. Moura and Y. A. Chang, “Lyapunov-based switched extremum
seeking for photovoltaic power maximization,” Control Eng. Pract.,
vol. 21, no. 7, p. 971–980, 2013.