Journal of Petroleum Science and Engineering 159 (2017) 314–330

Contents lists available at ScienceDirect

Journal of Petroleum Science and Engineering

journal homepage: www.elsevier.com/locate/petrol

An efficient adaptive algorithm for robust control optimization

using StoSAG
Ranran Lu, Fahim Forouzanfar *, Albert C. Reynolds
The University of Tulsa, Tulsa, OK, USA

A R T I C L E I N F O A B S T R A C T

Keywords: There is significant geological uncertainty in the reservoir description due to the limited knowledge about the
Robust reservoir optimization underground formation. The common approach is to use multiple plausible geostatistical realizations of the
Adaptive sub-set optimization reservoir model in order to quantify the uncertainty, however, usually a small ensemble of realizations is utilized
Well control optimization in the robust optimization due to the computational cost. This may results in erroneous estimation of uncertainty
Stochastic gradient in the predicted optimal reservoir production performance. We propose an efficient algorithm for robust opti-
Ensemble based optimization
mization where a large number of representative realizations are considered. The algorithm proposed in this work
assumes confidence in a subset of realizations for representing the full-set of realizations in the neighbourhood of
the current control estimate. The subset is chosen based on a ranking method and modified adaptively throughout
the optimization iterations. The optimizer in this paper is the steepest ascent method using the Stochastic Simplex
Approximate Gradient (StoSAG), but the proposed algorithm is suitable to be combined with other optimization
algorithms. Three synthetic reservoir examples generated with different geostatistical modelling methods are
tested to validate the proposed algorithm. All three examples show that compared with a full-set robust opti-
mization, the proposed adaptive robust optimization algorithm not only improves the optimization convergence
rate, but also find a higher optimal NPV when the optimization is terminated at a maximum affordable simulation
cost using the stochastic simplex gradient.

1. Introduction
Formulating the well control planning as an optimization problem is
becoming a common approach in the petroleum industry. The life-cycle
reservoir optimization problem aims at maximizing a production per-
formance measure, such as the net present value (NPV) or cumulative oil
production, over the presumed reservoir life. The well controls are usu-
ally continuous variables and the objective function surface of the well
control optimization problem is relatively smooth, compared to the well
placement optimization problem. Numerous previous work have shown
that the optimal well control problem by itself is amenable to solution by
gradient based method, see Kraaijevanger et al. (2007) and Sarma et al.
(2005). However, the adjoint gradient is difficult and tedious to code,
and any gradient based method would at best find a local optimum. In
addition, stochastic methods, i.e., PSO (Wang et al., 2015), CMA-ES
(Forouzanfar et al., 2015), StoSAG (Fonseca et al., 2016), SPSA (Li
et al., 2013), have been tried in the past to solve the reservoir optimi-
zation problem. StoSAG as proposed by Fonseca et al. (2016) is chosen as
the optimizer in this paper (Ramirez et al., 2017). applied the StoSAG for
the robust optimization of vertical well locations (Hanea et al., 2017).
applied the StoSAG for well trajectory optimization under structural
uncertainties. Lu et al. (2017) also developed a variant of the StoSAG for
the biobjective joint optimization of the well trajectories and controls.
To formulate the optimization problem, the reservoir life is split into a
set of control steps at which the well controls are tuned, e.g., Zhang et al.
(2017). In most of previous studies the number and length of control
steps are fixed and specified a priori. However, if the pre-specified con-
trol steps are too few, the solution may be suboptimal and if a large
number control steps are prescribed, the problem may be ill-conditioned.
This also usually increases the computational cost of solving the opti-
mization problem. Few previous work investigated parameterizing the
well controls or adaptively changing the number of control steps and
their length. Awotunde et al. (2014) proposed to use the polynomial and
trigonometry models to describe the rate change of each well, and then
optimize the coefficients of these two models. Forouzanfar et al. (2015)
parametrized the vector of well controls in terms of a set of truncated
singular vectors of a pre-specified temporal covariance matrix. Lien et al.
(2008) investigate the use of a multiscale regularization methods to

* Corresponding author. ExxonMobil Upstream Research Company, Houston, TX, USA.

E-mail address: fahim-forouzanfar@utulsa.edu (F. Forouzanfar).

https://doi.org/10.1016/j.petrol.2017.09.002
Received 22 May 2017; Received in revised form 16 August 2017; Accepted 5 September 2017
Available online 12 September 2017
0920-4105/© 2017 Elsevier B.V. All rights reserved.
R. Lu et al. Journal of Petroleum Science and Engineering 159 (2017) 314–330

achieve grouping of the control settings of the wells in both space and and the StoSAG algorithm are presented. Thereafter, we present the
time, starting from a very coarse grouping and refining the resolution proposed adaptive robust optimization procedure together with three
subsequently. Oliveira et al. (2015) proposed a similar multiscale different conditions under which we re-evaluate the full-set NPV cdf. The
approach, but his algorithm also allows merging of existing control steps. application of the proposed algorithm for three synthetic reservoir ex-
Other variants of formulating the well control optimization problem have amples is presented next, and the results are compared with the ones
also been investigated. Besides the well rate, Petvipusit et al. (2015) obtained with robust optimization performed on the full-set of the
optimized the conversion time of a well switching from a producer to an reservoir models. Finally, a summary of the observations and conclusions
injector using an ensemble based method. They concluded that the of the work is presented.
simultaneous optimization of well rates and conversion times out-
performed the optimization of only well controls or only well conversion 2. Well control optimization
times. Chen and Reynolds (2017) proposed to simultaneously optimize
well operating conditions and ICV settings for a water-alternating gas 2.1. Optimal well control problem
injection process. They concluded that the simultaneous optimization
yields better or equal optimal NPVs compared to optimizing only the In this work, we consider maximizing the net present value (NPV) of
controls or ICV settings depending on the initial guess. Yang et al. (2011) production over the presumed life of the reservoir. Given a reservoir
applied the operating condition optimization to the steam-assisted model, the NPV function for the control optimization problem is given by
gravity drainage (SAGD) recovery process and found that the optimiza- ( "
X
Nt NP 
X 
tion could not only increase the NPV but also make the recovery process Δt n
Jðx; mÞ ¼ tn ron ⋅qno;j  cnw ⋅qnw;j  cng ⋅qng;j
more energy efficient and environmentally friendly. In this work, we ð1 þ bÞ 365
n¼1 j¼1
assume the number and length of control steps which span the reservoir #)
NI 
X 
life are pre-specified and fixed. n n n n
 cwi ⋅qwi;k þ cgi ⋅qgi;k ; (1)
Probably one of the most important considerations in the decision- k¼1
making process based on the optimization results would be taking into
account the geological uncertainty in the reservoir description, which where x is a Nx dimensional vector of the well control variables,
leads to the uncertainty in the performance prediction of the reservoir. including the injection/production rates of the wells under a rate control
The most common approach for handling geological uncertainty is to or bottom-hole pressure of wells under BHP control; m is a Nm dimen-
generate multiple reservoir descriptions and do robust optimization in sional vector of reservoir model parameters; n denotes the nth time step
which, the expected value of the production performance computed over of the reservoir simulation; Nt is the total number of time steps; t n is the
the ensemble of realizations is optimized (Yeten et al., 2002; Essen et al., simulation time at the end of the nth time step; Δt n is the nth time step
2011; Durlofsky et al., 2004; Liu and Reynolds, 2014; Lu et al., 2017). size; NP and NI denote the number of producers and injectors; ron is the oil
However, the computational cost for evaluation of the ensemble of a revenue ($/STB); cnw is the water disposal cost ($/STB); cng is the gas
large number may be prohibitive. Hence, only a small ensemble size is disposal cost ($/Mscf); cnwi and cngi are the water injection cost ($/STB) and
usually used in robust optimization, which may not be able to properly
gas injection cost ($/Mscf), respectively; qno;j , qnw;j , qng;j , respectively,
capture the subsurface uncertainty. Thus, it is significant to develop
denote the average oil production rate (STB/day), the average water
strategies to improve the computational cost of the robust optimization
production rate (STB/day) and the average gas production rate (Mscf/
with larger ensemble size. Yang et al. (2011) proposed to carry out robust
day) at the jth producer over the nth simulation time step; qnwi;k and qngi;k
optimization on a small subset of representative realizations which is
selected based on a reliable ranking of the calculated performance of all denote the average water injection rate (STB/day) and the average gas
the realizations under a reference operating condition. However, the injection rate (Mscf/day) over the kth injector for the nth time step; and b
ranking of reservoir performance would change if the well operational is the annual discount rate. Note, in this work, we only focus on the oil-
controls are updated in the optimization process and the more is the water two phase waterflooding process in which no gas is injected.
performed iterations, the less likely that the chosen subset remains To take the geological uncertainty into account, the robust optimi-
representable of the full ensemble. Li et al. (2013) evaluated the objective zation of the expected NPV over an ensemble of plausible geostatistical
function with a subset of randomly selected realizations at each iteration realizations is considered. The robust optimization problem is defined by
of SPSA optimization algorithm in order to achieve the desired compu-
tational efficiency. However, this strategy results in an oscillatory 1 XNe
max JE ðxÞ ¼ Jðx; mk Þ (2a)
behaviour of the optimization objective function at the later SPSA iter- N
x2ℝ x Ne k¼1
ations, there is a potential risk of premature termination of the algorithm
specially when the geological uncertainty is large. Shirangi et al. (2015) subject to xlow
i  xi  xup
i ; i ¼ 1; 2; …; Nx ; (2b)
proposed to re-select the representative realizations based on the solu-
tions obtained with a previous complete subset optimization, and repeat where, Ne denotes the size of the full-set ensemble of representative re-
the subset optimization until a pre-specified relative improvement ratio alizations; mk , k ¼ 1; 2; …; Ne represents the kth reservoir model
is reached. However solving a whole optimization problem for each parameter vectors; and xilow and xiup represent the lower and upper bound
subset optimization is expensive, and also the obtained solution is not for the ith component of the control variable vector, respectively.
guaranteed to be the optimal solution of the full-set. For better scaling of the optimization problem, the optimization
In this paper, we focus on developing an efficient algorithm for robust variable vector x in domain ½xlow ; xup  is transformed to the set of vari-
optimization using StoSAG when a larger number of representative re- ables u in domain [0,1] according to Eq. (3). For evaluating the NPV, u is
alizations are considered. The algorithm proposed in this work adap- transformed back to x using Eq. (4).
tively selects the representative subset, and flexibly switches between the
full-set and subset optimizations when needed. Here, the number and xi  xlow
i
length of control steps are prescribed and fixed and the well operating ui ¼ (3)
xup
i  xi
low
conditions (i.e., bottomhole pressure of producers and rate of injection
wells) are the well control variables. The robust optimization algorithm is  
xi ¼ ui xup low
i  xi þ xlow
i (4)
solved using a variant of the steepest ascent algorithm using Stochastic
Simplex Approximate Gradient, StoSAG. The outline of the paper is as A commercial reservoir simulator is used in order to simulate the
follows: first, the details of the considered robust optimization problem,

315
R. Lu et al. Journal of Petroleum Science and Engineering 159 (2017) 314–330

dynamic performance of the flooding process. The NPV function defined

  T
ΔJðu; mk Þ ¼ δJðu1;k ; mk Þ; δJðu2;k ; mk Þ; …; δJ uNp ;k ; mk ; (9)
in Eq. (1) is computed using the output of the simulation run given a Nx
dimensional vector of the control variables x and a Nm dimensional where, δJðuj;k ; mk Þ ¼ Jðuj;k ; mk Þ  Jðu; mk Þ, and δuj;k ¼ uj;k  u is the jth
vector of the reservoir model parameters m. The expected NPV function random perturbation vector at the current estimate of u and is ob-
defined in Eq. (2) is computed using the output of the Ne reservoir tained by
simulation runs given x and Ne model parameter vec-
tors mk ; k ¼ 1; 2; ⋯; Ne . 1=2
δuj ¼ LZ j ¼ CU Z j : (10)

2.2. StoSAG search direction formulation Here, Z j is a Nu dimensional vector of random normal deviates and L
is the lower triangular matrix of the Cholesky decomposition of the
In this section, a summary of the procedure for calculating the covariance matrix CU which controls the perturbation size and correla-
approximate simplex gradient with respect to the transformed variable tions between the components of the perturbed vector of variables. Here,
set u is presented. From Eq. (2), an approximation of the gradient of JE we follow the discussion presented in Chen et al. (2015) to define CU as a
can be obtained by averaging the gradients of individual models over block diagonal matrix which is composed of Nwell diagonal sub-matrices,
the ensemble. CwU ; w ¼ 1; 2; ⋯; Nwell , where components of CwU are computed using a
spherical covariance function given by
1 X Ne
"   3 #
∇u JE ðuÞ ¼ ∇u Jðu; mk Þ (5) 3 ji  jj 1 ji  jj
Ne k¼1 w
CU;i;j ¼ σ2 1  þ ; (11)
2 Ns 2 Ns
In this work, we adopt the steepest ascent optimization method and in
order to impose a temporal smoothness on the well controls, we use a where σ is the standard deviation controlling the perturbation size; i and j
smoothed stochastic approximate gradient of the expected NPV, JE , denote the indices of the control steps; w denotes the well index; and Ns is
given by the number of correlated control steps. In this work, the control variable
uj;k is truncated to its nearest bounds once it goes beyond the bounds.
CU XNe
Hence, it is likely that some components of δuj;k would be set to 0 when a
d¼ dk ; (6)
Ne k¼1 positive perturbation is imposed to a component of the control's vector at
its upper bound or a negative perturbation is imposed to a component of
where CU is the temporal covariance matrix, and dk is the simplex the control's vector at its lower bound. In order to avoid taking 0 per-
gradient approximation of Jðu; mk Þ (Conn et al., 2009; Do and Reynolds, turbations, we take δuj;k;i ¼ δuj;k;i when δuj;k;i þ ui > 1 or δuj;k;i þ ui < 0.
2013) given by Note that the StoSAG algorithm doesn't require δuj;k to be samples from a
 þ Gaussian distribution, hence, this treatment is rigorous. This treatment is
dk ¼ ΔU k ΔU Tk ΔU k ΔJðu; mk Þ; (7) important since, Zandvliet et al. (2007) pointed out, if the only con-
straints for the optimal control problems of the water flooding reservoirs
where, Np denotes the total number of perturbations for computing the
are bound constraints, for various situations the optimal solution is
simplex gradient, and ΔU k and ΔJðu; mk Þ are defined by
sometimes either bang-bang or a bang-bang solution exists which is only
  slightly suboptimal. Thus, it is highly possible that many well might
ΔU k ¼ δu1;k ; δu2;k ; …; δuNp ;k ; (8)
operate at the bound value, especially at the later stage of the optimi-
zation process.
and
The optimization iterations follow the steepest ascent scheme com-
bined with the backtracking line search, shown in Algorithm 1.

Algorithm 1. Pseudo-code for steepset ascent using StoSAG

Initialize the parameters including the initial step size α0 , the maximum number of step size cuts nsc , the maximum number of the consecutive
search direction re-computation nss , the maximum number of simulation runs ns , and the number of perturbations for gradient computations Np .
Set the iteration index ℓ ¼ 0, the number of simulation runs nsim ¼ 0, the number of step size cuts ncuts ¼ 0, the number of search direction
computations is ¼ 0, and the step size αℓ ¼ α0 .
Do While (nsim ⩽ns )

 1. Compute the search direction with Eq. (6). Set is ¼ is þ 1.

ℓ
 2. Compute a trial update utrial;ℓþ1 ¼ uℓ þ αℓ kddℓ k.
 3. Check if the trial update is acceptable:
– If ðJðutrial;ℓþ1 Þ > Jðuℓ ÞÞ then
* uℓþ1 ¼ utrial;ℓþ1 ; ℓ ¼ ℓ þ 1; is ¼ 0; ncuts ¼ 0; αℓ ¼ α0 .
– else
* if ncuts  nsc , set αℓ ¼ αℓ =2, then goto step 2,
* if ncuts > nsc and is < nss , then goto step 1,
* if ncuts > nsc and is > nss , terminate.
– Endif
 4. Check convergence criteria defined in Eqs. (12) and (13). Terminate if both equations are satisfied.

Enddo

316
R. Lu et al. Journal of Petroleum Science and Engineering 159 (2017) 314–330

The optimization algorithm terminates if one of the following con- variables, we would like to keep the quality of the approximated gradient
ditions hold: by using a larger Np in the adaptive algorithm.
Denote the full-set ensemble of models by
 If
Ωf ¼ fmk jk ¼ 1; 2; ⋯; Ne g
jOðuℓþ1 Þ  Oðuℓ Þj
τu ¼ ⩽104 ; (12)
maxfjOðuℓ Þj; 1:0g and the subset ensemble of models by

 
and,
kuℓþ1  uℓ k2
τf ¼
 ⩽103 ; (13) examples presented in this work, we consider and Nes ¼ 5. We select the
max kuℓ k2 ; 1:0
 or if maximum number of simulation runs are reached,
 or if consecutive failures in improving the objective function within a
specified maximum number of optimization iterations are attained.
3. Adaptive robust optimization algorithm
The computational cost for the robust optimization is much higher
compared to the deterministic optimization since each objective function
evaluation requires Ne simulations. The proposed algorithm focuses on
how to adaptively choose a smaller subset of representative models to
lower the computational cost of the robust optimization.
We assume the realizations of reservoir models in the full-set could be
represented by a subset of realizations in the neighbourhood of the
current estimate of the control variables. Following this assumption, the
adaptive robust optimization is designed to (1) adaptively select a
representative subset based on model's ranking, (2) use more perturba-
tions per realization in StoSAG search direction computation in order to
improve the quality of the search direction obtained with the subset, and
(3) avoid evaluating NPV for each realization in the full-set at all opti-
mization iterations. In the proposed adaptive algorithm, an update is
initially estimated using the subset ensemble, and then is tested for the
whole ensemble models. Moreover, the enforced convergence criteria are
defined based on the full-set optimization. The adaptive optimization
process is able to switch between the subset and full-set optimization
if the subset is not adequate to represent the full-set. In this way,
many simulation runs are saved by avoiding evaluating the NPVs for the
full-set ensemble at every optimization iteration. One should pay atten-
tion to the underlying assumption that the full-set cdf should be repre-
sentable by a small subset. If the full-set NPV distribution is skewed or the
number of realizations in the subset is small, one may obtain no signifi-
cant gain in the full-set expected NPV by optimizing over the subset.
Typically, the curve of the optimization objective function versus the
iteration numbers consists of two parts, including a steeper part where
the objective function is most sensitive to the optimization parameters,
and a flatter part where the objective function is less sensitive to the
changes of the optimization variables. Numerical results based on the
Rosenbrook function and the Egg reservoir model presented by Fonseca
et al. (2015) showed that a good stochastic approximate of the gradient
could be obtained for the steeper part of the optimization curve with one
perturbation per realization (Np ¼ 1) when multiple realizations are
considered. However, for the flatter part of the optimization curve, Np
should be larger in order to obtain a good gradient approximation.
Although most previous work simply use Np ¼ 1 in the robust optimi-
zation, Fonseca et al. (2015) showed that in robust optimization a better
approximation of the gradient could be obtained by either increasing the
number of perturbations Np or the ensemble size Ne . By increasing the
control sample size Np , the quality of approximate gradient for each
model is improved, whereas, an increase in the ensemble size Ne , helps
with mitigating the roughness of the robust objective function. Assuming
the Ne realizations could be represented by a smaller ensemble of Nes
models in the neighbourhood of the current estimate of the control
317
Ωs ¼ mi mi 2 Ωf ; i ¼ 1; 2; ⋯; Nes ; Ne ¼ 50
where Nes is the number of realizations in the subset. In all computational
representative models in the subset as the models corresponding to the
equally spaced percentiles of the cumulative distribution function (cdf) of
the full-set ensemble NPV estimated at the current iterate. Suppose Ne ¼
50 and Nes ¼ 5, then Ωs includes the realizations corresponding to the
P10, P30, P50, P70, P90 (10th, 30th, 50th, 70th and 90th percentiles) of
the NPV cdf evaluated over the 50 realizations.
Based on the selected subset, the robust optimization is carried out
using Np perturbations per realization, i.e. Np ¼ 4 until a point at which
the subset is not a good representative of the full-set anymore.
If the well controls u obtained from the subset optimization could
increase the expected NPV for the subset JEΩs but not that for the full-set
Ω
JE f , it implies that the realizations in the subset Ωs are not enough or are
not selected appropriately to represent Ωf . In such cases, the subset is re-
constructed by selecting and/or adding new models from the full-set.
Consistently, if the subset size is increased, the Np is reduced in order
to keep the computational cost low. When the subset includes all the
models in the full-set, the Np is reduced to 1. Once an iterate that im-
proves the expected NPV of the full-set is found, the cdf of the NPV is
constructed at the current iterate and a new subset is formed, that is to
say, the full-set optimization is switched back to the subset optimization.
In this way, the subset representative models are selected dynamically
and adaptively to ensure the reliability of the subset, and reduce the
computational cost of the robust optimization over the full-set ensemble.
The adaptive robust optimization algorithm is summarized in
Algorithm 2. Here, v denotes the vector of optimization variables within
the subset optimization iterations and u denotes the solution vector of
optimization variables which has been validated over the full-set.
Below, we provide three different “conditions” that the user may
specify in Algorithm 2 for determining when to re-evaluate the cdf of
NPV and update the subset. Also, in all the examples presented in this
Ω0 Ω0
work we take Ne ¼ 50; Ness ¼ 5; Np s ¼ 4.
 Condition 1: The cdf of the full-set ensemble Ωf is updated by running
all the models when the subset optimization iterations is larger than a
pre-specified number Niter , i.e. Niter ¼ 3, or the subset optimization
failed to obtain an uphill search direction with a maximum number of
re-sampling, i.e., nΩ
ss ¼ 3 is reached.
s
Once condition 1 is satisfied, the full-set ensemble will be simulated.
If the subset solution vℓþ1 failed to improve the expected NPV for the full-
set, the algorithm switches to full-set optimization where Ωs is enlarged
to include all models in Ωf and Np is reduced to 1. This step is marked
with (    ) in the algorithm.
 Condition 2: The cdf of the full-set ensemble Ωf is updated when
the order of the subset model NPVs change, or the standard deviation
of the subset NPVs goes beyond a pre-specified range
(β1 σ Ω Ωs Ωs
0 < σ ℓ < β2 σ 0 ), or the number of simulation runs within the
s
subset optimization exceeds a pre-specified upper limit for the kth cdf
iteration, δNsim;k . The last criteria in condition 2 is a looser version of
condition 1. In this work we use, β1 ¼ 0:85 and β2 ¼ 1:15.
Similar to the one in condition 1, if condition 2 is satisfied and the
R. Lu et al. Journal of Petroleum Science and Engineering 159 (2017) 314–330

Algorithm 2. Pseudo-code for adaptive robust optimization

Ω0 Ω0
Initialize the size of the initial subset, Ness , and number of perturbations per realization for the initial subset Np s . Note that the superscripts
“Ωs ” and “Ωf ” represent the variables defined for the subset and the full-set, respectively. The superscript “0” represents the variables defined
for the initial subset chosen at current NPV cdf.
Evaluate the NPV for all realizations in Ωf .
Ω0 Ω0 Ω0
Select Ness representative models based on the cdf of initial NPVs and form the subset Ω0s . Set Nes ¼ Ness , Ωs ¼ Ω0s , Np ¼ Np s , k ¼ 0, ℓ ¼ 0,
uk ¼ u0 , vℓ ¼ u0 .
Do While (nsim ⩽ns )

 1. Calculate StoSAG search direction for the expected NPV over all realizations in Ωs . The value of Np depends on the ensemble size Nes .
ℓ
 2. Find the proper step size αℓ with a backtracking line search and take a trial update for the Nes realizations in Ωs , vtrial;ℓþ1 ¼ vℓ þ αℓ jjddℓ jj.
 3. Check if the objective function of subset is increased.
– If ðJEΩs ðvtrial;ℓþ1 Þ > JEΩs ðvℓ ÞÞ
* Set vℓþ1 ¼ vtrial;ℓþ1 ; ℓ ¼ ℓ þ 1; is ¼ 0; ncuts ¼ 0; αℓ ¼ α0 .
* If (Nes ¼ Ne ), set ukþ1 ¼ vℓ , k ¼ k þ 1.
– Else
* If (Nes < Ne and is < nΩ ss ), then set is ¼ is þ 1,
s

* If (Nes < Ne and is ¼ nΩ ss ), then set Ωs ¼ Ωf , Nes ¼ Ne ; is ¼ 0,

s

* If (Nes ¼ Ne and is < nss ), then set is ¼ is þ 1,

* If (Nes ¼ Ne and is ¼ nss ), then terminate.
– Endif
– If (Nes < Ne ), check the conditions for re-evaluating the cdf.
* If the prescribed Conditions are satisfied, then
 Run the full-set ensemble with vℓ .
 If (JE ðvℓ Þ > JE ðuk Þ) then
Set ukþ1 ¼ vℓ ; k ¼ k þ 1; choose Nes 0
representative models to form subset Ω0s based on current cdf of NPV; set Ωs ¼ Ω0s , Nes ¼ Nes0
,
Ω0
Np ¼ Np s , ℓ ¼ 0. Goto step 2 to switch to subset optimization.
 Else
Enlarge the subset Ωs and adjust value of Np . (    )
 Endif
* Endif
– Else
*Terminate the algorithm if the convergence criteria for the full-set ensemble are met.
– Endif

Enddo

subset solution vℓþ1 failed to improve the expected NPV for the full-set,
the algorithm switches to full-set optimization in the step marked
with (    ).
 Condition 3: Different from the first two Conditions, in this case the
Ω
adaptive algorithm evaluates the expected NPV for the full-set, JE f , at
each iteration of the subset optimization if the expected NPV for the
subset, JEΩs , is improved.
If the expected NPV for the subset JEΩs is improved and the updated
Ω optimal well control problem. Three synthetic reservoir examples are
control failed to improve the expected NPV for the full-set, JE f , we
gradually increases Nes and adjust Np accordingly. In this work, we use
three realizations and perturbations pairs, for this condition.
– Nes ¼ 5; Np ¼ 4,
– Nes ¼ 10; Np ¼ 2,
– Nes ¼ 50; Np ¼ 1.
As an example, lets consider that the initial subset Ωs includes Nes ¼ 5
realizations corresponding to the 10%; 30%; 50%; 70%; 90% percen-
tiles of the current cdf and Np ¼ 4 is used for the gradient calculation. If
Ω alizations are selected from Ne ¼ 50 realizations, and Np s ¼ 4 pertur-
the solution of this subset optimization fails to improve JE f , then another
Nes0 ¼ 5 realizations which correspond to the 20%; 40%; 60%; 80%; 100%
percentiles are added to Ωs . Thus Nes ¼ 10 and we set Np ¼ 2 to keep the
computational cost low. If the subset optimization fails again, then full
ensemble Ωf is used and Np is reduced to 1.
3.1. Numerical examples
In this section, we implemented the proposed adaptive robust opti-
mization procedure, with all three mentioned conditions, to solve the
considered which include, a 2D channelized reservoir model built with
object based modelling, a 2D reservoir model with Gaussian property
fields, and the 3D Brugge model. All three examples pertain to the water
flooding process optimization. For NPV computations, we consider the
oil revenue is $80:0/STB, water disposal cost is $5:0/STB, water injection
cost is $5:0/STB and the annual discount rate is 10:0%. The results from
the adaptive procedure with different conditions are compared with the
full-set robust optimization to show their superiority.
The numerical parameters for the robust 0optimizations are set as
Ω
following. To form the subset, initially Ness ¼ 5 representative re-
Ω0
bations per realization are used to compute the search direction. For the

318
R. Lu et al. Journal of Petroleum Science and Engineering 159 (2017) 314–330

Table 1
Hard data used for modelling the channelized reservoir.

P1 P2 P3 P4 P5 P6 P7 P8 P9 I1 I2 I3 I4

shale sand shale sand sand sand sand sand sand sand shale shale sand

Table 2 3.2. Example 1: channelized reservoir model

Initial values for the control settings and their bounds for the channelized reservoir.

Well-type Control-type Variables The first example is a a synthetic two-dimensional oil-water reservoir
Initial Guess Lower Bound Upper Bound with a uniform grid system of 50  50  1 grids. Each grid has the size
50 ft  50 ft  20 ft. The reservoir has three facies including high-
Producers BHP (psi) 3000 2000 3800
Injectors BHP (psi) 4500 3800 6000
permeable sand channels (1500 mD), low permeable shale background
(10 mD) and intermediate-permeable leeve (500 mD). The permeability
field is isotropic. The porosity is presumed to be homogeneous and set
adaptive procedure with conditions 1 & 2, a maximum of nΩ ss ¼ 3 sto-
s
equal to 0.2. 50 realizations of the facies field are generated with the
chastic gradient re-computations for the subset search direction is “facies modelling module” in Petrel in order to represent the geological
allowed. However, for the adaptive procedure with condition 3, no uncertainty. There are nine producers and four injectors in the reservoir.
subset gradient re-computation is allowed, i.e., nΩ ss ¼ 0, since the full-set
s
The hard data at each well location are used in the geo-statistical
NPVs are evaluated after each subset iteration. Moreover, for the full-set modelling of the field and are shown in Table 1. The reservoir life is
optimization, Np ¼ 1 is used when calculating the search direction. 180 days, which is divided into 6 equal control steps with each control
The perturbation size for the transformed variables u is set to 0.03
which corresponds to 3 percentage of the search domain. The initial step
size is set to α0k ¼ 0:2 at each iteration of the steepest ascent update, and a
Table 3
maximum number of nsc ¼ 5 step size cuts is allowed where at each cut
Optimal and average NPVs obtained with full-set optimization and the adaptive schemes
the old step size is multiplied by 0.5. The convergence of the algorithm is using three different seeds for the channelized reservoir.
determined based on the full-set optimization according to the conver-
Seeds Optimal NPVs, $106
gence criteria given in Eqs. (12) and (13). Moreover, the algorithm is
terminated whenever nss ¼ 3 successive unsuccessful search direction Adaptive optimization Full-set optimization (4000
simulations)
computations occur at the current estimate, or a maximum number of Condition Condition Condition
simulations ns is reached. The maximum affordable number of simula- 1 2 3
tions runs to perform the optimizations are, ns is set equal to 4000 for the Initial 64.436 64.436 64.436 64.436
channelized example, ns is set equal to 5000 for the Gaussian model case, Seed1 87.409 86.482 86.477 83.428
and ns is set to 3000 for the Brugge case. Seed2 86.305 86.709 85.721 83.955
Seed3 86.878 88.174 86.056 82.490
Average 86.864 87.122 86.084 83.291

Fig. 1. Representative subset realizations of the facies field corresponding to the 10th, 30th, 50th, 70th and 90th percentiles of the initial NPV cdf for the channelized reservoir.

319
R. Lu et al.
Fig. 2. Comparison of the NPV changes for the channelized reservoir obtained with full-set optimization and the adaptive schemes.
Fig. 3. Initial and optimal cdfs of NPV for the channelized reservoir obtained with full-set
optimization using seed 1; the red, green, cyan, blue and black stars represent realizations
corresponding to the 1st-20th, 21th-40th, 41st-60th, 61st-80th and 81st-100th percentiles
of the initial cdf; the red, green, cyan, blue and black circles represent the realizations
corresponding to the 10th, 30th, 50th, 70th, 90th percentiles of the initial cdf. (For
interpretation of the references to colour in this figure legend, the reader is referred to the
web version of this article.)
step length equal to 30 days. During the reservoir lifetime, all 13 wells
operate under a specified bottomhole pressure control, where the initial
values for the control settings and their bounds are shown in Table 2. To
honour the lifting and surface facility capacity of each well, a secondary
constraint on a maximum liquid production rate of 5000 bbl/day is
enforced on producers using the simulator.
The cdf of NPV of the full ensemble constructed with the initial well
settings, and the realizations of the facies field corresponding to the 10th,
30th, 50th, 70th and 90th percentiles of the initial NPV cdf are shown
in Fig. 1.
Fig. 4. Initial and optimal cdfs for the channelized reservoir obtained with the adaptive schemes with seed 1; the red, green, cyan, blue and black circles represent the realizations
corresponding to the 10th, 30th, 50th, 70th, 90th percentiles of the initial cdf. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version
of this article.)
320
Journal of Petroleum Science and Engineering 159 (2017) 314–330
The proposed adaptive optimization approach following Algorithm 2
with three presented conditions for keeping the consistency of the subset
with the full-set, are compared to the robust optimization based on the
full-set following Algorithm 1. Due to the stochastic characteristic of the
StoSAG algorithm, each optimization is repeated with three different
initial seeds for the random number generator in the algorithm. The
optimal NPV for each optimization and the average of optimal NPVs
obtained with different seeds are presented in Table 3. The full-set
optimization, as the reference, is terminated either at the convergence
or after 10,000 reservoir simulations. As we could observe from Fig. 2(a),
the full-set optimizations could not converge after 10,000 simulation
runs. From Table 3 and Fig. 2(b), it is observed that the proposed pro-
cedures not only improved the convergence rate, but also obtained
higher average optimal NPVs when the optimization is terminated at
4000 simulation runs. Note in Fig. 2 that each marker point on the plot
shows one full-set NPV evaluation. From Fig. 2(b) we see that although
almost same expected NPVs were obtained with the three presented
conditions, the convergence speed of the procedure with condition 3,
where the NPV cdf is updated at each optimization iteration of the subset,
is slightly slower. Although the convergence speed of conditions 1 and 2
were similar in this case, condition 1 required more full-set NPV evalu-
ations which makes it less efficient if a larger ensemble size is used. Thus,
based on this example results, we recommend the combination of the
adaptive procedure with condition 2.
Figs. 3 and 4, respectively, show the initial and optimal cdfs obtained
with the full-set optimization and the adaptive schemes using seed 1. The
NPVs obtained with the 5 representative models in the subset selected
based on the initial NPV cdf are shown on each cdf plot. Although the
shapes of the initial and optimal cdfs look similar, the representative
models selected based on the initial cdf are no longer representative for
the optimal cdf and a biased expected NPV would be obtained if the full-
set optimization is replaced by optimization with subset chosen on the
R. Lu et al. Journal of Petroleum Science and Engineering 159 (2017) 314–330

Fig. 5. Comparison of the expected NPV over the subset (shown with circles) and full-set expected NPV (shown with cross markers) for the channelized reservoir obtained with the
adaptive schemes.

Fig. 6. Relative uncertainty of NPVs of the full-set ensemble for the channelized reservoir.
initial NPV cdf. This indicates that it is necessary to update the subset
models frequently, or at least after a few iterations of the subset opti-
mization. Fig. 5 shows the comparison of the average NPV over the
subset, shown with the circles, and the full-set average NPV, shown with
a solid line with cross markers, obtained with seed 1. From this figure, the
expected NPV of the full-set is well approximated by that of the subset at
each cdf re-construction and gradually exceeds it after a few iterations of
subset updates. From Fig. 5, the adaptive procedure with different con-
ditions prevented the situation that the update in control variables lead
to a great increase in the expected NPV for the subset and negligible
increase for the full-set.
Fig. 7 shows the optimized well settings obtained with seed 1 at either
convergence or when the maximum simulation runs of 4000 is reached.
Though there is not big difference in the obtained expected NPV, the
optimal controls differ a lot. For the proposed adaptive optimizations,

Fig. 7. The optimal controls for the channelized reservoir obtained with the full-set optimization and the adaptive schemes using seed 1; the upper row corresponds to the bottomhole
pressure of producers and the lower row corresponds to the bottomhole pressure of injectors.

321
R. Lu et al. Journal of Petroleum Science and Engineering 159 (2017) 314–330

Fig. 8. Representative subset realizations of the log permeability field corresponding to the 10th, 30th, 50th, 70th and 90th percentiles of the initial NPV cdf for the Gaussian model.

most of the control values are either at upper or lower bounds. Table 4
Optimal and average NPVs obtained with the full-set optimization and the adaptive
schemes using three different seeds for the Gaussian model.
3.3. Example 2: reservoir with Gaussian property fields
Seeds Optimal NPVs, $106

Adaptive optimization Full-set optimization (5000

The proposed algorithm is also tested with a reservoir model with
simulations)
Gaussian permeability field. Multiple realizations of the log of horizontal Condition Condition Condition
permeability fields are generated from a spherical anisotropic variogram 1 2 3

model with a major correlation length of 2000 ft (corresponding to the Initial 80.341 80.341 80.341 80.341
width of 40 gridblocks) and a minor correlation length of 800 ft (i.e., 16 Seed1 97.769 97.166 97.186 95.503
Seed2 97.367 96.691 97.056 96.031
gridblocks). The direction of maximum continuity is along the 45
Seed3 97.691 97.771 96.299 96.089
measured from the positive x-axis. The mean of the log-permeability field Average 97.609 97.209 96.847 95.874
is 5.0 and the standard deviation of the log-permeability is set equal to
1.5. 50 realizations of the horizontal permeability fields are generated.
The well locations, control settings and reservoir life are kept the same as ensemble evaluations at each subset optimization iteration.
those specified in the previous example, except that there is no maximum By taking a closer look at the initial and optimal cdfs obtained with
liquid production rate constraints. The five realizations of the reservoir the full-set optimization in Fig. 10, the optimization results have a larger
model corresponding to the 10th, 30th, 50th, 70th and 90th percentiles variance in the optimal cdfs of NPV. This is reasonable since the StoSAG
of the initial NPV are shown in Fig. 8. algorithm is formulated to maximize the expected NPV and there is no
The adaptive optimization results using three proposed conditions guarantee that the NPV for each realization would be maximized. The
under which we re-evaluate the cdf of NPV and update the subset, are cdfs obtained with the adaptive procedure using different conditions are
compared with the full-set optimization results for three random seeds. also skewed with longer tails, similar to the cdfs obtained with full-set
The obtained optimal NPVs for optimizations with each seed and the optimization, see Fig. 11. Since the long tail of the optimal cdf mainly
average optimal NPV are presented in Table 4. Since the expected NPV results from four models corresponding to the 2nd, 4th, 6th and 8th
for the full-set optimization obtained with seed 2 is closest to the average percentiles of the optimal NPV cdf, the expected NPV for the full-set
value among all three seeds, results obtained with seed 2 is used to would be overestimated using subset realizations corresponding to the
represent the full-set robust optimization performance. The full-set 10th, 30th, 50th, 70th and 90th percentiles, see Fig. 12. This implies that
optimization is terminated either at convergence or at 10,000 simula- the average NPV of the selected subset would be biased in terms of the
tion runs. The performance of full-set optimizations are shown in representativeness of the full-set average NPV. This is consistent with
Fig. 9(a). A comparison of the adaptive and full-set robust optimization what is shown in Fig. 9(b) that there were more cdfs evaluations and also
using seed 2 is shown in Fig. 9(b). Table 4 and Fig. 9, show that the more frequent switchings between subset and full-set optimizations,
adaptive optimization not only improved the convergence rate, but also compared to example 1.
obtained higher average optimal NPVs when the optimization is termi- To alleviate the skewness of the cdfs and the risk of obtaining very
nated at a maximum simulation number of 5000. However, the conver- low NPV values, one way is to carry out a secondary optimization within
gence speed using condition 3 is the slowest due to the frequent full a lexicographic approach in order to maximize the minimum NPV subject

322
R. Lu et al. Journal of Petroleum Science and Engineering 159 (2017) 314–330

Fig. 9. Comparison of the NPV changes obtained with full-set optimization and the adaptive schemes for the Gaussian model.

Fig. 10. Initial and optimal cdfs of NPV for the Gaussian model obtained with full-set optimization using seed 2; the red, green, cyan, blue and black stars represent realizations cor-
responding to the 1st-20th, 21th-40th, 41st-60th, 61st-80th and 81st-100th percentiles of the initial cdf; the red, green, cyan, blue and black circles represent the realizations corresponding
to the 10th, 30th, 50th, 70th, 90th percentiles of the initial cdf. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 11. Initial and optimal cdfs for the Gaussian model obtained with the adaptive schemes using seed 2; the red, green, cyan, blue and black circles represent the realizations corre-
sponding to the 10th, 30th, 50th, 70th, 90th percentiles of the initial cdf. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of
this article.)

to the constraint that the expected NPV would not be greatly decreased,
see Lu et al. (2017). Another way is to carry out robust optimization with
a weighted sum method by placing a larger weight on the model with
minimum NPV. One may argue that the proposed adaptive procedure
would yield lower minimum NPVs compared to the full-set optimization
since the worst realization is not included in the representative subset.
However, since the adaptive procedure switches between the subset
optimization and the full-set optimization, the risk is comparable to the
full-set optimization, see Figs. 11 and 17.
Fig. 13 shows the optimal well control settings obtained with seed 2 at
either convergence or the maximum simulation runs of 5000. The results
recommend that producers operate at their lower bounds and injector
“I4” operates at higher bottomhole pressure in the beginning of the
reservoir life.
To generate a better acceptable solution which has a lower risk, a
different choice of the representative subset (corresponding to the 2nd,
25th, 50th, 75th and 100th percentile of the current cdf) for the adaptive
optimization is tried. By including the worst realization, the represen-
tative subset tends to underestimate the expected NPV of the full-set
when the cdf has a long tail, as shown in Fig. 16.
By including the two end points of a skewed cdf, the realization with
the minimum NPV is assigned a larger weight in the subset expected NPV
compared to the expected NPV over the full-set, which in nature is similar
to the bi-objective weighted sum method. Due to the poor performance of
the adaptive procedure with condition 3, only the procedures using
condition 1 and condition 2 are tested and compared with the full-set
optimization. Comparing Figs. 11 and 15, the new selection of the
representative models yields higher minimum NPVs for both adaptive

323
R. Lu et al.
Fig. 12. Comparison of the expected NPV for the subset and the full-set average NPV for the Gaussian model obtained with the adaptive schemes; the black squares represent switches from
subset optimization to full-set optimization.
Fig. 13. The optimal controls for the Gaussian model obtained with the full-set optimization and the adaptive schemes using seed 2; the upper row corresponds to the bottomhole pressure
of producers and the lower row corresponds to the bottomhole pressure of injectors.
Fig. 14. Comparison of the NPV changes obtained with full-set optimization and the
adaptive schemes using a new selection of the representative subset, for the
Gaussian model.
optimization runs. What is more, compared to the previous selection, the
new selection has lower standard deviations of NPVs for most iterations
of the optimization processes where the cdfs of NPVs are evaluated, (see
324
Journal of Petroleum Science and Engineering 159 (2017) 314–330
Fig. 17). However, as shown in Table 5, the optimal expected NPVs for
the full-set obtained with the new selection of the representative re-
alizations are smaller compared to the previous selection (see Fig. 14).
3.4. Example 3: brugge model
To benchmark current technology available for closed loop (real-
time) reservoir management, the Brugge field, was developed by TNO in
conjunction with the SPE Applied Technology Workshop held in Brugge
in June 2008. The stratigraphy of the Brugge field is modeled after a
typical North Sea Brent field and is an elongated half-dome with one
internal fault and one boundary fault. The top structure map with well
locations is shown in Fig. 18. The original model was constructed with
approximately 20 million gridblocks and then upscaled to a 450,000
gridblock model which formed the true case from which data were
generated by TNO. 104 geological realizations of the horizontal perme-
ability, vertical permeability, porosity, water oil contact and the net-
gross ratio are generated and upscaled to a 60,000 grid cell model to
describe the reservoir uncertainty. However in this work, due to our
limited computational resources, 50 realizations are selected for the full
R. Lu et al. Journal of Petroleum Science and Engineering 159 (2017) 314–330

Fig. 15. Initial and optimal NPV cdfs of the Gaussian model obtained with the adaptive schemes using a new selection of the representative subset; the red, green, cyan, blue and black
circles represent the realizations corresponding to the 2nd, 25th, 50th, 75th, 100th percentiles of the initial cdf. (For interpretation of the references to colour in this figure legend, the
reader is referred to the web version of this article.)

Fig. 16. Comparison of the expected NPV over the subset and full-set average NPV using a new selection of the representative subset for the Gaussian model obtained with the adaptive
schemes; the black squares represents switches from subset optimization to full-set optimization; the representative models are selected as 2nd, 25th, 50th, 75th, 100th percentiles on cdf.

Fig. 17. Relative uncertainty of NPVs for the full-set ensemble of the Gaussian model.

ensemble and the optimization runs are terminated at a maximum

Table 5 number of 3000 simulations. The 50 realizations are selected such that
Optimal and average NPVs obtained with full-set optimization and the adaptive schemes there are 10 realizations from the group of models built with each of the 5
using three different seeds and a new selection of the representative subset, for the
geo-statistical methods.
Gaussian model.
The reservoir life is set to 3650 days and is divided into 20 equal
Seeds Optimal NPVs, $106 control steps. The well control type, initial value and bounds for the
Adaptive optimization Full-set optimization (5000 simulations) controls are shown in Table 6. The representative realizations of hori-
Condition 1 Condtion 2 zontal permeability and porosity in the subset chosen from the initial cdf
are shown in Fig. 19. Similar to the presented two examples, the opti-
Initial 80.341 80.341 80.341
Seed1 96.021 96.380 95.503
mization procedures are tested with three different seeds. The compari-
Seed2 96.458 96.859 96.031 sons of the optimal expected NPVs for the full-set optimization and the
Seed3 96.611 96.941 96.089 adaptive procedure with different conditions are shown in Table 7. Since
Average 96.363 96.727 95.874 the second seed of the full-set optimization has an expected NPV closest

325
R. Lu et al. Journal of Petroleum Science and Engineering 159 (2017) 314–330

Table 7
Optimal and average NPVs obtained with the full-set optimization and the adaptive
schemes using three different seeds for the Brugge model.

Seeds Optimal NPVs, $106

Adaptive optimization Full-set optimization (3000

simulations)
Condition Condition Condition
1 2 3

Initial 545.24 545.24 545.24 545.24

Seed1 1157.9 1157.4 1130.5 1075.8
Seed2 1162.9 1164.1 1150.0 1096.9
Seed3 1142.8 1138.5 1121.1 1123.2
Average 1154.5 1153.3 1133.9 1098.6

procedure with condition 2 has the lowest number of full-set evaluations

while the adaptive procedure with condition 3 has the lowest conver-
gence speed among all three conditions (see Fig. 20). Fig. 21 shows that
the expected NPV over the subset (shown with circles) almost coincide
Fig. 18. Top structure of the Brugge model. with the expected NPV over the full set (shown with cross markers) for
the adaptive schemes, which implies the selected subset is representative.
Table 6 However, the obtained improvement of the convergence speed for the
Initial values for the control settings and their bounds for the Brugge model. Brugge case is not as impressive as the channelized reservoir example.
Well-type Control-type Variables This may be due to the fact that if the impact of the geological uncertainty
Initial Guess Lower Bound Upper Bound is small, the full-set optimization procedure which uses Ne ¼ 50 re-
alizations and Np ¼ 1 perturbation per realization would provide a better
Producers BHP (psi) 1595.33 725.00 2465.65
Injectors Rate (bbl/day) 3773.90 0 6289.81
search direction than those obtained with the adaptive procedure which
use Ne ¼ 5 realizations and Np ¼ 4 perturbations per realization or Ne ¼
10 realizations and Np ¼ 2 perturbations per realization. In such cases,
to the average expected NPV, we choose the optimization curve obtained the improvement mainly comes from the saved simulation cost when
with seed 2 to make the convergence comparisons. Note that in this accepting the control updates rather than obtaining the search directions
example, the maximum number of simulations between each two cdf with better quality. Hence, the proposed procedures would be less ad-
constructions are set to δNsim;k ¼ 250. From Table 7 and Fig. 20, we could vantageous for ensembles with smaller uncertainty (i.e. Brugge case, see
see the proposed adaptive algorithm using three different conditions Fig. 22) compared to models with larger uncertainty in the cdf of NPV
could yield faster convergence and higher average NPVs. The adaptive (i.e. Channelized case, see Fig. 6). Fig. 23 shows the initial and optimal

Fig. 19. Representative subset realizations of the horizontal permeability distribution (upper row) and porosity distribution (lower row) corresponding to the 10th, 30th, 50th, 70th and
90th percentiles of the initial NPV cdf for the Brugge reservoir.

326
R. Lu et al.
Fig. 20. Comparison of the NPV changes for the Brugge model obtained with the full-set optimization and the adaptive schemes.
Fig. 21. Comparison of the expected NPV over the subset (shown with circles) and full-set expected NPV (shown with cross markers) for the Brugge reservoir obtained with the adap-
tive schemes.
Fig. 22. Relative uncertainty of NPVs of the full-set ensemble for the brugge model.
cdfs obtained with the full-set optimization using seed 2 where a large
change in the ranking of the realizations could be observed. Fig. 24 shows
the cdfs constructed in the adaptive procedure with three different
conditions.
To further illustrate how the proposed procedure works, the changes
of the NPV of the selected realizations are studied during the subset
optimization iterations between every two neighboring cdf construc-
tions. Here we take the adaptive optimization run using condition 2 and
with seed 2 as the case to show these results for two optimization in-
tervals, which are the 4th to 5th cdf construction interval (on the steeper
part of the optimization curve), and the 9th to 10th cdf construction
interval (on the flatter part of the optimization curve). From Fig. 26 (a),
the variance of the subset NPV changes gradually with respect to the
subset optimization iterations. After ℓ ¼ 7 iterations, the NPV for each
subset realization increased greatly, and so does the expected NPV over
the subset. Also, the standard deviation of the subset NPV σ ℓ becomes
greater than β2 σ 0 , where σ 0 is the standard deviation of the subset NPV
evaluated at the 4th cdf construction. Hence, Condition 2 is satisfied and
327
Journal of Petroleum Science and Engineering 159 (2017) 314–330
Fig. 23. Initial and optimal cdfs of NPV for the channelized reservoir obtained with full-
set optimization using seed 2; the red, green, cyan, blue and black stars represent re-
alizations corresponding to the 1st-20th, 21th-40th, 41st-60th, 61st-80th and 81st-100th
percentiles of the initial cdf; the red, green, cyan, blue and black circles represent the
realizations corresponding to the 10th, 30th, 50th, 70th, 90th percentiles of the initial cdf.
(For interpretation of the references to colour in this figure legend, the reader is referred to
the web version of this article.)
the 5th NPV cdf for the whole ensemble is constructed. As is observed
from Fig. 26 (a), the subset realizations are no longer representative of
the 5th cdf and a new subset needs to be constructed. Fig. 25 (a) and (e),
respectively, show the control settings for producers and injectors at the
4th cdf construction; Fig. 27 (a) and (b), respectively, show the
normalized search direction for producers and injectors controls at the
iteration corresponding to the 4th cdf evaluation and obtained with the
subset; and Fig. 25 (b) and (f), respectively, show the control settings for
producers and injectors at the iteration corresponding yo the 5th cdf
evaluation. As we could see, on the steeper part of the optimization
curve, not only the expected NPV is increased greatly by the subset
optimization iterations, but also control variables obtained big changes.
R. Lu et al. Journal of Petroleum Science and Engineering 159 (2017) 314–330

Fig. 24. Initial and optimal cdfs for the Brugge model obtained with the adaptive schemes with seed 2; the red, green, cyan, blue and black circles represent the realizations corresponding
to the 10th, 30th, 50th, 70th, 90th percentiles of the initial cdf. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 25. The control settings at the 4th, 5th, 9th and 10th cdf constructions for the Brugge model obtained with the adaptive schemes using seed 2 and condition 2; the upper row
corresponds to the bottomhole pressure for producers and the lower row correspond to the injection rate of the injectors.

However, from Fig. 26 (b), the NPV for each realization only has a slight
increase after all ℓ ¼ 7 subset iterations compared to the NPVs obtained
at the 9th cdf evaluation. The 10th NPV cdf for the whole ensemble is
evaluated when the interval simulation number is greater than a pre-
specified upper limit, δNsim ¼ 250 for this case. As is observed from 26
(b), the subset realizations still cover various percentiles of the 10th cdf
and remain representative of the full set. Fig. 25 (c) and (g), respectively,
show the control settings for producers and injectors at the iteration
corresponding to the 9th cdf construction; Fig. 28 (a) and (b), respec-
tively, show the normalized search direction for producers and injectors
at the iteration corresponding to the 9th cdf evaluation obtained with the
subset; and Fig. 25 (d), (h), respectively, show the control settings for the
producers and injectors at the iteration corresponding to the 10th cdf
evaluation. As we could see, on the flatter part of the optimization curve,
through the subset optimization the expected NPV is increased sightly
and the changes in control variables are small.
Since the perturbations of control estimates are rarely subject to
truncation at the iteration corresponding to the 4th cdf evaluation, the
smoothed search direction conforms to a Gaussian distribution which is
consistent with Fig. 27 (c). However, at the iteration corresponding to the
9th cdf evaluation, most of the control values are close to their bounds
which require more truncations, and the distribution of the smoothed
gradient is no longer Gaussian, shown in Fig. 28 (c).
The optimal controls obtained are shown in Fig. 29. Similar to the
channelized example, the controls for the full-set optimization are
different from those obtained with the adaptive procedure using three
different conditions where most of the control values are either at their
upper or lower bounds due to the larger number of performed iterations.
All injectors inject at their upper limit of the injection rate at almost all
control steps. This is reasonable since the well control solutions highly
depend on the reservoir problem including, the number of wells, and
their locations, as well as the considered reservoir life-cycle for optimi-
zation. If too few injectors are placed or the reservoir life is too short to
flood the whole reservoir, which seems to be the case for the Brugge
model in this paper, a better solution tends to inject as much as possible,
i.e., injectors operate close to their upper limit of the injection rate, and
produce as much as possible, i.e., producers operate at their lower limit of
the bottomhole pressure except for those which perforate too close to the
water-oil contact or injectors. Similar results for the robust optimization
of the Brugge model is reported in Chen (2017).

328
R. Lu et al. Journal of Petroleum Science and Engineering 159 (2017) 314–330

Fig. 26. The subset NPV changes and the full-set cdf of NPV changes from the 4th to the 5th cdf constructions, and from the 9th to the 10th cdf constructions; the red, green, cyan, blue and
black circles represent NPVs for the subset realizations; the black stars represent the NPVs for realizations in the full-set. (For interpretation of the references to colour in this figure legend,
the reader is referred to the web version of this article.)

Fig. 27. The normalized sensitivity matrix for the control settings (estimated at the 4th cdf iteration) for the Brugge model obtained with the adaptive schemes using seed 2 and condition
2; (a) corresponds to sensitivity matrix for the bottomhole pressure of producers; (b) corresponds to the sensitivity matrix of the injection rate of the injectors; and (c) corresponds to the
histogram of the normalized search direction.

Fig. 28. The normalized sensitivity matrix for the control settings (estimated at the 9th cdf iteration) for the Brugge model obtained with the adaptive schemes using seed 2 and condition
2; (a) corresponds to sensitivity matrix for the bottomhole pressure of producers; (b) corresponds to the sensitivity matrix of the injection rate of the injectors; and (c) corresponds to the
normalized histogram of the search direction.

4. Discussion and conclusions
To relieve the heavy computational cost and better describe the
geological uncertainty, we developed an efficient adaptive optimization
algorithm to include a larger number of representative realizations. The
robust optimization is solved with the steepest ascent algorithm using the
StoSAG search direction. The proposed procedures automatically switch
between the subset optimization and full-set optimization. Three
different conditions under which we re-evaluate the NPV cdf of the full
ensemble and update of the representative subset are proposed and
tested. The proposed procedures helps to increase the convergence speed
for the tested numerical examples, especially for the flatter part of the
optimization curves. However, the proposed procedures have a set of
parameters which need to be specified empirically.
In conclusion, based on the applications we performed in this work,
the following conclusions are warranted.
 The proposed adaptive procedure using three different conditions
under which we re-evaluate the full-set NPV cdf could improve the
convergence rate by finding control updates using a representative

329
R. Lu et al.
Fig. 29. The optimal controls for the Brugge model obtained with the full-set optimization and the adaptive schemes using seed 2; the upper row corresponds to the bottomhole pressure of
producers and the lower row corresponds to the injection rate of injectors.
subset. The proposed procedures could keep the quality of the StoSAG
search direction and avoid simulating the full-set realizations at each
objective function evaluation.
 The combination of the adaptive procedure with condition 2 is rec-
ommended due to the least evaluations of the full-set expected NPV
which makes it more efficient when a much larger ensemble is used.
Condition 3 is conservative and slow since the subset is considered
representative only at the current estimate.
 It is recommended to include the realizations located at the two end
points of the current cdf to avoid obtaining solutions with high risk.
References
Awotunde, A.A., et al., 2014. On the joint optimization of well placement and control. In:
SPE Saudi Arabia Section Technical Symposium and Exhibition. Society of Petroleum
Engineers.
Chen, B., 2017. A stochastic simplex approximate gradient for production optimization of
WAG and continuous water flooding. Ph.D. thesis. The University of Tulsa.
Chen, B., Reynolds, A.C., 2017. Optimal control of ICV's and well operating conditions for
the water-alternating-gas injection process. J. Petroleum Sci. Eng. 149, 623–640.
Chen, B., Reynolds, A.C., et al., 2015. Ensemble-based optimization of the water-
alternating-gas-injection process. SPE J. 21 (3), 786–798.
Conn, A.R., Scheinberg, K., Vicente, L.N., 2009. Introduction to Derivative-free
Optimization, vol. 8. Siam.
Do, S.T., Reynolds, A.C., 2013. Theoretical connections between optimization algorithms
based on an approximate gradient. Comput. Geosci. 17 (6), 959–973.
Durlofsky, L., Aitokhuehi, I., Artus, V., Yeten, B., Aziz, K., 2004. Optimization of advanced
well type and performance. In: ECMOR IX-9th European Conference on the
Mathematics of Oil Recovery.
Essen, G.M.V., den Hof, P.M.J.V., Jansen, J.D., 2011. Hierarchical long-term and short-
term production optimization. SPE J. 16 (1), 191–199.
Fonseca, R., Chen, B., Jansen, J.D., Reynolds, A., 2016. A stochastic simplex approximate
gradient (StoSAG) for optimization under uncertainty. Int. J. Numer. Methods Eng.
109 (13), 1756–1776.
Fonseca, R., Kahrobaei, S., Van Gastel, L., Leeuwenburgh, O., Jansen, J., et al., 2015.
Quantification of the impact of ensemble size on the quality of an ensemble gradient
using principles of hypothesis testing. In: SPE Reservoir Simulation Symposium.
Society of Petroleum Engineers.
Forouzanfar, F., Poquioma, W.E., Reynolds, A.C., et al., 2015. Simultaneous and
sequential estimation of optimal placement and controls of wells using a covariance
matrix adaptation algorithm. SPE J. 21 (02), 501–521.
330
Journal of Petroleum Science and Engineering 159 (2017) 314–330
Hanea, R., Casanova, P., Wilschut, F.H., Fonseca, R., et al., 2017. Well trajectory
optimization constrained to structural uncertainties. In: SPE Reservoir Simulation
Conference. Society of Petroleum Engineers.
Kraaijevanger, J., Egberts, P., Valstar, J., Buurman, H., et al., 2007. Optimal waterflood
design using the adjoint method. In: SPE Reservoir Simulation Symposium. Society of
Petroleum Engineers.
Li, L., Jafarpour, B., Mohammad-Khaninezhad, M.R., 2013. A simultaneous perturbation
stochastic approximation algorithm for coupled well placement and control
optimization under geologic uncertainty. Comput. Geosci. 17 (1), 167–188.
Lien, M.E., Brouwer, D.R., Mannseth, T., Jansen, J.-D., et al., 2008. Multiscale
regularization of flooding optimization for smart field management. SPE J. 13 (02),
195–204.
Liu, X., Reynolds, A.C., 2014. Gradient-based multi-objective optimization with
applications to waterflooding optimization. Comput. Geosci. 1–17.
Lu, R., Forouzanfar, F., Reynolds, A., et al., 2017. Bi-objective optimization of well
placement and controls using StoSAG. In: SPE Reservoir Simulation Conference.
Society of Petroleum Engineers.
Oliveira, D.F., Reynolds, A.C., et al., 2015. Hierarchical multiscale methods for life-cycle-
production optimization: a field case study. SPE J. 20 (05), 896–907.
Petvipusit, K.R., Chang, Y., et al., 2015. Dynamic well conversion and rate optimisation
using ensemble-based method. In: SPE Bergen One Day Seminar. Society of
Petroleum Engineers.
Ramirez, B., Joosten, G., Kaleta, M., Gelderblom, P., et al., 2017. Model-based well
location optimization–a robust approach. Society of Petroleum Engineers, SPE
Reservoir Simulation Conference.
Sarma, P., Aziz, K., Durlofsky, L.J., et al., 2005. Implementation of adjoint solution for
optimal control of smart wells. In: SPE Reservoir Simulation Symposium. Society of
Petroleum Engineers.
Shirangi, M.G., Durlofsky, L.J., et al., 2015. Closed-loop field development optimization
under uncertainty. In: SPE Reservoir Simulation Symposium. Society of Petroleum
Engineers.
Wang, X., Haynes, R.D., Feng, Q., 2015. Well control optimization using derivative-free
algorithms and a multiscale approach arXiv preprint arXiv:1509.04693.
Yang, C., Card, C., Nghiem, L.X., Fedutenko, E., et al., 2011. Robust optimization of SAGD
operations under geological uncertainties. In: SPE Reservoir Simulation Symposium.
Society of Petroleum Engineers.
Yeten, B., Durlofsky, L.J., Aziz, K., et al., 2002. Optimization of nonconventional well
type, location and trajectory. In: SPE Annual Technical Conference and Exhibition.
Society of Petroleum Engineers.
Zandvliet, M., Bosgra, O., Jansen, J., Van den Hof, P., Kraaijevanger, J., 2007. Bang-bang
control and singular arcs in reservoir flooding. J. Petroleum Sci. Eng. 58 (1),
186–200.
Zhang, Y., Lu, R., Forouzanfar, F., Reynolds, A.C., 2017. Well placement and control
optimization for WAG/SAG processes using ensemble-based method. Comput. Chem.
Eng. 101, 193–209.