Sasd

REE200862 DOI: 10.
2118/200862-PA Date: 16-October-20 Stage: Page: 1298 Total Pages: 16
Machine-Learning-Assisted Closed-Loop
Reservoir Management Using Echo
State Network for Mature Fields
under Waterflood
Lichi Deng* and Yuewei Pan**, Texas A&M University
Downloaded from http://onepetro.org/REE/article-pdf/23/04/1298/2446263/spe-200862-pa.pdf/1 by Indian Institute of Technology (ISM) Dhanbad user on 23 March 2023
Summary
Closed-loop reservoir management (CLRM) consists of continuous application of history matching and optimization of model-
predictive control to maximize production or reservoir net present value (NPV) in any given period. Traditional field-scale implementa-
tion of CLRM by using a large number of reservoir models, in particular when uncertainty is accounted for, is computationally
impractical. This presented machine-learning-assisted workflow uses the echo state network (ESN) coupled with an empirical water
fractional flow relationship as a proxy to replace time-consuming simulations and improve the computational efficiency of the CLRM.
The ESN, under the paradigm of reservoir computing, provides a specific architecture and supervised learning principle for recurrent
neural networks (RNNs). ESNs, with randomly generated and invariant input weights and recurrent weights, greatly minimize the com-
putational load and solve potential problems during typical backpropagation through time in traditional RNNs while it still obtains the
benefits of RNNs to memorize temporal dependencies. Also, the linear readout layer makes the training much faster using analytical
ridge regression. Field-level well control and production-response data are fed into the workflow to obtain a trained ESN and fitted
fractional-flow relationship, which will represent/reproduce the dynamics of the reservoir under various well-control scenarios. Further
production optimization is directly applied to the matched models to maximize reservoir NPV. The optimized well-control scenario is
applied, and further observation is obtained to update the models. History matching and production optimization are performed again in
a closed-loop fashion. The previously mentioned advantages make ESN a very powerful tool for CLRM, with both history matching
and production optimization quickly accomplished, and make near-real-time CLRM possible. In this paper, two case studies will be pre-
sented to prove the effectiveness of the proposed workflow.
Introduction
CLRM has been extensively studied in the literature because of its effectiveness in significantly increasing life-cycle value by changing
reservoir management from a batch type to a near-continuous model-based controlled activity (Naevdal et al. 2006; Sarma et al. 2008;
Chen et al. 2009; Jansen et al. 2009; Wang et al. 2009). The overall workflow of CLRM consists of two major steps: data assimilation
(history matching) to decrease model uncertainty and production optimization to obtain the optimal well-control scheme. In the existing
literature, most of the research performed in this area is focused on the detailed optimization algorithm performed in these two steps
and coupling data uncertainty into the simulation models. Many ensemble-based algorithms have been studied in the past and are useful
to facilitate improved data assimilation and optimization workflow. For example, the ensemble smoother has great efficiency when per-
forming history matching (Emerick and Reynolds 2013). Fonseca et al. (2017) studied the stochastic simplex approximate gradient
(StoSAG) algorithm for optimization under uncertainty, which outperforms the traditional ensemble-based optimization algorithm.
Similar applications have been done by Liu and Forouzanfar (2018) and Liu and Reynolds (2019), which all showed powerful perfor-
mance compared with traditional approaches.
However, depending on the scale of the model, the computational load required for running multiple simulation runs could be
extremely high, making near-continuous implementation impossible (Guo and Reynolds 2018). Thus, many reduced-order modeling
approaches have been proposed to lower the computational load. Many of the reduced-order modeling approaches are based on mathe-
matical dimension reduction techniques such as proper orthogonal decomposition (Gildin et al. 2013), whereas other attempts aimed at
simplifying the underlying physics to represent the flow or displacement mechanisms have also been made and could potentially be
used in this context (Lerlertpakdee et al. 2014; Deng and King 2015, 2019; Zhao et al. 2015; Chen et al. 2020; Wang et al. 2020b). For
example, Guo and Reynolds (2019) introduced a novel workflow that uses an analytical water-saturation solution based on the
Buckley-Leverett solution to represent reservoir flow dynamics and replace the reservoir simulator. However, most of these techniques
still require some type of reservoir models to start with.
In contrast, pure data-driven models have been used in the literature to represent the dynamics of reservoir flow as well (Wang et al.
2020a). These approaches ideally would not require any model to start with, and they normally are built based on sensor-recorded data
from the field directly with the help of some basic information about the reservoir. A typical example would be the capacitance resis-
tance model, and it has been used with CLRM directly by other researchers (Stensgaard 2016). The advantage of a data-driven model
lies in its independence of complex and time-consuming reservoir simulations or the need for geological modeling. Machine-learning
algorithms such as support-vector-regression under nonlinear regression has been used to approximate the input/output relationship of
the subsurface flow, but most of these techniques are taking the time-series production data as static information without much interpre-
tation regarding the recurrent relationship within the data (Guo and Reynolds 2018).
In this research, the RNN-based algorithm ESN is proposed to accomplish the CLRM task. Machine learning has had considerable
success in complex signal processing tasks, such as in acoustics and digital vision. ESN is one special kind of RNN, and it has been
* now with Quantum Reservoir Impact LLC

** now with PetroChina Exploration & Production Company
Copyright V
C 2020 Society of Petroleum Engineers
Original SPE manuscript received for review 5 March 2020. Revised manuscript received for review 21 May 2020. Paper (SPE 200862) peer approved 25 May 2020.
1298 November 2020 SPE Reservoir Evaluation & Engineering
ID: jaganm Time: 15:55 I Path: S:/REE#/Vol00000/200030/Comp/APPFile/SA-REE#200030

REE200862 DOI: 10.2118/200862-PA Date: 16-October-20 Stage: Page: 1299 Total Pages: 16
demonstrated to be practical for chaotic time-series data by establishing feature-based classifiers and nonlinear predictive models (Pan
et al. 2019b, 2019c). For decades, the costly gradient-based training procedure with nonconvex objective functions hindered RNN
applications (Scardapane and Wang 2017). ESN was then developed to randomly assign networks’ weights to define a feature-based,
data-dependent mapping so that the resulting optimization task can be simplified as a linear least-squares problem (Dambre et al. 2012).
The entire ESN structure contains three layers: the input layer; a larger-than-normal layer of neurons also called dynamic reservoir,
which is used with random initialization; and a final output layer. The basic idea is intuitively the same as the kernel tricks, by nonli-
nearly transforming the input signals into a higher-dimensional reservoir space. After being nonlinearly transformed, each neuron per-
forms as a dynamic filter recording the chronological behavior in various temporal space. Similar to the kernel tricks, only the output
weights are linearly adaptable and can be acquired by ridge regression; superior to the kernel tricks, the reservoir space is constrained
as an approximation of kernel space to improve the computation expenses (Hermans and Schrauwen 2012).
By considering the inherent time dependence inside the data sequence, the recurrent nature of ESN plays an important role so that
systematic dynamics between the well injection/production pressure and rates in a flooding system can be captured. However, to per-
form well-control optimization, the water front and water behavior should also be quantified. In this research, we adopted the typical
workflow as normally used in the capacitance resistance model, which uses the empirical fractional-flow relationship to forecast the
water cut. More specifically, the Koval fractional-flow relationship is used here (Koval 1963). It is essential that data-driven and artifi-
cial intelligence models should be guided by the physics-based models and properties to achieve top-tier performance (Deng et al.
2020). Because of the nature of the empirical model, which needs data to perform proper training, the Koval fractional-flow model still
requires a good amount of water-cut data after breakthrough for the regression purpose and to further guarantee a reliable forecasting
capability. Thus, the proposed workflow is intended to be applicable to more mature waterflood projects where a good amount of water
breakthrough has been achieved in producers. The prediction of water breakthrough time is not within the scope of this research.
This paper is organized as follows. First, we will introduce and review the key component methodologies of our workflow and dem-
onstrate the complete workflow itself. Important algorithms and processes such as ESN, CLRM, and Koval method will be reviewed.
Then we will show two test examples on one synthetic 2D model and one 3D model from the literature to prove the effectiveness of the
proposed workflow. In the case study, the ESN model is constructed independently without explicit dependency over the simulation
model, and the simulation is only used as a benchmarking tool for validation purposes. The outcome of CLRM is compared with the
base case well-control scenario, and improved NPV has been observed in both cases. The final comparison results demonstrate that
through the utilization of ESN, the behavior of the reservoir could be captured effectively, and the reservoir long-term profit could be
maximized with continuous history matching and long-term production optimization.
Methodology
In this section, the entire workflow of ESN-CLRM is presented. Before that, a brief review of each one of the key components selected
for this research is provided. The key pieces include CLRM, ESN, and the empirical fractional-flow relationship.
CLRM. In a traditional CLRM workflow, near-continuous application of history matching and production optimization is implemented
to facilitate life-cycle optimization of a flooding system, and generally, this process relies on existing numerical simulation models. Pro-
duction optimization is performed based on numerical models, which are updated in a frequent mode through data assimilation. Fig. 1
illustrates the key elements of the CLRM process (Jansen et al. 2009).
Noise Input System Output Noise

(reservoir, wells,
and facilites)
Controllable
input
Optimization
Sensors
algorithms
Geology, seismics,
System models well logs, well tests,
fluid properties, etc.
Data
assimilation
Predicted output algorithms Measured output
Fig. 1—Key elements of CLRM (after Jansen et al. 2009).
As illustrated in Fig. 1, the system represents the actual physical system comprising a reservoir, wells, and facilities. With the output
signals captured by the sensors to keep track of the production variables, the system models are updated to achieve a better estimate of
the unknown geological properties (Hou et al. 2015). On the left side of the figure, production optimization is shown in the blue loop
representing the algorithm performed, which further influences the settings of well chokes, injection rates, production pressure, and so
on. From the output of the optimization algorithm, the optimal control scenario is proposed and further implemented in the system for
the next period.
November 2020 SPE Reservoir Evaluation & Engineering 1299

Typically, CLRM is performed using multiple realizations of system models to account for uncertainty. However, for the reservoirs
without any prebuilt model, a purely data-driven approach could be extremely beneficial as long as it could mimic the essential dynam-
ics of the system. In the research, the proposed workflow is aimed at achieving this goal by using a powerful machine-learning algo-
rithm that is easy to implement.
ESN. ESN is a machine-learning algorithm proposed under the paradigm of reservoir computing, and it is essentially a member of the
RNN class designated for supervised learning tasks. For the family of reservoir computing, the recurrent part is kept fixed and is either
generated randomly or by means of custom topologies for facilitating the information flow (Bianchi et al. 2018). ESN is probably the
most widely applied reservoir computing method, and it is also chosen in this research as part of the machine-learning-assisted proxy
model to mimic the field dynamic responses. Fig. 2 shows the components of a typical ESN system (Sun et al. 2012).
Dynamic reservoir
Input x(t) Output
u(t) y(t)
Linear
readout
W
Win Wout
Fig. 2—Illustration of general ESN dynamic system.
In Fig. 2, the input signals are denoted using uðtÞ 2 RNu , and the output signals are denoted using yðtÞ 2 RNy . The term t in this gen-
eral ESN context simply refers to the discrete timesteps. In the context of this research that is aimed at waterflood management and
optimization, if all producers are under bottomhole pressure (BHP) control and all injectors are under water-injection rate control, the
construction of the input and output signals would become
uðtÞ ¼ ½pwf Prod 1 ðtÞ; pwf Prod 2 ðtÞ; ……; pwf Prod m ðtÞ; I w Inject 1 ðtÞ; I w Inject 2 ðtÞ; ……; I w Inject n ðtÞ; . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ð1Þ
yðtÞ ¼ ½ql Prod 1 ðtÞ; ql Prod 2 ðtÞ; ……; ql Prod m ðtÞ: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ð2Þ
The xðtÞ 2 RNx is a vector consisting of reservoir neuron activations that essentially represent the hidden state of the current time-
step. The term “reservoir” (or “dynamic reservoir”) in the ESN context is extremely important because it not only represents the non-
linear high-dimensional expansion of inputs but also serves as the memory at the same time. This idea is similar to the kernel methods
used in machine learning. For the other parameters shown in Fig. 2, Win 2 RNx ð1þNu Þ is the input weight matrix, W 2 RNx Nx is the
recurrent matrix, and Wout 2 RNy ð1þNu þNx Þ is the output weight matrix. One major advantage of using ESN is that both Win and W
matrices are generated randomly and kept as constant during the training. This differs from traditional RNN where backpropagation
through time is needed, which would require extremely large computational overhead (Pan et al. 2019a). Training typical neural net-
works usually requires the optimization of nonconvex objective functions, and often, the training process is computationally expensive,
which thus hinders the applications in reality (Scardapane and Wang 2017). The alternative is to project the input space to a high-
dimensional feature space by randomly assigning a subset of the networks’ weights just like the approach ESN is taking here. Then, the
optimization task can be transformed into a linear adaptation problem. In our case, the randomization of the Win and W can be used as
an approximation of kernel functions. The neurons in the reservoir extract the dynamic features and create echoes from the chrono-
logical input data streams in which the original purposes are to perform classification or regression.
The following equation demonstrates the typical update equations of ESN with leaky integrator a:
x~ðtÞ ¼ ufWin ½1; uðtÞ þ Wxðt 1Þg; . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ð3Þ
xðtÞ ¼ ð1 aÞxðt 1Þ þ a~
x ðtÞ: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ð4Þ
From these two equations, the update of the hidden state xðtÞ has the contribution from not only the nonlinear activation function output
x~ðtÞ but also the hidden state from last step directly xðt 1Þ. The weight associated with these two contributing components are con-
trolled by the leaky integrator a 2 ð0; 1. Sometimes the leaky integration is not used during the update,
which makes
a special case
1 e2x
where xðtÞ ¼ x~ðtÞ. The activation function uðxÞ used here is the hyperbolic tangent function tanhðxÞ ¼ , which is the most
1 þ e2x
common choice. However, certain other activation functions such as sigmoid function could still be used.
It could also be observed from these two equations that the input weight matrix Win , recurrent matrix W, and leaky integrator a are
very important because they together control the entire high-dimensional dynamic reservoir of ESN. Both Win and W matrices are typi-
cally randomly generated using a uniform distribution. W is typically very sparse, whereas Win tends to be denser. To ensure the sparsity
of the recurrent matrix, as a general ESN recommendation proposed by Jaeger and Haas (2004), the sparse interconnectivity is set to be
1% within the reservoir. This condition helps the reservoir to decompose into loosely coupled subsystems, establishing a richly structured
reservoir of excitable dynamics (Jaeger and Haas 2004). Also, Lukoševičius (2012) noted that, although sparse connections tend to give
better performance, it has a low priority to be optimized. In our actual implementation, the uniformly sampled sparse matrix W is

generated using MATLABV R function sprand by explicitly specifying the density of the matrix according to the predetermined 1% reser-
voir interconnectivity (MathWorks 2020). This guarantees the output matrix is sparse and uniformly sampled as we are expecting.
One important parameter used when generating the recurrent weight matrix is the spectral radius qðWÞ. This parameter controls the
scaling of the matrix, and it has been proved that qðWÞ typically should remain less than 1 for the ESN to hold the echo state property.
Thus, when W is initially generated by uniform sampling, it is divided by its spectral radius to yield a unit-spectral-radius matrix as an
initial guess, and then it is further adjusted in the hyperparameter tuning process. As for the input weight matrix Win , it is initialized by
a uniform distribution from [a, a], where a stands for the input scaling. Input scaling determines the nonlinearity of the reservoir
neuron responses with larger input scaling factor bringing more nonlinearity. In our experience, when the input data are normalized to
be bounded, choosing input scaling to be 1 seems effective in handling all test cases we have worked with. Ultimately, the spectral
radius of the recurrent weight matrix is more important because too large of a qðWÞ value may cause unstable reservoir activation.
Regarding the details and specifics about these parameters, they have been well documented in the literature, and we would refer the
readers to the work by Bianchi et al. (2018) for more details.
Moving into the readout layer, the ultimate goal of training an ESN is to obtain a model that could approximate the universal func-
tion y ¼ f ðxÞ, which could further be used for prediction. To achieve this goal, the training process should essentially minimize an error
measure of Eðy; ytarget Þ, which is typically represented using mean-square error or root-mean-square error. In our specific workflow, the
mean-square error is used where the objective function to be minimized is
Ny
( )
1 X 1X Nt
2
Eðy; ytarget Þ ¼ ½yi ðtÞ yi target ðtÞ þ bkWout
i k : . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ð5Þ
Ny i¼1 Nt t¼1
2
In this representation, on top of minimizing the discrepancy between the outputs, an additional term bkWout i k is added to penalize the
large weights in the output weight matrix using the regularization parameter b. This is called Tikhonov regularization, which is also
known as ridge regression. In general, this method improves the efficiency of the parameter estimation of any linear regression problem
(Gruber 1998). If we combine the input signals and hidden states of all timesteps to be X ¼ ½1; uðtÞ; xðtÞ 2 Rð1þNu þNx ÞT and also define
the output signal matrix to be Y ¼ ½yðtÞ; 2 RNy T , then the final linear system to be solved at the readout layer is simply expressed as
Y ¼ Wout X: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ð6Þ
To find the optimal output weight matrix that minimizes the objective function as shown in Eq. 5 using ridge regression, there exists
a universal and stable analytical solution:
Wout ¼ Ytarget XT ðXXT þ bIÞ1 : . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ð7Þ
In this equation, I 2 Rð1þNu þNx Þð1þNu þNx Þ is simply an identity matrix. This analytical form of the solution at the readout layer
reveals another large advantage of ESN, which is that the training process contains no gradient calculation for minimization, and this
greatly decreases the computational load again.
The previous section briefly reviewed the typical ESN system components with the leaky integrator. Overall, there are four important
global parameters (hyperparameters) that control the behavior of the ESN: dynamic reservoir size Nx , spectral radius q(W), leaky integra-
tor a, and Tikhonov regularization parameter b. These four hyperparameters should be tuned/trained carefully to ensure the successful
application of ESN. We will also show the results from the hyperparameter tuning step in the later test examples sections.
Koval Empirical Fractional-Flow Relationship. As shown in Eq. 2, in our workflow, the outputs of the ESN for all producers are
liquid rates without trying to infer water cut from the data. Thus, an empirical fractional-flow relationship is used to calculate the
expected water cut based on the existing production history. This is identical to the step normally implemented in a capacitance resis-
tance model workflow, and many fractional-flow relationships have been used in the literature (Timmerman 1971; Ershaghi and
Omorigie 1978; Gentil 2005; Temizel et al. 2019). In this research, the specific fractional-flow relationship selected is the Koval
method, which was originally proposed to address the issue of fingering in unstable miscible displacement process (Koval 1963). This
method is an analog to the Buckley-Leverett theory, and it has been proved to have a broader application such as in immiscible dis-
placement and is suitable for the whole life of a waterflood system (Buckley and Leverett 1942; Lake 2014; Cao et al. 2015). The origi-
nal Koval model is slightly changed in this exposure for easier adoption. First, the following equation shows the Buckley-Leverett
fractional flow fw equation under segregated water and oil flow conditions:
1 Sw Swi
fw ¼ ; where S¼ : . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ð8Þ
1 1S 1 Swi Sorw
1þ
Kval S
In this equation, Kval represents the Koval factor, S denotes the normalized water saturation, Swi is the irreducible water saturation,
and Sorw is the residual oil saturation. With this fractional-flow representation, the derivative of it represents the saturation speed:
1 xD
fw0 ¼ 2 ¼ ; . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ð9Þ
1 tD
Kval S þ ð1 SÞ
Kval
where xD represents the rescaled dimensionless distance that equals the distance along the streamline or streamtube normalized by the
total length of it. The term tD represents the rescaled dimensionless time. In this specific research, tD is defined as Q=Vp , where Q stands
for the cumulative produced liquid volume of a producer, and Vp stands for the drainage pore volume (PV) of the corresponding producer.
To obtain an algebraic equation for the saturation term in Eq. 9:
rffiffiffiffiffiffiffiffiffiffiffiffiffiffi
tD
Kval 1
xD
S¼ : . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ð10Þ
Kval 1
Finally, the water fractional flow at the producer (xD ¼ 1 or outlet of streamline/streamtube), which is essentially the empirical
Koval fractional-flow relationship, could be expressed as follows:

8
>
> 1
>
> 0; tD <
>
> Kval
>
< rffiffiffiffiffiffiffiffi
Kval
fw ¼ Kval
>
> tD 1
>
> ; tD Kval
>
> K 1 Kval
>
: val
1; tD > Kval : ð11Þ
Based on Eq. 11, there exist effectively two parameters that define a unique Koval relationship given any input data. The parameters
are the drainage PV Vp and the Koval factor Kval . In the actual implementation, these two parameters are first given an initial guess using
the best guess based on the reservoir knowledge, and then they are tuned for each producer when fitting the Koval model by minimizing
the mismatch between the actual data and model predictions. The initial guess of the drainage PV could be treated as an acreage-based
estimate coupled with rough understanding of the net pay and porosity of the field. During the CLRM cycles, the values for each producer
will update based on the regression process, and ideally it will become more and more realistic as more water-cut information is acquired.
In the next section, the details regarding how the Koval fractional-flow relationship is used will be illustrated in the complete work-
flow. Essentially, for every producer in the field with production history, the previous model could be used to fit the water-cut data and
obtain a regression model for future water-cut prediction.
ESN–CLRM Workflow. The previous three sections briefly reviewed the key components of the ESN-CLRM workflow. In Fig. 3, the
complete illustration of this workflow is demonstrated, and this will be directly applied in the later test examples to show its effectiveness.
System
(reservoir, wells
Well control input: BHP or etc.) Dynamic response
production/injection rate
Sensor
Optimization
Measurement
Well liquid
ESN Training production/injection
rate or BHP
Data-driven Koval fractional-

model Well water cut
flow relationship
Regression
Fig. 3—Complete workflow of ESN-CLRM.
From the workflow in Fig. 3, the connections between all important modules introduced in the previous methodology sections are
illustrated. Like the traditional CLRM workflow, the starting point and fundamental pillar of the workflow is the system that represents
the oilfield asset consisting of reservoir, wellbore, choke, and so on. For the system under a typical waterflood recovery mechanism, the
input would be either the well injection/production BHP or the well injection/production liquid rate. Under the specified well-control
scenario, the dynamic response of the system is measured by sensors deployed in the field to record the resulting liquid flow rate or
BHP and well-production water cut.
With the measured responses, a data-driven model proposed by this research that consists of the ESN and Koval fractional-flow rela-
tionship is used to approximate the true reservoir state by assimilating the recorded data. In this typical workflow, the data assimilation
step is broken down into two components. First, the Koval fractional-flow relationship is used to fit the water-cut data on the well level
through a simple regression algorithm, and the tuning parameters are Kval and Vp as shown in the previous section. In the meantime, an
ESN model is trained using the injection well control and output dynamic response data. One key advantage for integrating with ESN is
that the training algorithm uses ridge regression, providing an analytic solution. Thus, no optimization or gradient calculation is needed
for this part of the training process.
Once both components of the data-driven model are properly history matched, this model could be used in conjunction with any
expected future well-control data to estimate the future reservoir dynamic responses. These calculated response data could be turned
into an objective function using the following relationship:
( "N Ninj
#)
XNt X prod X
O¼ Dtn ðio qo; j n cw qw; j n Þ ðci qwi;k n Þ : . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ð12Þ
n¼1 j¼1 k¼1
This objective function is essentially the cost function used to measure the benefit for the life-cycle production optimization.
It could also be considered as the NPV without the discounting factor of time. In the previous equation, the superscript n denotes the
timesteps, and Nt represents the total number of timesteps. The io , cw , and ci represent the income through oil production, the cost
by disposing of produced water, and the cost of water injection, respectively. The term qno; j represents the timestep-average oil produc-
tion rate of the jth producer, qnw; j represents the timestep-average water production rate of the jth producer, and qnwi;k represents the

timestep-average water injection rate of the kth producer, all at the nth timestep. Nprod refers to the total number of producers, and
Ninj represents the total number of injectors.
To maximize the objective function defined by Eq. 12, any optimization algorithm could be used to optimize the NPV by tuning the
future well-control parameters, and the resulting optimized control would be fed into the system to obtain new response data for the
next period. This process is performed in a cyclic fashion to close the loop of the smart management of the reservoir. For this research
specifically, a global optimization technique, mesh adaptive direct search, is used to maximize the NPV through the NOMAD
MATLAB package (Audet and Dennis 2006; Le Digabel 2011).
Test Example 1. In this part, the proposed workflow is first applied to a 2D synthetic reservoir model. The 2D model’s dimension is
51 51, which results in 2,601 cells. There are synthetic wells placed within the model, and they form a typical inverted five-spot pat-
tern where the producers are placed at the corners of the model while the injector is placed in the middle. In this analysis, all producers
are assumed to be under BHP control, and the injector is assumed to be under injection-rate control, all with upper and lower bound
constraints. Fig. 4 shows the permeability field of this synthetic 2D model.
log10 (md)
Prod 1 Prod 3
2.5
2.0
1.5
Inject 1
1.0
0.5
Prod 2 Prod 4
Fig. 4—Permeability field of the synthetic 2D model.
The previous model is assumed to be under production for 5 years before we started the CLRM workflow. Thus, the well-level pres-
sure and rate responses could be used as training data to initialize our ESN model. The data-recording frequency is assumed to be every
10 days, and the existing history has 180 data points recorded for training in total. We have split this data series into three pieces: the
first 60% of the data points are used for training the ESN model, the middle 20% of the data points are used for validation purposes,
and the last 20% are used for blind test to verify the model robustness. The training and validation data points are used to give an esti-
mate of model skill while tuning the model’s hyperparameters. Recall that in the specific research, four global parameters that affect the
ESN performance the most are tuned: dynamic reservoir size Nx , spectral radius qðWÞ, leaky integrator a, and Tikhonov regularization
parameter b. Figs. 5 and 6 show the input and output responses for the first 5 years of this synthetic test example, respectively. As we
can see, for all producers to have seen water breakthrough, the ranges of BHP for each producer are different because of the heterogene-
ity of the reservoir model itself.
Fig. 7 shows the match between the model output and the true output signal for both the validation period and the blind test period.
The blind test data points are not used during the training process, and they are intended to give an unbiased estimate of the skill of the
final tuned model. From Fig. 7, we could see that both the validation and the blind test data set show good agreement between the
trained model’s predictions and true output signals. This proves that the ESN model, after the training, has learned to mimic the under-
lying dynamic relationship between the controlling well BHPs and well production and injection liquid rates.
As the other part of the data-driven model, the Koval fractional-flow relationship is applied to the existing water-cut history of the four
producers to fit the production water behavior. Fig. 8 shows the match between the Koval model after the regression and the true water-
cut data. From the match between the Koval model and the existing data, the overall trend of the water fractional flow has been success-
fully captured by the regression. These two training and regression steps show the initial preparation results of the model to be used.
According to the workflow, the next step would be to initiate CLRM and ultimately optimize the NPV at the end of the CLRM life.
To compare the result of the CLRM, one base case is setup for this test example. The base case is assumed to be that the last well-
control scheme at the end of the existing production history will remain the same for another 2 years. In this sense, the total time period
for CLRM is 2 years after the initial 5-year production. Ideally, the frequency of the data assimilation and production optimization
should be high. However, for simplicity, the well-control change frequency is set to be every 3 months or 90 days. The optimization
tuning variables are the BHPs for the four producers and the injection water rate for the injector. The tuning parameter bounds are set to
be similar to the existing bounds of BHP history for producers and 1,000 to 2,000 STB/D for the injector. This range covers the original
well-control history in the first 5 years but still provides a slight margin to allow improvement on the performance. The purpose of
allowing a slight margin on the tuning bounds is to constrain the extent of extrapolation needed by the ESN model, which could be
highly uncertain. For the fields with a reasonable amount of production history, this condition should be easily satisfied. The initial
guess used for the production optimization at every cycle is the same as the value from the previous data point. The prices used for the
cost function to be optimized are as follows: io ¼ 50, cw ¼ 2, and ci ¼ 1, and all units are in USD/STB.
Figs. 9 and 10 show the comparison between the base case well-control scenario and the final optimized well-control scenario for
the entire duration of CLRM, as well as the NPV comparison between the two cases, respectively. From Fig. 9, it is interesting to notice
that the optimized well control, to a certain extent, follows the bang-bang control theory for the optimization of the waterflood problem
(Sudaryanto and Yortsos 2000, 2001); that is, the components of the control vector are taking values close to the bound.

Prod 1 Prod 2 Prod 3

800 2,600 2,400
2,200
BHP (psi)
BHP (psi)
BHP (psi)
600 2,400
2,000
400 2,200
1,800
200 2,000 1,600

500 1,000 1,500 500 1,000 1,500 500 1,000 1,500
Time (days) Time (days) Time (days)
Prod 4 Inject 1
2,600 2,000
Injection Rate (STB/D)
BHP (psi)
2,400 Training
1,500 Validation
Blind test
2,200
2,000 1,000
500 1,000 1,500 500 1,000 1,500
Time (days) Time (days)
Fig. 5—Well-control data for the injector (rate) and all producers (BHP) in the first 5 years for Test Example 1: black portion used as
training input, red portion used as validation input, and blue portion used as blind test input.
Prod 1 Prod 2
300 1,200
Liquid Production (STB/D)
1,000
250
800
200
600
150 400
500 1,000 1,500 500 1,000 1,500
Time (days) Time (days) Training
Validation
Blind test
Prod 3 Prod 4
400 600
500
300
400
200
300
100 200
500 1,000 1,500 500 1,000 1,500
Fig. 6—Well-production liquid rate for all producers in the first 5 years for Test Example 1: black portion used as training output,
red portion used as validation output, and blue portion used as blind test output.
From Fig. 10, it is clearly observed that the optimized scenario yields better return at any time during the last 2-year period com-
pared with the base case. Using this, the final NPV at the end of the 7-year production is USD 83,220,026, which is USD 3,350,698
more in return compared with the NPV of USD 79,869,328 for the base case. The improvement is mainly achieved by improving the
total oil production by 105,708 STB. The total water production is enlarged by 609,281 STB, and the total water injection volume has
increased in the optimized case by 716,156 STB compared with the base case scenario. Although both water production and injection
have been increased, because of the high profit margin provided by oil production, the overall NPV has increased significantly. Fig. 11
shows cumulative oil-production, water-production, and water-injection volumes for the last 2 years when CLRM was performed.
To further validate the workflow and prove the effectiveness of the proposed ESN-based proxy model, we compare the performance
of the prediction of the proposed data-driven model with the prediction obtained from the true simulation model during CLRM. Fig. 12
shows the corresponding results for both field-level liquid production rate and field-level water cut at each cycle during CLRM, and for
each of them, four cases are presented: optimized scenario prediction by ESN þ Koval; optimized scenario prediction by simulation;

base scenario prediction by ESN þ Koval; and base scenario prediction by simulation. As demonstrated in Fig. 12, at each cycle, the
future prediction from the ESN þ Koval method is in close agreement with the true simulated responses, which further proves the effec-
tiveness of our proposed ESN-based workflow in capturing the dynamics during waterflood.
Prod 1 Prod 2
1.0 1.0
Normalized True Data

0.8 0.8
0.6 0.6
0.4 0.4
0.2 0.2
0 0
0 0.5 1.0 0 0.5 1.0
Normalized ESN Prediction Normalized ESN Prediction Validation
Blind test
Prod 3 Prod 4 Unit slope
1.0 1.0

0.8 0.8
0.6 0.6
0.4 0.4
0.2 0.2
0 0
0 0.5 1.0 0 0.5 1.0
Normalized ESN Prediction Normalized ESN Prediction
Fig. 7—Matching between ESN prediction and true output signal after hyperparameter tuning for both validation and blind test
data set for Test Example 1.
Prod 1 Prod 2
1.0 1.0
Data
Fitted model
Water Cut
Water Cut
0.5 0.5
Data
Fitted model
0 0
0 500 1,000 1,500 2,000 0 500 1,000 1,500 2,000
Prod 3 Prod 4
1.0 1.0
Data
Fitted model
Water Cut
Water Cut
0.5 0.5
Data
Fitted model
0 0
0 500 1,000 1,500 2,000 0 500 1,000 1,500 2,000
Fig. 8—Matching between Koval fractional-flow relationship and true water cut for all producers for the first 5 years for Test
Example 1.
Test Example 2. In this section, one additional 3D case study is presented to further illustrate the implementation of the previously
mentioned workflow. In this case study, the first realization of the egg model (Jansen et al. 2014) is used along with a reservoir simula-
tor to represent the true reservoir responses (Lie et al. 2012). The grid dimension is 60 60 7, which results in 25,200 cells, within
which 18,553 cells are active. The model has eight injectors and four producers. In this analysis, all producers are assumed to be under
BHP control and all injectors are under rate control, all with upper- and lower-bound constraints. The logarithmic representation of the
horizontal-permeability field of the model is shown in Fig. 13.

Prod 1 Prod 2 Prod 3

2,500
700 2,200
2,400
2,100
BHP (psi)
BHP (psi)
BHP (psi)
600
2,300
500 2,000
2,200
400 1,900
2,100
300 1,800
2,000
1,800 2,000 2,200 2,400 1,800 2,000 2,200 2,400 1,800 2,000 2,200 2,400
Prod 4 Inject 1
2,500 2,000
2,400
BHP (psi)
2,300 Base scenario

1,500 Optimized scenario
2,200
2,100
2,000 1,000
1,800 2,000 2,200 2,400 1,800 2,000 2,200 2,400
Fig. 9—Well-control comparison between base case and final optimized case for Test Example 1.
×107
8.4
Base scenario
Optimized scenario
8.2
NPV (USD)
8.0
7.8
7.6
7.4
1,800 1,900 2,000 2,100 2,200 2,300 2,400 2,500
Time (days)
Fig. 10—NPV/cost function comparison between base case and final optimized case for Test Example 1.
×106 ×106 ×106

1.85 2.6 4.4
Base scenario Base scenario Base scenario
Cumulative Water Production (STB)
Cumulative Oil Production (STB)
Cumulative Water Injection (STB)
Optimized scenario 2.4 Optimized scenario 4.2 Optimized scenario

1.80
4.0
2.2
3.8
1.75
2.0
3.6
1.8
1.70 3.4
1.6
3.2
1.65
1.4 3.0
1.60 1.2 2.8

1,800 2,000 2,200 2,400 1,800 2,000 2,200 2,400 1,800 2,000 2,200 2,400
(a) (b) (c)
Fig. 11—(a) Cumulative oil-production volume comparison between base case and final optimized case for Test Example 1. (b)
Cumulative water-production volume comparison between base case and final optimized case for Test Example 1. (c) Cumulative
water-injection volume comparison between base case and final optimized case for Test Example 1.

Liquid Rate (STB/D) Liquid Rate (STB/D) Liquid Rate (STB/D) Liquid Rate (STB/D)
Liquid Rate (STB/D) Liquid Rate (STB/D) Liquid Rate (STB/D) Liquid Rate (STB/D)
Cycle 1–Field Liquid Rate Cycle 1–Field Water Cut Cycle 2–Field Liquid Rate Cycle 2–Field Water Cut
10,000 1.0 6,000 1.0
Water Cut
Water Cut
0.8 0.8
5,000 4,000
0.6 0.6
0 2,000
1,800 2,000 2,200 2,400 1,800 2,000 2,200 2,400 1,800 2,000 2,200 2,400 1,800 2,000 2,200 2,400
Time (days) Time (days) Time (days) Time (days)
6,000 1.0 6,000 1.0
Water Cut
Water Cut
0.8 0.8
4,000 4,000
0.6 0.6
2,000 2,000
1,800 2,000 2,200 2,400 1,800 2,000 2,200 2,400 1,800 2,000 2,200 2,400 1,800 2,000 2,200 2,400
6,000 1.0 6,000 1.0
Water Cut
Water Cut
0.8 0.8
4,000 4,000
0.6 0.6
2,000 2,000
1,800 2,000 2,200 2,400 1,800 2,000 2,200 2,400 1,800 2,000 2,200 2,400 1,800 2,000 2,200 2,400
6,000 1.0 6,000 1.0
Water Cut
Water Cut
0.8 0.8
4,000 4,000
0.6 0.6
2,000 2,000
1,800 2,000 2,200 2,400 1,800 2,000 2,200 2,400 1,800 2,000 2,200 2,400 1,800 2,000 2,200 2,400
History
Future prediction: simulation result–optimized scenario
Future prediction: simulation result–base scenario
Future prediction: ESN+Koval–optimized scenario
Future prediction: ESN+Koval–base scenario
Fig. 12—Comparison of field-level liquid rate and water cut between proposed data-driven model and true simulation result at
each cycle during CLRM for Test Example 1.
Log10 (md)
3.6
Inject 5 3.4
Inject 2 Prod 2 Inject 8

3.2
Prod 4
3.0
Inject 4
Inject 1 Prod 1
Prod 3 Inject 7 2.8
Inject 3 2.6
Inject 6
2.4
2.2
Fig. 13—Permeability field of the 3D egg model.
First, we performed a 1.5-year simulation under variable well control to mimic the existing production history of the field, and the
goal is to perform CLRM for the period of the following half a year (essentially the contract duration of the lease is 2 years). The data-
recording frequency is assumed to be 3 days, and the existing history has 180 data points recorded for training in total. Figs. 14 and 15
show the existing history of the well control (input for ESN) and producers’ liquid production rates (output for ESN), respectively. This
entire history again is split into three parts (60%, 20%, and 20%), the same as what we performed in Test Example 1.
With these input and output signals, we could successfully train our model and perform the hyperparameter tuning again. Fig. 16
shows the match between the model output and the true output signal during both the validation period and the blind test period in Test
Example 2. From the results, we can conclude that the hyperparameter tuning provides a reasonably good set of global parameters.
With a well-tuned ESN model as one part of the data-driven model, the Koval fractional-flow model, as the other half, is again fitted
with the water-cut data to facilitate the prediction of future water production trend as shown in Fig. 17. As we can see, all producers
have seen water breakthrough at the end of the 1.5 years.

Prod 1 Prod 2 Prod 3 Prod 4
5,700 5,700 5,700 5,700
BHP (psi)
BHP (psi)
BHP (psi)
BHP (psi)
5,650 5,650 5,650 5,650
5,600 5,600 5,600 5,600

100 200 300 400 500 100 200 300 400 500 100 200 300 400 500 100 200 300 400 500
Inject 1 Inject 2 Inject 3 Inject 4

700 700 700 700
650 650 650 650
600 600 600 600
550 550 550 550
100 200 300 400 500 100 200 300 400 500 100 200 300 400 500 100 200 300 400 500

700 700 700 700
650 650 650 650
600 600 600 600
550 550 550 550

100 200 300 400 500 100 200 300 400 500 100 200 300 400 500 100 200 300 400 500
Training Validation Blind test
Fig. 14—Well-control data for all injectors (rate) and producers (BHP) in the first 1.5 years for Test Example 2: black portion used
as training input, red portion used as validation input, and blue portion used as blind test input.
Prod 1 Prod 2
2,500 2,000
2,000 1,500
1,500 1,000
1,000 500
500 0
100 200 300 400 500 100 200 300 400 500 Training
Validation
Prod 3 Prod 4 Blind test
2,500
2,000
2,000
1,500
1,500
1,000
1,000
500 500
100 200 300 400 500 100 200 300 400 500
Fig. 15—Producer liquid rate in the first 1.5 years for Test Example 2: black portion used as training output, red portion used as val-
idation output, and blue portion used as blind test output.
In this case, the CLRM is performed for the next half a year by assuming the lease of the field lasts for 2 years. The frequency of the
well-control update in this test example is set to be every month. The upper and lower bounds for the injection rates and producer BHPs
are shown as the y-axes limits in Fig. 18. Just as stated in Test Example 1, the bounds for the tuning variable are determined based on
the previous control history, and the initial guess used for the production optimization at every cycle is the same as the value from the
previous data point. The prices used for the cost function are the same as in Test Example 1 as well. The base case to be compared
against is again set to be using the last well-control scheme at the end of the first year. Fig. 18 shows the well-control comparison
between the base case and the final optimized case after all cycles of CLRM for the latter half of the second year.
By feeding the final optimized well-control scenario and the base case well-control scenario to the reservoir simulator, the compari-
son between the NPVs from both control scenarios could be calculated as a function of time. Fig. 19 shows the NPV values for the last
half a year. It still concludes that, by using the ESN and Koval relationship as a proxy in CLRM, an optimized production strategy

could be obtained. The final NPV at the end of the 2-year production is USD 124,093,474, and compared with the value of
USD 121,046,761 for the base case, the increment is USD 3,046,713. The total oil production from the four producers is increased by
62,835 STB. The total water production is increased by 10,712 STB. However, the field water cut at the end of the second year is
78.66% for the optimized case, which is less than the water cut of 80.97% for the base case. Finally, the total water-injection volume
has been increased by 73,654 STB compared with the base case scenario. Fig. 20 shows the cumulative oil-production, water-production,
and water-injection volumes for the second year.
Prod 1 Prod 2
1.0 1.0

0.8 0.8
0.6 0.6
0.4 0.4
0.2 0.2
0 0
0 0.5 1.0 0 0.5 1.0
Validation
Blind test
Prod 3 Prod 4 Unit slope
1.0 1.0

0.8 0.8
0.6 0.6
0.4 0.4
0.2 0.2
0 0
0 0.5 1.0 0 0.5 1.0
Fig. 16—Matching between ESN prediction and true output signal after hyperparameter tuning for both validation and blind test
data set for Test Example 2.
Prod 1 Prod 2
1.0 1.0
Data Data
Water Cut
Water Cut
Fitted model Fitted model
0.5 0.5
0 0
0 200 400 600 0 200 400 600
Prod 3 Prod 4
1.0 1.0
Data Data
Water Cut
Water Cut
Fitted model Fitted model
0.5 0.5
0 0
0 200 400 600 0 200 400 600
Fig. 17—Matching between Koval fractional-flow relationship and true water cut for all producers for the first 1.5 years for Test
Example 2.
Identical to the comparison performed in Test Example 1, Fig. 21 still shows the prediction performance comparison between the
ESN þ Koval method and true simulation responses for field-level liquid production rates and water cuts at each cycle during CLRM.
Again, on top of the blind test during the hyperparameter tuning step, this comparison further proves the capability of the ESN þ Koval
method in mimicking the waterflood dynamics.

Prod 1 Prod 2 Prod 3 Prod 4
5,680 5,680 5,680 5,680
BHP (psi)
BHP (psi)
BHP (psi)
BHP (psi)
5,660 5,660 5,660 5,660
5,640 5,640 5,640 5,640
550 600 650 700 550 600 650 700 550 600 650 700 550 600 650 700


680 680 680 680
660 660 660 660
640 640 640 640
620 620 620 620
600 600 600 600
580 580 580 580
550 600 650 700 550 600 650 700 550 600 650 700 550 600 650 700


680 680 680 680
660 660 660 660
640 640 640 640
620 620 620 620
600 600 600 600
580 580 580 580
550 600 650 700 550 600 650 700 550 600 650 700 550 600 650 700
Base scenario Optimized scenario
Fig. 18—Well-control comparison between base case and final optimized case for Test Example 2.
×108
1.26
Base scenario
1.24 Optimized scenario
1.22
NPV (USD)
1.20
1.18
1.16
1.14
1.12
540 560 580 600 620 640 660 680 700 720
Time (days)
Fig. 19—NPV/cost function comparison between base case and final optimized case for Test Example 2.
×106 ×105 ×106

2.60 11 3.8
Cumulative Water Production (STB)
Cumulative Water Injection (STB)

Cumulative Oil Production (STB)
Base scenario Base scenario Base scenario

2.55 Optimized scenario 10 Optimized scenario 3.6 Optimized scenario
9
2.50 3.4
8
2.45 3.2
7
2.40 3.0
6
2.35 5 2.8
2.30 4 2.6
550 600 650 700 550 600 650 700 550 600 650 700
(a) (b) (c)
Fig. 20—(a) Cumulative oil-production volume comparison between base case and final optimized case for Test Example 2. (b)
Cumulative water-production volume comparison between base case and final optimized case for Test Example 2. (c) Cumulative
water-injection volume comparison between base case and final optimized case for Test Example 2.

Liquid Rate (STB/D)
Liquid Rate (STB/D)

1.0 1.0
Water Cut
Water Cut
5,500 5,500
0.8 0.8
5,000 0.6 5,000 0.6
550 600 650 700 550 600 650 700 550 600 650 700 550 600 650 700
Liquid Rate (STB/D)
Liquid Rate (STB/D)

1.0 1.0
Water Cut
Water Cut
5,500 5,500
0.8 0.8
5,000 0.6 5,000 0.6
550 600 650 700 550 600 650 700 550 600 650 700 550 600 650 700
Liquid Rate (STB/D)
Liquid Rate (STB/D)

1.0 1.0
Water Cut
Water Cut
5,500 5,500
0.8 0.8
5,000 0.6 5,000 0.6
550 600 650 700 550 600 650 700 550 600 650 700 550 600 650 700
History
Future prediction: simulation result–optimized scenario
Future prediction: simulation result–base scenario
Future prediction: ESN+Koval–optimized scenario
Future prediction: ESN+Koval–base scenario
Fig. 21—Comparison of field-level liquid rate and water cut between proposed data-driven model and true simulation result at
each cycle during CLRM for Test Example 2.
Discussion and Conclusions

This research presents a novel workflow for CLRM using a proxy model that is inspired by an RNN-based machine-learning algorithm
ESN with successful applications on two test examples. The coupling of the machine-learning algorithm with an empirical data-driven
fractional-flow model shows promising results and further demonstrates the capability of our workflow facilitating field-wise operation
optimization. The process shows reasonably good accuracy of mimicking the dynamic input/output relationship in addition to the
improved end-cycle optimization target values. With the accuracy and simplicity of our workflow, it exhibits a much decreased compu-
tational load compared with the traditional CLRM workflow using a model-based simulation. Although these techniques are an approxi-
mation in nature and may not completely substitute full-order simulation, they still can be used to seek good initial solutions to
significantly decrease the number of iterations in CLRM with full models.
The direct application of this workflow would greatly benefit reservoir management with respect to any mature field under water-
flood with enough data after water breakthrough and would be specifically beneficial for fields without reservoir models because of its
simplicity by building on only production data. In the future, the work could be extended to perform robust training with available ini-
tial models. In that particular case, the ESN would no longer require existing production history to initiate the training. Additionally,
the ESN can be trained using the response generated by reservoir simulators using the provided model. Last, by considering capturing
more uncertainty, Bayesian inversion techniques may also be considered in the future for more robust and reliable CLRM results.
References
Audet, C. and Dennis, J. E. 2006. Mesh Adaptive Direct Search Algorithms for Constrained Optimization. SIAM J Optim 17 (1): 188–217. https://
doi.org/10.1137/060671267.
Bianchi, F. M., Scardapane, S., Lokse, S. et al. 2018. Reservoir Computing Approaches for Representation and Classification of Multivariate Time
Series. https://arxiv.org/abs/1803.07870.
Buckley, S. E. and Leverett, M. C. 1942. Mechanism of Fluid Displacement in Sands. Trans AIME 146 (1): 107–116. SPE-942107-G. https://doi.org/
10.2118/942107-G.
Cao, F., Luo, H., and Lake, L. W. 2015. Oil-Rate Forecast by Inferring Fractional-Flow Models from Field Data with Koval Method Combined with the
Capacitance/Resistance Model. SPE Res Eval & Eng 18 (4): 534–553. SPE-173315-PA. https://doi.org/10.2118/173315-PA.
Chen, H., Onishi, T., Olalotiti-Lawal, F. et al. 2020. Streamline Tracing and Applications in Embedded Discrete Fracture Models. J Pet Sci & Eng 188:
1–20. https://doi.org/10.1016/j.petrol.2019.106865.
Chen, Y., Oliver, D. S., and Zhang, D. 2009. Efficient Ensemble-Based Closed-Loop Production Optimization. SPE J. 14 (4): 634–645. SPE-112873-
PA. https://doi.org/10.2118/112873-PA.
Dambre, J., Verstraeten, D., Schrauwen, B. et al. 2012. Information Processing Capacity of Dynamical Systems. Sci Rep 2: 512. https://doi.org/
10.1038/srep00514.
Deng, L. and King, M. J. 2015. Capillary Corrections to Buckley-Leverett Flow. Paper presented at the SPE Annual Technical Conference and Exhibi-
tion, Houston, Texas, USA, 28–30 September. SPE-175150-MS. https://doi.org/10.2118/175150-MS.
Deng, L. and King, M. J. 2019. Theoretical Investigation of the Transition from Spontaneous to Forced Imbibition. SPE J. 24 (1): 215–229. SPE-
190309-PA. https://doi.org/10.2118/190309-PA.
Deng, L., Salehi, A., Benhallam, W. et al. 2020. Artificial-Intelligence Based Horizontal Well Placement Optimisation Leveraging Geological and Engi-
neering Attributes, and Expert-Based Workflows. Paper presented at the SPE Conference at Oman Petroleum & Energy Show, Muscat, Oman, 9–11
March. SPE-200069-MS. https://doi.org/10.2118/200069-MS.
Emerick, A. A. and Reynolds, A. C. 2013. History-Matching Production and Seismic Data in a Real Field Case Using the Ensemble Smoother with Mul-
tiple Data Assimilation. Paper presented at the SPE Reservoir Simulation Symposium, The Woodlands, Texas, USA, 18–20 February. SPE-163675-
MS. https://doi.org/10.2118/163675-MS.

Ershaghi, I. and Omorigie, O. 1978. A Method for Extrapolation of Cut vs Recovery Curves. J Pet Technol 30 (2): 203–204. SPE-6977-PA. https://
doi.org/10.2118/6977-PA.
Fonseca, R. R. M., Chen, B., Jansen, J. D. et al. 2017. A Stochastic Simplex Approximate Gradient (StoSAG) for Optimization under Uncertainty. Int J
Numer Meth Eng 109: 1756–1776. https://doi.org/10.1002/nme.5342.
Gentil, P. H. 2005. The Use of Multilinear Regression Models in Patterned Waterflood: Physical Meaning of the Regression Coefficients. MS thesis, The
University of Texas at Austin, Austin, Texas, USA.
Gildin, E., Ghasemi, M., Romanovskay, A. et al. 2013. Nonlinear Complexity Reduction for Fast Simulation of Flow in Heterogeneous Porous Media.
Paper presented at the SPE Reservoir Simulation Symposium, The Woodlands, Texas, USA, 18–20 February. SPE-163618-MS. https://doi.org/
10.2118/163618-MS.
Gruber, M. 1998. Improving Efficiency by Shrinkage: The James–Stein and Ridge Regression Estimators. Boca Raton, Florida, USA: CRC Press.
Guo, Z. and Reynolds, A. C. 2018. Robust Life-Cycle Production Optimization with a Support-Vector-Regression Proxy. SPE J. 23 (6): 2409–2427.
SPE-191378-PA. https://doi.org/10.2118/191378-PA.
Guo, Z. and Reynolds, A. C. 2019. INSIM-FT in Three-Dimensions with Gravity. J Comput Physics 380: 143–169. https://doi.org/10.1016/
j.jcp.2018.12.016.
Hermans, M. and Schrauwen, B. 2012. Recurrent Kernel Machines: Computing with Infinite Echo State Networks. Neural Comput 24 (1): 104–133.
https://doi.org/10.1162/NECO_a_00200.
Hou, J., Zhou, K., Zhang, X. et al. 2015. A Review of Closed-Loop Reservoir Management. Pet Sci 12: 114–128. https://doi.org/10.1007/s12182-
014-0005-6.
Jaeger, H. and Haas, H. 2004. Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication. Science 304 (5667):
78–80. https://doi.org/10.1126/science.1091277.
Jansen, J. D., Douma, S. D., Brouwer, D. R. et al. 2009. Closed-Loop Reservoir Management. Paper presented at the SPE Reservoir Simulation Sympo-
sium, The Woodlands, Texas, USA, 2–4 February. SPE-119098-MS. https://doi.org/10.2118/119098-MS.
Jansen, J. D., Fonseca, R. M., Kahrobaei, S. et al. 2014. The Egg Model—A Geological Ensemble for Reservoir Simulation. Geosci Data J 1: 192–195.
https://doi.org/10.1002/gdj3.21.
Koval, E. J. 1963. A Method for Predicting the Performance of Unstable Miscible Displacement in Heterogeneous Media. SPE J. 3 (2): 145–154. SPE-
Lake, L. W. 2014. Enhanced Oil Recovery. Englewood Cliffs, New Jersey, USA: Prentice Hall.
Lerlertpakdee, P., Jafarpour, B., and Gildin, E. 2014. Efficient Production Optimization with Flow-Network Models. SPE J. 19 (6): 1083–1095. SPE-
Le Digabel, S. 2011. Algorithm 909: NOMAD: Nonlinear Optimization with the MADS Algorithm. ACM Trans Math Softw 37 (4): 44:1–44:15. https://
doi.org/10.1145/1916461.1916468.
Lie, K. A., Krogstad, S., Ligaarden, I. S. et al. 2012. Open Source MATLAB Implementation of Consistent Discretisations on Complex Grids. Comput
Geosci 16 (2): 297–322. https://doi.org/10.1007/s10596-011-9244-4.
Liu, Z. and Forouzanfar, F. 2018. Ensemble Clustering for Efficient Robust Optimization of Naturally Fractured Reservoirs. Comput Geosci 22:
283–296. https://doi.org/10.1007/s10596-017-9689-1.
Liu, Z. and Reynolds, A. C. 2019. An SQP-Filter Algorithm with an Improved Stochastic Gradient for Robust Life-Cycle Optimization Problems with
Nonlinear Constraints. Paper presented at the SPE Reservoir Simulation Conference, Galveston, Texas, USA, 10–11 April. SPE-193925-MS. https://
doi.org/10.2118/193925-MS.
Lukoševičius M. 2012. A Practical Guide to Applying Echo State Networks. In Neural Networks: Tricks of the Trade, ed. G. Montavon, G. B. Orr, and
K. R. Müller, Lecture Notes in Computer Science, Vol. 7700, Chapter 27, 659–686. Berlin, Germany: Springer. https://doi.org/10.1007/978-3-642-
35289-8_36.
MathWorks. 2020. sprand, https://www.mathworks.com/help/matlab/ref/sprand.html (accessed 19 May 2020).
Naevdal, G., Brouwer, D. R., and Jansen, J. D. 2006. Waterflooding Using Closed-Loop Control. Comput Geosci 10 (1): 37–60. https://doi.org/10.1007/
s10596-005-9010-6.
Pan, Y., Bi, R., and Zhou, P. et al. 2019a. An Effective Physics-Based Deep Learning Model for Enhancing Production Surveillance and Analysis in
Unconventional Reservoirs. Paper presented at the SPE/AAPG/SEG Unconventional Resources Technology Conference, Denver, Colorado, USA,
22–24 July. URTEC-2019-145-MS. https://doi.org/10.15530/urtec-2019-145.
Pan, Y., Deng, L., and Lee, J. 2019b. Data-Driven Deconvolution Using Echo-State Networks Enhances Production Data Analysis in Unconventional
Reservoirs. Paper presented at the SPE Eastern Regional Meeting, Charleston, West Virginia, USA, 15–17 October. SPE-196598-MS. https://
doi.org/10.2118/196598-MS.
Pan, Y., Zhou, P, Deng, L. et al. 2019c. Production Analysis and Forecasting for Unconventional Reservoirs Using Laplacian Echo-State Networks. Paper pre-
sented at the SPE Western Regional Meeting, San Jose, California, USA, 23–26 April. SPE-195243-MS. https://doi.org/10.2118/195243-MS.
Sarma, P., Durlofsky, L. J., and Aziz, K. 2008. Computational Techniques for Closed-Loop Reservoir Modeling with Application to a Realistic Reser-
voir. Pet Sci Technol 26 (10–11): 1120–1140. https://doi.org/10.1080/10916460701829580.
Scardapane, S. and Wang, D. 2017. Randomness in Neural Networks: An Overview. Wiley Interdiscipl Rev: Data Mining & Knowledge Discovery 7 (2):
1–18. https://doi.org/10.1002/widm.1200.
Stensgaard, A. D. D. 2016. Estimating the Value of Information Using Closed Loop Reservoir Management of Capacitance Resistive Models. MS thesis,
The Norwegian University of Science and Technology, Trondheim, Norway.
Sudaryanto, B. and Yortsos, Y. C. 2000. Optimization of Fluid Front Dynamics in Porous Media Using Rate Control. I. Equal Mobility Fluids. Physics
Fluids 12 (7): 1656–1670. https://doi.org/10.1063/1.870417.
Sudaryanto, B. and Yortsos, Y. C. 2001. Optimization of Displacement in Porous Media using Rate Control. Paper presented at the SPE Annual Techni-
cal Conference and Exhibition, New Orleans, Louisiana, USA, 30 September–3 October. SPE-71509-MS. https://doi.org/10.2118/71509-MS.
Sun, X., Cui, H., Liu, R. et al. 2012. Modeling Deterministic Echo State Network with Loop Reservoir. J Zhejiang Univ Sci C 13 (9): 689–701. https://
doi.org/10.1631/jzus.C1200069.
Temizel, C., Artun, E., and Yang, Z. 2019. Improving Oil-Rate Estimate in Capacitance/Resistance Modeling Using the Y-Function Method for Reser-
voirs under Waterflood. SPE Res Eval & Eng 22 (3): 1161–1171. SPE-194497-PA. https://doi.org/10.2118/194497-PA.
Timmerman, E. H. 1971. Predict Performance of Water Floods Graphically. Pet Eng 43 (12): 77–80.
Wang, C., Li, G., and Reynolds, A.C. 2009. Production Optimization in Closed-Loop Reservoir Management. SPE J. 14 (3): 506–523. SPE-109805-PA.
https://doi.org/10.2118/109805-PA.

Wang, Z., He, J., Milliken, W. J., et al. 2020a. Fast History Matching and Optimization Using a Novel Physics-Based Data-Driven Model: An Applica-
tion to a Diatomite Reservoir. Paper presented at the SPE Western Regional Meeting, Bakersfield, California, USA, 27 April–1 May 2020. SPE-
200772-MS. https://doi.org/10.2118/200772-MS.
Wang, Z., Malone, A., and King, M. J. 2020b. Quantitative Production Analysis and EUR Prediction from Unconventional Reservoirs Using a Data-
Driven Drainage Volume Formulation. Comput Geosci 24: 853–870. https://doi.org/10.1007/s10596-019-09833-8.
Zhao, H., Kang, Z., Zhang, X. et al. 2015. INSIM: A Data-Driven Model for History Matching and Prediction for Waterflooding Monitoring and Man-
agement with a Field Application. Paper presented at the SPE Reservoir Simulation Symposium, Houston, Texas, USA, 23–25 February. SPE-
173213-MS. https://doi.org/10.2118/173213-MS.

Sasd

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Sasd

Uploaded by

Copyright:

Available Formats

REE200862 DOI: 10.

2118/200862-PA Date: 16-October-20 Stage: Page: 1298 Total Pages: 16

* now with Quantum Reservoir Impact LLC

1298 November 2020 SPE Reservoir Evaluation & Engineering

ID: jaganm Time: 15:55 I Path: S:/REE#/Vol00000/200030/Comp/APPFile/SA-REE#200030

Noise Input System Output Noise

Fig. 1—Key elements of CLRM (after Jansen et al. 2009).

November 2020 SPE Reservoir Evaluation & Engineering 1299

ID: jaganm Time: 15:55 I Path: S:/REE#/Vol00000/200030/Comp/APPFile/SA-REE#200030

Fig. 2—Illustration of general ESN dynamic system.

x~ðtÞ ¼ ufWin ½1; uðtÞ þ Wxðt 1Þg; . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ð3Þ

1300 November 2020 SPE Reservoir Evaluation & Engineering

ID: jaganm Time: 15:55 I Path: S:/REE#/Vol00000/200030/Comp/APPFile/SA-REE#200030

November 2020 SPE Reservoir Evaluation & Engineering 1301

ID: jaganm Time: 15:55 I Path: S:/REE#/Vol00000/200030/Comp/APPFile/SA-REE#200030

Data-driven Koval fractional-

Fig. 3—Complete workflow of ESN-CLRM.

1302 November 2020 SPE Reservoir Evaluation & Engineering

ID: jaganm Time: 15:55 I Path: S:/REE#/Vol00000/200030/Comp/APPFile/SA-REE#200030

Fig. 4—Permeability field of the synthetic 2D model.

November 2020 SPE Reservoir Evaluation & Engineering 1303

ID: jaganm Time: 15:55 I Path: S:/REE#/Vol00000/200030/Comp/APPFile/SA-REE#200030

Prod 1 Prod 2 Prod 3

200 2,000 1,600

Liquid Production (STB/D)

1304 November 2020 SPE Reservoir Evaluation & Engineering

ID: jaganm Time: 15:55 I Path: S:/REE#/Vol00000/200030/Comp/APPFile/SA-REE#200030

Normalized True Data

Normalized True Data

Normalized True Data

November 2020 SPE Reservoir Evaluation & Engineering 1305

ID: jaganm Time: 15:55 I Path: S:/REE#/Vol00000/200030/Comp/APPFile/SA-REE#200030

Prod 1 Prod 2 Prod 3

Injection Rate (STB/D)

2,300 Base scenario

×106 ×106 ×106

Cumulative Water Injection (STB)

Optimized scenario 2.4 Optimized scenario 4.2 Optimized scenario

1.60 1.2 2.8

1306 November 2020 SPE Reservoir Evaluation & Engineering

ID: jaganm Time: 15:55 I Path: S:/REE#/Vol00000/200030/Comp/APPFile/SA-REE#200030

Inject 2 Prod 2 Inject 8

Fig. 13—Permeability field of the 3D egg model.

November 2020 SPE Reservoir Evaluation & Engineering 1307

ID: jaganm Time: 15:55 I Path: S:/REE#/Vol00000/200030/Comp/APPFile/SA-REE#200030

Prod 1 Prod 2 Prod 3 Prod 4

5,700 5,700 5,700 5,700

5,600 5,600 5,600 5,600

Injection Rate (STB/D)

Injection Rate (STB/D)

Injection Rate (STB/D)

650 650 650 650

600 600 600 600

Injection Rate (STB/D)

700 700 700 700

650 650 650 650

600 600 600 600

550 550 550 550

Training Validation Blind test

Liquid Production (STB/D)

1308 November 2020 SPE Reservoir Evaluation & Engineering

ID: jaganm Time: 15:55 I Path: S:/REE#/Vol00000/200030/Comp/APPFile/SA-REE#200030

Normalized True Data