energies
Article
Modeling Subsurface Performance of a Geothermal Reservoir
Using Machine Learning
Dmitry Duplyakin 1, * , Koenraad F. Beckers 1 , Drew L. Siler 2 , Michael J. Martin 1 and Henry E. Johnston 1






Citation: Duplyakin, D.; Beckers,
K.F.; Siler, D.L.; Martin, M.J.;
Johnston, H.E. Modeling Subsurface
Performance of a Geothermal
Reservoir Using Machine Learning.
Energies 2022, 15, 967.
https://doi.org/10.3390/en15030967
Academic Editors: Maruti
Kumar Mudunuru, Kalyana
Babu Nakshatrala and Zhao Hao
Received: 30 December 2021
Accepted: 27 January 2022
Published: 28 January 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Energies 2022, 15, 967. https://doi.org/10.3390/en15030967
1 National Renewable Energy Laboratory, Golden, CO 80401, USA; koenraad.beckers@nrel.gov (K.F.B.);
michael.martin@nrel.gov (M.J.M.); henry.johnston@nrel.gov (H.E.J.)
2 U.S. Geological Survey, Moffett Field, CA 94035, USA; dsiler@usgs.gov
* Correspondence: dmitry.duplyakin@nrel.gov
Abstract: Geothermal power plants typically show decreasing heat and power production rates over
time. Mitigation strategies include optimizing the management of existing wells—increasing or
decreasing the fluid flow rates across the wells—and drilling new wells at appropriate locations. The
latter is expensive, time-consuming, and subject to many engineering constraints, but the former
is a viable mechanism for periodic adjustment of the available fluid allocations. In this study, we
describe a new approach combining reservoir modeling and machine learning to produce models
that enable such a strategy. Our computational approach allows us, first, to translate sets of potential
flow rates for the active wells into reservoir-wide estimates of produced energy, and second, to
find optimal flow allocations among the studied sets. In our computational experiments, we utilize
collections of simulations for a specific reservoir (which capture subsurface characterization and
realize history matching) along with machine learning models that predict temperature and pressure
timeseries for production wells. We evaluate this approach using an “open-source” reservoir we
have constructed that captures many of the characteristics of Brady Hot Springs, a commercially
operational geothermal field in Nevada, USA. Selected results from a reservoir model of Brady Hot
Springs itself are presented to show successful application to an existing system. In both cases,
energy predictions prove to be highly accurate: all observed prediction errors do not exceed 3.68%
for temperatures and 4.75% for pressures. In a cumulative energy estimation, we observe prediction
errors that are less than 4.04%. A typical reservoir simulation for Brady Hot Springs completes in
approximately 4 h, whereas our machine learning models yield accurate 20-year predictions for
temperatures, pressures, and produced energy in 0.9 s. This paper aims to demonstrate how the
models and techniques from our study can be applied to achieve rapid exploration of controlled
parameters and optimization of other geothermal reservoirs.
Keywords: geothermal reservoir modeling; machine learning; energy predictions; geothermal
reservoir management
1. Introduction
Geothermal energy technology is continuing to evolve to provide heat and electricity
for both industrial and residential applications [1–3]. The need to provide low-carbon
electricity and heat has increased interest in geothermal energy as a significant contributor
to the low-carbon energy mix [4]. Geothermal energy is projected to contribute around
2–3% of global electricity generation by 2050 [5] and produce approximately 5% of global
heat load [6]. However, significant environmental, economic, and social challenges re-
main [7,8].
Application of machine learning (ML) and artificial intelligence (AI) methods has the
potential to improve the economic competitiveness of geothermal energy while reducing
the environmental impact. ML has already been successfully applied to the identification
of lithologies [9], extraction of hidden yet important features in hydrological data [10],
https://www.mdpi.com/journal/energies
Energies
Energies2022,
2022,15,
15,x967
FOR PEER REVIEW 22ofof20
20

generation
generationofofgeothermal
geothermalpotential
potentialmaps
mapsbased
basedon ongeological
geologicalandandgeophysical
geophysicaldata data[11],
[11],
identification
identification of key production-associated geologic factors [12,13], and creation ofdigital
of key production-associated geologic factors [12,13], and creation of digital
twins
twinsofofgeothermal
geothermalpower
powerplants
plants[14].
[14].
ML
MLcan canalso
alsobebeused
usedtotooptimize
optimizethethemanagement
managementofofgeothermal
geothermalreservoirs,
reservoirs,allowing
allowing
for
formaximum
maximumpower powerproduction
productionwhile
whileminimizing
minimizingthe theremoval
removalofofwater
waterfrom
fromthe thesystem.
system.
However,
However,geothermal
geothermalreservoirs
reservoirsarearelow-data
low-datasystems;
systems;fieldfielddata
dataisisdifficult,
difficult,ififnot
notimpos-
impos-
sible,
sible, to obtain in large enough quantities to provide a training set for ML. However,an
to obtain in large enough quantities to provide a training set for ML. However, an
alternate
alternateapproach,
approach,based
basedon onreservoir
reservoirsimulation,
simulation,isisavailable.
available.Reservoir
Reservoirengineers
engineershave have
decades
decadesofofexperience
experienceininusing
usingaacombination
combinationof ofgeologic
geologicmeasurements
measurementsand andreservoir
reservoir
outputs
outputsto tocreate
createphysics-based
physics-based numerical
numerical simulations
simulations of of reservoirs. These reservoir simu- sim-
ulations can then
lations can then be used to predict the performance
performance of of aa geothermal
geothermalreservoir
reservoirover overtime
time
undervarious
under variousoperating
operatingconditions.
conditions.As Aslarge
largeensembles
ensemblesofofsimulations
simulationscan canbebeperformed
performed
overaarange
over rangeof of operating
operatingconditions,
conditions,these
thesephysics-based
physics-basedreservoir
reservoirsimulations
simulationscan canbe be
usedtotocreate
used createdata
data sets
sets that
that enable
enable MLML in ways
in ways fieldfield
datadata
alonealone
would would
not benotable beto.
able
One to.
One potential
potential application
application is to is to create
create reduced-order
reduced-order models
models that
that allow
allow therapid
the rapidprediction
prediction
andoptimization
and optimizationofofsystem
systemperformance.
performance.This Thisisisthe
theapproach
approachthat thatwewearearetaking
takingininthe the
currentwork.
current work.ItsItskey
keysteps
stepsare
areshown
shownin inFigure
Figure1.1.

Field data High-

Traditional Geologic
& Fidelity
Reservoir experience
Models Reservoir
Engineering Models

Optimal
Ensembles
Machine Trained Model of
Learning Models for Training Simulation
Decision
Data
Making

Figure
Figure1.1.The
Theprocess
processthat
thatcombines
combinesreservoir
reservoirsimulations
simulationsand
andmachine
machinelearning.
learning.

Ouranalysis
Our analysisisisfocused
focusedon onBrady
BradyHot HotSprings
Springs(BHS),
(BHS),aageothermal
geothermalfield fieldininnorthern
northern
Nevada, USA, near the town of Fernley. Initial geothermal exploration
Nevada, USA, near the town of Fernley. Initial geothermal exploration in the late 1950s in the late 1950s
andearly
and early1960s
1960sled
ledtotothe
theconstruction
constructionofofaafoodfoodprocessing
processingfacility
facilityininthe
thelate
late1970s
1970sthat
that
usedgeothermal
used geothermalhot hotwater
waterfor
foronion
oniondehydration.
dehydration.AAgeothermal
geothermalflash flashpower
powerplant
plantwas
was
builtininthe
built the1990s,
1990s,and
andan anadditional
additionalbinary
binarycycle
cyclepower
powerplant
plantwaswasbuilt
builtin inthe
the2000s.
2000s.The
The
current combined nameplate capacity is 26 MWe (Megawatts electric)
current combined nameplate capacity is 26 MWe (Megawatts electric) [15]. The geother- [15]. The geothermal
power
mal plants
power are are
plants ownedownedandand
operated by Ormat
operated by OrmatTechnologies
Technologies Inc. Inc.
(Reno, NV,NV,
(Reno, USA).USA).
The hydrothermal system at BHS is controlled by a north-northeast-striking,
The hydrothermal system at BHS is controlled by a north-northeast-striking, west- west-
dipping basin-and-range-type fault system. Geologic mapping
dipping basin-and-range-type fault system. Geologic mapping [16] and paleoseismic [16] and paleoseismic
studies[17]
studies [17]indicate
indicatethat
thatthe
thefault
faultsystem
systemhas hasbeen
beenactive
activeininthe
the last
last ~15,500
~15,500years.
years.Fluid
Fluid
advectionisiscontrolled
advection controlledlocally
locallybybyaa~1-km-wide
~1-km-wideleft leftstep
stepininthe
thefault
faultsystem,
system,where
wherefracture
fracture
permeability in the fault system is maximized [18–21]. There is a three-dimensional
permeability in the fault system is maximized [18–21]. There is a three-dimensional geo- geologic
map [22] of BHS.
logic map [22] of BHS.
We study BHS through a large number of subsurface simulations, run using the reser-
We study BHS through a large number of subsurface simulations, run using the res-
voir simulators STARS by the Computer Modelling Group Ltd. (CMG, Alberta, Canada)
ervoir simulators STARS by the Computer Modelling Group Ltd. (CMG, Alberta, Canada)
and TETRAD. These simulations provide the data used for ML to inform Ormat, the
and TETRAD. These simulations provide the data used for ML to inform Ormat, the res-
reservoir operator. To avoid the publication of competition-sensitive data in the scientific
ervoir operator. To avoid the publication of competition-sensitive data in the scientific
literature, we created a second reservoir model by modifying the subsurface characteristics
literature, we created a second reservoir model by modifying the subsurface characteris-
of BHS. These modifications include changing well locations and modifying temperature
tics of BHS. These modifications include changing well locations and modifying temper-
and permeability characteristics. The modified system, with the generated synthetic, yet
ature and permeability characteristics. The modified system, with the generated synthetic,
realistic, data, will be referred to as the open-source reservoir (OSR). For the OSR, we present
yet realistic, data, will be referred to as the open-source reservoir (OSR). For the OSR, we
all relevant data and analysis artifacts. In contrast, for BHS, we only publish and discuss
present all relevant data and analysis artifacts. In contrast, for BHS, we only publish and
Energies 2022, 15, 967 3 of 20

scaled, sufficiently anonymized results and results that characterize the quality of the
applied ML algorithms rather than the reservoir itself.
In our previous work [23,24], we relied solely on ML for timeseries prediction [25] and
used neural network (NN) architectures such as multilayer perceptron (MLP), long short-
term memory (LSTM), and convolutional neural networks (CNN). While these approaches
showed satisfying results, we continued searching for ways to increase the efficiency of the
model training and, specifically, to avoid using single-step prediction schemes recursively
(which potentially leads to an accumulation of errors). As a result, we have developed
a technique that includes different processing of the simulation data followed by direct
mapping of the selected well flows to the sets of coefficients that define the curves for the
timeseries being modeled. We call this approach curve prediction and discuss it further in
the Methodology section of this paper.
The key contributions of the current study are the following:
• We show the validity of the proposed ML-based modeling by applying it to two geothermal
reservoirs (BHS and OSR), one real and one artificially constructed (synthetic but upholding
realistic constraints and properties).
• We document all computational activities—from the preprocessing of training data
to the analysis of prediction accuracy and the estimation of reservoir-wide energy
characteristics—and release the developed code artifacts.
• We demonstrate how ML models can be used in the search for optimal well flow rates,
resulting in increased energy production, and quantify the possible gains.
• For both BHS and OSR, we perform sensitivity analysis, which allows us to reason
about the reliability of our predictions and describe our findings.
The remainder of the paper is organized as follows. Section 2 introduces the data that
are generated and used in this study. In Section 3, we describe the design and implementa-
tion of our computational pipeline for modeling geothermal reservoirs. Sections 4 and 5
focus on the results of our analysis for OSR and BHS, respectively. In Section 6, we share
several high-level perspectives about our work. We conclude in Section 7.

2. Data Set Creation

The term “model” has different meanings to the reservoir engineering and ML com-
munities. In reservoir engineering, the term “model” is frequently used to describe an
estimate of the subsurface conditions in the reservoir. In ML, the term “model” is frequently
used to describe the process of estimating the outputs of a system from the known inputs.
To avoid confusion, the term “geologic modeling” or “reservoir modeling” will be used to
describe estimates of subsurface conditions based on reservoir engineering, while the term
“model” will be used to refer to ML-based predictions.

2.1. Role of Simulation in Data Set Creation

The role of subsurface models in generating the data sets for this study is shown in
Figure 2. The simulation takes three sets of inputs: (1) a geologic model of the reservoir,
such as the structure of fractures and the permeability and porosity at each location; (2) the
location and depth of the injection and production wells; and (3) the flow rate and fluid
temperature of each injection. The simulation, which can be performed using commercial
reservoir engineering packages such as TETRAD [26] or CMG STARS [27], then solves
mass, momentum, and conservation equations for the system to provide the output flow
rates, temperatures, and pressures for each of the production wells. These simulations are
grouped together in the “Traditional Reservoir Engineering” portion of Figure 1, and they
provide the data for the ML steps from Figure 1, which take place after completion of the
numerical simulations illustrated in Figure 2.
rgies 2022, 15, x FOR PEER REVIEW 4 of 20

Energies 2022, 15, x FOR PEER REVIEW 4 of 20

Energies 2022, 15, 967 4 of 20
Subsurface Injection and
fractures, production
porosity, and well locations
temperatures Subsurface
and depths Injection and
fractures, production
porosity, and well locations
temperatures and depths

Injection well Numeric solution of Production well

temperatures mass, momentum, and temperatures and
and flow rates energy equations flow rates
Injection well Numeric solution of Production well
temperatures mass, momentum, and temperatures and
and flow rates energy equations flow rates

Figure 2. Inputs and outputs for high-fidelity reservoir models.

Figure 2. Inputs and outputs for high-fidelity reservoir models.
2.2. Subsurface Characterization
Figure and History
2. Inputs and outputs Matching reservoir models.
for high-fidelity
As part2.2. Subsurface
of this project,Characterization
we upgraded Ormat’s and History Matching
existing BHS static geothermal reservoir
2.2.
model and usedAs Subsurface
part
it to Characterization
of this project,
generate ML training and History
we upgraded Matching
data. TheOrmat’s
system existing
has fourBHS static wells
injection geothermal
and reservoir
model
six production As andWe
part
wells. used
of this itproject,
modifiedto generate ML
we upgraded
reservoir training
fractureOrmat’s
flow data. The
existing
paths, system
BHS hasgeothermal
static
permeabilities, four
and injection wells
po- reservoir
rosities andandransix
model and
86 production
used it towells.
simulations usingWe
generate modified
theML training
TETRAD reservoir
data. The
model tofracture
system
improve flowhas paths, permeabilities,
fourmatching
history injection and
wells and
from 1979sixporosities
production
through and ran
2020.wells. 86 simulations using
We modified reservoir
The best-performing the TETRAD
model fracture
obtainedflow model
a mean to improve
paths,absolute history
permeabilities, matching
error and po-
6from
(MAE) of rosities
°C for 1979
and through
ran 86 2020.
production The best-performing
simulations
temperatures using
acrosstheall model model
TETRAD
production obtained to aimprove
wells. mean
The absolute
model per-error
history (MAE)
matching
of 6 ◦ C for production temperatures across all production wells. The model performed
formed wellfrom 1979 through
in matching 2020.results
the tracer The best-performing
from the tracer model obtainedina2019,
test conducted meanwhereabsolute error
well inof
(MAE)
multiple conservative matching
6 °C for the
tracers tracer
production
were results
injected in from
temperatures the across
tracerwells
four injection test conducted
all production
and their responsein 2019,The
wells. where
wasmodel multiple
per-
conservative
formed well intracers
matchingwere injected
the tracer in four
results injection
from the
monitored in several production wells. The updated reservoir model also closely matches wells
tracerand their
test response
conducted inwas monitored
2019, where
in several
multiple
the early 2021 temperature production
conservativesurvey wells.
tracers The updated
were
(on average within reservoir
injected in
+/−four
3 °C) model
injection
in also closely
wells
a 4000-ft-deep matches
and observa-
their response the early
was
2021 temperature survey (on average within +/ − 3 ◦ C) in a 4000-ft-deep observation well
monitored
tion well that in severalasproduction
was conducted part of thiswells.
study. The Weupdated
refer toreservoir
this set of model also closely
simulations as matches
that
BHS data. the
A was2021
early
second conducted
temperature
reservoir as part
model, inofwhich
survey this(on
study.
average
the We locations,
well refer
within to+/−
this3 set
°C)of insimulations
temperature, a 4000-ft-deep
and per- as BHS data.
observa-
A second
tion well that was
meability characteristics reservoir model,
wereconducted
arbitrarilyasin which the
part of this
changed, well locations,
study. to
is referred We asrefertemperature,
the OSR.to thisThe setOSRand permeability
of simulations
nu- as
characteristics
BHS data. A were
second arbitrarily
reservoir changed,
model, in is
which referred
the
merical model in CMG, color-coded by initial temperature, is shown in Figure 3. The do- wellto as the
locations, OSR. The OSR
temperature, numerical
and per-
modelare
meability
main dimensions incharacteristics
CMG,
19 km ×color-coded
10 kmwere by
km.initial
× 3.5arbitrarily
A “hot temperature,
changed,
spot” is is is shown
referred
visible neartothe incenter
as Figure
the of3.the
OSR. TheOSR
The domain
nu-
dimensions
merical model arein 19
CMG, × 10 km × 3.5
km color-coded byandkm. Atemperature,
initial “hot spot” isisvisible shown near the center
in Figure of the
domain, where labels indicate locations of wells boundary conditions (fumaroles at 3. The do-
domain, where labels indicate
km × 10locations
km × 3.5ofkm. wells and boundary conditions (fumaroles
center ofatthe
the
the surfacemain
and dimensions
sources at the arebottom).
19 A “hot spot” is visible near the
surface and sources at the bottom).
domain, where labels indicate locations of wells and boundary conditions (fumaroles at
the surface and sources at the bottom).

3. CMG reservoir domain ◦

Figure 3. CMGFigure
reservoir domain of OSR. The of OSR.represent
colors The colors represent
initial initial temperatures
temperatures (in °C). Gray(in C). Gray
labels above labels above
the field theatfield
point point at wells.
the existing the existing wells.domain
The shown The shown
is 19domain
km × 10iskm 3.5×km.
19 ×km 10 km × 3.5 km.
Figure 3. CMG reservoir domain of OSR. The colors represent initial temperatures (in °C). Gray
2.3. Data Set Generation
2.3. Data Set Generation
labels above the field point at the existing wells. The shown domain is 19 km × 10 km × 3.5 km.
To generate the data needed for this study, we ran 80 simulations for BHS and com-
To generate the data needed for this study, we ran 80 simulations for BHS and com-
pleted
Data101
Setsimulations for OSR, all covering the 20-year period from 2020 to 2040. We chose
pleted 1012.3.
simulations Generation
for OSR, all covering the 20-year period from 2020 to 2040. We
to run 80 and 101 simulations with the idea of having two slightly different data points for
chose to run 80To generate
and the data needed
101 simulations foridea
with the this study, we ran
of having two80 simulations
slightly for data
different BHS and com-
the size of the training data for ML rather than studying a single size. At the same time,
pleted 101 simulations for OSR, all covering the 20-year period from 2020 to 2040. We
these numbers are large enough to represent a wide variety of scenarios, but not too large to
chose to run 80 and 101 simulations with the idea of having two slightly different data
become computationally intractable, considering the time required to run each simulation.
Energies 2022, 15, x FOR PEER REVIEW 5 of 20

points for the size of the training data for ML rather than studying a single size. At the
Energies 2022, 15, 967 same time, these numbers are large enough to represent a wide variety of scenarios,5but of 20

not too large to become computationally intractable, considering the time required to run
each simulation. The flow constraints were imposed upon individual wells, and the flow
rates
The varied within thewere
flow constraints specific ranges,
imposed as shown
upon in Table
individual 1. and
wells, The the
injection temperature
flow rates was
varied within
the specific
about ranges,
70 °C, and as shownpressure
the injection in Table varied
1. The with
injection temperature
the flow rate. was about 70 ◦ C, and
the injection pressure varied with the flow rate.
Table 1. Ranges of the well flows used in OSR simulations.
Table 1. Ranges of the well flows used in OSR simulations.
Simulated Flow Rate (kg/day)
Well Type
Min Simulated Flow RateMax
(kg/day)
Well Type
I1 Injection 3,409,880 Min 10,229,650
Max
I2 I1 Injection Injection 5,056,776 3,409,880 15,170,330
10,229,650
I3 I2 Injection Injection 10,982,070 5,056,776 32,946,200
15,170,330
I4 I3 Injection Injection 5,389,298 10,982,070 32,946,200
16,167,890
P1 I4 Production Injection 7,522,530 5,389,298 16,167,890
22,567,590
P1 Production 7,522,530 22,567,590
P2 P2
Production Production
3,641,546 3,641,546 10,924,640
10,924,640
P3 P3 Production Production 3,926,900 3,926,900 11,780,710
11,780,710
P4 P4 Production Production 3,906,016 3,906,016 11,718,050
11,718,050
P5 P5 Production Production 7,680,623 7,680,623 23,041,870
23,041,870
P6 Production 3,555,400 10,666,210
P6 Production 3,555,400 10,666,210

AArepresentative
representativetime
timehistory
historyfor
forthe
theOSR
OSRmodel
modelisisshown
shownininFigure
Figure4.4.The
Theshown
shown
temperatureprofiles
temperature profilesvisualize
visualizeone
oneexample
examplefrom
fromthe
thesimulated
simulatedcases
caseswe
wehave
havereleased
releasedasas
partofofthe
part theOSR
OSRdataset,
dataset,asasdetailed
detailedunder
underthe
theData
DataAvailability
AvailabilityStatement
Statementnear
nearthe
theend
endofof
thepaper.
the paper.

Figure
Figure4.4.Production
Productiontemperature
temperature profiles for
for the
thesix
sixOSR
OSRproduction
productionwells,
wells,
P1P1 through
through P6,P6, simu-
simulated
lated for 2020–2040
for 2020–2040 (simulated
(simulated years
years areare shown
shown alongthe
along thex-axes).
x-axes). Simulations were
wereconducted
conductedusing
using
CMG
CMGSTARS.
STARS.

InInour
ourwork
workweweassume
assumeno nochange
changeininexternal
externalconditions.
conditions.Any
Anychange
changeininthetheabove-
above-
ground
groundtemperature
temperatureand
andprecipitation
precipitationpatterns
patternsisisconsidered
consideredtotohave
haveaaminor
minorimpact
impacton on
subsurface
subsurfacephysics
physics and geothermal
geothermalenergy
energyproduction
productioncomparing
comparing to to
thethe injection
injection wellwell
flow
allocations
flow being
allocations studied.
being Potential
studied. impacts
Potential of climate
impacts change,
of climate change,with variations
with variationsin in
surface
sur-
temperatures
face andand
temperatures precipitation, are are
precipitation, not not
explicitly considered
explicitly in this
considered work.
in this work.

3. Design and Implementation of Computational Pipeline

In this section, we detail our ML-based approach—from model training to inference—
and discuss the nuances of all required data processing steps. We also describe the tech-
nologies that we leveraged when we turned the proposed ideas into code.
 3. Design and Implementation of Computational Pipeline
In this section, we detail our ML-based approach—from model training to infer-
Energies 2022, 15, 967 6 of 20
ence—and discuss the nuances of all required data processing steps. We also describe the
technologies that we leveraged when we turned the proposed ideas into code.
We trained our ML models to capture relationships in the reservoir data sets—more
We trained
specifically, in theour ML models
timeseries to capture
produced relationships
by the simulations in with
the reservoir data sets—more
the aforementioned sub-
specifically,
surface in the
models. timeseries
Thus, produced
after the necessaryby the simulations
training, with theare
these models aforementioned
capable of reproduc- subsur-
facethose
ing models. Thus, after
relationships andtheproducing
necessaryestimates
training, of these
the models
quantities areofcapable
interestof(e.g.,
reproducing
pressure
those relationships and producing estimates of the quantities of interest
and temperature values and then, after additional postprocessing, energy estimates), even (e.g., pressure and
temperature values and then, after additional postprocessing, energy
for the scenarios that were not part of the training data sets. To be clear, these new scenar- estimates), even for
the scenarios that were not part of the training data sets. To be clear,
ios should be similar to the ones in the training data, in the sense that the same key phys- these new scenarios
should
ical be similarare
phenomena to in
theeffect
onesforin the
the training
modeleddata, in the
system; sense that
otherwise, thethe same key physical
predictions may be-
phenomena
come are inFor
inaccurate. effect for theifmodeled
instance, system;
new scenarios otherwise,
allow previouslythe predictions
inactive wells may become
to become
inaccurate. For instance, if new scenarios allow previously inactive
active, the models cannot adequately predict the outcomes without a sufficient amount wells to become active,
of
the models cannot adequately predict the outcomes without a sufficient
representative training data. At the same time, for a fixed number of active wells, it should amount of rep-
resentative
be possible training
to vary the data. At the same
allocation and gettime, for a fixed
accurate modelnumber of active wells,
predictions—this it should
is exactly the
be possible to vary the allocation and
hypothesis that we are validating through this study. get accurate model predictions—this is exactly the
hypothesis that we are validating through this study.
It is worth highlighting the ways in which our current ML work differs from our
It is worth
previous efforts highlighting the ways
[23,24]. Previously, weinstarted
whichmodeling
our current BHSMLbywork differs ML
leveraging fromtech-
our
previous efforts [23,24]. Previously, we started modeling BHS by
niques. Currently, we use an approach based on a fully interconnected NN with two hid- leveraging ML techniques.
Currently,
den layerswe (of use an approach
varying based are
sizes), which on atrained
fully interconnected NN with of
to predict coefficients two hidden
the curveslayers
that
(of varying sizes), which are trained to predict coefficients of the curves that approximate
approximate the timeseries of interest. This approximation conveniently serves the func-
the timeseries of interest. This approximation conveniently serves the function of data
tion of data reduction, and the goal of the NN becomes more computationally tractable.
reduction, and the goal of the NN becomes more computationally tractable. Thus, multi-
Thus, multi-headed NNs (i.e., networks with multiple neurons in the output layers) can
headed NNs (i.e., networks with multiple neurons in the output layers) can predict all
predict all required coefficients in a single pass. In contrast, our previous work was fo-
required coefficients in a single pass. In contrast, our previous work was focused on the
cused on the evaluation of MLP, LSTM, and CNN network architectures in the context of
evaluation of MLP, LSTM, and CNN network architectures in the context of timeseries
timeseries prediction, with numerous single-step predictions. In Appendix B, we further
prediction, with numerous single-step predictions. In Appendix B, we further compare and
compare and contrast these approaches and share information about the parameters of
contrast these approaches and share information about the parameters of the corresponding
the corresponding NNs. We further note that the timeseries prediction from our previous
NNs. We further note that the timeseries prediction from our previous work was similar to
work was similar to the ML methods applied to oil production modeling [28].
the ML methods applied to oil production modeling [28].
In the current formulation, the ML models’ goal is to find an accurate and reliable
In the current formulation, the ML models’ goal is to find an accurate and reliable
mapping
mapping between
between 10 10 flow
flow levels
levels (for
(for four
four injection
injection wells
wells and
and six
six production
production wells—these
wells—these
numbers are the same for BHS and OSR) and D + 1 coefficients
numbers are the same for BHS and OSR) and D + 1 coefficients of the curves of the curves for theforpoly-
the
nomial
polynomial approximation
approximation of degree D. If we
of degree D. Ifwere to consider
we were each of
to consider eachthese
of coefficients sepa-
these coefficients
rately, we could
separately, modelmodel
we could them individually
them individually using regression techniques
using regression (ML-based
techniques or even
(ML-based
simpler
or even simpler fitting techniques). However, for modeling entire combinations ofcoeffi-
fitting techniques). However, for modeling entire combinations of these these
cients, we choose
coefficients, to leverage
we choose multi-headed
to leverage multi-headedNN. NN.

3.1. Learning
Learning and and Inference
Inference
We perform
performmodel
modeltraining
trainingononthe
the simulation
simulation data
data and
and create
create multiple
multiple ML ML models,
models, one
one for each
for each quantity
quantity we want
we want to predict.
to predict. ThisThis includes
includes modelsmodels for temperature
for temperature andand pres-
pressure
sure timeseries
timeseries for of
for each each
the of the production
production wells.
wells. In In 5,Figure
Figure 5, wethis
we depict depict this workflow,
workflow, listing all
listing all preprocessing
preprocessing and modeland model steps.
handling handling steps.

Figure 5. Computational tasks involved in creation of ML models.

Next, we
Next, we discuss
discuss each
each of
of these
these steps
steps in
in further
further detail.
detail.
• Renaming: Since TETRAD and CMG STARS produce data in the form of tables that
have different column names, there is a need for input standardization. We map names
like “Flow PXX (kkg/h)” onto names like “pm1” (i.e., mass flow for production #1).
This mapping can be undone at the end of the pipeline to present the results in the
original terms.
Energies 2022, 15, 967 7 of 20

• Time conversion: We map individual dates in the studied timeseries, which represent
the time intervals between 2020 and 2040, onto points in the [0, 1] range. Similar to
the previous step, original dates can be obtained at the end by inverting this linear
transformation. We use this time conversion step to make sure that the following ML
process remains general enough, supporting time periods of arbitrary lengths.
• Scaling: For each input variable, we scale the values to the [0, 1] range, using global
minimum and maximum values for all values of the corresponding type: temperature,
pressure, or mass flow. In other words, all temperature timeseries are scaled using the
same minimum and maximum values found in the arrays of historic and simulated
data; the same is true for the pressure and mass flow scaling. This type-specific scaling
allows us to preserve data relationships that would otherwise be lost if each timeseries
was scaled irrespective of the rest of the data.
• Input consolidation: Although each flow rate can change over time in an arbitrarily
complex way, we consider a simpler case: a step function according to which the
historic flow rate changes to the new, studied value at the beginning of the simulated
period. Obviously, a sequence of flow adjustments can be represented by a series of
such steps. Using this consolidation, we create the input data set for ML training,
which takes the shape of N values for N wells for which the flow can be controlled.
These N values represent the individual flows for the active injection and production
wells and together constitute the new flow allocation. We refer to the individual unique
allocations as the studied scenarios. For C scenarios in the training data, we end up
with a C * N matrix, which becomes the input for ML training (typically referred to as
matrix X in ML terms).
• Polynomial approximation for output: Similar to input consolidation, we are looking
for a way to represent a lengthy timeseries of interest with only a handful of numbers
that represent an approximation for a given timeseries with pressure or temperature
values for each well. Counting on the fact that we can decide how many such numbers
are going to be involved in this approximation in each specific case, we propose
obtaining these numbers from the values in the output layers of the NNs we are
going to train. A polynomial approximation meets the needs of this step for the
following reasons: (1) obtaining coefficients for a given timeseries’ approximation is
a computationally inexpensive task (typically performed using the method of least
squares); (2) reconstructing the timeseries using a set of polynomial coefficients is
also straightforward and fast; and (3) based on the timeseries for temperatures and
pressures obtained in the simulations we have inspected in this work, we conclude
that their smooth shapes can be approximated using polynomials very accurately.
Appendix A includes the figures that illustrate these shapes and characterize the
errors of the curve fitting. In summary, using a polynomial of degree D will result
in D + 1 coefficients, and, if used consistently across C scenarios, it will produce the
C * (D + 1) matrix that will be used in training (referred to as matrix Y in ML terms).
For each of the considered scenarios, a scaled approximated output value (temperature
or pressure), represented by yˆs and corresponding to the (scaled) moment of time
ts ∈ [0, 1], would be calculated as:

yˆs = a0 + a1 ts + a2 t2s + . . . + a D tsD ,

where a0 , . . . , a D are coefficients of the approximating polynomial.

• Model training: This refers to the optimization process that minimizes the specified
loss function. We use the MAE loss and rely on the code provided by TensorFlow [29],
a commonly used open-source platform for ML, in driving this optimization. Model
training takes place in both the cross-validation process, as explained below, and also
in training of the “production” models, i.e., the ones that are ready for final evaluation
and application in decision making.
Energies 2022, 15, x FOR PEER REVIEW 8 of 20

Energies 2022, 15, 967 8 of 20

Model training takes place in both the cross-validation process, as explained below,
and also in training of the “production” models, i.e., the ones that are ready for final
evaluation and application in decision making.
• Model evaluation and selection of models with smallest prediction errors: Each of
• Model evaluation and selection of models with smallest prediction errors: Each of the
the trained models is evaluated on the data that was not used in the training using
trained models is evaluated on the data that was not used in the training using the
the mean absolute percentage error (MAPE). This analysis of the models’ predictive
mean absolute percentage error (MAPE). This analysis of the models’ predictive abil-
ability allows us to choose the best parameters of the models (e.g., the size of the
ity allows us to choose the best parameters of the models (e.g., the size of the NNs
NNs being trained) and their training (e.g., the number of iterations in the training
being trained) and their training (e.g., the number of iterations in the training algo-
algorithm), which is the main goal of the cross-validation process. When the values of
rithm), which is the main goal of the cross-validation process. When the values of
those parameters are identified, we can proceed to train the production models.
those parameters are identified, we can proceed to train the production models.
In the inference phase, we use the trained production models, as shown in Figure 6.
In the inference phase, we use the trained production models, as shown in Figure 6.
We select a particular flow allocation to be studied, scale the flow values to make them con-
We select a particular flow allocation to be studied, scale the flow values to make them
sistent with the training data, and then use them as input for the pressure and temperature
consistent with the training data, and then use them as input for the pressure and temper-
models. The models produce predictions of the curve coefficients, which we then convert
ature models. The models produce predictions of the curve coefficients, which we then
into timeseries. At the end, those predicted timeseries are comparable to the timeseries
convert into timeseries. At the end, those predicted timeseries are comparable to the
in the simulation data, i.e., they represent the same 2020–2040 period and have unscaled
timeseries in the simulation data, i.e., they represent the same 2020–2040 period and have
values. The inference is repeated for as many cases as we need to study and can be run in
unscaled values. The inference is repeated for as many cases as we need to study and can
parallel (e.g., on several compute nodes on a supercomputer) for time efficiency.
be run in parallel (e.g., on several compute nodes on a supercomputer) for time efficiency.

Figure 6. The inference process and creation of timeseries predictions. “Q” could be a quantity such
Figure 6. The inference process and creation of timeseries predictions. “Q” could be a quantity
as temperature or pressure for one of the production wells. Specific numbers are shown in the dia-
such as temperature or pressure for one of the production wells. Specific numbers are shown in the
gram for illustrative purposes only.
diagram for illustrative purposes only.
After we obtain pressure and temperature predictions for individual wells, as de-
After we obtain pressure and temperature predictions for individual wells, as de-
scribed above, we can estimate reservoir-wide characteristics, such as the amounts of en-
scribed above, we can estimate reservoir-wide characteristics, such as the amounts of
ergy produced. Specifically, we calculate the total amount of produced energy (in units of
energy produced. Specifically, we calculate the total amount of produced energy (in units
W, later converted to MW) by summing up contributions from the active production
of W, later converted to MW) by summing up contributions from the active production
wells. One production well’s contribution is estimated as the difference between pro-
wells. One production well’s contribution is estimated as the difference between produced
duced enthalpy,
enthalpy, ℎ in
h p in J/kg, J/kg,
and and injected
injected enthalpy,
enthalpy, ℎ
hinj in J/kg in J/kg
(the (theislatter
latter kept is kept constant
constant for all
for all wells) multiplied by the flow rate for the well,
wells) multiplied by the flow rate for the well, m in kg/s: 𝑚 in kg/s:
p

𝑇𝑜𝑡𝑎𝑙 𝑃𝑟𝑜𝑑𝑢𝑐𝑒𝑑 𝑃𝑜𝑤𝑒𝑟 𝑡ℎ𝑒𝑟𝑚𝑎𝑙 = 𝑚 ℎ ℎ
.

Total Produced Power (thermal ) = ∑ m p h p − hinj .
production wells
In these calculations, we assume the dead-state condition to be at 20 ◦°C and 1 bar. In
In these calculations,
the presented weenthalpy
equation, the assume the
for dead-state condition
each production welltoisbeaatfunction
20 C and 1 bar.
of the In
pro-
the presented equation, the enthalpy for each production well is a function of
duced temperature and pressure, and we calculate them by using the thermo-physicalthe produced
temperature and pressure, and we calculate them by using the thermo-physical property
property tables for pure water, also known as the steam tables [30]. We integrate the cal-
tables for pure water, also known as the steam tables [30]. We integrate the calculated
culated power estimates over time to estimate the amount of cumulative thermal energy
power estimates over time to estimate the amount of cumulative thermal energy (in MWh)
(in MWh) produced between 2020 and 2040.
produced between 2020 and 2040.
4.
4. Studying
Studying the
the Open-Source
Open-Source Reservoir
Reservoir
We begin our OSR analysis
We begin our OSR analysis with with cross-validation
cross-validation for
for ML,
ML, as
as outlined
outlined in
in the
the previous
previous
section.
section. As demonstrated in Appendix C, we use the k-fold cross-validation technique [31]
As demonstrated in Appendix C, we use the k-fold cross-validation technique [31]
to select the
to select the best
best model
model andand training
training parameters
parameters for
for the studied data.
the studied data. Over
Over 8000
8000 “trial”
“trial”
models
models were
were fit
fit using
using aa computing
computing allocation
allocation on
on Eagle
Eagle [32],
[32], NREL’s
NREL’s supercomputer, with
supercomputer, with
the goal of finding the model configurations that would allow the models to generalize,
the goal of finding the model configurations that would allow the models to generalize, i.e.,
perform well on the unseen data. Although an evaluation of this kind may be computation-
ally intractable for models of larger sizes and larger data sets, we were able to perform a
grid-search-style optimization and assess 72 combinations of parameters that characterize
 acterize the models and the training process for 12 quantities of interest using just a frac-
tion of the Eagle’s allocation dedicated to this project. Thus, this cross-validation was com-
pleted after 39 h of model fitting and evaluation on one of Eagle’s compute nodes.
After cross-validation, we trained the models on the entire training set, with only the
test subset (a randomly selected 10%) being held out for final model evaluation. We fol-
Energies 2022, 15, 967 lowed the process depicted in Figure 6 to obtain pressure and temperature curves, which 9 of 20
we then compared with the simulation data in the test data set. In each case, we calculated
the MAPE, capturing the magnitude of relative difference. The distributions of these error
measures are shown in Figure 7. In the same figure, we show a visualization of one pre-
the models and the training process for 12 quantities of interest using just a fraction of the
dicted curve and how it closely follows the simulation data, which, for the purposes of
Eagle’s allocation dedicated to this project. Thus, this cross-validation was completed after
this evaluation, are treated as “ground truth.” To summarize, the errors in pressure pre-
39 h of model
dictions fitting
are in and evaluation
the range between 0.13% on one
and of Eagle’s
3.68%, withcompute
an averagenodes.
of 0.57%; similarly,
After cross-validation, we trained the models on the
for the errors in temperature predictions, the range is between 0.17% and entire training
4.75%,set,
andwith
the only
theaverage
test subset (a
is 1.34%. randomly selected 10%) being held out for final model evaluation. We
followed the process depicted in Figure 6 to obtain pressure and temperature
Using the selected trained models, we obtained energy estimates, as described in Sec- curves, which
wetion
then3.compared with thethe
We then calculated simulation data in energy
total cumulative the testproduced
data set.by
Inthe
each case,
end we 2040
of year calculated
theusing
MAPE,bothcapturing
the simulationthe data and our predictions,
magnitude of relativeand we compared
difference. Thethe two results. We
distributions of these
show
error our results
measures arefor the 11in
shown test cases in
Figure 7. Figure
In the 8. In the
same same figure,
figure, we show we also visualize oneof one
a visualization
specific curve
predicted test case
andandhowhowititclosely
develops over time,
follows in the interval
the simulation from
data, 2020–2040.
which, for theThese
purposes
results, where the average percent error is as low as 0.32% and the worst-case
of this evaluation, are treated as “ground truth.” To summarize, the errors in pressure error is
4.04%, indicate that the models fit the data very accurately. It is worth noting that in this
predictions are in the range between 0.13% and 3.68%, with an average of 0.57%; similarly,
analysis we averaged the real prediction errors—positive and negative (as opposed to ab-
for the errors in temperature predictions, the range is between 0.17% and 4.75%, and the
solute values)—obtained for the test cases, and the average of 0.32% indicates that the bias
average
in our is 1.34%.
predications is small.

Figure
Figure 7. (left)
7. (left) Summaryof
Summary of the
the models
models trained
trainedforfor
OSR andand
OSR evaluated on the
evaluated onunseen data (11
the unseen test(11 test
data
cases that were set aside before model training) and (right) one illustrative example of the curve
cases that were set aside before model training) and (right) one illustrative example of the curve
predicted for a selected test case.
predicted for a selected test case.

Using the selected trained models, we obtained energy estimates, as described in

Section 3. We then calculated the total cumulative energy produced by the end of year
2040 using both the simulation data and our predictions, and we compared the two results.
We show our results for the 11 test cases in Figure 8. In the same figure, we also visualize
one specific test case and how it develops over time, in the interval from 2020–2040. These
results, where the average percent error is as low as 0.32% and the worst-case error is 4.04%,
indicate that the models fit the data very accurately. It is worth noting that in this analysis
we averaged the real prediction errors—positive and negative (as opposed to absolute
values)—obtained for the test cases, and the average of 0.32% indicates that the bias in our
predications is small.
The error summaries shown in Figures 7 and 8 indicate that the trained NN produced
models accurately predict characteristics of OSR and generalize to the unseen data. It is also
evident that the amount of used training data (i.e., 101 simulated scenarios) is sufficient for
obtaining satisfying prediction accuracy.
 Energies2022,
Energies 2022,15,
15,967
x FOR PEER REVIEW 10ofof2020
10

Figure8.8.(left)
Figure (left)Relative
Relativeerrors
errorsininpredictions
predictionsfor
fortotal
totalcumulative
cumulativeenergy
energy(for
(forthe
theyear
year2040)
2040)and
and
(right) one illustrative example (observed case with the largest relative error) of cumulative thermal
(right) one illustrative example (observed case with the largest relative error) of cumulative thermal
energy produced as a function of time.
energy produced as a function of time.

Thefurther
We error summaries
characterized shown in Figures 7ofand
the robustness the 8trained
indicate that the
models bytrained NN produced
performing sensitiv-
models
ity accurately
analysis. The idea predict
here ischaracteristics
that, for goodofmodels,
OSR and generalize
relatively smallto changes
the unseen data.
in the It is
input
also evident that the amount of used training data (i.e., 101 simulated
should not lead to disproportionally high variability in the predicted metrics. Furthermore, scenarios) is suffi-
cient
by for obtaining
varying inputs one satisfying
at a time,prediction
we can detect accuracy.
where the prediction sensitivity is the highest
in relative terms, which would be informativeoftothe
We further characterized the robustness trained
anyone whomodels by performing
is interested in usingsensi-
this
type of model. To proceed with such analysis, we leveraged a formulation for thein“classical
tivity analysis. The idea here is that, for good models, relatively small changes the input
should not lead
one-at-a-time” to disproportionally
sensitivity index [33], defined high as
variability in the predicted
a partial derivative metrics.R,Further-
of the function which
more,the
takes byinput
varying inputs
S, and one at a at
is evaluated time, we can detect
a particular point inwhere the prediction
the input space S0 : sensitivity is
the highest in relative terms, which would be informative to anyone who is interested in
using this type of model. SI To proceed 0 + ∆S
[ R(Swith such R(S0 − ∆S
) − analysis, we)] leveraged a formulation for
RS = ,
the “classical one-at-a-time” sensitivity index2∆S [33], defined as a partial derivative of the
function R, which takes the input S, and is evaluated at a particular point in the input
where ∆S is the change applied to the input, and [ ] in the numerator denotes the absolute
space 𝑆 :
value. We adapted this formulation to our energy estimation as follows:
𝑅 𝑆0 + Δ𝑆 𝑅 𝑆0 Δ𝑆
𝑆𝐼𝑅𝑆 = ,
[ E2040 ( F + ∆F2Δ𝑆
) − E2040 ( F − ∆F )]
SI
where Δ𝑆 is the change appliedE = to the input,2∆F and ,
in the numerator denotes the abso-
lute value. We adapted this formulation to our energy estimation as follows:
where E2040 is the ML-based estimate of the cumulative
𝐸2040 𝐹 + Δ𝐹 𝐸2040 𝐹 Δ𝐹
amount of energy produced by
the end of the year 2040,𝑆𝐼F is one of the flow combinations, from the training set, and
𝐸 =
2Δ𝐹
∆F is the𝐸 change
where applied
is the to one
ML-based of the flows
estimate of the in the studied
cumulative combination.
amount of energyThe resulting
produced by
index characterizes the cumulative amount of thermal energy and has the unit of MWh
the end of the year 2040, F is one of the flow combinations from the training set, and Δ𝐹
per kg/day, because MWh and kg/day are the units used for energy values and flow rates
is the change applied to one of the flows in the studied combination. The resulting index
in this work, respectively. In Figure 9, we show the distributions of the sensitivity index
characterizes the cumulative amount of thermal energy and has the unit of MWh per
values, SIE , we obtained in evaluating this index on all 101 cases in the OSR data set. For all
kg/day, because MWh and kg/day are the units used for energy values and flow rates in
these cases, this index is between 0.12 and 0.87 MWh per kg/day, and the sensitivity is the
this work, respectively. In Figure 9, we show the distributions of the sensitivity index val-
highest (based on average SIE ) for production well 6, production well 5, and production
ues, 𝑆𝐼 , we obtained in evaluating this index on all 101 cases in the OSR data set. For all
well 1 (P6, P5, and P1 in the figure, respectively). Among the injection wells, SIE are more
these cases, this index is between 0.12 and 0.87 MWh per kg/day, and the sensitivity is the
alike, although the average is slightly higher for injection well 2 (I2).
highest (based on average 𝑆𝐼 ) for production well 6, production well 5, and production
well 1 (P6, P5, and P1 in the figure, respectively). Among the injection wells, 𝑆𝐼 are more
alike, although the average is slightly higher for injection well 2 (I2).
As an illustrative example, we can consider 𝑆𝐼 = 0.80 (an index value on the higher
end), which was observed for P6. This particular value refers to the instance in which P6′s
flow is varied by +/− 5%, and 𝐸 varies in response, going from 2.29 10 to 2.34
10 MWh, which is, in relative terms, only a 1.86% change. For a single-well change in
the input, this appears to be a reasonable and modest response in a metric such as the
 data increases. Our initial assessment of the observed 𝑆𝐼 values leads us to believe that
there is currently no problematic variability in the model outputs. This sensitivity analysis
is also an initial step toward bridging the gap of model interpretability, one of the common
shortcomings in ML work, and the effort can be further expanded through the use of tech-
niques such as individual conditional expectations and partial dependence plots [34]. Fur-
Energies 2022, 15, 967 thermore, shapely additive explanations [35] can help with interpretability and 11 of 20
reveal
more insights into relative feature importance.

9. Sensitivity
Figure 9. Sensitivity index
indexdistributions
distributionsfor
forOSR.
OSR.I1–I4
I1–I4refer
refertotothe injection
the wells,
injection and
wells, P1–P6
and refer
P1–P6 to
refer
the production
to the productionwells. In the
wells. histograms,
In the y-axis
histograms, values
y-axis characterize
values characterizethe the
number of occurrences
number of the
of occurrences of
the corresponding
corresponding x-axis
x-axis values
values (SIE(𝑆𝐼
). ).

As an illustrative
5. Application example, we
to a Real-World can consider SIE = 0.80 (an index value on the higher
Reservoir
end),Similar
which to was observed for P6.
our OSR analysis, we This particular
perform errorvalue refers
analysis for to
thethe instance in
predictions which
obtained
P6’s flow is varied by +/ − 5%, and E2040 varies in response, going from
using the7 models trained on the BHS data. Through cross-validation, we confirm that 2.29 × 107theto
2.34 × 10 MWh, which is, in relative terms, only a 1.86% change. For a single-well change
NN architecture and the chosen amount of training provide the best results among the
in the input, this appears to be a reasonable and modest response in a metric such as the
studied configurations. Results for the production models are shown in Figure 10. Predic-
aggregate E2040 estimate. We do not have reference numbers with which this result can be
tion errors can be summarized as follows. Predictions for pressure values show errors
compared, and we believe more work is needed to thoroughly study sensitivity measures
between 0.03% and 0.52%, with an average of 0.25%, and the temperature prediction er-
and, for instance, track how such measures change as the amount of training data increases.
rors are between 0.11% and 3.54%, with an average of 0.83%. For the cumulative energy
Our initial assessment of the observed SIE values leads us to believe that there is currently
estimates, we observe errors between 0.59% and 2.20%, with an average of 0.95% (we av-
no problematic variability in the model outputs. This sensitivity analysis is also an initial
eraged positive and negative error estimates to characterize bias). We notice that these
step toward bridging the gap of model interpretability, one of the common shortcomings
models show similar level of accuracy to the models we trained for OSR (see the results
in ML work, and the effort can be further expanded through the use of techniques such
presented in the previous section). Minor differences are likely originating from the fact
as individual conditional expectations and partial dependence plots [34]. Furthermore,
that one out of the 10 wells studied at BHS had zero flow (the model was developed for a
shapely additive explanations [35] can help with interpretability and reveal more insights
ten-well
into reservoir,
relative featurebut the data from the operator suggested that one specific well was no
importance.
longer considered active); thus, the BHS models focused on learning the relationships in
theApplication
5. data for a nine-well system,Reservoir
to a Real-World as opposed to the full 10-well system represented by the
OSR Similar
data. Forto our OSR analysis, we we
these two reservoirs, used error
perform the same hyperparameters
analysis (i.e., degree
for the predictions obtained of
approximating polynomial, sizes of NN layers, and amount of model
using the models trained on the BHS data. Through cross-validation, we confirm that training), but the
actual
the NNtraining (selection
architecture of model
and the chosenweights)
amountwas performed
of training for each
provide reservoir
the best results and each
among
wellstudied
the independently from the Results
configurations. rest. There
for might be additional
the production insights
models comingin
are shown from a thor-
Figure 10.
Prediction errors can be summarized as follows. Predictions for pressure valuesofshow
ough statistical comparison of the two reservoirs’ models, yet for the purposes this
errors between 0.03% and 0.52%, with an average of 0.25%, and the temperature prediction
errors are between 0.11% and 3.54%, with an average of 0.83%. For the cumulative energy
estimates, we observe errors between 0.59% and 2.20%, with an average of 0.95% (we
averaged positive and negative error estimates to characterize bias). We notice that these
models show similar level of accuracy to the models we trained for OSR (see the results
presented in the previous section). Minor differences are likely originating from the fact
that one out of the 10 wells studied at BHS had zero flow (the model was developed for
a ten-well reservoir, but the data from the operator suggested that one specific well was
no longer considered active); thus, the BHS models focused on learning the relationships
in the data for a nine-well system, as opposed to the full 10-well system represented by
the OSR data. For these two reservoirs, we used the same hyperparameters (i.e., degree
Energies 2022, 15, 967 12 of 20

Energies 2022, 15, x FOR PEER REVIEW 12 of 20

of approximating polynomial, sizes of NN layers, and amount of model training), but
the actual training (selection of model weights) was performed for each reservoir and
each well independently from the rest. There might be additional insights coming from a
thorough
study, statistical
we find comparison
it sufficient of the twothat
to demonstrate reservoirs’ models,
all prediction yet for
errors wethe purposes
have observedof this
for
study,
two we find are
reservoirs it sufficient
relativelyto small
demonstrate
(under that all prediction
4.75%). Moreover,errors we have
we could haveobserved
made BHS for
two reservoirs are relatively small (under 4.75%). Moreover, we could have
and OSR match exactly in terms of the numbers of simulations and active wells, but in- made BHS and
OSR match
stead exactly
we chose in terms
in favor of the numbers
of studying of simulations
two slightly differentand active wells,
datasets but instead we
and summarizing a
chose range
wider in favor of studying
of analysis two slightly different datasets and summarizing a wider range
results.
of analysis results.

Figure 10. Error analysis for BHS predictions. (left) MAPE for pressure and temperature predictions.
Figure 10. Error analysis for BHS predictions. (left) MAPE for pressure and temperature predic-
(right) Percent errors are shown for E2040 . All results represent predictions for the subset of the data
tions. (right) Percent errors are shown for 𝐸 . All results represent predictions for the subset of
not used in the training.
the data not used in the training.
We continue our model evaluation with a sensitivity analysis, in the same way we
We continue our model evaluation with a sensitivity analysis, in the same way we
performed it for OSR. We use our ML models for BHS and the code for the cumulative
performed it forestimation,
thermal energy OSR. We use E2040our
, toML models
calculate thefor BHS andindex,
sensitivity the code
SIE . for
Thethe cumulative
distributions of
thermal energy estimation, 𝐸 , to calculate the sensitivity index, 𝑆𝐼 .
the observed index values for different wells (nine wells with nonzero flows) are shown in The distributions
of the observed
Figure index values
11. The largest index for different
value, SIE =wells (nine wells
0.49 MWh with nonzero
per kg/day, which is flows)
smallerarethan
shownthe
in Figure
index 11. The
values we largest index
estimated forvalue, 𝑆𝐼 = 0.49
OSR, indicates MWh
that perchange
a small kg/day,inwhich
the modelis smaller
inputthan
(5%)
the index
results invalues we estimated
a relatively for OSR,
small output indicates
change, that a the
confirming small change
model’s in the
high modelSimilar
stability. input
to the
(5%) OSR results,
results model sensitivity
in a relatively small output to the change
change, in the injection
confirming wells’ high
the model’s flowsstability.
is lower
than thetosensitivity
Similar to the change
the OSR results, modelin the production
sensitivity to the wells’
change flows.
in theConsidering
injection wells’all our results
flows is
for BHS and OSR, we conclude that the selected ML approach is indeed
lower than the sensitivity to the change in the production wells’ flows. Considering all reliable and has
potential
our resultsfor
foraccurately
BHS and OSR, predicting characteristics
we conclude of otherML
that the selected geothermal
approachreservoirs given a
is indeed reliable
similar amount of training simulation data. On the computing side, the
and has potential for accurately predicting characteristics of other geothermal reservoirs results shown in
Figure 11 were obtained using the trained NNs relatively quickly, but, in contrast, running
given a similar amount of training simulation data. On the computing side, the results
all 1440 additional simulations needed for such per-quantity sensitivity summary would
shown in Figure 11 were obtained using the trained NNs relatively quickly, but, in con-
potentially require as much as 240 days of computing time, at 4 h per simulation.
trast, running all 1440 additional simulations needed for such per-quantity sensitivity
Our final analysis step involves optimization. In this step, we selected the cases with
summary
the highestwould potentially
predicted require
cumulative as much
energy values,asE240 days of computing time, at 4 h per
2040 . For this optimization specifically, we
simulation.
generated a large number (G = 5000) of potential flow allocations to be evaluated using the
same script that we used to generate the input for the TETRAD simulations. This allowed
us to study the allocations quickly, through the predictions obtained using our trained
models, and choose the best-performing cases. In Figure 12, we show three distributions
of energy estimates: (1) E2040 for 80 training cases, on which the models were trained (in
red), (2) E2040 for G generated cases (in light blue), and (3) 5% of results from the latter (in
dark blue). While we cannot release the specific estimates for BHS, we can highlight the
following in relative terms:
Energies 2022, 15, 967 13 of 20

• The 250 flow allocations selected as part of the top 5% have an average E2040 that
approaches the maximum E2040 for the training set. At the same time, the maximum
E2040 for the top 5% is approximately 6.5% higher than the maximum E2040 for the
entire training set. This relative difference can potentially translate into a substantial
energy gain, considering that the absolute values for E2040 are measured in millions of
MWh. It is also worth pointing out that the maximum E2040 for the training set may
refer to the flow allocation, demonstrating an additional nontrivial gain with respect
to the current allocation at BHS. It is important to treat this summary as a statement
about the potential optimization enabled by the proposed computational approach,
rather than a commentary on the efficiency of the current reservoir operation.
• These 250 best-performing flow allocations provide enough data to further study the
flows for individual wells, and, specifically, to look for noticeable differences between
flow values in the training cases and in the best allocations. Such differences can
indicate the directions in which the respective flows need to change in order to achieve
energy gains. For BHS, we compared the current flows, the distributions of flows in
the training data, and the distributions of flows in these 250 cases to arrive at well-
specific recommendations. These recommendations can be used to inform operational
Energies 2022, 15, x FOR PEER REVIEW 13 of 20
decisions or, at minimum, can serve as the basis for additional investigation through
simulations using TETRAD or CMG STARS.

Energies 2022, 15, x FOR PEER REVIEW

Over the course of this optimization, we collected prediction runtimes, capt

how much computing time (in seconds) is needed to produce all predictions—pres
temperatures, and energy estimates—for a given case. Our results indicate that, on
age, the inference takes only 0.9 s (the runtimes were measured on a single Eagle’s
pute node, with dual Intel Xeon Gold Skylake 6154 (3.0 GHz, 18-core) processor a
GB of RAM). This is several orders of magnitude faster than running simulations
TETRAD. 3D reservoir modeling with TETRAD or another simulator which takes
to run, because it simulates changes in rock and fluid properties associated with th
Sensitivity index
index distributions
distributions for
for BHS.
BHS. For
For anonymity,
Figure 11.
Figure 11. Sensitivity
and hydraulic processes across theanonymity, we refer
reservoir, we refer to
which istoathe
the injection
injection
much wells as
morewells as
computationa
I1–I4 and the production wells as P1–P6. P2 has zero flow in all simulations and was skipped in the
tensive process (if done at sufficient temporal and spatial resolutions) compared to
sensitivity analysis. In
In the
the histograms,
histograms, y-axis
y-axis values
values characterize
characterize the
the number
number of occurrences of the
the weights of the trained NNs for translating given inputs into outputs, even for
corresponding x-axis values (SI (𝑆𝐼 E).).
NNs.
Our final analysis step involves optimization. In this step, we selected the cases with
the highest predicted cumulative energy values, 𝐸 . For this optimization specifically,
we generated a large number (𝐺 = 5000) of potential flow allocations to be evaluated us-
ing the same script that we used to generate the input for the TETRAD simulations. This
allowed us to study the allocations quickly, through the predictions obtained using our
trained models, and choose the best-performing cases. In Figure 12, we show three distri-
butions of energy estimates: (1) 𝐸 for 80 training cases, on which the models were
trained (in red), (2) 𝐸 for G generated cases (in light blue), and (3) 5% of results from
the latter (in dark blue). While we cannot release the specific estimates for BHS, we can
highlight the following
Figure 12. Histogram
Figureof12.
in relativeenergy
cumulative
Histogram
terms:
of cumulative E2040 . The
estimates,energy values 𝐸on the
estimates, x-axis
. The are omitted
values to
on the x-axis are o
•avoidThe 250 flow
releasing allocations
sensitive
to avoid selected
sensitive as
information.
releasing part of the top 5% have an average 𝐸
information. that
approaches the maximum 𝐸 for the training set. At the same time, the maximum
𝐸 for 6. top 5% is approximately 6.5% higher than the maximum 𝐸
theDiscussion for the
entire training Inset.our
This relative difference can potentially translate into a substantial
work, we have proposed and evaluated a computational approach that
energy gain, considering
bines that the absolute
geologic reservoir modelingvalues for 𝐸We show
with ML. are measured in millions
how existing methods in ge
of MWh. malIt is reservoir
also worth pointing out that the maximum 𝐸 for the training set
engineering can be leveraged to produce data for training surrogate m
Energies 2022, 15, 967 14 of 20

Over the course of this optimization, we collected prediction runtimes, capturing

how much computing time (in seconds) is needed to produce all predictions—pressures,
temperatures, and energy estimates—for a given case. Our results indicate that, on average,
the inference takes only 0.9 s (the runtimes were measured on a single Eagle’s compute node,
with dual Intel Xeon Gold Skylake 6154 (3.0 GHz, 18-core) processor and 96 GB of RAM).
This is several orders of magnitude faster than running simulations with TETRAD. 3D
reservoir modeling with TETRAD or another simulator which takes hours to run, because
it simulates changes in rock and fluid properties associated with thermal and hydraulic
processes across the reservoir, which is a much more computationally intensive process (if
done at sufficient temporal and spatial resolutions) compared to using the weights of the
trained NNs for translating given inputs into outputs, even for large NNs.

6. Discussion
In our work, we have proposed and evaluated a computational approach that com-
bines geologic reservoir modeling with ML. We show how existing methods in geothermal
reservoir engineering can be leveraged to produce data for training surrogate models
implemented using NNs. In this section, we share several high-level perspectives on the
current work and what it can enable in future work.
We can summarize our ML work through the lens of the ML-focused recommendations
shared in a recent editorial by Physical Review Fluids [36]. We found the editorial to be
useful and informative, and we applied its recommendations to our work as follows:
• Physical mechanisms and their interpretation: Our ML approach is based on training
a set of models to predict pressures and temperatures for individual wells. We used
those model predictions in a closed-form expression to obtain reservoir-wide energy
estimates, as discussed in Section 3. By obtaining multiple well-specific predictions
instead of training a single model for energy produced reservoir-wide, we created a
more interpretable solution where we could reason about the possible causes of high
or low energy estimates; we could investigate our predictions for the wells and find
the pressure or temperature timeseries that were influencing the outcomes more than
the rest of the predictions.
• Reproducibility: We plan to publish all relevant data for OSR and the code we have
developed to make it possible for others to reproduce our findings.
• Validation, verification, and uncertainty quantification: We believe that our error
analysis, sensitivity analysis, and the energy optimization process provide enough
empirical evidence about the models to consider their results trustworthy. The great
similarity between the results for BHS and OSR is another piece of supporting evidence.
Further validation of BHS models and predictions can take place next time Ormat
changes the flow allocation (and the models will need to be retrained using the newest
data), but that work is outside the scope of our current investigation.
We envision that our ML efforts will pave the way for reinforcement learning (RL) [37]
in future research. RL can introduce mechanisms for further coupling of ML models
and simulators. For instance, the top 5% of cases described in Section 5 can form the
“second round” of TETRAD or CMG STARS simulations, which would then create even
more data for model training. Then, the retrained models would help find new best-case
candidates again. This iterative optimization process can be repeated as many times as we
see a noticeable increase in the energy estimates. At each iteration, model validation can be
performed to further inform the decisions about continuing or stopping the RL process.
The training of predictive models described in this study is an important prerequisite for
applying RL and potentially discovering additional energy gains.
Finally, we note that open access to high-quality data is a major prerequisite to con-
ducting research using modern, data-centric techniques. Data availability can actually
stimulate the rapid advancement of an entire field. In geothermal reservoir engineering,
this is a concern of particular importance, as the availability of real-world data sets—both
simulation results and real reservoir data—is relatively limited. We are not aware of any
Energies 2022, 15, 967 15 of 20

other published subsurface characterization data sets, analogous to the ones we trained
the models on in our work, that we could have used. Our aim was to bridge this gap
when creating OSR and running an ensemble of simulations for the reservoir. Now that
the relevant OSR data is published, the research community can take advantage of it in a
variety of ways. For example, other researchers can take the data and evaluate a broader
set of ML predictive models, searching for further optimizations and using the results from
this paper as a reference point. In a different line of research, researchers can compare the
published data with the simulation data from other geothermal reservoirs, the reservoirs
to which they have access, and reason about the potential of ML-based optimization for
those fields.

7. Conclusions
In this study, we demonstrate how geologic reservoir modeling can be used in com-
bination with machine learning. In fact, the latter does not replace the former in this
application, but rather complements it. The combination of the two computational ap-
proaches allows us to train accurate prediction models on simulation data that capture
physical subsurface phenomena. We develop this novel method with the goal of enabling
optimization of the energy produced at a geothermal reservoir. We tested the method on
the data for Brady Hot Springs in Nevada, USA, as well as a synthetic 10-well geothermal
system with realistic characteristics. In each case, we followed the proposed process—from
using simulators like TETRAD and CMG STARS to model training and evaluation—and
documented the details of each computational step. Our results indicate that a high level of
prediction accuracy can be achieved for these systems even with medium-sized data sets
(80–101 simulations). The relative error of 4.75% was the largest error we observed among
all studied predictions of pressure, temperature, and energy. For Brady Hot Springs, we
showed how the trained models can be used in the optimization process, i.e., in the search
for optimal flow allocations and increased energy production. Among the most significant
practical benefits enabled by our work is computational efficiency: whereas a typical reser-
voir simulation for Brady Hot Springs completes in approximately 4 h, our trained ML
models yield accurate 20-year predictions for temperatures, pressures, and the amount of
produced thermal energy for the entire reservoir in only 0.9 s. This computational efficiency
allows the rapid exploration of many candidate flow allocations, and, based on our findings,
we expect this method to be applicable to studying other geothermal reservoirs.

Author Contributions: D.D. is the lead author who worked on data curation, formal analysis, investi-
gation, methodology, software, validation, visualization, and all stages of writing. K.F.B. contributed
to investigation, validation, visualization, and writing. D.L.S. provided valuable guidance in the areas
of conceptualization, funding acquisition, and resource management, as well as helping to review and
edit the manuscript. M.J.M.’s involvement includes conceptualization, funding acquisition, resource
management, and writing. H.E.J. is a principal investigator on this project who coordinated activities
and advised the team through conceptualization, funding acquisition, data curation, formal analysis,
investigation, methodology, project administration, resource management, and writing. All authors
have read and agreed to the published version of the manuscript.
Funding: We acknowledge the U.S. Department of Energy’s Office of Energy Efficiency and Renew-
able Energy Geothermal Technologies Office for providing funding under award number DE-FOA-
0001956-1551 to the National Renewable Energy Laboratory (NREL) and the U.S. Geological Survey
Geothermal Resources Investigation Project. Partial funding was also provided by the U.S. Geological
Survey Mendenhall and Energy and Minerals Program. Additional material support was provided
by Ormat, CMG, and Quantum Reservoir Impact LLC (QRI). This work was authored in part by the
National Renewable Energy Laboratory, operated by the Alliance for Sustainable Energy, LLC, for the
U.S. Department of Energy (DOE) under Contract No. DE-AC36-08GO28308. The views expressed in
the article do not necessarily represent the views of the Department of Energy. The U.S. Government
retains and the publisher, by accepting the article for publication, acknowledges that the U.S. Govern-
ment retains a non-exclusive, paid-up, irrevocable, worldwide license to publish or reproduce the
published form of this work, or allow others to do so, for U.S. Government purposes. Any use of
Energies 2022, 15, 967 16 of 20

trade, firm, or product names is for descriptive purposes only and does not imply endorsement by
the U.S. Government.
Data Availability Statement: The repository with the data for Open-Source Reservoir, along with the
ML code we have developed, can be found at: https://github.com/NREL/geothermal_osr (accessed
on 27 January 2022).
Acknowledgments: We would like to thank John Akerley and John Murphy (Ormat Technologies Inc.)
for their continued collaboration and important contributions to this study. We thank the Computer
Modelling Group for donating the software licenses necessary to enable these simulations. Computational
experiments were performed using Eagle, the supercomputer sponsored by the Department of Energy’s
Office of Energy Efficiency and Renewable Energy and located at NREL. The authors would like to also
thank Caleb Phillips and Kristi Potter of NREL, and David Castineira of QRI for useful discussions.
Conflicts of Interest: The authors declare no conflict of interest.

Abbreviations

AI artificial intelligence
BHS Brady Hot Springs, a geothermal field in northern Nevada, USA
CNN convolutional neural networks
Eagle NREL’s supercomputer
LSTM long short-term memory
MAE mean absolute error
MAPE mean absolute percentage error
ML machine learning
MLP multilayer perceptron
NN neural network
open-source reservoir, a contrived geothermal reservoir that resembles BHS
OSR
at a high level but has a number of sufficiently modified characteristics
RL reinforcement learning
RMSE root mean square error

Appendix A
In determining what type of approximation to use for the studied data, we based
our decision on the shapes of the functions being approximated, their smoothness, and
complexity. As shown in Figure A1, the temperature and pressure curves are quite smooth,
and they can be approximated using polynomial functions. To support this claim by
the numbers, we calculate the root mean square error (RMSE) values for the individual
instances of this approximation (37 scenarios × 6 production wells). It appears that the
errors coming from the temperature approximation tend to be smaller than the errors of
pressure approximation, but, more importantly, all observed approximation errors are
small, with RMSE < 0.015 for all cases.
 stances of this approximation
stances
stances of (37
of this thisscenarios
approximation
approximation 6 production
(37 scenarios wells).
(37 scenarios 6Itproduction
appears
6 production that
wells). the er-
It appears
wells). thatthat
It appears the er-
the er-
rors coming from the
rors temperature
coming
rors fromfrom
coming approximation
the temperature
the temperaturetend to be smaller
approximation
approximationtendthan the
to be
tend errors
tosmaller
be of than
than
smaller the errors of of
the errors
pressure
Energies 2022, 15, 967 approximation,
pressure but,approximation,
pressure more importantly,
approximation, but,but,
more all
more observed
importantly,approximation
all observed
importantly, errors
all observed are
approximation
approximationerrors are17are
errors of 20
small, with RMSE < small,
0.015
small, for
with all
RMSE cases.
< 0.015 for all cases.
with RMSE < 0.015 for all cases.

Figure A1. ScaledFigure Figure

temperature
A1. Scaled
Figure A1.A1. Scaled
(left) temperature
andtemperature
pressure
temperature
Scaled (left) (left)
(right)
and
(left) and pressure
timeseries
pressure
and for one
(right)
pressure (right)
of the
(right) timeseries
production
timeseries forfor
for one
timeseries of
oneone
the of
thethe production wells
ofproduction
production
wells at BHS. The wells
timeseries
wells atrepresent
atatBHS.
BHS. The
BHS. The37timeseries
CMGrepresent
Thetimeseries
timeseries STARS
represent simulations
37CMG
37
represent CMG forSTARS
STARS
STARS
37 CMG different flow for
simulations
simulations allocations.
for different
simulations different flowflow
flow
for different allocations.
allocations.
allocations.

Appendix B Appendix
Appendix
Appendix B BB
In Table A1, we In In In
summarize
Table Table
A1,
Table weA1,
two
A1, wewe
ML summarize
approaches:
summarize
summarize twotwoML two
the ML
one approaches:
used
approaches:
ML in our
approaches: thethe one
previous
the one used
one used
in in
work,
in
used our our
previous
our previous
work,
previous work,
work,
based on point based based
prediction, on
and
on point point prediction,
the prediction,
one used
prediction, andinand and
thisthe
the the
study,
one one used
usedcurve in this
prediction.
in this study, study,
We
curve curve
report prediction.
prediction. We We We
reportreport
based on point one used in this study, curve prediction. report
several key prosseveralseveral
andseveral
cons
keyfor key
each
pros pros andfor
approach.
and cons cons
We
eachfor each
also approach.
briefly
approach. discuss
We alsoWethealso briefly
structure
briefly discuss discuss
of the the structure
structure of the of the
key pros and cons for each approach. We also briefly discuss the structure of the
corresponding NNs incorresponding
terms ofNNs
corresponding
corresponding the NNsNNs
numbers
in in terms
terms of
in terms the of thethat
of neurons
of numbers
numbers
the numbersform of neurons
ofdifferent
of neurons that different
thatlayers.
neurons form
that
form different
form differentlayers.layers.
layers.

Table A1. Comparison

Table Table
of ML
A1. A1. Comparison
approaches
Comparison offor
ML of ML approaches
prediction forcharacteristics.
of reservoir
approaches prediction
for prediction of reservoir
of reservoir characteristics.
characteristics.
Table A1. Comparison of ML approaches for prediction of reservoir characteristics.

Point Prediction Point Prediction Curve Prediction Curve

Curve Prediction
Prediction
Curve Prediction
Point Prediction
Work
(Used in Previous(Used
Work [23,24])
[23,24])
in Previous Work [23,24]) (Used in Current Study)
(Used
(Used in
in
(UsedCurrent
Current
in Study)
Study)
Current Study)
(Used in Previous Work [23,24])

Pro: This approach is more

Pro: ThisThis
Pro: expressive
approachapproach isand iscon-
more expressive
more and con-con-
expressive
Pro: Many
Pro: examples
Many examplesin the
Pro:literature,
inPro:
theMany
Many with
literature,
examples a variety
with
examples in the of
theNN
ainvari-
literature, withPro:
literature, This
a vari-
with approach
a vari- is more expressive and concise, and making it
architectures being used (Brownlee, 2018). cise, making it straightforward
cise,
straightforward making
cise, tomaking
go to
it
from goit from
straightforward curve
straightforward
curve to
parameters gotofrom
togo curve
from
timeseries.curve
ety of NN architectures
ety of ety being
NN used
of architectures
NN [25]. being
architectures usedused
being [25].[25].
Pro: Allows learning and prediction of arbitrarily complex parameters Pro:to Prediction
timeseries.
parameters
parameters
errors tofor
timeseries.
tomoments
timeseries. in the distant future are not
temporal patterns. Pro: Prediction
necessarilyerrors for
Pro:worse
Pro: moments
Prediction
than prediction
Prediction in errors
errors thefordistant
moments
errors for in
for moments thethe indistant
near thefuture.
distant
Pro: Allows learning Pro:and Allows
Pro: prediction
Allowslearning of and
learning arbitrarily
prediction
and predictionof arbitrarily
future areof Con:
arbitrarily
not necessarily
future
Requires worse
are not
specific
future are than
necessarily
structure
not prediction
of
necessarily worse
input than
data.
worse prediction
We
than rely on the
prediction
complex temporal
Con: Accumulation patterns.
complex
of error iscomplextemporal
likely totemporal
occur patterns.
if this single-step
patterns. errors for flows
the near being
errorsset
future. to specific
for
errors the the values
fornear future.
near at the beginning of the time
future.
prediction scheme is used recursively (to obtain prediction for a
Con: Accumulation of
Con: error is
Accumulation likely toof occur
error ifis Con:
likely to period
Requires
occur if being
specific
Con:
Con: Accumulation of error is likely to occur if Con: Requires specific structurestudied
structure
Requires andof remaining
input
specific data.
structureconstant throughout
of input
of data.data.
input
whole period of time).
this single-step prediction scheme
this single-step is used
prediction recur-
scheme We rely
is used that
on the
recur-period.
flows
We being
rely set
on theto specific
flows values
being set to specific values
this single-step prediction scheme is used recur- We rely on the flows being set to specific values
Parameters
sivelyof(to
the NN: prediction
obtain sively (to for
obtain a whole period
prediction forof aat the
whole Parameters
beginning
period of of
at theoftime
the the NN:
period
beginning being
of the studied
time period being studied
sively (to obtain prediction for a whole period of at the beginning of the time period being studied
input layer: number of neurons is set to the number of input layer: number of neurons is set to the number of values
time). time). time). and remaining constant and and throughout
remaining
remaining that
constant period.
constantthroughout
throughoutthat that
period.
period.
consecutive time steps used in prediction (in previous work, we (flows) that define each studied scenario.
Parameters of theParameters NN:Parameters of the NN:
ofnumber
the NN:of neurons in one or
used L Parameters of theParameters
= 12 for 12 monthly NN:
steps) plusofthe
Parameters the NN:
number
of the NN: of additional hidden layer(s): arbitrary
input layer:factors
“channels”—other number inputofinput
being neurons
layer:
considered is
number set toof
(flowsthe
neurons
in our input
is set layer:
to the
more number input
layers. of neurons
layer:
input layer: is
number set
number toofthe
neurons
of neuronsis setistosettheto the
layer: number of neurons is set to the
number
particular of consecutive
problem). number time of steps used
consecutive in predic-
time stepsnumber
number of consecutive time steps used in predic- used of
in values
predic-
output (flows)
number
layer: number that
of
number define
values
ofofvalues each
(flows)
neurons stud-
that
(flows)
is set define
that
to each
define
the number stud-
each ofstud-
hiddention
layer(s): arbitrary
(in previous number
work,
tion of previous
weprevious
(in
tion used
(in neurons
L =work, in
12 for one
we12 used
work, we ied
or usedL =scenario.
12
L =for
12 12for 12 ied that
coefficients scenario.
ied define
scenario. the curves within the selected type of
more layers.
monthly steps) plus the
monthly number
monthly steps) of
plus
steps) additional
the number
plus the number hidden layer(s):
of additional
of additional arbitrary
approximatinghiddenhidden number
functions;
layer(s): of
layer(s):forneurons
arbitrarypolynomials
arbitrary in number
number of
ofdegree
neurons D,in
of neurons D+1in
layer: single neuron
output“channels”—other for
factors single-step
being considered
“channels”—other
“channels”—other predictions.
factors being
factors one
considered
being neurons
orconsidered
more layers. are used.
one or
onemore or morelayers. layers.
(flows in our particular
(flows problem).
in our
(flows in particular
our particular problem). output layer: number
problem). output of neurons
outputlayer: is set
number
layer: numbertoofthe
neurons
of neuronsis setistosettheto the
hidden layer(s): arbitrary
hidden hidden number
layer(s): of neurons
arbitrary
layer(s): arbitrary in number
number of neurons
number ofofneuronsin number
coefficients thatofdefine
in number of the curves
coefficients
coefficients that that
define the curves
define the curves
one or more layers. one or Appendix C
onemore or morelayers.layers. within the selectedwithin type of
withintheapproximating
selected
the selected typetype func-
of approximating
of approximating func-func-
output layer: single neuron
output output for
layer: Using
layer: OSR
single-step
single neuron
single data,
pre-
neuronfor we fortrained
single-step
tions; 8640
pre-
single-step
for polynomials individual
pre-
tions; offor
tions; formodels
degree D, D+1for
polynomials
polynomials 12oftimeseries
neurons
of degree D, D+1
degree (pressure
neurons
D, D+1 neuronsand
dictions. dictions. temperature
dictions. values for six
are used.production wells).
are used. Simultaneously, we studied the effects of
are used.
three parameters—the degree of the approximating polynomials, the number of neurons in
the NN’s hidden layers, and the amount of model training—on the accuracy of predictions.
We performed this analysis using a common approach called k-fold cross-validation (with
k = 10) [31], where the training data set was repeatedly shuffled and split into k subsets, out
of which k − 1 were used for model training and the remaining one was used for validation.
This process is designed to prevent overfitting in the course of hyperparameter selection.
 temperature values for six production wells). Simultaneously, we studied the effects of
three parameters—the degree of the approximating polynomials, the number of neurons
in the NN’s hidden layers, and the amount of model training—on the accuracy of predic-
tions. We performed this analysis using a common approach called k-fold cross-validation
(with k = 10) [31], where the training data set was repeatedly shuffled and split into k
Energies 2022, 15, 967 subsets, out of which k − 1 were used for model training and the remaining one was18used of 20

for validation. This process is designed to prevent overfitting in the course of hyperpa-
rameter selection. In each iteration of the validation process, we obtained MAPE estimates
Incompare
to each iteration of the validation
the simulation data andprocess, we predictions.
the model obtained MAPE estimates to compare the
simulation dataA2,
In Figure andwetheshow
modelour
predictions.
cross-validation results, with the 20 best-performing
modelInconfigurations
Figure A2, we depicted
show ourfor
cross-validation results,
one pressure and onewith the 20 best-performing
temperature model
timeseries. Through
configurations
this depicted
cross-validation, for one the
we learned pressure and one temperature timeseries. Through this
following:
•cross-validation, we learned
Curve approximation the following:
should use polynomials of degree four: They consistently pro-
• Curve
duce approximation
models with lower should
errorsusethanpolynomials
for degreesoffivedegree four:
and six onThey consistently
the given data. pro-
• duce models with lower errors than for degrees five and
The best models are trained with the number of epochs set to 1000: An epochsix on the given data. refers
• to one complete pass over the data set made by an ML algorithm, and ourrefers
The best models are trained with the number of epochs set to 1000: An epoch to
results
one complete
indicate pass such
that 1000 over the dataresult
passes set made by an models
in better ML algorithm,
than the and our results
lower tested indicate
values,
that and
250 1000 500,
such with
passestheresult in better
control models thanenabled
for overfitting the lower bytested
k-foldvalues, 250 and 500,
cross-validation.
with thehigher
Epochs controlthan
for overfitting
1000 wereenabled by k-foldincross-validation.
not considered our study because Epochs
the higher
resultsthan
we
obtained for 1000 epochs with our datasets showed sufficiently high accuracy.epochs
1000 were not considered in our study because the results we obtained for 1000
• with our
With datasets showedselections,
the aforementioned sufficientlythe high accuracy.
hidden layer sizes of the NNs should be 12
• With the aforementioned selections, the hidden
and 6: This combination consistently results in the best-performinglayer sizes of the NNs should
models, as we canbe
12 and 6: This combination consistently results in the best-performing
see in Figure A2 (there, it is the second-best configuration), as well as resulting in the models, as we
can see
best in Figure
models createdA2for(there,
all sixitproduction
is the second-best configuration),
wells. These particular as well
sizes of as
theresulting
hidden
in the best models created for all six production wells. These particular
layers result in the models with 245 weights being fitted to the data, sufficiently sizes ofcap-
the
hidden layers result in the models with 245 weights
turing complex relationships between the model inputs and outputs. being fitted to the data, sufficiently
capturing complex relationships between the model inputs and outputs.

Figure A2. Analysis of ML models trained with k-fold cross-validation (k = 10). The figures show
Figure A2. Analysis of ML models trained with k-fold cross-validation (k = 10). The figures show
error summaries
error summariesfor
forpressure
pressure(left)
(left) and
and temperature
temperature (right)
(right) predictions
predictions for
for one
one of
of the
the production
production
wells. Each plot shows 20 model/training configurations (out of 72 evaluated) with the lowest
average MAPE estimates, depicted in ascending order.

Overall, it is worth noting that the magnitudes of the observed MAPE values, where
the averages and the medians are below 2% and the rare, worst-case values are around 7%,
indicate that the models with these identified parameters are highly accurate.
Energies 2022, 15, 967 19 of 20

References
1. Robins, J.C.R.; Kolker, A.; Flores-Espino, K.; Pettitt, W.; Schmidt, K.; Beckers, K.; Pauling, H.; Anderson, B. 2021 U.S. Geothermal
Power Production and District Heating Market Report. 2021. Available online: https://www.nrel.gov/docs/fy21osti/78291.pdf
(accessed on 1 August 2021).
2. Ball, P.J. A Review of Geothermal Technologies and Their Role in Reducing Greenhouse Gas Emissions in the USA. J. Energy
Resour. Technol. 2020, 143, 010903. [CrossRef]
3. DOE. GeoVision: Harnessing the Heat Beneath Our Feet—Analysis Inputs and Results. Geothermal Technologies Office,
US Department of Energy, 2019. Available online: https://www.osti.gov/dataexplorer/biblio/dataset/1572361 (accessed on
21 November 2021).
4. Ball, P.J. Macro Energy Trends and the Future of Geothermal Within the Low-Carbon Energy Portfolio. J. Energy Resour. Technol.
2020, 143, 010904. [CrossRef]
5. van der Zwaan, B.; Longa, F.D. Integrated Assessment Projections for Global Geothermal Energy Use. Geothermics
2019, 82, 203–211. [CrossRef]
6. Edenhofer, O.; Pichs-Madruga, R.; Sokona, Y.; Seyboth, K.; Kadner, S.; Zwickel, T.; Eickemeier, P.; Hansen, G.; Schlomer, S.;
von Stechow, C. Renewable Energy Sources and Climate Change Mitigation: Special Report of the Intergovernmental Panel on Climate
Change (IPCC); Cambridge University Press: Cambridge, UK, 2011.
7. Pan, S.-Y.; Gao, M.; Shah, K.J.; Zheng, J.; Pei, S.-L.; Chiang, P.-C. Establishment of Enhanced Geothermal Energy Utilization Plans:
Barriers and Strategies. Renew. Energy 2019, 132, 19–32. [CrossRef]
8. Soltani, M.; Kashkooli, F.M.; Souri, M.; Rafiei, B.; Jabarifar, M.; Gharali, K.; Nathwani, J.S. Environmental, Economic, and Social
Impacts of Geothermal Energy Systems. Renew. Sustain. Energy Rev. 2021, 140, 110750. [CrossRef]
9. Matzel, E.; Magana-Zook, S.; Mellors, R.J.; Pullimmanappallil, S.; Gasperikova, E. Looking for Permeability on Combined 3D
Seismic and Magnetotelluric Datasets With Machine Learning. In Proceedings of the 46th Workshop on Geothermal Reservoir
Engineering, Stanford, CA, USA, 16–18 February 2021.
10. Ahmmed, B.; Lautze, N.C.; Vesselinov, V.V.; Dores, D.E.; Mudunuru, M.K. Unsupervised Machine Learning to Extract Dominant
Geothermal Attributes in Hawaii Island Play Fairway Data. In Proceedings of the Geothermal Resources Council’s Annual
Meeting & Expo, Reno, NV, USA, 18–21 October 2020.
11. Smith, C.M.; Faulds, J.E.; Brown, S.; Coolbaugh, M.; Lindsey, C.; Treitel, S.; Ayling, B.; Fehler, M.; Gu, C.; Mlawsky, E.
Characterizing Signatures of Geothermal Exploration Data with Machine Learning Techniques: An Application to the Nevada
Play Fairway Analysis. In Proceedings of the 46th Workshop on Geothermal Reservoir Engineering, Stanford, CA, USA,
16–18 February 2021.
12. Siler, D.L.; Pepin, J.D.; Vesselinov, V.V.; Mudunuru, M.K.; Ahmmed, B. Machine Learning To Identify Geologic Factors Associated
With Production in Geothermal Fields: A Case-Study Using 3D Geologic Data, Brady Geothermal Field, Nevada. Geother. Energy
2021, 9, 1–17. [CrossRef]
13. Siler, D.L.; Pepin, J.D. 3-D Geologic Controls of Hydrothermal Fluid Flow at Brady Geothermal Field, Nevada, USA. Geothermics
2021, 94, 102112. [CrossRef]
14. Buster, G.; Siratovich, P.; Taverna, N.; Rossol, M.; Weers, J.; Blair, A.; Huggins, J.; Siega, C.; Mannington, W.; Urgel, A.; et al.
A New Modeling Framework for Geothermal Operational Optimization with Machine Learning (GOOML). Energies 2021,
14, 6852. [CrossRef]
15. U.S. Securities and Exchange Commission, Ormat Technologies, Inc. (Commission File Number: 001-32347). Available on-
line: https://d18rn0p25nwr6d.cloudfront.net/CIK-0001296445/d7111d58-b48a-49a1-aa60-7c72223b25ba.html (accessed on
21 December 2021).
16. Faulds, J.; Ramelli, A.; Coolbaugh, M.; Hinz, N.; Garside, L.; Queen, J. Preliminary Geologic Map of the Bradys Geothermal Area,
Churchill County, Nevada; Open-File Report; Nevada Bureau of Mines and Geology: Reno, NV, USA, 2017.
17. Wesnousky, S.G.; Barron, A.D.; Briggs, R.W.; Caskey, S.J.; Kumar, S.; Owen, L. Paleoseismic Transect Across the Northern
Great Basin. J. Geophys. Res. Solid Earth 2005, 110. Available online: https://webcentral.uc.edu/eProf/media/attachment/
eprofmediafile_371.pdf (accessed on 21 November 2021). [CrossRef]
18. Faulds, J.E.; Garside, L.J.; Oppliger, G.L. Structural Analysis of the Desert Peak-Brady Geothermal Field, Northwestern Nevada:
Implications for Understanding Linkages between Northeast-Trending Structures and Geothermal Reservoirs in the Humboldt Structural
Zone; Transactions-Geothermal Resources Council: Davis, CA, USA, 2003.
19. Faulds, J.; Moeck, I.; Drakos, P.; Zemach, E. Structural Assessment and 3D Geological Modeling of the Brady’s Geothermal
Area, Churchill County (Nevada, USA): A Preliminary Report. In Proceedings of the 35th Workshop on Geothermal Reservoir
Engineering, Stanford, CA, USA, 1–3 February 2010; Stanford University: Stanford, CA, USA, 2010.
20. Jolie, E.; Moeck, I.; Faulds, J.E. Quantitative Structural—Geological Exploration of Fault-Controlled Geothermal Systems—A Case
Study From the Basin-And-Range Province, Nevada (USA). Geometrics 2015, 54, 54–67. [CrossRef]
21. Siler, D.L.; Hinz, N.H.; Faulds, J.E. Stress Concentrations at Structural Discontinuities in Active Fault Zones in the Western United
States: Implications for Permeability and Fluid Flow in Geothermal Fields. Bulletin 2018, 130, 1273–1288. [CrossRef]
22. Siler, D.L.; Faulds, J.E.; Hinz, N.H.; Queen, J.H. Three-Dimensional Geologic Map of the Brady Geothermal Area, Nevada. 2021.
Available online: https://pubs.er.usgs.gov/publication/sim3469 (accessed on 2 December 2021).
Energies 2022, 15, 967 20 of 20

23. Beckers, K.F.; Duplyakin, D.; Martin, M.J.; Johnston, H.E.; Siler, D.L. Subsurface Characterization and Machine Learning
Predictions at Brady Hot Springs. In Proceedings of the 46th Workshop on Geothermal Reservoir Engineering, Stanford, CA, USA,
15–17 February 2021.
24. Duplyakin, D.; Siler, D.L.; Johnston, H.; Beckers, K.; Martin, M. Using Machine Learning To Predict Future Temperature Outputs
in Geothermal Systems. In GRC 2020 Virtual Annual Meeting & Expo; National Renewable Energy Lab: Reno, NV, USA, 2020.
25. Brownlee, J. Deep Learning for Time Series Forecasting: Predict the Future with MLPs, CNNs and LSTMs in Python; Machine Learning
Mastery: San Juan, PR, USA, 2018.
26. Shook, G.M.; Faulder, D.D. Validation of a Geothermal Simulator; DOEEEGTP (USDOE Office of Energy Efficiency and Renewable
Energy Geothermal Tech Pgm): Idaho Falls, ID, USA, 1991.
27. CMG LTD. STARS Thermal & Advanced Processes Simulator. 2021. Available online: https://www.cmgl.ca/stars (accessed on
21 November 2021).
28. Nwachukwu, C. Machine Learning Solutions for Reservoir Characterization, Management, and Optimization. Ph.D. Thesis,
University of Texas at Austin, Austin, TX, USA, 2018. Available online: https://repositories.lib.utexas.edu/handle/2152/74252
(accessed on 22 April 2020).
29. Abadi, M.; Agarwal, A.; Barham, P.; Brevdo, E.; Chen, Z.; Citro, C.; Corrado, G.S.; Davis, A.; Dean, J.; Devin, M.; et al.
TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. arXiv 2015, arXiv:1603.04467. Available online: https:
//www.tensorflow.org/2015 (accessed on 27 January 2022).
30. Wagner, W.; Kruse, A. Properties of Water and Steam: The Industrial Standard IAPWS-IF97 for the Thermodynamic Properties and
Supplementary Equations for Other Properties: Tables Based on These Equations; Springer: Berlin/Heidelberg, Germany, 1998.
31. scikit-learn.org, 3.1. Cross-Validation: Evaluating Estimator Performance. 2021. Available online: https://scikit-learn.org/stable/
modules/cross_validation.html (accessed on 22 November 2021).
32. NREL, Eagle Computing System. 2021. Available online: https://www.nrel.gov/hpc/eagle-system.html (accessed on
22 November 2011).
33. Tunkiel, A.T.; Sui, D.; Wiktorski, T. Data-Driven Sensitivity Analysis of Complex Machine Learning Models: A Case Study of
Directional Drilling. J. Petrol. Sci. Eng. 2020, 195, 107630. [CrossRef]
34. Molnar, C.; Konig, G.; Herbinger, J.; Freiesleben, T.; Dandl, S.; Scholbeck, C.A.; Casalicchio, G.; Grosse-Wentrup, M.; Bischl, B.
General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models. arXiv 2020, arXiv:2007.04131.
35. Lundberg, S.; Lee, S. A Unified Approach to Interpreting Model Predictions. 2017. Available online: https://arxiv.org/abs/1705
.07874 (accessed on 5 January 2022).
36. American Physical Society. Machine Learning and Physical Review Fluids: An Editorial Perspective. 2021. Available online:
https://journals.aps.org/prfluids/edannounce/10.1103/PhysRevFluids.6.070001 (accessed on 4 December 2021).
37. Arulkumaran, K.; Deisenroth, M.P.; Brundage, M.; Bharath, A.A. Deep Reinforcement Learning: A Brief Survey. IEEE
2017, 34, 26–38. [CrossRef]