You are on page 1of 6

5th

5th IFAC
IFAC International
International Conference
Conference on
on
5th
5th IFAC
IFAC International
International Conference
Conference on
on
Intelligent
Intelligent
5th Control
Control and
IFAC International Automation
andConference Sciences
Automationon
Sciences
Available online at www.sciencedirect.com
Intelligent
Intelligent
5th IFAC Control and Automation
ControlIreland,
andConference
International Automation Sciences
Sciences
on
Belfast,
Belfast, Northern
Intelligent Control
Northern and August
Automation
Ireland, August 21-23, 2019
Sciences
21-23, 2019
Belfast, Northern
Belfast, Northern
Intelligent ControlIreland,
Ireland,
and August 21-23,
August
Automation21-23, 2019
2019
Sciences
Belfast, Northern Ireland, August 21-23, 2019
Belfast, Northern Ireland, August 21-23, 2019
ScienceDirect
IFAC PapersOnLine 52-11 (2019) 212–217

Machine
Machine Learning
Learning approaches
approaches for
for Anomaly
Anomaly
Machine
Machine Learning
Learning
Detection in approaches
approaches
Multiphase for
for
Flow Anomaly
Anomaly
Meters
Detection
Detection in
in Multiphase
Multiphase Flow
Flow Meters
Meters
Detection in Multiphase Flow Meters ∗∗∗
Tommaso
Tommaso Barbariol
Barbariol ∗∗∗ Enrico

Barbariol
∗∗
Enrico Feltresi
Enrico Feltresi ∗∗ Gian
Gian Antonio Susto
Gian Antonio
Antonio Susto
∗ ∗∗ ∗∗∗
Tommaso
Tommaso Feltresi ∗∗ Gian Antonio Susto ∗∗∗
Tommaso Barbariol
Barbariol ∗ Enrico
Enrico Feltresi
Feltresi ∗∗ Susto ∗∗∗
∗∗ Gian Antonio Susto ∗∗∗
∗∗∗
Tommaso Barbariol Enrico Feltresi Gian Antonio Susto
∗ Department of Information Engineering, University of Padova, Italy


Department of Information Engineering, University of Padova, Italy
∗ Department
Department of
of Information
Information Engineering,
Engineering, University
University of
of Padova,
Padova, Italy
Italy

∗∗ Pietro
∗∗
Department of Fiorentini
Information S.p.A., Arcugnano
Engineering, (VI),
University ofItaly
Padova, Italy
∗ ∗∗
Department
∗∗ Pietro
Pietro
of Fiorentini
Fiorentini
Information
Pietro Fiorentini S.p.A.,
S.p.A., Arcugnano
Arcugnano
Engineering,
S.p.A., (VI),
(VI),
University
Arcugnano (VI),ofItaly
Italy
Padova,
Italy Italy
∗∗∗
∗∗∗ Department
∗∗
Pietro
Department
∗∗∗ Department of
of Information
Fiorentini
Information Engineering
S.p.A., Arcugnano
Engineering and
and Human-Inspired
(VI), Italy
Human-Inspired
∗∗∗ ∗∗
Pietroof
∗∗∗ Department
Technology of Information
Fiorentini
Information
Center,
Engineering
S.p.A., Arcugnano
Engineering
University of
and
and
Padova,
Human-Inspired
(VI), Italy
Human-Inspired
Italy. E-mail:
∗∗∗ Department
Technology
Technology ofCenter,
Information Engineering
University of andItaly.
of Padova, Human-Inspired
E-mail:
Department
Technology ofCenter, University
Information
Center, Padova,
Engineering andItaly. E-mail:
Human-Inspired
Technology Center, University
University of
of Padova,
gianantonio.susto@dei.unipd.it
gianantonio.susto@dei.unipd.it Padova, Italy.
Italy. E-mail:
E-mail:
Technology Center, University of Padova, Italy. E-mail:
gianantonio.susto@dei.unipd.it
gianantonio.susto@dei.unipd.it
gianantonio.susto@dei.unipd.it
gianantonio.susto@dei.unipd.it
Abstract:
Abstract:
Abstract: Multiphase
Multiphase
Multiphase Flow
Flow
Flow Meters
Meters
Meters (MPFM)
(MPFM)
(MPFM) are
are
are important
important
important metering
metering
metering tools
tools
tools in
in
in the
the
the oil
oil
oil and
and
and
Abstract:
gas industry.
Abstract: Multiphase
A
Multiphase MPFM Flow
Flow Meters
provides
Meters (MPFM)
real-time
(MPFM) are
measurements
are important
important of metering
gas,
meteringoil tools
and
tools waterin
in the oil
flows
the oil and
of
and aaa
gas
gas industry.
industry.
Abstract:
gas industry. A
A
Multiphase
A MPFM
MPFM
MPFM Flowprovides
provides
Meters
provides real-time
real-time
(MPFM)
real-time measurements
measurements
are
measurements important of
of
of gas,
gas,
metering
gas, oil
oil
oil and
and
tools
and water
water
waterin flows
flows
the oil
flows of
of
and
of aa
well
gas
well
well without
industry.
without
without the
A
the
the need
MPFM
need
need to
to
to separate
provides
separate
separate the
real-time
the
the phases,
phases,
phases, a
measurements
a time-consuming
time-consuming
aa time-consuming of gas, procedure
oil and
procedure
procedure water that
that
that has
flows
has
has been
of
been
been
gas industry.
well without
classically A MPFM
the
adopted need
in theprovides
to separate
industry. real-time
the measurements
phases,
Evaluating the time-consuming
composition of gas,ofoil and
procedure
the flow water
is thatflows
has
fundamental of a
been
well without
classically
classically the
adopted
adopted need
in
in to
the
the separate
industry. the phases,
Evaluating a
the time-consuming
composition of of procedure
the flow is that
isthat has
fundamentalbeen
well without
classically the need
adopted in to industry.
separate Evaluating
the phases, the composition
a time-consuming the flow
procedure fundamental
has been
for
for
for the
classically
the
the well
well
well management
adopted
management
management in the
the andindustry.
and
and
Evaluating
productivity
industry. Evaluating
productivity
productivity
the
the composition
prediction;
prediction;
prediction; therefore,
composition
therefore,
therefore,
of the
the flow
flow is
ofprocedures
procedures
procedures
fundamental
isfor measuring
fundamental
for
for measuring
measuring
classically
for the
quality welladopted
management in the industry.
and Evaluating
productivity the
prediction; composition
therefore, of the flow
procedures is fundamental
for measuring
for the assessment
quality
quality well management
assessment are
are of
of crucial
crucial importance.
and productivity
importance. In
In this
prediction;
this work
work we
we propose
therefore,
propose an
procedures
an Anomaly
Anomaly Detection
for measuring
Detection
for the assessment
quality
approach well
to management
assessment
MPFM
are
are
that
of
of crucial
and
crucial
is
importance.
productivity
importance.
effectively able to
In
In this
prediction;
this
hand
work
work
the
we
we
complexity
propose
therefore,
propose and
an
procedures
an Anomaly
Anomaly
variability
Detection
for measuring
Detection
associated
quality
approach
approach assessment
to
to MPFM
MPFM are
that
that of crucial
is
is importance.
effectively
effectively able
able to
to In this
hand
hand work
the
the we
complexity
complexitypropose and
and an Anomaly
variability
variability Detection
associated
associated
quality
approach
with MPFM assessment
to MPFM
data. are
that
The of crucial
is importance.
effectively able to In this
hand work
the we
complexitypropose and an Anomaly
variability Detection
associated
approach
with
with MPFM
MPFM to MPFM
data.
data. thatproposed
The
The is effectively
proposed
proposed approach
able to
approach
approach is
is
is designed
designed
designed for
hand the complexity
for
for embedded
embedded
embedded andimplementation
variability associated
implementation
implementation and
and
and it
it
it
approach
with
exploitsMPFM to MPFM
data.
unsupervised that
The is
proposed
Anomaly effectively able
approach
Detection to
is hand
designed
approaches the complexity
for
like embedded
Cluster and
Based variability
implementation
Local associated
Outlier and
Factor it
with
exploits
exploitsMPFM data.
unsupervised
unsupervised The proposed
Anomaly
Anomaly approach
Detection
Detection is designed
approaches
approaches for
like
like embedded
Cluster
Cluster Based
Based implementation
Local
Local Outlier
Outlier and
Factor
Factor it
with
exploits
and MPFM data.
unsupervised
Isolation Forest. The proposed
Anomaly approach
Detection is designed
approaches for
like embedded
Cluster Based implementation
Local Outlier and
Factor it
exploits
and
and unsupervised
Isolation
Isolation Forest.
Forest. Anomaly Detection approaches like Cluster Based Local Outlier Factor
exploits
and unsupervised
Isolation Forest. Anomaly Detection approaches like Cluster Based Local Outlier Factor
and Isolation Forest.
and
© 2019,Isolation Forest.
IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved.
Keywords:
Keywords:
Keywords: Anomaly
Anomaly
Anomaly Detection,
Detection,
Detection, Data
Data
Data Mining,
Mining,
Mining, Data
Data
Data Fusion,
Fusion,
Fusion, Machine
Machine
Machine Learning,
Learning,
Learning, Multiphase
Multiphase
Multiphase
Keywords:
Flow Meter,
Keywords: Anomaly
Oil
Anomaly & Detection,
Gas Industry,
Detection, Data Mining,
Self-Diagnosis.
Data Mining, Data
Data Fusion,
Fusion, Machine
Machine Learning,
Learning, Multiphase
Multiphase
Flow
Flow Meter,
Meter,
Keywords:
Flow Meter, Oil
Oil
Anomaly
Oil &
&
& Gas
Gas Industry,
Industry,
Detection,
Gas Industry, Self-Diagnosis.
Self-Diagnosis.
Data Mining, Data Fusion, Machine Learning, Multiphase
Self-Diagnosis.
Flow Meter, Oil & Gas Industry,
Flow Meter, Oil & Gas Industry, Self-Diagnosis. Self-Diagnosis.
1.
1. INTRODUCTION
1. INTRODUCTION
INTRODUCTION
INTRODUCTION tection (Meneghetti
tection (Meneghetti
(Meneghetti et et al.,
et al., 2018b),
al., 2018b),
2018b), SmartSmart Monitoring
Smart Monitoring
Monitoring and and
and
1. tection
tection (Meneghetti et al., 2018b), Smart Monitoring and
1. INTRODUCTION Self-Diagnosis
tection
Self-Diagnosis
Self-Diagnosis (Meneghettitechnologies
et
technologies
technologies al., in
2018b),
in smart
smart
in smart
smart devices
Smart (Meneghetti
Monitoring
devices
devices (Meneghetti
(Meneghetti and
1. INTRODUCTION tection
Self-Diagnosis (Meneghetti et al., 2018b),
technologies in Smart Monitoring and
Monitoring of wells flow production and composition et
et al.,
Self-Diagnosis
et al., 2018a)
al., 2018a)
2018a) and
2018a) and
and
and products
technologies
productsin
products smart devices
(Theissler,
(Theissler,
(Theissler, 2017),
devices
2017),
2017),
(Meneghetti
industrial
(Meneghetti
industrial
industrial
Monitoring
Monitoring of of wells
of wells
wells flow flow production
flow production
production and and composition
and composition
composition et Self-Diagnosis
et al.,
processes (Sustotechnologies
etproducts
al., in
2017a) smart
(Theissler,
and devices
2017),
productions (Meneghetti
industrial
(Susto
Monitoring al., 2018a) and etproducts (Theissler, 2017),
has
has
has become
Monitoring
become
become very
Monitoring
very
of wells
very
of
significant
significant
significant
wells flow
in
flow production
in oil
in oil and
oil and
production and gas gas
gas
and
industry,
andindustry,
composition
industry,
composition
par-
par- processes
processes
et al., al., 2017b).
processes
par- processes
et 2018a)(Susto
(Susto
andet
(Susto et al., 2017a)
al.,
products
al., 2017a)
2017a) and
(Theissler,
and 2017), industrial
and productions
productions
productions (Susto
(Susto
industrial
(Susto
has become
ticularly
has become as very
fields
very significant
become
significant in oil and
economically
in oil and gas
gas industry,
marginal
industry, par-
and
par- et
et al.,
al., 2017b).
2017b).(Susto et al., 2017a) and productions (Susto
ticularly
ticularly
has become as fields
as very become
become economically
fieldssignificant economically
in oil and gas marginal
marginal
industry, and
and et
par- processes
et al.,
al., 2017b).
2017b).(Susto et al., 2017a) and productions (Susto
ticularly
reservoirs
ticularly as
as fields
deplete
fields become
(AL-Qutami
become economically
et al.,
economically 2018).marginal
In
marginalthis and
con-
and
reservoirs
reservoirs
ticularly deplete
asdeplete
fields (AL-Qutami
(AL-Qutami
become et
et al.,
al.,
economically 2018).
2018). In
In
marginalthis
this con-
con-
and In
et
In al., particular,
2017b).
In particular,
particular, in
particular, in
in this
in this
this work
this work
work we
work we
we present
we present a
present aaa AD
present AD
AD system
AD system
system to
system to
to
to
reservoirs
text, the
reservoirs pastdeplete
decade
deplete (AL-Qutami
has
(AL-Qutamiseen et increased
an
et al., 2018).
al., 2018). In this
this con-
attention
In con-
to- In
text,
text, the past
the past
reservoirs past decade
decade
deplete has seen
has seen
(AL-Qutamiseen an an
an
et increased
increased
al., 2018). attention
attention
In this to-
to- be
con- be
In
be
be implemented
particular,
implemented
implemented in on
this
on
on board
work
board
board the
we
the
the MPFM
present
MPFM
MPFM and
a
and
and AD that
that
that will
system
will
will be
to
be
be
text,
wards
text, the
Multiphase
the past decade
decade has
Flow
has Meters
seen an increased
(MPFM)
increased attention
(Corneliussen
attention to-
to- In particular,
implemented in this work
onquality
board of we present
theproduced
MPFM and a
and AD system
that will
will to
be
wards
wards
text, Multiphase
Multiphase
the past decade Flow
Flow
has Meters
Meters
seen an (MPFM)
(MPFM)
increased (Corneliussen
(Corneliussen
attention to- used
be
used
used to
to assess
implemented
to assess
assess the
assess the
theon
the board
quality
quality the
of
of MPFM
produced
produced metrology
metrology
metrology that mea-
mea-
mea-be
wards
et al.,
wards Multiphase
2005).
MultiphaseThe MPFMFlow
Flow Meters
is (MPFM)
a diagnostics
diagnostics (Corneliussen
and metering be
used implemented
to on board
quality the
of MPFM
produced and
metrology that will
mea-be
et
et al.,
al., 2005).
wards
et al.,
tool
2005).
Multiphase
2005).
that
The
The MPFM
The
provides
MPFM
MPFMFlow Meters
real-time
is
is a
Meters
is
(MPFM)
aa diagnostics
diagnostics (Corneliussen
and
and metering
(MPFM) measurements
diagnostics
simultaneous (Corneliussen
and metering
surements.
used to assess
surements.
metering surements.
surements.
used to assess
Detecting
Detecting
Detecting metrology
the quality of produced
metrology
metrology
the quality
Detecting of (i)
metrology
anomalies
anomalies
anomalies
produced
anomalies
metrology
metrology
in
in
in
MPFM
in MPFM
MPFM
MPFM mea-
mea-
is
is
is
is
et al.,
tool
tool
et 2005).
that
that
al., The
provides
provides
2005). The MPFM
real-time
real-time
MPFM is
is a
simultaneous
simultaneous
a diagnostics and metering
measurements
measurements
and metering challenging
surements.
challenging mainly
Detecting
mainly because:
metrology
because: (i) we
we must
anomalies
must decouple
in
decouple MPFM data
datais
tool
of thethat provides
commingled real-time
flow of simultaneous
oil, water and measurements
gas. It combines challenging
surements.
challenging mainly
Detecting because:
metrology
mainly because:
because: (i) we
(i) we
we must
anomalies
must decouple
in MPFM
decouple data
datais
tool
of
of thethat
the
tool provides
commingled
commingled
that provides of real-time
flow
flow of
real-timeof oil,simultaneous
oil, water
water and
simultaneous and gas. measurements
gas. It
It combines related
combines challenging
measurements related to metrology
mainly
to metrology
metrology anomalies
anomalies (i) from data
frommust associated
datadecouple
associated to
data
to
of
thethe commingled
measurement flow of
several oil, water
sensors and
withoutgas. It
thecombines
need to related
challenging
related to
to metrology
mainly anomalies
because:
anomalies (i) from
we
from data
must
data associated
decouple
associated to
data
to
of
the
thethe commingled
measurement
measurement
of the commingled of flow
offlow of
several
several oil,
of oil, water
sensors
sensors
water and
without
without
and gas.
gas. It
the
the
It combines
need
need to
combines new
to related
new operating
to conditions;
metrology
operating (ii)
anomalies
conditions; MPFM
(ii) MPFM from
MPFM generated
data
generatedassociateddata are
data areto
are
the measurement of several sensors without the need to new
related
new operating
toaffected conditions;
metrology (ii)
anomalies MPFM
from generated
dataconditions;
associateddata are
to
separate
the
separate
separate
the
the
measurement
the
the
measurement
phases.
of
phases.
phases.
of
Recently
several
Recently
Recently
several
Machine
sensors
Machine
Machine
sensors without
without
Learning
the
Learning
Learning
the
(ML)
need
(ML)
(ML)
need to
to new operating
strongly
strongly
strongly operating
affected
affected
conditions;
by
by
by the
conditions;
the
the flow
flow
flow
(ii)
(ii)and
and
and MPFMthe
the
the
generated
well
generated
well
well conditions;
conditions;
data
data (iii)
are
(iii)
(iii)
separate
approaches
separate the phases.
have been Recently
developed Machine
to enhance Learning
the (ML)
MPFM new
strongly operating
affected conditions;
by the flow (ii)
and MPFMthe generated
well conditions; data are
(iii)
separate the
approaches
approaches
approaches
capabilitiesthe(Yan phases.
have
have beenRecently
been
phases.
have been
et al.,
developed
developed
Recently
developed
2018).
Machine
to
to enhance
Machine
to enhance
enhance
Learning
the
the MPFM
Learning
the
(ML)
MPFM
MPFM
MPFM
(ML) strongly
MPFM
MPFM
strongly
MPFM
data
data are
affected
data are
affected
data are
multivariate.
by
by the
are multivariate.flow
flow and
multivariate.
multivariate.
the
The
Thethe
The
and
The
proposed
well
proposed
the
proposed
system
well conditions;
proposed system aims
system
conditions;
system
aims
(iii)
aims
aims
(iii)
approaches
capabilities
capabilities have
(Yan
(Yan been
et al.,
etbeen developed
2018).
al., 2018).
2018). to enhance the MPFM at
MPFM
at overcoming
overcoming data arethose challenges
multivariate.
those challenges The and
and provides
proposed
provides a reliability
system
a aims
reliability
approaches (Yan
capabilities haveet al., developed to enhance the MPFM at at
MPFMovercoming
overcoming data arethose challenges
multivariate.
those challenges The and
and providessystem
proposed
provides a reliability
a reliability
aims
capabilities (Yan et
et al.,
al., 2018). score
at associated with the measurements.
An
An
An ensemble
capabilities
ensemble
ensemble (Yan approach,
approach,
approach, based
2018).
based
based on on Neural
on Neural Networks,
Neural Networks,
Networks, is is at overcoming
score
score associatedthose
associated
overcoming
score
is score associated withchallenges
with
those
with the measurements.
the
the
and
and provides
measurements.
measurements.
challenges provides aa reliability
reliability
An
An ensemble
presented
ensemble in approach,
(AL-Qutami
approach, based
et al.,
based on
2018)
on Neural
to
Neural Networks,
improve the
Networks, flow is
is associated with the measurements.
presented
presented
An ensemble in (AL-Qutami
in (AL-Qutami
(AL-Qutami
approach, based et al.,
et al., 2018)
al., 2018)
2018)
on to improve
to improve
Neural improve the flow
the flow
Networks, flow is TheThe
score
The contribution
associated
contribution
The contribution
contribution of withof
of this
the
this
of this work
measurements.
work
this work
work can can
can
can bebe
be summarized
summarized
be summarized
summarized as as
as
as
presented
composition
presented in
in estimations.
(AL-Qutami et
etWith
al., the
2018) to
same
to aim,
improve the
a
theLong-
flow
composition
composition
presented estimations.
estimations.
inMemory-based
(AL-Qutami With
With the
the same
same aim,
aim, a
a Long-
Long- follows:
The
follows: contribution of this work can be summarized as
composition
Short Term
composition estimations. etWith
estimations.
al., 2018)
With
approach the same
the
to
same
is improve
shownaim,inaathe
aim,
flow follows:
Long-
(Andri-
Long-
follows:
The contribution of this work can be summarized as
Short
Short Term
Term Memory-based
composition
Short Term Memory-based
estimations.
Memory-based approach
approach
With
approach the is
is shown
same
is shown
shownaim, in
ina(Andri-
in (Andri-
Long-
(Andri- follows:
anov, 2018). Also in (AL-Qutami et al., 2017) Neural Net- • This
follows: This is is one
is one
one ofof the
of the first
the first Machine
first Machine Learning-based
Machine Learning-based
Learning-based tech- tech-
tech-
Short
anov,
anov, Term
2018).Memory-based
2018). Also
Also in
Short architectures
anov, Term
2018). in (AL-Qutami
Memory-based
Also in (AL-Qutami
(AL-Qutami
approach
approach et
et al.,is
et al.,
al.,
shown
is2017)
2017)
shown
2017)
in
Neural
Neural (Andri-
inthe
Neural Net-
Net-
(Andri-
Net- ••• This
This
This
nology
is
is
one
for
one
of
MPFM
of
the
the
first
in
first
Machine
literature;
Machine
Learning-based
Learning-based
tech-
tech-
work
anov,
work 2018). Also in are used
(AL-Qutami for theet estimation
al., 2017) of
Neural flow
Net- • nology
nology for MPFM
for MPFM
MPFM in literature;
in literature;
literature;
anov, architectures
work
work architectures
2018). Also
architectures
composition inare
(Radial
are used
used for
(AL-Qutami
are used
Basis
for the
for the
the
Network), et estimation
estimation
al., while
estimation in
of
of the
2017) Neural
of the flow
the flow
Net-
flow
(Ristanto
Thisthe
nology
•••• nology
To
To
is one
for
best
for
the best
best
ofofthe
MPFM ourfirst
ourin
of our
in Machine Learning-based
knowledge,
literature;
knowledge, this
this is is
is thethe tech-
first
the first ap-
first ap-
ap-
work architectures
composition
composition
work (Radial
(Radial
architectures are
are used
Basis
Basis
used for the
Network),
Network),
for the estimation
while
while
estimation in
inof
of the flow
(Ristanto
(Ristanto
the flow • To the
nology
To the best
for of
MPFM
of our inknowledge,
literature;
knowledge, this
this is the first ap-
composition
and Horne,
composition (Radial
2018)
(Radial a Basis
broader
Basis Network),
variety
Network), of while
ML
while in (Ristanto
algorithms
in (Ristanto is • proach
To the
proach in
best
in literature
of our
literature that
knowledge,
that applies
applies unsupervised
this is
unsupervisedthe first ML
ap-
ML
and
and Horne, 2018)
Horne, 2018)
2018) a broader variety of ML algorithms is • proach
To the
proach in
best literature
of our
in literature
literature that
knowledge,
that applies
applies unsupervised
this is
unsupervisedthe first ML
ap-
ML
composition
and Horne,
compared.
and Horne,
(Radial
2018) aa broader
a
broader
broader variety of
Basis Network),
variety
variety
ofwhile
of
ML algorithms
ML
ML
algorithms
in (Ristanto
algorithms
is
is
is techniques
proach
techniques in to MPFM
MPFMthat
to MPFM and implements
andapplies unsupervised
implements ML method-
ML method-
method-ML
compared.
compared.
and Horne, 2018) a broader variety of ML algorithms is techniques
proach
techniques in to
to MPFM
literature and
that
and implements
applies unsupervised
implements ML
ML method-ML
compared.
compared. ologies
techniques
ologies onboard
to MPFM
onboard the MPFM;
the MPFM;and implements ML method-
MPFM;
However,
compared. no work adopting unsupervised learning ap- ologies
techniques
ologies onboard
to MPFM
onboard the
the MPFM;
and implements ML method-
However,
However, no no work
no work adopting
work adopting unsupervised
adopting unsupervised
unsupervised learning learning
learning ap- ap-
ap- • The
ologies
The
• The
The work
work
work employs
onboard
employs
employs the state-of-the-art
MPFM;
state-of-the-art
state-of-the-art unsupervised
unsupervised AD
unsupervised AD
AD
AD
However,
proaches
However, to
no enhance
work MPFM
adopting capabilities
unsupervised has been
learning pre-
ap- • • ologieswork onboard
employs the MPFM;
state-of-the-art unsupervised
proaches
proaches
However, to
to
no enhance
enhance
work MPFM capabilities
MPFM unsupervised
adopting capabilities has has
has been pre-
been pre-
learning pre-
ap- • approaches
The work
approaches
approaches like
employs
like
like Isolation
Isolation Forest
state-of-the-art
Isolation Forest
Forest and
and
and Cluster-based
unsupervised
Cluster-based
Cluster-based AD
proaches
sented
proaches in to
the enhance
scientific
to enhance MPFM
literature. capabilities
This work aims been
at filling • The work
approaches employs
like state-of-the-art
Isolation Forest Forest andunsupervised
and Cluster-based
Cluster-based AD
sented
sented
proaches
sented in
in the
in the
to
the scientific MPFM
scientific
enhance
scientific literature.
literature.
MPFM
literature.
capabilities
This workhas
This work
capabilities
This work aimsbeen
aims
has
aims at
been
at
pre-
at filling
filling
pre-
filling
Local
Local Outlier
approaches
Local Outlierlike
Outlier
approaches
Local Outlier
Factor.
Isolation
Factor.
Factor.
like Isolation
Factor. Forest and Cluster-based
this gap
sented
gapin by
the introducing
scientific ML approaches
literature. This work for Anomaly
for aims at De-
this
this gap
sented
this gap
tection inby
theintroducing
by
by
(AD)
introducing
scientific
introducing
in MPFM.
ML
ML approaches
AD
approaches
literature.
ML is an This
approaches work
important
for
for Anomaly
Anomaly
aims
Anomaly
area at filling
of
De-
De- The
filling
De-
unsu- The
Local Outlier Factor.
rest
Local
rest ofof the
ofOutlier paper
Factor.
the paper
paper is organized
is organized
organized as as follows:
as follows:
follows: MPFM MPFM
MPFM
this gap
tection
tection by
(AD)
(AD)
this gap(AD)
by introducing
in MPFM.
in MPFM.
MPFM. AD
introducing ML
AD
AD
ML is approaches
is
is an important
an important
important
approaches for
for Anomaly
area
area of
Anomaly of De-
unsu-
of unsu-
unsu-
De- The The
The rest
rest of the
the paper is
is organized as follows: MPFM
tection in an area and
and its
rest
its principles
of the
principles are
paper
are described
is organized
described in
in Section
as
Section 2.
follows:
2. In
In Section
MPFM
Section 3
pervised
tection
pervised
pervised
tection
pervised
learning
(AD) in
in MPFM.
learning
learning
(AD)
learning MPFM.
that
that
that
aims
that aimsAD
aims
AD
aims
at
is
atan
isat
at
detecting
important
detecting
detecting
an important
detecting
anomalous
area
anomalous
anomalous of
area observa-
anomalous
data
of unsu-
data
data and
unsu-
data
and
Theproposed
the
and
its
its
its
principles
restprinciples
of the are described
paper
are described
Self-Diagnosis
principles are
is organized
described
in Section
in
solution
in
Section 2. In
as presented
is
Section
follows:
2.
2.
In Section
In
Section
MPFM
with
Section 33a33
that
that differ
pervised
differ significantly
learning that
significantly from
aims
from at previously
detecting
previously seen
anomalous
seen data the
the proposed
proposed Self-Diagnosis
Self-Diagnosis solution
solution is presented
is presented
presented with aaa3
with
that
that differ
pervised
differ significantly
learning that have
significantly from
aims
from at detecting
previously seen observa-
previouslyanomalous
seen observa-
data
observa- andproposed
the
brief
the
its principles
review
proposed of the are described
Self-Diagnosis
employed
Self-Diagnosis
in Section
solution
unsupervised
solution is
is
2.AD
presented
In methods.
Section
with
with
tions.
that
tions.
tions. AD
differ
AD approaches
significantly
approaches
AD approaches
that differ
tions. AD approaches
significantly have
have
have
from
from
been
been
been
been
employed
previously
employed
employedseen
previously
employed
for
seen
for Fault
Fault
for Fault
for Fault De-
observa-
De-
De- brief
observa-
De-
brief
brief
the proposed
brief review
review
review of of
of the
the employed
employed
ofSelf-Diagnosis
employed solution
the employed unsupervised
unsupervised
unsupervised AD
AD
is presented
AD methods. with aa
methods.
methods.
methods.
tions. review the unsupervised AD
tions. AD
AD approaches
approaches have have beenbeen employed
employed for for Fault
Fault De-
De- brief review of the employed unsupervised AD methods.
2405-8963 © 2019, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved.
Copyright
Peer review©
Copyright 2019
2019 IFAC
© under IFAC 212
212 Control.
responsibility of International Federation of Automatic
Copyright
Copyright ©
© 2019
2019 IFAC
IFAC 212
212
Copyright © 2019 IFAC
10.1016/j.ifacol.2019.09.143 212
Copyright © 2019 IFAC 212
2019 IFAC ICONS

Belfast, Tommaso Barbariol et al. / IFAC PapersOnLine 52-11 (2019) 212–217
Northern Ireland, August 21-23, 2019 213

Experimental settings and results are detailed in Section between capacitive and conductive modes, measuring
4; finally, conclusive remarks and future works are dis- respectively the permittivity and conductivity of the flow.
cussed in Section 5. Since multiple electrode couples are used, linear velocity
of the flow is determined through cross-correlation of the
2. MULTIPHASE FLOW METER measurements of each couple of electrodes.

Gamma Module - This technology is based on the prop-


The MPFM is non-intrusive, in-line meter for measuring erty of gamma rays to interact with matter and to expo-
flow rates of oil, water and gas in the dispersed phase of nentially attenuate as they pass through it. The attenua-
the flow. MPFMs are used for onshore and offshore oil tion is proportional to the amount of the passed material,
wells, both topside and subsea. In this work data have therefore the module detects the density variations in the
been collected using a MPFM (PietroFiorentini, 2011) flow. The source of gamma rays is the radioactive isotope
which combines the input of five sensors to compute the of Cesium 137 Cs, it has a half-life of almost 30 years.
flow rate of oil, water and gas (Falcone et al., 2009).
MPFMs exploit the combination of the following mea- 3. ML-BASED MPFM SELF-DIAGNOSIS APPROACH
surement principles:
To cope with the challenges described in the previous
• Venturi differential pressure;
section, we resort to modern unsupervised learning AD
• Electrical impedance;
approaches like Isolation Forest and Histogram-based
• cross-correlation of impedance measurements (ca-
Outlier Score. Moreover, the self-diagnosis system (as
pacitance/conductance);
shown on Figure 2) is composed of different procedures
• gamma ray densitometry;
to adapt to the peculiarity of the MPFM structure. This
• absolute pressure and temperature.
instrument is made up of different metrology modules to
Oil, water and gas flow rates are calculated based on the allow redundancy and robustness in the flow estimation.
measurements obtained by the electrodes and the mea- In particular, a windowing is in place to allow continuous
surement of the differential pressure (DP) across a Venturi monitoring, and an ad-hoc feature extraction (FE) pro-
throat. The capacitance or conductance of the mixture cedure designs quantities that capture the inter-module
flowing through the meter is measured in the impedance differences.
section. Velocity is measured by cross-correlating the high
The motivations that pushed the authors to develop a
resolution time signals from the electrodes. The MPFM is
new kind of FE will be analyzed in the first part of this
also equipped with a gamma densitometer. In the follow-
section, while the implemented method will be described
ing the various metrology modules are briefly described.
in the second part.
Venturi Module - The DP is measured at the inlet and the As explained in Section 2, the MPFM continuously mea-
throat of the venturi and is proportional to the velocity of sures different properties of the flow. The flow conditions
the fluids passing by and the density of the mixture. In the (namely the composition and mass discharge) can vary
same module both the absolute temperature and pressure widely over time and can be very different among dif-
are measured. ferent wells. According to literature (Brennen, 2005) the
flow can be classified as homogeneous, bubble, anular or
Impedance Module - Impedance module is made up of a slug flow. When the type of flow evolves, also the relation
series of electrods. Depending on flow regime, it works between the measured quantities changes: data form new
and different manifolds (Figure 3). For this reason the task
of detecting anomalous patterns in the measuring instru-
mentation can be particularly challenging. The aim of the
following method is to filter out the complexity given by
the flow dynamics, leaving to the AD algorithm only the
task of detecting anomalies in the instrumentation. In this
way the algorithm works independently on the operating
point.
The flowchart of the algorithm is presented in Figure
2. As previously stated in Section 2, the signals are
continuously collected from different modules X (i) , i ∈
{1, . . . , M }. Later the signals are preprocessed in order
to generate features that will be analyzed by an AD. The
output is a stream of labels that tags every observation as
outlier or inlier depending on the anomaly score.
3.1 Preprocessing

Since the algorithm works on fixed size batches, the sig-


nals need to be windowed. The windowing box splits the
original signals in small overlapping intervals of size τ
Figure 1. Example of a Subsea MPFM and time-overlap α.

213
2019 IFAC ICONS
Belfast,
214 Northern Ireland, August 21-23, 2019
Tommaso Barbariol et al. / IFAC PapersOnLine 52-11 (2019) 212–217

Windowing
(1)
Signal x1
Anomaly
Module X (1) ...
(1) at time t∗
Signal xm1 Cross
correlation
alignment Anomaly
... Input
detection
signals
algorithm
Normalization
(M )
Signal x1 Automated Anomaly at
Module X (M ) ... Root Cause time t∗ in
(M )
Signal xmM Analysis Module X∗
Feature
design

PREPROCESSING

Figure 2. Flowchart of the proposed algorithm. The Root Cause Analysis box is not discussed in this paper.

• all sensors at any given moment measure a differ-


ent property of the same underlying physical phe-
nomenon;
• the measurements from two different sensors are
linearly (or approx. linearly) correlated.
Our assumptions are supported by visual inspection of
the designed features (see Fig. 3 and Fig. 6) and by the
effectiveness of monitoring such quantities with AD as
proved by the experimental results reported in Section 4.
The efficacy of such feature design is obtained only when
the modules measure the same event at the same time,
namely when the signals are well aligned in time and
the correlation between them is sufficiently high. To en-
sure such alignment we resort to a cross correlation pro-
cedure; such choice over more sophisticated time align-
ment algorithms like Dynamic Time Warping (Keogh and
Ratanamahatana, 2005), was motivated by the computa-
tional resources constrains imposed by the MPFM.

3.2 Feature Extraction

The following step is the normalization of the windowed


Figure 3. Example of scatter-distribution plot during slug and aligned signals. In order to compare the features
flow conditions. Manifolds and clusters are typical from different signal windows, we have defined a refer-
of this condition, in this example there are not instru- ence batch, whose median and median absolute deviation
mental anomalies. G (Gamma), I (Impedance), DP have been used to normalize the other batches. The refer-
(Pressure Difference) time-aligned signals. Cross cor- ence batch is collected in controlled conditions and, thus,
relation KG,I = 0.94, KG,DP = −0.88. The acronym it does not contain anomalies.
[a.u] means arbitrary units.
In order to define features that encode the explained in-
sights, we consider the difference between the normal-
ized time series of all the modules. Therefore, the features
The insight behind this novel feature extraction is that zij are defined as:
all the correlated modules see the same physical process
from different points of view. An anomaly is likely to zij = xi − sign(Kij )xj zij = −zji
be present when the modules disagree: it occurs when where xi is a generic normalized windowed signal and
the informative content of one signal differs from all Kij is the correlation matrix between all the selected
the others. This intuition motivates the employed feature signals. The presence of the sign(·) is necessary when the
design approach that is based on two basic assumptions: considered data are negatively correlated.

214
2019 IFAC ICONS

Belfast, Tommaso Barbariol et al. / IFAC PapersOnLine 52-11 (2019) 212–217
Northern Ireland, August 21-23, 2019 215

3.3 Anomaly detection algorithms

The MPFM self-diagnosis system is finally equipped with


an AD algorithm that provides flags for anomalous con-
ditions and an Anomaly Score, a quantitative index that
defines the degree of anomaly of the considered data. The
MPFM will provide the users and the maintenance oper-
(a) Bias (b) Complete failure
ators, with useful information on the equipment health
status.
A typical approach in Fault Detection is to resort to clas-
sification methods, that rely on data tagged with faulty
and un-faulty conditions. However, labelling data in real-
world scenario is typically unfeasible, therefore we resort
to unsupervised AD algorithms.
Four different AD algorithms have been considered
in this work: (a) Cluster Based Local Outlier Factor (c) Drifting (d) Precision degradation
(CBLOF), (b) Principal Component Analysis (PCA), (c)
Histogram-based Outlier Score (HBOS) and (d) Isolation Figure 4. Type of synthetic anomalies used to test the
Forest (IF). A brief introduction to the aforementioned algorithm
AD methods is reported in the following. Some AD ap-
proaches that require long evaluation time, like k-Nearest The implementation employed in this work for the AD
Neighbour, have not been considered since they are not algorithms detailed in this Section is the one provided by
suitable for the application at hand. (Zhao et al., 2019).

Unsupervised AD approaches can be divided into differ- 4. EXPERIMENTAL RESULTS


ent families. One of these is represented by the so-called
density-based AD approaches; the most famous approach
4.1 Synthetic data generation
is the Local Outlier Factor (LOF). The degree of anomaly
in data is determined in LOF by taking into account the
clustering structure in a bounded neighborhood of the The algorithm explained in section 3 has been applied to
observation (He et al., 2003). In this work we resort to a semi-synthetic dataset. It is composed of three sets of
an optimized extension of LOF called Cluster-based Local data collected in dedicated facilities for testing MPFM.
Outlier Factor (CBLOF) (He et al., 2003). A simpler den- A set contains 150 operating points of 3 million samples
sity approach considered in this work is Histogram-based each. Even if we run unsupervised approaches, we need
Outlier Score (HBOS) (Goldstein and Dengel, 2012): for labeled data for evaluating the performances. Therefore
each considered feature, an univariate histogram is con- we create a semi-synthetic dataset injecting anomalies
structed first. The frequency (relative amount) of samples inside the real data. In this way the added noise is used as
falling into each bin is used as an estimate of the density a ’sure’ anomaly label, although ’natural’ anomalies may
(height of the bins). still be present in the data.

Another family of AD approaches is represented by Four types of sensor failures are considered in the lit-
Isolation-based methods (Liu et al., 2012), whose Anomaly erature (Narasimhan and Jordache, 2000), namely bias,
Score is defined by space partitioning. In Isolation For- complete failure, drifting and precision degradation (see
est (IF) (Liu et al., 2012), the most famous Isolation ap- Fig. 4 where examples of synthetic faults are depicted). In
proach, a procedure to partition the space based on ran- the following, only the drifting case will be discussed due
dom choices of variables and splitting points is in place; to the fact that this type of anomaly is the most frequent
the procedure iterates until the observation under exam and relevant in the application at hand.
is isolated, i.e. no other observations are contained in the
remaining space. The underlying idea of this approach is 4.2 Feature extraction
that outliers, being different from normal data, are easy to
be isolated, while inliers will require a more complicated Figure 5 shows the application of the FE procedure de-
and longer procedure. scribed in 3.1. We stress the effectiveness of the align-
ment procedure employed in the proposed MPFM self-
Finally a Principal Component Classifier-based approach diagnosis scheme: normal operating conditions of the
(PCA) is considered; PCA is a well know procedure used equipment are in fact associated with high cross correla-
for defying a set of uncorrelated variables, ordered by tion, as it can be appreciated in the data reported in Figure
explained variability, from a dataset. In the approach 5a.
proposed by (Shyu et al., 2003), an anomaly predictive
model is constructed from the major and minor principal As said, one of the main benefits of the proposed pre-
components of inliers. A measure of the difference of an processing phase is the decoupling of (a) different oper-
anomaly from the normal instance is the distance in the ating conditions (associated with normal process physi-
principal component space. cal dynamics), that the measuring system aims at detect-

215
2019 IFAC ICONS
216
Belfast, Tommaso Barbariol et al. / IFAC PapersOnLine 52-11 (2019) 212–217
Northern Ireland, August 21-23, 2019

(a) Windowed signals (b) Aligned signal

Figure 7. F1 score obtained by the different AD algo-


rithms at the change of the evaluation window. The
coloured dots show the score computed for every
(c) Normalized signal (d) Feature extracted experiment repetition while the solid line represents
the median. Maximum and minimum values are con-
Figure 5. Example of preprocessing on a synthetic dataset.
nected with a dashed line.
Precision degradation of the blue signal. z13 = x1 −
x3 .
disappears. By having a unique cluster, the work of AD
algorithms is greatly facilitated; in particular IF will ben-
efit from the proposed procedure, since it suffers for the
presence of multiple clusters (Hariri et al., 2018).

4.3 Comparison of anomaly detection algorithms

The classification ability of an AD algorithm can be sum-


marized by the F -metric (Saito and Rehmsmeier, 2015).
Figure 7 shows the F 1-score computed for every window
on a synthetic dataset where a drifting has been added to
real experimental data (Fig. 4c): in our settings, the 20%
of the data have an added degradation, while the rest are
retained as the original data. The experiment has been
repeated 10 times with the same settings. As expected, we
notice an increasing evolution of F 1 along the windows:
the outliers are much more difficult to detect in the first
window than in the last one since the drifting values
increase over time.
On average HBOS and IF perform better in the first win-
dows, but starting from window-2 CBLOF grows at much
Figure 6. Feature extraction applied to the slug flow in higher rate. The drawback of CBLOF is the high vari-
Figure 3. The flow dynamics has been removed from ance associated with its F 1-score. PCA is always outper-
data and the AD problem has become a lot simpler. formed by the other methods until window-6. It’s inter-
esting to notice that IF trend stabilizes around window-5.
ing, and (b) anomalous measures, the target of our self-
Table 1 shows all the metrics used to test the algorithms.
diagnosis system.
They refer to the threshold that maximises the F 1-score.
In fact, the effectiveness of the proposed pre-processing Moreover, given the importance for the application at
phase is demonstrated by Fig. 6 and Fig. 3. In presence hand, time complexity has been reported, it was measured
of different operating conditions (for example, slugs or in seconds on a 2.7 GHz Intel Core 5 Processor; w.r.t.
bubble flows), original data exhibit many clusters and this metric it can be appreciated there is no significant
complex manifolds but, exploiting the proposed features, difference between CBLOF and IF, while PCA and HBOS
the detection of instrumentation anomalies is greatly sim- are much faster, with a reduced time complexity of 3
plified and the influence of process physical dynamics orders of magnitude.

216
2019 IFAC ICONS

Belfast, Tommaso Barbariol et al. / IFAC PapersOnLine 52-11 (2019) 212–217
Northern Ireland, August 21-23, 2019 217

Table 1. Performance metrics for the tested AD algo- Goldstein, M. and Dengel, A. (2012). Histogram-based
rithms. These results pertain to one repetition done in the outlier score (hbos): A fast unsupervised anomaly de-
window-4 (see Fig. 7). tection algorithm. KI-2012: Poster and Demo Track, 59–
63.
CBLOF HBOS IF PCA Hariri, S., Kind, M.C., and Brunner, R.J. (2018). Extended
Time complexity [sec] 284.2 0.7 192.8 2.3 Isolation Forest. 1–11. URL http://arxiv.org/
F1 score [%] 86.8 74.7 66.8 66.0
Precision [%] 84.6 61.7 51.6 59.1
abs/1811.02141.
Recall [%] 89.2 94.8 95.0 74.9 He, Z., Xu, X., and Deng, S. (2003). Discovering cluster-
based local outliers. Pattern Recognition Letters, 24(9-10),
5. CONCLUSIONS AND FUTURE WORK 1641–1650.
Keogh, E. and Ratanamahatana, C.A. (2005). Exact index-
In this work we have presented a Machine Learning- ing of dynamic time warping. Knowledge and informa-
based approach to Anomaly Detection in Multiphase tion systems, 7(3), 358–386.
flow meters. The system allows the end user to promptly Liu, F.T., Ting, K.M., and Zhou, Z.H. (2012). Isolation-
detect anomalous measures and to have an indication of based anomaly detection. ACM Transactions on Knowl-
measure reliability on historical data. edge Discovery from Data (TKDD), 6(1), 3.
Meneghetti, L., Terzi, M., Del Favero, S., Susto, G.A., and
In particular the system exploits a preprocessing proce- Cobelli, C. (2018a). Data-driven anomaly recognition
dure that is very effective for distinguishing metrology for unsupervised model-free fault detection in artificial
anomalies within the typical physical dynamic of the un- pancreas. IEEE Transactions on Control Systems Technol-
derlying process. This model has been tested over semi- ogy.
synthetic datasets, based on real data, with multiple AD Meneghetti, L., Terzi, M., Susto, G.A., Del Favero, S., and
algorithms and type of anomalies. Cobelli, C. (2018b). Fault detection in artificial pan-
This study is the first step towards Multiphase Flow creas: A model-free approach. In 2018 IEEE Conference
Meters able to self-diagnose their metrology modules. on Decision and Control (CDC), 303–308. IEEE.
Our approach could be applied to other appliances that Narasimhan, S. and Jordache, C. (2000). Data Reconcilia-
have many different but correlated modules, electric cars, tion & Gross Error Detection. 1–57.
batteries, redundant systems. The proposed approach has PietroFiorentini (2011). (access february 27, 2019),
been designed for Plug & Play implementations, without mpfm flowatch hs datasheet. URL https://www.
the need of tuning the module for the well that hosts the fiorentini.com/ww/en/product/components/
MPFM. mpfm_eng/flowatchhs/.
Ristanto, T. and Horne, R. (2018). Machine Learning Ap-
As a future research direction, we will investigate dif- plied to Multiphase Production Problems. Ph.D. thesis, MS
ferent types of anomalies (ie. bias, complete failure) and Thesis. Stanford University.
different types of AD algorithms. In particular, we are Saito, T. and Rehmsmeier, M. (2015). The precision-
currently investigating approaches for Root Cause Anal- recall plot is more informative than the roc plot when
ysis that will allow the users and maintenance operator evaluating binary classifiers on imbalanced datasets.
to move from Anomaly Detection to Fault Classification. PloS one, 10(3), e0118432.
Shyu, M.L., Chen, S.C., Sarinnapakorn, K., and Chang,
REFERENCES L. (2003). A novel anomaly detection scheme based
on principal component classifier. Technical report,
AL-Qutami, T.A., Ibrahim, R., Ismail, I., and Ishak, M.A. MIAMI UNIV CORAL GABLES FL DEPT OF ELEC-
(2017). Radial basis function network to predict gas TRICAL AND COMPUTER ENGINEERING.
flow rate in multiphase flow. In Proceedings of the 9th Susto, G.A., Beghi, A., and McLoone, S. (2017a). Anomaly
International Conference on Machine Learning and Com- detection through on-line isolation forest: An applica-
puting, 141–146. ACM. tion to plasma etching. In 2017 28th Annual SEMI Ad-
AL-Qutami, T.A., Ibrahim, R., Ismail, I., and Ishak, M.A. vanced Semiconductor Manufacturing Conference (ASMC),
(2018). Virtual multiphase flow metering using diverse 89–94. IEEE.
neural network ensemble and adaptive simulated an- Susto, G.A., Terzi, M., and Beghi, A. (2017b). Anomaly de-
nealing. Expert Systems with Applications, 93, 72–85. tection approaches for semiconductor manufacturing.
Andrianov, N. (2018). A machine learning approach Procedia Manufacturing, 11, 2018–2024.
for virtual flow metering and forecasting. IFAC- Theissler, A. (2017). Detecting known and unknown
PapersOnLine, 51(8), 191–196. faults in automotive systems using ensemble-based
Brennen, C.E. (2005). Fundamentals of Multiphase anomaly detection. Knowledge-Based Systems, 123, 163–
Flow. Cambridge University Press. doi:10.1017/ 173.
CBO9780511807169. Yan, Y., Wang, L., Wang, T., Wang, X., Hu, Y., and Duan,
Corneliussen, S., Couput, J.P., Dahl, E., Dykesteen, E., Q. (2018). Application of soft computing techniques
Frøysa, K.E., Malde, E., Moestue, H., Moksnes, P.O., to multiphase flow measurement: A review. Flow Mea-
Scheers, L., and Tunheim, H. (2005). Handbook of surement and Instrumentation, 60, 30–43.
multiphase flow metering. Norwegian Society for Oil and Zhao, Y., Nasrullah, Z., and Li, Z. (2019). Pyod: A python
Gas Measurement (NFOGM), Revision, 2. toolbox for scalable outlier detection. arXiv preprint
Falcone, G., Hewitt, G., and Alimonti, C. (2009). Mul- arXiv:1901.01588.
tiphase flow metering: principles and applications, vol-
ume 54. Elsevier.

217

You might also like