SPE-167817-MS

Implementation of Complex Event Processing for Intelligent-Field

Muhammad Al-Gosayir, Moayad Al-Nammi, Waleed Awadh, Abdullah Al-Bar, Nasser Nasser, Saudi Aramco
Copyright 2014, Society of Petroleum Engineers

This paper was prepared for presentation at the SPE Intelligent Energy Conference and Exhibition held in Utrecht, The Netherlands, 13 April 2014.

This paper was selected for presentation by an SPE program committee following review of information contained in an abstract submitted by the author(s). Contents of the paper have not been
reviewed by the Society of Petroleum Engineers and are subject to correction by the author(s). The material does not necessarily reflect any position of the Society of Petroleum Engineers, its
officers, or members. Electronic reproduction, distribution, or storage of any part of this paper without the written consent of the Society of Petroleum Engineers is prohibited. Permission to
reproduce in print is restricted to an abstract of not more than 300 words; illustrations may not be copied. The abstract must contain conspicuous acknowledgment of SPE copyright.


Abstract
One of the main challenges for Intelligent Field engineers is handling large amount of data stream in real-time. Finding
meaningful patterns among this huge amount of data is just like trying to find a needle in a haystack. Every day reservoir and
production engineers are being bombarded with streams of massive amount of real-time data coming from all kinds of
Intelligent Field equipments including but not limited to Permanent Downhole Monitoring Systems (PDHMS), Multiphase
Flowmeters (MPFM), MicroMotion Meters, Wellhead Gauges, Smart Well Completion (SWC), and Electrical Submersible
Pumps (ESP). They spend significant amount of time and effort looking at the trends and analyzing the data to find
anomalies. Moreover, the existing systems for data cleansing and summarization are based on batch processing, hence,
engineers cannot make the right decision on time and they do not have the mechanism to instantly detect interesting patterns
as data coming in streams.
The objective of this paper is to share Saudi ARAMCO experience with Complex Event Processing (CEP) as an emerging
technology that is designed for low-latency and high-throughput event processing for data stream. This paper addresses the
architecture, the implementation, and the benefits of CEP as a solution for Intelligent Field. The implementation will cover
three common applications of CEP namely real-time data cleansing, pattern detection, and Event-Driven computation. Data
cleansing covers handling out-of-bound, negative, and frozen values. Patten detection enables the detection of anomalous
behavior in the data stream, such as unusual buildup or drawdown in well pressure in real-time. Event-Driven computation is
trigged by events in the field, such as change in downhole pressure to perform advanced calculation logic, such as average
reservoir pressure. Implementing CEP will help reservoir and production engineers obtain clean data in real time and receive
notification in case of any significant event detected.

Introduction
Technological innovation is perhaps the most significant driver to the production of any industry. Every year we see more
and more advancements in the capability of field equipment to monitor real-time data and provide benefits to the production
potential, recovery factor, and efficiency of wells. Saudi Aramco realizes this potential and already has plans to install real-
time monitoring systems to majority of wells under its domain. This will result in the generation of millions of data records
from the thousands of wells, each with tens of thousands of sensors. Such a large stream of data would require a robust,
reliable, and efficient data management scheme to ensure the continuous flow of data to allow for timely decision making and
better management of wells and reservoirs.
In short, real-time Intelligent Field Data Management is a new, complex, multidisciplinary, and expensive undertaking for
Saudi Aramco. The process of installing sensors deep inside oil and gas wells, at wellheads, at flow meters, and from points
along the flow lines is a challenge. This mission is adequately known as Saudi Aramcos Intelligent-Field program
(AbdulKarim et. al. 2010).
Complex Event Processing (CEP) is an innovative, rising technology that is designed to handle large amounts of data
events with minimal latency in real time. This technology can aid in the detection of trends and anomalies in the data stream
such as unusual buildup or drawdown in well pressure in real-time. Furthermore, CEP is an Event-Driven solution, meaning
it is triggered by events such as changes in the downhole pressure to perform computational logic to calculate the average
reservoir pressure. This in turn will provide reservoir and production engineers with clean real-time data and notifications of
any prominent events.

2 SPE-167817-MS

Overview of Intelligent Field Structure
For the implementation of the Intelligent Field, Saudi Aramco has adopted a four-layered architecture. These layers are
Surveillance, Integration, Optimization and Innovation as shown in Figure 1. The Surveillance layer is responsible for
continuous monitoring of real-time production data and makes use of data management tools to ensure the validity of the
data. The Integration layer processes real-time data to detect trends and anomalies. These anomalies are referred to reservoir
engineers for analysis and resolution. The Optimization layer streamlines field optimization capabilities and management
recommendations. The Innovation layer stores event knowledge and triggers optimization processes and actions throughout
the fields life-cycle. This knowledge management and lessons-learned layer captures and injects intelligence into the
system. (AbdulKarim et. al. 2010)


Figure 1: Intelligent Field Development Layers

Real-time data at Saudi Aramco traverses multiple zones of instrumentation, networks, and data servers before it reaches
the corporate database and engineers desktop. A typical piece of data captured by a sensor will be automatically transmitted
from the sensor and passes through multiple data servers connected together through wired/wireless networks. The data
eventually resides in the corporate database. (Naser and Awaji 2011)


Figure 2: Intelligent Field Infrastructure (Naser and Awaji 2011)



SPE-167817-MS 3

Complex Event Processing (CEP)
Complex Event Processing (CEP) is the continuous and incremental processing of event streams from multiple sources
based on a declarative query and pattern specifications with near-zero latency (Sezgin and Schindlauer 2009). CEP has many
applications, to name a few:
Web Analytics: Click-stream data analysis and online consumer behaviors.
Power Utilities: Power consumption monitoring.
Banking Services: Detecting anomalies in consuming habits to detect fraudulent behavior.

Over the years CEP has been receiving increasing attention in the Upstream industry. Patri et. al. (2012) highlighted how
CEP can be employed in Production optimization and pump failure detection. To put things into perspective, let us compare
between CEP and traditional relational database technology. In traditional databases, a user issues a request for a query and
the database sends back the response as shown in Figure 3.


Figure 3: Database Query Flow (Sezgin and Schindlauer 2009)

As an example, in a relational database the following query can be used to calculate the average pressure for the past
hour.

Relational database technology lacks support for temporal analytic capabilities and therefore will struggle with more
complex queries, such as calculating the running hourly average of the pressure every twenty seconds. Such a task would
require the use of a scheduled batch job that will run every 20 seconds to calculate the average. This approach suffers from a
number of issues; firstly, it does not scale well, with thousands of wells the database will be overwhelmed. Second, for every
new query it requires a scheduler, which can cause maintainability issues. Third, scheduling conflicts can arise when
complicated queries are used as input to other queries and this can be hard to debug. Therefore, CEP can resolve these
complications through the use of standing queries to process the data. Figure 4 showcases the query flow paradigm used by
CEP, as shown; the input stream is constantly processed as a standing query and generated through the output stream.


Figure 4: CEP Query Flow (Sezgin and Schindlauer 2009)
SelecL AvC(8eadlng_value) from Sensors_8eadlngs
Where Sensors 1ag=ressureSensor1" and
(8eadlng_1lme beLween 1lme-1h and 1lme)
4 SPE-167817-MS

This paradigm shift from the request/response of relational databases, CEP will process data on the fly as it comes from
the data source. All event processing is achieved in-memory and this contributes to the high-throughput low-latency goal.
Furthermore, CEP is already integrated with temporal analytical capabilities that allow it to perform complex queries such
as running averages and real-time pattern detection. To exemplify, CEP has temporal windowing features that allow query
builders to specify the period of time for each window of events. One such window is the hopping window, which defines a
subset of events that fall within some period, and over which you can perform some set-based computation such as an
aggregation (See Figure 5). A window will hop forward by a fixed time period based on two time spans: the hop size H and
the window size S. Thus, for every H time unit, a window of size S is created. (MSDN 2013)


Figure 5: Hopping Window (Sezgin and Schindlauer 2009)

Take our example of calculating the hourly-average every 10 seconds. CEP can achieve this by making use of the
hopping window feature and since this is a standing query, the results are updated automatically every ten seconds.



Another window feature is the tumbling window (Figure 6), which shares similarities with the hopping window except
that the windows are nonoverlapping. (Sezgin and Schindlauer 2009).


Figure 6: Tumbling Window (Sezgin and Schindlauer 2009)

This is just a taste of what CEP can offer and there are more advanced temporal features, such as snapshot windows and
counting windows. To summarize, the following Table 1 showcases the main differences between CEP and traditional
database technologies.

SPE-167817-MS 5

Table 1: Database vs. CEP (Sezgin and Schindlauer 2009)
Database CEP
Query Paradigm Ad-hoc queries or requests Continuous Standing queries
Latency Seconds, hours, days Milliseconds or less
Data Rate Hundreds of events/second More than tens of thousands of events/second

Architecture
A CEP system comprises of three constructs: the event source, the event-processing agent (EPA), and the event sink (See
Figure 7). The event source is responsible for pushing the data from the source, such as a database or data-generating
equipment, to the EPA in a format that is understood by the EPA. The EPA will then process these events coming from the
source and perform some computation like aggregation and/or trend analysis. Once the EPA is done with processing the
events, it will push the result towards another EPA for further processing or toward an event sink. The event sink will then
consume the results from the EPA and perform any necessary transformations to store it in a database, send an email
notification, or even initiate a custom workflow. (Hobach and Seeger 2013)

Figure 7: CEP Main Component (Hobach and Seeger 2013)

Recollecting our previous examples of the PI system and the Upstream database, we see that these two can be considered
as data sources. The proposed solution is to push the data in the PI system into the CEP engine, and use the Upstream
databases for fetching configuration information about the sensors, instrumentation, wells, reservoirs, and formations. Input
adapters will need to be implemented to transform the data from these two data sources into a format understood by the CEP
engine. After which, the event-processing agents come in to play. At the moment, there are three event-processing agents:
data cleansing and quality assurance, statistical analysis and transformation, and computation and pattern detection. After
processing the data, each EPA will push the results toward the appropriate next-in-line handler, which can either be another
EPA or an event sink. The event sinks can have a range of different tasks such as sending email notifications in the case of an
alerting scenario, for business reporting purposes, or even to store the results in the Upstream databases for use by other
business applications. To get the full picture, Figure 8 illustrates the architecture of the CEP engine.

6 SPE-167817-MS


Figure 8: CEP Implementation Components (Al-Bar et. al. 2013)

Applications
Data Cleansing and Quality Assurance
Engineers rely on clean data to ensure that their decisions can be as accurate as possible. Hence, it is of the utmost
importance that the data be filtered from erroneous values that can result from faulty instrumentation or from unaccounted
randomness. Such filtering can be categorized into two types: single event cleansing and multiple events cleansing. The
foremost is filtered based solely on that single event alone. Whereas the latter requires multiple events to apply the filtering
logic.
Single event cleansing includes errors such as out-of-range or negative values. By utilizing CEP technology, we can
devise a standing query that will filter out out-of-range values from the stream and divert those violations to another process.
To exemplify look at the following query:


As a result, any out-of-range value will be filtered out, as illustrated in the below Figure 9.


Figure 9: Out of range filtering

SPE-167817-MS 7

The removal of other types of invalid values such as stuck, frozen, outliers or noise would require the multiple events.
Take into consideration the filtering of stuck or frozen values. To achieve this efficiently using CEP, we would need to
compare consecutive events in a tumbling window. The period of this window can vary according to the business need or to
the period of acceptance of nonchanging values. For the purpose of this paper, we exemplify with the use of one hour
tumbling windows. See Figure 10.


Figure 10: Frozen/Stuck values Detection
Statistical Analysis and Data Normalization
To ease the extraction of knowledge from the large amount of real-time data coming in, two techniques are applied using
CEP to normalize and simplify the data. One technique is aggregation using statistical analysis that allows for the
computation of the sums, averages, standard deviations and regressions. The second technique involves the transformation of
individual sensor measurements data format to a more general well data format. The transformation will convert tag readings
from the Sensor ID, Timestamp, and Value layout, to a summarized well layout that includes: Well ID, Timestamp, Oil Rate,
Gas Rate, Water Rate, Choke Position, Bottom-hole Pressure, etc.

Pattern Detection
Detecting patterns and anomalies in the production environment of wells and reservoirs can be a challenge. Engineers
take an interest in a number of events that may arise such as when the pressure rapidly increases or decreases. An engineer
may also want to be alerted when a pressure buildup occurs and for how long this buildup has been taking. For this, a report
is generated daily to provide the engineer with the pressure build-up information, but it is not provided real-time. Using CEP,
it is possible to detect such a build-up by calculating the change in pressure over a specified tumbling window. Figure 11
shows an example of bottom-hole pressure buildup and drawdown during the shut-in period, transit state, or choke position
change over 25 days.
8 SPE-167817-MS


Figure 11: Pattern Detection

Event-Driven Computation
Event-driven computation is the ability to process data as it arrives, driven by the frequency of the data source. This
produces results as soon as, and only when, new input is available, and without any delay or unnecessary duplicate results.
(Nguyen et. al. 2011). One example of Event-Driven computation is calculating the datum pressure for a well. Because data
are coming from different sensors, the first step is to create a data stream for each tag.


A problem that may arise during calculation is that sensors may have different timestamps. To remedy this concern, the
solution is to convert the point event into a signal event. This will prolong the duration of the point event until the next event
timestamp. (See Figure 12)
SPE-167817-MS 9


Figure 12: Converting Point Event to Signal (MSDN 2013)

The following code displays the conversion of point events into signal events.



The last step is to join the two streams using the well name and perform the datum pressure calculation.



Conclusion
To conclude, this paper demonstrated the implementation and use of CEP technology for the Intelligent Field in areas
such as data cleansing, pattern detection and Event-Driven computation. Given the huge amount of data that the Intelligent
Field processes and the need to analyze this data in real-time to provide engineers with accurate information, we strongly feel
that utilizing technologies such as CEP in the Intelligent Field is becoming a necessity to maximize the benefits.
Acknowledgements
The authors would like to express their appreciation of Saudi Aramco management for permission to publish this paper.
10 SPE-167817-MS


Nomenclature
CEP: Complex Event Processing
EPA: Event-Processing Agent
ESP: Electrical Submersible Pumps
GOSP: Gas Oil Separation Plant
KPI: Key Performance Index/Indicator
MPFM: Multiphase Flowmeters
PDHMS: Permanent Downhole Monitoring Systems
PI: Plant Information
SCADA: Supervisory Control and Data Acquisition
SWC: Smart Well Completion

References
AbdulKarim, A., Al-Dhubaib, T., Elrafie, E. and Alamoudi, M.O. 2010. Overview of Saudi Aramcos Intelligent Field
Program, Paper SPE 129706, presented at the Intelligent Energy Conference and Exhibition, Utrecht, the Netherlands.
Al-Bar, A., Gosayir, M., Awadh, W., Nasser, N, Towaileb, H. 2013. Complex Event Processing Implementation for
Intelligent-Field. Paper SPE 167386, presented at SPE Middle East Intelligent Energy Conference and Exhibiiton,
Dubai, UAE, 2830 October.
Hobach, Bastian, and Bernhard Seeger. 2013. "Anomaly management using complex event processing: extending data base
technology paper." Proceedings of the 16th International Conference on Extending Database Technology. ACM
MSDN Library, 2013. "Time Stamp Modifications," Microsoft StreamInsight 2.0, Developer's Guide (StreamInsight),
Writing Query Templates in LINQ. MSDN http://msdn.microsoft.com/en-us/library/ee362414.aspx.
Nguyen, P.,Schindlauer R.,Sawyer J. 2011. "Event-Driven Solutions for the Oil & Gas Industry." Microsoft Technical White
Paper.
Patri, Om Prasad, Vikrambhai Sorathia, and Viktor Prasanna. 2012. "Event-driven Information Integration for the Digital
Oilfield." Paper SPE 159835, presented at SPE Annual Technical Conference and Exhibition, San Antonio, Texas, USA,
8-10 October.
Sezgin, B., Schindlauer, R. 2009. Advanced Microsoft SQL Server 2008 R2 StreamInsight Presented at Professional
Developers Conference (PDC), Redmond, Washington, United States, 28-29 October.
Sybase Inc., Complex Event Processing: Ten Design Patterns. Technical White Paper, 2010
Naser, A., Awaji, M. A., 2011. Intelligent Field Data Management: Case Study, SPE 149040, presented at SPE/DGS
Saudi Arabia Section Technical Symposium and Exhibition.