Manufacturing Letters 15 (2018) 139–142

Contents lists available at ScienceDirect

Manufacturing Letters
journal homepage: www.elsevier.com/locate/mfglet

Letters

A fog computing industrial cyber-physical system for embedded

low-latency machine learning Industry 4.0 applications
Peter O’Donovan ⇑, Colm Gallagher, Ken Bruton, Dominic T.J. O’Sullivan
School of Engineering, University College Cork, Ireland

a r t i c l e i n f o a b s t r a c t

Article history: Industrial cyber-physical systems are the primary enabling technology for Industry 4.0, which refers to an
Received 30 August 2017 emerging data-driven paradigm focused on the creation of manufacturing intelligence using real-time
Received in revised form 8 January 2018 pervasive networks and operational data streams. These cyber-physical systems enable objects and pro-
Accepted 11 January 2018
cesses residing in the physical world (e.g. manufacturing facility), to be tightly coupled and evaluated by
Available online 31 January 2018
advanced predictive analytics (e.g. machine learning models) and simulation models in the cyber world,
with the intention of realising self-configuring operations. Thus, this research presents an industrial
Keywords:
cyber-physical system based on the emerging fog computing paradigm, which can embed production-
Industry 4.0
Industrial cyber-physical systems
ready PMML-encoded machine learning models in factory operations, and adhere to Industry 4.0 design
Fog computing concerns pertaining to decentralisation, security, privacy and reliability.
Machine learning Ó 2018 Society of Manufacturing Engineers (SME). Published by Elsevier Ltd. All rights reserved.
Embedded analytics

1. Introduction At present, cloud and service-oriented computing appear to be

Industrial cyber-physical systems enable objects and processes
from the physical world, to be tightly coupled with compute, com-
munication and control systems in the cyber world [1]. Cyber-
physical interfaces connecting both worlds facilitate transmissions
using wireless sensors, smart phones, and tablets, to name a few
[2]. Conceptually, these cyber-physical interfaces manifest ‘cyber
twins’, where real-world physical objects are represented as virtual
objects in the cyber-world. In turn, these virtual objects may be
individually and/or collectively analysed, interrogated or simu-
lated to derive operational insights and inform decision-making.
A prominent emerging network paradigm promising to bridge
physical and cyber-worlds is that of the internet-of-things, which
comprises internet-enabled devices and gateways to sense, collect,
send and receive data [3]. In terms of manufacturing, this may
involve interactions with sensors, controllers, actuators, radio-
frequency-identification (RFID) tags, global positioning systems
(GPS), and high-definition cameras [3], to name a few. Naturally,
these continuous and pervasive interactions produce large data
repositories (i.e. big data) that describe factory operations [1]. Once
enough high-quality data has been captured, these large datasets
can be analysed using machine learning to make useful predictions
(e.g. equipment failures).
⇑ Corresponding author.
E-mail address: peter_odonovan@umail.ucc.ie (P. O’Donovan).
the most prominent compute paradigms used to implement indus-
trial cyber-physical systems [4–15]. However, traditional cloud
computing naturally conflicts with Industry 4.0 principles relating
to decentralised decision-making and reliable real-time control.
Although cloud and service-oriented computing can support dis-
tributed engineering scenarios, intelligence and processing (e.g.
decision-making) typically remain central (e.g. cloud server),
which means distributed clients depend on consistent and resilient
connections with the cloud. However, given industrial cyber-
physical systems may comprise networks-of-networks with uncer-
tain bandwidth, compute paradigms dependent on persistent con-
nections to centralised services are not suited to real-time
automation and control scenarios. To better address these Industry
4.0 concerns, compute paradigms supporting decentralised and
autonomous decision-making may be considered. Both multi-
agent systems [16,17] and fog computing [18–21] exemplify such
paradigms, where compute nodes operate autonomously to deliver
intelligence on the outer edge of pervasive networks, without
being concerned with persistent connectivity. In addition to
removing dependencies on external connectivity, the local and
autonomous operation of these paradigms can also reduce network
traffic (i.e. less requests), improve scalability (i.e. deploy more
nodes as needed) and enhance data security (i.e. data does not
leave the facility). Hence, this research presents an industrial
cyber-physical system that employs fog computing to deploy and
embed production-ready machine learning models, and compares

https://doi.org/10.1016/j.mfglet.2018.01.005
2213-8463/Ó 2018 Society of Manufacturing Engineers (SME). Published by Elsevier Ltd. All rights reserved.
140 P. O’Donovan et al. / Manufacturing Letters 15 (2018) 139–142
the reliability and consistency of the implemented fog cyber-
physical interface with that of a traditional cloud interface.
2. Industrial cyber-physical system
2.1. Proposed fog topology
Fig. 1 illustrates the composition of the fog computing topology
for delivering real-time embedded machine learning using cyber-
physical interactions. The cloud platform stores production-ready
machine learning models encoded as Predictive Modelling Markup
Language (PMML) for different engineering applications (e.g.
equipment prognostics), which are disseminated and executed by
fog nodes deployed within the facility’s local network. Although
these local operations promote data security and privacy,
factory-to-cloud communications depend on the facility’s existing
security policies and services governing internet communications.
Once communications from the factory are received by the cloud,
the request is authenticated using the fog node’s 128-bit Global
Unique Identifier (GUID). A cloud database of registered devices
is used to lookup the GUID, identify the engineering applications
handled by the node, and return relevant PMML models to down-
load or synchronise. The downloaded PMML models are stored on
the fog so they may be executed within the physical boundaries of
the factory, and deliver real-time predictions and decision-making
(e.g. control changes) without persistent connections to the cloud.
2.2. Technical architecture
Fig. 2 illustrates the technical components used to implement
the industrial cyber-physical system. First, the sensing layer con-
tains the industrial equipment and systems to continuously
acquire real-time measurements, and an embedded software agent
to mediate communications between physical and cyber environ-
ments. Second, the fog layer contains technical components to
receive inbound data streams, execute PMML-encoded machine
Fig. 1. Composition of fog computing with cyber-physical interactions.
learning models, and return results. Finally, the cloud layer contains
technical components to maintain metadata about each fog gate-
way deployed in the factory (e.g. engineering applications etc.),
persist PMML-encoded models in a global repository, and dis-
charge relevant PMML models to fog gateways as machine learning
models are added or updated.
2.3. Performance assessment
A series of load/stress tests were applied to the implemented
fog and cloud cyber-physical interfaces to evaluate their (a) relia-
bility: maximum execution time, and (b) consistency: number of
failed communications. These performance parameters were cho-
sen given their fundamental importance to time-dependent con-
trol and engineering applications. The cloud and fog cyber-
physical technologies used during the experiments employed stan-
dard out-of-the-box configurations to protect from potential
biases, while experiments were executed in close proximity to
reduce fluctuating environmental conditions (e.g. broadband
throughput, local network activity etc.) contaminating perfor-
mance measurements. These measurements were captured using
a test computer hosting the open source load testing application
JMeter. The JMeter agent was configured with experiment param-
eters to send, receive and measure transmissions for each cyber-
physical interface. The OpenScoring engine was installed behind
both interfaces to handle JMeter requests, and execute the
PMML-encoded model using the data provided. The PMML model
was derived from an existing Support Vector Machine (SVM)
model that predicts faulty heating operations within industrial
air handling units.
JMeter was configured with several scenarios that continuously
increased the load on the cyber-physical interfaces. These stress
tests instructed JMeter to simulate 50, 100, 250 and 500 concurrent
connections (e.g. controllers, smart sensors etc.), and execute 1000
requests for each connection (e.g. 50 concurrent connections
would result in 50,000 requests).
 P. O’Donovan et al. / Manufacturing Letters 15 (2018) 139–142 141

Fig. 2. Fog computing architecture, components and technologies.

3. Results Table 2 summarises the quantity and percentage of communi-

Table 1 summarises the maximum execution times observed for
each stress test, which may indicate worst case execution times. The
findings show the fog interface recorded significantly lower maxi-
mum execution times compared to the cloud interface, with differ-
ences in maximum latency measured at 92.9% (50 connections),
99.4% (100 connections), 67.7% (250 connections) and 91.0% (500
connections). These differences may be attributed to the localised
and embedded characteristics of fog computing, which possess
fewer dependencies on network routing and external connectivity.
cation failures, with the best performing cyber-physical interface
highlighted as the percentage difference of failure rates between
the fog and cloud interface. Assuming there are no hardware, soft-
ware or network issues, failure rates may arise from timeouts due
to excessive loads being placed on the cyber-physical interfaces.
These failures can be easily identified within the stress testing
dataset given they are recorded as zero milliseconds. The fog inter-
face did not record any failed communications for the stress testing
scenarios, successfully responding to 50,000, 100,000, 250,000, and
500,000 requests. Although the cloud interface demonstrated

Table 1
Summary of maximum execution times for fog and cloud interfaces (milliseconds).

No. Connections Fog Max. Cloud Max. Difference Max. Difference Max. (%)
50 2571 36,034 33,463 92.9%
100 436 70,397 69,961 99.4%
250 32,798 101,401 68,603 67.7%
500 7536 83,370 75,834 91.0%

Table 2
Summary of failed communications for fog and cloud interfaces.

No. Connections Fog (Failed) Fog (% of All) Cloud (Failed) Cloud (% of All) Failed Diff. (%)
50 0 0.00% 0 0.00% 0.00%
100 0 0.00% 112 0.11% 0.11%
250 0 0.00% 3556 1.42% 1.42%
500 0 0.00% 32,994 6.60% 6.60%
142 P. O’Donovan et al. / Manufacturing Letters 15 (2018) 139–142
comparable failure rates for 50 connections (0%), modest failures
were evident for 100 connections (0.11%) and 250 connections
(1.42%), with more substantial failures observed for 500 connec-
tions (6.6%).
4. Conclusions
This research considers the use of fog computing to deliver real-
time embedded machine learning applications for Industry 4.0
operations. The initial findings highlight the fog’s ability to deliver
consistent and reliable cyber-physical interactions for real-time
engineering scenarios (e.g. industrial control processes), while
cloud computing may support engineering scenarios that are toler-
ant to occasional failures and unencumbered by time constraints.
However, those factories adopting Industry 4.0 design principles,
real-time decision-making, and self-optimising operations should
find the fog’s inherent characteristics better suited to minimising
failures and high-latency communications.
In addition to the performance benefits, the proposed fog topol-
ogy also promotes data security and privacy by ensuring operating
data and model execution remain within the physical boundaries
of the facility. Although not the primary focus of this research,
other decisions influenced by security included (a) employing the
HTTP protocol due to the maturity of policies governing internet
communications, and (b) utilising hash-based authorisation
scheme to identify and associate fog nodes residing in facilities.
A more thorough investigation of cyber-physical system security
should be undertaken as part of future work.
Of course, fog computing also has limitations that require fur-
ther investigation. Firstly, given fog nodes possess modest compute
capacity, the benefit of local and embedded topologies may be off-
set when engineering applications require significant processing
power. Secondly, although the proposed fog topology can dis-
tribute work amongst nodes to scale resources, such approaches
depend on the availability or deployment of physical devices in
the factory, which contrasts with the instantaneous auto-scaling
capability of commercial cloud platforms. Finally, there are fewer
libraries, frameworks and platforms supporting fog computing,
which naturally requires teams and stakeholders to invest more
resources in development, support and security.
References
[1] Bagheri B, Yang S, Kao H, Lee J. Cyber-physical Systems Architecture for Self-
Aware Machines in Industry 4. 0 Environment. IFAC Pap.; 2015. pp. 1622–
1627.
[2] Riedl M, Zipper H, Meier M, Diedrich C. Cyber-physical systems alter
automation architectures. Annu Rev Control 2014;38(1):123–33.
[3] Babiceanu RF, Seker R. Big data and virtualization for manufacturing cyber-
physical systems: a survey of the current status and future outlook. Comput.
Ind., no. 2015. p. 2016.
[4] Wen Z, Liu X, Xu Y, Zou J. A RESTful framework for Internet of things based on
software defined network in modern manufacturing. Int J Adv Manuf Technol
2016;84(1–4):361–9.
[5] Lu Y, Cecil J. An Internet of Things (IoT)-based collaborative framework for
advanced manufacturing. Int J Adv Manuf Technol 2016;84(5–8):1141–52.
[6] Brizzi P, Conzon D, Khaleel H, Tomasi R, Pastrone C, Spirito AM, et al. Bringing
the Internet of Things along the manufacturing line: a case study in controlling
industrial robot and monitoring energy consumption remotely. ETFA: IEEE Int.
Conf. Emerg. Technol. Fact. Autom; 2013.
[7] Brusaferri A, Ballarino A, Cavadini FA, Manzocchi D, Mazzolini M. CPS-based
hierarchical and self-similar automation architecture for the control and
verification of reconfigurable manufacturing systems. In: 19th IEEE Int. Conf.
Emerg. Technol. Fact. Autom. ETFA 2014; 2014.
[8] Iarovyi BS, Mohammed WM, Lobov A, Ferrer BR, Lastra JLM. Cyber–physical
systems for open-knowledge-driven manufacturing execution systems. Proc
IEEE 2016;104(5):1142–54.
[9] Ruan J, Yu W, Yang Y, Hu J. Design and realize of tire production process
monitoring system based on cyber-physical systems. In: Proc. – 2015 Int. Conf.
Comput. Sci. Mech. Autom. CSMA 2015; 2016. p. 175–9.
[10] Schulz D. FDI and the industrial internet of things: protection of investment for
industrie 4.0. In: IEEE Int. Conf. Emerg. Technol. Fact. Autom. ETFA vol. 2015–
Octob, 2015.
[11] Bangemann T, Riedl M, Thron M, Diedrich C. Integration of classical
components into industrial cyber-physical systems. Proc IEEE 2016;104
(5):947–59.
[12] Zhang DQ, Jinadasa KBSN, Gersberg RM, Liu Y, Ng WJ, Tan SK. Application of
constructed wetlands for wastewater treatment in developing countries – a
review of recent developments (2000–2013). J Environ Manage
2014;141:116–31.
[13] Tao F, Wang Y, Zuo Y, Yang H, Zhang M. Internet of things in product life-cycle
energy management. J Indus Inf Integr 2016:1–14.
[14] Qu T, Lei S, Chen Y, Wang Z, Luo H, Huang GQ. Internet-of-things-enabled
smart production logistics execution system based on cloud manufacturing;
2014. pp. 1–7.
[15] Lee J, Bagheri B, Kao HA, Lapira E. Industry 4.0 and manufacturing
transformation. Manuf Leadersh J 2015;6(1):36–43.
[16] Mařík V, McFarlane D. Industrial adoption of agent-based technologies. IEEE
Intell Syst February 2005;20:27–35.
[17] Fischer K, Schillo M, Siekmann J. Holonic multiagent systems: a foundation for
the organisation of multiagent systems. Holonic Multi-Agent Syst Manuf
2003;2744:1083–4.
[18] More P. Review of implementing fog computing. Int J Res Eng Technol 2015;4
(6):2319–22.
[19] Chen R-Y. An intelligent value stream-based approach to collaboration of food
traceability cyber physical system by fog computing. Food Control
2017;71:124–36.
[20] Varshney P, Simmhan Y. Demystifying fog computing: characterizing
architectures, applications and abstractions; 2017. pp. 1–23.
[21] Wu D, Liu S, Zhang L, Terpenny J, Gao RX, Kurfess T, et al. A fog computing-
based framework for process monitoring and prognosis in cyber-
manufacturing. J Manuf Syst April 2017;43:25–34.