Available online at www.sciencedirect.

com

ScienceDirect
Journal of Electrical Systems and Information Technology 1 (2014) 266–275

Comparisons process-to-bay level peer-to-peer network delay in IEC

61850 substation communication systems
Nasser Hasan Ali * , Borhanuddin Mohd. Ali, Mohammed A. Abdala,
Mohammad Lutfi Othman, Fazirulhisyam b. Hashim
Department of Computer and Communication System, Faculty of Engineering, University Putra Malaysia, UPM Serdang, 43400 Selangor,
Malaysia
Available online 19 December 2014

Abstract
This paper presents the application of IEC 61850 protocol in electrical power engineering industry for data communication systems
between substations. This IEC 61850 protocol presents new challenges for real-time communication performance between Intelligent
Electronic Devices (IEDs) within substation because of the Generic Object Oriented Substation Event (GOOSE) messages. The
analyses of substation Ethernet and WLAN (wireless LAN) communication delay, its impact factors, various methods and different
network topologies which can improve the real-time performance are discussed. For basic analysis of data flow within a substation, the
Optimized Network Engineering Tool (OPNET) software is used. The process-to-bay level network simulation model is performed
by using the OPNET software. The Ethernet delay and WLAN peer-to-peer performance of the process-to-bay level network
simulation results are analyzed which is based on AP (access point), switched, shared Ethernet network or peer-to-peer network.
© 2015 Electronics Research Institute (ERI). Production and hosting by Elsevier B.V. All rights reserved.

Keywords: IEC 61850; Substation automation system (SAS); Real-time peer-to-peer network delay; Power system communications; OPNET
Modeler; Access point (AP)

1. Introduction

IEC 61850 protocol is a new generation of smart substation automation system communication networks and
communication protocol standards. Through to the unified information integration modeling of the substation internal
all IEDs data object, the unified language has been used to describe a definition according to the object service
technology and abstract communication service standard interface. In the substation, the seamless communication of
all the layered distributed intelligent IEDs data information resources realizes time sharing. In the smart substation
automation system the application of IEC 61850 standard communication protocol can realize interoperability between

∗ Corresponding author.
E-mail addresses: nasir200420022000@yahoo.ca (N. Hasan Ali), borhan@eng.upm.edu.my (B. Mohd. Ali), lutfi@eng.upm.edu.my
(M.L. Othman), fazirul@eng.upm.edu.my (F.b. Hashim).
Peer review under responsibility of Electronics Research Institute (ERI).

http://dx.doi.org/10.1016/j.jesit.2014.12.008
2314-7172/© 2015 Electronics Research Institute (ERI). Production and hosting by Elsevier B.V. All rights reserved.
 N. Hasan Ali et al. / Journal of Electrical Systems and Information Technology 1 (2014) 266–275 267

Fig. 1. Substation levels.

all the IEDs, and realize system automation function extension compatibility and operation and long-term accurate
stability. It is also the fundamental premise to realize time sharing of the data information resources, is an important
assure to realize the based intelligent of the process level IEDs information integration modeling, is an important
direction to research and development the bus communication technology (Anderson et al., 2003). The structure of
smart substation system in physics can be divided into two categories: intelligence of primary equipment and network
of secondary equipment. According to the IEC 61850 communication protocol definitions in the logical structure can
be divided into “process level”, “bay level”, and “station level” (Brand et al., 2003). Internal and middle of all levels
use high speed network communication, and the relation of three levels is shown in Fig. 1.
Substation communication network required high level real-time message transmission, especially from process-
to-bay level communication network performance. Transmission protocol, network load and networking mode are the
main factors affecting the real-time performance of the SAS network (Hoang, 2004; Kaneda et al., 2008; Haizhu,
2011). Among the transmission delay constitution factors, message packaging and disassembling take up most of the
time, which relates to the transmission model and protocol (Haizhu, 2011). According to the IEC 61850-5, the message
transmission time requirements for the SAS network must be ensured under any operating conditions and contingencies
inside the substation to be consistent. Moreover, the dynamic performance of the SAS network must be analyzed during
the planning stage in order to find the network performance problems ahead of the deployment stage (Sidhu and Yin,
2006). The modeling of communication systems requires the definition of objects (such as data objects, data sets,
report control, and log control) and services provided by the objects (such as get, set, report, create, and delete) (Sidhu
and Yin, 2007; IEC, 2003). However, IEC 61850 has selected mainstream technology for the communication stack.
The stack structure is based on the International Standard Organization (ISO) Open Systems Interconnection (OSI)
layers consisting of Ethernet (layers 1 and 2), TCP/IP (layers 3 and 4) and manufacturing messaging specification
(MMS) layers 5–7. The object model and its services are mapped to the MMS application layer (i.e., layer 7). Only
time-critical services, such as Sampled Values and Generic Object Oriented Substation Event (GOOSE) messages, are
mapped directly to the Ethernet link layer (i.e., layer 2) in order to ensure openness and interoperability of systems
(Kanabar and Sidhu, 2011). Additional mappings to other communication stacks are possible. The network topology
is critical when using the GOOSE messaging. Since that message may include protective signals, redundancy of the
Ethernet LAN might be required (IEC, 2013). As part of the IEC 61850 standard, special GOOSE messages are also
planned for a quick exchange of information between the IEDs (Sidhu and Yin, 2007; Ozansoy et al., 2007). Fig. 2
shows an example of Ethernet communication mapping (Ozansoy et al., 2007).
IEC 61850 can work in WLAN (wireless local area network) (Brand et al., 2004). WLAN uses RF (radio frequency)
technology, which replaces a local area network constituted by the old twisted copper. It can make wireless local area
network to make use of simple access architecture to allow users to access to data. It constitutes a network system
which can communicate with each other and realize resources sharing, so the network construction and the moving
of the terminal is more flexible. Wireless network according to the application area can be divided into three aspects:
268 N. Hasan Ali et al. / Journal of Electrical Systems and Information Technology 1 (2014) 266–275

wireless personal area network (WPAN) have ZigBee, Bluetooth, UWB, etc.; wireless local area network (WLAN)
has 802.11 standard class, ad hoc; wireless metropolitan area network (WMAN) has 802.16/WiMAX.
The wireless network building and applications of substation mainly adopt the wireless local area network (WLAN)
to achieve reliable communication. The Optimized Network Engineering Tool (OPNET) modeler 14.5 is used to
simulate various types of SAS network under different network and scenarios, allowing user to set the raw sample rate,
fault time, number of faults, background traffic and other configuration parameters (Sidhu and Gangadharan, 2005; IEC,
2003). In this paper, peer-to-peer and star network communication architectures have been used for the modeling of
IEC 61850 based communication protocol and analyzed the simulated results. For example, (i) according to substation
communication network performance requirements, substation communication network architecture solutions based
on IEC 61850 standard have been analyzed. The feasibility of the Ethernet substation communication network has
also been analyzed. (ii) Substation messages transmission delay and its impact factors using several methods that
can improve the real-time performances are discussed. A comprehensive study of substation communication network
packet transmission delay of the composition and the impact of this delay are considered. There are several ways to
improve Ethernet real time effective measures which are included as well. (iii) It has proposed to characterize the data
stream for transmission of data in substation communication network as well as grouping the IEDs communication ports
for particular protection in the SAS. Using the OPNET network simulation, the IEC 61850 based substation network
simulation model is established and the real-time performances in substation network with different bandwidths,
configurations and types of networks are analyzed (Kanabar and Sidhu, 2011; OPNET, 2014; Ozansoy et al., 2007;
CDTPR, 2005; Wester et al., 2011; Skeie et al., 2002; IEC TC, 2001).
This paper is organized as follows: introduction is presented in Section 1, IEC 61850 standards in Section 2, model
of a substation automation system (SAS) in Section 3, and simulation setup is provided in Section 4. Section 5 presents
the results and discussion. Finally, the conclusion is given in Section 6.

2. Overview of IEC 61850

The first edition of the IEC 61850 standard comprises 12 parts in total. Each part describes different aspects of the
communications taking place within a substation automation system. For example, part 7 presents the logical view of
the communication system, i.e., logical models of devices, abstract communication service interface (ACSI), etc., while
parts 8 and 9 specify how these logical concepts ought to be mapped into a specific communications infrastructure, i.e.,
protocol stack. Throughout this investigation part 9-2 of the standard was scrutinized in depth since it describes how
Sampled Values protocol should be designed and mapped to Ethernet (ISO/IEC 8802-3). Part 9-1 was also analyzed,
however, in less detail since it is likely to be withdrawn from the future releases of the standard (Kim and Lee, 2005).
IEC 61850 defines five types of communication services as shown in Fig. 2. The first three types are time-critical and
are used in protection and control schemes of the substation. They include Sampled Values (SV) and Generic Object
Oriented Substation Event (GOOSE) protocols which are mapped directly to Data Link layer for reduced protocol
overhead and hence increased performance; and Generic Substation State Event (GSSE) protocol which features its

Fig. 2. Ethernet communication mapping based on IEC 61850 protocol.

 N. Hasan Ali et al. / Journal of Electrical Systems and Information Technology 1 (2014) 266–275 269

Table 1
Performance requirements for smart-grid applications in a distribution substation (Parikh, 2013).
Smart distribution substation application IEC 61850 message Allowed delay at distribution (s) QoS priority levels

Control and monitoring

Automatic capacitor bank control GOOSE 0.5–1 4
LTC control and monitoring GOOSE 0.25–0.5 6
Fast transfer trip scheme for bus GOOSE 0.4–0.5 5
Automation and metering application
Watt/VAR measurement IP message/Sample Values 1–5 3
Centralized IED configuration IP messages 1–10 2
Protection
Feeder over current protection Sample Value and GOOSE 0.02 7

own custom protocol mapping. The remaining two types of services, i.e., Time Sync and Client-Server Manufacturing
Message Specification (MMS), deal with time synchronization and management of the substation devices respectively.

3. Model of a substation automation system (SAS)

3.1. Substation level structure

SAS involved several items, such as control, monitoring, alarm, protection and live reporting for the status of
equipment’s in the substation. Substation based on IEC 61850 logically can be divided into three sections: (i) station
level, (ii) bay level, and (iii) process level, as shown in Fig. 1. The station level device has human machine interface
computers, operator workstations, and remote communication interfaces and devices configuration. The bay level
devices are control relays, switch gears and metering gauge and the process level device has typical and remote
I/O’s, smart sensors and actuators as well as merging unit IEDs (Skeie et al., 2002). Protection and control data were
exchanged between bay and station levels or direct among station level. From the station level, remote engineers are
able to monitor and configure through the satellite or fiber optic communication networks. Bay and process levels were
exchanged instantaneous control and live data as well as direct control data exchange between the bay levels (IEC TC,
2001).

3.2. Substation message type and data size

Based on IEC 61850 protocols, there are several types of messages or data flow within the substation. To allow for
different requirements of the substations, some message types are also subdivided into performance classes. There are
two independent groups of performance classes: (a) control and protection and (b) metering and power quality appli-
cations. Since the performance classes are defined according to the functionality requirements, they are independent
from the size of the substation (Janssen and Koreman, 2013). The message transmission requirements time and bit
resolution in the IEC 61850 protocol is listed in Table 1.

4. Simulation setup

4.1. Simulation model

IEC 61850 uses different levels for different network topologies. Fig. 3 shows a simple process-to-bay level network
simulation model. A peer-to-peer network topology has been used in a substation process-to-bay level network for
protection functions and other functions which is required for fast delivery of critical messages. The IEDs between the
bay level and process level are connected together through the switches.
OPNET Modeler constructs the models using object-oriented modeling approach. Network devices like IEDs,
switches and workstations are called node models. Node model consists of modules connected by packet streams
or static wires. Each module is assigned to a process module to achieve the required behaviors. OPNET’s process
model uses finite state machine (FSM) approach to support the implementation of protocols, resources, applications,
270 N. Hasan Ali et al. / Journal of Electrical Systems and Information Technology 1 (2014) 266–275

Fig. 3. Process-to-bay level.

Fig. 4. The process model for the raw data source.

algorithms, and queuing policies. States and transitions graphically define the progression of a process in response to
events. Each state of a process model contains embedded C/C++ code, supported by an extensive library of functions
designed for network programming. Fig. 4 shows the process model for the raw data source in the merging unit (MU)
IED model.
The modeling of merging unit IED is based on IEC 61850-9-1 (Brand et al., 2004). The Ethernet broadcast address is
used as a default destination address although this model also supports unicast and multicast. The user could configure
the sample rate, start time, stop time, packet size, address, and multicast group address if multicast is used as the
transmission type. The communication stack for merging unit IED is very simple. It contains an application layer,
Ethernet layer, and physical layer. Fig. 5 shows the node model diagram for MU IED.
The OPNET Modeler software is used to analyze the real time performances of the SAS communication network
based on IEC 61850. Each network node was connected with the Ethernet communication link by 10-BaseT. The
OPNET software is used to simulate the data communication between the process and bay levels (i.e., the publisher
and subscriber). The application configuration was set with application parameters, which describe the applications
of the message type, including the size of a packet or the data sending interval. During a simulation process, packets
sent from the IEDs are received by the hub receiver object (hub rx 0 0) and processed up the protocol stack to the
application module. After processing, they are sent down the stack to the transmitter (hub tx 0 0), and then back to
 N. Hasan Ali et al. / Journal of Electrical Systems and Information Technology 1 (2014) 266–275 271

Fig. 5. Node model diagram for MU IED.

Fig. 6. Data configuration for the network model.

Fig. 7. IEC 61850 simple source packet format.

the IEDs (Ozansoy et al., 2007; CDTPR, 2005; Wester et al., 2011; Skeie et al., 2002; IEC TC, 2001; Janssen and
Koreman, 2013; Lebakken and Orlando, 2003; Preiss and Wegmannn, 2003; Das et al., 2012). Fig. 6 shows the data
configuration for the network model as shown in Fig. 3 (i.e., process-to-bay level network simulation model). This data
configuration is used for the network simulation. In this case, the raw data source is simple source with finite state
machine as shown in Fig. 4 and packet format as shown in Fig. 7. And the packet inter-arrival time was set as 0.0025 s
with 128 bytes and the IEC 61850 protocol compliance with Ethernet IEC 802.3. The ETE (end-to-end delay) result
of the above scenario simulation is shown in Fig. 8.
Fig. 9 shows the same configuration using TCP protocol to compare with the above scenario. While the setting of
this scenario is shown in Fig. 10.
272 N. Hasan Ali et al. / Journal of Electrical Systems and Information Technology 1 (2014) 266–275

Fig. 8. ETE (end-to-end) delay for peer-to-peer using station-adv.

Fig. 9. Peer-to-peer using workstation-adv. under TCP-IP protocol.

Fig. 10. Data configuration table for Fig. 9.

 N. Hasan Ali et al. / Journal of Electrical Systems and Information Technology 1 (2014) 266–275 273

Fig. 11. End-to-End delay for workstation-adv. under TCP-IP Protocol at different size packets.

Fig. 12. Peer-to-peer WLAN.

Finally simulation peer-to-peer WLAN scenario is shown in Fig. 12. Using station-adv. with physical layer char-
acteristic IEEE 802.11b direct sequence with 11 Mbps data rate. The end-to-end delay characteristic is shown in
Fig. 13.

5. Results and discussion

This section analyzes the overall performance of the peer-to-peer and customized configuration with star topology
network, the sending packet size, sending interval and the impact of end-to-end data caused by different bandwidth
of media link as well as customize data configuration for the particular protection in the SAS. By changing a single
variable in the simulation, simply it is possible to analyze the real-time performance of the message transmission in
substation’s IEDs communication (Ozansoy et al., 2007; Das et al., 2012).
The end-to-end delay of message in different packet sizes is shown in Figs. 8, 11 and 13. The packet sizes were
128, 256, 512, and 1024, respectively. It shows clearly that with the increase of message packet sizes, the end-to-end
delay time increases with the constant data generation time interval of 0.0025 s which equivalent to 400 Hz sampling
frequency that enough for MU to take eight sample from one cycle 50 Hz original signal during over current protection.
Also in Fig. 13, since only two workstations are connected to WLAN network, then no collisions, and congestions
problems, the average delay was taken then it is appear approximately constant for each adopted packet size this delay
274 N. Hasan Ali et al. / Journal of Electrical Systems and Information Technology 1 (2014) 266–275

Fig. 13. End-to-end delay performance of WLAN at different size packets.

actually vibrate between maximum and minimum value if it is taken as it is, also this delay depend on load packet of
P&C IED, and number of retransmission GOOSE message.
When a data packet size is 256 bytes, the end-to-end delay is about 0.0001 s for station-adv., and about 0.0003 for
workstation-adv., and 0.0017 s for WLAN. For this reason, in the actual or a GOOSE message configuration, it should
avoid sending a variety of data streams at the same time as well as too large packets to meet the real-time requirements
(Sidhu and Yin, 2007; Das et al., 2012). It is easily verified that the end-to-end Ethernet delay time was affected by
the data generation interval, which means that sampling rate or GOOSE messages generation from the instruments or
IEDs must not be varied. Also the fast data generation resulted in longer delay time for the end-to-end Ethernet data
transmission. We can conclude from these simulations that the specific protection IEDs must be connected on the same
Ethernet switch in order to get the fast and reliable delivery of the trip messages or other important messages.

6. Conclusion

This paper presented the performance of IEC 61850 for peer-to-peer communications between substations. Based
on the simulation results, it is recommended for practical applications, and the size of the packet must use small packets
and avoid sending a non-uniform data streams to ensure real time and reliable data transmission. To meet the high
real-time requirements for substation protection relay message data stream, with modeling of communication between
bay level protection and process level instruments IEDs as GOOSE messages, the main factors affecting delay were
packet sizes, Ethernet link bandwidth and data generation or sampling interval time as well as the IED’s communication
architecture of the intended protection. Those communication cables must be connected on the same Ethernet switch or
grouping the port on the Ethernet switch. Therefore, for practical applications, it is suggested to consider a reasonable
configuration of parameters and communication network architecture for real time data transmission and ensure the
normal operation of the substations. Also we see that the end-to-end delay for WLAN was much greater than Ethernet
network; this is due to the fact that WLAN was influenced by invariant noise and distance and other physical factors.

References

Anderson, L., Bmnner, Ch., Engler, F., 2003. Substation automation based on JEC 61850 with new process-close technologies. In: IEEE Power
Tech., Bologna, Italy, June 23–26.
Brand, K.-P., Lohmann, V., Winuner, W., 2003. Substation Automation Handbook. Utility Automation Consulting Lohmann.
Brand, J.K.-P., Brunner, Ch., Wimmer, W., 2004. Design of IEC 61850 based substation automation systems according to customer requirements.
In: Cigre 2004. Paper 195-04, Paris.
 N. Hasan Ali et al. / Journal of Electrical Systems and Information Technology 1 (2014) 266–275 275

February 2005. Communication Delivery Time Performance Requirements for Electric Power Substation Automation, Std. 1646-2004. IEEE Power
Eng. Soc. (Power Eng. Soc.)/IEEE Std., NY, USA.
Das, N., Ma, W., Islam, S.,2012. Bay-level network end-to-end delay in IEC61850 substation communication system. In: Proc. of the 11th
Postgraduate Electrical Engineering and Computing Symposium (PEECS 2012). Curtin University, Perth, Australia, November 9.
Haizhu, W., 2011. Analysis of real-time performances of process-level networks based on IEC61850 SCSM model. In: Proc. of the 4th Int’l Conf.
on Electric Utility Deregulation and Restructuring and Power Technologies (DRPT), Weihai, Shandong, China, July 6–9.
Hoang, H., 2004. Real-time communication for industrial embedded systems using switched Ethernet. In: Proc. of the 18th International Parallel
and Distributed Processing Symposium (IPDPS’04), Santa Fe, NM, USA, April 26–30, pp. 127–130.
2003. IEC 61850-5 Communication Requirements for Functions and Device Models, first ed., pp. 43–48.
2013. IEC 61850 – Communication Networks and Systems in Substations. Parts Issued as International Standard (IS) Between 2002 and 2004 by
International Electrotechnical Commission IEC.
IEC TC-57, 2001. Communication Networks and Systems in Substations Part 7-4: Basic Communication Structure for Substation and Feeder
Equipment-Compatible Logical Node Classes and Data Classes, Draft Std. 61850-7-4. Int. Electrotech. Comm. (IEC), Geneva, Switzerland.
Janssen, M.C., Koreman, C.G.A., 2013. Substation Components Plug and Play Instead of Plug and Pray: The Impact of IEC 61850. Kema T&D
Power, Netherlands.
Kanabar, M.G., Sidhu, T.S., 2011. Performance of IEC 61850-9-2 process bus and corrective measure for digital relaying. IEEE Trans. Power Deliv.
26 (April (2)), 725–735.
Kaneda, K., Tamura, S., Fujiyama, N., Arata, Y., lto, H., 2008. IEC 61850 based substation automation system. In: Proc. of the Joint Int’l Conf. on
Power System Tech. and IEEE Power India Conference (POWERCON 2008), New Delhi, India, October 12–15.
Kim, G.-S., Lee, H.-H., 2005. A study on IEC 61850 based communication for intelligent electronic devices. In: KORUS’2005, pp. 765–770.
Lebakken, T.M., Orlando, D.R., 2003. Substation automation and communication standards: IEC 61850 and DNP3. In: Electric Light & Power.
2014. OPNET Modeler-OPNET Technologies, Sighted from the website: http://www.opnet.com
Ozansoy, C.R., Zayegh, A., Kalam, A., 2007. The real-time publisher/subscriber communication model for distributed substation systems. IEEE
Trans. Power Deliv. 22 (July (3)), 1411–1423.
Parikh, P.P., Sidhu, T.S., Shami, A., 2013. A comprehensive investigation of wireless LAN for IEC 61850-based smart distribution substation
applications. IEEE Trans. Ind. Inform. 9 (August (3)), 1466–1476.
Preiss, O., Wegmannn, A., 2003. Towards a composition problem based on IEC 61850. J. Syst. Softw. 65 (3), 227–236.
Sidhu, T.S., Gangadharan, P.K., 2005. Control and automation of power system substation using IEC61850 communication. In: Proc. of 2005 IEEE
Conference on Control Applications, Toronto, Canada, August 28–31.
Sidhu, T.S., Yin, Y., 2006. IED modelling for IEC 61850 based substation automation system performance simulation. In: IEEE Power Engineering
Society General Meeting, Canada, June 18–22.
Sidhu, T.S., Yin, Y., 2007. Modelling and simulation for performance evaluation of IEC 61850 based substation communication systems. IEEE
Trans. Power Deliv. 22 (July (3)), 1482–1489.
Skeie, T., Johanessen, S., Brunner, C., 2002. ETHERNET in substation automation. IEEE Control Syst. J. Mag. 23 (June (3)), 43–51.
Wester, C., Adamiak, M., Vico, J., 2011. IEC 61850 protocol – practical applications in industrial facilities. In: Proc. of the 2011 IEEE Industry
Applications Society Annual Meeting, Orlando, FL, USA, October 9–13.