Communication system for Smart Grids EE-576

M.ENGG. ELECTRICAL ENGINEERING PROGRAMME, FALL SEMESTER 2022

INSTRUCTOR: DR. SUNDUS ALI, ASSISTANT PROFESSOR, TELECOMMUNICATION ENGINEERING, NED UET

Week 12 TOPIC:
Smart Grid Network Design Process

Email: sundus@neduet.edu.pk, sundus@cloud.neduet.edu.pk

Office Location: Telecommunication Engineering Building Faculty Offices
 2
Network Reliability

 Smart Grid network reliability is fundamental to power system reliability. As a result,

power system reliability requirements for utility operation applications (that require
network communication) drive network reliability requirements.
 Network reliability requirements are often specified in terms of (the probability of) the
availability of a network connection between endpoints. Thus, 99.999 % reliability for a
network connection translates to network unavailability probability of 0.00001 (due to
failures in the network).
 Often, such unavailability is informally specified as the amount of time that the network
will be unavailable in a year. Thus, for 99.999 % reliability, the unavailability
(0.00001*365*24*60) is less than 5.3 min on average per year. However, reliability
specification being a probability, such informal interpretation does not indicate a
definitive period of unavailability in a year.
 3
Network Reliability

 High values for end-to-end reliability require that the individual network equipment (routers,
network switches, optical multiplexers, wireless base stations, etc.) and network links are
themselves very reliable. It is advisable that “utility-grade” network equipment is deployed for
increased reliability.
 Among other things, utility grade equipment may imply redundant configurations of processors,
switching fabric, and power supplies, reliable software, “hardening” of the systems in the
equipment to withstand and not lose its performance under extreme conditions such as in an
outdoor substation environment, and other features.
 The reliability of network links depends on many factors including the communication medium,
environment (such as underground or over the ground), weather, and quality of mechanical
network connections.
 4
Network Reliability

 For wireless networks, communication over licensed spectrum is more reliable than
over unlicensed spectrum due to high interference in the unlicensed spectrum from
other users of the spectrum.
 Underground cabling for communication is often more reliable than over-the-
ground communication.
 Network availability is dependent on the frequency of failures of the nodes and the
links in the network. Reliability of each of these components is generally specified in
terms of Mean Time Between Failure (MTBF) of the components.
 Another similar metric used particularly for network equipment and/or components is
the failure rate called Failure in Thousand (FIT), the expected number of failures in a
device in 109 (one billion) hours of operation.
 5
Network Reliability

 In addition to the information on the failure of its components, network availability also depends
on the time to repair (or replace) the failed component(s). The Mean Time to Repair (MTTR)
metric is used to quantify this metric.
 In many situations, replacing a component is faster than repair assuming that spares are readily
available.
 Network reliability can be improved by using multiple physical paths as either parallel and
separate links between adjacent network elements or two or more paths between two
endpoints, each going over a separate set of intermediate network elements. Such multiple
physical paths must be managed, often using standards based network redundancy tools.
 6
Network Reliability

1. Link or Path Protection with SONET/SDH Rings: SONET/SDH connection between multiple nodes
forms a ring with two paths between every pair of nodes in opposite direction around the ring. If
there is a fault in any link, every pair of nodes remains connected through the remaining links. The
reconfiguration to the alternate path takes 50 ms.
2. Ethernet Link Aggregation: Two or more links connecting a pair of Ethernet switches are
configured as one single Ethernet connection between them. If a link fails, the Ethernet
connectivity is not affected with the Ethernet connection being defined over the remaining link(s).
3. Spanning Tree for Ethernet Network: In an Ethernet network, only a spanning tree of switches in
the network can be used as active links carrying traffic even if other links (if present in the network)
are in good working condition and available. Thus, if the Ethernet network of n nodes has more
than n-1 links, only n-1 links can be active at any time (since the spanning tree is a minimally
connected graph of the Ethernet switches). If one of the links in the current spanning tree
configuration (active link) fails, the network can reconfigure the spanning tree, provided that a
spanning tree can be formed from the available and working links in the network.
 7
Network Reliability

4. Routing Protocols: As described earlier, routing protocols such as OSPF and IS–IS are designed to
find an alternate path in an IP network if a node or a link fails in the network, if an alternate path
exists.
 8
Network Reliability

 In a large network, a combination of these and other network redundancy schemes may be used.
For example, for a reliable WAN, there must be at least two separate paths between every pair of
WRs.
 The reliability required for connecting the DCCs and other critical utility locations such as some
substations may not be satisfied by connecting these locations to only one WR, since if the CR–WR
link fails, these critical locations will be disconnected from the network.
 Therefore, these locations connect to the WAN over two separate links connecting to two
separate WRs, if increased reliability is necessary.
 If the dually-connected CR is collocated with a WR, one of these two connections will be
supported over the LAN at that location, with the other being a FAN connection to another WR.
 Some of the reliability requirements may be satisfied by connecting the CRs with each other
directly, in addition to their individual connections to the respective WRs.
9
 10
Network Reliability

 Thus, it is not necessary for the Smart Grid network to be “trees” outside of the WAN. In some
cases, connections between CRs may be preferred (such as between CRs at substations C and
D) over connecting a CR (such as at substation B) to two different WRs based on cost
considerations as well as for supporting an application that requires very low delay.
 For example, to support the delay requirement of less than 10 ms for teleprotection between
substations, a direct link between the substations is preferred over sending the teleprotection
traffic over the WAN.
 There may already be one or more physical links between some pairs of (transmission)
substations for teleprotection. Thus, these links may be used in the Smart Grid network for carrying
traffic for other applications to improve the network reliability without incurring additional costs of
new FAN links – provided the performance, reliability, and security of teleprotection application
are not affected.
 If required for reliability and as dictated by the design, links between some pairs of substation
routers may be deployed if they did not already exist.
 11
Network Security Elements

 Configuration of access control lists (ACLs) in routers (WRs, IRs, and CRs) helps filter unwanted data
traffic based on the IP headers in every packet entering the router. For example, an ACL may allow or
block traffic entering the router that originated from a specific endpoint (based on its IP address).
Similarly, another ACL element may allow or block traffic destined for a particular IP address. Other
ACLs may allow or block traffic satisfying other contents of the IP header. While ACLs do serve
important security functions, they are often not adequate for security for most mission-critical
applications.
 Firewall (FW) and/or intrusion detection (IDS) and/or intrusion detection and prevention (IPS) systems
are deployed in the network for additional security. Firewalls may be integrated in some router
products or may be stand-alone products. IPS/IDS systems are often stand-alone products. These
security systems have evolved through the years often providing overlapping security functions. It is
difficult to define the firewall, IDS, and IPS systems individually.
 Functionality of such security appliances includes deep packet inspection and detection algorithms
based on application signatures, anomalies, and behaviors. For mission-critical applications, these
systems can include signatures for protocols specific to the critical infrastructure. Increasingly, router
products are integrating some of these functions. Whether integrated with the routers or as stand-
alone products, we will refer to these network security functions simply as firewall/Unified Threat
Management (UTM).
 12
Network Security Elements

 For security, data entering or leaving a router passes through the FW/UTM system.
 The FW/UTM system inspects the incoming traffic to examine any security threats and takes appropriate
action based on the system configuration that may include implementation of utility network security policies.
The level of traffic inspection goes beyond just the inspection of IP headers of individual packets
independently in each packet of the traffic.
 Higher-level headers and even the data in the packet payload may be inspected (deep packet inspection)
to identify application signatures corresponding to harmful applications to ascertain if the traffic is legitimate.
Traffic inspection may be stateful in that a stream of packets is observed to determine security threat instead
of just a single packet at a time.
 Based on the security threats identified, the FW/UTM system may discard the unwanted traffic, close the
session between the end systems, report security violation, or take other actions supported by the system and
based on the security policies. The FW/UTM systems are also placed at the WR locations and routers
connecting to the external networks.
 Data encryption may be implemented between the endpoints of an application for additional security. IP
Security (IPsec) standard [RFC4301] provides for secure tunneling between two endpoints of an IP network to
transfer encrypted traffic between them. IPSec provides security against threats to data privacy.
13
 14
Network Scalability

 Presently, Smart Grid is in its infancy. Many applications described in this book will be introduced
by utilities over the next few years. There will be many other applications and new application
functions that will be developed as Smart Grid matures.
 It is extremely important that the Smart Grid network is able to support new applications with as
few modifications in the network architecture as possible.
 The Smart Grid network architecture developed here is well suited to support new applications
with little churn in the network itself.
 Therefore, the network design should allow for easy accommodation of application introduction
with only network configuration modifications and connections to a few new endpoints, if any.
 Often there is no need for increasing network bandwidth or deployment of new physical network
connections except for connecting the new endpoints that will need to be placed in new
locations.
 As more endpoints are added over time with successful introduction of the application, the task
of supporting the large number of additional endpoints is the capacity management task and
not really a new application introduction task.