Electric Power Systems Research 122 (2015) 140–151

Contents lists available at ScienceDirect

Electric Power Systems Research

journal homepage: www.elsevier.com/locate/epsr

Situation awareness in power systems: Theory, challenges and

applications
Mathaios Panteli a,∗ , Daniel S. Kirschen b
a
School of Electrical and Electronic Engineering, The University of Manchester, Sackville Street, Manchester M13 9PL, UK
b
Department of Electrical Engineering, University of Washington, Campus Box 352500, Seattle, WA 98195, USA

a r t i c l e i n f o a b s t r a c t

Article history: Due to the increasing size and operational complexity of modern power systems, transmission system
Received 21 July 2014 operators often have difficulties forming a complete and accurate picture of the state of the part of the
Received in revised form system for which they are responsible. These difficulties may prevent them from achieving the level
27 November 2014
of situation awareness (SA) that they need to make the right decisions and respond effectively to an
Accepted 11 January 2015
incident. Inadequate SA has indeed been identified as one of the contributing factors in several recent large
electrical disturbances worldwide. This paper first reviews the fundamentals of SA and then discusses
Keywords:
the main sources of operator errors due to insufficient SA in power systems and how these affect the
Control center
Decision-making
operational decision-making process. It then discusses tools and standards that can help system operators
Power systems improve their level of SA. Finally, a generic procedure for achieving sufficient SA is presented, which aims
Situation awareness to guide the design of an information system that is both technology- and user-oriented.
Power System Observability © 2015 Elsevier B.V. All rights reserved.
Power System Operation

Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
2. Role of situation awareness in recent major disturbances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
3. What is situation awareness? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
3.1. Individual SA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
3.2. SA for team and collaborative operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
3.3. SA in the context of power systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
4. SA challenges in power system control centers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
4.1. Hardware & software applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
4.2. Real-time measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
4.3. Environmental factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
4.4. Automation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
4.5. Individual factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
4.6. Data & information sharing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
5. Filters in operational decision-making process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
6. Supporting situation awareness in power systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
6.1. Increasing the accuracy of state estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
6.1.1. Bad data processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
6.1.2. Topology error processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
6.2. Improving GUI effectiveness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
6.3. Dealing with highly automated systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
6.4. How to reduce operators’ individual errors? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

∗ Corresponding author. Tel.: +44 1613068721.

E-mail addresses: mathaios.panteli@manchester.ac.uk (M. Panteli), kirschen@uw.edu (D.S. Kirschen).

http://dx.doi.org/10.1016/j.epsr.2015.01.008
0378-7796/© 2015 Elsevier B.V. All rights reserved.
 M. Panteli, D.S. Kirschen / Electric Power Systems Research 122 (2015) 140–151 141

6.5. Wide-area monitoring and collaborative visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

6.6. Ensure the functionality of hardware and software applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
7. Framework for achieving adequate situation awareness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
8. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

1. Introduction processing system, and a lack of information sharing between sys-

tem operators led to a degraded OSA and consequently to a delayed
Numerous incidents in various domains demonstrate the impor- response to the initial electrical failure, resulting in the spreading of
tance of situation awareness (SA) in carrying out operations safely the disturbance. In the Italian blackout of 28th September 2003 [4],
and reliably. For example, in 1989, a major aircraft accident insufficient information exchange between the Italian and Swiss
occurred during take-off from an airport in Canada, because the operators prevented the development of a common understanding
pilot was not aware that a large amount of snow and ice on the of the problem, which resulted in a lack of effective joint control
wings, reduced the lift of the aircraft. This led to a loss of control and actions and to the isolation and complete blackout of the entire
the crash of the aircraft [1]. Several power system blackouts, some Italian peninsula. In the UCTE system disturbance on 4th November
of which are discussed in Section 2, also illustrate what can happen 2006 [5], the operators based their decisions on empirical assess-
in a complex system, when the people involved in its operation are ments without performing a contingency analysis using updated
not fully cognizant of what they need to know. Driven by numerous data. The problem was compounded by a lack of coordination
similar incidents, organizations from different domains have devel- between operators in different control centers. The initial electri-
oped advanced information systems to support the development of cal outage could then propagate and split the European Grid into
a sufficient SA. three islands (West, North–East, and South–East), with a significant
Acquiring adequate SA is particularly critical during electrical power imbalance in each area. Due to a lack of proper operational
disturbances. In general, the progress of a blackout can be divided planning and adequate cognition of the evolving system conditions,
into two main phases: the pre-cascading phase, where the situa- part of the Western US Interconnection was operated in an N-1
tion evolves slowly enough that operators have time to react, and insecure state. Following a routine electrical problem, this led to
the cascading phase, where components trip so rapidly that opera- a blackout in Arizona and Southern California on 8th September
tors are not able to respond in a timely manner and fast automatic 2011 [6].
devices take over when emergency action is required, either to The following main conclusions can be derived from these
protect components or to try to salvage part of the system using examples of blackouts where OSA played a key role:
under-frequency load shedding. Panteli et al. [2] show that inade-
quate operators SA (OSA) has a significant impact on the probability • Despite the increasing use of automation, human operators
of the system entering the cascading phase of a blackout. To pre-
remain an integral part of modern power systems.
vent cascading outages it is therefore vitally important to support • OSA is a key factor in preserving power system security. Inade-
the effective and timely decision-making during the pre-cascading
quate OSA may result in a delayed, incorrect or deficient response
phase, before the situation becomes uncontrollable.
by the operators to an electrical event or even cause errors dur-
However, operating power systems is an extremely challeng-
ing routine operational procedures, endangering power system
ing task because of their size and complexity as well as the large
reliability.
number of contingencies that can occur. Operators should there- • It is critical to develop and maintain sufficient SA at the early
fore be provided with the data and the information that they need
stages of a disturbance, before the situation becomes uncontrol-
to understand the current state of the system and be able to project
lable. This enables effective and fast reaction by the operators,
its future behavior. Because power systems are getting increasingly
preventing cascading outages.
interconnected, system operators need to be aware not only of the • As power systems have become highly interconnected, it is vitally
state of their own control area, but also of the state of neighboring
important for the system operators to develop a wide-area SA
control areas.
that goes beyond the administrative borders of their control area.
The aims of this paper are to review the fundamentals of SA,
They need to be aware of events that may occur in neighboring
provide an overview of existing tools and methods for enhancing
networks, which even though they may be geographically distant,
OSA, and discuss how OSA could be further improved in transmis-
might be electrically close enough to affect the stability of their
sion control centers. Section 2 discusses the impact of situation
own network.
awareness on the severity of recent major electrical disturbances.
Section 3 defines SA and discusses the factors that influence it. Sec-
tion 4 discusses the main sources of lack of SA in power system 3. What is situation awareness?
control centers. Section 5 discusses how the information required
for operational decision-making is filtered by the ICT infrastructure Situation awareness has attracted the interest of researchers
and human perception. Section 6 reviews existing SA technolo- and practitioners in a variety of domains, including military
gies, while a generic procedure for achieving adequate OSA is [7], aviation [8,9], air traffic control [10], automotive [11] and
provided in Section 7. Section 8 summarizes and concludes the C4i (command, control, communication, computers and intelli-
paper. gence) [12] and environment [13]. While early research on SA
took place in the military and aviation domains, driven by the
2. Role of situation awareness in recent major disturbances recognition that SA played a key role in numerous incidents in dif-
ferent domains, it has developed into a critical research theme in
Inadequate OSA has been identified as a contributing factor to almost any area that involves humans performing tasks in com-
the development and severity of recent electrical disturbances. In plex and challenging environments. A distinction should be made
the Northeast USA blackout of 14th August 2003 [3], numerous fail- between two main categories of SA: individual and team or shared
ures in the information system, such as state estimator and alarm SA.
142 M. Panteli, D.S. Kirschen / Electric Power Systems Research 122 (2015) 140–151

• System Capability
• Interface Design
• Stress & Workload
• Complexity
• Automation

Task/System Factors

Feedback

SITUATION AWARENESS

State of Perception of Elements in Comprehension of Projection of Performance

Decision
Environment Current Situation Current Situation Future Status of actions

Level 1 Level 2 Level 3

Individual Factors

Information Processing
• Goals & Objectives Mechanisms
• Preconceptions
(Expectations) Long-term Memory Stores Automaticity

• Abilities
• Experience
• Training

Fig. 1. The three-level model of situation awareness [15].

3.1. Individual SA
Even though several attempts have been made to form a uni-
versally accepted definition of SA, it is still rather difficult to find
a definition that comprehensively captures every aspect of SA.
According to Stanton, Chambers and Piggott [14], three main defi-
nitions dominate: the three-level model [15], the perceptual cycle
model [16] (which is based on Niesser’s perceptual cycle [17]) and
the activity theory model [18]. Table 1 summarizes these SA def-
initions and main characteristics of these models, while Figs. 1–3
provide their graphical representation.
These SA models have fundamental differences, in terms of the
means for acquiring SA and the interactions between the individ-
ual and the environment. Endsley’s three-level model sets apart
the product of SA from the processes that are used to achieve it,
Actual environment
(potentially available
information)
Environmental
Information
in contrast with the Smith and Hancock’s perceptual cycle model
which does not distinguish between the SA product and the pro-
cesses. This characteristic of Endsley’s model makes it appropriate
for measuring SA as discussed in [19]. On the other hand, the
perceptual cycle and the activity theory models emphasize the
impact of the interactions between the individual and the world on
the development and maintenance of SA. These interactions make
these models more dynamic than Endsley’s three-level model,
which according to Uhlarik and Comerford [20] is both static and
finite. Nevertheless, we believe that the intuitive description of
SA provided by the three-level model offers the capability to eas-
ily understand and accurately measure SA and determine the SA
requirements at each level. These are probably the main reasons of
its popularity in the research community.
In addition to these three dominant SA definitions, several other
models have attracted less interest, such as the models of Taylor
[21], Sarter and Woods [22] and Hourizi and Johnson [23]. A com-
prehensive evaluation and comparison of these models, including
the models of Table 1, is provided in [24], which also discusses the
criteria used for such a comparison.

3.2. SA for team and collaborative operations

Modifies Samples

In complex and dynamic environments, such as aviation and

military operations, humans need to work as a team for achieving a
Modifies Samples
common goal. As a result, team SA has been receiving an increasing
amount of attention from the human factors community. To achieve
Schema of Perceptual
Present Directs
Exploration high performance, team SA requires an adequate level of individual
Cognitive Environment
map of the Locomotion SA and the effective sharing of the information relevant to each
world and its and action individual’s task. If any of the team members has poor SA, it can
possibilities
lead to a critical error that can undermine the success of the entire
team.
In general, team performance depends on two components,
Directs
teamwork (team members collaborating) and taskwork (individ-
ual tasks) [24]. Salas [25] defines a team as a group of two or more
Fig. 2. The perceptual cycle model of situation awareness [16]. people, working together to achieve a shared goal. There may be
 M. Panteli, D.S. Kirschen / Electric Power Systems Research 122 (2015) 140–151 143

Table 1
Dominant SA definitions and characteristics.

SA model SA definition Characteristics

Three-level (Endsley,
[15])
Perceptual cycle (Smith
and Hancock, [16])
Activity theory (Bedny
and Meister, [18])
Situational awareness is the perception of the elements in Three levels: perception, comprehension, projection
the environment within a volume of time and space, the Information processing theory
comprehension of their meaning and a projection of their
status in the near future
Situational awareness is the invariant in the Based on Niesser’s perceptual cycle [17]
agent-environment system that generates the momentary Infinite cycle of directed interaction and modification between
knowledge and behavior required to attain the goals individual and world, driven by internal schemata
specified by an arbiter of performance in the environment
Situational awareness is the conscious dynamic reflection Activity stages: orientational, executive and evaluative
on the situation by an individual. It provides dynamic Eight functional blocks with dedicated purposes, connected
orientation to the situation, the opportunity to reflect not through forward and feedback loops.
only the past, present and future, but the potential
features of the situation. The dynamic reflection contains
logical-conceptual, imaginative, conscious and
unconscious components which enables individuals to
develop mental models of external events

multiple information sources and the interaction between the team
members may be over time and space via advanced communication
technologies.
Several authors have proposed definitions of team SA and
descriptions of the mechanisms driving the development of a
shared understanding of a situation. Wellens [26] describes SA
during collaborative activities through a model of distributed
decision-making. He argues that the arrangement of the teams
must allow sufficient overlapping between the team members
to support collaborative tasks, but must also be physically and
spatially separate enough to allow individual SA acquisition. Fur-
thermore, Wellens defines team SA as “the sharing of a common
perspective between two or more individuals regarding current envi-
ronmental events, their meaning and projected future”. This definition
of team SA is in line with Endsley’s three-level model of individual
SA, which demonstrates that the simplicity of this model makes it
appropriate for a wide range of SA-related studies.
Most of the researchers’ attention has been focused on under-
standing team SA in the context of a shared understanding of the
same situation. For instance, Nofi [27] defines team SA as “a shared
awareness of a particular situation” and Perla et al. [28] suggest that
“shared SA implies that we all understand a given situation in the same
way”. Shu and Furuta [29] define team SA as “two or more individuals
share the common environment, up-to-the-moment understanding of
situation of the environment, and another person’s interaction with
the cooperative task”.
Endsley and Jones’ [30,31] approach is different in that they
make a distinction between team and shared SA. According to Ends-
ley [19], team SA is “the degree to which every team member possesses
the SA required for his or her responsibilities”, while shared SA is “the
degree to which team members have the same SA on shared SA require-
ments”. Fig. 4 demonstrates the construct of team and shared SA
[15] and Table 2 lists the shared SA requirements [19].
The concept of team and shared SA can be captured using the
illustrative example of Fig. 5. Here, there are four different and com-
pletely separated control and decision-making centers (elliptical
shapes).
A decision-making center may consist of a single operator (B
and C) or a group of two (A), three (D) or more operators. Commu-
nication and sharing of data and information takes place between

Fig. 3. Activity theory model of situation awareness [18].

144 M. Panteli, D.S. Kirschen / Electric Power Systems Research 122 (2015) 140–151

3.3. SA in the context of power systems

In the context of power systems, SA can be defined as the cog-

nition of those elements that enable an effective reaction by the
operators during routine procedures and electrical disturbances.
Power system operators must be able to determine at any time the
following:

- the current and future state of their responsibility area,

- the state of adjacent networks as it might affect their own respon-
sibility area,
- the actions that have to be implemented to restore or maintain
the system in a normal and secure state, and
- the maximum amount of time available to implement these
Fig. 4. Team SA vs. Shared SA [15]. actions.

Table 2 Based on the three-level SA model by Endsley, Sobajic [32]

Shared SA requirements [19].
Level 1 SA: perception System
Environment
Other team members
Level 2 SA: Status relevant to own goals/requirements
comprehension Status relevant to other’s goals/requirements
Impact of own actions on others
Impact of other’s actions on self and overall
goal
Level 3 SA: projection Actions of team members
different control centers, e.g. A–B, B–C and C–D, and within the
same control center, e.g. A-1 and D-1. Both are vitally important in
developing and maintaining a common understanding of a situa-
tion. For example, sufficient team SA is required in control centers
A and D for achieving their own individual goals, but this may have
to be distributed to the other control centers as it may affect their
decision-making and goal setting, and vice-versa. Therefore, in the
concept of team cognition, SA needs to be shared not only between
team members within a common environment, but also between
operators in different locations. Depending on the structure and
organization of the system, different technologies and means of
sharing SA can be used, such as emails, phone calls and shared
displays within a common environment (D).
discusses the OSA requirements in each level. Fig. 6 shows that
operators need to gradually receive the information related to their
goals, form a complete and accurate picture of their control area and
predict the future system state. Using the perceived information,
system operators need to detect any deviations between the actual
system state and the planned/expected state. These deviations can
involve equipment outages, abnormal frequency or voltage levels,
etc. Next, they need to remain aware of the status of equipment and
the system capabilities (e.g. available active and reactive reserves,
power transfer capabilities, power imports and exports from and
to neighboring networks, etc.). Then, they need to identify possi-
ble vulnerabilities and identify what actions they might be able to
take to prevent a worsening of the situation. The projection of the
future system state helps them determine the time available for
implementing these actions. System operators need to be able to
decide on the course of actions that minimize the probability of
customer disconnections, which have to be used as a last resort for
saving the system.
The picture of the system that operators manage to develop in
their minds is the output of the dynamic interactions that take place
between the operators and the environment in which they work in,
i.e. the control center, as described by the perceptual cycle and the
activity theory SA models. Therefore, the design of the user inter-
face and of the applications available to the system operators, as
internal schemata, play a critical role in understanding the rapidly
changing system conditions. Fig. 7 shows the applications that are
required for the reliable monitoring and control of a bulk electric
system and the maintenance of an adequate OSA [33].
The degree of difficulty in developing an adequate OSA also
depends on the size and complexity of the electrical network, which
affects the amount of information and alarms that operators need
to perceive and comprehend. For example, in an isolated network,
the operators only need to be aware of the state of their own con-
trol area. On the other hand, if their control area is interconnected
with others, they also need to be aware of aspects of the state of
adjacent networks that may affect the reliability of their control
area. A shared understanding of the situation between the TSOs is
then critically important for jointly dealing with emergencies, as
discussed in Section 3.2.

4. SA challenges in power system control centers

The electric power system control center is a complex environ-

Fig. 5. Distributed SA (elliptical shapes represent different control and decision-
making locations).
ment, with dynamic interactions between human operators and
between the human operators and the human machine interface
(HMI). This makes the development and maintenance of a sufficient
OSA level a highly challenging task. Endsley and Connors [34] dis-
cuss “SA demons”, such as workload and data overload, which are
 M. Panteli, D.S. Kirschen / Electric Power Systems Research 122 (2015) 140–151 145

Perception Comprehension Projection

• Generation data • Deviation • Future system
• Transmission between actual state
planned outages and planned • Time for actions
• Load forecast states implementation
• Transaction • Status of
schedules equipment
• Planned and • System
actual reserves capabilities and
• Switch/breaker vulnerabilities
status • Available actions
• Information on
neighboring
systems
• Identification of
limit violations

Fig. 6. Required information for sufficient SA [32].

software applications in preserving power system integrity. Loss of

any of these tools, such as the state estimator or the contingency
analysis, can result in the absence of early indications of the dete-
rioration of the system state, which may result in an ineffective or
absent response from the operators.

4.2. Real-time measurements

There are several ways of providing operators with data on key

Fig. 7. Reliability toolbox for real-time operation [33].
applicable to every domain. In [34,35], these authors (along with
L. Jones) discuss the particular OSA challenges that arise in power
transmission and distribution.
Fig. 8 summarizes the factors governing OSA in power system
control centers [2]. The impact of each factor varies depending on
the status of the electrical network when an event occurs and on the
severity of this event. For example, the impact of losing the alarm
processing system is much higher during an emergency than during
normal system conditions.
4.1. Hardware & software applications
The North America blackout of August 2003 [3] highlighted the
importance of the energy management system (EMS) hardware and
parameters of their power system, ranging from simple measur-
ing sensors to advanced systems, such as the phasor measurement
units (PMU) and wide-area measurement systems (WAMS). Miss-
ing or inconsistent information, due to failure in a measuring device
or in the communication system, can result in insufficient opera-
tors’ SA. In addition, the use of non-real-time data can provide a
false sense of security and prevent the timely implementation of
appropriate corrective or preventive control actions, as was the case
in the North America blackout of 2003 [3] and the UCTE disturbance
of 2006 [5] discussed in Section 2.
4.3. Environmental factors
During emergencies, system operators may have to cope with
a large amount of data and a huge number of alarms. An ineffec-
tive integration and presentation of the data and alarms can make
the graphical user interface (GUI) unintuitive, and hence make the
perception and comprehension of the information provided a very
challenging task. As a result, critical information may be neglected,
or the attention of the operators may be attracted by pieces of data
that may be less important or unrelated to the main problem. In
general, GUIs are effective in helping operators detect an abnormal

Fig. 8. SA challenges in power system control centers.

146 M. Panteli, D.S. Kirschen / Electric Power Systems Research 122 (2015) 140–151

situation, but less efficient at supporting them reach a decision in

a timely manner. Operators often have to scan several computer
monitors and many pages of SCADA data to reach a single decision
[35].

4.4. Automation

The increasing use of automation in power systems has many

benefits because it reduces the operators’ workload. However,
highly automated systems can leave the operators detached from
the system and possibly unaware of its actual state. This is defined
by Endsley [36] as “out-of-the-loop” syndrome. Operators may fail
to understand the system in sufficient depth to detect situations
where human intervention is needed, resulting in lack of reaction
to dangerous events. In addition, if the operators are not vigilant
enough, they may fail to detect automation failures or problems
for which manual actions are required.

4.5. Individual factors

Another source of poor performance is factors related to the

operators themselves, such as experience and training. For exam-
ple, the available information sources in a power system control
center may be insufficiently or incorrectly used by the operators
because of a lack of experience with a computer system or insuf-
ficient training on the latest technologies. The UCTE disturbance Fig. 9. Filters introduced in the decision-making process by the ICT and the opera-
tors’ SA.
of 2006 [5] illustrates the importance of the operators’ individual
factors, because the operators proceeded to actions based on their
experience with the system, rather than running an updated N-1
the system that matches closely the actual state. However, as dis-
analysis first.
cussed in Section 4, several factors govern their awareness, and
consequently their performance. The operators’ cognitive limita-
4.6. Data & information sharing
tions thus create a filter between the presented and the perceived
power system states. Based on the perceived state, the operators
The Italian blackout of 2003 [4] demonstrates the impact of
can predict the future behavior of the system, supported by tools
insufficient data and information exchange between system oper-
enabling “what-if” analyses. The perceived and predicted states are
ators, especially in highly interconnected networks. Insufficient
of crucial importance because they are the states that operators
communication between operators located in one or more control
actually use to decide upon the course of actions to implement and
rooms can result in a lack of awareness of key elements that may
the effect of these actions on their system.
affect the stability of more than one area. This leads to inadequate
Following the decision-making, the actions are implemented
shared SA between the system operators, which results in absent or
using the ICT infrastructure for transmitting the control signals
ineffective coordinated actions when required to tackle a problem.
to the system elements. If this infrastructure does not work as
designed, then these actions will not have the desired effect on
5. Filters in operational decision-making process
the system. Therefore, the ICT system does introduce an additional
filter in the feedback loop of the schematic representation of the
The analysis of the previous sections shows that instead of
operational decision-making process shown on Fig. 9.
assuming that the state of the system is perfectly presented by the
This feedback loop suggests an active learning process involving
information system and unambiguously perceived by the system
the system’s reaction to the actions implemented by the operators.
operators, it is more realistic to distinguish the three power system
It therefore reflects the effectiveness of the operators’ response to
states illustrated in Fig. 9 [2]: actual, presented and perceived.
the electrical disturbance, which in turn shows the level of their SA.
The actual state of a power system refers, as its name indicates,
to the true state of each electrical element in the field. Knowing
this state would require an exhaustive instrumentation of every 6. Supporting situation awareness in power systems
component and a perfectly reliable ICT infrastructure. Such an ICT
system is technically infeasible and economically unjustifiable. As highlighted in the previous sections, acquiring and maintain-
The presented state is thus the state as depicted imperfectly ing a sufficient SA level is a challenging task, especially in highly
and incompletely at any given time by the information infrastruc- complex and dynamic environments like power systems. Several
ture. Failures, limitations or design flaws in the information system research projects and private companies worldwide are working on
result in discrepancies between the actual and the presented power the development of tools and methods for supporting OSA in power
system states, affecting in turn OSA. In a sense, the ICT infrastruc- system control centers. The remainder of this section provides a
ture introduces a filter between these two states in the operational comprehensive overview of these efforts, focusing on tackling the
decision-making process. SA challenges discussed in Section 4 [37].
The perceived state refers to the state of the system as the oper-
ators comprehend it. It is basically the picture of the system that 6.1. Increasing the accuracy of state estimation
operators manage to form in their minds. Under familiar circum-
stances, well-trained, alert and experienced operators who have The aim of the state estimator (SE), which constitutes the core
access to complete and reliable data will have a perception of of the on-line security assessment function, is to provide a reliable
 M. Panteli, D.S. Kirschen / Electric Power Systems Research 122 (2015) 140–151
real-time snapshot of the state of the system. Its accuracy criti-
cally depends on the validity of the steady-state model and on the
reliability of the input signals, mainly the real-time analog mea-
surements and the output of the topology processor. It is thus vitally
important to detect and eliminate bad analog measurements and
erroneous breaker/switch status.
6.1.1. Bad data processing
Bad data processing involves two steps: detection and iden-
tification. Detection is the process of determining whether there
are any bad data in the measurement set [38–40]. Identification
is the procedure of determining which measurement is bad and
should be removed [38,41–47] in order to obtain an unbiased state
estimation.
6.1.2. Topology error processing
Errors in the bus/branch model provided by the topology pro-
cessor usually cause the SE to diverge or suffer serious convergence
problems. This can happen when:
(a) the status of a circuit breaker is not telemetered due to a com-
munications failure,
(b) the status of a circuit breaker is incorrectly telemetered, or
(c) the manual manipulation of a circuit breaker is not reported by
a maintenance team.
The techniques for topology error analysis can be applied either
before the state estimation, where the digital data can be validated
in advance by local consistency checks or rule-based techniques,
or after the state estimation by incorporating the possibility of
topology errors in the processing of bad data [38]. Rule-based
techniques are proposed and effectively used in existing EMS of
central Europe in [48] and [49]. According to these methods, if
the amount of compatible information is in favor of a given switch
position, then it is considered true; otherwise it is assumed false.
These techniques require data redundancy in order to carry out
the consistency checking. A two-stage SE is proposed in [50] and a
user-friendly software has been developed for the real-time illus-
tration of this method [51]. This two-stage SE first identifies the set
of data that contains the topology errors, and then it provides the
set of corrected breaker status and flows to the system operators.
Jakovljevic and Kezunovic [52] suggest the addition of advanced
data collection and processing at the substation level to the conven-
tional state estimation, in order to utilize all the available analog
and digital measurements in the substation. Several other advanced
techniques have been developed for addressing the issue of topol-
ogy errors, such as [53–57] based on generalized state estimation
and [58–61].
One of the latest technologies that enable the enhancement
of state estimation and power systems observability is phasor
measurement units (PMUs), which in combination with high-
bandwidth and high-speed communication networks, provide
time-synchronized measurements across the entire network. This
enables the dynamic and real-time detection of the system state,
improving the accuracy and speed of state estimation. In addi-
tion, a highly accurate and real-time estimation of the unfolding
system state may enable the design of smart and adaptive pro-
tection schemes, such as PMU-based system integrity protection
schemes (SIPS) [62,63], which would contribute to the blackout
prevention. However, PMUs are currently considered an expensive
solution, and as a result they are not installed at every bus of the
network. Therefore, different studies have been recently carried out
for determining the optimum PMU placement, such as [64–67].
147
Fig. 10. 3D representation of a power system [77].
6.2. Improving GUI effectiveness
Because the amount of data and the number of alarms that oper-
ators have to monitor has become very large, innovative solutions
are required for the visualization and data integration in control
centers. Even though several technologies have been developed
in the last decades to improve the GUI efficiency, there are some
crucial questions yet to be answered:
• What information do the operators really need among the large
amount of data that are available?
• How does this change depending on the situation?
• What is the best way to integrate and present information to max-
imize the operators’ performance and minimize the probability
of human errors?
• How should this information be organized and what should be
emphasized?
Alarm processing and filtering helps operators integrate more
efficiently the received information and cope with the excess infor-
mation. While the necessity of such functions has been recognized
since the 1960s [68], this problem has not yet been fully solved,
mainly due to the increased size, complexity and interconnectivity
of modern power systems. Intelligent Alarm Processing [69], [70]
provides a number of tools including alarm suppression, prioriti-
zation and reduction, which helps operators better comprehend a
situation and make accurate and faster decisions [71].
Advanced visualization methods are also currently used for
enhancing operators’ SA. In contrast to the traditional approach
of representing the system state using numerical fields on one-
line diagram using a static map board, different technologies are
currently used to make the system representation more dynamic
and improve the system observability. Such technologies include
color contours, animated arrows and dynamically sized pie charts
[72–76] for representing highly stressed areas of the system, high-
lighting thermal overloads, over- and under-voltages, etc. These
visualization techniques have been integrated into different soft-
ware packages [77], and have been effectively applied in various
control centers [73]. In addition, 3D visualization of power sys-
tem information can be used, to highlight lines loading and voltage
levels [78]. Fig. 10 provides an example of these visualization tech-
nologies. Even though these technologies provide numerous visual-
ization capabilities, their effectiveness depends on the training and
ability of the system operators to use them properly and effectively.
148 M. Panteli, D.S. Kirschen / Electric Power Systems Research 122 (2015) 140–151
6.3. Dealing with highly automated systems
The increasing use of automation in power systems results in a
change in the operators’ role from highly active to a more passive
involvement in system operation. Electrical utilities currently try
to minimize the impact of automation on operators’ alertness and
awareness by designing systems that support the human–system
interaction. Measures that can be applied to reduce the effect of
automation include [19]:
- Automate only if necessary: The functions to be automated should
be carefully chosen. The use of automation should be limited to
the functions that human operators cannot perform effectively in
a timely manner.
- Keep the operator in control and in the decision-making loop: Keep-
ing the system operators actively involved in system monitoring
and operation enables the development of a sufficient SA and
thus their effective reaction when required. This is achieved by
designing the control centers in a way that keeps the operators
constantly updated on the system state.
- Provide automation transparency: System operators should be
constantly aware of what the automation is doing and why it is
doing it. Information regarding the state of the automation and
the state of the parameters being controlled should be clearly
presented to the system operators.
6.4. How to reduce operators’ individual errors?
As evidenced by the discussion of blackouts in Section 2, it is
highly possible that operators proceed to incorrect actions, even
during routine procedures. It is therefore necessary that system
operators, even the most experienced ones, be frequently trained
to deal with various situations, ranging from routine procedures
during normal conditions to emergencies.
Because modern power systems are highly interconnected, the
training scenarios should cover not only electrical events within
the control area of the operators, but also events that are electri-
cally close to their responsibility area. This is because an event in
a neighboring system can trigger a sequence of events that require
the implementation of coordinated control actions. Therefore, the
individual operators should be adequately aware and trained on
the joint emergency procedures to be followed in the case of an
electrical disturbance affecting more than one control areas.
6.5. Wide-area monitoring and collaborative visualization
A 2008 survey conducted by the North American Electric
Reliability Corporation (NERC) [79] revealed the limitations in
telemetry and sharing of data from adjacent networks. Even though
this sharing is considered necessary to maintain wide-area security,
the vast majority of the respondents stated that they do not receive
telemetry data from neighboring Reliability Coordinators (RCs).
NERC defines wide-area as “the entire Reliability Coordinator
Area as well as the critical flow and status information from adja-
cent Reliability Coordinator (RC) Areas as determined by detailed
system studies to allow the calculation of Interconnected Reliabil-
ity Operating Limits” [80]. For its part, ENTSO-E uses the concepts
of responsibility and observability areas (Fig. 11) [81]:
• Responsibility area: The transmission system operator (TSO) is
responsible for the secure operation of its own grid and all the
interconnectors to adjacent TSOs. The equipment comprising this
network is called responsibility area.
• Observability area: The branches defined by the external observ-
ability list and their terminal buses may not constitute a fully
connected consistent external network. Thus they may need to
Observability
Area
TSO B TSO A
Responsibility
Area
Fig. 11. Responsibility and observability areas [81].
be complemented with additional network elements to obtain a
consistent observability area, which is usually implemented in
the SCADA system. The observability area includes the responsi-
bility area.
The lack of specific criteria and guidelines on the amount and
type of adjacent-area telemetry data to be shared between con-
trol centers, as well as the lack of a clear and universal definition
of what is wide-area, are critical obstacles in fulfilling the require-
ments of wide-area monitoring and collaborative visualization of
power systems.
The importance of this issue has been recently recognized,
and electrical utilities are working toward setting standards for
effective data and information sharing, which would focus on the
following:
(a) Define what constitutes wide-area view boundaries: Defining
which parts of the adjacent networks are needed to be mon-
itored would help determine the parts of the electrically close
networks that should be included in the wide-area security
tools for providing a picture of the state of adjacent networks.
An interesting study is presented in [82], where a methodol-
ogy is described for identifying and evaluating the influence
of external elements on the TSO’s responsibility area, which
helps determine the observability area. The proposed method
is applied on the Spanish and Portuguese networks.
(b) Establish a list of critical components to monitor closely: This
requires a what-if analysis using a comprehensive model of
the interconnected network in order to determine the impact
of these critical components on the reliability of a wide-area.
These components can be tie-lines, substation switchgear, gen-
erators or any component electrically close to other networks.
(c) Development of wide-area visualization and security analysis
tools: Following the identification of wide-area view boundaries
and the critical components of neighboring systems, tools that
can provide operators with the ability to be constantly aware of
what is going on in adjacent networks can be developed. This
would enable the effective collaboration and coordination of
RCs.
If such visualization tools had been in place during several
recent large-area disturbances, such as the Italian and Northeast
USA blackouts discussed in Section 2, then the cascading outages
could have been prevented, or at least their impact could have been
mitigated through the implementation of joint control actions.
However, there are several challenges in developing the exter-
nal network model required for wide-area monitoring. A review
of these challenges and the various external network modeling
methods is provided in [83], where each method is assessed based
on its accuracy, computation requirements and load-flow com-
patibility. A unified approach for external network modeling is
proposed based on this review. A method that combines the inter-
nal state estimation and external network modeling is presented
 M. Panteli, D.S. Kirschen / Electric Power Systems Research 122 (2015) 140–151
Determine SA Technology- and User-
Requirements Oriented Principles
Develop SA
Enhancement Plan
SA Assessment
Implement SA
Enhancement Plan
Fig. 12. Procedure for enhancing situation awareness.
in [84], where the external system is represented by an unreduced
load flow model. Kim and Abur [85] enhance the external sys-
tem modeling by detecting topology errors in the external system.
A partitioned SE method for real-time external network model-
ing to support contingency analysis is presented in [86], where
an unreduced external network model is utilized and the entire
system state is estimated using two individual SE solutions, one
for the internal system and one of the external system. In [87],
a multi-area state estimation method suitable for a decentralized
interconnected networks is discussed. It uses Ward-type equiva-
lents and takes advantage of the fast data acquisition rates that
are available from PMU measurements. An extensive discussion
on multi-area distributed SE, along with related challenges in bor-
der data and information exchanging among TSOs is provided in
[88–94].
6.6. Ensure the functionality of hardware and software
applications
The functionality of the hardware and software applications
used by system operators needs to be ensured under any con-
ditions. However, failures in the information systems still occur,
affecting the observability of the power network. In such cases, an
alarm should be raised to the operators to seek for alternatives to
continue monitoring and controlling the system effectively. Rec-
ommendations can be provided to the system operators to help
them use the available means to remain adequately aware of the
evolving situation. In addition, frequent testing and maintenance
would help identify and replace faulted components or compo-
nents with degraded reliability in order to improve the reliability
of the hardware and software applications.
7. Framework for achieving adequate situation awareness
Based on the discussions in the previous sections, this section
presents a generic framework for achieving sufficient OSA (Fig. 12).
The aim of this framework is to determine the procedure that needs
to be followed for designing an information system that is both
technology- and user-oriented, optimizing in this way the OSA.
First, the SA requirements need to be determined, which would
depend on the size, complexity and interconnectivity of the elec-
trical network and should support the development of individual
149
and shared SA, if needed, as discussed in Section 3. These require-
ments have to dynamically interact with the technology- and
user-oriented principles for optimizing the use of the available
technologies, on the one hand, and designing the user interface
in a user-friendly way, on the other hand. The aim of these inter-
actions should be the prevention of the sources of operator errors
discussed in Section 4. If the human factors and ergonomics are
considered during the interface design, then the information could
be integrated in a way that fits the capabilities and needs of the
user. This will result in higher user acceptance and satisfaction,
leading to higher productivity and efficiency and minimizing the
probability of inadequate OSA and consequently of operator errors.
After determining the SA requirements, the SA technologies and the
user-oriented principles to be followed, the SA enhancement plan is
developed. The technologies and SA-oriented approaches provided
in Section 6 can be applied here.
Before implementing this plan, the level of SA achieved needs to
be assessed. One way of doing this is by integrating the information
system and user interface under consideration into the operators’
training simulators, which will give the opportunity to the opera-
tors to experience the capabilities of their control center. Different
scenarios can also be included in the training, such as losing a soft-
ware application or data and alarm overload. As a result, system
operators would have to deal with the type of events that may occur
in their control center. An analysis of these simulations would then
be used to evaluate the operators’ performance and response time.
This will provide useful input on the limitations or deficiencies of
the SA applications and control center design and on the possible
sources of operator errors.
Another way of assessing SA would be to apply an SA measure-
ment technique. Salmon et al. [13] provide a comprehensive review
of SA measurement techniques that have been successfully applied
across a variety of domains, such as aviation (e.g. SA-SWORD [95])
and military (e.g. SAGAT [96], SARS [97] and SART [98]). Similarly,
SA measurement techniques could be applied in power systems
to evaluate the inter-company and inter-TSO OSA, which will help
target OSA enhancement methods. Since applications of these tech-
niques in power systems are currently limited (see [99] for an
example), it is difficult to provide any recommendations on which
are the most appropriate and effective techniques for the specific
needs of power systems.
Finally, using the input from SA assessment, the SA enhance-
ment plan can be implemented. The infinite loop in Fig. 12 indicates
the need to continuously upgrade the control center applications
and design in order to ensure an adequate level of OSA. As power
system evolve, the SA requirements are becoming higher and
higher, but new SA technologies are becoming available at the same
time. In order to fulfill these SA requirements and enable the effec-
tive monitoring of the system by the operators, new SA technologies
will have to be put in place. A balance between the technologies
added to the control center and the ability of the operators to handle
these technologies needs to be realized for achieving adequate OSA.
8. Conclusions
The state of the ICT infrastructure and situation awareness play
a key role in preserving the integrity of modern power systems.
Inadequate individual and/or shared OSA can result in the ineffec-
tive response of power system operators when needed, threatening
system reliability. It is therefore a necessity to develop tools and
interfaces that can effectively support the decision-making process
in the complex and dynamic environment of power system control
centers.
This paper provided the fundamentals of SA, including defini-
tions, sources of operator errors due to insufficient SA and their
impact on operational decision-making process. By determining
150 M. Panteli, D.S. Kirschen / Electric Power Systems Research 122 (2015) 140–151
the OSA challenges in a control room, targeted improvements can
be applied to enhance SA and measures can be taken to prevent sim-
ilar design errors in the future. When considering the state of the
information systems and the human operator as an independent
element of the system, a distinction between the actual, presented
and perceived power system states has to be made.
Several existing measures for supporting system operators have
also been discussed in this paper. However, as evidenced by black-
outs caused by operating errors in the last few decades, there is
still a lot of effort to be made to optimize the performance of sys-
tem operators. While it is probably impossible to avoid completely
operator errors, additional measures can be taken to reduce the
likelihood of such events. Within this context, the paper discusses
a generic framework for designing the information system in a way
that helps achieve sufficient OSA.
References
[1] V.P. Moshansky, Commission of Inquiry Into the Air Ontario Crash at Dryden,
Ontario: Final Report, 1992.
[2] M. Panteli, P.A. Crossley, D.S. Kirschen, D.J. Sobajic, Assessing the impact of
insufficient situation awareness on power system operation, IEEE Trans. Power
Syst. 28 (2013) 2967–2977.
[3] U.S.–Canada Power System Outage Task Force, Final Report on the August 14,
2003 Blackout in the United States and Canada: Causes and Recommendations,
U.S.–Canada Power System Outage Task Force, 2004.
[4] UCTE, Interim Report of the Investigation Committee on the 28 September 2003
Blackout in Italy, UCTE, 2003.
[5] UCTE, Final Report System Disturbance on 4 November 2006, UCTE, 2007.
[6] FERC, NERC, Arizona—Southern California Outages on September 8, 2011:
Causes and Recommendations, FERC, NERC, 2012.
[7] M.R. Endsley, Theoritical underpinnings of situation awareness: a critical
overview, in: M.R. Endsley, D.J. Garland (Eds.), Situation Awareness Analysis
and Measurement, New Jersey, Lawrence Erlbaum Associates, Inc., 2000, pp.
1–24.
[8] D.B. Kaber, M.R. Endsley, M.C.W.H. Wright, The Effects of Levels of Automation
on Performance, Situation Awareness, and Workload in an Advanced Commer-
cial Aircraft Flight Simulation, Final Report, NASA Langley Research Center,
2002.
[9] J. Keller, C. Lebiere, R. Shay, K. Latorella, Cockpit system situational awareness
modelling tool, in: Proc. of the fifth Human Performance, Situation Awareness
and Automation Conference, 2004.
[10] Y. Hauss, K. Eyferth, Securing future ATM-concepts’ safety by measuring situ-
ation awareness in ATC, Aerosp. Sci. Technol. 7 (2003) 417–427.
[11] X.S. Zheng, G.W. McConkie, Y. Tayi, Dynamic monitoring of traffic flow: the
Driver’s situation awareness, in: Proc. of the Fifth Human Performance, Situa-
tion Awareness and Automation Conference, 2004.
[12] G.H. Walker, N.A. Stanton, P. Salmon, D. Green, Measuring and predicting SA
in C4i: development and testing of a refined SA measurement technique, and
a new concept for SA prediction, in: Proc. of the Fifth Human Performance,
Situation Awareness and Automation Conference, 2004.
[13] P. Salmon, N. Stanton, G. Walker, D. Green, Situation awareness measurement:
a review of applicability for C4i environments, Appl. Ergon. 37 (2006) 225–238.
[14] N.A. Stanton, P.R.G. Chambers, J. Piggott, Situational awareness and safety, Saf.
Sci. 39 (2001) 189–204.
[15] M.R. Endsley, Towards a theory of situation awareness in dynamic systems,
Hum. Factors 37 (1995) 32–64.
[16] K. Smith, P.A. Hancock, Situation awareness is adaptive, externally directed
consciousness, Hum. Factors 37 (1995) 137–148 (The J. of the Hum. Factors
and Ergonomics Soc.).
[17] U. Niesser, Cognition Reality: Principles and Implications of Cognitive Psychol-
ogy, W.H. Freeman, San Francisco, CA, 1976.
[18] G. Bedny, D. Meister, Theory of activity and situation awareness, Int. J. Cogn.
Ergon. 3 (1999) 63–72.
[19] M.R. Endsley, B. Bolte, D.G. Jones, Designing for Situation Awareness: An
Approach to User-centered Design, Taylor and Francis Group, New York, NY,
2003.
[20] J. Uhlarik, D.A. Comerford, A Review of Situation Awareness Literature Relevant
to Pilot Surveillance Functions, U.S. Department of Transportation, Manhattan,
KS, 2002.
[21] R.M. Taylor, Situational awareness rating technique SART: the development
of a tool for aircrew systems design, situational aware, in: Aerosp. Operations
(AGARD-CP-478), 1990.
[22] N.B. Sarter, D.D. Woods, Situation Awareness, A critical but Ill-defined phe-
nomenon, Int. J. Aviat. Psychol. 1 (1991) 45–57.
[23] R. Hourizi, P. Johnson, Towards an explanatory, predictive account of aware-
ness, Comput. Graphics 27 (2003) 859–872.
[24] P.M. Salmon, N.A. Stanton, G.H. Walker, C. Baber, D.P. Jenkins, R. McMaster,
Mark S. Young, What really is going on? Review of situation awareness models
for individuals and teams, Theor. Issues Ergon. Sci. 9 (2008) 297–323.
[25] E. Salas, Handbook of Human Factors Methods, CRC Press, Boca Raton, FL, 2005.
[26] A.R. Wellens, Group situation awareness and distributed decision making: from
military to civilian applications, in: N.J. Castellan (Ed.), Individual and Group
Decision Making: Current Issues, New Jersey, Lawrence Erlbaum Associates,
Inc., 1993, pp. 267–287.
[27] A. Nofi, Defining and Measuring Shared Situational Awareness, DARPA, Alexan-
dria, Virginia, 2000.
[28] P. Perla, M. Markowitz, A. Nofi, C. Weuve, J. Loughran, Gaming and Shared
Situation Awareness, DARPA, Alexandria, Virginia, 2000.
[29] Y. Shu, K. Furuta, An inference method of team situation awareness based on
mutual awareness, Cogn. Technol. Work 7 (2005) 272–287.
[30] M.R. Endsley, Final Report, Situation Awareness in an Advanced Strategic Mis-
sion, Northrop Corporation, Hawthorne, CA, 1989.
[31] M.R. Endsley, W.M. Jones, A model of inter- and intra-team situation awareness:
implications for design, training and measurement, in: M. McNeese, E. Salas,
M.R. Endsley (Eds.), New Trends in Cooperative Activities: Understanding Sys-
tem Dynamics in Complex Environments, Santa Monica, CA, Human Factors
and Ergonomics Society, 2001, pp. 46–67.
[32] D.J. Sobajic, About situation awareness, in: Proc. of the 11th Workshop on
Electrical Power Control Centers: Experiences and Trends in Generation, Trans-
mission and Distribution Control Centers, 2011.
[33] NERC, Real-Time Tools Survey Analysis and Recommendations, NERC, Berkeley,
California, 2008.
[34] M.R. Endsley, E.S. Connors, Situation awareness: state of the art, in: Proc. of
the Power and Energy Society General Meeting—Conversion and Delivery of
Electrical Energy in the 21st Century, 2008.
[35] E.S. Connors, M.R. Endsley, L. Jones, Situation awareness in the power transmis-
sion and distribution industry, in: Proc. of the Human Factors and Ergonomics
Society 31st Annual Meeting, 2007.
[36] M.R. Endsley, Automation and situation awareness, in: R. Parasuraman, M.
Mouloua (Eds.), Automation and Human Performance: Theory and Applica-
tions, Lawrence Erlbaum Associates, Inc., New Jersey, 1996, pp. 163–181.
[37] M. Panteli, D.S. Kirschen, P.A. Crossley, D.J. Sobajic, Enhancing Situation Aware-
ness in Power System Control Centers, in: Proc. of the IEEE International
Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness
and Decision Support (CogSIMA), 2013.
[38] A. Abur, A.G. Exposito, Power System State Estimation: Theory and Implemen-
tation, Marcel Dekker, Inc., New York, NY, 2004.
[39] K.L. Lo, P.S. Ong, R.D. McColl, A.M. Moffatt, J.L. Sulley, Development of a static
state estimator Part I: Estimation and bad data suppression, IEEE Trans. Power
Appar. Syst. PAS 102 (1983) 2486–2491.
[40] A. Monticelli, A. Garcia, Reliable bad data processing for real-time state estima-
tion, IEEE Trans. Power Appar. Syst. PAS 102 (1983) 1126–1139.
[41] A. Monticelli, State Estimation in Electric Power Systems—A Generalized
Approach, Kluwer Academic Publishers, Boston, MA, 1999.
[42] G.N. Korres, G.C. Contaxis, A reduced model for bad data processing in state
estimation, IEEE Trans. Power Syst. 6 (1991) 550–557.
[43] A. Monticelli, F.F. Wu, M. Yen, Mutiple bad data identification for state esti-
mation by combinatorial oftimization, IEEE Trans. Power Delivery 1 (1986)
361–369.
[44] B.M. Zhang, K.L. Lo, A recursive measurement error estimation identification
method for bad data analysis in power system state estimation, IEEE Trans.
Power Syst. 6 (1991) 191–198.
[45] B.M. Zhang, S.Y. Wang, N.D. Xiang, A linear recursive bad data identification
method with real-time application to power system state estimation, IEEE
Trans. Power Syst. 7 (1992) 1378–1385.
[46] T. Van Cutsem, M. Ribbens-Pavella, L. Mili, Hypothesis testing identification: a
new method for bad data analysis in power system state estimation, IEEE Trans.
Power Appar. Syst. PAS 103 (1984) 3239–3252.
[47] F. Zhuang, R. Balasubramanian, Bad data processing in power system state esti-
mation by direct data deletion and hypothesis tests, IEEE Trans. Power Syst. 2
(1987) 321–327.
[48] N. Singh, H. Glavitsch, Detection and identification of topological errors
in online power system analysis, IEEE Trans. Power Syst. 6 (1991) 324–
331.
[49] N. Singh, F. Oesch, Practical experience with rule-based on-line topology error
detection, IEEE Trans. Power Syst. 9 (1994) 841–847.
[50] A. Abur, M. Kezunovic, S. Meliopoulos, Enhanced state estimation by
advanced substation monitoring, in: Power Systems Engineering Research
Center (PSERC), Final Project Report, PSERC publication, 2002, 02–48,
November.
[51] A. Abur, Power Education Toolbox (P.E.T) User Manual, Texas A&M University,
Available from: http://www3.ece.neu.edu/∼abur/pet.html.
[52] S. Jakovljevic, M. Kezunovic, Advanced substation data collecting and
processing for state estimation enhancement, in: Proc. of the IEEE Power Engi-
neering Society Summer Meeting, 2002.
[53] O. Alsac, N. Vempati, B. Stott, A. Monticelli, Generalized state estimation, IEEE
Trans. Power Syst. 13 (1998) 1069–1075.
[54] G.N. Korres, P.J. Katsikas, Identification of circuit breaker statuses in WLS state
estimator, IEEE Trans. Power Syst. 17 (2002) 818–825.
[55] G.N. Korres, P.J. Katsikas, G.E. Chatzarakis, Substation topology identification
in generalized state estimation, Int. J. Electr. Power Energy Syst. 28 (2006)
195–206.
[56] E.M. Lourenco, A.S. Costa, K.A. Clements, Bayesian-based hypothesis testing for
topology error identification in generalized state estimation, IEEE Trans. Power
Syst. 19 (2004) 1206–1215.
 M. Panteli, D.S. Kirschen / Electric Power Systems Research 122 (2015) 140–151
[57] E.M. Lourenco, A.S.A. Costa, K.A. Clements, R.A. Cernev, A topology error iden-
tification method directly based on collinearity tests, IEEE Trans. Power Syst.
21 (2006) 1920–1929.
[58] I.S. Costa, J.A. Leao, Identification of topology errors in power system state
estimation, IEEE Trans. Power Syst. 8 (1993) 1531–1538.
[59] L. Mili, G. Steeno, F. Dobraca, D. French, A robust estimation method for topology
error identification, IEEE Trans. Power Syst. 14 (1999) 1469–1476.
[60] K.A. Clements, A.S. Costa, Topology error identification using normalized
lagrange multipliers, IEEE Trans. Power Syst. 13 (1998) 347–353.
[61] C. Jian, A. Abur, Enhanced topology error processing via optimal measurement
design, IEEE Trans. Power Syst. 23 (2008) 845–852.
[62] V. Madani, D. Novosel, R. King, Technological breakthroughs in system integrity
protection schemes, in: Proc. of the 16th Power Systems Computation Confer-
ence (PSCC), 2008.
[63] V. Madani, D. Novosel, S. Horowitz, M. Adamiak, J. Amantegui, D. Karlsson,
et al., IEEE PSRC report on global industry experiences with system integrity
protection schemes (SIPS), IEEE Trans. Power Delivery 25 (2010) 2143–2155.
[64] R. Emami, A. Abur, Robust measurement design by placing synchronized phasor
measurements on network branches, IEEE Trans. Power Syst. 25 (2010) 38–43.
[65] N.M. Manousakis, G.N. Korres, A weighted least squares algorithm for optimal
PMU placement, IEEE Trans. Power Syst. 28 (2013) 3499–3500.
[66] S. Chakrabarti, E. Kyriakides, Optimal placement of phasor measurement units
for power system observability, IEEE Trans. Power Syst. 23 (2008) 1433–1440.
[67] F. Aminifar, A. Khodaei, M. Fotuhi-Firuzabad, M. Shahidehpour, Contingency-
constrained PMU placement in power networks, IEEE Trans. Power Syst. 25
(2010) 516–523.
[68] Human factors review of electric power dispatch control centers, in: EL-1960,
EPRI, Palo Alto, California, 1960.
[69] R.W. Bijoch, S.H. Harris, T.L. Volkmann, J.J. Bann, B.F. Wollenberg, Development
and implementation of the NSP intelligent alarm processor [for power systems],
IEEE Trans. Power Syst. 6 (1991) 806–812.
[70] A.H. Shoop, S. Silverman, A real-time alarm processor, Int. J. Electr. Power
Energy Syst. 14 (1992) 108–113.
[71] E. Kyriakides, J.W. Stahlhut, G.T. Heydt, A next generation alarm processing
algorithm incorporating recommendations and decisions on wide area control,
in: Proc. of the IEEE Power Engineering Society General Meeting, 2007.
[72] T.J. Overbye, D.A. Wiegmann, A.M. Rich, Y. Sun, Human factors aspects of power
system voltage contour visualizations, in: Proc. of the IEEE Power Engineering
Society General Meeting, 2003.
[73] R.P. Klump, J.D. Weber, Real-time data retrieval and new visualization tech-
niques for the energy industry, in: Proc. of the 35th Annual Hawaii International
Conference on System Sciences (HICSS), 2002.
[74] T.J. Overbye, J.D. Weber, Visualization of power system data, in: Proc. of the 33rd
Annual Hawaii International Conference on System Sciences (HICSS), 2000.
[75] T.J. Overbye, J.D. Weber, Visualizing the electric grid, IEEE Spectr. 38 (2001)
52–58.
[76] J.D. Weber, T.J. Overbye, Voltage contours for power system visualization, IEEE
Trans. Power Syst. 15 (2000) 404–409.
[77] PowerWorld Corporation & Simulator, Available from: (http://www.
powerworld.com/).
[78] T. Overbye, R. Klump, J. Weber, Interactive 3D visualization of power system
information, Electr. Power Compon. Syst. 31 (2003) 1205–1215.
151
[79] NERC, Real-Time Tools Survey Analysis and Recommendations, Real-Time Tools
Test Practices Task Force, NERC, Berkeley, California, 2008.
[80] NERC, Glossary of Terms Used in Reliability Standards, NERC, Washington, DC,
2008.
[81] ENTSO-E, Operational Handbook, ENTSO-E, Brussels, 2004.
[82] P.I. Domingues dos Santos e Abreu, C. Machado Ferreira, R. Pestana, F.P.M.
Barbosa, Identification and influence of the external elements on the trans-
mission system operator’s responsibility area, in: Proc. of the IEEE PowerTech
Conference, 2013.
[83] F.F. Wu, A. Monticelli, Critical review of external network modelling for online
security analysis, Int. J. Electr. Power Energy Syst. 5 (1983) 222–235.
[84] A. Monticelli, F.F. Wu, A method that combines internal state estimation and
external network modeling, IEEE Trans. Power Appar. Syst. PAS 104 (1985)
91–103.
[85] H. Kim, A. Abur, Enhancement of external system modeling for state estimation
[power systems], IEEE Trans. Power Syst. 11 (1996) 1380–1386.
[86] G.N. Korres, A partitioned state estimator for external network modeling, IEEE
Trans. Power Syst. 17 (2002) 834–842.
[87] E.W.S. Angelos, E.N. Asada, Multi-area state estimation with PMU and exter-
nal system modeling, in: Proc. of the IEEE Power and Energy Society General
Meeting, 2012.
[88] W. Jiang, V. Vittal, G.T. Heydt, Diakoptic state estimation using phasor mea-
surement units, IEEE Trans. Power Syst. 23 (2008) 1580–1589.
[89] G.N. Korres, A distributed multiarea state estimation, IEEE Trans. Power Syst.
26 (2011) 73–84.
[90] A. Gomez-Exposito, A. Abur, A. de la Villa Jaen, C. Gomez-Quiles, A mul-
tilevel state estimation paradigm for smart grids, IEEE Proc. 99 (2011)
952–976.
[91] L. Xie, D.-H. Choi, S. Kar, H.V. Poor, Fully distributed state estimation for wide-
area monitoring systems, IEEE Trans. Smart Grid 3 (2012) 1154–1169.
[92] S. Jin, Y. Chen, M. Rice, Y. Liu, I. Gorton, A testbed for deploying distributed
state estimation in power grid, in: Proc. of the IEEE Power and Energy Society
General Meeting, 2012.
[93] G.N. Korres, A. Tzavellas, E. Galinas, A distributed implementation of multi-area
power system state estimation on a cluster of computers, Electr. Power Syst.
Res. 102 (2013) 20–32.
[94] V. Kekatos, G.B. Giannakis, Distributed robust power system state estimation,
IEEE Trans. Power Syst. 28 (2013) 1617–1626.
[95] M.A. Vidulich, E.R. Hughes, Testing a subjective metric of situation awareness,
in: Proc. of the Human Factors and Ergonomics Society Annual Meeting, 1991.
[96] M.R. Endsley, Measurement of situation awareness in dynamic systems, Hum.
Factors 37 (1995) 65–84 (The J. of the Hum. Factors and Ergonomics Soc.).
[97] W.L. Waag, M.R. Houck, Tools for assessing situational awareness in an
operational fighter environment, Aviat. Space Environ. Med. 5 (1994)
13–19.
[98] R.M. Taylor, Situational awareness rating technique (SART): the development
of a tool for aircrew systems design, in: Proc. of the AGARD AMP Symposium
on Situational Awareness in Aerospace Operations, 1989.
[99] M.M. Lenox, E.S. Connors, M.R. Endsley, A baseline evaluation of situation
awareness for electric power system operation supervisors, in: Proc. of the
Human Factors and Ergonomics Society Annual Meeting, 2011.