Nuclear Engineering and Design 322 (2017) 266–276

Contents lists available at ScienceDirect

Nuclear Engineering and Design

journal homepage: www.elsevier.com/locate/nucengdes

Effect of user interface layout on the operators’ mental workload

in emergency operating procedures in nuclear power plants
Shengyuan Yan a, Cong Chi Tran a,⇑, Yu Chen b, Ke Tan c, Jean Luc Habiyaremye a
a
College of Mechanical and Electrical Engineering, Harbin Engineering University, 150001 Harbin, Heilongjiang, China
b
College of Mechanical Engineering, Heilongjiang University of Science & Technology, 150022 Harbin, Heilongjiang, China
c
China Nuclear Power Engineering Co Ltd, 518000 Shenzhen, Guangdong, China

h i g h l i g h t s

 Different interface design has a significant effect on the operators’ mental workload.
 Improving interface design based on human factor plays a significant role in reducing operator’s mental workload.
 NASA Task Load Index tool and eye movement technique are useful for assessing human – computer interface designs.

a r t i c l e i n f o a b s t r a c t

Article history: The improved control interface designs in nuclear power plants (NPPs) have been constantly developed in
Received 30 March 2017 order to satisfy actual requirements. In this article, the effect of user interface layout on mental workload
Received in revised form 10 July 2017 (MWL) of operators in emergency operating procedures in NPPs is analyzed. The control circuit interface
Accepted 11 July 2017
of the Chemical and Volume Control system (RCV) was redesigned based on some human factor criterions
and it was compared to the original interface during the procedure of system boots. The multi-index eval-
uation method based on performance measures (i.e., time to complete and error rate), subjective rating
Keywords:
(i.e., NASA Task Load Index (NASA-TLX)) and physiological measure (i.e., eye movement) was used to
User interface
Mental workload
evaluate MWL of operators. Twenty-two engineering students tested both interfaces in randomized
Nuclear power plants order. The results showed that interface design has a significant effect on operators’ MWL. Specifically,
Eye-tracking the redesigned interface was rated significantly lower in overall workload, time to complete, and fixation
NASA-TLX indices, but higher blink rate. The results suggested that human factor criterions have a significant effect
on operators’ MWL so they should be considered during designing of interface, and using of NASA-TLX
tool and eye movement technique is useful for assessing of human-computer interface (HCI) designs
in NPPs.
Ó 2017 Elsevier B.V. All rights reserved.

1. Introduction
The interface design of control room is constantly improved in
order to make it more consistent with actual requirements because
poor interface design can cause user stress, lower work rates,
decreased job satisfaction and even cause absenteeism (Booth,
1989). In nuclear power plants (NPPs), user control interface is
often complex and has many emergency operating procedures
(EOPs) needed to process a large amount of information in a short
⇑ Corresponding author at: College of Mechanical and Electrical Engineering,
Harbin Engineering University, NO. 145-1, Nantong Street, Nangang District, Harbin
150001, China.
E-mail addresses: yanshengyuan@hrbeu.edu.cn (S. Yan), trancongchi_bk@yahoo.
com (C.C. Tran), chenyubeautiful@163.com (Y. Chen), tanke@cgnpc.com.cn (K. Tan),
habijealuc@yahoo.fr (J.L. Habiyaremye).
time and to respond quickly with the decision, which easily leads
to generation of very high mental workload (MWL). Poor user
interfaces have effect on operators’ MWL, which further affects
efficiency and reliability of the entire system. Statistics show that
human error is a causal factor in significant number of industrial
accidents. Although, human error is rarely the sole cause of acci-
dents in NPPs, human error is major contributor of about 70–90%
of accidents (Isaac et al., 2002). Hitherto, the number of occurred
nuclear incidents is relatively small, but if serious accident hap-
pens it will be a big disaster for people and environment. There-
fore, operators have to be attentive all the time and good
interface design will enable operators to accomplish their tasks
efficiently and effectively with minimal errors (Ikuma et al., 2014).
Presently, there are no structured methods for all human-
computer interface (HCI) design. Reality shows that it is easy to find

http://dx.doi.org/10.1016/j.nucengdes.2017.07.012
0029-5493/Ó 2017 Elsevier B.V. All rights reserved.
 S. Yan et al. / Nuclear Engineering and Design 322 (2017) 266–276
interface design defects when we have a physical interface to anal-
ysis and evaluation. It is also important to separate design of the
interface from other components in the system, so defects in the
interface do not propagate faults through the system, yet this
method sometimes is complicated and expensive. Burns (1991) pro-
posed architecture to separate HCI from hard real-time systems,
which reduces complexity and timing constraints. Currently, one
of the best available methods is iterative design. This methodology
is based on a cyclic process of prototyping, testing, analyzing, evalu-
ating, and redesign. Based on these results many iterations of design,
changes and refinements have been made recently. Namely, if we
could perform efficient evaluations and correctly identify as many
defects as possible, the interface would be significantly improved.
One of the commonly used methods for interface evaluation is
the evaluation based on operators’ MWL. This technique has become
an important measure for evaluation of HCI in work environment,
and the number of studies on MWL measurement has been increas-
ing since 1960s (Seker, 2014). Moray (1988) demonstrated that opti-
mization of MWL allocation could reduce human errors, improve
system safety, and increase users’ satisfaction. Many of studies focus
on optimization of MWL by creating the appropriate designs and
interfaces (Hertzum and Holmegaard, 2013). There are many studies
which show that a well-designed interface can provide a suitable
operator MWL (Parasuraman et al., 2000). Niemelä and Saarinen
(2000) studied the effect of icons and spatial grouping on scanning
speed in computer interface and results showed that design of
graphics-based interfaces, such as effective arrangement of icons
in interface, had a significant positive effect on scanning speed,
which means there is a change in user’s MWL. In addition, there
are also studies related to spatial menu layout and color contrast
of user interface and their impact on visual search performance,
and their conclusion showed similar result (Michalski and
Grobelny, 2008; Van Schaik and Ling, 2001).
Interface evaluation is not only understanding of task, its envi-
ronment and the way the user interacts with the system, but also
an appreciation of capabilities and cognitive demands of future
users. However, the evaluation of industrial control interface based
on MWL still has some limitations, because the focus is mainly on
impact of workload level, and there is a lack of studies on impact of
interface design. Especially in NPPs, many information sources
should be monitored, but the operators have limited capacity of
attention and memory (Ha and Seong, 2009). Since it is impossible
to monitor all information sources, the operators continuously
decide where to allocate their attention in order to select impor-
tant things and ignore irrelevant things. Particularly in emergency
situations, the provision of a set of well-designed procedures is
necessary to reduce MWL of operators and to compensate degrada-
tion of operations (Jimmieson and Terry, 1998; Wood et al., 1990).
Therefore, user interfaces should be designed such that can achieve
suitable operator MWL. Thus, this study focuses on effect of user
interface layout on operators’ MWL in emergency operating proce-
dures. The interface of control circuit in Chemical and Volume Con-
trol system (RCV) was redesigned after its limitations were
analyzed based on cognitive psychology principles in layout design
and then it was compared to original interface during operating the
procedure of system boots. It was concluded that interface layout
has a significant effect on MWL because redesigned interface had
lower operators’ MWL than original interface.
2. Methodology
2.1. Participants
This study considers the effect of interface layout on operators’
MWL during searching for information in NPP user interface.
267
Presently, operator performance in this domain necessitates mas-
tering of all knowledge and capabilities accompanied by a great
deal of concentrated training and repetition to attain such mastery.
However, access to these experts is not readily available, hence this
study presents experiment with participants who were trained for
the task they were asked to do. Based on the results of these par-
ticipants, we expect similar patterns exist for real operators, but
further study is needed to confirm it.
Twenty-two man engineering students who were 22 ± 1.6
(mean ± SD) years old voluntarily participated in our experiment.
All participants were familiar with the computer operations, but
none of them was familiar with the operation of NPPs. They were
also required to have good vision ability and to be right-handed.
The experimental process was overseen by professors and experts
who are specialized in human factors and ergonomics. All partici-
pants completed and signed an informed consent form approved
by the university institutional review board and were compen-
sated with extra credit in extracurricular activities in their course.
2.2. Mental workload measures
The MWL measurement methods can be divided into three
main categories: performance measurement methods, physiologi-
cal measurement methods and subjective rating methods (Tsang
and Vidulich, 2006). Performance measurement methods evaluate
synchronized data collected during primary tasks and secondary
tasks in designed modules. Physiological measurement methods
are based on indirect measurement of observed physiological
changes, such as heartbeat, nasal tip temperature, eye movement,
body and face temperature, and so on, while subjective measures
can be measured by asking users about their experience of
demands imposed by current or recently completed task.
Although there are many different ways to measure human
MWL, there is no universal method because each method has its
disadvantage. For instance, psychophysiological signals are mea-
sured using specialized equipment and that equipment could be
costly because of capabilities of purchased system. Besides, when
in experiments some apparatuses are attached to user body that
constrains user movements and breaks naturalness of interaction.
Physiological methods also can be influenced by physical and emo-
tional factors. In subjective rating methods, rating scale results can
be affected by characteristics of respondents, like biases, response
sets, errors and protest attitudes (Dyer et al., 1976). Hence, use of
workload rating scales is not recommended, when it is likely that
raters would fake results, have low motivation and prejudices,
guess randomly, etc. However, compared with performance mea-
surement and physiological measurement methods, subjective
measurement methods are more efficient regarding the application
requirements and they are frequently used because of their large
MWL measurement range, lower cost and less required effort. With
the aim to reduce disadvantages of MWL measurement method
and to improve the reliability of assessment results, this study
applies the multi-index evaluation method, including performance
measurement, physiological measurement and subjective ratings,
to evaluate operators’ MWL.
Performance measures can be classified into many categories
such as accuracy, task time, worst-case performance, and so on
(Gawron, 2008). In this study, time to complete (or speed of task)
and error rate are used as performance measures because of fol-
lowing reasons: (1) Operating time is an important criterion of
operating emergency procedures; in order to prevent incidents
during operation of these procedures, operators need to process
information and to respond with the correct decision quickly. (2)
Operating errors involve risky behaviors that need to be under-
stood in order to prevent incidents, because that directly relates
to the effectiveness and safety of operating of NPPs system.
268 S. Yan et al. / Nuclear Engineering and Design 322 (2017) 266–276
Additionally, many studies have shown that error rate is sensitive
to differences in visual environment (Baker et al., 1960; Downing
and Sanders, 1987; Hwang et al., 2008); and (3) time to complete
and error rate are easy to collect using video playback feature. In
this study, time to complete (given in seconds) is defined as the
time that participant spend on each session of experiment, and
error rate (%) is defined as the ratio of the number of incorrect
operations and the number of all operations.
NASA-TLX method(Hart and Staveland, 1988) is used to evalu-
ate operators’ subjective workload in this study, because it is one
of the most widely used methods and it has provided solid results
in human factors research (Rubio et al., 2004). Namely, this tool
has been successfully used to measure MWL of HCI in various
domains including automobile (Götze et al., 2015), aerospace (De
la Torre et al., 2016; Lehrer et al., 2010; Wei et al., 2014) and NPPs
(Gao et al., 2013, 2015; Jou et al., 2009; Naderpour et al., 2016).
This method is a multi-dimensional rating scale using six dimen-
sions of workload to provide diagnostic information about the nat-
ure and relative contribution of each dimension in influencing
overall operator workload. Six dimensions to assess MWL includ-
ing mental demand (MD), physical demand (PD), temporal demand
(TD), own performance (OP), effort (EF) and frustration (FR). Each
of these is on a scale from 0 to 100. Each subscale has been pre-
sented as a 12 cm line with a title and bipolar descriptors at each
end. Numerical values have not been displayed, but values are
ranging from 1 to 100 have been assigned to scale positions during
data analysis. The process of MWL evaluation through NASA-TLX
involves three steps. At the first step, weight in each of the six sub-
scales is determined to reveal the priority of the six subscales. All
subscales are self-evaluated and selected by the operator in a
paired form and 15 different comparisons, and then, each workload
dimension is scored 0–5. At the second step, to allocate the rating
of workload, each of the six subscales is rated with the goal of
determination of each scale’s effect on the MWL. At this step, the
operator scores each of these six subscales from 1 to 100 based
on his/her working condition. At the third and the last step, after
determination of weight and rating in previous steps, total MW
is calculated in the range 1–100 through the following formula
‘‘Weighted Workload (WWL) is R (rating  weight)/15.”
Physiological measurement based on eye movement is a tech-
nique that captures eye behavior as a response to visual stimulus,
and this technique is widely used to analyze user behavior in mar-
keting, neuroscience, human-computer interaction, visualization
research and user interface evaluation (Ikuma et al., 2014; Toker
et al., 2013). Recordings of user eye movements during interaction
(i.e., doing a task) with user interface can show where on the inter-
face user looks and for how long. The particular pattern of eye
movements can signal whether the user interface elicits usage
problems or facilitates efficient and effective task realization,
which can help designers to evaluate changes in operator workload
during design and evaluation of complex systems (Ahlstrom and
Friedman-Berg, 2006).
Eye response parameters that are used as MWL measures
include pupil dilation, blink rate and fixation. Human pupil dilation
can be used as a measure of psychological load because it is related
to the amount of cognitive control, attention, and cognitive pro-
cessing required for a given task (Moresi et al., 2008; Wierda
et al., 2012). In addition, it was previously shown that it correlates
with the cognitive workload, whereby increased frequency of dila-
tion is associated with increased degree of difficulty of a task
(Hampson et al., 2010). Gao et al. (2013) also showed that in NPPs
operation the pupil size in a high complexity EOP is larger than in a
low complexity EOP.
Blink rate (‘frequency’) is defined as a number of blinks per sec-
ond or minute. Some researchers in ophthalmology prefer to use
the inverse blink rate, called the ‘inter-blink interval’, which is
defined as a time interval between blinks and it is given in seconds
(Montés-Micó et al., 2005). Human pupil dilation can measure the
average arousal underlying the cognitive tasks, and blink rate can
produce an opposite effect. Whereas MWL of cognitive tasks will
increase blink rate, visual demand will inhibit it (Recarte et al.,
2008). Thus, many studies on relationship between average blink
rate and workload yield conflicting results. Nonetheless, some
researchers found that a decrease in blink rate is associated with
an increase in task demands, particularly visual demands of tasks
(Gao et al., 2013; Wilson, 2002), and other researchers failed to find
such a significant relationship (Casali and Wierwille, 1983).
Eye fixations are detected and their properties calculated by an
event-detection algorithm. Fixation duration is sometimes called
‘fixation time’, but also ‘dwell time’, or ‘dwell time of the fixation’
(Holmqvist et al., 2011). This index is extensively used measure
and it is believed that it increases with increasing of mental task
demands (Recarte and Nunes, 2000; Tole et al., 1982). Fixation rate
is the number of fixations divided by certain time period, such as
duration of a trial in seconds, giving the unit persecond. It is found
that it is negatively correlated to task difficulty, and it can be used
as a measure of MWL (Van Orden et al., 2001).Recently,fixation
indices have been investigated in a series of studies on MWL
(Ahlstrom and Friedman-Berg, 2006; Di Stasi et al., 2013), assess
product design (Guo et al., 2016; Ho and Lu, 2014), assessing user
interface (Mat Zain et al., 2011; Scott et al., 2002; Sharma et al.,
2016), and these indices often show high reliability. In this study,
the area of interest (AOI) is defined as a simulation screen. Total
fixation duration (duration of all fixations within AOI) and fixation
rate were collected within the fixation lengths that varied from
100 ms to 600 ms.
2.3. Equipment
2.3.1. Eye tracking system
The participants’ eye movement data was recorded by the SMI
HED of SensoMotoric Instruments (SMI). Software configuration
has the video recording and the BeGaze 3.0 eye movement data
analysis, sampling rate 50/60 Hz (optional 200 Hz), tracking reso-
lution, pupil/CR < 0.1° (typ.), gaze position accuracy < 0.5°–1.0°
(typ.), tracking range ±30° horizontal and ±25° vertical. The exper-
imental interface presented on the 17 inches LCD monitor, dis-
played resolution of 1024  768 pixels, 60 Hz of refresh rate.
Calibration was done before each using session.
2.3.2. RCV interface
The RCV system is essential for safe and reliable operation of
NPPs. Among its numerous functions, this system is designed to
maintain a continuous primary fluid letdown and charging flow
via letdown line and charging line. The letdown line reduces the
primary water to pressure and temperature conditions that are
compatible with the purification/degassing systems. It is also
designed to fulfill functions of injection into primary pump seals
and auxiliary spray. Failure or degradation of system components
might significantly affect plant operations (reactivity and pressur-
izer level control). Failures which affect system non-safety-related
functions do not usually result in increase of plant risk.
The RCV interface used in this study is the interface of circuit
preparation for S-1-RCV-001, which is used in NPP in China,
Fig. 1. The interface displays system and its different components.
The display also provides operators the information necessary to
monitor the system state. Flow lines connect different system com-
ponents with arrows that indicate flow direction. Gauge values
adjacent to each of valves indicate flow rate, pressure, and temper-
ature. When the interface of RCV system is used, operators must
search and locate many valves, pumps place and set required
parameters. Although the original design is used in practice, some
 S. Yan et al. / Nuclear Engineering and Design 322 (2017) 266–276
Fig. 1. Original interfaces of circuit preparation for S-1-RCV-001.
design characteristics suggest that consideration of avoiding
human error and decreasing MWL of operators as much as possible
should be addressed. Therefore, this study applies few layout prin-
ciples based on cognitive psychology, which are used for analysis
and redesign (Deng et al., 2016; Yunpeng, 2005).
Principle 1. Cognition corresponds to objective. Using the inter-
face in line with users’ experience and knowledge and adopting the
appropriate processing method conform to the old habits and con-
cepts, could reduce learning time and memory time and avoid mis-
takes. For instance, similar functions should be arranged in the
same area, because operators would automatically search the
manipulator at the familiar area of interface.
Principle 2. Task flow design. The operating sequence design
can reduce users’ workload and improve working efficiency, also,
the main tasks of operator that need to be done should be given
and execution process of tasks should be analyzed. Moreover,
according to operators’ cognitive habits to merge or reduce unnec-
essary actions or implement automation, the dialogue process
should be simplified, which would speed up information process-
ing and reduce users’ cognitive load and time of information
processing.
Principle 3. User interface matches with users’ cognitive strat-
egy. User interface is very difficult to adapt to the dynamic cogni-
tive differences of all kinds of users. So, based on users’ knowledge
level, cognitive ability, and habits, and by judging the different
levels of users’ cognitive strategy, user interface should be
269
designed intelligently, in order to adapt it to users’ cognitive pro-
cess dynamically.
Principle 4. User interface in accordance with information orga-
nization law. The discrete stimulations in view could be organized
together forming a whole vision by certain relationship between
them, and this phenomenon is known as the visual organization
features. Visual organizational principles mainly include proximity
principle, similarity principle, and closeness principle, which have
a certain guiding significance for interface design.
After analysis, we found some limitations in this design that
might affect efficiency of operator performance in information
searching and processing, and they are labeled by 1–6 in Fig. 1.
(1) The RCV startup protocol interface does not specify task con-
tent required for completion. It also lacks low-level informa-
tion, high-level information, and links between reference
information support and services, which might not be
important to experienced operators, but it is essential for
less experienced operators.
(2) Some critical parameters and variables in RCP001YFU,
RCV001YCD, and RIS001YCD interface display unclearness
and inconsistency, so it is difficult for operators to read the
information. For instance, the data of 120MT and 109MT
are the same type as the data of 110MT, 112MT, 113MT.
However, 120MT and 109MT are located on the right side
of the main control window, while the indicators that
270 S. Yan et al. / Nuclear Engineering and Design 322 (2017) 266–276
display 110MT, 112MT, 113MT are located on the left side of
the screen. The lack of grouping reduces perception that data
are related to a common task. Furthermore, visual searching
time increases during manipulation of flow controllers
because operators’ gaze must switch between flow con-
trollers and indicators.
(3) Information flows in CP001YFU interface are not prominent,
namely they are thin and poorly visible. The more salient
solid line depicting active flow serves as an emergent feature
that can quickly provide operators the information they
need. Thus their display should be changed in term of color
or flow size to make the flows clearly displayed.
(4) Some of display information does not distinguish clearly
information type, thus it is easy to be confused and delayed
in searching of information. Therefore, an explicit mapping
between system characteristics and functions should be rep-
resented and features should be clearly displayed. For
instance, RCP001PO, RCP002PO, and RCP003PO are not dis-
played clearly. Their physical form and displayed functions
should be explicitly tied to their meaning with respect to
their functions.
(5) The information in the interface needs to be compared by
operators’ psychological integration; the information to be
compared should be arranged close to each other or aligned
if possible. For instance, in RCV001YCD interface, 310VP CL/
OPEN and 036KGMANU/AUTO need to be compared with
409KU, 409KG, 413KU in 013VP data.
Fig. 2. Redesigned interface of circuit preparation for S-1-RCV-001.
(6) Some display information is not necessary; namely, the sur-
plus information is easily disturbed to the operator. For
instance, in RCV001YCD interface, displayed information of
RCP007MN, RCP008MN and RCP011MN is not necessary,
so it is possible to hide them using certain display modes
in order to make interface neater.
According to the above analysis, the interface is redesigned
based on some standards of NUREG (O’Hara et al., 2002). The rede-
signed interface is shown in Fig. 2. Specifically, navigational links
to and from high-level and lower-level information and to refer-
ence and supporting information that should be provided when
needed for operators’ tasks are added using NUREG-07001.1-27
criterion (Navigational Links to Related Information). The criterion
of grouping of related information (NUREG-07001.1-28) is used to
suggest some groups and relevant information with the aim to
minimize user distraction, which minimizes disadvantages of
divided attention and number of attention shifts within a display
page and between them. For instance, based on this criterion,
120MT and 109MT data are moved and grouped with 110MT,
112MT, 113MT on the left side of the screen. NUREG-07001.2.8-6
(Line Coding) is used to design important display information
flows, and NUREG-07001.1-5 (Correspondence Mapping) is used
to display information that does not distinguish clearly between
other types. The information that needs to be compare by opera-
tors’ psychological integration in interface is designed based on
NUREG-07001.1-29 (Spatial Proximity for Related Information).
 S. Yan et al. / Nuclear Engineering and Design 322 (2017) 266–276
This criterion proposes that all information that must be compared
or mentally integrated should be presented in close spatial proxim-
ity. If possible, the information items should be contained in the
same display page and grouped together. Spatial proximity can
be also achieved by presenting display pages in adjacent display
windows or on adjacent display devices that can be viewed
together. For instance, for convenience of comparing data of
310VPCL and 036KG with 409KU, 409KG and 413KU of 013VP data,
in redesigned interface they are horizontally aligned and have the
same design shape. Finally, NUREG-07001.5-7 (Redundancy) is
applied to display the information that is not necessary.
The processes of interface analysis, evaluation and redesigning
were conducted carefully. In addition to designing based on cogni-
tive principles, interface layout must also meet the requirements of
technology and system functions. Therefore, interface analysis and
redesigning process were combined with comments from some
professors and experts who are specialized in human factors and
NPP technology.
2.4. Experimental task
The main objective of this study is to consider the effect of
interface layout on operators’ MWL during searching for informa-
tion in complex user interface. To accomplish this, the participants
were asked to execute the tasks of operation of system boots for S-
1-RCV-001 using two interfaces, the original interface that is used
in NPP and the redesigned interface. Task steps and actions to be
performed, such as search and locate valve/pump, set parameters,
and turn on/off valve/pump, are presented in Fig. 3 and partici-
pants had to perform these actions following the required order.
In order to use RCV procedure, 011MN and 012MN parameters
were set to less than 3 m. Participants started the system boot with
opening the RIS012VP valve and completed the procedure when
RIS013VP valve closed. The process did not have any automatic
controllers and had to be monitored and controlled by operators.
2.5. Procedures
Each participant tested both mentioned interfaces in random-
ized order. All participants were asked to complete task seriously,
and no information was given about any specific hypotheses under
investigation. The procedure of experiment was as follows. (1)
Firstly, all participants got about 30 min of training. During the
training, the participants read and listened to the electronic lec-
ture, which provided background information about location and
functioning of system components. They were taught how to use
the interface simulators, eye tracking equipment and NASA-TLX
questionnaire. (2) Then, each participant received the list of oper-
ations and practiced the procedures. The user interface for practice
was the same as that in the formal experiment except that there
was no pressure on time. The practice ended when participant
was sure that he understood all procedures. Moreover, each partic-
ipant was interviewed by a control room expert to ensure that he
was adequately prepared for the task. After that, the participant
took a five minutes rest. (3) Afterwards, each participant was pro-
vided with eye-tracker and started the first session on one of two
interfaces. In the experiment, participant was allowed to use the
list of operations if he needed it. (4) After first session, the partic-
ipant filled out NASA-TLX pairwise comparisons and completed
NASA-TLX questionnaire, and then rested for 5 min before he
started to use the other interface. It should be noted that, to
improve the reliability of subjective rating results, the NASA-TLX
questionnaire was conducted after each completed experiment
session. Certain studies indicated that evaluation results are signif-
icantly different when ratings is done before each condition and
when it is done after all conditions (Ikuma et al., 2014). (5) The par-
271
ticipant repeated steps (2) to (4) using the other interface. Each
session (steps (2) to (4)) lasted approximately 40 min.
3. Results
In all cases, a level of 0.05 was used to determine statistical sig-
nificance. The observed significance level (p-value) for a specific
statistical test is the probability of observing a value of the test
statistic. One rejects the null hypothesis if the p-value is smaller
than or equal to a level. If the level is 0.05, then the results are only
5% likely to be as extraordinary as just seen, given that the null
hypothesis is true. The statistical analysis was conducted using
SPSS software, version 20.
3.1. Performance data
The performance data of participants were analyzed to distin-
guish the differences between two interfaces, and results are sum-
marized in Table 1. The results showed that average time to
complete of two interfaces is significantly different. The partici-
pants were slower using the original interface than redesigned
interface (t = 1.97; p = 0.043). However, the error rate was not sig-
nificant different (t = 0.27; p = 0.792). The obtained performances
are presented graphically in Fig. 4.
3.2. Subjective workload
Subjective workload data obtained from participants using
NASA-TLX method were statistically evaluated, and results are
summarized in Table 2. There was a significant difference between
WWL in operating with original interface and redesigned interface
(t = 3.11; p = 0.005). Comparing six different dimensions of NASA-
TLX, significant differences were also found for MD (t = 2.81;
p = 0.022) and FR (t = 2.52; p = 0.042). NASA-TLX dimensions and
WWL score of two interfaces are presented graphically in Fig. 5.
3.3. Physiological measures
The mean value of all participants’ eye movement data of two
interfaces is illustrated in Table 3. The results of t-Test indicate that
there are significant differences in average blink rate (t = -2.37;
p = 0.027), total fixation duration (t = 2.86; p = 0.009) and fixation
rate (t = 2.01; p = 0.044). The eye movement indices of two inter-
faces are plotted in Fig. 6.
4. Discussion
This study analyzes effect of user interface layout on operators’
MWL in emergency operating procedures in NPPs. Two interfaces
of RCV control were used to analyze the interface effect based on
performance measures, subjective rating and physiological mea-
sures. Time to complete and error rate of task were used as perfor-
mance measure indices, because they directly relate to
effectiveness and safety of system operation. The obtained results
have shown that participants spent less time to finish task on rede-
signed interface but there was no significant difference in error
rate. This result can be explained by the fact that human perfor-
mance should be considered in terms of both MWL and situation
awareness. Theoretically, if interface design could reduce MWL
and enhance situation awareness, error rate would be decreased
(Salvucci and Bogunovich, 2010). However, error rate is often not
sensitive to workload in a short experimental time (Hwang et al.,
2008; Lanzetta et al., 1987). Additionally, in experimental condi-
tions, participants often do not have psychological comfort because
they suffer the consequences of their mistakes when the operation
272 S. Yan et al. / Nuclear Engineering and Design 322 (2017) 266–276

Fig. 3. Operation/check list of system boots for S-1-RCV-001.

Table 1 ing to the workload of participant for both interfaces. This partially
t-Test of performance data between two interfaces. confirms the research problem statement that operation of NPPs is
Original interface Redesign p-value
mentally demanding job. In addition to MD, TD also received high
(Mean ± SD) (Mean ± SD) rating in contributing to the workload of operator for both HCIs.
Time to complete (s) 275.6 ± 96.8 231.9 ± 81.4 0.043*
Namely, TD is influenced by parameters such as time pressure
Error rate (%) 10.2 ± 8.2 9.6 ± 7.1 0.792 and task speed. Due to the emergency operating procedures and
*
fact that each participant was asked to complete task seriously
p  0.05.
and hence provides higher ratings of workload. EF and FR dimen-
sions were also found as important parameters that contribute to
fails or does not fulfill task requirements which might affect oper- operator workload, because all persons who tested the interfaces
ation performance discussed in this study. were novices, and they did not have relevant experience with oper-
The results of subjective rating have shown that operators’ ating of NPPs and they had to complete task in a short time, so the
MWL for different interfaces is different. The WWL of NASA-TLX amount of required effort was significantly increased. Additionally,
of redesigned interface was reduced for 9.8% in comparison to the frustration originated from searching and processing of infor-
the original interface, which indicates that interface design has sig- mation in the task. Namely, the RCV interface had many icons
nificant effect on operators’ MWL. The results of NASA-TLX score and data which operator needs to distinguish, which makes oper-
have shown that MD dimension received high ratings in contribut- ator to feel difficult to complete the task.
 S. Yan et al. / Nuclear Engineering and Design 322 (2017) 266–276 273

Fig. 4. Performance data of two interfaces. Vertical bars denote the mean standard error.

Table 2 significant differences in pupil dilation between two interfaces,

t-Test of NASA-TLX between two interfaces. but there were significant differences in the mean blink rate and
Original interface Redesign p-value fixations.
(Mean ± SD) (Mean ± SD) Several studies associated the blink rate with MWL (Wolkoff
Mental demand (MD) 68.4 ± 9.8 58.9 ± 13.2 0.022* et al., 2005). Van Orden et al. (2000) found that blink rate increases
Physical demand (PD) 47.9 ± 12.2 43.5 ± 11.4 0.202 as a function of tracking error in human factors tracking task. How-
Temporal demand (TD) 61.7 ± 14.6 58.6 ± 13.4 0.495 ever, Veltman and Gaillard (1998) concluded that blinking is inde-
Own Performance (OP) 46.6 ± 12.8 48.9 ± 13.0 0.845
pendent of MWL. These results indicate that more research is
Efforts (EF) 53.0 ± 12.1 47.3 ± 14.3 0.081
Frustration (FR) 59.8 ± 12.5 53.5 ± 10.1 0.042* required before blink rate could be used as a measure of MWL
Overall workload (WWL) 58.0 ± 7.1 52.3 ± 5.6 0.005** (Kramer, 1991). In this study, we found a significant difference in
* the mean blink rate between two interfaces. Compared to rede-
p  0.05.
**
p  0.01. signed interface, the average blink rate of original interface was
higher for 14.8%. Therefore, operator had to process more visual
information in original interface, so duration of blinks was lower.
Although, the eye movement method has certain limitations,
such as the fact that eye is easily interfered by lighting environ-
ment and impact of eye-tracking camera’s ability to record eye Table 3
t-Test of eye movement between two interfaces.
movements, eye movement is useful measure to determine the
areas of interface that draw user’s attention, whereas the other Original interface Redesign p-value
tools only provide information on interface as a whole (Lehrer (Mean ± SD) (Mean ± SD)
et al., 2010). In addition, many indexes of this method directly Pupil dilation (pixel) 53.1 ± 6.9 51.5 ± 5.7 0.330
relate to MWL and they are commonly used in assessment of Blink rate (times/min) 18.5 ± 5.2 21.6 ± 6.2 0.027*
Total fixation duration (s) 87.9 ± 29.7 64.5 ± 21.5 0.009**
MWL. The size of human eye pupil is often used as a measure
Fixation rate (times/s) 1.57 ± 0.92 1.14 ± 0.64 0.044*
of mental effort because it is assumed that pupil size is related
*
to the amount of cognitive control, attention, and cognitive pro- p  0.05.
**
p  0.01.
cessing required by a given task. This experiment did not found

Fig. 5. NASA-TLX scores of two interfaces. Vertical bars denote the mean standard error.
274 S. Yan et al. / Nuclear Engineering and Design 322 (2017) 266–276

Fig. 6. Eye movement indices of two interfaces. Vertical bars denote the mean standard error.

It complies with many researches, wherein it was suggested that improved interface design based on human factor criterions has a
blink rate decrease reflects a strategy to maximize the time avail- significant effect on operator MWL. However, this is just an assess-
able for visual processing when visual demands are the highest. ment of user interface effect based on ergonomics, thus we are not
Further, it is possible that differences in blink rate and mental/ able to conclude that the redesigned interface presented in this
visual workload reflect different types of task (Veltman and study is the best design. Rather, this interface design can be useful
Gaillard, 1998). for interface designers in the future.
Recently, numerous parameters that characterize eye fixation
have been studied in relation with human’s MWL. Many research-
4.1. Limitations
ers, such as Tole et al. (1982), found an increase in fixation duration
when MWL increases, and Goldberg and Kotval (1999) found a
There are several limitations of this study. First, a small number
negative correlation between fixation time and performance. In
of samples was used to test interface designs in experiments.
this study, in the task of searching and processing of information
Moreover, a small size of samples reduced statistical power, partic-
in user interface, there were significant differences between two
ularly for interface factor evaluation. Additionally, all participants
interfaces in terms of fixation duration and fixation rate. These
were students, who cannot represent the real operators of NPPs.
results have indicated that redesigned interface is more effective
Although after training and practice they had adequate knowledge
in information searching because higher fixation rate on a particu-
and skills for solving the problem used in the experiment, their
lar area can be indicative of greater interest in the target, or it can
experience might be very different from experience of real opera-
be a sign that the target is complex in some way and more difficult
tors who have been trained for years. Thus, further validation with
to be searched (Nakayama et al., 2002).
professional operators or experts is needed before applying these
The goal of user interface design is to produce a user interface
results in the industry. Also, the experiment was not performed
which makes it easy to use, efficient, and friendly to operate the
in real environment (real environment would make the study more
system in the way which produces the desired result and improv-
realistic and reliable) which may cause significant differences
ing the interface layout often gains significant efficiency (Niemelä
among the outcomes, thus results have limited reliability. Despite
and Saarinen, 2000). Some similar experiments with slightly differ-
these limitations, we believe this study contributes to understand-
ent icons were also reported in other studies on evaluation of user
ing of interface layout design effect on operators’ MWL during the
interface system, wherein an eye-tracking device was used for data
operation of EOPs.
gathering, and analysis of layout and its impact on visual search
performance (Murata and Furukawa, 2005; Van Schaik and Ling,
2001). They demonstrated that speed of task execution is signifi- 5. Conclusions
cantly higher for good interface layout. Comparative evaluation
results of this study show that new interface is more effective in Reality shows that HCI design greatly influences information
information searching and processing, which means that the acquisition in operator control, time of feedback and final results,
 S. Yan et al. / Nuclear Engineering and Design 322 (2017) 266–276
leading to possible human factor accidents. Therefore, the
improvement of HCI design is essential, especially in the field of
NPPs. This paper reports the effect of user interface layout on oper-
ators’ MWL based on human performances, NASA-TLX rating and
eye movement. The original design is used in practice, but some
design characteristics suggest that consideration of avoiding
human error and decreasing MWL of operators as much as possible
should be addressed. According to the obtained results, the rede-
signed interface is more effective than the original interface in both
information searching and processing, and some evaluation tools
are sensitive to differences between interface designs. Although
this is just an assessment of user interface effect based on ergo-
nomics, the results of this study can be a real basis for reference
and development of new interface designs in the future.
Acknowledgements
This study was conducted in the Lab of the College of Mechan-
ical Engineering at Harbin Engineering University, Harbin, Hei-
longjiang, China. The authors would like to thank the reviewers
for their valuable remarks and comments. Also, the authors thank
the experts and students who helped conduct this research.
References
Ahlstrom, U., Friedman-Berg, F.J., 2006. Using eye movement activity as a correlate
of cognitive workload. Int. J. Ind. Ergon. 36, 623–636.
Baker, C.A., Morris, D.F., Steedman, W.C., 1960. Target recognition on complex
displays. Hum. Fact.: J. Hum. Fact. Ergon. Soc. 2, 51–60.
Booth, P., 1989. An Introduction to Human Computer Interaction. Lawrence
Erlbaum Associates Ltd, Hills dale.
Burns, A., 1991. The HCI component of dependable real-time systems. Softw. Eng. J.
6, 168–174.
Casali, J.G., Wierwille, W.W., 1983. A comparison of rating scale, secondary-task,
physiological, and primary-task workload estimation techniques in a simulated
flight task emphasizing communications load. Hum. Fact. 25, 623–641.
De la Torre, G.G., Ramallo, M.A., Cervantes, E., 2016. Workload perception in drone
flight training simulators. Comput. Hum. Behav. 64, 449–454.
Deng, L., Wang, G., Yu, S., 2016. Layout design of human-machine interaction
interface of cabin based on cognitive ergonomics and GA-ACA. Comput. Intell.
Neurosci. 2016, 2.
Di Stasi, L.L., Antolí, A., Cañas, J.J., 2013. Evaluating mental workload while
interacting with computer-generated artificial environments. Entertain.
Comput. 4, 63–69.
Downing, J.V., Sanders, M.S., 1987. The effects of panel arrangement and locus of
attention on performance. Hum. Fact. 29, 551–562.
Dyer, R.F., Matthews, J.J., Wright, C.E., Yudowitch, K.L., 1976. Questionnaire
construction manual. DTIC Document.
Gao, Q., Wang, Y., Song, F., Li, Z., Dong, X., 2013. Mental workload measurement for
emergency operating procedures in digital nuclear power plants. Ergonomics
56, 1070–1085.
Gao, Q., Yu, W., Jiang, X., Song, F., Pan, J., Li, Z., 2015. An integrated computer-based
procedure for teamwork in digital nuclear power plants. Ergonomics 58, 1303–
1313.
Gawron, V.J., 2008. Human Performance, Workload, and Situational Awareness
Measures Handbook. CRC Press.
Goldberg, J.H., Kotval, X.P., 999. Computer interface evaluation using eye
movements: methods and constructs. Int. J. Ind. Ergon. 24, 631–645.
Götze, M., Ruff, F., Bengler, K., 2015. Optimal information output in urban traffic
scenarios: an evaluation of different HMI concepts. Procedia Manuf. 3, 739–746.
Guo, F., Ding, Y., Liu, W., Liu, C., Zhang, X., 2016. Can eye-tracking data be measured
to assess product design?: visual attention mechanism should be considered.
Int. J. Ind. Ergon. 53, 229–235.
Ha, J.S., Seong, P.H., 2009. A human–machine interface evaluation method: a
difficulty evaluation method in information searching (DEMIS). Reliab. Eng.
Syst. Saf. 94, 1557–1567.
Hampson, R., Opris, I., Deadwyler, S., 2010. Neural correlates of fast pupil dilation in
nonhuman primates: relation to behavioral performance and cognitive
workload. Behav. Brain Res. 212, 1–11.
Hart, S., Staveland, L., 1988. Development of NASA-TLX (Task Load Index): results of
empirical and theoretical research. Hum. Mental Workload, 139–183.
Hertzum, M., Holmegaard, K.D., 2013. Perceived time as a measure of mental
workload: effects of time constraints and task success. Int. J. Hum. Comput.
Interact. 29, 26–39.
Ho, C.-H., Lu, Y.-N., 2014. Can pupil size be measured to assess design products? Int.
J. Ind. Ergon. 44, 436–441.
275
Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., Van de Weijer,
J., 2011. Eye tracking: A comprehensive guide to methods and measures. OUP
Oxford.
Hwang, S.-L., Yau, Y.-J., Lin, Y.-T., Chen, J.-H., Huang, T.-H., Yenn, T.-C., Hsu, C.-C., 2008.
Predicting work performance in nuclear power plants. Saf. Sci. 46, 1115–1124.
Ikuma, L.H., Harvey, C., Taylor, C.F., Handal, C., 2014. A guide for assessing control
room operator performance using speed and accuracy, perceived workload,
situation awareness, and eye tracking. J. Loss Prev. Process Ind. 32, 454–465.
Isaac, A., Shorrock, S.T., Kirwan, B., 2002. Human error in European air traffic
management: the HERA project. Reliab. Eng. Syst. Saf. 75, 257–272.
Jimmieson, N.L., Terry, D.J., 998. An experimental study of the effects of work stress,
work control, and task information on adjustment. Appl. Psychol. 47, 343–369.
Jou, Y.-T., Yenn, T.-C., Lin, C.J., Yang, C.-W., Chiang, C.-C., 2009. Evaluation of
operators’ mental workload of human–system interface automation in the
advanced nuclear power plants. Nucl. Eng. Des. 239, 2537–2542.
Kramer, A.F., 1991. Physiological metrics of mental workload: a review of recent
progress. Multiple-Task Perform., 279–328
Lanzetta, T.M., Dember, W.N., Warm, J.S., Berch, D.B., 1987. Effects of task type and
stimulus heterogeneity on the event rate function in sustained attention. Hum.
Fact.: J. Hum. Fact. Ergon. Soc. 29, 625–633.
Lehrer, P., Karavidas, M., Lu, S.E., Vaschillo, E., Vaschillo, B., Cheng, A., 2010. Cardiac
data increase association between self-report and both expert ratings of task
load and task performance in flight simulator tasks: an exploratory study. Int. J.
Psychophysiol. 76, 80–87.
Mat Zain, N., Abdul Razak, F., Jaafar, A., Zulkipli, M., 2011. Eye tracking in
educational games environment: evaluating user interface design through eye
tracking patterns. Visual Informatics: Sustaining Research and Innovations, pp.
64-73.
Michalski, R., Grobelny, J., 2008. The role of colour preattentive processing in
human–computer interaction task efficiency: a preliminary study. Int. J. Ind.
Ergon. 38, 321–332.
Montés-Micó, R., Alió, J.L., Charman, W.N., 2005. Dynamic changes in the tear film in
dry eyes. Invest. Ophthalmol. Vis. Sci. 46, 1615–1619.
Moray, N., 1988. Mental workload since 1979. Int. Rev. Ergon. 2, 123–150.
Moresi, S., Adam, J.J., Rijcken, J., Van Gerven, P.W., Kuipers, H., Jolles, J., 2008. Pupil
dilation in response preparation. Int. J. Psychophysiol. 67, 124–130.
Murata, A., Furukawa, N., 2005. Relationships among display features, eye
movement characteristics, and reaction time in visual search. Hum. Fact. 47,
598–612.
Naderpour, M., Lu, J., Zhang, G., 2016. A safety-critical decision support system
evaluation using situation awareness and workload measures. Reliab. Eng. Syst.
Saf. 150, 147–159.
Nakayama, M., Takahashi, K., Shimizu, Y., 2002. The act of task difficulty and eye-
movement frequency for the’Oculo-motor indices’, Proceedings of the 2002
symposium on Eye tracking research & applications. ACM, pp.37–42.
Niemelä, M., Saarinen, J., 2000. Visual search for grouped versus ungrouped icons in
a computer interface. Hum. Fact. 42, 630–635.
O’Hara, J., Brown, W., Lewis, P., Persensky, J., 2002. Human-System Interface Design
Review Guidelines (NUREG-0700, Rev 2). US Nuclear Regulatory Commission,
Washington, DC, p. 12.
Parasuraman, R., Sheridan, T.B., Wickens, C.D., 2000. A model for types and levels of
human interaction with automation. IEEE Trans. Syst. Man Cybernetics-Part A:
Syst. Hum. 30, 286–297.
Recarte, M.A., Nunes, L.M., 2000. Effects of verbal and spatial-imagery tasks on eye
fixations while driving. J. Experim. Psychol. Appl. 6, 31.
Recarte, M.Á., Pérez, E., Conchillo, Á., Nunes, L.M., 2008. Mental workload and visual
impairment: differences between pupil, blink, and subjective rating. Spanish J.
Psychol. 11, 374–385.
Rubio, S., Díaz, E., Martín, J., Puente, J.M., 2004. Evaluation of subjective mental
workload: a comparison of SWAT, NASA-TLX, and workload profile methods.
Appl. Psychol. 53, 61–86.
Salvucci, D.D., Bogunovich, P., 2010. Multitasking and monotasking: the effects of
mental workload on deferred task interruptions, Proceedings of the SIGCHI
conference on human factors in computing systems. ACM, pp. 85–88.
Scott, I.U., Feuer, W.J., Jacko, J.A., 2002. Impact of graphical user interface screen
features on computer task accuracy and speed in a cohort of patients with age-
related macular degeneration. Am. J. Ophthalmol. 134, 857–862.
Seker, A., 2014. Using outputs of NASA-TLX for building a mental workload expert
system. Gazi Univ. J. Sci. 27, 1131–1142.
Sharma, C., Bhavsar, P., Srinivasan, B., Srinivasan, R., 2016. Eye gaze movement
studies of control room operators: a novel approach to improve process safety.
Comput. Chem. Eng. 85, 43–57.
Toker, D., Conati, C., Steichen, B., Carenini, G., 2013. Individual user characteristics
and information visualization: connecting the dots through eye tracking,
Proceedings of the SIGCHI conference on human factors in computing
systems, Paris, France, pp. 295–304.
Tole, J., Stephens, A.T., Harris, R., Ephrath, A., 1982. Visual scanning behavior and
mental workload in aircraft pilots. Aviation, Space, and EnvironmentalMedicine.
Tsang, P.S., Vidulich, M.A., 2006. Mental Workload and Situation Awareness. Wiley,
New York, NY.
Van Orden, K.F., Jung, T.-P., Makeig, S., 2000. Combined eye activity measures
accurately estimatechanges in sustained visual task performance. Biol. Psychol.
52, 221–240.
Van Orden, K.F., Limbert, W., Makeig, S., Jung, T.-P., 2001. Eye activity correlates of
workload during a visuospatial memory task. Hum. Fact.: J. Hum. Fact. Ergon.
Soc. 43, 111–121.
276 S. Yan et al. / Nuclear Engineering and Design 322 (2017) 266–276
Van Schaik, P., Ling, J., 2001. The effects of frame layout and differential background
contrast on visual search performance in web pages. Interact. Comput. 13, 513–
525.
Veltman, J., Gaillard, A., 1998. Physiological workload reactions to increasing levels
of task difficulty. Ergonomics 41, 656–669.
Wei, Z., Zhuang, D., Wanyan, X., Liu, C., Zhuang, H., 2014. A model for discrimination
and prediction of mental workload of aircraft cockpit display interface. Chin. J.
Aeronaut. 27, 1070–1077.
Wierda, S.M., van Rijn, H., Taatgen, N.A., Martens, S., 2012. Pupil dilation
deconvolution reveals the dynamics of attention at high temporal resolution.
P. Natl. A. Sci. 109, 8456–8460.
Wilson, G.F., 2002. An analysis of mental workload in pilots during flight using
multiple psychophysiological measures. Int. J. Aviat. Psychol. 12, 3–18.
Wolkoff, P., Nøjgaard, J.K., Troiano, P., Piccoli, B., 2005. Eye complaints in the office
environment: precorneal tear film integrity influenced by eye blinking
efficiency. Occup. Environ. Med. 62, 4–12.
Wood, R., Bandura, A., Bailey, T., 1990. Mechanisms governing organizational
performance in complex decision-making environments. Organ. Behav. Hum.
Decis. Process. 46, 181–201.
Yunpeng, Z., 2005. Human-computer interface design based on knowledge of
cognitive psychology. Comput. Eng. Appl. 30, 034.