International Journal of Industrial Ergonomics 78 (2020) 102974

Contents lists available at ScienceDirect

International Journal of Industrial Ergonomics

journal homepage: http://www.elsevier.com/locate/ergon

Workload measurement using physiological and activity measures for

validation test: A case study for the main control room of a nuclear
power plant
Yu Gan a, b, Xiaolu Dong b, c, Yijing Zhang a, b, Xuegang Zhang a, Ming Jia a, Zhaopeng Liu a,
Zhizhong Li b, a, *
a
State Key Laboratory of Nuclear Power Safety Monitoring Technology and Equipment, China Nuclear Power Engineering Co., Ltd (CNPEC), Shenzhen, Guangdong,
China
b
Department of Industrial Engineering, Tsinghua University, Beijing, China
c
Nuclear and Radiation Safety Center, Ministry of Ecology and Environment, Beijing, China

A R T I C L E I N F O A B S T R A C T

Keywords: In design of safety-critical and social-technical systems such as a nuclear power plant, practitioners are required
Workload to conduct a performance-based Integrated System Validation (ISV) test to verify that the system design could
Task complexity support the safe operation of the plant. Measurement of workload should be included. However, subjective
Physiological measures
workload measurements could not provide detailed information and continuous monitoring of the changing
Nuclear power plants
Validation test
workload. This study compared physiological (heart rate difference, heart rate variability, respiration rate and
breathing wave amplitude) and activity (number of walking steps, peak acceleration, activity level, and incli­
nation) measures with workload defined as intensity of task demand and estimated with a task complexity
measure in an ISV test. The test was conducted on a full-scale simulator using a beyond design-basis accident
scenario. The results show that heart rate difference and respiration rate are positively correlated with the
estimated workload, while heart rate variability and breathing wave amplitude are negatively correlated with
the estimated workload. For operations using traditional panels, high workload is accompanied by larger number
of walking steps, higher activity level, and smaller angles of inclination. It is suggested that continuous moni­
toring of cardiovascular, respiration, and activity measures can detect workload change during the ISV test.
Relevance to industry: This study provides recommendations for continuous monitoring of workload during an
ISV test of a nuclear power plant. The identified physiological and activity measures can be applied in detecting
workload change. The findings are supportive in meeting regulatory requirements and improving system design
in the nuclear domain.

1. Introduction
In design of safety-critical and social-technical systems such as a
nuclear power plant, designers are eager to know the performance of
human operators in tasks, because any errors or miss judgement might
lead to unwanted situations. Workload of human operators during task
performance is of special interest to the investigators. Regulations and
standards in the nuclear industry also have specific requirements on the
evaluation of workload both during the analysis stage and validation
stage (IEC, 1995, 2000; 2018; O’Hara et al., 2004). Practitioners are
required to conduct a performance-based Integrated System Validation
(ISV) test to verify that the integrated system design could support the
safe operation of the plant, which should employ the measurement of
cognitive workload (O’Hara et al., 2004; O’Hara et al., 1997).
Workload could be described as the relationship between the quan­
titative resource demands imposed by a task and the ability to supply
those resources by the task performer (Wickens, 2002; Young et al.,
2015). Therefore, task analysis that included the human information
processing resources and activities could be adopted as an estimation of
workload. From this perspective, VACP (Visual, Auditory, Cognitive,
and Psychomotor) model has been regarded as an analytical assessment
of workload that is composed of three information processing stages:

* Corresponding author. Department of Industrial Engineering, Tsinghua University, Beijing, China.

E-mail address: zzli@tsinghua.edu.cn (Z. Li).

https://doi.org/10.1016/j.ergon.2020.102974
Received 19 March 2020; Received in revised form 30 April 2020; Accepted 11 May 2020
Available online 28 May 2020
0169-8141/© 2020 Elsevier B.V. All rights reserved.
Y. Gan et al.
sensory, cognitive, and psychomotor (Aldrich et al., 1989; Liang et al.,
2014; McCracken and Aldrich, 1984; Rusnock and Borghetti, 2018). As
an example, Task Complexity (TC) measure proposed by Liu et al. (2012)
was proposed to estimate resource requirements (the task complexity)
using the VACP rating scales. The tasks were first decomposed into
meta-operations, and then the VACP rating scales were used to assess the
complexity scores of the meta-operations. The complexity of the task
was calculated as the summation of the complexity scores of its
meta-operations (Liu et al., 2012). The predictive validity of the TC
measure has been studied using emergency operating procedure (EOP)
performance data collected under laboratory conditions (Liu et al.,
2012; Xu et al., 2009). By definition, the result of TC measure could be
considered as workload to be imposed by a task, since it describes the
resource demands and information processing requirements of the task.
As a product of task complexity, workload should normally increase
with the increasing task complexity (Wickens et al., 2015). Human
response to stress could be complicated and context dependent. There
are cases where workload dissociates from task demands when task
demands are excessively high. For our case, i.e. accident handling in the
main control room of a nuclear power plant, the highly professional
crew would adopt an active coping strategy to maintain the level of
performance as the demands increase, at the expense of increased sub­
jective effort, and behavioral and physiological costs (Robert and
Hockey, 1997). Thus, it is assumed that crew activities would increase
with the increasing task demands and such influence could be detected
by physiological measures.
Physiological measures have been tested as indicators of task de­
mands. Physiological measures of workload include cardiac measures,
brain activities, and eye activities (Ahlstrom and Friedman-Berg, 2006;
Charles and Nixon, 2019; Ryu and Myung, 2005; Shakouri et al., 2018;
Wierwille and Eggemeier, 1993). Heart rate has been found to increase
with the overall arousal and physical exertion, and emotional responses
associated with task demands in general (Wierwille and Eggemeier,
1993), but some studies suggested opposite influence of different task
demands on heart rate (Kramer, 1991). Variation of heart beat intervals
has been shown to decrease with the increase of workload due to
increased level of arousal (De Rivecourt et al., 2008). This measure was
also found to be sensitive to fatigue and interact with respiration (Gao
et al.; Veltman and Gaillard, 1998). As for the respiration related vari­
ables, duration of a respiratory cycle has been found sensitive to changes
in workload (Veltman and Gaillard, 1998). These physiological mea­
sures could provide continuous measurements of the change of
workload.
In the present practices of ISV tests, evaluators use subjective
workload measurements such as NASA-TLX and Overall Workload
which have been adopted in many domains (Braarud, 2020; Gawron,
2008; Grier, 2015; Hart, 2006; Hart and Staveland, 1988). However,
subjective and discontinuous measurements of workload could not
provide detailed information on the trend of workload. In addition, the
limited number of evaluations taken during or after the tests could not
support real time monitoring of workload (Rusnock and Borghetti,
2018). Furthermore, subjective measurements could be suffered from
rater bias (Gawron, 2008). Therefore, adopting continuous physiolog­
ical measures as objective indicators of workload could be a possible
solution to these problems and a good supplementary to workload
measurement in ISV tests.
In this study, the researchers observed and recorded a beyond
design-basis accident handled by a four-member crew on a nuclear
power plant full-scale simulator during an ISV test. Physiological and
activity data of heart rate difference, heart rate variability, respiration
rate, breathing wave amplitude, number of walking steps, peak accel­
eration, physical activity level, and inclination were collected along the
test. These variables were compared with the analysis of workload based
on the TC method in order to: 1) identify physiological and activity
measures that correlate with workload; 2) explore the relationships
between these measures and workload; 3) provide recommendations for
2
International Journal of Industrial Ergonomics 78 (2020) 102974
continuous monitoring of workload during an ISV test. In the following
sections, the test scenario, workload analysis, measures recorded, and
data analysis results will be reported.
2. Method
2.1. Scenario
During an ISV test of a beyond design-basis accident on a full-scale
simulator, physiological and activity measurements were taken for an
operating crew with four members, namely first loop operator, second
loop operator, coordinator (or unit supervisor), and safety engineer. The
scenario was a small LOCA (Loss Of Coolant Accident) combined with a
failure in the Reactor Protection Monitoring System (RPMS). For the
main control room in question, a Diversity Hardware Panel (DHP) and a
Severe Accident Panel (SAP) were added to the original backup panels,
the application of which was the main focus of this ISV test.
Four crew members were included in the ISV test. The first loop
operator is responsible for operations concerning nuclear island; the
second loop operator is responsible for operations concerning conven­
tional island; the coordinator, also named unit supervisor in this case, is
responsible for the direct supervision of two reactor operators; finally,
the safety engineer, on the other hand, is responsible for providing in­
dependent supervision and evaluation of plant safety conditions. The
two reactor operators hold valid certifications of reactor operator issued
by the regulation department. The coordinator and safety engineer hold
valid senior reactor operator certifications.
The traditional hardware panels include integrated panels of Back
Up Panel (BUP), DHP, and SAP. They are located in front of four
computerized workstations. The four workstations are arranged in two
rows, with two at the front and two at the back. The transfer switches for
the DHP and SAP are installed on the Emergency Control Panel (ECP)
which is located between two computerized workstations in the front
row.
The details of the simulated scenario were not revealed to the
operating crew before the test. Before running the simulated scenario,
the experimenter informed the crew of the initial status of the unit. The
crew were told to perform according the requirements of the work
specifications and procedures. They were also informed of the general
process of the test.
At the beginning of the scenario, the unit operated at full-power and
the operating crew performed the task on the computerized worksta­
tions. After an accident of small LOCA (Loss Of Coolant Accident) was
introduced into the simulated scenario, the crew entered the Symptom-
based Operating Procedures (SOPs) by certain alarms. While the crew
was performing accident diagnose using SOP, the signals of reactor trip
and safety injection were triggered by the dropping pressure. Because of
the failure in RPMS, the reactor failed to trip and a red alarm was trig­
gered. By following the instructions on the corresponding alarm sheet,
the crew stopped the execution of SOP and transfer to Diversity Driven
System (KDS) procedure. According to the instructions of KDS proced­
ure, the crew first transferred the control to DHP by turning the switch
on the Emergency Control Panel (ECP) and then performed KDS pro­
cedure on the related traditional hardware panels. The switch took place
approximately 10 min after the beginning of the test.
The operating crew performed the KDS procedure on the traditional
hardware panels in this scenario. The procedures are specially devel­
oped for every crew member’s responsibility. Once the switch has been
made, the coordinator would stand behind the reactor operators and
supervise the operating process. The activities of operators in interacting
with the traditional panels included locating indicators, checking system
status, reading parameters, pressing a button, turning a knob, and
locating alarms, etc. (Table 1). Major operations in this scenario
included leakage detection, primary pumps shutdown, safety injection
mode switching to charge mode, operation of letdown system, lowering
the temperature and pressure, and preparing for the connect to the
Y. Gan et al. International Journal of Industrial Ergonomics 78 (2020) 102974

Table 1 2.3. Measures

VACP scores for meta-operations.
ID Meta-operations Resources V A C P Sum The physiological and activity data were recorded and processed
using Zephyr™ BioHarness™ 3 physiology monitoring system and
A Interpreting procedures V&C 5.9 5.3 11.2
B Locating indicators V 5 5 OminiSense 5.0 software. Meanwhile, video and audio recordings were
C Checking conditions/status V 4 4 taken along the ISV test. Data of the following measures were collected
D Reading parameters/status V 5.9 5.9 with around 1Hz reporting frequency:
E Decision making and C 4.6 4.6
diagnosing (single
decision)
(1) Heart rate difference. Heart rate was determined from analysis of
F Decision making and C 6.8 6.8 250Hz ECG data. Heart rate difference was the difference of heart
diagnosing (multiple rates between task performing and resting, in order to minimize
decisions) subject-to-subject variations.
G Calculating C 7 7
(2) Heart rate variability. Heart rate variability was a measure of the
H Pressing a button C&P 1 2.2 3.2
I Turning a knob C&P 1 5.8 6.8 variations in R to R intervals. It was a rolling 300 beat Standard
J Selecting a procedure C&P 1.2 2.2 3.4 Deviation of Normal-Normal intervals (SDNN) value.
branch (3) Respiration rate (breaths per minute). Respiration rate was
K Listening to verbal A 4.9 4.9 detected by a pressure sensor that detected torso expansion and
directions
L Transmitting verbal A&P 1 1 2
contraction due to breathing.
messages (4) Breathing wave amplitude (volt). Breathing wave amplitude was
M Locating/identifying A&C 4.2 3.7 7.9 the average of breathing sensor output during previous 1 s, which
alarms by sound was an indicative measure of breathing depth.
(5) Number of walking steps. It was the count of detected walking
steps.
residual heat removal system. When the letdown system was in service
(6) Peak acceleration. Peak acceleration was the maximum absolute
and the crew began to lower the temperature and pressure, the experi­
value of three-axis acceleration magnitude achieved during pre­
menter announced the end of the scenario. The test lasted for about an
vious 1 s period.
hour. An interview with the operating crew was conducted soon after
(7) Activity level (g). Activity level was a measure of mechanical
the test.
activity with unit of Velocity Magnitude Unit (VMU) measured in
g (gravitational acceleration). VMU was calculated with formula
2.2. Task analysis
(a), where x, y and z were the averages of the three axial accel­
eration magnitudes over the previous 1 s period, sampled at
The Task Complexity (TC) method was adopted in this study to
100Hz.
provide an estimation of workload (Liu et al., 2012). It is assumed that
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
the physiological and activity measurements recorded during task per­ VMU ¼ x2 þ y2 þ z2 (a)
formance would be correlated with this estimated workload. Twelve
meta-operations were identified for the execution of the operating
procedures in the control room of a nuclear power plant using tradi­ (8) Inclination (degree). Inclination of a subject was the absolute
tional panels (Table 1). The meta-operations and VACP scores were value of the deviation from vertical position measured in degrees.
determined based on video and audio recordings by the researchers. Lateral (side to side) changes in posture were not detected.
There were five steps in the evaluation of workload:
In this study, data for each of the above measures were averaged
(1) Decomposition of the task into units. Firstly, the task was broken across the reported values within every task unit. The analysis was
down into several task units using both procedures and video conducted using these averaged measurements.
recordings. Task units were relatively separable task blocks that
constituted the execution of procedures along the timeline. For 3. Results
each member of the operating crew, the 1-h ISV test in this study
was divided into several task units. The starting and ending time The descriptive statistics for estimated workload of task units in
of each task unit were determined by watching the video re­ Table 2 indicate a comparatively higher mean score for the first loop
cordings. The task decomposition results were different for the operator (M ¼ 6.29, SD ¼ 4.70). The resource requirement for the safety
four crew members because of their different duties during the engineer was less severe than that for other members of the crew (M ¼
process. A total of 120 task units were identified for the crew. 1.81, SD ¼ 1.50). The data of estimated workload did not pass the
(2) Decomposition of each unit into steps. Each task unit was divided normality test (Shapiro-Wilk test, W ¼ 0.75, p < 0.001). The descriptive
into specific steps according to the corresponding operating statistics for task units show relatively short time span of task units in
procedures. seconds (M ¼ 109.05, SD ¼ 53, Min ¼ 11, Max ¼ 264). Fig. 1 presents the
(3) Decomposition of each step into meta-operations. Each step was trend of estimated workload for the four crew members during the ISV
disassembled into meta-operations that required visual, auditory, test. It provides an illustration of the resource and coordination re­
cognitive, and psychomotor resources. The meta-operations quirements of the accident scenario in this study.
applied in this study are indicated in Table 1.
(4) Estimation of workload for each task unit. The complexity score
Table 2
for each step was the summation of VACP scores for all of its
Descriptive statistics for the estimated workload.
meta-operations. And the complexity score for each unit was the
Number of task units Mean SD
summation of scores for all of its steps. In order to avoid the
accumulation of complexity scores for relatively long task units, First loop operator 33 6.29 4.70
the estimated workload of each task unit was calculated as the Second loop operator 30 4.33 3.18
Coordinator 31 2.39 2.22
complexity score per 10 s. Safety engineer 26 1.81 1.50
Total 120 3.82 3.64

3
Y. Gan et al. International Journal of Industrial Ergonomics 78 (2020) 102974

The empirical data show deep and long breathings accompanied by

higher heart rate variability, which is consistent with the findings in
precious studies (Gao et al., 2013; Veltman and Gaillard, 1998). How­
ever, these characteristics were only shown when estimated workload
was low in this study. This fact disagrees with the inference made by Gao
et al. (2013) that these characteristics would appear under high work­
load. Further exploration into the step VACP scores of the low and high
complexity procedures in Gao’s study revealed higher average step
VACP score for the low complexity procedure level. Inference could be
made that the measure of heart rate variability was sensitive not to
accumulated workload, but to workload change in short time periods.

4.2. Activity measures

Fig 1. Estimated workload for the four crew members. Number of walking steps, activity level, and inclination of operators
could be considered as indicators of changing workload. Logically, the
Apart from peak acceleration, the other selected measures were physical activities of operating crews were the result of task re­
significantly correlated with estimated workload of task units at 0.05 quirements. By just observing crew members’ behavior, time periods
significant level (Table 3). Among the physiological measures, the cor­ with high workload could be easily detected, although with less preci­
relation coefficients between heart rate variability and estimated sion. During high workload periods, the operators moved about more
workload (ρ¼-0.42, p < 0.001), as well as between breathing wave frequently, changed directions more quickly, and leaned forward (or
amplitude and estimated workload (ρ¼-0.46, p < 0.001) were negative backward) with smaller angles for examinations. In other words,
with medium to large effect sizes. For the measures of heart rate dif­ because decisions and manipulations were made in relatively shorter
ference and respiration rate, the correlation coefficients were positive time periods, the operators appeared to be more active in front of the
(ρ¼0.40, p < 0.001, ρ¼0.27, p ¼ 0.01). Fig. 2 illustrates changes of hardware panels. The negative correlation between inclination and
physiological measures against estimated workload for the first loop estimated workload suggested that the operators reduced the time for
operator. The relationships of positive or negative correlations could be information gathering to compensate for the higher task requirements.
observed by similar or opposite shapes of trend curves. As for the ac­ Attention should be paid to this phenomenon in that the risk of
tivity measures, number of walking steps and activity level were posi­ misreading system status would increase.
tively correlated with estimated workload (ρ¼0.24, p ¼ 0.01, ρ¼0.48, p It should be noted that the original outputs of posture included
< 0.001). While for the measure of inclination, the correlation coeffi­ positive values for leaning forward and negative values for leaning
cient was negative (ρ¼-0.24, p ¼ 0.01). backward. For the purpose of this study, the absolute values of the de­
viation from vertical position were used in the analysis. In addition,
4. Discussion large angles or too many backward inclinations were possible in­
dications of inadequate display of information, which means flawed
4.1. Physiological measures design with information displayed too small, too high, or too far away
from the corresponding actuators. Evaluators of the ISV test should pay
The results showed that the measures of heart rate (heart rate dif­ attention to such behaviors.
ference and heart rate variability) and respiration (respiration rate and
breathing wave amplitude) could be good indicators of the workload 4.3. Estimated workload
change. As the workload increased, the heart rates of the crew members
also increased with dropping variations in R to R intervals. This finding The task complexity measure applied in this study was an effective
was in consensus with a previous study on cardiovascular measures as task analysis method in estimating workload (Aldrich et al., 1989; Liu
indices for mental effort during simulated flight (De Rivecourt et al., et al., 2012; McCracken and Aldrich, 1984; Wickens et al., 2015; Xu
2008). For the respiration measures, breathing rates increased with et al., 2009). The description of the ISV scenario and the statistically
workload increase, while the breathing wave amplitude decreased. It significant correlations found in the analysis could be considered as
indicated faster and shallower breathings as workload increased. In confirmation for its validity. In this study, task units were considered as
addition, after excluding the effects of workload on the four physio­ basic units for calculating physiological and activity measures. For each
logical measures (linear relationships assumed), no significant trends task unit, estimated workload was calculated by dividing the complexity
could be detected for the residuals along the timeline. Therefore, no score with the unit time span. It provided a reasonable way in predicting
evidence of fatigue was discovered during this 1-h ISV test. workload change along the timeline. For monitoring and cognitive tasks
that lasted for a period of time, there was a possibility of under­
estimating the related workload. Since the durations of task units in this
Table 3
study were comparatively short (below 5 min), such influence on esti­
Correlations between estimated workload and physiological & activity
mated workload was limited.
measures.
However, this objective method was not the same as subjective
Mean SD Spearman’s ρ p multi-dimension rating scales that included mental demand, physical
Heart rate difference (beats/min) 14.37 5.84 0.40 <0.001a demand, temporal demand, performance, effort, and frustration. A post-
Heart rate variability (ms) 45.91 8.98 0.42 <0.001a test survey with the crew members revealed subjective workload
Respiration rate (breaths/min) 18.71 2.10 0.27 0.01a
measured by NASA-TLX ranking from high to low in the order of coor­
Breathing wave amplitude (V) 15747 4403 0.46 <0.001a
Number of walking steps 26.37 22.59 0.24 0.01a dinator, safety engineer, second loop operator, and first loop operator.
Peak acceleration (g) 0.14 0.06 0.03 0.72 This result was contrary to estimated workload ranking in the order of
Activity level (g) 0.10 0.06 0.48 <0.001a first loop operator, second loop operator, coordinator, and safety engi­
Inclination (� ) 11.93 5.89 0.24 0.01a neer. Apart from individual differences in rating, coordinator and safety
Note. engineer as supervisors of the procedure execution process had higher
a
Significant at the level of 0.05. self-evaluation for the task demands. They also subjectively believed

4
Y. Gan et al.
Fig 2. Trend curves of physiological measures against estimated workload for the first loop operator.
that they worked really hard, although the objective physiological and
activity measures indicated otherwise. In other words, duties of the
coordinator and safety engineer might be the reason for higher ratings of
subjective workload. The subjective ratings of workload could be biased
in this case.
In the context of accident handling in the main control room of a
nuclear power plant, mental workload is considered to have stronger
influence than physical workload. It is part of the reason why this study
applied task complexity method based on VACP rating scales to estimate
workload. However, physical activities are essential for implementing
procedures using traditional hardware panels, as was observed during
the test. In this case, operations with high mental workload also required
more physical activities. Therefore, the physical and mental workload
could not be easily separated in this scenario. In addition, during the
post-test interview, the participants were asked to sort the six di­
mensions of NASA-TLX according to their contributions to the total
workload. All participants put “mental demand” ahead of “physical
demand”. It is consistent with the assumption that the influence of
physical workload on performance is limited in this test.
4.4. Intrusiveness of measures
In this study, the physiological and activity sensors were placed on a
strap around the operators’ chests. Prior to the ISV test, the position and
tension of the strap were adjusted for good recording performance and
the participants’ comfort. Interviews after the test showed that operators
did not feel the existence of BioHarness™ garment once the test had
begun. Thus, the intrusiveness of the measures was acceptable.
5. Conclusions
In this study, continuously recorded physiological and activity
measures during an ISV test were compared with the estimated work­
load calculated by the TC method. The ISV test of a beyond design-basis
accident was conducted on a full-scale simulator by licensed operators.
The conditions were almost the same as in the real control room of a
nuclear power plant, except for the pressure imposed on the operating
crew. The physiological and activity measures of heart rate difference,
heart rate variability, respiration rate, breathing wave amplitude,
number of walking steps, peak acceleration, activity level, and inclina­
tion were recorded. No significant effect of fatigue was detected in this
study. Major findings are as followed:
5
International Journal of Industrial Ergonomics 78 (2020) 102974
1) Heart rate difference was positively correlated with estimated
workload, while heart rate variability was negatively correlated with
estimated workload. They were sensitive to workload change.
2) Respiration rate was positively correlated with estimated workload,
while breathing wave amplitude was negatively correlated with
estimated workload. They were sensitive to workload change.
3) For operations using traditional panels, high workload was accom­
panied by larger number of walking steps, higher activity level, and
smaller angles of inclination.
Suggestions can be provided to evaluators of the ISV test that moving
and leaning activities of operators are good indicators of workload for
operations using traditional panels. Continuous monitoring of cardio­
vascular and respiration measures can detect workload peaks during the
test.
As this ISV test was conducted using traditional hardware panels, the
task types were different from those using the digital platform. The latter
had different ways of interaction between operators and the system (Liu
and Li, 2016a, 2016b). Furthermore, increasing automation level of the
digital operating platform would increase the percentage of cognitive
tasks. Tasks using digital operating platform should be considered in
further research on workload measurement in ISV test.
As for the physiological measures selected, eye activity was not
included. Although eye activity measures such as pupil size, blink rate,
and blink duration were proved to be correlated with workload, espe­
cially cognitive workload in many studies (Gao et al., 2013; Hwang
et al., 2008; Marquart et al., 2015; Siegle et al., 2008; Yan et al., 2019;
Van Acker et al., 2020), for the procedure execution using traditional
panels in this study, eye response measures were not adopted because of
the relatively high intrusiveness and low precision in measurement
when the participants were moving. For further studies using digital
operating platform, eye activity should be considerable.
Since this study was not a controlled experiment, causal relation­
ships could not be obtained from the analysis. Possible influence of
underlying factors on workload and physiological measures, as well as
their interaction effects should be explored with more carefully designed
experiments.
Unlisted references
Van Acker et al., 2020;.
Y. Gan et al.
Declaration of competing interest
The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influence
the work reported in this paper.
CRediT authorship contribution statement
Yu Gan: Investigation, Formal analysis, Writing - original draft.
Xiaolu Dong: Validation, Writing - original draft. Yijing Zhang:
Conceptualization, Methodology, Project administration. Xuegang
Zhang: Resources, Investigation. Ming Jia: Resources, Investigation,
Project administration. Zhaopeng Liu: Resources, Investigation. Zhiz­
hong Li: Conceptualization, Methodology, Supervision.
Acknowledgments
This study was partly supported by China Nuclear Power Engineer­
ing Co., Ltd (CNPEC) Science and Technology Innovation Project
(Project No. K-A2017.401).
References
Ahlstrom, U., Friedman-Berg, F.J., 2006. Using eye movement activity as a correlate of
cognitive workload. Int. J. Ind. Ergon. 36, 623–636. https://doi.org/10.1016/J.
ERGON.2006.04.002.
Aldrich, T.B., Szabo, S.M., Bierbaum, C.R., 1989. The development and application of
models to predict operator workload during system design. In: McMillan, G.R.,
Beevis, D., Salas, E., Strub, M.H., Sutton, R., Van, L.B. (Eds.), Applications of Human
Performance Models to System Design. Springer, Boston, MA.
Braarud, P.Ø., 2020. An efficient screening technique for acceptable mental workload
based on the NASA Task Load Index—development and application to control room
validation. Int. J. Ind. Ergon. 76, 102904. https://doi.org/10.1016/J.
ERGON.2019.102904.
Charles, R.L., Nixon, J., 2019. Measuring mental workload using physiological measures:
a systematic review. Appl. Ergon. 74, 221–232. https://doi.org/10.1016/J.
APERGO.2018.08.028.
De Rivecourt, M., Kuperus, M.N., Post, W.J., Mulder, L.J.M., 2008. Cardiovascular and
eye activity measures as indices for momentary changes in mental effort during
simulated flight. Ergonomics 51, 1295–1319. https://doi.org/10.1080/
00140130802120267.
Gao, Q., Wang, Y., Song, F., Li, Z., Dong, X., 2013. Mental workload measurement for
emergency operating procedures in digital nuclear power plants. Ergonomics 56,
1070–1085. https://doi.org/10.1080/00140139.2013.790483.
Gawron, V.J., 2008. Chapter 3 human workload. In: Human Performance, Workload, and
Situational Awareness Measures Handbook. CRC Press, Boca Raton.
Grier, R.A., 2015. How high is high? A meta-analysis of NASA-TLX global workload
scores. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 59, 1727–1731. https://doi.org/
10.1177/1541931215591373.
Hart, S.G., 2006. NASA-task load index (NASA-TLX): 20 years later. In: Proceedings of
the Human Factors and Ergonomics Society Annual Meeting. Sage Publications Sage
CA, Los Angeles, CA, pp. 904–908. https://doi.org/10.1177/154193120605000909.
Hart, S.G., Staveland, L.E., 1988. Development of NASA-TLX (task load index): results of
empirical and theoretical research. Advances in Psychology. Elsevier, pp. 139–183.
https://doi.org/10.1016/S0166-4115(08)62386-9.
Hwang, S.-L., Yau, Y.-J., Lin, Y.-T., Chen, J.-H., Huang, T.-H., Yenn, T.-C., Hsu, C.-C.,
2008. Predicting work performance in nuclear power plants. Saf. Sci. 46,
1115–1124. https://doi.org/10.1016/j.ssci.2007.06.005.
IEC, 1995. IEC61771-1995: nuclear power plants - main control-room - verification and
validation of design. International Electrotechnical Commission.
IEC, 2000. IEC61839-2000: nuclear power plants - design of control rooms - functional
analysis and assignment. International Electrotechnical Commission.
IEC, 2018. IEC60964-2018: nuclear power plants - control rooms - Design. International
Electrotechnical Commission.
International Journal of Industrial Ergonomics 78 (2020) 102974
Kramer, A.F., 1991. Physiological metrics of mental workload: a review of recent
progress. Mult. Perform. 279–328.
Liang, S.-F.M., Rau, C.-L., Tsai, P.-F., Chen, W.-S., 2014. Validation of a task demand
measure for predicting mental workloads of physical therapists. Int. J. Ind. Ergon.
44, 747–752. https://doi.org/10.1016/J.ERGON.2014.08.002.
Liu, P., Li, Z., 2016a. Comparison between conventional and digital nuclear power plant
main control rooms: a task complexity perspective, part I: overall results and
analysis. Int. J. Ind. Ergon. 51, 2–9. https://doi.org/10.1016/j.ergon.2014.06.006.
Liu, P., Li, Z., 2016b. Comparison between conventional and digital nuclear power plant
main control rooms: a task complexity perspective, Part II: detailed results and
analysis. Int. J. Ind. Ergon. 51, 10–20. https://doi.org/10.1016/j.
ergon.2014.06.011.
Liu, P., Li, Z.Z., Wang, Z., 2012. Task complexity measure for emergency operating
procedures based on resource requirements in human information processing. In:
Proceedings of the 11th International Probabilistic Safety Assessment and
Management Conference (PSAM11) and the Annual European Safety and Reliability
Conference (ESREL 2012). Finland, Helsinki.
Marquart, G., Cabrall, C., de Winter, J., 2015. Review of eye-related measures of drivers’
mental workload. Procedia Manuf 3, 2854–2861. https://doi.org/10.1016/j.
promfg.2015.07.783.
McCracken, J.H., Aldrich, T.B., 1984. Analyses of Selected LHX Mission Functions:
Implications for Operator Workload and System Automation Goals (No. ASI479-024-
84). ANACAPA Sci. INC FORT RUCKER AL.
O’Hara, J., Stubler, W., Higgins, J., Brown, W., 1997. NUREG/CR-6393 Integrated
System Validation: Methodology and Review Criteria. US Nuclear Regulatory
Commission, Washington, DC.
O’Hara, J.M., Higgins, J.C., Persensky, J.J., Lewis, P.M., Bongarra, J.P., 2004. NUREG-
0711 Human Factors Engineering Program Review Model. U.S. Nuclear Regulatory
Commission, Washington, DC.
Robert, G., Hockey, J., 1997. Compensatory control in the regulation of human
performance under stress and high workload: a cognitive-energetical framework.
Biol. Psychol. 45, 73–93. https://doi.org/10.1016/S0301-0511(96)05223-4.
Rusnock, C.F., Borghetti, B.J., 2018. Workload profiles: a continuous measure of mental
workload. Int. J. Ind. Ergon. 63, 49–64. https://doi.org/10.1016/J.
ERGON.2016.09.003.
Ryu, K., Myung, R., 2005. Evaluation of mental workload with a combined measure
based on physiological indices during a dual task of tracking and mental arithmetic.
Int. J. Ind. Ergon. 35, 991–1009. https://doi.org/10.1016/J.ERGON.2005.04.005.
Shakouri, M., Ikuma, L.H., Aghazadeh, F., Nahmens, I., 2018. Analysis of the sensitivity
of heart rate variability and subjective workload measures in a driving simulator: the
case of highway work zones. Int. J. Ind. Ergon. 66, 136–145. https://doi.org/
10.1016/J.ERGON.2018.02.015.
Siegle, G.J., Ichikawa, N., Steinhauer, S., 2008. Blink before and after you think: Blinks
occur prior to and following cognitive load indexed by pupillary responses.
Psychophysiology 45, 679–687. https://doi.org/10.1111/j.1469-8986.2008.00681.
x.
Van Acker, B.B., Bombeke, K., Durnez, W., Parmentier, D.D., Mateus, J.C., Biondi, A.,
Saldien, J., Vlerick, P., 2020. Mobile pupillometry in manual assembly: a pilot study
exploring the wearability and external validity of a renowned mental workload lab
measure. Int. J. Ind. Ergon. 75, 102891. https://doi.org/10.1016/J.
ERGON.2019.102891.
Veltman, J.A., Gaillard, A.W.K., 1998. Physiological workload reactions to increasing
levels of task difficulty. Ergonomics 41, 656–669. https://doi.org/10.1080/
001401398186829.
Wickens, C.D., 2002. Multiple resources and performance prediction. Theor. Issues
Ergon. Sci. 3, 159–177. https://doi.org/10.1080/14639220210123806.
Wickens, C.D., Hollands, J.G., Banbury, S., Parasuraman, R., 2015. Engineering
Psychology and Human Performance. Psychology Press.
Wierwille, W.W., Eggemeier, F.T., 1993. Recommendations for mental workload
measurement in a test and evaluation environment. Hum. Factors 35, 263–281.
https://doi.org/10.1177/001872089303500205.
Xu, S., Li, Z., Song, F., Luo, W., Zhao, Q., Salvendy, G., 2009. Influence of step complexity
and presentation style on step performance of computerized emergency operating
procedures. Reliab. Eng. Syst. Saf. 94, 670–674. https://doi.org/10.1016/j.
ress.2008.07.001.
Yan, S., Wei, Y., Tran, C.C., 2019. Evaluation and prediction mental workload in user
interface of maritime operations using eye response. Int. J. Ind. Ergon. 71, 117–127.
https://doi.org/10.1016/J.ERGON.2019.03.002.
Young, M.S., Brookhuis, K.A., Wickens, C.D., Hancock, P.A., 2015. State of science:
mental workload in ergonomics. Ergonomics 58, 1–17. https://doi.org/10.1080/
00140139.2014.956151.