Safety Science 47 (2009) 425–435

Contents lists available at ScienceDirect

Safety Science
journal homepage: www.elsevier.com/locate/ssci

A real-time warning model for teamwork performance and system safety

in nuclear power plants
Sheue-Ling Hwang a,*, Guo-Feng Liang a, Jhih-Tsong Lin a, Yi-Jan Yau a, Tzu-Chung Yenn b,1,
Chong-Cheng Hsu b,1, Chang-Fu Chuang c,2
a
Department of Industrial Engineering and Engineering Management, National Tsing Hua University, 101, Section 2, Kuang-Fu Road, Hsinchu 300, Taiwan
b
Institute of Nuclear Energy Research, Atomic Energy Council, Executive Yuan No. 1000, Wunhua Road, Jiaan Village, Longtan Township, Taoyuan County 325, Taiwan
c
Atomic Energy Council, 6F, No. 80, Section 1, Cheng Kung Road, Yung-Ho City, Taipei Country 234, Taiwan

a r t i c l e i n f o a b s t r a c t

Article history: In order to increase system safety and team performance, this study aimed to develop a real-time warn-
Received 6 November 2007 ing model (RTWM) by assessing team response time, error rates, and mental workload. Toward this goal,
Received in revised form 26 April 2008 the group method of data handling (GMDH) algorithm was applied to physiological indices to predict
Accepted 14 July 2008
team performance. Then fuzzy logic, fuzzy inference and linguistic variable sets representing the Team
Performance and Safety Index were applied to construct the RTWM. To model the RTWM, experiments
were conducted on computer-supported cooperative work (CSCW) in the personal computer transient
Keywords:
analyzer (PCTRAN) simulator. The simulator and teamwork are designed to simulate the real tasks of
Real-time warning model (RTWM)
Teamwork
the control room of the Fourth Nuclear Power Plant (FNPP) in Taiwan. In addition, important physiolog-
NPPs ical parameters, the NASA-TLX questionnaire, team response time, and team error rates were collected
Mental Workload from 39 participants. The results revealed that there was a significant positive correlation between the
GMDH error rates of teamwork and the interval of event arrival time. This indicated that a pre-alarm device is
Fuzzy logic necessary because vigilance decreased with time. Moreover, a predictive teamwork performance model
applying the GMDH algorithm and the RTWM with a fuzzy inference system was developed in this study.
The proposed model can efficiently predict teamwork performance to maintain appropriate mental work-
load as well as ensure system safety.
Ó 2008 Elsevier Ltd. All rights reserved.

1. Introduction
Recently, there has been increased interest in the study of team
collaboration. In particular, one of the most popular topic areas in
teamwork research has focused on computer-supported coopera-
tive work (CSCW) which provides an automated system and shared
environment to support teamwork. Team members have to moni-
tor tasks and share responsibility. However, some key problems in
such an environment need to be resolved, such as (1) how to main-
tain high team performance, and (2) how to ensure human and
system safety when sharing responsibility in monitoring tasks.
Numerous studies have noted that team performance measure-
ment is more complicated than that of individual performance be-
cause teamwork relies heavily on communication, supervision of a
common situation, and the sharing of the mental workload (Carv-
* Corresponding author. Tel.: +886 3 5742694; fax: +886 3 5722685.
E-mail addresses: slhwang@ie.nthu.edu.tw (S.-L. Hwang), d947803@oz.nthu.
edu.tw (G.-F. Liang), d947808@oz.nthu.edu.tw (J.-T. Lin), d927814@oz.nthu.edu.tw
(Y.-J. Yau), tcyenn@iner.gov.tw (T.-C. Yenn), worm@iner.gov.tw (C.-C. Hsu),
chuang@aec.gov.tw (C.-F. Chuang).
1
Tel.: +886 3 4711400; fax: +886 3 4712358.
2
Tel.: +886 2 22322118; fax: +886 2 22322113.
alho and Vidal, 2007; Sebok, 2000). Various methods can be taken
to measure team performance, and most of these methods focus on
human reliability analysis (HRA), cognition, team interactions, and
objective performance (Artman, 2000; Sebok, 2000; Shu and
Furuta, 2005; Marseguerra and Zio, 2006).
In addition, some studies have revealed that mental workload is
related to human performance (Xie and Salvendy, 2000; DiDome-
nico and Nussbaum, 2005; Desmond and Hoyes, 1996). Hollnagel
(2003) indicated that cognitive workload is regarded as that subset
of mental workload which requires conscious effort (e.g., recogniz-
ing familiar objects or driving a car). Moray (1988) suggested that
the appropriate mental workload could reduce human errors and
enhance system safety. Rubio and Diaz (2004) pointed out that
proper mental workload ensured the safety and long-term produc-
tion efficiency of operators. Greef et al. (2007) reviewed the symbi-
otic relation between man and machine which aimed to
compensate for temporal limitations in human information pro-
cessing, overload, cognitive lockup, and underload by a combina-
tion of performance, effort, and task information.
In the nuclear power plant control room, team members have
been assigned roles, responsibility, and areas of specialization but
were often cross-trained to be able to deal with operating and

0925-7535/$ - see front matter Ó 2008 Elsevier Ltd. All rights reserved.
doi:10.1016/j.ssci.2008.07.011
426 S.-L. Hwang et al. / Safety Science 47 (2009) 425–435
monitoring tasks. The sharing of responsibility, also termed diffu-
sion of responsibility, with a computer for system monitoring
and decision making tasks may achieve shared team workload
and shared team mental models (Skitka et al., 2000; Rentsch and
Klimoski, 2001). However, diffusion of responsibility may reduce
the effort and workload of group members who have capable part-
ners free-riding on their efforts (Mclntyre et al., 2003). For such
team members with diffusion of responsibility, the underload of
each team member may influence the overall team performance
especially in monitoring tasks.
Several methods have been developed to measure mental work-
load. These methods are classified into three categories: perfor-
mance-based measures, subjective measures (e.g., NASA-Task
Load Index), and physiological measures (e.g., eye blink, respira-
tion, electroencephalogram (EEG), heart rate (HR), heart rate vari-
ability (HRV)) (Luximon and Goonetilleke, 2001; Braarud, 2001;
Laine and Bauer, 2002; Wilson and Russell, 2003; Rubio and Diaz,
2004). Among these methods, heart rate variability can provide the
clearest evidence and can provide real-time support for the use of
physiological activity in a test (Kamada et al., 1992a,b; Kageyama
and Kabuto, 1995; Nishikido et al., 1997; Scerbo and Freeman,
2001; Mikulka et al., 2002; Stanton et al., 2005). Tattersal and
Hockey (1995) showed that the heart rate increased and heart rate
variability spectrum (HRVs) decreased in flight engineers when
given three levels of cognitive task demands during the flight tasks
(i.e., system monitoring, routine fault correction, and problem solv-
ing). In addition, Sammer (1998) compared a physical task, a cog-
nitive task, and a combination of both task with computing the
heart period (IBI) and heart rate variability (HRV) in the low
(0.01–0.05 Hz), mid (0.06–0.16 Hz), and high (0.2–0.4 Hz) range.
The results showed that the HRV was less for the dual task and
greater for the physical and cognitive tasks. Scerbo and Freeman
(2001) reported that the heart rate (HR) increases and heart rate
variability (HRV) decreases with increased mental workload, and
one of the advantages with using HRV as a measure is the capabil-
ity to have continuous, on-line recordings. Therefore, heart rate
variability is suitable for constructing a real-time predictive model
to ensure both system and human safety.
Safety is an important criterion during design and management
(Hale et al., 1997, 2007; Schupp et al., 2006). According to previous
studies, team model, team performance measure methods, and
mental workload are proposed to ensure safety; however, these re-
searches have focused on concepts, and very few studies have been
conducted on the predicting performance and deciding the safety
threshold in practical work. The purpose of this study was (1) to
design a predictive teamwork performance model using the group
method of data handling (GMDH) algorithm, and (2) to determine
the safety threshold by fuzzy logic when team members were
underloaded mentally.
The group method of data handling (GMDH) algorithm has been
widely applied in various fields (e.g., education, economic systems,
weather modeling, manufacturing, pattern recognition, physiolog-
ical experiment (Baker and Richards, 1999; Pavel and Miroslav,
2003; Sarycheva, 2003; Kim et al., 2001; Ivakhnenko, 1993, 1995;
Hwang et al., 2008)). This paper extended the GMDH algorithm to
develop the proposed team performance predictive model.
Fuzzy logic is based on the fuzzy set theory which provides a
methodology that simulates human thinking by explicitly model-
ing and managing the linguistic imprecision and uncertain (Zadeh,
1965). Currently, it has been used for modeling in many fields
including workload assessment, intelligent patient monitoring
and alarm systems, human–machine systems, cognitive workload,
electrocardiograms and electroencephalogram signal design, and
automated assistance system (Becker et al., 1997; Ntuen, 1999;
Moon et al., 2002; Liu and Su, 2006; Gregoriades and Sutcliffe,
2006; Rani et al., 2007; Yang et al., 2008). In this study, fuzzy logic
was used to determine the linguistic index, performance, and then
define the linguistic threshold and Safety Index to avoid underload
from the diffusion of responsibility in monitoring tasks.
Therefore, firstly the methodology of GMDH and fuzzy logic are
described in Sections 2 and 3. The details of the experimental
methods and design are presented in Section 4. The team perfor-
mance prediction model and the threshold of Safety Index using
GMDH and fuzzy logic are described in Section 5. Section 6 dis-
cusses the findings and limitations of the proposed models in
teamwork. Finally, Section 7 provides some comments about this
study.
2. The group method of data handling (GMDH)
The GMDH algorithm, a self-organizing approach, was a pio-
neering proposition by Ivakhnenko (1968) for identifying the best
predicting polynomial equation. The algorithm found the only
optimal model by fully sorting out model-candidates and evaluat-
ing their operation by using external criteria of accuracy or differ-
ence types (Madala and Ivakhnenko, 1994). Recently, genetic
algorithms and neural network methods have been combined into
GMDH polynomials, and some software packages have been devel-
oped by the Glushkov Institute of Cybernetics and Algorithm for
Synthesis of Polynomial Networks (ASPN) such as NeueroShell2
and ModelQuest (WSG, 1995).
GMDH nets derive a mathematical formula which is a nonlinear
polynomial expression relating the values of the most important
inputs to predict the output variable. The general connection be-
tween input and output variables can be expressed by the Volterra
functional series, the discrete analogue of which is the Kolmogo-
rov–Gabor polynomial (Madala and Ivakhnenko, 1994):
X
M X
M X
M X
M X
M X
M
y ¼ a0 þ ai xi þ aij xi xj þ aijk xi xj xk ð1Þ
i¼1 i¼1 j¼1 i¼1 j¼1 k¼1
where X(x1, x2, ... , xM) is the vector of inputs and A(a1, a2, ... , aM) is the
vector of coefficients or weights. The components of the input vec-
tor X can be independent variables, functional forms, or finite differ-
ence terms. Other non-linear reference functions such as difference,
logistic, and harmonic can be used for model construction.
In this study, a team performance prediction model using phys-
iological data as the input variable and using the NeueroShell2 tool
was established. Furthermore, the output values of this model
were transferred into the input values of the fuzzy interference
system as described in the next section.
3. Fuzzy Logic
Zadeh introduced the term fuzzy logic (FL) describing the math-
ematics of the fuzzy set theory in 1965. Fuzzy logic provides the
ability to mimic the human mind to employ modes of reasoning
that are approximate rather than exact. It can also address linguis-
tic imprecision and tolerance. A rule-based system, the fuzzy infer-
ence system (FIS) is one of the most famous applications of fuzzy
logic and the fuzzy sets theory. In this section, we shall briefly
introduce the fuzzy set theory and fuzzy inference system which
is applied in this paper.
3.1. Fuzzy sets and membership functions
The classical set theory has a crisp definition on whether an ele-
ment is a member of a set or not. However, a fuzzy set is an exten-
sion of a crisp set. Crisp sets allow only full membership or non
membership, whereas fuzzy sets allow partial membership in
which a degree of membership ranging between zero and one is as-
signed to each element. A fuzzy set A, symbol A;~ on a universe U is
 S.-L. Hwang et al. / Safety Science 47 (2009) 425–435
X
Fuzzification Inference
engine
Rule base
Fig. 1. Fuzzy inference system.
characterized by a membership function lA~ ðxÞ. Various types of
membership function are used, including the triangular type, trap-
ezoidal type, generalized bell shape, Gaussian curves, polynomial
curves, and sigmoid function (Klir and Yuan, 1995).
3.2. Fuzzy inference system
A fuzzy inference system (FIS) uses the fuzzy rule in a nonlinear
mapping of input linguistic variables into a scalar output. A model
of FIS contains five stages which are as follows (see Fig. 1): (1) fuzz-
ification, (2) inference engine, (3) rule base, (4) aggregation, and (5)
defuzzification (Mamdami and Assilina, 1975; Kruse et al., 1994;
Kulkarni, 2001).
Fuzzification is a process which takes input values and deter-
mines the degree to which they belong to each of the fuzzy sets
via membership functions. The inference engine determines the
degree to which the antecedent is satisfied for each rule. The rules
are obtained from expert knowledge, sample data points, or train-
ing data (Wang and Mendel, 1992; Dagdeviren et al., 2008). In or-
der to get a crisp value for the output, we needed a defuzzification
process. Many defuzzification techniques are being proposed in
various studies including methods of center of gravity (COG), max-
imum decomposition, center of maximum, and the height method.
The most commonly used method is the center of gravity, which
determines the center of an area and uses this value as the output.
To model the fuzzy inference system, the input linguistic vari-
able and teamwork performance should be obtained. The team-
work performance can be predicted by the physiological data
from this study. Thus, an experiment was conducted to collect
the physiological data and teamwork performance data.
Procedure:
BP1~29
Fig. 2. Power generator control system (PGCS).
427
Y
Agregation Defuzzification
4. Method
4.1. Participants
Forty-five participants in fifteen teams took part in the experi-
ment. They included 28 students (average age = 26.68 years;
SD = 4.52 years) of the National Tsing Hua University, 14 experts
(average age = 45.86 years; SD = 12.46 years) with an average of
19.64 working years in the Institute of Nuclear Energy Research,
and three operators (average age = 39 years; SD = 1.73 years) with
an average of 13.67 working years at the Fourth Nuclear Power
Plant in Taiwan. Before running the experiment, six students in
two teams joined a preliminary experiment. The remaining 39 par-
ticipants attended the formal experiment.
4.2. Apparatus
The Cardio Tens portable device made by Meditech was used an
electrocardiogram (ECG). The ECG signals were calculated using
the software from cardio visions in order to transfer the signals
into heart rate variability (HRV) indices including time domain
and frequency domain. The personal computer transient analyzer
(PCTRAN) system including the power generator control system
(PGCS), the reactor recirculation system (RCIR), and the rod control
and information system (RCIS) were used to simulate the startup
reactor task. A program in the training simulator was designed to
represent the procedures of integrated operating procedure (IOP)
201.2, reactor startup with PGCS, and 202.2, power changes with
PGCS. In addition, operators executed the tasks by mouse following
the operating procedure.
Continuous
Bottom
428 S.-L. Hwang et al. / Safety Science 47 (2009) 425–435
Dryer
Wetwell
Fig. 3. Reactor parameters display.
4.3. Experimental tasks and variables
The team members organized by the reactor operator (RO),
assistant reactor operator (ARO), and supervisor were given differ-
ent tasks as part of the teamwork. The primary task of the team-
work was to start up the reactor safely in the PCTRAN system
provided by the Institute of Nuclear Energy Research (INER). More-
over, the seven setting points and 29 procedures from break-point
1 (BP1) to break-point 29 (BP29) were established in the PGCS of
PCTRAN. In the teamwork, the tasks of the reactor operator were
to continuous push the button in PGCS (see Fig. 2) in order to carry
out 29 procedures, and input the target value of the core flow and
power value at each setting point by the standard operation proce-
dure (SOP); meanwhile, the tasks of the assistant reactor operator
were to monitor the parameters of the core, the vessel, dryer, and
wetwell, and to judge if the parameters were under control (see
Fig. 3). Afterwards, the assistant reactor operator wrote down the
parameters of the water level, temperature, pressure, core flow,
and power value in the standard operation procedure (SOP) and re-
layed these values to the supervisor when the core flow or power
PGCS interface Core flow and
Power interface
RCIS
interface
RCIR
interface
Fig. 4. Interface layouts of teamwork.
Core
Vessel
value had reached each setting point. Similarly, the supervisor
supervises the reactor operator and assistant reactor operator in
performing the procedures correctly as stated in the standard oper-
ation procedure (SOP). The supervisor recorded all of these values
at the same time.
The team members’ position and the interface layout in this
experiment are shown in Fig. 4. The participants in this experiment
as shown from left to right were the reactor operator, the supervi-
sor, and the assistant reactor operator, respectively. The devices
using in this experiment consisted of seven monitors, two comput-
ers, two mice, and two keyboards.
The independent variable in this experiment was the interval of
event arrival time, and the dependent variables were physiological
indices (HRV indices), teamwork performance (response time and
correct rates), and mental workload which are described as
follows.
The PCTRAN system provided the system default value and
interval time; for example, (1)–(2) means the interval of the arrival
time between event 1 and event 2 in the scenario.
Target value Alarm items
requiring interface interface
Turbine system
interface
Alarm priority
list interface
 S.-L. Hwang et al. / Safety Science 47 (2009) 425–435
The heart rate variability (HRV) indices were calculated from
short-term (5 min) and long-term (24 h) recordings from the elec-
trocardiogram (ECG) such as time-domain and frequency-domain
analysis presented as follows (McNames et al., 2003):
(1) Time-domain metrics
(1) NN number (Num): number of normal-to-normal (NN)
intervals.
(2) NN average (ms): average on NN intervals in ms.
(3) SDNN (ms): the standard deviation of the NN intervals.
(4) pNN50 (%): number of NN interval pairs with over
50 m s difference.
(5) HRVti: HRV triangular index which is a measure of the
shape of the NN interval distribution. Uniform distribu-
tions representing large variability have large values
and distributions, while single large peaks have small
values.
(6) RMSSD: the root mean square of successive NN interval
differences.
(2) Frequency-domain metrics
(1) LF: the low frequency (LF) power which was calculated
as the total signal power in the frequency range of
0.04–0.15 Hz.
(2) HF: the high frequency (HF) power which was calcu-
lated as the total signal power in the frequency range
of 0.15–0.40 Hz.
(3) LF/HF: the low frequency–high frequency ratio.
(4) TP: the total power which was the integral of the power
spectral density (PSD) estimated over the full frequency
range of 0.0–1.5 Hz.
In this experiment, the teamwork performance of the primary
task provided data of error rates (or correct rates) and the response
time of the teams. Furthermore, the subjective mental workload
was evaluated using the questionnaire NASA task load index
(TLX) at the end of the experiment.
4.4. Experimental procedure
There were several stages before the experiment. First, each
participant wore the Cardio Tens portable device and to check
whether the electrode was regular or not. The default settings on
the device began to collect the heart rate variability data after
10 min of wearing the apparatus. The experimental tasks for the
teamwork were enumerated, and each participant was guided in
reading the standard operation procedure (SOP) and took the
NASA-TLX questionnaire. They were asked to sign an informed
consent once the procedure was explained to them. Afterwards,
the initial heart rate variability (HRV) indices were automatically
measured around 5 min as a base line before the experiment. At
the beginning of the experiment, each team was given 10 min to
familiarize themselves and to practice the control procedure
according to the standard operation procedure (SOP) and the
instructions. Then the team members implemented and monitored
the startup reactor including tasks such as communicating, input-
ting the target value, recording the vital parameters, and so on.
Moreover, the HRV indices were recorded continuously by the de-
vice until the startup reactor finished which was after approxi-
mately 40 min. Finally, the NASA-TLX questionnaire was filled
out by the team member to evaluate the subjective mental work-
load after the experiment.
5. Results
In developing a real-time warning model (RTWM) firstly, in Sec-
tion 5.1 we surveyed the team members’ mental workload in the
429
monitoring tasks of this experiment. In addition, the correlation
between the interval of event arrival time and the error rates of
the 13 teams were analyzed in Section 5.2. Then, the GMDH with
the neruoshell2 tool and fuzzy logic theory were applied. The pro-
cess of developing the RTWM contains two models as shown in
Sections 5.3 and 5.4. Finally, the two models were validated in Sec-
tion 5.5.
5.1. Subjective questionnaire assessment of mental workload
The NASA-TLX was used as a subjective workload measure. The
TLX has six subscales: mental demand, physical demand, temporal
demand, performance, effort, and frustration. To investigate the
mental workload of teamwork undertaken in this experiment,
the questionnaire integrated NASA-TLX with five-point Likert
scales (strongly low, low, middle, high, and strongly high) and fuz-
zy calculation. The fuzzy scale of each band of the five-point scale
was calculated from the responses of 39 participants in 13 teams.
Each team included a reactor operator (RO), an assistant reactor
operator (ARO), and a supervisor. The mental workload scores
(mean and standard deviation) of the reactor operator, the assis-
tant reactor operator, and the supervisor were calculated as
wR = 37.96 ± 14.29, wA = 39.37 ± 10.08, and wS = 35.65 ± 16.46,
respectively. The fuzzy membership was obtained as follows:
RO : fwR ; li ðwR Þg ¼fðl2 ðwR Þ; 0:6Þ; ðl3 ðwR Þ; 0:66Þ;
ðl4 ðwR Þ; 0:13Þg ð2Þ
ARO : fwA ; li ðwA Þg ¼fðl2 ðwA Þ; 0:52Þ; ðl3 ðwA Þ; 0:7Þ;
ðl4 ðwA Þ; 0:16Þg ð3Þ
Supervisor : fwS ; li ðwS Þg ¼ fðl2 ðwS Þ; 0:71Þ; ðl3 ðwS Þ; 0:58Þ;
ðl4 ðwS Þ; 0:02Þg ð4Þ
where the fuzzy membership of wR, wA and wS is represented as
li(wR), li(wA),and li(wS), i = 1, 2, 3, 4, and 5 which means the degree
of membership of strongly low, low, middle, high, and strongly
high.
In Eqs. (2)–(4) the mental workload of the reactor operator,
assistant reactor operator, and supervisor belongs to low and mid-
dle in the PGCS startup reactor and monitoring tasks. This implies
that all participants engaged in this semi-automated teamwork
task perceived a low to middle mental workload.
5.2. Correlation between the interval of event arrival time and error
rates
The person correlation analysis was used to examine the rela-
tionship between the interval of the event arrival time and error
rates of the 13 teams using statistical products and services solu-
tion (SPSS). Table 1 indicated that the interval of event arrival time
and error rates were positively correlated with each other. The cor-
relation coefficient of 0.966 was found to be statistically significant
(p < 0.01, two-tailed). In other words, the longer the interval of
event arrival time, the higher the error rates of the 13 teams
(Fig. 5). Therefore, a teamwork performance prediction model
and warning model are deemed necessary to increase the vigilance
of team members especially during a longer event arrival time.
5.3. Teamwork performance prediction model
The first model was used to establish the relationship between
the physiological indices and correction rate/response time (C/R ra-
tio). Thirty sets of data from 10 teams were used to construct the
model while nine sets of data from three teams were used to val-
idate this model using NeuroShell software 2 with the GMDH algo-
rithm method. The predictors of the eight heart rate variability
430 S.-L. Hwang et al. / Safety Science 47 (2009) 425–435

Table 1 1
Pearson Correlation analysis between error rates and interval of event arrival time

Correlations Interval of the event Error rates made 0.8

arrival time of 13 teams
0.6
0.966a

value
Interval of the event Pearson 1
arrival time Correlation
Significance – 0.001 0.4
(two-tailed)
R squared
N 6 6 0.2
Corr. coeff.
Error rates made of Pearson 0.966a 1
13 teams Correlation 0
Y1 Y2 Y3 Y4 Y5 Y6 Y7
Significance 0.001 –
(two-tailed) C/R Rate
N 6 6
Fig. 6. Sensitivity test in performance index.
a
Correlation is significant at the 0.01 level (two-tailed).

pffiffiffiffiffi
0.25 on y2 has the highest correlation coefficient (0.91) and R square
(3)-(4)
Error rates made of 13 teams (%)

(4)-(5) value (0.82), and all individual variables except for the standard
0.20 deviation of the normal-to-normal intervals (SDNN) are significant.
Furthermore, the parameter variation values of the predictive
(6)-(7) model were scattered in Fig. 6 where the performance index Y3 is
0.15
(5)-(6)
better than the others. Thus, given the values for X1, X2, X4, X5, X6,
0.10 X7, and X8, a work performance predictive model can be expressed
by
(2)-(3)
0.05
(1)-(2)
Error rates (%) Y ¼ 0:68 þ 0:99X 5 þ 0:24X 8  4:7X 2 þ 0:67X 4  19X 21
0.00
0 50 100 150 200  0:15X 27  0:51X 34  5:1X 1 X 4  6:6X 1 X 7 þ 0:47X 4 X 7
Interval of event arrival time (seconds)
þ 15X 1 X 4 X 7 þ 5X 26 þ 2:20:002 X 32  8:9X 36  2:6X 25  1:1X 28
Fig. 5. Relationship between interval of the event arrival time and error rates.
þ 3:4X 5 X 8  8X 22 ð7Þ

(HRV) indices collected from the Cardio Tens as physiological indi- To diagnose the team Safety Index, the prediction performance
ces including NN count (X1), NN average (X2), SDNN (X3), pNN50 value of team members found using Eq. (7) could be used as an in-
(X4), HRVti (X5), RMSSD (X6), LF/HF (X7), and TP (X8) were adopted put into the fuzzy inference system.
to predict the performance index from the algorithm. To reduce
individual differences and eliminate the unit from each index, all 5.4. Teamwork real-time warning model (RTWM)
of the predictors (Xi) were transferred from:
The purpose of the second model is to maintain high team per-
ðX iA  X iB Þ=X iA ¼ X i ð5Þ formance using a RTWM. To construct the RTWM, five steps could
be conducted as follows:
where XiA is the value during the experiment and XiB is the value be-
fore the experiment. By Eq. (5), the range of the Xi scale is from 1 Step 1: Draw the correct rate/response time (C/R) value to realize
to 1. In addition, the performance index (Y, %) is combined with the the data scatter. From the practical value of this experi-
correct rate (y1) and the response time (y2) of teamwork at the same ment, the performance index of C/R in 10 teams were
time, which can be represented as sorted from low to high as shown in Fig. 7 whereas the
rest of the three teams remained to validate the model.
pffiffiffiffiffi
½ðy1 Þ3 = y2   10 ¼ Y ð6Þ The meaning of each dot represents the number of partic-
ipants whose C/R was the same value.
pffiffiffiffiffi
where (y1)3 and y2 are simulated from the GMDH algorithm using Step 2: Determine the membership function and fuzzy numbers
the neruoshell2 tool. The concept of simulation is similar to sensitiv- of performance indices. According to the scatters in
ity analysis which investigates the effect on the optimal solution if Fig. 7, three fuzzy numbers and membership function
the parameters take on other possible values. The results in Table 2 (l~j ðY i Þ; i = R, A, S; j = low, middle, high) were modeled as
reveal that taking the correct rate on (y1)3 and a fixed response time Eqs. (8)–(10).

Table 2
Sensitivity analysis

Correct rate Response time C/R R squared Correlation coefficient Insignificant

pffiffiffiffiffi pffiffiffiffiffi
y1 y y1 = y2 ¼ Y 1 0.77 0.88 rMSSD
pffiffiffiffiffi2 pffiffiffiffiffi
y21 y y21 = y2 ¼ Y 2 0.80 0.89 HRVti
pffiffiffiffiffi2 pffiffiffiffiffi
y31 y y31 = y2 ¼ Y 3 0.82 0.91 SDNN
pffiffiffiffiffi2 pffiffiffiffiffi
y41 y y41 = y2 ¼ Y 4 0.67 0.82 rMSSD
pffiffiffiffiffi2 pffiffiffiffiffi
y51 y y51 = y2 ¼ Y 5 0.74 0.86 rMSSD
pffiffiffiffiffi2 pffiffiffiffiffi
y61 y y61 = y2 ¼ Y 6 0.73 0.85 SDNN
pffiffiffiffiffi2 pffiffiffiffiffi
y71 y2 y71 = y2 ¼ Y 7 0.74 0.86 SDNN
 S.-L. Hwang et al. / Safety Science 47 (2009) 425–435 431

4
number of participants

1
number of participants
0
0. 01
0. 03
0. 05
0. 07
0. 09
0. 11
0. 13
0. 15
0. 17
0. 19
0. 21
0. 23
0. 25
0. 27
0. 29
0. 31
0. 33
0. 35
0. 37
0. 39
0. 41
0. 43
0. 45
0. 47
0. 49
0. 51
0. 53
0. 55
0. 57
0. 59
0. 61
0. 63
0. 65
0. 67
0. 69
0. 71
0. 73
0. 75
0. 77
0. 79
0. 81
0. 83
0. 85
0. 87
0. 89
0. 91
0. 93
0. 95
0. 97
0. 99
C/R value

Fig. 7. C/R value of team members.

8
< 1;
> Y i < 0:26
llow ðY i Þ ¼ ð0:5  Y i Þ=0:24; 0:26 6 Y i < 0:5 ð8Þ
>
:
0; Y i P 0:5
8
< 0;
> Y i < 0:26 or Y i P 0:69
lmiddle ðY i Þ ¼ ðY i  0:26Þ=0:24; 0:26 6 Y i < 0:5 ð9Þ
>
:
ð0:69  Y i Þ=0:19; 0:5 6 Y i < 0:69
8
< 0;
> Y i < 0:5
lhigh ðY i Þ ¼ ðY i  0:5Þ=0:19; 0:5 6 Y i < 0:69 ð10Þ
>
:
1; Y i P 0:69
where l~j ðY i Þ is the membership function that the C/R ratio
of the reactor operator, the assistant reactor operator and
the supervisor belong to. Fig. 8. Example of fuzzy rule bases.

Step 3: Transfer the linguistic variable, which is the performance

index of the team members, as the input variable of the
fuzzy inference system from the crisp value into the fuzzy
set a process called fuzzification. To validate the proposed
system, the data from the 10 teams was adopted in ran-
dom, and then the rest of the three teams were retained.
For instance, in the first team, the C/R ratio of RO was 0.34,
implying that the degree of membership was
llow(0.34) = 0.66 and lmiddle(0.34) = 0.32. Similarly, the
C/R ratio of ARO is 0.54 lmiddle(0.54) = 0.79 and
lhigh(0.54) = 0.21), and that of the supervisor was 0.34
(llow(0.34) = 0.66 and lmiddle(0.34) = 0.32). The calcula-
tion for the rest of the nine teams is the same as this step.
Step 4: Construct the inference engine and rule base. The infer-
ence engine determines the degree to which the anteced-
ent is satisfied for each rule. The rule base is derived from
the combination of fuzzy sets and the fuzzy numbers of
lj(Yi), where i = R, A, S; j = low, middle, high, which has
3  3  3 = 27 different combinations. For example, in
the first team, the membership value was 0.66 for the
RO or the supervisor with low performance, and 0.32 with
middle performance. Similarly, the membership value
was 0.79 for the ARO with middle performance, and
0.21 with high performance. The rule base in this case
had 2  2  2 = 8 combinations as shown in Fig. 8.
In Fig. 8, the fuzzy set output is defined as the linguistic notion,
the team safety performance, described by three attributes: safety
(Green signal), attention (Blue signal), and danger (Red signal). Let
SI (Safety Index) represent the degree of team safety performance
and the range be [0, 10]. If SI is lower, it means that the team per-
formance is poorer and the warning condition is more critical. To
ensure that all of the team members are in suitable performance
level, when the premise of each team member falls into the low
threshold, the conclusion would be in the Red of the SI0c where c
is 1, 2, . . . , r and r is the number of the rules.
Step 5: Execute the aggregation. According to the inference
engine and rule base in the first team, the firing strength
(ai) with an implication process of eight antecedents and
conclusion is calculated as shown in Fig. 9. Then, the
output of the aggregation process becomes the com-
bined output fuzzy set by lSI ðYÞ ¼ max½l01l ðYÞ; l02l ðYÞ;
l03l ðYÞ; :l0kl ðYÞ . . . ; l0nl ðYÞ, where l0kl ðYÞ means team per-
formance (Y) in rule k (k = 1, . . . , n) and signal l (l = Red,
Blue, Green). lSI(Y) represents the Safety Index (SI) of
team performance (Y).
Step 6: Calculate the defuzzification. The most commonly used
method, the center of gravity (COG) method introduced
by Mamdami in 1975, was applied in this system. As
shown in Fig. 10, the Safety Index (SI) of eight l0kl ðYÞ are
Safety Index = Red and Safety Index = Blue where the
interval is between [yL = 1, yR = 8] and is divided into t
areas. The crisp value is calculated as:
81
D ¼ 1; q ¼ þ1¼ 8
( 1 ),
X q X
q
SI ¼ Y t lSI ðY t Þ lSI ðY t Þ
t¼1 t¼1
0:66  ð1 þ 2 þ 3Þ þ 0:33  4 þ 0  5 þ 0:32  ð6 þ 7Þ þ 0  8
¼
0:66  3 þ 0:33 þ 0  2 þ 0:32  2
¼ 3:37
Step 7: Determine the signal of the Safety Index of the teamwork
by Eqs. (11)–(13). The signal (Red, Bule, and Green) can be
decided by the maximum of the lRed(SI), lBlue(SI), and
lGreen(SI). The results are shown in Table 3. From the sig-
nal of the Safety Index, the proposed fuzzy inference sys-
tem can diagnose the degree of safety in the current
teamwork in real-time. The purpose of this is to ensure
that the team members can maintain their team
performance.
432 S.-L. Hwang et al. / Safety Science 47 (2009) 425–435

Fig. 9. Calculation of firing strength (ai).


μ 1; SI 6 2
lRed ðSIÞ ¼ ð11Þ
Red Blue Green ð5  SIÞ=3; 2 6 SI 6 5

1 ðSI  2Þ=3; 2 6 SI 6 5
lBlue ðSIÞ ¼ ð12Þ
ð8  SIÞ=3; 5 6 SI 6 8

0.66 ðSI  5Þ=3; 5 6 SI 6 8
lGreen ðSIÞ ¼ ð13Þ
1; SI P 8

0.32 where lRed(SI), lBlue(SI), and lGreen(SI) represent the

membership degree of the Safety Index as Red, Blue, and
0 Green, respectively in the current team.
Y
1 2 3 4 5 6 7 8 9 10
yL yR
5.5. Model validation
Fig. 10. Defuzzification of safety index.
Hollnagel (1996) argued that in the operator model, the data
collection is unrealistically simple, and so the results will be of lim-
ited value. Thus, the models must be able to reflect the human sit-
uation in an adequate way. Therefore, in this study, we further
validated the proposed (1) teamwork performance prediction
Table 3
Safety Index of teamwork and signal type (modeling data) model, and (2) teamwork real-time warning model (RTWM).
The nine participants’ data from the three teams were used as
Team No. RO ARO Supervisor Safety Index Signal
an input to validate the accuracy of the first proposed model, and
Real Estimative the results are shown in Table 4. The nine estimated values were
1 0.34 0.54 0.34 3.37 2.41 Red very close to the real values, with the real values all within the
2 0.65 0.24 0.24 2.21 2.14 Red 95% confidence intervals. Thus, this implies that the predictive
3 0.22 0.22 0.22 2.11 2.2 Red
ability of the first model can attain a 95% confidence level. In addi-
4 0.71 0.26 0.26 2.10 2.1 Red
5 0.69 0.69 0.69 8.42 7.89 Green tion, this result revealed that the data from the first model may
6 0.83 0.83 0.83 8.42 8.41 Green effectively be used to construct the second model.
7 0.35 0.04 0.04 2.11 2.18 Red To validate the second model, the validation data from the three
8 0.89 0.89 0.89 8.42 7.77 Green teams was used as input for the fuzzy inference system. The result
9 0.58 0.58 0.58 6.44 5.22 Blue
of the validation is shown in Table 5. The estimated and real mem-
10 0.45 0.45 0.26 2.22 3 Red
bership values and signals that belong to Red, Blue, or Green cate-

Table 4
Model 1: performance model validation (validation data)

Team No. x1 x2 x3 x4 x5 x6 x7 x8 Estimated value Real value Low bound of 95% CI Upper bound of 95% CI
11 0.07 0.03 0.47 1.00 0.40 0.13 0.26 0.78 0.70 0.72 0.67 0.75
0.05 0.00 0.20 1.00 0.05 0.21 0.23 0.20 0.70 0.75 0.70 0.78
0.11 0.01 0.05 0.00 0.11 0.37 0.28 0.74 0.70 0.68 0.63 0.71
12 0.10 0.08 0.09 0.50 0.44 0.29 0.42 0.59 0.48 0.46 0.42 0.50
0.01 0.04 0.39 0.00 0.20 0.05 0.00 0.32 0.48 0.44 0.41 0.49
0.18 0.03 0.08 0.54 0.00 0.53 0.67 0.23 0.30 0.33 0.29 0.37
13 0.22 0.20 0.16 0.85 0.00 0.21 0.71 0.58 0.61 0.63 0.59 0.67
0.15 0.10 0.56 0.91 0.55 0.37 0.32 0.42 0.61 0.64 0.60 0.68
0.02 0.07 0.32 0.63 0.25 0.20 0.21 0.75 0.61 0.64 0.60 0.68
 S.-L. Hwang et al. / Safety Science 47 (2009) 425–435 433

Table 5
Model 2: Safety Index of teamwork and signal type (validation data)

Team No. RO ARO Supervisor Safety Index Membership Signal

Red Blue Green Estimated Real
11(e) 0.70 0.70 0.70 8.41 1 Green
12(e) 0.48 0.48 0.30 2.72 1 Red
13(e) 0.61 0.61 0.61 6.93 0.36 0.64 Green
11(r) 0.72 0.75 0.68 8.26 1 Green
12(r) 0.46 0.44 0.33 3.24 0.59 0.41 Red
13(r) 0.63 0.64 0.64 7.21 0.26 0.74 Green

(e) Represents the estimated value; (r) represents the real value.

Table 6
Pearson correlation analysis between the real and estimated values of validation data
Correlations Real value
Real value Pearson Correlation 1 0.999a
Significance (two-tailed) – 0.034
N 10
Estimated value Pearson Correlation 0.999a
Significance (two-tailed) 0.034
N 10
a
Correlation is significant at the 0.01 level (two-tailed).
gories were determined by Eqs. (11)–(13). The Pearson Correlation
analysis between the estimated and real Safety Index in Table 6
were significant (p < 0.05) and the correlation coefficient turned
out to be very high (r = 0.99). Therefore, the second model can
accurately provide teams with the warning signal.
6. Discussion
6.1. Physiological indices on mental workload assessment
Physiological measurement and signals could be obtained in
real-time and recorded continuously in the present investigation
(Luximon and Goonetilleke, 2001; Braarud, 2001; Laine and Bauer,
2002; Wilson and Russell, 2003; Rubio and Diaz, 2004).
In this experiment, eight heart rate variety indices that were
accomplished standards of the Task Force of the European Society
of Cardiology and the North American Society of Pacing and Elec-
trophysiology in 1996 were collected and analyzed. Compared
with previous studies that measure mental workload, the heart
rate variability is more efficient than eye blink frequency, blood
pressure, electroencephalogram (EEG), and other tests. Although
subjective mental workload assessment in teamwork proved to
be insignificant, the physiological indices revealed that some heart
rate variety indices among the team members were significant. The
finding of heart rate variety indices among them indicated that the
standard deviation of the normal-to-normal intervals (SDNN), HRV
triangular index (HRVti), and total power (TP) between the assis-
tant reactor operator and supervisor are insignificant; however,
they are significant with the reactor operator. Furthermore, the
standard deviation of the normal-to-normal intervals (SDNN), the
number of normal-to-normal interval pairs with over 50 m s differ-
ence (pNN50), HRV triangular index (HRVti), the root mean square
of successive normal-to-normal interval differences (RMSSD), and
total power (TP) of the heart rate variability indices of the assistant
reactor operator and supervisor are higher than those of the reac-
tor operator. These can be due to the following reasons: (1) each
team member has a different responsibility for teamwork; and
(2) the reactor operator has to operate more procedures than the
assistant reactor operator and supervisor. These findings are con-
sistent with previous heart rate variability studies that show as
task complexity increases, the heart rate variability decreases (Task
Force of the European Society, 1996; Hwang et al., 2008).
Estimated value
6.2. Team performance index assessment
10 Under the semi-automation task in this experiment, the error
1 rates and response time of teamwork were collected as perfor-
– mance indices. The team errors were classified into the following
10 categories: (1) the reactor operator neglected to input the next tar-
get value, (2) the assistant reactor operator and supervisor ignored
recording parameters, (3) the recording parameters were different
between the assistant reactor operator and supervisor, and (4) all
of the team members were not able to notice that the messages
of 29 break-points (BP1  BP29) and the seven setting points ap-
peared. The relationship between the error rates and response time
showed that the longer the interval of the event arrival time, the
higher the error rates of the teams. This result was in agreement
with the curve of mental workload and performance as proposed
by Veltman and Jansen (2006). This means that the lower the men-
tal workload of team members, the lower the teamwork perfor-
mance. Gauging from the responsibility of team members, the
tasks of the reactor operator were more complex than those of
the assistant reactor operator and supervisor. Thus, the mental
workload of the assistant reactor operator and supervisor was low-
er than normal. In addition, the special team of practical operators
from the Fourth Nuclear Power Plant in Taiwan spends a lot of time
checking whether the parameters, instructions, and actions are cor-
rect or not. Their accuracy rate is 100%, but they spend longer in
their response time.
6.3. Teamwork performance prediction model analysis
The results of the GMDH algorithm revealed that when the per-
3 pffiffiffiffiffi
formance index is applied ðy1 Þ = y2 , the R-square and correlation
coefficient are higher (refer Table 2). Although there were only
30 data sets used in the model development, all predictive values
fell in the 95% confidence intervals. The validation result of the
proposed model is the same as that found in the work of Hwang
et al. (2008).
6.4. Advantages, limitations, and application of the real-time warning
model (RTWM)
Fuzzy inference system and fuzzy logic have been used for mod-
eling in many fields (Becker et al., 1997; Ntuen, 1999; Moon et al.,
2002; Liu and Su, 2006; Gregoriades and Sutcliffe, 2006; Rani et al.,
2007; Yang et al., 2008). This study extended fuzzy applications to
develop a RTWM to detect the low performance of team members.
The proposed RTWM could predict teamwork performance in real-
time.
Furthermore, the RTWM have several advantages. One is that
the manager can adjust the fuzzy numbers to set stricter or more
434 S.-L. Hwang et al. / Safety Science 47 (2009) 425–435
lenient management policy. Second, the model can be expanded
into a multiple input and multiple output system. Lastly, the
GMDH algorithm can immediately combine individual perfor-
mance values into team performance and then determine the
team’s safety level. However, some limitations exist in this model:
(1) theoretical limitation, and (2) application limitation.
Firstly, the theoretical limitation with this work is that the cur-
rently proposed real-time warning model (RTWM) did not focused
so much on team communication and coordination but considered
more the monitoring of team cooperative tasks of low mental
workload. Secondly, the model can effectively predict team’s per-
formance and give the warning signal to team members based on
the simulator; however, in practice, there are still many task com-
plexities to be considered (e.g., component complexity, coordina-
tive complexity, dynamic complexity, and written manual or
operating procedures) (Wood, 1986; Park et al., 2003). Moreover,
the application of the RTWM in practice should overcome these
technical limitations. For example, the data transferred from the
Cardio Tens portable device to the GMDH software and then to
the fuzzy inference system was a challenging task.
7. Conclusions
The control room of the Fourth Nuclear Power Plant (FNPP) of
Taiwan is a digitally controlled system. Automation is one of the
characteristics of digital control systems. Many researches have
shown that automation may decrease operator’s workload, afford
operators to control more complex systems, and reduce the vari-
ability of human performance (Young and Stanton, 1997; Parasur-
aman, 2000; Metzger and Parasuraman, 2005; Liu and Su, 2006).
This study simulated the teamwork of FNPP control room where
a reactor operator (RO), an assistant reactor operator (ARO), and
a supervisor constitute a team to conduct the startup and monitor-
ing tasks with the PCTran simulator. The experimental results re-
vealed that (1) the operators’ psychological status changed
according to the degree of complexity of the tasks, and (2) the team
members made more errors when the interval of the event arrival
time increased. Thus, in such computer-supported cooperative
work (CSCW), one of the important things to avoid is the low men-
tal workload of any team member which may result in human er-
rors and accidents. Therefore, the teamwork performance
prediction model and the real-time warning model (RTWM) have
been developed in this study. The proposed model can efficiently
predict teamwork performance in real-time to increase both sys-
tem and human safety.
Acknowledgements
This research has been supported by the Institute of National
Science Council of Taiwan (Project No. 962001INER005). The
authors would like to thank the operators of Forth Nuclear Power
Plant engaged this study and the Institute of Nuclear Energy Re-
search for providing such an opportunity.
References
Artman, H., 2000. Team situation assessment and information distribution.
Ergonomics 43 (8), 1111–1128.
Baker, B.D., Richards, C.E., 1999. A comparison of conventional linear regression and
neural network methods for forecasting educational spending. Economics of
Education Review 18 (4), 405–415.
Becker, K., Thull, B., Käsmacher-Leidinger, H., Stemmer, J., Rau, G., Kalff, G.,
Zimmermann, H.-J., 1997. Design and validation of an intelligent patient
monitoring and alarm system based on a fuzzy logic process model. Artificial
Intelligence in Medicine 11, 33–53.
Braarud, P., 2001. Subjective task complexity and subjective workload: criterion
validity for complex team tasks. International Journal of Cognitive Ergonomics 5
(3), 261–273.
Carvalho, P.V.R., Vidal, M.C.R., 2007. Nuclear power plant communications in
Normative and actual practice. A field study of control room operator’s
communications. Human Factors and Ergonomics in Manufacturing 17 (1),
43–78.
Dagdeviren, M., Yuksel, I., Kurt, M., 2008. A fuzzy analytic network process (ANP)
model to identify faulty behavior risk (FBR) in work system. Safety Science 46
(5), 771–783.
Desmond, P.A., Hoyes, T.W., 1996. Workload variation, intrinsic risk and utility in a
simulated air traffic control task: evidence for compensatory effects. Safety
Science 22 (1–3), 87–101.
DiDomenico, A., Nussbaum, M.A., 2005. Interactive effects of mental and postural
demands on subjective assessment of mental workload and postural stability.
Safety Science 43, 485–495.
Greef, T., Dongen, K., Grootjen, M., Lindenberg, J., 2007. Augmenting cognition:
reviewing the symbiotic relation between man and machine. In: The Ninth
International Conference on Human–Computer Interaction, Beijing, PR China,
12–15 November, pp. 439–448.
Gregoriades, A., Sutcliffe, A.G., 2006. Automated assistance for human factors
analysis in complex systems. Ergonomics 49 (12/13), 1265–1287.
Hale, A.R., Heming, B.H.J., Carthey, J., Kirwan, B., 1997. Modelling of safety
management systems. Safety Science 26 (1/2), 121–140.
Hale, A., Kirwan, B., Kjellen, U., 2007. Safe by design: where are we now? Safety
Science 45, 305–327.
Hollnagel, E., 1996. Reliability analysis and operator modeling. Reliability
Engineering and System Safety 52, 327–337.
Hollnagel, E., 2003. Handbook of Cognitive Task Design. Erlbaum, Mahwah, NJ.
Hwang, S.L., Yau, Y.J., Lin, Y.T., Chen, J.H., Huang, T.H., Yenn, T.C., Hsu, C.C., 2008.
Predicting work performance in nuclear power plants. Safety science 46 (7),
1115–1124.
Ivakhnenko, A.G., 1968. Group method of data handling – rival of method of
stochastic approximation. Soviet Automatic Control 13 (1), 43–55.
Ivakhnenko, G.A., 1995. Self-organization of neuronet with active neurons for
effects of nuclear test explosions forecasting. System Analysis Modeling
Simulation (SAMS) 20, 107–116.
Ivakhnenko, A.G., Ivakhnenko, G.A., Müller, J.A., 1993. Self-organization of optimum
physical clustering of the data sample for weakened description and forecasting
of fuzzy objects. Pattern Recognition and Image Analysis 3 (4), 415–421.
Kageyama, T., Kabuto, M., 1995. Combined effects of mental workload and following
exposure to road traffic noise on sympathetic activities assessed by a spectral
component of heart rate variability. Arch Complex Environ Studies 7, 13–20.
Kamada, T., Sato, N., Miyake, S., Kumashiro, M., Monou, H., 1992a. Power spectral
analysis of heart rate variability in type as during solo and competitive mental
arithmetic task. Journal of Psychosomatic Research 36 (6), 543–551.
Kamada, T., Miyake, S., Kumashiro, M., 1992b. Power spectral analysis of heart rate
variability in Type As and Type Bs during mental workload. Psychosomatic
Medicine 54 (4), 462–470.
Kim, S., Kim, J., Lee, C.B., 2001. Fuzzy decision support system to the prediction of
ozone concentrations. In: IEEE Conference, Pusan, Korea.
Klir, G.J., Yuan, B., 1995. Fuzzy Sets and Fuzzy Logic: Theory and Applications.
Prentice Hall.
Kruse, R., Gebhardt, R., Klawonn, F., 1994. Foundations of Fuzzy Systems. Wiley,
Chichester.
Kulkarni, A.D., 2001. Computer Vision and Fuzzy-Neural Systems. Prentice Hall.
Laine, T.I., Bauer, K.W., 2002. Selection of input features across subjects for
classifying crewmember workload using artificial neural networks. IEEE
Transactions on Systems 32 (6), 691–704.
Liu, C.L., Su, K.W., 2006. A fuzzy logical vigilance alarm system for improving
situation awareness and trust in supervisory control. Human Factors and
Ergonomics in Manufacturing 16 (4), 409–426.
Luximon, A., Goonetilleke, R.S., 2001. Simplified subjective workload assessment
technique. Ergonomics 44 (3), 229–243.
Madala, H.R., Ivakhnenko, A.G., 1994. Inductive Learning Algorithms for Complex
Systems Modeling. CRC Press Inc, Boca Raton.
Mamdami, E.H., Assilina, S., 1975. An experiment in linguistic synthesis with a fuzzy
logic controller. International Journal of Man–Machine Studies 7 (1), 1–13.
Marseguerra, M., Zio, E., 2006. Quantitative developments in the cognitive reliability
and error analysis method (CREAM) for the assessment of human performance.
Annals of Nuclear Energy 33, 894–910.
Mclntyre, R.M., Strobel, K., Hanner, H., Cunningham, A., Tedrow, L., 2003. Toward an
Understanding of Team Performance and Team Cohesion over Time through the
Lens of Time Series Analysis. Army Research Institute (ARI) Research Note
2003–2007.
McNames, J., Thong, T., Goldstein, B., 2003. Reliability and accuracy of heart rate
variability metrics versus ECG segment duration. Engineering in Medicine and
Biology Society. Proceedings of the 25th Annual International Conference of the
IEEE 1 (17–21), 212–215.
Metzger, U., Parasuraman, R., 2005. Automation in future air traffic management:
effects of decision aid reliability on controller performance and mental
workload. Human Factors 47 (1), 35–49.
Mikulka, P.J., Scerbo, M.W., Freeman, F.G., 2002. Effects of a biocybernetic system on
vigilance performance. Human Factors 44, 654–664.
Moon, B.S., Lee, H.C., Lee, Y.H., Park, J.C., Oh, I.S., Lee, J.W., 2002. Fuzzy systems to
process ECG and EEG signals for quantification of the mental workload.
Information Sciences 142, 23–35.
Moray, N., 1988. Mental workload since 1979. International Review of Ergonomics
2, 123–150.
 S.-L. Hwang et al. / Safety Science 47 (2009) 425–435
Nishikido, T.K., Kurokawa, Y.H., Kobayashi, H.I., Kabuto, T.K., 1997. Effects of obesity,
current smoking status, and alcohol consumption on heart rate variability in
male white-collar workers. International Archives Occupation and Environment
Health 69, 447–454.
Ntuen, C.A., 1999. The application of fuzzy set theory to cognitive workload
evaluation of electronic circuit board inspectors. Human Factors and
Ergonomics in Manufacturing 9 (3), 291–301.
Parasuraman, R., 2000. Designing automation for human use: empirical studies and
quantitative models. Ergonomics 43 (7), 931–951.
Park, J., Jung, W., Kim, J., Ha, J., 2003. Step complexity measure for emergency
operating procedures-determining weighting factors. Nuclear Plant Operations
and Control 143, 290–308.
Pavel, N., Miroslav, S., 2003. Modeling of student’s quality by means of GMDH
algorithms. Systems Analysis Modeling Simulation 43 (10), 1415–1426.
Rani, P., Sarkar, N., Adams, J., 2007. Anxiety-based affective communication for
implicit human–machine interaction. Advanced Engineering Informatics 21,
323–334.
Rentsch, J.R., Klimoski, R.J., 2001. Why do ‘great minds’ think alike?: antecedents of
team member schema agreement. Journal of Organization Behavior 22, 107–120.
Rubio, S., Diaz, E., 2004. Evaluation of subjective mental workload: a comparison of
SWAT, NASA-TLX, and workload profile methods. Applied Psychology: An
international Review 53 (1), 61.
Sammer, G., 1998. Heart period variability and respiratory changes associated with
physical and mental load: non-linear analysis. Ergonomics 41, 746–755.
Sarycheva, L., 2003. Using GMDH In ecological and socio-economical monitoring
problems. Systems Analysis Modeling Simulation (SAMS) 43 (10), 1409–1414.
Scerbo, M.W., Freeman, F.G., Mikulka, P.J., 2001. The Efficacy of Psychophysioligical
Measures for Implementing Adaptive Technology. NASA/TP-2001-211018.
Schupp, B., Hale, A., Pasman, H., Lemkovitz, S., Goossens, L., 2006. Design support for
the systematic integration of risk reduction into early chemical process design.
Safety Science 44, 37–54.
Sebok, A., 2000. Team performance in process control: influences of interface design
and staffing levels. Ergonomics 43 (8), 1210–1236.
Shu, Y., Furuta, K., 2005. An inference method of team situation awareness based on
mutual awareness. International Journal Cognition, Technology, and Work 7 (4),
272–287.
435
Skitka, L.J., Mosier, K.L., Burdick, M., Rosenblatt, B., 2000. Automation bias and
errors: are crews better than individuals? The International Journal of Aviation
Psychology 10 (1), 85–97.
Stanton, N., Hedge, A., Brookhuis, K., Salas, E., Hendrick, H., 2005.
Handbook of Human Factors and Ergonomics Methods. CRC Press,
Boca Raton, FL.
Task Force of The European Society of Cardiology and The North American Society of
Pacing and Electrophysiology, 1996. Heart rate variability: standards of
measurement, physiological interpretation, and clinical use. European Heart
Journal 17, 354–381.
Tattersal, A.J., Hockey, G.R.J., 1995. Level of operator control and changes in heart
rate variability during simulated flight maintenance. Human Factor 37, 682–
698.
Veltman, J.A., Jansen C., 2006. The Role of Operator State Assessment in Adaptive.
TNO Report, TNO-DV3 2005 A245.
Wang, L.X., Mendel, J.M., 1992. Generating fuzzy rules by learning from examples.
In: Proceedings of the IEEE International Symposium on Intelligent Control, vol.
22(6). Arlington, pp. 1414–1427.
Wilson, G.F., Russell, C.A., 2003. Real-time assessment of mental workload using
psychophysiological measures and artificial neural networks. Human Factors 45
(4), 635–643.
Wood, R.E., 1986. Task complexity: definition of the construct. Organizational
Behavior and Human Decision Processes 37, 60–82.
Ward Systems Group (WSG), 1995. Neuroshell 2. User’s Guide. WSG, Frederick, MD.
Xie, B., Salvendy, G., 2000. Review and reappraisal of modeling and predicting
mental workload in single- and multi-task environments. Work and Stress 14
(1), 74–99.
Yang, G., Lin, Y., Bhattacharya, P., 2008. Multimodality inferring of human cognitive
states based on integration of neuron-fuzzy network and information fusion
techniques. EURASIP Journal on Advances in Signal Processing, Article ID
371621, 14 p., doi:10.1155/2008/371621.
Young, M.S., Stanton, N.A., 1997. Automotive automation: investigating the impact
on drivers’ mental workload. International Journal of Cognitive Ergonomics 1
(4), 325–336.
Zadeh, L., 1965. Fuzzy sets. Information and Control 8 (3), 338–353.