This article was downloaded by: [University of Maastricht]

On: 11 June 2014, At: 13:40

Publisher: Taylor & Francis
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

International Journal of Human-

Computer Interaction
Publication details, including instructions for authors and
subscription information:
http://www.tandfonline.com/loi/hihc20

Real-Time Analysis of EEG Indexes

of Alertness, Cognition, and Memory
Acquired With a Wireless EEG Headset
Chris Berka , Daniel J. Levendowski , Milenko M. Cvetinovic ,
Miroslav M. Petrovic , Gene Davis , Michelle N. Lumicao , Vladimir T.
Zivkovic , Miodrag V. Popovic & Richard Olmstead
Published online: 09 Jun 2010.

To cite this article: Chris Berka , Daniel J. Levendowski , Milenko M. Cvetinovic , Miroslav M.
Petrovic , Gene Davis , Michelle N. Lumicao , Vladimir T. Zivkovic , Miodrag V. Popovic & Richard
Olmstead (2004) Real-Time Analysis of EEG Indexes of Alertness, Cognition, and Memory Acquired
With a Wireless EEG Headset, International Journal of Human-Computer Interaction, 17:2, 151-170,
DOI: 10.1207/s15327590ijhc1702_3

To link to this article: http://dx.doi.org/10.1207/s15327590ijhc1702_3

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the
“Content”) contained in the publications on our platform. However, Taylor & Francis,
our agents, and our licensors make no representations or warranties whatsoever as to
the accuracy, completeness, or suitability for any purpose of the Content. Any opinions
and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content
should not be relied upon and should be independently verified with primary sources
of information. Taylor and Francis shall not be liable for any losses, actions, claims,
proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or
howsoever caused arising directly or indirectly in connection with, in relation to or arising
out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
Conditions of access and use can be found at http://www.tandfonline.com/page/terms-
and-conditions
 INTERNATIONAL JOURNAL OF HUMAN–COMPUTER INTERACTION, 17(2), 151–170
Copyright © 2004, Lawrence Erlbaum Associates, Inc.

Real-Time Analysis of EEG Indexes of Alertness,

Cognition, and Memory Acquired With a Wireless
EEG Headset

Chris Berka
Daniel J. Levendowski
Downloaded by [University of Maastricht] at 13:40 11 June 2014

Milenko M. Cvetinovic
Miroslav M. Petrovic
Gene Davis
Michelle N. Lumicao
Vladimir T. Zivkovic
Advanced Brain Monitoring, Inc., Carlsbad, CA

Miodrag V. Popovic
Faculty of Electrical Engineering, University of Belgrade, Belgrade, Serbia & Montenegro

Richard Olmstead
Veterans Affairs Greater Los Angeles Healthcare System

The integration of brain monitoring into the man–machine interface holds great prom-
ise for real-time assessment of operator status and intelligent allocation of tasks be-
tween machines and humans. This article presents an integrated hardware and soft-
ware solution for acquisition and real-time analysis of the electroencephalogram
(EEG) to monitor indexes of alertness, cognition, and memory. Three experimental
paradigms were evaluated in a total of 45 participants to identify EEG indexes associ-
ated with changes in cognitive workload: the Warship Commander Task (WCT), a
simulated navy command and control environment that allowed workload levels to
be systematically manipulated; a cognitive task with three levels of difficulty and con-
sistent sensory inputs and motor outputs; and a multisession image learning and rec-

We thank Tim Zavora and Roy Dalati for their assistance in data collection and analysis, and Philip
Westbrook for his editorial comments. We also thank Mark St. John, Dave Kobus, Jeff Morrison, and
their colleagues at Pacific Science and Engineering for organizing and hosting the pre-TIE and TIE data
acquisition sessions.
This research was supported by grants and contracts from DARPA and the National Institute of Neu-
rological Disease and Stroke, the National Institute of Mental Health, and the National Heart Lung and
Blood Institute divisions of the National Institutes of Health.
Requests for reprints should be sent to Chris Berka, Advanced Brain Monitoring, Inc., 2850 Pio Pico
Drive, Suite A, Carlsbad, CA 92008. E-mail: chris@b-alert.com
 152 Berka et al.

ognition memory test. Across tasks and participants, specific changes in the EEG were
identified that were reliably associated with levels of cognitive workload. The EEG in-
dexes were also shown to change as a function of training on the WCT and the learn-
ing and memory task. Future applications of the system to augment cognition in mili-
tary and industrial environments are discussed.

1. INTRODUCTION

The technical complexity and 24-hr schedule of contemporary industrial and mili-
tary operations increasingly demand rapid skill acquisition and the ability to sus-
tain high levels of performance for extended periods of time (Moore–Ede, 1993). Ef-
ficient and reliable interfaces between human operators and machines are required
Downloaded by [University of Maastricht] at 13:40 11 June 2014

to facilitate the integration and adoption of sophisticated technologies. With in-

creased automation, the operator’s role is often simply to monitor, and maintaining
vigilance becomes more difficult, with performance decrements increasing with
time-on-task (Parasuraman, Molloy, & Singh, 1993; Singh, Molloy, & Parasuraman,
1993). The integration of physiological monitoring into the man–machine interface
holds great promise both for real-time assessment of operator status and for pro-
viding a means to allocate tasks between machines and humans based on the oper-
ator status. Once meaningful real-time monitoring is achieved, intelligent feedback
or “closed-loop” systems can facilitate active intervention by the operator or
through a third party, increasing safety, efficiency, and productivity (Parasuraman,
Bahri, Deaton, Morrison, & Barnes, 1992; Parasuraman et al., 1993; Parasuraman,
Mouloua, & Molloy, 1996; Singh et al., 1993).
Research conducted over the past 40 years has established electroencephalogra-
phy (EEG) and event-related potentials (ERPs) as the primary tools available to sci-
entists investigating neural indexes of cognition (Fabiani, Gratton, & Coles, 2000).
Characteristic changes in the EEG and ERPs that reflect levels of alertness and
drowsiness, selective attention, workload, memory, and executive function have
been identified (Akerstedt & Folkard, 1997; Fabiani et al., 2000; Gevins et al., 1996;
Gevins et al., 1998; Gevins, Smith, McEvoy, & Yu, 1997; Hillyard, Hink, Schwent, &
Picton, 1973; Polich, 2002; Polich & Herbst, 2000). EEG measures have been applied
in areas as diverse as human factors, lie detection, pharmacological research, and
assessment of cognitive dysfunction caused by neurological disorders (Fabiani et
al., 2000). EEG indexes of real-time cognitive state changes offer the capability of
providing real-time input to adaptively automated systems (Parasuraman,
Mouloua, & Hilburn, 1999). This article presents an integrated hardware and soft-
ware solution for acquisition and real-time analysis of EEG to monitor indexes of
alertness, cognition, and memory. The system will translate the data into interpret-
able feedback for state modification by the operator, an observer, or a machine.
EEG has traditionally been confined to laboratory settings due to the technical
obstacles of recording high-quality data and the computational demands of
real-time analysis. A reasonable expectation for success with ambulatory EEG ap-
plications in operational environments is that the setup and acquisition of high
quality recordings can be obtained by the user without technical assistance. A re-
 Real-Time EEG Indexes of Cognition 153

cently developed wireless EEG sensor headset achieves this goal by combining
battery-powered hardware with a sensor placement system to provide a light-
weight, easy-to-apply method to acquire and analyze six channels of high-quality
EEG (Figure 1). The EEG sensor headset requires no scalp preparation and pro-
vides a comfortable and secure sensor-scalp interface for 12 to 24 hr of continu-
ous use. The headset was designed with fixed sensor locations for three sizes
(e.g., small, medium, and large). Sensor placement was determined using a data-
base of over 225 participants so that each sensor is no more than one centimeter
from the international 10 to 20 system coordinates. The workload studies de-
scribed in this article required only three EEG channels. The remaining channels
are utilized to monitor other cognitive states including attention, learning, and
memory.
Amplification, digitization, and radio frequency (RF) transmission of the signals
Downloaded by [University of Maastricht] at 13:40 11 June 2014

are accomplished with miniaturized electronics in a portable unit worn on the

head. The combination of amplification and digitization of the EEG close to the sen-
sors and wireless transmission of the data facilitates the acquisition of high-quality
signals even in high electromagnetic interference environments.
Quantification of the EEG in real time, referred to as the B-Alert® system, is
achieved using signal analysis techniques to identify and decontaminate fast and
slow eye blinks, and identify and reject data points contaminated with
electromyography (EMG), amplifier saturation, or excursions due to movement ar-
tifacts. Each 1-sec EEG epoch is then classified into one of four states of alertness:
“high vigilance,” “low vigilance,” “relaxed wakefulness,” and “sleepy.” These four
states were empirically derived using EEG acquired from individuals participating
in sleep deprivation studies. The high and low vigilance states were modeled by
varying the level of task engagement. Relaxed wakefulness is the state induced
when participants are instructed to relax with eyes closed and is generally charac-
terized by predominance of EEG in the alpha frequency band (8–12 Hz.). Data for
the sleepy class were obtained using EEG samples acquired just subsequent to
sleep onset.
The classification model utilizes discriminant function analysis derived from a
large normative database and is fitted to each individual’s unique EEG patterns

FIGURE 1 Wireless EEG sensor headset: (a) Front view, (b) Rear view, and (c) Aug-
mented-Cognition Technical Integration Experiment participant.
 154 Berka et al.

with data acquired from three baseline conditions. The B-Alert® model was devel-
oped and evaluated for its capability to provide early warnings of the onset of
drowsiness. The B-Alert® system was validated in sleep deprivation studies with
performance in a driving simulator (Levendowski, Berka, Olmstead, & Jarvik,
1999), accuracy and reaction time during a psychomotor vigilance task, behavioral
evidence as measured by cessation of finger tapping, visually scored observations
of facial signs of drowsiness (eye closures, head nods), and responses to a subjec-
tive sleepiness questionnaire (Levendowski et al., 2001; Levendowski, Olmstead,
Konstantinovic, Berka, & Westbrook, 2000). The B-Alert® model was independ-
ently validated with visual inspection of the EEG signals and observations of facial
signs of drowsiness conducted by two board-certified sleep specialists
(Levendowski et al., 2000; Mitler et al., 2002). The model was demonstrated to be ef-
fective in characterizing excessive daytime drowsiness in patients with sleep apnea
Downloaded by [University of Maastricht] at 13:40 11 June 2014

(Westbrook et al., 2002). Analysis of the B-Alert® indexes during 44 hr of sleep de-
privation revealed that changes in the indexes could predict performance deficits
(Mitler et al., 2002) and confirmed the previously reported observation (Balkin,
2001; Doran, Van Dongen, & Dinges, 2001) that individuals differ in their vulnera-
bility to sleep deprivation. Sleep deprivation studies (Levendowski et al., 2000;
Mitler et al., 2002) revealed that highly engaging or difficult tasks induce higher
overall levels of vigilance as measured by B-Alert®, suggesting that the B-Alert® in-
dexes may have utility in monitoring cognitive workload.
Other investigators have reported EEG measures of workload that reflected dif-
ferences in task-related cognitive resource allocation, task mastery, and task over-
load (Byrne & Parasuraman, 1996; Kramer, 1991; Pope, Bogart, & Bartolome, 1995;
Prinzel, Freeman, Scerbo, Mikulka, & Pope, 2000; Sterman, 1995). The EEG vari-
ables employed in these models to monitor workload included alpha suppression,
increased beta, increased frontal midline theta, and ratios such as beta–alpha plus
theta and alpha plus theta–beta. The B-Alert® model incorporates information
from the alpha, beta, and theta bands in assigning a class to each 1-sec of EEG. It
was hypothesized that increasing the workload would result in increasing levels of
high vigilance as measured by the percentage of B-Alert® high vigilance.
The studies presented in this article were conducted to evaluate the efficacy
of the sensor headset and B-Alert® system in monitoring mental workload dur-
ing complex cognitive tasks for the Defense Advance Research Projects Agency
(DARPA) Augmented Cognition (Aug-Cog) program. Figure 1c presents a par-
ticipant during a typical Aug-Cog data acquisition session with the EEG sensor
headset, eye tracking and pupillometry (Marshall, St. John, Knust, & Binning,
2003), and functional near infrared imaging headgear (Onaral, 2003). Three cog-
nitive test paradigms were employed to evaluate workload measures. The War-
ship Commander Task (WCT) simulated a multitasking Navy command and
control environment and allowed workload levels to be systematically manipu-
lated (St. John, Kobus, & Morrison, 2002). A cognitive task with three levels of
difficulty provided consistent sensory inputs and motor outputs while varying
workload levels (Mathiak et al., 2002). Image Learning and Recognition Mem-
ory Tests (Levendowski et al., 2002) were utilized to evaluate the B-Alert® in-
dexes during learning and recognition memory.
 Real-Time EEG Indexes of Cognition 155

2. METHODS

2.1. Acquisition Hardware System

The sensor headset acquires six channels of EEG or electrooculography (EOG) us-
ing either a unipolar or bipolar montage. Data are sampled at 256 samples per sec-
ond with a bandpass from 0.5 Hz and 65 Hz (at 3 dB attenuation) obtained digitally
with Sigma-Delta A/D converters. The RF link is frequency-modulated to transmit
at a rate of 57 kBaud in the 915 MHz ISM band. When utilized in the bidirectional
mode, the firmware allows the host computer to initiate impedance monitoring of
the sensors, select the transmission channel (so two or more headsets can be used in
the same room), and monitor battery power of the headset. Data are acquired
across the RF link on a host computer via an RS232 interface. Data acquisition soft-
Downloaded by [University of Maastricht] at 13:40 11 June 2014

ware stores the EEG data and synchronizes event codes generated by the cognitive
tasks in the EEG record for ERP analysis. For all of the results presented later, the
standard hardware montage includes bipolar recordings from Fz to POz and Cz to
POz (required for the B-Alert® system), unipolar recordings from Fz, Cz, and POz
referenced to linked mastoids (available for ERP analysis), and a bipolar configura-
tion for horizontal and vertical EOG (to confirm the accuracy of the eye blink iden-
tification and decontamination algorithm).

2.2. Artifact Identification and Decontamination, and Signal Processing

Artifacts automatically detected and decontaminated in the time-domain EEG sig-

nal include 3, 5, or 7 data point spikes with amplitudes greater than 40 mV (caused
by tapping or bumping of the sensors), amplifier saturation, and excursions that
occur during the onset or recovery of saturations. For each of these artifacts, data
points with 0 µ V are inserted, starting at the last zero crossing prior to and ending
at the first zero crossing after the artifact. A 60 Hz notch filter is applied to all EEG
data. Three sets of filtered EEG data are then derived using a 0.5 Hz 256th order
high-pass FIR filter, a 4 Hz 640th order FIR high-pass filter, and a 7 Hz IIR low-pass
filter. To obtain faster computations, both high-pass filters are realized by subtract-
ing the output of the corresponding low-pass filter from the original signal.
Identification of eye blinks in the EEG without the use of a reference EOG chan-
nel is achieved by filtering the fast component of the FzPOz channel with a 7 Hz IIR
low-pass filter, applying cross-correlation analysis to the filtered signal using the
positive half of a 40 µ V 1.33 Hz sine wave as the target shape, and applying thresh-
olds to the outputs from the cross-correlation analysis. Minima and maxima analy-
sis in each direction from the point of maximum correlation is used to identify the
data points corresponding to the range between the start and end of each eye blink.
Once eye blink ranges have been determined, the 0.5 Hz high-pass filtered EEG sig-
nal from each channel is decontaminated by replacing the data points in the eye
blink region with the corresponding data after application of the 4 Hz filter.
Decontaminated EEG is then segmented into overlapping 256 data-point win-
dows called overlays. An epoch consists of three consecutive overlays. Fast-Fourier
 156 Berka et al.

transform is applied to each overlay of the decontaminated EEG signal multiplied

by the Kaiser window (α = 6.0) to compute the power spectral densities (PSD). The
PSD values are adjusted to take into account zero values inserted for artifact-con-
taminated data points. The PSD between 70 and 128 Hz is used to detect EMG arti-
fact. Overlays with excessive EMG artifact (“EMG”) or with fewer than 128 data
points (“missing data”) are rejected. The remaining overlays are then averaged to
derive PSD for each epoch with a 50% overlapping window. Epochs with two or
more overlays with EMG or missing data are classified as invalid. For each channel,
PSD values are derived for each 1-Hz bin (“bin”) from 3 Hz to 40 Hz and the total
PSD from 3 Hz to 40 Hz (“band”).

2.3. Classification Model

Downloaded by [University of Maastricht] at 13:40 11 June 2014

The four-class model was developed with data from three 5-min baseline condi-
tions (i.e., baseline conditions = finger-tapping with eyes open [EO] and eyes
closed [EC] and three-choice psychomotor vigilance task [PVT]) and sleepy epochs
selected from sleep-deprivation data using a database of 150 healthy participants.
For each epoch from these four conditions, five variables were computed for each
1-Hz bin between 3 Hz and 40 Hz (5 variables × 38 bins) for each channel: the
logged PSD, the relative power compared to the total power between 3 Hz and 40
Hz, and the z scores for EO, EC, and PVT. The z scores were computed using the
means and standard deviations from each of the three baseline conditions. The
identification of fast blinks was also used as a predictive variable. A total of 381
variables were available for each epoch based on a two-channel classification
model (i.e., 190 variables each for FzPOz and CzPOz, plus fast-blink).
The variables from each artifact-free epoch for the four conditions were submit-
ted to stepwise analysis to select those variables most predictive in a four-class
model (high vigilance, low vigilance, relaxed wakefulness, and sleepy). A total of
19 variables were selected. The most predictive variables were as follows: (a) the z
score of the 10 Hz bin from CzPOz relative to the PVT task (partial r2 = 0.46), (b) the
presence of a fast blink (partial r2 = 0.09), and (c) the z score of 11 Hz from CzPOz
relative to eyes closed (partial r2 = 0.06). The r2 for each of the remaining 16 vari-
ables was less than 0.03.
The approach developed for the B-Alert® system was to utilize population data
to establish the underlying model and then refine the discriminant function by ad-
justing for individual differences in the EEG using data from the three baseline con-
ditions. Although baseline data could be readily acquired for development of the
classification models for new individuals, sleep data could not be obtained a priori.
Rather, the mean values of all variables from the three baseline conditions (190 vari-
ables × two channels × three conditions) for all participants in the database were
submitted to stepwise linear regressions to derive equations to predict the
discriminant function coefficients for the “sleepy” classification for the each of the
19 predictive variables. Matrices were then derived using the aforementioned anal-
yses to fit the four-class model to the individual and compute the probability of cor-
 Real-Time EEG Indexes of Cognition 157

rect classification into each of the four output classes on a second-by-second basis
offline or in real time.

2.4. Warship Commander Task

The WCT requires the user to monitor groups (“waves”) of incoming aircraft
(“tracks”), to identify tracks as friendly or hostile, and to warn and then destroy
hostile tracks. Each wave is 75 sec in duration with workload level manipulated by
increasing numbers of tracks per wave (6, 12, 18, or 24 tracks). The learning curve
for WCT is steep due to the complexity of task demand. Performance measures
computed for each wave, including reaction times to identify, warn, and destroy
tracks and total game scores (as a percentage of possible points), correlate directly
Downloaded by [University of Maastricht] at 13:40 11 June 2014

with workload levels (number of tracks) and can be used as behavioral measures of
participant workload (St. John et al., 2002).
Of a total of 15 “novices” evaluated in an initial study, one was eliminated from
the study for fatigue and another for failing to reach proficiency on WCT, leaving a
total of 13 participants in the analysis. Novices were given 35 min of training to
achieve sufficient proficiency to complete the basic protocol that included three
waves of 6, 12, and 18 tracks. As part of the DARPA Augmented Cognition Techni-
cal Integration Experiment (TIE), 10 participants were studied with additional lev-
els of WCT training, ranging from 1 hr to over 300 hr. This group of 10 participants
completed multiple sessions with 3 waves of 6, 12, 18, and 24 tracks (presented in
the order 6, 18, 12, 24).

2.5. Three-Level Cognitive Task

One of the limitations of the WCT in validating a cognitive workload measure was
the fact that the number of stimuli and the amount of required motor activity (mouse
and keypress) varied directly as a function of the “workload” manipulations. To
evaluate the B-Alert® EEG indexes without the sensory and motor confounds associ-
ated with workload levels in the WCT, a three-level cognitive task, developed as part
of the DARPAAugmented Cognition project (Mathiak et al., 2002), was evaluated in
16 healthy participants. In this task, both the stimuli and motor demands are kept
constant during three levels of increasing task difficulty (easy, moderate, hard) by
employing the same number of stimuli and targets for each level of difficulty. For
each level of difficulty, 250 trials of single integers between 1 and 8 are presented with
a 1.6-sec Interstimulus Interval (ISI). The order of the digit presentation is identical
for each of the levels of difficulty to maintain consistent visual inputs. For level 1
(easy), participants are instructed to press the space bar with the index finger of both
hands only when they see the number 5. In level 2 (moderate), participants respond
only after any three consecutive even numbers and for level 3 (hard), they respond
only to a number the same as the number 2 trials earlier (2-back task). Performance
 158 Berka et al.

measures, including reaction times and percentage of correct responses, were com-
puted to verify the levels of task difficulty.

2.6. Image Learning and Recognition Memory Task

The recognition memory task requires participants to memorize 20 images, and

then recognize those images when randomly interspersed among 80 new images of
the same category (e.g. animals, food, travel, sports). The categories were selected
and evaluated for equivalence in test performance across categories. During the
training period, each of the 20 images is presented sequentially twice for 1.25 sec
with a 1.5-sec ISI. During the testing period, each of the 100 images is presented for
100 msec. with a 2.1-sec ISI, while the participant identifies the training and new
Downloaded by [University of Maastricht] at 13:40 11 June 2014

images. Each participant completed two recognition memory sessions with two
different image categories.

2.7. Participants

A total of 45 fully-rested, healthy individuals (ages 18–50) participated in the WCT

novice (n = 13), three-level cognitive task (n = 16), Image Learning and Recognition
Memory Task (n = 19), and DARPA Augmented Cognition TIE (n = 10) studies.
Thirteen participants took part in both the WCT novice and three-level cognitive
task All studies were conducted at the Advanced Brain Monitoring laboratory in
Carlsbad, CA, between the hours of 9 a.m. and 2 p.m. with the exception of the
DARPA Augmented Cognition TIE, conducted at Pacific Science and Engineering
in San Diego, CA.

3. RESULTS

3.1. WCT Novices

B-Alert® percentage high vigilance (HV) and WCT performance measures were
averaged across the three waves for each of the workload levels (i.e., 6, 12, 18
tracks) by and across participants. A repeated-measures analysis of variance
(ANOVA) applied across the three WCT workload levels revealed an increasing
percentage of HV classifications as a result of increasing workload with a signifi-
cant main effect for workload (F = 7.369, p < 0.005; Figure 2). Significance levels
reflect an adjustment to the degrees of freedom using the Greenhouse–Geisser
procedure to correct for violations of the sphericity assumption in repeated-mea-
sures designs, when appropriate.
Post hoc comparisons between the easy, moderate, and hard levels of the WCT
revealed significant differences for easy versus moderate (F = 5.095, p < 0.05) and
easy versus hard (F = 10.423, p < 0.01), with the moderate versus hard (F = 4.074, p <
0.066) approaching significance. As expected, repeated measures ANOVAs across
 Real-Time EEG Indexes of Cognition 159

FIGURE 2 Mean ± SE percentage of EEG epochs classified as high vigilance (% HV) dur-
Downloaded by [University of Maastricht] at 13:40 11 June 2014

ing WCT in 13 novices for easy, moderate, and hard levels (6, 12, and 18 tracks, respectively).

the three levels showed significant increases in the reaction time to identify tracks
(WCT-Rtiff; F = 88.913, p < 0.001) and decreases in the game scores (WCT-%score; F
= 117.78, p < 0.001) across the three levels (Figure 3). The Pearson product correla-
tions between HV and at least one of the WCT performance measures were r ≥ 0.85
in the majority of participants.

3.2. WCT TIE

Repeated measures ANOVAs across the four WCT workload levels revealed an in-
creasing percentage of high vigilance classifications (Figure 4; F = 3.573, p < 0.001),
significant increases in the reaction time to identify tracks (WCT-Rtiff; F = 27.485, p
< 0.001), and decreases in the game scores (WCT-%score; F = 14.585, p < 0.001)
across the four levels of increasing workload with a significant main effect for

FIGURE 3 Mean ± SE percentage game scores (%Score) and reaction time (RTiff) during
WCT in 13 novices for easy, moderate, and hard levels (6, 12, and 18 tracks, respectively).
 160 Berka et al.

FIGURE 4 Mean ± SE percentage of epochs classified as high vigilance (% HV) dur-

Downloaded by [University of Maastricht] at 13:40 11 June 2014

ing WCT in 10 Technical Integration Experiment participants for 6, 12, 18, and 24
tracks.

FIGURE 5 Mean reaction time (RTiff) and percentage of epochs classified as high
vigilance (% HV) across 12 waves in 10 Technical Integration Experiment participants
during WCT.

workload. Figure 5 presents the wave-by-wave relation between the mean B-Alert®
indexes and mean WCT-RTiff across participants, illustrating that the B-Alert® in-
dexes maintained the temporal characteristics of the WCT protocol. The correlation
between B-Alert® indexes and the WCT-RTiff across waves and participants was
significant but weak (r = .432, p < 0.01) as a result of within- and between-subject
variability.

3.3. Effects of increasing levels of training in WCT

An inspection of data from individual participants suggested that the percentage

of HV classifications during WCT was influenced by the level of training. Figure 6
illustrates this observation with data from four individual participants with in-
 Real-Time EEG Indexes of Cognition 161

FIGURE 6 Percentage epochs classified as high vigilance (% HV) in 6, 12, 18, and 24
Downloaded by [University of Maastricht] at 13:40 11 June 2014

tracks for four participants with 0.6, 5, 10, and 200 hr of WCT training, respectively.

creasing levels of WCT training: 35-min training, 5-hr training, 10-hr training, and
over 40 hr of training. The workload effect is demonstrated by an increasing per-
centage of high vigilance at each of the levels and the training effect is indicated by
an overall decrease in the percentage of HV. These findings suggest that differences
in training among the TIE participants (ranging from 1 hr to 300 hr) might explain
some of the between-subject variability. To investigate this hypothesis, the 10 TIE
participants were stratified into three groups based on amount of WCT training
(i.e., 1–3 hr, n = 3; 6–8 hr, n = 4; and 40–300 hr, n = 3), group means were calculated
for the percentage of HV, and an ANOVA was applied across workload levels by
group (see Figure 7). Despite the small sample size, the results revealed a signifi-

FIGURE 7 Mean ± SE percentage of epochs classified as high vigilance (% HV) in 6,

12, 18, and 24 tracks for three groups stratified based on WCT training: group 1 with 1
to 3 hr (n = 3), group 2 with 6 to 8 hr (n = 4), and group 3 with 40 to 300 hr (n = 3) of WCT
training. Probabilities for the comparisons between workload levels are designated as
*p < .05 and **p < .01.
 162 Berka et al.

cant interaction between level of expertise and workload level (number of tracks; F
= 3.118, p < 0.01).
Figure 7 illustrates that the B-Alert® percentage HV increases across WCT work-
load levels for all three groups, however, the overall levels of HV across the four
workload levels decreased dramatically across groups as a result of training. This
suggests that as participants gain expertise, the level of vigilance is modulated to
meet task demands. For the participants with over 40 hr of training, there were no
statistically significant differences between the 12- to 18-track and 24-track condi-
tions (Figure 7). Due to the small sample size (n = 3), it is premature to draw any
conclusions from these data.

3.4. Three-Level Cognitive Task

Downloaded by [University of Maastricht] at 13:40 11 June 2014

The mean percentage of B-Alert® HV classifications for each of the three difficulty
levels is illustrated in Figure 8. Repeated measures ANOVAs across the three levels
of difficulty revealed a significant increase in the percentage of high vigilance clas-
sifications as a result of increasing task difficulty (F = 21.962, p < 0.001). Compari-
sons between the easy, moderate, and hard levels revealed significant differences
for easy versus moderate (F = 6.419, p < 0.05), easy versus hard (F = 24.608, p <
0.001), and moderate versus hard (F = 27.321, p < 0.001). Repeated measures
ANOVAs across the three difficulty levels revealed significant increases in reaction
time (F = 17.267, p < 0.001) and decreases in the percentage of correct responses (F =
38.217, p < 0.001), confirming the actual increase in the levels of task difficulty (Fig-
ure 9). The correlations between HV and at least one of the performance measures
were r ≥ 0.85 in the majority of participants.

FIGURE 8 Mean ± SE percentage of epochs classified as high vigilance (% HV) for

three levels of cognitive task.
 Real-Time EEG Indexes of Cognition 163
Downloaded by [University of Maastricht] at 13:40 11 June 2014

FIGURE 9 Mean ± SE for percentage of correct responses (% correct) and reaction time
(RT) during easy, moderate, and hard difficulty levels of a three-level cognitive task.

3.5. Image Learning and Recognition Memory Task

Mean performance (percentage of correct responses) was computed for each of the
two sessions. Mean percentages of B-Alert® HV classifications were computed for
the image memorization period and the recognition memory period for each of the
two sessions of the memory test. Comparison of the performance results between
the two sessions revealed significant improvements in performance between ses-
sion 1 and session 2 (Figure 10), suggesting a practice effect (t = –2.983, p < .01). Sim-
ilar practice-related results were obtained for the B-Alert® indexes with percentage
HV decreasing from session 1 to session 2 (t = 4.765, p < .001). In addition, signifi-

FIGURE 10 Mean ± SE for percentage of correct responses (% correct) during image

learning and recognition memory test for two sessions (categories: animals and travel).
 164 Berka et al.

FIGURE 11 Mean ± SE percentage of epochs classified as high vigilance (% HV) for the
Downloaded by [University of Maastricht] at 13:40 11 June 2014

image memorization period and the recognition memory period for the two sessions.

cant within-session differences in the B-Alert® HV indexes were obtained with

higher levels of HV during the image memorization period when compared to the
recognition memory period (Figure 11). This effect was enhanced for session 2 (t =
3.291, p < .005) in comparison to session 1 (t = 1.989, p = .06).

4. DISCUSSION

The wireless sensor headset provided a reliable method for EEG acquisition and
analysis even within the constraints of the challenging TIE environment (see Figure
1c). Although previous investigators have reported high quality EEG acquisition in
operational settings including airplane cockpits, long haul truck cabins, and train
operator quarters (Caldwell, 1995; Kecklund & Akerstedt, 1993; Miller, 1995;
Mitler, Miller, Lipsitz, Walsh, & Wylie, 1997; Sterman & Mann, 1995), the wireless
sensor headset represents a significant advance in technology for operational mon-
itoring. The limited channel montage has now been demonstrated effective for the
detection of EEG indexes of alertness–drowsiness (Levendowski et al., 2001;
Levendowski et al., 2000), attention- and memory-related ERPs (Levendowski et
al., 2000; Mitler et al., 2002), and the quantification of cognitive workload during
the WCT and the three-level cognitive task. The limited channel approach has also
been utilized by other investigators to achieve highly sensitive and reliable correla-
tions between EEG and performance, including the ability to predict performance
on a second-by-second basis (Makeig & Inlow, 1993; Makeig & Jung, 1995, 1996;
Sterman & Mann, 1995; Torsvall & Akerstedt, 1987).
The B-Alert® system was designed to detect and predict vigilance decrements,
and was validated in sleep deprivation studies using a variety of objective and sub-
jective measures to confirm its sensitivity and reliability (Levendowski et al., 1999;
Levendowski et al., 2001; Levendowski et al., 2000). In studies of more than 400
healthy participants and sleep disorder patients, a number of tonic influences on
the B-Alert® indexes were observed including homeostatic and circadian effects,
 Real-Time EEG Indexes of Cognition 165

individual differences in vulnerability to sleep deprivation, and disease states such

as sleep and neurological disorders (Westbrook et al., 2002). Quantifiable changes
in the B-Alert® indexes were predictive of fatigue-related decrements in accuracy
and reaction time that occurred hours before the actual onset of sleep. B-Alert® in-
dexes also identified lapses in alertness on a second-by-second basis that were pre-
dictive of missed responses or errors on vigilance tests (Levendowski et al., 2001;
Levendowski et al., 2000; Mitler et al., 2002).
These studies also revealed a number of phasic influences on the B-Alert® in-
dexes including the type of task the in which the participant was engaged, task dif-
ficulty, and novelty (Levendowski et al., 2001; Levendowski et al., 2000; Mitler et
al., 2002). Caffeine and nicotine were shown to affect the B-Alert® indexes (Berka et
al., 2000). Although the B-Alert® model was designed for maximum sensitivity to
the vigilance and performance decrements resulting from the transition from wak-
Downloaded by [University of Maastricht] at 13:40 11 June 2014

ing to drowsiness and sleep onset, these data suggested that the B-Alert® indexes
may have utility for monitoring workload.

4.1. B-Alert® Indexes as Measures of Cognitive Workload

Cognitive workload has been conceptualized as the allocation of mental re-

sources or effort required to maintain adequate performance on one or more
tasks (Kahneman, 1973; Kramer, 1991; Wickens, 1984, 1992; Wickens & Holland,
1984). Several investigators have reported EEG measures of workload and task
difficulty, and in a review of studies of EEG and workload, Sterman and Mann
(1995; Sterman, 1995) concluded that EEG had the potential to be a valid and ob-
jective measure of mental workload. Studies of air traffic controllers (Brookings,
Wilson, & Swain, 1996), airline pilots (Sterman & Mann, 1995; Sterman, Mann, &
Kaiser, 1992;), drivers (Brookhuis & de Waard, 1993), and participants performing
cognitive tasks (Smith, Gevins, Brown, Karnik, & Du, 2001), have related EEG
shifts to changes in task complexity and task difficulty. The EEG variables com-
monly employed by these investigators (e.g. changes in alpha, beta, and theta
bands; Byrne & Parasuraman, 1996; Kramer, 1991; Pope et al., 1995; Prinzel et al.,
2000; Sterman & Mann, 1995) are incorporated into the B-Alert® classification
model.
The WCT developed for the DARPA project had several advantages for validat-
ing workload indexes: The task was highly engaging with a series of multitasking
subtests, the workload levels could be systematically varied, and the task resem-
bled a navy command and control scenario (St. John et al., 2002). One problem with
the WCT was that the number of stimuli and the motor demands increased as a
function of increasing workload levels, leaving open the possibility that perceptual
processes or motor activity were being quantified and not cognitive workload per
se. Although changes in sensory processing and motor output are important ele-
ments of increasing engagement in a task, it has been proposed that the neural re-
sources involved in cognitive processes are relatively independent of those in-
volved in the motor output (Donchin, Karis, Bashore, Coles, & Gratton, 1986;
Gopher, 1992; Wickens, 1991).
 166 Berka et al.

The three-level cognitive task allowed a more precise definition of the validity of
the B-Alert® indexes in the measurement of workload by keeping constant the
number and type of stimuli as well as the motor demands while manipulating the
cognitive requirements. The combined results from these two tasks confirm that
the B-Alert® indexes are related to cognitive effort associated with task difficulty
and not to the number of sensory inputs or the amount of motor output required
for the different levels.
The image memory and recognition task offered a third paradigm for evaluating
the B-Alert® indexes. The percentage of B-Alert® HV was significantly higher dur-
ing the image memorization period when compared to the recognition memory pe-
riod (Figure 11), presumably as a result of increased effort during the encoding pe-
riod in comparison to the less demanding recognition period.
Downloaded by [University of Maastricht] at 13:40 11 June 2014

4.2. Training Effects

Although the WCT experiments were not designed specifically to evaluate changes
in EEG parameters as a function of training, the level of WCT expertise clearly had
a substantial impact on the B-Alert® indexes, with overall high vigilance percent-
ages decreasing dramatically as a result of training. Thus, in the context of the
WCT, the B-Alert® indexes reflect changes in task difficulty as well as the total ef-
fort expended by the operator. These results, in conjunction with performance data
and subjective reports, confirm that the cognitive, and by inference, neural, re-
sources required for WCT decrease as a function of practice and are consistent with
current theories in cognitive skill acquisition (Fisk & Schneider, 1984; Schneider &
Fisk, 1982; Schneider & Shiffrin, 1977). Similarly, brain imaging studies show a re-
duction in the distribution and the magnitude of cortical activation as skill acquisi-
tion progresses (Haier et al., 1992; Raichle et al., 1994). Further investigation is re-
quired to determine whether the B-Alert® indexes can be used as predictive
variables to estimate, for example, the number of tracks that can be successfully
handled by an operator at a specified time. Interestingly, the WCT “workload ef-
fect” appeared to diminish after 40 to 300 hr of training. It would be of value for
practical applications to determine whether additional training would ultimately
eliminate this distinction.
The image learning and recognition memory tests provide additional support
for the B-Alert® indexes reflecting practice effects that were correlated with im-
proved performance (Figures 10 and 11). It is tempting to speculate that these data
imply that the B-Alert® indexes reflect practice-related changes in cognitive re-
source allocation and that, as participants gain expertise, the level of “vigilance” as
measured by B-Alert® is modulated to meet task demands. To support this conclu-
sion, additional studies are required utilizing repeated measures testing on indi-
viduals at regular intervals during the skill acquisition process.
If the B-Alert® indexes related to skill acquisition can be replicated in a con-
trolled experiment, they may have utility for monitoring progress during a variety
of computer-based training exercises. An intelligent brain-computer interface
could be designed to assure that trainees move efficiently through various levels of
 Real-Time EEG Indexes of Cognition 167

expertise using the B-Alert® indexes to maintain an optimal pace of information

presentation.

5. CONCLUSION

The wireless sensor headset and the B-Alert® EEG analysis software provide a ro-
bust and reliable method for monitoring alertness and cognitive workload in oper-
ational environments. The B-Alert indexes are sensitive to long-term and tran-
sient fluctuations in the neural activity involved in alertness and cognitive
workload. In future applications, the global B-Alert indexes could be applied in
combination with other analytic techniques such as ERPs (Fabiani et al., 2000; Kok,
2001; Levendowski et al., 2000) or event-related desynchronization (Klimesch,
Downloaded by [University of Maastricht] at 13:40 11 June 2014

Russegger, Doppelmayr, & Pachinger, 1998) to capture a more detailed image of in-
formation processing in the brain.
Additional research is required to develop an intelligent interface that utilizes
the outputs from the B-Alert® to drive an adaptive automation system. The data
suggest that in addition to predicting performance decrements resulting from ex-
cessive workload, the B-Alert® EEG indexes could have utility in optimizing the
speed and efficiency of computer-based training programs.

REFERENCES

Akerstedt, T., & Folkard, S. (1997). The three-process model of alertness and its extension to
performance, sleep latency, and sleep length. Chronobiolology International, 14(2), 115–123.
Balkin, T. J. (2001, August). Sleep deprivation research at WRAIR. Paper presented at the
DARPA Workshop, Las Vegas, NV.
Berka, C., Levendowski, D. J., Konstantinovic, Z. R., Olmstead, R. E., Davis, G., & Lumicao,
M. N. (2000, July). Detection of drowsiness with realtime analysis of the electroencephalogram
(EEG). Paper presented at the National Institutes of Health SBIR Conference, Bethesda,
MD.
Brookhuis, K. A., & de Waard, D. (1993). The use of psychophysiology to assess driver status.
Ergonomics, 36, 1099–1110.
Brookings, J. B., Wilson, G. F., & Swain, C. R. (1996). Psychophysiological responses to
changes in workload during simulated air traffic control. Biological Psychology, 42,
361–377.
Byrne, E. A., & Parasuraman, R. (1996). Psychophysiology and adaptive automation.
Biological Psychology, 42, 249–268.
Caldwell, J. (1995). Assessing the impact of stressors on performance: Observations on levels
of analyses. Biological Psychology, 40, 197–208.
Donchin, E., Karis, D., Bashore, T. R., Coles, M. G., & Gratton, G. (1986). Cognitive
psychophysiology and human information processing. In M. G. Coles, E. Donchin, & S.
Porges (Eds.), Psychophysiology: Systems, processes and applications (pp. 244–266). New
York: Guilford.
Doran, S. M., Van Dongen, H. P., & Dinges, D. F. (2001). Sustained attention performance
during sleep deprivation: Evidence of state instability. Archives Italiennes de Biologie,
139(3), 253–267.
 168 Berka et al.

Fabiani, M., Gratton, G., & Coles, M. G. (2000). Event-related brain potentials. In J. T.
Caciooppo, L. G. Tassinary, & G. G. Berntson (Eds.), Handbook of psychophysiology (pp.
53–84). Cambridge, England: Cambridge University Press.
Fisk, A. D., & Schneider, W. (1984). Memory as a function of attention, level of processing,
and automatization. Journal of Experimental Psychology, Learning, Memory, and Cognition,
10, 181–197.
Gevins, A., Smith, M. E., Le, J., Leong, H., Bennett, J., Martin, N., et al. (1996). High resolution
evoked potential imaging of the cortical dynamics of human working memory. Electroen-
cephalography and Clinical Neurophysiology, 98, 327–348.
Gevins, A., Smith, M. E., Leong, H., McEvoy, L., Whitfield, S., Du, R., et al. (1998). Monitoring
working memory load during computer-based tasks with EEG pattern recognition meth-
ods. Human Factors, 40, 79–91.
Gevins, A., Smith, M. E., McEvoy, L., & Yu, D. (1997). High-resolution EEG mapping of corti-
cal activation related to working memory: Effects of task difficulty, type of processing,
and practice. Cerebral Cortex, 7, 374–385.
Downloaded by [University of Maastricht] at 13:40 11 June 2014

Gopher, D. (1992). The skill of attention control: Acquisition and execution of attention strat-
egies. In D. E. Meyer & S. Kornblum (Eds.), Attention and performance XIV (pp. 299–322).
Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
Haier, R. J., Siegel, B. V., Jr., MacLachlan, A., Soderling, E., Lottenberg, S., & Buchsbaum, M.
S. (1992). Regional glucose metabolic changes after learning a complex visuospatial/mo-
tor task: A positron emission tomographic study. Brain Research, 570, 134–143.
Hillyard, S. A., Hink, R. F., Schwent, V. L., & Picton, T. W. (1973). Electrical signs of selective
attention in the human brain. Science, 182, 177–180.
Kahneman, D. (1973). Attention and effort. Englewood Cliffs, NJ: Prentice Hall.
Kecklund, G., & Akerstedt, T. (1993). Sleepiness in long distance truck driving: An ambula-
tory EEG study of night driving. Ergonomics, 36, 1007–1017.
Klimesch, W., Russegger, H., Doppelmayr, M., & Pachinger, T. (1998). A method for the cal-
culation of induced band power: Implications for the significance of brain oscillations.
Electroencephalography and Clinical Neurophysiology, 108, 123–130.
Kok, A. (2001). On the utility of P3 amplitude as a measure of processing capacity.
Psychophysiology, 38, 557–577.
Kramer, A. F. (1991). Physiological metrics of mental workload: A review of recent progress.
In D. L. Damos (Ed.), Multiple task performance (pp. 279–328). Washington, DC: Taylor &
Francis.
Levendowski, D. J., Berka, C., Olmstead, R. E., & Jarvik, M. (1999, October). Correlations be-
tween EEG indices of alertness measures of performance and self-reported states while operating a
driving simulator. Paper presented at the 29th annual meeting of the Society for Neurosci-
ence, Miami, FL.
Levendowski, D. J., Berka, C., Olmstead, R. E., Konstantinovic, Z. R., Davis, G., Lumicao, M.
N., et al. (2001). Electroencephalographic indices predict future vulnerability to fatigue
induced by sleep deprivation. Sleep, 24(Abstract Suppl.), A243–A244.
Levendowski, D. J., Olmstead, R. E., Konstantinovic, Z. R., Berka, C., & Westbrook, P. R.
(2000). Detection of electroencephalographic indices of drowsiness in realtime using
a multi-level discriminant function analysis. Sleep, 23(Abstract Suppl. 2),
A243–A244.
Levendowski, D. J., Westbrook, P., Berka, C., Popovic, M. V., Pineda, J. A., Zavora, T. M., et al.
(2002). Event-related potentials during a test of working memory differentiate sleep
apnea patients from healthy subjects. Sleep, 25(Abstract Suppl.), A460–A461.
Makeig, S. (1993). Lapses in alertness: Coherence of fluctuations in performance and EEG
spectrum. Electroencephalography and Clinical Neurophysiology, 86, 23–35.
 Real-Time EEG Indexes of Cognition 169

Makeig, S., & Jung, T. P. (1995). Changes in alertness are a principal component of variance in
the EEG spectrum. Neuroreport, 7, 213–216.
Makeig, S., & Jung, T. P. (1996). Tonic, phasic, and transient EEG correlates of auditory
awareness in drowsiness. Brain Research, Cognitive Brain Research, 4, 15–25.
Mathiak, K., Hertrich, I., Rothe, S., Kincses, W. E., Lutzenberger, W., & Ackerman, H. (2002).
Involuntary crossmodal enhancement of preattentive auditory processing is selective in space.
Manuscript submitted for publication.
Miller, J. C. (1995). Batch processing of 10,000 h of truck driver EEG data. Biological Psy-
chology, 40, 209–222.
Mitler, M. M., Miller, J. C., Lipsitz, J. J., Walsh, J. K., & Wylie, C. D. (1997). The sleep of
long-haul truck drivers. New England Journal of Medicine, 337, 755–761.
Mitler, M. M., Westbrook, P., Levendowski, D. J., Ensign, W. Y., Olmstead, R. E., Berka, C., et al.
(2002). Validation of automated EEG quantification of alertness: Methods for early identification
of individuals most susceptible to sleep deprivation. Sleep, 25(Abstract Suppl.), A147–A148.
Moore–Ede, M. C. (1993). The twenty-four-hour society: Understanding human limits in a world
Downloaded by [University of Maastricht] at 13:40 11 June 2014

that never stops. Reading, MA: Addison–Wesley.

Parasuraman, R., Bahri, T., Deaton, J. E., Morrison, J. G., & Barnes, M. (1992). Theory and design
of adaptive automation in adaptive systems (Progress Rep. No. NAWCADIWAR–92033–60).
Warminster, PA: Naval Air Warfare Center, Aircraft Division.
Parasuraman, R., Molloy, R., & Singh, I. L. (1993). Performance consequences of automation
induced “complacency”. International Journal of Aviation Psychology, 3, 1–23.
Parasuraman, R., Mouloua, M., & Hilburn, B. (1999). Adaptive aiding and adaptive task allo-
cation enhance human–machine interaction. In M. W. Scerbo & M. Mouloua (Eds.), Auto-
mation technology and human performance: Current research and trends (pp. 119–123).
Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Parasuraman, R., Mouloua, M., & Molloy, R. (1996). Effects of adaptive task allocation on
monitoring of automated systems. Human Factors, 38, 665–679.
Polich, J. (2002). Neuropsychology of P3a and P3b: A theoretical overview. Advanced in
Electrophysiology in Clinical Practice and Research.
Polich, J., & Herbst, K. L. (2000). P300 as a clinical assay: Rationale, evaluation, and findings.
International Journal of Psychophysiology, 38, 3–19.
Pope, A. T., Bogart, E. H., & Bartolome, D. S. (1995). Biocybernetic system evaluates indices
of operator engagement in automated task. Biological Psychology, 40, 187–195.
Prinzel, L. J., Freeman, F. G., Scerbo, M. W., Mikulka, P. J., & Pope, A. T. (2000). A closed-loop
system for examining psychophysiological measures for adaptive task allocation.
International Journal of Aviation Psychology, 10, 393–410.
Raichle, M. E., Fiez, J. A., Videen, T. O., MacLeod, A. M., Pardo, J. V., Fox, P. T., et al. (1994).
Practice-related changes in human brain functional anatomy during nonmotor learning.
Cerebral Cortex, 4, 8–26.
Schneider, W., & Fisk, A. D. (1982). Concurrent automatic and controlled visual search: Can
processing occur without cost? Journal of Experimental Psychology: Learning, Memory, and
Cognition, 8, 261–278.
Schneider, W., & Shiffrin, R. M. (1977). Controlled and automatic human information pro-
cessing I: Detection, search, and attention. Psychological Reviews, 84, 1–66.
Singh, I. L., Molloy, R., & Parasuraman, R. (1993). Automation-induced “complacency”: De-
velopment of the complacency-potential rating scale. International Journal of Aviation Psy-
chology, 3, 111–121.
Smith, M. E., Gevins, A., Brown, H., Karnik, A., & Du, R. (2001). Monitoring task loading
with multivariate EEG measures during complex forms of human–computer interaction.
Human Factors, 43, 366–380.
 170 Berka et al.

St. John, M., Kobus, D. A., & Morrison, J. G. (2002). A multi-tasking environment for manipu-
lating and measuring neural correlates of cognitive workload. In Proceedings of the 2002
IEEE 7th Conference on Human Factors and Power Plants (pp. 7.14–7.14). New York: IEEE.
Sterman, M. B., & Mann, C. A. (1995). Concepts and applications of EEG analysis in aviation
performance evaluation. Biological Psychology, 40, 115–130.
Sterman, M. B., Mann, C. A., & Kaiser, D. A. (1992). Quantitative EEG patterns of differential
in-flight workload (Abstract from Soar ’92). Sepulveda, VA: Sepulveda VA Medical Center.
Torsvall, L., & Akerstedt, T. (1987). Sleepiness on the job: Continuously measured EEG
changes in train drivers. Electroencephalography and Clinical Neurophysiology, 66, 502–511.
Westbrook, P., Berka, C., Levendowski, D., Lumicao, M. N., Davis, G., Olmstead, R. E., et al.
(2002). Biobehavioral quantification of alertness and memory in patients with sleep
apnea. Sleep, 25(Abstract Suppl.), A49–A50.
Wickens, C. D. (1984). Engineering psychology and human performance. Columbus, OH: Merrill.
Wickens, C. D. (1991). Processing resources and attention. In D. L. Damos (Ed.), Multiple-task
performance (pp. 1–34). London: Taylor & Francis.
Downloaded by [University of Maastricht] at 13:40 11 June 2014

Wickens, C. D. (1992). Engineering psychology and human performance (2nd ed.). New York:
HarperCollins.
Wickens, C. D., & Holland, J. G. (1984). Engineering psychology and human performance (3rd
ed.). Englewood Cliffs, NJ: Prentice Hall.