You are on page 1of 4

32nd Annual International Conference of the IEEE EMBS

Buenos Aires, Argentina, August 31 - September 4, 2010

Toward BCI Wizard - best BCI approach for each user


Ivan Volosyak, Christoph Guger and Axel Gräser

Abstract— Modern brain-computer interface (BCI) systems However, each lab tends to focus on only one BCI
use different types of neural activity for control. Most BCI approach and it still remains unclear which approach is best
systems only allow the customization of very few parameters suited for the individual user. To date, the first and only study
and focus only on one type of BCI approach. Many articles
reported that a certain BCI did not work for some users that explored different mental strategies with the same users
(so called BCI illiteracy). We are introducing the BCI wizard was done by Birbaumer and colleagues in collaboration with
as a system that automatically identifies key parameters to the Wadsworth Center in Albany, New York [11]. The cited
customize the best BCI paradigm for each user. With a BCI work compared three non-invasive approaches (SCP, P300,
wizard it is possible to develop an interface that relies on the and SMR) with each user in their home environment. Eight
best mental strategy for each user and therefore makes the
difference between an ineffective system and a working BCI. severely paralyzed patients with amyotrophic lateral sclerosis
This work presents a preliminary study that aims to develop a (ALS) participated in the study. All subjects received 20
BCI wizard exploring the two most effective BCI approaches sessions of SMR training, six had ten sessions of P300
(SSVEP and P300). These types of non-invasive BCIs were training and in addition, seven participated in 20 sessions
tested and evaluated in a group of 14 healthy subjects. During of SCP training. Results showed that SCP performance was
online tests all subjects were asked to spell three words with
two spelling applications and at the end of the experiment they not sufficient for communication after 20 sessions for any of
chose their preferred approach. Results showed that all subjects the seven subjects. In contrast, SMR training was successful
could communicate with the P300-based BCI with an accuracy for half of the subjects. P300 was the best with most subjects.
above 69% (5 reached 100% accuracy), 10 out of 14 subjects These results suggest that the P300-based BCI might be the
could effectively use the SSVEP-based BCI (2 reached 100% most efficient BCI for ALS patients [12]. This study also
accuracy). These promising results confirm that BCI wizard will
enable BCIs customized to each user with considerably greater confirms that for each BCI approach, some subjects could
flexibility and independence than present systems allow. not produce the neuronal activity needed for control. Other
studies reveal that about 10-25% of users are unable to attain
I. I NTRODUCTION
effective control [13]. The motor imagery BCI system was
A brain-computer interface (BCI) allows communication also tested with 99 subjects at an exhibition in Graz [14].
and control without requiring limb articulation or any move- The subjects were trained for 6 minutes to imagine left or
ment from the user. The normal communication channels right hand movement for a few seconds (20 times each) to
such as speech and movement are not used, but instead brain produce SMR changes. The BCI system was then trained
activity is directly recorded and transformed into control on the individual EEG data for a subsequent session with
signals [1]. There exist different neural activities that could visual feedback of cursor movement. The subjects were
be used as features for a BCI. Slow cortical potentials (SCPs) able to control the cursor to the left or right side of the
constitute one of the first non-invasive BCI approaches, computer screen. 6.2% of the subjects were able to learn
which has been used until very recently [2]. Motion-onset this control with 100% accuracy in this short training session.
visual evoked potentials (mVEPs) [3], auditory evoked po- About 93.3% showed a control above 59% accuracy (50%
tentials (AEP) [4], [5], and steady-state movement related corresponds to random classification). The P300 speller was
potentials (ssMRPs) [6] are modern examples of sources tested with 100 subjects with a row/column (RC) and a single
also used in BCIs. The commonly used and effective ap- character (SC) speller [15]. 72.8% (81 subjects) were able
proaches among non-invasive BCIs are the steady state visual to spell with 100% accuracy in the RC paradigm and 55.3%
evoked potential (SSVEP) [7], [8], the 300-ms component (N=38) spelled with 100% accuracy in the SC paradigm.
of an evoked potential (P300) [9], and sensorimotor rhythms Less than 3% of the subjects did not spell any character
(SMR) [10]. These approaches have been studied extensively correctly. The SSVEP based Bremen-BCI system was tested
in healthy participants and subjects with disabilities, and have with 37 subjects at an exhibition in Düsseldorf [16]. 86% (32
been validated as input signals that can be used in a BCI. subjects) were able to complete spelling tasks successfully.
I. Volosyak* and A. Gräser are with the Institute of Automation (IAT), Therefore, flexible and reliable BCIs systems that can
University of Bremen, 28359 Bremen, Germany, www.iat.uni-bremen.de automatically identify and optimize important parameters not
C. Guger is with g.tec medical engineering GmbH, Guger Technologies only for some specific BCI paradigm, but across different
OEG, 8020 Graz, Austria, www.gtec.at
This document is a collaborative effort of Institute of Automation, Uni- BCI approaches are needed. The building of a system that
versity of Bremen, Germany and g.tec Medical Engineering GmbH, Austria. performs a series of tests with the user with parameter
This research was supported within the 7th European Community Frame- adaptation within each paradigm and the incorporation of
work Program by a Marie Curie European Re-Integration Grant RehaBCI,
and by the EU ICT grants BRAIN, SM4ll and Presenccia. *Corresponding advanced and flexible signal processing tools may help to
author is Ivan Volosyak, volosyak@iat.uni-bremen.de develop a BCI best suited for the individual user.
978-1-4244-4124-2/10/$25.00 ©2010 IEEE 4201
II. M ETHODS AND M ATERIALS
The present work constitutes the first effort in building
FPZ
an expert system that walks the user through a series of
tests to determine optimal parameters across different BCI
approaches. We called this system the BCI wizard. For each Fz
test subject, the wizard will determine the appropriate spatial
filters, electrode sites, and further optimal parameters. For
example, for the P300 paradigm the wizard will determine Cz

the best inter-stimulus timing, stimulus duration, matrix size A2


and number of sequences. Different stimulus frequencies,
P3 Pz P4
time segment lengths, and display parameters will be as-
sessed for SSVEP BCIs. Since these two approaches are PO 7 PO8

very reliable and exhibited higher information throughput Oz


than other approaches, there was no reason to pursue other
approaches in this preliminary study.
Additionally, the user may also choose to answer questions Fig. 1. The electrode montage used in the current study. The 10 used
about training, application parameters and other personal electrode sites are indicated by the dotted circles; the FP Z was used for
preferences. Based on this information, an expert system will ground (circled in black) and right ear lobe A2 for the reference electrode
according to the system of electrode placement [18].
then determine the best BCI approach and parameters for
each subject. A BCI wizard will enable BCIs customized to times at the beginning of the experiment, the minimum
each user with considerably greater flexibility and indepen- necessary number of electrodes was used. Data were ref-
dence than present systems allow. erenced to the right earlobe with a ground at site F P z. This
The principal goal of the present study was to compare and configuration was used for both approaches, but channels
evaluate the most effective BCI approaches and to identify used were utilized depending on the specific mental strategy
key parameters that can be adjusted for each user to improve as follows: Fz , Cz , P3 , Pz , P4 , P O7 , Oz and P O8 for P300
BCI performance. The comparison between the different BCI and P3 , Pz , P4 , P O7 , Oz and P O8 for SSVEP. Figure 1
approaches was done by measuring the information transfer shows the electrode arrangement used in this study. Standard
rate (ITR). This measure is a standard method for measuring abrasive electrolytic electrode gel was applied between the
communication and control that incorporates speed and ac- electrodes and the skin in order to bring impedances below
curacy in a single value [17]. This study also assessed with 5kΩ. An EEG amplifier g.USBamp (Guger Technologies,
brief questionnaires subjective information such as which Graz, Austria) was used and the sampling frequency was
paradigm was preferred by the user. This is very important, 128Hz for SSVEP and 256Hz for P300 approach. During
because when a user e.g. dislikes stimuli that flash more the EEG acquisition, an analog bandpass filter between 2
quickly than twice per second, the BCI wizard rules out and 30Hz, and a notch filter around 50Hz (mains frequency
an SSVEP approach, and instead recommends the next best in Europe) were applied directly in the amplifier.
approach, for example, a P300 BCI. Or e.g. if the user had
C. Experimental protocol
either been willing to undergo training, or shown very strong
SMR activity, an SMR BCI might have been recommended 1) P300: The experiment started with the P300 session,
instead. which consisted of the presentation of a 6 × 6 row-column
(RC) matrix containing letters and numbers (see Figure 2).
A. Subjects The individual rows and columns flashed rapidly in a random
A total of 14 subjects participated in the study. Subjects sequence, such that a row or column was never followed by
mean age was 30.29 years, range 22-41 with standard devia- the same row and column. In one sequence of intensifications
tion of 4.78. Five subjects were female and nine male. None the target character flashed twice, once in a column and once
of the subjects have had any prior experience with P300 BCI. in a row. One character epoch consisted of 15 flashes of each
All subjects completed six spelling tasks across the P300 and row and column. Between two characters there was an in-
SSVEP approaches during one day (P300 first, followed by terval of 6 seconds where the next character was highlighted
SSVEP). Subjects were seated approximately 60cm from the in the matrix so that the person was able to find it quickly.
LCD screen (15.4” with resolution of 1280 x 800 pixels) of This is an advantage for new users who are unfamiliar with
the notebook running the BCI software. The entire session the P300 matrix. The rows and columns were intensified for
took on average about 40 minutes per subject. Before each 100ms with 60ms breaks between intensifications.
session started, the experimenter explained to the subject the Subjects were instructed to silently count flashes that
task to perform. included the letter or symbol that they wanted to select, while
ignoring other flashes. Only the row and column that contain
B. Hardware the desired letter evoked large P300 potentials. An initial
After completing a brief questionnaire each subject was run consisted of spelling the word WATER. These data were
prepared for EEG recording. In order to reduce preparation recorded and used to calibrate the system. The P300 system
4202
Start

Cursor: N

SNR
Target: N

A B C D E F
Threshold 170

SubjectNr 1
Accuracy: 100.00
Please spell
BRAIN Next word Word OK
ITR: 50.63 bpm

G H I J K L Clr L Del
F T J
M N O P Q R Q V M I O K .
, U A N E S D C
S T U V W X G P W R B Y -
X H Z
Y Z 0 1 2 3

4 5 6 7 8 9
BRAIN

Fig. 2. P300 paradigm presented on a LCD screen. Fig. 3. SSVEP speller application presented on a LCD screen.

TABLE I
D ETAILED RESULTS OVER 14 SUBJECTS .
collects EEG data for each flashing character and extracts an
interval 100 ms before the flash and 700 ms after the flash. Subject SSVEP paradigm (A) P300 paradigm (B) Preference
Then for each character a buffer is filled up with each flash. # Trials ACC ITR Trials ACC ITR Paradigm
After all characters were shown the EEG data is classified. A [-] [-] [%] [bpm] [-] [%] [bpm] [A, B]
linear discriminant analysis was applied to derive the EEG 1 34 100.00 57.98 13 100.00 10.77 A
weighting parameters for classification. The individualized 2 - - - 11 100.00 10.77 B
classifier was used for the next three runs where the user 3 - - - 12 83.33 7.64 B
was instructed to spell the words BCI and BRAIN (copy 4 88 77.27 12.72 15 93.33 9.32 A
5 - - - 13 84.62 7.84 B
spelling), and the third word was free and chosen by the
6 47 95.74 23.66 12 91.67 9.02 A
user (free spelling). Misspellings should be ignored, as far
7 35 100.00 31.09 13 69.23 5.63 A
as no correcting option is available in the current P300 matrix 8 - - - 14 100.00 10.77 B
(i.e. all letters of the word should appear in the spelled text). 9 54 87.04 15.22 12 83.33 7.64 B
The complete P300 software was provided by g.tec Medical 10 46 95.65 26.91 13 92.31 9.13 A
Engineering GmbH, Austria. This RC-system was already 11 55 87.27 23.95 14 85.71 8.01 A
tested within 81 subjects, 72.8% were able to spell with 12 52 96.15 20.43 14 92.86 9.23 A
100% accuracy (for more details about this study see [15]). 13 43 88.37 11.93 12 100.00 10.77 A
2) SSVEP: SSVEP patterns are elicited in a visual at- 14 37 91.89 24.36 12 100.00 10.77 A
Mean 49.10 91.94 24.83 12.86 91.17 9.09
tention task in which users focus their attention on light
S.D. 15.65 7.07 13.21 1.10 9.10 1.60
sources that flicker with frequencies above 5Hz. Figure 3
shows the display used for the evaluation of this paradigm.
structed to complete the second session following the P300
The centre of the display contained 32 characters (letters
session. The task was to spell exactly the same three words
and other special characters). At the beginning of each run,
as in the previous session. Misspellings should be corrected
a cursor was presented over the “E” character in the middle.
with the ‘Del’ option located at the top-right of the matrix.
Subjects spelled three words by focusing on one of the
five boxes presented on the screen. Each box contained
either an arrow (indicating left, right, up, or down) or the III. R ESULTS
word “Select”. Each box oscillated at a different constant
frequency: 7.5Hz, 8.57Hz, 10Hz, 12Hz and 6.67Hz to encode Table I shows the results from 14 subjects. The paradigms
the command: left, right, up, down and select, respectively. were compared by measuring number of trials, accuracy and
These frequencies, and the best arrangement of characters in information transfer rate. Trials correspond to the number
the display, were determined through prior work [19]. The of correct and incorrect selections. Accuracy was calculated
complete SSVEP system has been developed at the Institute as the ratio between the number of correct selections or
of Automation, University of Bremen and evaluated under hits and the sum of the number of targets or trials. The
real world conditions [16]. During this prior study, spelling information transfer rate (ITR) represented the amount of
tasks were successfully completed by 32 participants with information communicated per unit time. The number of
a mean accuracy of over 92% and an average information possible choices was different for each BCI approach. In the
transfer rate of 22.6 bits/min. EEG signals are processed P300 paradigm, the number of choices corresponded to the
to find signal-to noise ratios (SNR) every 100ms for each number of elements in the matrix (N = 36). In the SSVEP
frequency. The SSVEP detection uses a time window of 2 paradigm, N was the number of flickering boxes available
seconds of EEG data to classify the incoming signals. If (N = 5). Additionally, the last column of Table I shows the
the SNR at a specific frequency exceeded a predetermined answer to the question about preferred paradigm. Subjects
threshold, the corresponding command was executed. This had to choose one of the paradigms (A or B) based on their
system does not require any calibration. Subjects were in- personal experience.
4203
IV. C ONCLUSIONS AND F UTURE WORK learn the time behavior of specific BCI application and could
react foresighted, what is definitely needed e.g. for play-
A. Conclusions
ing pinball with non-invasive BCI [20]) could considerably
Direct comparison can be done between paradigms A and shorten the response times of the BCI system. Additional
B because the words spelled were exactly the same. By tests should be done to analyze the possible correlations
analyzing the number of trials it can be seen that the number between different BCI paradigms (e.g. high ITR with SSVEP
of commands required to complete the task was different BCI could correspond to a good response to the ssMRPs and
depending on the approach. For example, for the word “BCI” vice versa).
the number of commands was 9 using the SSVEP BCI,
R EFERENCES
while using the P300 BCI the number of commands was
3. Accuracies between 77.27 and 100% were achieved for [1] G. Dornhege, J. del R. Millan, T. Hinterberger, D. J. McFarland, and
K.-R. Müller, Toward Brain-Computer Interfacing. MIT Press, 2007.
the SSVEP paradigm, while with the P300 paradigm the [2] N. Birbaumer, “Slow cortical potentials: Plasticity, operant control,
accuracy ranged between 69.23 and 100%. and behavioral effects,” Neuroscientist, vol. 5, no. 2, pp. 74–78, 1999.
The P300 information transfer rate was calculated without [3] F. Guo, B. Hong, X. Gao, and S. Gao, “A Brain-Computer Interface
using motion-onset visual evoked potential,” J. Neural Eng., vol. 5,
considering the break between the single characters that no. 4, pp. 477–485, 2008.
were spelled, because it was used to show the person the [4] S. Kanoh, K.-I. Miyamoto, and T. Yoshinobu, “A brain-computer
position of the next character. Trained users can reduce this interface (BCI) system based on auditory stream segregation,” in Proc.
30th Annual International Conference of the IEEE Engineering in
time period significantly. It must also be mentioned that the Medicine and Biology Society EMBS 2008, 20–25 Aug. 2008, pp.
P300 speller used here was not optimized for speed and was 642–645.
not individually adjusted to each person. We used the same [5] J. Guo, B. Hong, F. Guo, X. Gao, and S. Gao, “An Auditory BCI Using
Voluntary Mental Response,” in Proc. 4th International IEEE/EMBS
settings as in the study performed by Guger et al. [15] to Conference on Neural Engineering NER 09, May 2009, pp. 455–458.
be able to compare the results (88.9% reached 80-100% [6] K. Nazarpour, P. Praamstra, R. C. Miall, and S. Sanei, “Steady-state
accuracy and 6.2% reached 60-79%). In this study 92.8% movement related potentials for brain-computer interfacing,” IEEE
Trans. Biomed. Eng., vol. 56, no. 8, pp. 2104–2113, Aug 2009.
were able to reach accuracy between 80-100% and 7.14% [7] O. Friman, I. Volosyak, and A. Gräser, “Multiple channel detection
reached an accuracy of 60-79%. This shows that the results of steady-state visual evoked potentials for brain-computer interfaces,”
of both studies are very similar. IEEE Trans. Biomed. Eng., vol. 54, no. 4, pp. 742–750, Apr. 2007.
[8] Y. Wang, X. Gao, B. Hong, C. Jia, and S. Gao, “Brain-computer
Four subjects exhibited weak SSVEP and therefore could interfaces based on visual evoked potentials,” IEEE Eng. Med. Biol.
not communicate using the SSVEP-based BCI, while all Mag., vol. 27, no. 5, pp. 64–71, 2008.
[9] U. Hoffmann, J.-M. Vesin, T. Ebrahimi, and K. Diserens, “An effi-
subjects could spell using the P300-based BCI. Subjects that cient P300-based brain-computer interface for disabled subjects,” J.
could not control the SSVEP system consequently chose the Neurosci. Methods, vol. 167, no. 1, pp. 115–125, Jan 2008.
P300 as favored approach, only one subject who was able to [10] G. Pfurtscheller and F. Lopes da Silva, “Event-related EEG/MEG
synchronization and desynchronization: Basic principles,” Clin. Neu-
spell with the SSVEP paradigm selected P300 as preferred rophysiol., vol. 110, pp. 1842–1857, 1999.
approach. ITR values for the SSVEP paradigm were higher [11] N. Birbaumer and L. G. Cohen, “Brain-computer interfaces: commu-
than for the P300 paradigm, which suggest it was the nication and restoration of movement in paralysis,” J. Physiol., vol.
579, no. 3, pp. 621–636, 2007.
reason why most subjects chose the SSVEP paradigm as the [12] T. Hinterberger et al., Brain-Computer Interfaces for Communication
preferred paradigm. The other reason could be the feedback in Paralysis: A Clinical Experimental Approach. Cambridge, Mas-
type - in case of SSVEP every command classification was sachusetts: The MIT Press, 2007, ch. 3, pp. 41–64.
[13] A. Nijholt and D. Tan, “Brain-Computer Interfacing for Intelligent
followed by the visual feedback (highlighting actual letter Systems,” IEEE Intell. Syst., vol. 23, no. 3, pp. 72–79, 2008.
background), additionally auditory signals played the name [14] C. Guger, G. Edlinger, W. Harkam, I. Niedermayer, and G. Pfurtschel-
of the received navigation command or selected character ler, “How Many People are Able to Operate an EEG-Based Brain-
Computer Interface (BCI)?” IEEE Trans. Neural Syst. Rehabil. Eng.,
simultaneously with cursor movements or letter selections. vol. 11, no. 2, pp. 145–147, Jun 2003.
For the P300 the character selection was performed silently [15] C. Guger, S. Daban, E. Sellers, C. Holzner, G. Krausz, R. Carabalona,
only after all characters flashed up 30 times (15 repetitions F. Gramatica, and G. Edlinger, “How many people are able to control
a P300-based brain-computer interface (BCI)?” Neurosci. Lett., vol.
for each row and column), this type of BCI gave feedback 462, no. 1, pp. 94–98, Jun 2009.
much more rarely. Therefore, the results of this study need [16] I. Volosyak, H. Cecotti, D. Valbuena, and A. Gräser, “Evaluation of the
further investigation with optimized speller applications and Bremen SSVEP based BCI in real world conditions,” in Proc. IEEE
ICORR’09, Jun. 2009, pp. 322–331.
with a larger number of subjects. [17] B. Dal Seno, M. Matteucci, and L. Mainardi, “The Utility Metric: A
Novel Method to Assess the Overall Performance of Discrete Brain-
B. Future work Computer Interfaces,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 18,
pp. 20–28, 2010.
The further BCI research shall be extended to the deve- [18] R. Oostenveld and P. Praamstra, “The five percent electrode system
lopment of friendly, straightforward wizard that will walk for high-resolution EEG and ERP measurements,” Clin. Neurophysiol.,
each user through a series of tests to determine optimal vol. 112, no. 4, pp. 713–719, Apr 2001.
[19] I. Volosyak, H. Cecotti, and A. Gräser, “Impact of Frequency Selection
parameters. The comparatively shortened tests with the BCI on LCD Screens for SSVEP Based Brain-Computer Interfaces,” in
wizard will allow a rapid choice of the most suitable BCI IWANN 2009, Part I, LNCS 5517. Springer, 2009, pp. 706–713.
paradigm for each user. Nevertheless, training still plays [20] M. Tangermann, M. Krauledat, K. Grzeska, M. Sagebaum, C. Vi-
daurre, B. Blankertz, and K.-R. Müller, “Playing Pinball with Non-
an important role for the brain-machine interaction. The Invasive BCI,” in Neural Information Processing Systems (NIPS),
inclusion of the cognitive abilities of the users (subject can 2009.

4204

You might also like