IEEJ TRANSACTIONS ON ELECTRICAL AND ELECTRONIC ENGINEERING

IEEJ Trans 2018

Published online in Wiley Online Library (wileyonlinelibrary.com). DOI:10.1002/tee.22742

Paper

BCI-Based Control of Electric Wheelchair Using Fractal Characteristics of

EEG
Nobuaki Kobayashi*a , Member
Masahiro Nakagawa** , Non-member

Brain–computer interfaces (BCIs) have been attracting attention as interfaces that connect a brain to an external device. The
implementation of a BCI requires establishing a technique that accurately recognizes the brain state, and a host of challenges
remains for the implementation. This paper uses the emotion fractal analysis method (EFAM), which quantifies emotions based
on data obtained from an electroencephalogram (EEG), to propose a BCI system that accurately discriminates and recognizes
emotions such as delight, anger, sorrow, and pleasure, and uses that information to manipulate an electric wheelchair. The
EFAM’s emotion recognition rates for the four emotions, namely delight, anger, sorrow, and pleasure, were 81.11, 79.25, 73.16,
and 85.42%, respectively, and its emotion isolation rates were 97.88, 98.08, 98.07, and 97.86%, respectively. Based on emotion
data obtained in real time by the EFAM, we developed a novel BCI circuit that manipulates an electric wheelchair. Using this
BCI circuit allows us to adjust the speed of an electric wheelchair in proportion to the intensity of the emotion. © 2018 Institute
of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Keywords: electroencephalogram (EEG); fractal dimension; emotion fractal analysis method; brain–computer interface (BCI)

Received 18 October 2017; Revised 9 March 2018

1. Introduction
The establishment of brain–computer interfaces (BCIs) that
enable users to move equipment based solely on their biologi-
cal information from electroencephalography without using their
limbs by connecting a brain to an external device is desired. Using
BCIs allows us to support people with amyotrophic lateral sclerosis
(ALS), people with paralysis due to brain lesions, or elderly peo-
ple who face difficulties moving on their own. Our research aims
are set out below. A small-size, wearable electroencephalograph is
placed on the head of a person in need of nursing care, such as one
with ALS or a brain lesion, to keep monitoring her/his electroen-
cephalograms (EEGs). Based on the EEGs, the emotions of the per-
son are analyzed using an emotion analysis method later described
in this paper. Based on the emotion thus analyzed, a wheelchair,
home electronics, or electronic equipment is automatically manipu-
lated. This emotion analysis allows us to check through a monitor
how the person in need of nursing care feels, and what she/he
wants, even if she/he has difficulty communicating with her/his
caregiver by reason of ALS or paralysis due to brain lesion. The
first advantage of this approach is that it helps in reducing the
workload of caregivers. In other words, reducing human costs can
lead to the resolution of the labor shortage in an aging society with
a declining birth rate. The second advantage is that it is expected to
improve the quality of life (QoL) of people in need of nursing care.
This study aims to support an aging society by realizing this con-
cept. This paper proposes a novel form of a BCI system that ana-
lyzes emotions based on human EEGs and manipulates an electric
wheelchair according to the estimated emotion, in order to confirm
that we are indeed on a right path toward realizing this possibility.
a Correspondence to: Nobuaki Kobayashi. E-mail:
kobayashi.nobuaki@nihon-u.ac.jp
* College of Science and Technology, Nihon University, 7-24-1 Narashin-
odai Funabashi-shi, Chiba 274-8501, Japan
** Faculty of Engineering, Nagaoka University of Technology, 1603-1
Kamitomioka Nagaoka-shi, Niigata 940-2188, Japan
Figure 1 shows a standard BCI system [1]. A BCI system refers
to an interface that measures brain functions, extracts features
from biological information thus measured, and generates some
kind of device control signals using these features to manipulate
an external device. Noninvasive techniques for measuring brain
function include functional magnetic resonance imaging (fMRI),
positron emission tomography, magnetoencephalography (MEG),
and electroencephalography; an EEG is used most often owing
to its lower constraints and cost variations [1–6]. A BCI control
system can be developed by extracting features from the EEG
signals using some kind of technique and developing a novel
algorithm that converts these features. Concerning wheelchair
control through BCI, systems using motor imagery EEGs [7],
SSVEP (steady-state visual evoked potential) [8], and P300s
have been proposed previously. To date, however, there are no
examples using emotions. The advantages of applying emotions
to machine controls are many; for example, where detecting
feelings of fear in the user, it can apply an emergency stop to
the wheelchair, and by considering that, these things contribute to
ergonomic aspects. From a safety standpoint, there are measures
to prevent malfunctions in the relatively large machine controls of
wheelchairs; however, because of the characteristics of BCI, even
in the methods proposed so far it has been difficult to achieve
100% recognition rate. Therefore, it is important to construct safer
systems by combining multiple methods. In this paper, we show
the utility of wheelchair controls through emotion in order to
investigate the possibilities of using emotions in BCI development.
Musha et al. proposed a typical feature extraction technique
called the emotion spectrum analysis method (ESAM) [9,10]. The
ESAM is reported to have a recognition rate of around 50% with
regard to the isolation of four types of emotions [9]. This paper
proposes the emotion fractal analysis method (EFAM), which
quantifies emotions. EFAM can accurately recognize affects (such
as comfortable and uncomfortable) and emotions (such as delight,
anger, sorrow, and pleasure) by quantifying emotions based on
biological information obtained via an EEG. What distinguishes
the EFAM from all other feature extraction methods the most

© 2018 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.
 N. KOBAYASHI AND M. NAKAGAWA
BCI System
Brain-signal Feature Translation
acquisition extraction algorithm
Device
commands
Computer,
wheelchair, etc.
Fig. 1. Example of a BCI system
is its ability to detect higher forms of emotion, such as affects
and emotions. As stated previously, the application of emotions
to a BCI (emotion BCI) allows people in need of nursing care to
manipulate devices around them at their own will without the help
of caregivers. This study focuses on a wheelchair using an emotion
BCI. Emotion is used to manipulate a wheelchair because it permits
implementation of functions that, for example, urgently halt the
device when the user feels fear. In this study, we developed a novel
BCI control circuit that can manipulate an external device based
on an emotion identified by the EFAM in order to demonstrate
such a possibility. Using this circuit allows us to adjust the speed
of an electric wheelchair in proportion to the intensity of emotion;
that is, it allows us to control the speed of a wheelchair depending
on the degree of emotion intensity.
This paper gives a detailed account of the fundamental char-
acteristics of EFAM and a speed-adjustable electric wheelchair
system to which the proposed BCI circuit is applied.
2. Emotion Fractal Analysis Method
2.1. Acquisition of potential difference signals
between electrodes First, the EEGs used for learning were
measured using electrodes placed on the scalp. EFAM uses the
difference signals between electrodes for EEGs. The difference
signals between electrodes coincide with potential differences
between the electrodes. Second, using features developed from
reference electroencephalographic data thus measured, classifiers
necessary for discriminating emotions (emotion matrix C , bias
vector d ) were determined. In the end, features calculated for
assessment from electroencephalographic data with unknown
emotion state and features calculated from reference electroen-
cephalographic data beforehand were matched to estimate the
emotion. The most distinct feature of this approach is that it uses
the fractal dimension for difference signals.
2.2. Calculating features: Scaling property of vari-
ance (SPV) A feature was calculated in terms of the fractal
dimension for difference signals of the EEGs. A fractal dimension
estimation approach based on the SPV was used to calculate the
fractal dimension [11]. Let f (t) denote a time series dataset where
the fractal dimension is D q . Then, the q-dimensional moment for
a dataset f (t + τ ), which is separated in time from the time series
data set f (t) by τ , can be expressed by the following formula:
σq (τ ) = |f (t + τ ) − f (t)|q 
Here, <• > denotes statistical average. Suppose that a statistical
distribution function for a variation in f (t) X = f (t + τ ) – f (t)
can be given by a stable distribution, like a Gaussian distribution;
dependence of the Husrt exponent H q on the order of moment q
reflects a fractal property. In (1), as τ changes, a scaling property
expressed in the following formula can be obtained between σ q (τ )
and q; hence, the generalized Hurst exponent be calculated.
1 log|f (t + τ ) − f (t)||f (t + τ ) − f (t)|q 
Hq = (2)
q log|τ |
At this moment, the fractal dimension D q (generalized self-
affine dimension) is given in the following formula:
Dq = d + 1 − Hq (3)
Here, d denotes the dimensionality of the space.
2.3. Emotion estimation based on classifiers: Deter-
mination of emotion matrix C and bias vector d
Figure 2 shows an example of analyzing the fractal dimension for
EEGs as an example of calculating a time series-dependent fea-
ture. Figures 2(a)–(c) show electroencephalographic signals for a
time series, an example of fractal dimension estimation in terms of
SPV, and a time series-dependent fractal dimension obtained from
the electroencephalographic signals, respectively. Figure 2(a) is an
EEG signal when resting for 70 s and concentrating for 230 s. Mea-
surement points are F 3 , based on the International 10–20 system
(Fig. 3). At this time, the Hurst exponent is calculated using (2)
and obtained from the slope of the scaling characteristic shown
in Fig. 2(b). Figure 2(b) shows the scaling characteristic when
extracted in a 30-s rest interval). In practice, the Hurst exponent is
calculated from the slope at the limit τ → 0. Where dimensionality
q = 2, the time series data of the distribution of the scaling charac-
teristics is obtained, and from this distribution of scaling character-
istics, the self-affine fractal dimension is estimated. The estimated
value of the self-affine fractal dimension D q is obtained from (3).
Specifically, SPV was applied to the rectangular window
data with the analysis window length W to calculate the fractal
dimension D q . The rectangular window was then slid by the
window sliding width W s to calculate the fractal dimension at
the next time-point. By repeating this procedure, temporal change
in the fractal dimension against the original signal,Fig. 2(a), can
be obtained, Fig. 2(c). In Fig. 2(a) it is difficult to distinguish
the EEG signals at rest and during concentration. However, by
converting to the fractal dimension estimate value D q shown in
Fig. 2(c), even at the visual level it can be seen that a definite
difference between resting and concentration values D q , is
generated. Classifiers were defined using features of these fractal
dimensions. Let y denote a time series of a feature used for
learning, and let C denote an emotion matrix that maps y onto
the emotion output z ; and assume the linear relation shown in (4)
is obtained. Here, d denotes a bias vector.
z = Cy + d (4)
As regards the y used specifically for this analysis, for any com-
bination of electroencephalographic signals from the electrodes,
the fractal dimension time series against difference signals between
electrodes were used, as stated in Section 2.1. A difference sig-
nal between two electrodes coincides with a potential difference
between them, and if t denotes a time point, x i (t) denotes an
input from the i th electrode and y ij (t) denotes a potential differ-
ence between the i th electrode and j th electrode; then the potential
difference is given by
(1) yij (t) = xi (t) − xj (t) (5)
2 IEEJ Trans (2018)
 BCI-BASED CONTROL OF ELECTRIC WHEELCHAIR

100 101

50

Voltage (μV)

log σq (τ)
100
2
10–1
–50
Rest Concentration
–100 10–2 –5
0 50 100 150 200 10 10–4 10–3 10–2 10–1 100
Time (s) log τ
(a) (b)
2.0

Fractal dimension
1.7

1.4

1.1

0.8
Rest Concentration
0.5
0 50 100 150 200
Time (s)
(c)
Fig. 2. Example of fractal dimension analysis of brain waves. (a) Brain wave signal. (b) Scaling property of variance (SPV). (c) Time-
dependent fractal dimension analysis

C and d that minimize <ε2 > are given in the following

formulas:

C = R zy (R yy ) − 1 (8)
Fp1 Fp2

F7 F8 d = z ∗  − R zy (R yy ) − 1y (9)
F3 Fz F4

 yy  
A1 Rj ,n = yj yn − yj yn 
T3 C3 Cz C4 T4 A2 (10)
Rmn = zm ∗ yn  − zm ∗ yn 
yy

Note that the row direction component of z corresponds to an

P3 P4 emotion, and suppose there were four kinds of emotions, namely
Pz
T5 T6 delight, anger, sorrow, and pleasure; the first row of z was mapped
onto ‘delight’ and the second row to ‘anger’. A teacher signal
O1 02 z = [1,0,0,0]T was given to a location that corresponds to the
fractal dimension when ‘delight’ was recalled, and z = [0,1,0,0]T
was given to a location that corresponds to the fractal dimension
when ‘angry’ was recalled. Here, T denotes transposition.
Fig. 3. Positioning of electrodes (International 10–20 system) Applying the above to the case of four emotions (delight, anger,
sorrow, pleasure), in the actual analysis, the subjects were asked
Applying SPV to N (= m C2 ) potential difference signals to think of each previously experienced emotion for 30 s, and this
between electrodes y ij (t) obtained from data from m channels, data was added as an instruction signal z i (i = 1,2,3,4) in (6).
we will get an N -dimensional, length T input signal vector y. Therefore, we measure two types of brainwaves, as learning
Equation (4) can be expressed using components as in the follow- data and evaluation data. As described above, the time series
ing formula: data of the fractal dimension is obtained by applying (2) and
(3) to the time series data of four channels (emotion number)

N
taking the difference between each channel for n channels of
zi ,t = Ci ,j yj ,t + di (6)
j =1
learning data z i . With this D(t) as input, using linear mapping
allows learning and recognition so as to give independent outputs
The subscripts i ,j for each parameter denote the row and column for each emotion. Where linear transformation is applied to the
components of a matrix, respectively, and the matrix size for each four-dimensional vector z = (z 1 ,z 2 ,z 3 ,z 4 )T using linear mapping
parameter is z = ET , C = EN , s = NT , d = E each. Here, E is C , (6) can be related through the following expression:
the number of emotions used for analysis, T is the discrete time ⎡ ⎤⎡ ⎤ ⎡ ⎤ ⎡ ⎤
length, and N is the number of feature vectors. C1,1 · · · C1,M D1 (t) d1 z1 (t)
⎢ .. .. .. ⎥ ⎢ .. ⎥ + ⎢ .. ⎥ = ⎢ .. ⎥
Let z * be a teacher signal, and ε be mean error; then the mean ⎣ . . . ⎦⎣ . ⎦ ⎣ . ⎦ ⎣ . ⎦ (11)
squared error can be given by the following formula:
C4,1 ··· C4,M DM (t) d4 z4 (t)
 E ⎧ ⎛ ⎞⎫2 
⎨ N ⎬ Here, M is the number of electrode channels. Thus, where the
ε  =
2
zi − ⎝
∗
Ci ,j yj − di ⎠ . (7) delight learning data is input, C and d are calculated such that
⎩ ⎭
i =1 j =1 z = [1,0,0,0]T , anger is z = [0,1,0,0]T , sorrow, z = [0,0,1,0]T , and

3 IEEJ Trans (2018)

 N. KOBAYASHI AND M. NAKAGAWA

pleasure z = [0,0,0,1]T . If C , d , and D(t) obtained from evaluation Table I. Average recognition rate of four emotions (delight,
data are input into (11), a vector of evaluative emotion output is anger, sorrow, and pleasure) (%)
obtained.
‘Emotions’ measured in the present approach denote ‘moods Pleasure Anger Sorrow Delight
or feelings’ based on the experience of each study subject rather Pleasure 81.11 6.6 6.14 3.44
than temporary emotions such as ‘joy’, ‘sadness’, and ‘annoyance’ Anger 8.95 79.25 10.34 4.52
experienced during their daily living. Sorrow 5.36 8.27 73.16 6.62
Delight 4.58 5.88 10.36 85.42
3. Characteristics of EFAM The numbers in bold indicate that the emotions from the recollection
(column) and the assessment (row) match.
3.1. Measuring conditions EEG measurements were
performed using a Polymate (Digitex Lab. Co., Ltd., Japan), a
biosignal recorder, and the EFAM performance was assessed by Table II. Average isolation rate of four emotions (delight, anger,
implementing a real-time EFAM analytical program that estimated sorrow, and pleasure) (%)
emotions in real time using C++. The measurement conditions
were as follows: sampling frequency of 2 kHz, the digital filter was Pleasure Anger Sorrow Delight
a 50 Hz notch filter, with a low-cut filter (LCF), a high-cut filter Pleasure 97.88 0.54 0.55 0.56
(HCF), and an anti-aliasing filter that had a cutoff frequency of Anger 0.79 98.08 0.76 0.76
600 Hz. The electrode sites conformed to the International 10–20 Sorrow 0.78 0.82 98.07 0.83
system, as shown in Figs. 3, and 19 electrodes were used. The Delight 0.55 0.56 0.62 97.85
reference electrode was placed on the right earlobe. The study
subjects were five healthy adult males. Informed consent was The numbers in bold indicate that the emotions from the recollection
obtained from subjects prior to the experiment. The subjects were (column) and the assessment (row) match.
aged 24.40 ± 4.506 years.
obtained at 0.25-s intervals. For the 30-s recall of the target emo-
3.2. Assessing recognition rates and isolation rates tion, for each attempt, a 120-element emotion output vector (four
Figure 4 shows the time table of the measurements. First, the dimensions) (maximum value of 1) was obtained. In each of the
study subjects were asked to recall four emotions (delight, anger, 120-element emotion output vectors, emotions exceeding a certain
sorrow, and pleasure) for 60 s each, and the results were used value (threshold value) were counted as emotions in the evaluation
as learning data. Second, the measured data were recorded. The data, the ratio of which was the recognition rate. On this occasion,
study subjects were asked to recall any one of the four emotions the threshold rate was 0.3. Therefore, in some cases there are
(delight, anger, sorrow, and pleasure) for 30 s, and then to recall instances where two or three emotions exceed the threshold values
the same emotion for 20 s. Subsequently, they were asked to recall and are output. Thus, among the 120-element emotion output
the same emotion for 30 s. These tasks were taken together as a vectors, by counting each case where only one emotion exceeds
set lasting 80 s, and two sets were performed for each of the four the threshold value and each case where two or three emotions
emotions. The analytical program used learning data as a reference exceed the threshold values, their ratio becomes the isolation rate.
and estimated the emotion of a study subject from his actual data. Table I shows the mean recognition rates to the evaluation
In addition, the subjects were not previously trained in recalling data (mean values of two trials) for the four emotions. The
emotions, but they were instructed to recall events from past expe- column of the table shows recollection and the row shows emotion
riences that had left a strong impression. The visual analog scale assessments. The mean recognition rates were all high, with the
(VAS) was implemented during each experiment. The VAS values figures for delight, anger, sorrow, and pleasure being 81.11, 79.25,
for each emotion were as follows: delight, 71.01 ± 4.506; anger, 73.16, and 85.42%, respectively. Furthermore, Table II shows the
82.94 ± 6.870; sorrow, 71.20 ± 8.207; and pleasure, 79.20 ± 6.853. mean isolation rates for the four emotions (mean values of two
The recognition rate was defined as the rate at which the trials). The mean isolation rates were extremely high, with the
program correctly identified the emotion of a study subject, and figures for delight, anger, sorrow, and pleasure being 97.88, 98.08,
the isolation rate was defined as the rate at which a single emotion 98.07, 97.86%, respectively, and all figures were greater than 97%.
was identified by the analytical program. Because the analysis As seen above, using the EFAM allows us to recognize and isolate
window in this analysis was W = 2000 (1 s), and the window tran- the four emotions very accurately.
sition width W s = 500 (0.25 s), the evaluation emotion output was

4. BCI-Based Electric Wheelchair Control System

Pleasure Delight Pleasure 4.1. Configuration of the proposed system Figure 5
shows an overview of the BCI-based electric wheelchair control
0 3 0 50 80 (s)
system that we are going to propose. This system consists of an
0 2 0 EEG element, an emotion discrimination element provided by real-
time EFAM (analytical software), a BCI circuit (hardware), and a
0 30 control target (electric wheelchair). In what follows, this system
Anger Delight Anger
will be referred to as the proposed system. The proposed system is
composed of a Polymate mentioned in Section 3, which measures
EEG; real-time EFAM software, which estimates the emotion; and
Sorrow Delight Sorrw the BCI control circuit, which generates an electrical signal based
on the estimated emotion, which in turn manipulates an electric
wheelchair. The details of the real-time EFAM were described
Delight Delight Delight
in Sections 2 and 3. We will provide a detailed account of the
operating principle of a moving wheelchair in Section 4.2, and the
Fig. 4. Flowchart of EEG measurement BCI control circuit in Section 4.3.

4 IEEJ Trans (2018)

 BCI-BASED CONTROL OF ELECTRIC WHEELCHAIR

v
Proposed BCI system Driving sytem (back wheel)
Left Right
Real-time EFAM 0°
Forward
Steering

+6
(km/h)
(V)
θ

5
–60° +60° system
Brain-signal Feature Translation
Front (front wheel)
acquisition extraction algorithm
(degree)
0
Left –60 Stop
+60 Right
θ
turn 0 2.5 5 turn
v (V)
+ –
BCI Back

–6
controller

0
Backward
Device (a) (b)
commands
Fig. 6. Operating principle of the electric wheelchair. (a) The
system of the front and back wheels. (b) The operational voltage

Electric
wheelchair BCI circuit (hardware)

A Control
Micro
Se singnal
controller
generator

Fig. 5. Diagram of BCI controlling the electric wheelchair system

4.2. Operating principle of the electric wheelchair
An EMC-150 (Imasen Engineering Corporation) was used as
the electric wheelchair. Figure 6 shows its operating principle.
As shown in Fig. 6(a), in this wheelchair, the front wheels
constitute the steering system, where the wheelchair direction can
be adjusted in terms of the rotation angles (−60◦ ≤ θ ≤ +60◦ )
of the wheels, with wheels on both sides performing the same
movement; (−) corresponds to a left turn, and (+) corresponds
to a right turn. Back wheels constitute the driving system, where
the speed and direction can be adjusted in terms of the rotation
speeds (−6 km/h ≤ v ≤ +6 km/h) of the wheels, with wheels on
both sides performing the same movement; (+) corresponds to a
forward movement, and (−) corresponds to a backward movement.
The two systems are both DC systems; the range of input for the
control voltage (C = (C d (driving system), C s (steering system)))
is 0 V ≤ C ≤ 5 V, and the movement is determined by the voltage
level. Figure 6(b) corresponds to a straight direction either forward
or backward. We illustrate this as follows: In the steering system,
if C s = 2.5 V, then θ = 0◦ ; and if C s > 2.5 V, θ rotates in the
(+) direction (right-turn), in proportion to the value of C s ; if
C s < 2.5 V, it rotates in the (−) direction (left-turn) in proportion
to the value of C s . Thus, for example, if C s = 0 V, then θ = −60◦ ,
and if C s = 5 V, then θ = +60◦ . On the other hand, in the driving
system, if C d = 2.5 V, then v = 0 km/h. This signifies that the
wheelchair is being stopped. Using this as reference, if C d > 2.5 V,
v rotates in the (+) direction (forward movement) in proportion to
the value of C d , and if C d < 2.5 V, it rotates in the (−) direction
(backward movement) in proportion to the value of C d . Thus, for
example, if C d = 0 V, then v = −6 km/h, and if C d = 5 V, then
v = +6 km/h.
USB PC
Real time Wheelchair
EFAM
(software)
Fig. 7. BCI circuit
(forward movement); if ‘anger’, it outputs ‘L’ (left-turn); if
‘sorrow’, it outputs ‘R’ (right-turn); if ‘pleasure’, it outputs ‘B’
(backward movement). In addition, it simultaneously outputs the
intensity of the emotion on a 0–100 scale. Here, the intensity
denotes the similarity between the estimated emotion and the
original reference emotion. In other words, it is a rate that denotes
the probable degree of matching between the EEG data given as
a teacher signal and the actually measured EEG. If, for example,
the estimated emotion is ‘delight’, and its intensity is a maximum,
i.e., if the EEG obtained for the reference exactly matches the
actually measured EEG, then S e is ‘F100’.
The microcontroller is a circuit that converts the estimated
emotion signal S e into the movement parameter A for a wheelchair.
Table III shows A that corresponds to S e . Here, the pulse width
modulation (PWM) is a square wave with a frequency of 50 Hz,
where Low is 0 V and High is 5 V, and the +duty cycle (the
fraction of a period in which the signal is High in a cycle)
corresponds to the intensity of emotion. In other words, the
intensity of the emotion is in proportion to the +duty cycle of
the PWM; as the intensity increases, the +duty cycle of the PWM
also increases. The detailed account of the control-signal generator
will be given in the next section.

4.3. Configuration and movements of BCI circuit
Figure 7 shows the BCI circuit configuration. This control circuit
consists of a microcontroller, which receives as input a serial
signal (S e ) that contains information about the emotion and its
intensity estimated by real-time EFAM software on a PC and
determines the movement parameter (A = {Af , Ab , Ar , Al }) of the
wheelchair. The control signal generator (CSG) then receives A as
input, and generates a movement control signal C (Section 4.2)
for the electric wheelchair. If ‘delight’ is given, it outputs ‘F’
4.4. Configuration and working of control signal
generator Figure 8 shows the configuration of the control-
signal generator (CSG). It is split into two blocks: a driving system
block, and a steering system block, and both share the same circuit
architecture. Each of these blocks is composed of two PhotoMOS
relays and voltage dividers (two serially connected 1-k resistors).
The PhotoMOS relays used are ones available on the market.
A PhotoMOS relay produces light when a current flows in the
LED in the input stage; then the light-sensitive elements receive

5 IEEJ Trans (2018)

 N. KOBAYASHI AND M. NAKAGAWA

Table III. Average recognition rate of four emotions (delight,

anger, sorrow, and pleasure) (%)

Se Af Ab Ar Al
Pleasure ‘F’a PWM 0 0 0
Anger ‘La’ PWM 0 0 PWM
Sorrow ‘Ra’ PWM 0 PWM 0
Delight ‘Ba’ 0 PWM 0 0
a Intensity of the emotion (0–100).

Driving system block 5V

PhotoMOS relay a Voltage

divider
Af

LED FET 1 kΩ

b
0V Cd
Ab

1 kΩ

c
0V

0V

Ar
Steering system block Cs
Al

Fig. 8. Control signal generator (CSG)

the light, and the quantity of light is converted into voltage. As
the converted voltage is applied to the gate of MOSFET (FET)
and the voltage exceeds the thresholds, the FET is turned ON.
In contrast, a voltage divider consists of two serially connected
resistors with the same resistance level. If 0 is input to every
A = {Af , Ab , Ar , Al }, the gate voltage of FET is 0 V; hence, it is
turned OFF. At this moment, point b in Fig. 8 has a half of 5 V
owing to the voltage division ratio (1 k/1 k + 1 k) of two equal
resistance levels. In short, the control voltage C of the wheelchair
is 2.5 V. This corresponds to the resting state (v = 0 km/h) for the
driving system (C d = 2.5 V), and to a straight movement, forward
or backward (θ = 0◦ ), for the steering system (C s = 2.5 V). If, on
the other hand, a PWM signal is input to A (here, as the PWM
signal is 5 V when High, it is converted into a gate voltage that
will exceed the thresholds of FET in order to turn it ON), the FET
is turned ON when the PWM is High, and it is turned OFF when
the PWM is Low. For example, if a PWM figure with a + duty
cycle of 50% is input into Af in Fig. 8, the FET is turned ON
for half the time on average; hence, a and b are short-circuited
during this period. At the same time, the FET is turned OFF during
the rest of the time, on average. At this moment, the resistance
between a and b is equal to 50% of 1 k on average, i.e., the
resistor works as if the resistance is 500 . In other words, the
resistor operates as a variable resistor with a minimum resistance
of 0  (+duty cycle = 100%) and a maximum resistance of 1 k
Fig. 9. Example of PWM generation by the BCI controller. (a)
Serial code ‘B50’. (b) Serial code’ F90’
(+duty cycle = 0%) in inverse proportion to the +duty cycle of
the PWM. On the other hand, if the signal input into Ab is 0, the
resistance between b and c is 1 k because the FET is turned OFF.
Therefore, the values for C d vary from 2.5 to 5 V, in proportion
to changes in the +duty cycle (0–100%) of the PWM input into
Af . In other words, the rotation speed of the back wheels of the
wheelchair changes from the resting state (0 km/h) to +6 km/h
(forward direction) in proportion to the +duty cycle of the PWM
input into Af . On the other hand, suppose the signal input into Af
is 0, and PWM is input into Ab , the resistance is 1 k between a
and b, and 0–1 k (variable resistance) between b and c in inverse
proportion to the +duty cycle; hence, C d changes from 2.5 to 0 V
in inverse proportion to the change in the +duty cycle (0–100%)
of the PWM. In other words, rotation speed v of the back wheels of
the wheelchair changes from the resting state (0 km/h) to −6 km/h
(backward direction) in inverse proportion to the +duty cycle of
the PWM input into Ab .
Similar remarks apply to the steering system block. If a signal
input into Al is 0, and the PWM is input to Ar , C s varies from 2.5
to 5 V, that is, the rotation angle of the front wheels changes from
0◦ (straight movement either forward or backward) to +60◦ (right-
turn) in proportion to the change in the +duty cycle (0–100%) of
the PWM. On the other hand, suppose the signal input into Ar

6 IEEJ Trans (2018)

 BCI-BASED CONTROL OF ELECTRIC WHEELCHAIR

“Pleasure” “Anger” “Sorrow” “Delight”

80
60
“Pleasure” 40
20
0
80
Intensity of emotion (%) 60
“Anger” 40
20
0
80
60
“Sorrow” 40
20
0
80
60
“Delight” 40
20
0
0 20 40 60 0 20 40 60 0 20 40 60 0 20 40 60
Time (s)
(a)

“Pleasure” “Anger” “Sorrow” “Delight”

80
60
40
Af

20
0
80
60
40
+Duty cycle (%)
Al

20
0
80
60
40
Ar

20
0
80
60
40
Ab

20
0
0 20 40 60 0 20 40 60 0 20 40 60 0 20 40 60
Time (s)
(b)

“Pleasure” “Anger” “Sorrow” “Delight”

5.0
4.0
3.0
Cd (V)

2.0
1.0
0.0
5.0
4.0
3.0
Cs (V)

2.0
1.0
0.0
0 20 40 60 0 20 40 60 0 20 40 60 0 20 40 60
Time (s)
(c)

“Pleasure” “Anger” “Sorrow” “Delight”

+6.0
+3.0
v (km/h)

0.0
–3.0
–6.0
+60
+30
0
θ (°)

–30
–60
0 20 40 60 0 20 40 60 0 20 40 60 0 20 40 60
Time (s)
(d)
Fig. 10. Example of controlling electric wheelchair. (a) Strength of emotional states. (b) Operational parameter A. (c) Control voltages C .
(d) Rotational speed v and angle θ ◦

7 IEEJ Trans (2018)

 N. KOBAYASHI AND M. NAKAGAWA
is 0, and the PWM is input into Al , C s varies from 2.5 to 0 V,
that is, the rotation angle of the front wheels changes from 0◦
(straight movement either forward or backward) to −60◦ (left-turn)
in inverse proportion to changes in the +duty cycle (0–100%) of
the PWM. As mentioned above, appropriate input of a PWM signal
corresponding to Table III into A enables the desired movements
(forward movement, backward movement, right-turn, left-turn) of
the wheelchair.
4.5. Operation instance of proposed BCI system
A microcontroller within the BCI control circuit generates the
movement parameter A based on the emotion, and its intensity
is estimated by real-time EFAM. Figure 9 shows an instance of an
actually observed wave pattern of a PWM signal generated as A via
the oscilloscope. Figure 9(a) is an instance in which the emotion
estimated by EFAM is ‘pleasure’ and its intensity is 54%. The
actual emotion recalled by the study subject was also ‘pleasure’,
so the subject’s recalled emotion and the EFAM-estimated emotion
coincide. As a serial code, ‘B54’ is input to the microcontroller,
and PWM is output as Ab . Other movement parameters Af , Ar , and
Al are all 0. As the +duty cycle of the PWM signal was around
54%, one can tell that the PWM has been accurately output in
accordance with the intensity of the emotion. Figure 9(b) is an
instance in which the emotion estimated by the EFAM is ‘delight’
and its intensity is 90%. The actual emotion recalled by the study
subject was also ‘pleasure’, so the subject’s recalled emotion and
the EFAM-estimated emotion coincide. As a serial code, ‘F90’
is input to the microcontroller, and PWM is output as Af . Other
movement parameters Ab , Ar , and Al are all 0. As the +duty cycle
of the PWM signal is around 90%, we can confirm that the PWM
has been accurately output in accordance with the intensity of
emotion.
Figure 10 shows an instance of controlling an electric
wheelchair using the proposed system. Emotions recalled for 60 s
by a study subject are shown from the left in the following order:
‘delight’, ‘anger’, ‘sorrow’, and ‘pleasure’. All the horizontal
axes represent time. Figure 10(a) is an emotion estimated by
real time EFAM, and the vertical axis represents the intensity
of emotion. Figure 10(b) is the movement parameter A of the
wheelchair output from the microcontroller, and is ordered Af ,
Al , Ar , and Ab from the top. Figure 10(c) is the control voltage
C of the wheelchair that is output from CSG, and is ordered
C d , C s from the top. Figure 10(d) is the rotation velocity v of
the back wheels and the rotation angle θ of the front wheels,
lined up in this order from the top. Fig. 10(a) shows that the
intensities of each emotion matched by the emotion recalled by
the study subject are high. Thus, we can confirm that EFAM
has accurately estimated the remembered emotion. The emotion
intensities outputs were around 50–60% for ‘delight’, 40–60%
for ‘anger’, 50–70% for ‘sorrow’, and 50–60% for ‘pleasure’.
Despite the fact that emotion intensities other than those of the
remembered emotions are output, in the present system, only an
emotion whose intensity hits the highest mark and its intensity
level are transmitted to the microcontroller in the next stage as
a serial code. The microcontroller will convert the serial code
into the movement parameter A, which will be output as a + duty
cycle PWM signal, which is in proportion to the intensity of the
emotion. Figure 10(b) shows that a PWM with +duty cycle of
around 50–60% was output to Af for ‘delight’; around 40–60%
was output to Af and Al for ‘anger’; around 50–70% was output
to Af and Ar for ‘sorrow’; and around 50–60% was output to Ab
for ‘pleasure’. Afterwards, CSG will generate a control voltage C
of the wheelchair using the PWM signal. As shown in Fig. 10(c),
C d and C s were set to around 4 and 2.5 V, respectively, for
‘delight’; around 4 and 1 V, respectively, for ‘anger’; around
4 and 4 V, respectively, for ‘sorrow’; and around 1 and 2.5 V,
8 IEEJ Trans (2018)
respectively, for ‘pleasure’. The rotation speed v of the back
wheels is determined according to the value of C d , and the
rotation angle θ of the front wheels is determined according to
the value of C s , resulting in a movement of the wheelchair. As
shown in Fig. 10(d), with ‘delight’, v is set to around +3 km/h
and θ to 0◦ , producing a forward movement; with ‘anger’, v is
set to around +3 km/h and θ to around −45◦ to -–0◦ , producing
a left-turn movement; with ‘sorrow’, v is set to around +3 to
+4.5 km/h and θ to around +30◦ to +45◦ , producing a right-turn
movement; and with ‘pleasure’, v is set to around −3 km/h and
θ to 0◦ , making a backward movement. As mentioned above, this
system realized an electric wheelchair that produces a forward
movement with ‘delight’, a left-turn movement with ‘anger’, a
right-turn movement with ‘sorrow’, and a backward movement
with ‘pleasure’. It also demonstrated that the speed of wheelchair
movement and the turning angle (either left or right) can be
adjusted.
5. Conclusion
This study demonstrated that an electric wheelchair can be
accurately operated in real time using information only of EEG by
applying EFAM, which discriminates emotions of study subjects
to a BCI using electrode difference signals obtained through
noninvasive, less constraining EEG. In addition, using a newly
developed BCI control circuit enables users to adjust the speed and
angle of an electric wheelchair movement based on the intensity
of their emotion.
In the future, we will seek to improve the emotion recogni-
tion rate, using not only EEG information but also biological
information obtainable from near-infrared spectroscopy (NIRS).
Furthermore, we aim to develop our system into a more practical,
general one by building a brain–affective interface (BAI), which
can discriminate between multiple high-level emotions, to enable,
for example, the electric wheelchair to halt urgently when the user
feels fear.
References
(1) Vidal JJ. Toward direct brain-computer communication. Annual
Review of Biophysics and Bioengineering 1973; 2:157–180.
(2) Ramoser H, Muller-Gerking J, Pfurtscheller G. Optimal spatial
filtering of single trial EEG during imagined hand movement. IEEE
Transactions on Rehabilitation Engineering 2000; 8(4):441–446.
(3) Wolpaw JR, Birbaumer N, Heetderks WJ, McFarland DJ, Peckham
PH, Schalk G, Donchin E, Quatrano LA, Robinson CJ, Vaughan
TM. Brain-computer Interface technology: A review of the first inter-
national meeting. IEEE Transactions on Rehabilitation Engineering
2000; 8(2):164–173.
(4) Cheng M, Gao X, Gao S, Xu D. Design and implementation of a
brain-computer Interface with high transfer rates. IEEE Transactions
of Biomedical Engineering 2002; 49(10):1181–1186.
(5) Naveen RS, Julian A. Brain computing interface for wheel chair
control. Computing, Communications and Networking Technologies
(ICCCNT), 2013 Forth International Conference on, 2013; 1–5.
(6) Khare V, Santhosh S, Anand S, Bhatia M. Brain computer Interface
based real time control of wheelchair using electroencephalogram.
International Journal of Soft Computing and Engineering (IJSCE)
2011; 1(5):41–45.
(7) Pinheiro OR, Alves LRG, de Souza JR. EEG signals classification:
Motor imagery for driving an intelligent wheelchair. IEEE Latin
America Transaction 2018; 16(1):254–259.
(8) Waytowich NR, Krusienski DJ. Development of an extensible
SSVEP-BCI software platform and application to wheelchair control.
Neural Engineering (NER) 2017; 1(1):259–264.
(9) Musha T, Terasaki Y, Haque HA, Ivamitsky GA. Feature extraction
from EEGs associated with emotions. Artificial Life and Robotics
1997; 1(1):15–19.
 BCI-BASED CONTROL OF ELECTRIC WHEELCHAIR
(10) Araga Y, Musha T. Digitalization and quantification of human
emotions using electroencephalogram. Bioscience and Industry 2001;
59(10):693–694.
(11) Kobayashi N, Nakagawa M. BCI-based control of electric wheelchair.
Proceedings of IEEE Global Conference on Consumer Electronics
(IEEE GCCE’2015), 2015; 429–430.
Nobuaki Kobayashi (Member) received the B.S. degree from the
Faculty of Science and Engineering, Chuo
University, in 2004, and the Ph.D. degree
in engineering from the Graduate School of
the same university in 2011. He is currently
with the College of Science and Technol-
ogy, Nihon University. His current research
interests include CMOS video and image
encoding processors, CMOS memory LSI,
and brain–computer interface (BCI). Dr Kobayashi is a member
of the Institute of Electronics, Information and Communication
Engineers (IEICE) and IEEE.
9 IEEJ Trans (2018)
Masahiro Nakagawa (Non-member) Masahiro Nakagawa
obtained the Ph.D. degree from in Engineer-
ing from Faculty of Engineering, Nagaoka
University of Technology (NUT), in 1988.
He was an Assistant Researcher in the
Faculty of Engineering, NUT, in 1982.
He was a foreign-based researcher from
March 1988 to January 1989 appointed
by the Ministry of Education, Science,
Sports and Culture, Japan, in the Department of Mathematics,
University of Strathclyde, UK. He is currently a Professor with
NUT. His research interests include chaos and fractal informatics,
mathematical engineering, emotion information engineering,
fractal image coding, chaos neural network, and physics of
liquid crystals. Prof. Nakagawa is a member of the Japan Neural
Network Society, The Physical Society of Japan, The Japan
Society of Applied Physics, Japan Society for Fuzzy Theory and
Intelligent Informatics, The Institute of Electronics, Information
and Communication Engineers (IEICE), and The Japanese Liquid
Crystal Society. He is the author of numerous books, including
Chaos and Fractals in Engineering (WorldScientific), Liquid
Crystal Phase Transition (Springer-Verlag Japan) (in Japanese),
and Chaos and Fractal Emotion Information Technology (Nikkan
Kogyo Shimbun, Ltd) (in Japanese).