See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/369934177

Ensemble Median Empirical Mode Decomposition for Emotion Recognition

Using EEG Signal

Article in IEEE Sensors Letters · May 2023

DOI: 10.1109/LSENS.2023.3265682

CITATIONS READS
6 13

2 authors, including:

Priyadarsini Samal
National Institute of Technology, Warangal
4 PUBLICATIONS 12 CITATIONS

SEE PROFILE

All content following this page was uploaded by Priyadarsini Samal on 14 February 2024.

The user has requested enhancement of the downloaded file.

 VOL. 7, NO. 5, MAY 2023 7001704

Sensor signal processing

Ensemble Median Empirical Mode Decomposition for Emotion Recognition

Using EEG Signal
Priyadarsini Samal and Mohammad Farukh Hashmi∗
National Institute of Technology, Warangal 506004, India
∗ Senior Member, IEEE

Manuscript received 28 March 2023; accepted 4 April 2023. Date of publication 10 April 2023; date of current version 25 April 2023.

Abstract—This letter investigates ensemble median empirical mode decomposition (MEEMD), an extension model of
ensemble empirical mode decomposition, and its improved characteristics for emotion recognition. It is tough to extract
the hidden patterns in the electroencephalography (EEG) signal due to the signals’ nonstationary nature, which is caused
by the brain’s complex neuronal activity. This makes it difficult to identify emotions using EEG. This research presents a
feature extraction method based on MEEMD for decoding EEG signals for emotion recognition. Analysis is done on the
intrinsic mode functions (IMFs) that are retrieved by EEMD and MEEMD. When identifying emotions using multichannel
EEG signals, features like power spectral density, relative powers, power ratios, entropies, mean, standard deviation,
and variance are used as indications of valence and arousal scales. The results indicate that the suggested method has
achieved accuracy rates of 74.3% for valence and 78% for arousal classes. DEAP EEG emotion dataset is used, and both
EEMD and MEEMD models are used to evaluate the results.

Index Terms—Sensor signal processing, electroencephalography (EEG), emotion recognition, ensemble empirical mode decomposition
(EEMD), ensemble median empirical mode decomposition (MEEMD), intrinsic mode functions (IMFs).

I. INTRODUCTION
Emotion plays an important role in communication between human
kind. Since artificial intelligence has advanced so quickly, emotion
recognition has been used in a wide range of industries, including
smart homes, health monitoring, etc. Human emotions can be rec-
ognized using a variety of behavioral cues, including gestures and
body language, voice, and physiological markers. The first three,
however, might be ineffective because people sometimes conceal their
genuine emotions either intentionally or unconsciously. More precise
and objective emotion recognition can be achieved using physiological
cues [1]. Many machine learning and deep learning-based techniques
for recognizing emotions based on electroencephalogram (EEG) signal
have been introduced during the past ten years. Variety of strategies
are available to read and analyze the patterns of the EEG signal.
Empirical mode decomposition (EMD) method is one of those,
which used to process nonlinear signals like EEG. It breaks down
the signal into intrinsic mode functions (IMFs) without making any a
priori assumptions or defining a standard. The expanded versions of
EMD [2] are bivariate [3], trivariate [4], and multivariate [5] models
to analyze multichannel data. The frequent occurrence of noise and
intermittent signal strength real-world information typically results in
mode splitting and mode mixing in EMD, from which mode mixing
is described as one IMF featuring various scales, while mode splitting
relates to the expansion of one scale over two IMFs or more. To
solve this problem, ensemble empirical mode decomposition (EEMD)
has been developed by applying the EMD across an ensemble of
the signal with white Gaussian noise [6], [7]. Later on some more
improved models of EMD are proposed named as fast multivariate
EMD, complementary ensemble EMD [8], etc. Many works have done
by researchers for recognizing emotions of human brain from different
aspects. For the analysis of working memory of brain under different
emotional states, Barkana et al. [9] used statistical as well as energy
Corresponding author: Mohammad Farukh Hashmi (e-mail: mdfarukh@nitw.
ac.in).
Associate Editor: F. Falcone.
Digital Object Identifier 10.1109/LSENS.2023.3265682
2475-1472 © 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.
features for different emotion states. They considered three classes of
emotions: positive, negative, and neutral emotions and SVM and KNN
were used as classifiers giving accuracy 61.45%, 53.7%, respectively.
Mehmood et al. [10] proposed a novel feature extraction algorithm
using late positive potential for recognizing emotions in EEG signal,
where they also used SVM and KNN classifiers and obtained 57.9%,
56.2% accuracies, respectively. In [11], EMD as well as variational
mode decomposition, another method for signal decomposition, are
used. Using GAMEEMO dataset and DeepBiLSTM, SVM, KNN, and
RF as classifiers, they obtained an 70.89% accuracy for binary class
classification. Mert et al. [5] used the multivariate extension of EMD
for extracting various statistical and frequency domain features for
emotion classification and obtained an accuracy of 75% and 72.87%
for valence and arousal classes respectively.
The EMD and its extension methods have given higher classification
performance for different applications of biomedical signal. However,
in this EEMD method the addition of noise causes the mode mixing
problem. To overcome this problem a novel method is used in this
study. In this letter, we proposed EEG-MEEMD (median ensemble
empirical mode decomposition)-based feature extractions techniques
for emotion recognition where during the IMF extraction phase median
signal is used in place of mean signal. To know the efficiency of this al-
gorithm, we have also implemented the conventional EEMD algorithm
and compared the results, in this letter. Various machine learning al-
gorithms were used for providing improved performance measures for
this experiment. The workflow of the proposed model is shown in Fig 1.
II. DATASET
DEAP dataset is a multimodal dataset made up of 32-channel EEG
data collected from 32 people who rated the valence and arousal of
video clips on a scale of 1 to 9. Each subject goes through 40 trials
and each trial’s 60 s and the baseline trial’s 3 s are separated from
the signal for further analysis. The signal is subjected to a band pass
filter with a range of 0.4–45 Hz, and the sampling frequency was set
to 128 Hz. [12]. Data of 32 subjects is used for processing, and 18 of
the 32 channels are chosen for the experiment because they are more
important for emotion identification.

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY WARANGAL. Downloaded on February 14,2024 at 06:01:29 UTC from IEEE Xplore. Restrictions apply.
 7001704 VOL. 7, NO. 5, MAY 2023

Fig. 1. Work flow of the proposed model.

III. METHODS TABLE 1. Correlation Coefficients Between the Original Signal of

Subject1 and the Corresponding Nine IMFs
In this study, both MEEMD and EEMD models are used for anal-
ysis of EEG signals. The EMD approach has a drawback known as
mode mixing, which is described as an oscillation of a single IMF
having oscillations of significantly different scales, or a component
of a comparable scale existing in distinct IMFs. To overcome these
issues, EEMD came into picture [13]. Zhaoha et al. [14] presented
an expanded variant of EMD called EEMD in 2010. Without the use
of arbitrary a priori criteria, the ensemble technique can distinguish
between signals on different scales of genuineness. The following is a
description of the EEMD algorithm [11].
1) Implement the mth trial for the signal with additional white
noise.
a) Add the white noise series with the specified amplitude to
the signal under investigation, i.e., xm (t ) = x(t ) + nm (t ), A. Feature Extraction
where nm (t ) denotes the mth added white noise and xm (t )
denotes the noise-added signal of the mth trial. Each signal had a total of 19 components, including time-domain,
b) Utilizing the EMD approach, divide the noise- frequency-domain, and nonlinear features. The time domain features
added signal xm (t ) into l IMFs cim (i = 1, 2, chosen are mean, standard deviation, and variance.
. . . , l, m = 1, 2, . . . , M ), where cim denotes the ith IMF For a signal, X = {x1 , x2 , x3 , …., xN }, mean (μ), standard deviation
of the mth trial; l is the number of IMFs; and M is the (σ ), variance(σ 2 ) can be expressed as follows:
number of the ensemble members.
1 
N
c) If m < M, then let m = m + 1 and repeat the steps (a)
μ= xi (2)
and (b) until m = M, but each time using a white noise. N i=1
2) Calculate the collective mean ci of the M trials for each IMF to 
find 1 N
σ = (xi − μ)2 (3)
N i=1
1 
M
ci = ci,m . 1 
N
(1)
M m=1 σ2 = (xi − μ)2 . (4)
N i=1
3) Note down the mean ci (i = 1, 2, . . . .., l ) of each of l IMFs as
the final ith IMF. Let the normalized signal be XNorm. Then, it is divided into
However, in this ensemble empirical mode decomposition method, frequency bands using DWT, and the signal’s power spectra were
the addition of noise causes the mode mixing problem. To overcome obtained using FFT. Band power, relative power, and power ratios for
this problem MEEMD method is used here. Here, the process is similar all five EEG frequency bands are extracted as the frequency domain
to EEMD. Only difference is during the extraction phase the median features, which are Pδ , Pθ , Pα , Pβ , Pγ , Rδ , Rθ , Rα , Rβ , Rγ , r1 , r2 , and
signal of lower envelope and upper envelope is used for obtaining r3 . The relative powers are obtained by dividing the corresponding
IMFs by subtracting from the original signal. The EEMD algorithm band power to the total power of all five frequency bands
is implemented on DEAP dataset and shown inFig. 2. The maximum r1 = Pθ /Pα (5)
number of IMF is set to 9 here.
r2 = Pθ /Pβ (6)
r3 = Pθ /Pβ + Pα . (7)
IV. EXPERIMENT
In a similar manner, the signal’s energy, wavelet Shannon entropy,
The EEG data is subjected to the EEMD and MEEMD models, and approximation entropy are also measured. With 32 subjects, 40
and the models’ respective nine IMFs are derived. The IMFs are then trials, and 18 channels, we were able to collect total of 23 040∗ 19
normalized in the range of 0.1 to 0.9 for easier analysis. After analyzing features for each subject. For 32 subjects the size of the feature set we
all the nine IMFS, the most appropriate one among those is utilized constructed is 23 040∗ 19∗ 32.
for the research. Table 1 provides the correlation coefficients between
the original signal and the nine IMFs obtained by MEEMD. B. Classification
The correlation coefficients show higher value for IMF2 whereas
correlation coefficients of IMF5 to 9 show minimum values. Seven classifiers use the feature vector, that has been constructed to
Similarly, by analyzing the spectrums, histograms, p-values, and sta- recognize of emotions. Both high and low values of valence and arousal
tistical feature values can help in selecting the optimal IMF. are considered for classification of emotions. Support vector machine
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY WARANGAL. Downloaded on February 14,2024 at 06:01:29 UTC from IEEE Xplore. Restrictions apply.
 VOL. 7, NO. 5, MAY 2023
Fig. 2. EEMD implementation.
TABLE 2. Classification Accuracy (%) of EEMD and MEEMD
TABLE 3. Accuracy Comparison With Existing Emotion Recognition
Methods Using DEAP Database
(SVM), naïve bayes (NB), neural network (NN), linear discriminant
analysis (LDA), K nearest neighbor (KNN), logistic regression (LR),
and ensemble bagged trees classifiers are employed. Ten-fold cross-
validation and 80:20 train to testing ratio are chosen for analysis.
V. RESULTS AND DISCUSSION
Table 2 displays the classification results for all seven classifiers learning. Similar to this, when recognizing emotions from EEG sig-
using both EEMD and MEEMD models for high (greater than 5) and
low (less than 5) values of valence and arousal. For multiclass, the
combined class includes both valence and arousal for classification. It
is observed that SVM, KNN, and ensemble classifiers have achieved
higher classification accuracy for both models. When employing
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY WARANGAL. Downloaded on February 14,2024 at 06:01:29 UTC from IEEE Xplore. Restrictions apply.
7001704
Fig. 3. ROC curves for valence (a) and (c) and arousal (b) and (d)
using EEMD and MEEMD, respectively.
bagged trees classifier, MEEMD achieved the best accuracy (74.3% for
valence, 78% for arousal), but EEMD only achieved 70.4% for valence
and 74.1% for arousal using the same. In multiclass classification,
MEEMD is provided 65.2% accuracy. It can be shown from comparing
the results of EEMD and MEEMD that MEEMD has produced results
that are more accurate in recognizing emotions. Average results for
classification in both scenarios have been produced by other classifiers.
It can be seen that, the issues with EEMD can be resolved by using
its extension version MEEMD, which is more efficient than that. Fig. 3
shows the ROC curves for both valence and arousal classes using
MEEEMD and EEMD methods. Figs. 4 and 5 show the subject wise
accuracies using ensemble bagged trees classifier for high and low
values of valence and arousal respectively. It is evident from these
figures as well that MEEMD produces better results for all subjects
when identifying emotions.
Using the same DEAP EEG signal database, Table 3 compares
the proposed MEEMD approach with existing emotion recognition
techniques. It has been noted that, in terms of the accuracy measure,
the suggested approach has outperformed CN-LSTM based ensemble
nals, the EEMD method has demonstrated greater accuracy than the
Deep BiLSTM network.
In addition, compared to the Discriminative graph regularized
extreme machine learning method, our approach produced better
outcomes. Most of the methods now in use, use differential entropy as
 7001704
Fig. 4. Subject wise accuracy for arousal class using EEMD and MEEMD.
Fig. 5. Subject wise accuracy for valence class using EEMD and MEEMD.
an important indicator of emotions in EEG signals since it is a useful
tool for analysis. Similar to this, various entropy measurements are
beneficial for determining emotions. In addition, the wavelet Shannon
entropy and approximation entropy features in the suggested approach
improve model outcomes after being included to the feature vector.
VI. CONCLUSION
In this letter, a novel method based on MEEMD for recognizing
emotions in multichannel EEG signals is proposed. The EEG signals
from 32 subjects from the DEAP dataset were used to achieve the
recognition performance of the suggested approach. Using ensemble
bagged trees classifier with time, frequency, and nonlinear features,
the EEG-MEEMD technique has obtained an accuracy of 74.3% for
high/low value of valence and 78% for high/low value of arousal.
The proposed strategy has outperformed the existing methods for
recognizing emotions using EEG signals from the same database,
according to the performance comparisons with the existing method-
ologies. In the future, the performance of the suggested method can be
evaluated using the EEG signals from more channels. Only the baseline
machine learning classifiers are used for this study. In future different
advanced machine learning and deep learning techniques can be used
for proposed model. For the multichannel analysis of EEG signals for
the recognition of emotions, more extensions of EMD approaches can
be investigated.
REFERENCES
[1] E. H. Houssein, A. Hammad, and A. A. Ali, Human Emotion Recognition From
EEG-Based Brain–Computer Interface Using Machine Learning: A Comprehensive
Review, vol. 4. Berlin, Germany: Springer, 2022.
[2] N. Salankar, P. Mishra, and L. Garg, “Emotion recognition from EEG signals using
empirical mode decomposition and second-order difference plot,” Biomed. Signal
Process. Control, vol. 65, 2021, Art. no. 102389.
[3] A. Ahrabian, N. U. Rehman, and D. Mandic, “Bivariate empirical mode decompo-
sition for unbalanced real-world signals,” IEEE Signal Process. Lett., vol. 20, no. 3,
pp. 245–248, Mar. 2013.
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY WARANGAL. Downloaded on February 14,2024 at 06:01:29 UTC from IEEE Xplore. Restrictions apply.
View publication stats
VOL. 7, NO. 5, MAY 2023
[4] N. Ur Rehman and D. P. Mandic, “Empirical mode decomposition for trivariate
signals,” IEEE Trans. Signal Process., vol. 58, no. 3, pp. 1059–1068, Mar. 2010.
[5] A. Mert and A. Akan, “Emotion recognition from EEG signals by using multivariate
empirical mode decomposition,” Pattern Anal. Appl., vol. 21, no. 1, pp. 81–89, 2018.
[6] X. Lang, N. ur Rehman, Y. Zhang, L. Xie, and H. Su, “Median ensemble empirical
mode decomposition,” Signal Process., vol. 176, 2020, Art. no. 107686.
[7] D. Chen, D. Li, M. Xiong, H. Bao, and X. Li, “GPGPU-aided ensemble empirical-
mode decomposition for EEG analysis during anesthesia,” IEEE Trans. Inf. Technol.
Biomed., vol. 14, no. 6, pp. 1417–1427, Nov. 2010.
[8] W. Tang and Y. Geng, “EEG denoising method based on improved CEEMD with
cosine similarity,” J. Nanjing Univ. Posts Telecommun., Natural Sci., vol. 40, no. 3,
2020.
[9] B. D. Barkana, Y. Ozkan, and J. A. Badara, “Analysis of working memory from EEG
signals under different emotional states,” Biomed. Signal Process. Control, vol. 71,
2022, Art. no. 103249.
[10] R. M. Mehmood and H. J. Lee, “A novel feature extraction method based on late
positive potential for emotion recognition in human brain signal patterns,” Comput.
Elect. Eng., vol. 53, pp. 444–457, 2016.
[11] A. Abdulrahman, M. Baykara, and T. B. Alakus, “A novel approach for emotion
recognition based on EEG signal using deep learning,” Appl. Sci., vol. 12, no. 19,
2022, Art. no. 10028.
[12] S. Koelstra et al., “DEAP: A database for emotion analysis using physiological
signals,” IEEE Trans. Affect. Comput., vol. 3, no. 1, pp. 18–31, Jan.-Mar. 2012.
[13] Z. Zhao, L. Yang, D. Chen, and Y. Luo, “A human ECG identification system based
on ensemble empirical mode decomposition,” Sensors (Switzerland), vol. 13, no. 5,
pp. 6832–6864, 2013.
[14] Z. Wu and N. E. Huang, “Ensemble empirical mode decomposition: A noise-assisted
data analysis method,” Adv. Adaptive Data Anal., vol. 1, no. 1, pp. 1–41, 2009.
[15] N. Kumar, K. Khaund, and S. M. Hazarika, “Bispectral analysis of EEG for emotion
recognition,” Procedia Comput. Sci., vol. 84, pp. 31–35, 2016.
[16] A. Iyer, S. S. Das, R. Teotia, S. Maheshwari, and R. R. Sharma, “CNN and LSTM
based ensemble learning for human emotion recognition using EEG recordings,”
Multimedia Tools Appl., vol. 82, no. 4, pp. 4883–4896, 2022.
[17] V. M. Joshi, R. B. Ghongade, A. M. Joshi, and R. V. Kulkarni, “Deep BiLSTM neural
network model for emotion detection using cross-dataset approach,” Biomed. Signal
Process. Control, vol. 73, 2022, Art. no. 103407.
[18] S. Sheykhivand, T. R. Yousefi, Z. Mousavi, and S. Meshini, “Automatic stage
scoring of single-channel sleep EEG using CEEMD of genetic algorithm and neural
network,” Comput. Intell. Elect. Eng., vol. 9, no. 1, pp. 15–28, 2018.
[19] X. W. Ding, Z. T. Liu, D. Y. Li, Y. He, and M. Wu, “Electroencephalogram
emotion recognition based on dispersion entropy feature extraction using random
oversampling imbalanced data processing,” IEEE Trans. Cogn. Devlop. Syst., vol. 14,
no. 3, pp. 882–891, Sep. 2022.