Pathological Speech Processing: Review
1
Máté HIREŠ (1st year),
Supervisor: 2 Peter DROTÁR
1,2 Department of Computers and Informatics, FEI TU of Košice, Slovak Republic
1 mate.hires@tuke.sk, 2 peter.drotar@tuke.sk
Abstract—This paper describes an investigation of voice pathol-
ogy detection. This is a preliminary study in the field of automatic
detection of voice disorders. We compare some of the available
voice databases, which contains both healthy and pathological
voice recordings. We also compare some methods, which use
neural networks for the detection and achieved significant results.
Finally we discuss the presented methods and their results in term
of their accuracy.
Keywords—voice pathology, speech disorders, machine learn-
ing, signal processing, deep learning.
I. I NTRODUCTION
The human voice can have a radical effect on the health
condition of an individual. It can be radically affected by neo-
plasm, vocal palsy and other phonotraumatic diseases. Auto-
matic detection of these pathological voice disorders is a very
challenging and significant medical classification problem.
Voice pathology includes evaluation and treatment of voice
production related diseases, which are affecting fluency, in-
tonation, phonation and breathing. Since the computer-aided
analysis and diagnosis of human voice can be an effective and
low-cost tool for patients around the world, voice pathology
has congested specific interest amongst machine learning
and signal processing scientists. The current activity of the
detection of voice disorders includes a special procedure called
laryngeal endoscopy [1], [2]. This is a very complicated and
expensive procedure and it also requires an expert to perform
the evaluation. To detect voice disorders without intrusion, we
can analyze the voice directly.
Many scientists work on the automatic detection of voice
disorders, exactly in the last few years. Van der Merwe [3]
provides a foundation and accentuates the necessity of the
research related to voice disorder detection. Kent [4] discusses
the connection between voice production and its dysfunctions.
Some studies found specific speech dysfunctions within some
particular population groups [5], [6].
Thanks to the amount of available data and the advance-
ment in the computational power, a lot of appropriate Deep
Learning models are provided for speech processing. Hence,
we are allowed to use complex model architectures. We can
teach convolutional layers [7] to detect various features that
could help us to differentiate pathological and healthy voice.
Some Interspeech challenges [8], [9] also attracted interest in
the application of Machine Learning and signal processing
techniques for voice pathology detection.
A lot of available datasets contain recordings, which are
recorded in more different environments, thus it makes hard
to find common features in the samples. The Saarbruecken
Voice Database (SVD) [10] could be an ideal set of data to start
the research in this area, since all of its recordings are sampled
at 50 kHz with 16-bit resolution.
II. R ELATED WORK
Deep learning models got popular from the introduction
of the deep Boltzman machine in 2006 [11]. From this time
using these models, researchers and scientist have achieved
significant results in many fields, such as medical analysis
[2], [12], [13], computer vision [14], cybersecurity [15], etc.
There are a lot of research articles in the field of voice
pathology detection, where the researchers used SVD for their
work [16], [17], [18], [19], [20]. They yielded certain fea-
tures from the voice recordings. These features contains mel-
frequency cepstral coefficients, energy, entropy, harmonics-to-
noise ratio, normalized noise energy, multidimensional voice
program parameters, etc. Hence, multiple classifiers have been
used. Most of the researchers used traditional methods like
Support Vector Machine classifier (SVM), Dimensionality
Reduction Method (DRM), glottal source excitation, Gaussian
mixture models, k-means clustering. Deep Neural Network
(DNN) methods were also significantly useful. Notable results
were achieved for example with the Far Eastern Memorial
Hospital (FEMH) dataset using transfer learning techniques
[21]. It consists of transferring the knowledge from a small
dataset to another previously trained model, which was trained
on a similar domain [22].
Voices are mostly affected by cancer, nodules, polyps, cysts,
laryngitis, vocal tremors, spasmodic dysphonia, vocal fold
paralysis and sulcus diseases [1]. The results of the voice
pathology detection depends on the data used. Dankovičová
et al. differentiated healthy and dysfunctional voice samples
with the accuracy of over 70% using traditional methods
[23]. Kasuya et al. effectively identified glottic cancer, vocal
cord polyp and other nerve paralysis diseases [24]. Fang
et al. identified polyp, cyst, nodules, neoplasm, and other
diseases as well by using deep learning models [1]. They
achieved over 90% accuracy in female and over 94% accuracy
in male subjects. Dibazar et al. connected pitch dynamics,
Mel-frequency spectral coefficients and Hidden Markov Model
classifier (HMM) to identify some voice disorders. Working
with data from the Massachusets Eye and Ear Infirmary
(MEEI) they implemented a model, which resulted with the
accuracy of over 99% [25]. Souissi et al. achieved over
86% accuracy using 2000 recordings [17]. Al-nasheri et al.
pushed the accuracy of 99% [18]. Hemmerling at al. achieved
the highest accuracy of 100%, who detected the disorders
for women and men separately [19]. However, since these
results were achieved on specific datasets, the high accuracy
can be questionable.
III. DATASETS
A. The Saarbruecken Voice Database (SVD)
The SVD is a collection of voice recordings from more than
2000 patients [10]. One recording session contains the follow-
ing recordings:
• Recording of the vowels [i, a, u] produced at normal,
high and low pitch,
• Recordings of the vowels [i, a, u] with rising-falling pitch,
• Recording of the sentence ”Guten Morgen, wie geht es
Ihnen?” ("Good morning, how are you?").
This is a total of 13 files per session. The signals for each
case are stored in separate files. All the samples are recorded
at 50 kHz with 16 bit resolution. This dataset contains 71
different pathologies including organic and functional.
B. The Far Eastern Memorial Hospital database (FEMH)
This dataset includes 60 normal voice samples and 402
samples of common voice disorders, including vocal nodules,
polyps, andcysts, glottic neoplasm, vocal atrophy, laryngeal
dystonia, unilateral vocal paralysis and sulcus vocalis.
It contains 3-second long voice samples of patients sustaining
the vowel sound /a/ [1]. These samples were recorded
using a high-quality microphone with a digital amplifier
under a background noise level between 40 and 45 dBA.
The sampling rate was 44,100 Hz with a 16-bit resolution.
C. The Massachusetts Eye and Infirmary Voice Disorders
Database (MEEI)
The MEEI Voice Disorders Database was delivered in 1994
at the MEEI Voice and Speech Lab and partly at Kay Elemet-
rics Corp. [26]. It contains over 1400 samples of recordings
of:
• sustained of vowel /a/, from which 53 is normal and 657
pathological,
• continuous speech, from which 53 is normal and 661
pathological.
All the samples have 16 bit resolution, but as a disadvantage,
the samples are recorded in two different environments.
The frequency for normal samples is 50 kHZ, while the
frequency of the pathological samples is either 25 or 50 kHz.
(53 normal and 657 pathological files) and continuous speech
(53 normal and 661 pathological).
D. The Arabic Voice Pathology Database (AVPD)
The AVDP dataset was developed at the Communication
and Swallowing Disorders Unit of King Abdul Aziz University
Hospital, Riyadh, Saudi Arabia [27]. The idea behind the cre-
ation of this dataset is to overcome the environmental problems
of the MEEI dataset, which could lead to non-accurate results.
The dataset contains:
• voice samples of successive speech tasks,
• sustained phonation of the vowels /a/, /e/ and /o/,
• counting from 0-10,
• a standardized Arabic passage,
• reading of three common words.
The sample frequency of all collected samples in AVPD is 50
kHz.
E. New Spanish Speech corpus database
This database contains speech recordings of 50 people
with Parkinson’s disease and 50 healthy recordings with 25
male and 25 female on each category. The participants are
Colombians, speaking native Spanish. This database is bal-
anced in terms of age and gender as well. The average
age of the participants is approximately 60 in each category
and subcategory. The recordings were sampled at 44100 Hz
with 16 bits of resolution.
The participants had to perform different tasks to analyze
some aspects of their voice. The tasks can be grouped into
the following aspects: (i) phonation, (ii) articulation, (iii)
prosody. Detailed description of each task is described in [28].
IV. M ETHODOLOGY
A. DNN Architecture
The following Deep Neural Network (DNN) architecture
was presented by Harar et al. [20]. They used two stacks
of convolutional layers so their model can transform the input
into a set of more abstract repeating patterns, that could be
important for the decrease of the network cost. Between each
stack there is a max pooling layer to reduce the dimension
of the input vector. Moreover, they wrapped all the convolution
and pooling layers in a TimeDistributed layer to keep the time
axis unchanged. The resulting matrices are then reshaped to be
connected to a recurrent Long-Short-Term-Memory (LSTM)
layer. This LSTM layer is set so it can remember the changes
in time. At last, there is a stack of 3 fully connected layers,
which ends with a 2 neuron Softmax layer for the final
classification. One of the neurons are for the healthy and one
for the pathological class.
The detailed DNN architecture with its input and output
parameters is shown in Fig. 1.
B. CNN Architecture
Convolutional Neural Networks (CNN) are variants of the
standard neural network. They are not using fully connected
hidden layers. They have a special network architecture, which
consists of alternating convolution and pooling layers, most
of them are hidden. In the following text we will discuss the
CNN architecture used by Wu at al. in [29].
The weights are shared among all the units, where each unit
is calculated as
NW
I X
X
hkm = σ( k
vl,n+m−1 wl,n + w0k ), (1)
l=1 n=1
where vl,m is the m-th unit of the l-th input layer V and hk,m
is the m-th unit of the k-th convolutional layer H. NW is the
k
size of all the weights, where wl,n is the n-th unit of the given
weight. The feature extraction is an automatic process by the
shared-weights.
A pooling layer is fundamental for dimensional reduction
to decrease the complexity of the computations. To build such

Fig. 1. The detailed DNN architecture by Harar et al. [20]
a pooling layer, maximization function is used. The pooling
layer is defined as follows:
G i∈input
pkm = max hl,(m−1)×s+n ,
n=1
where G is the size of the pooling window using maximization
function. The value s is a step, when the pooling window shifts
among the convolutional layer.
They used 10 hidden layers. After the first layer, which has
the shape of 8∗ 3 with the step 1, each following layer was
convolved with 8 filters with the shape 8∗ 3∗ 8 and the step
1. They set 4∗ 4 pooling windows and a RELU activation
function. Next, they formed the feature map into a fully
connected layer to train the model classification. To overcome
the over-fitting problems, L2-regularization is used.
C. Feature Extraction Procedure
The following model was created by Banerjee et al. for fea-
ture extraction [13].
Firstly, there is a 25ms segment size and a 10ms shift size used
to extract the features for each patient. This model extracts
three types of features for each segment:
• Prosodic features,
• Vocal-Tract features,
• Excitation features.
These three types produce overall 54 features for each seg-
ment. Next, there are 162 features formed by taking the first
and second time derivates for each segment. Finally there is
a concatenation of 15 consecutive feature vectors to create
a final vector of size 2430, which serve for diagnosis. The
detailed description of the features is shown in Table I.
TABLE I
D ESCRIPTION OF SPEECH FRAME FEATURES [13]
Feature Type N. of Features
Prosodic features
Short-time energy, Average power, Average magni- 12
tude, Zero crossings, Mean, Standard deviation, Me-
dian, Max, Min, Range, Dynamic range, Interquartile
range
Vocal-tract features
MFCC (Mel Frequency Cepstrum Coefficients) 39
Teager Energy Operator 1
Excitation features
Jitter 1
Shimmer 1
Total number of raw features per frame 54
First order time derivative of raw features 54
Second order time derivative of raw features 54
Total number of features per frame 162
D. Deep Belief Network
Once the features are extracted, the training of a deep
belief network can be done. As an example method we have
chosen the transfer learning approach by Islam et al. [21].
They trained their Deep Belief Network (DBN) on the TIMIT
dataset [30]. The classification is performed in two steps, pre-
training and fine-tuning. In the first step, the DBN learns the
input feature in an unsupervised manner in different layers.
A stack of layers are learned using the restricted Boltzman
machine algorithm. The states are notable only in the visible
layer. The DBN learns the hidden features via the following
energy function:
X X X
E(v, h) = − bi vi − bj hj − vi hj wij , (3)
j∈f ea i,j
(2)
where vi and hj represents the binary states of input i
and feature j, respectively. Attributes bi and bj are the biases
for input i and feature j and wij is the weight between them.
In the second, fine-tuning step the truth is attached on the
topmost layer with the pre-trained weights. The fine-tuning
is a supervised procedure, thus a soft-max layer is added
on top of the learned weight so the weights could be fine-tuned
for the phoneme cases. The model uses stochastic gradient
descent algorithm for weight optimization. To determine the
probability of the visible vector, the following function is used:
X P
exp(−E(v, h))
p(v) = p(v, h) = P h , (4)
h∈H u,g exp(−E(u, g))
where H is the set of all possible binary hidden vectors in the
DBN.
E. Transfer Learning
Transfer learning is proved to be notably useful within all
the deep learning methods. This technique is useful especially
when there is limited training data. It consists of transferring
the knowledge from a small dataset to another previously
trained model, which was trained on a similar domain [22].
Once the model is trained, transfer learning approach can be
applied on different layers of the trained model. The follow-
ing transfer learning approach is modelled by Islam et al.
[21] following the DBN modelling (see previous subsection)
and it contains the following steps:
 1) Training of a DBN model using all the samples from
the TIMIT dataset for 39 phoneme classes, see Table I.
2) The FEMH training dataset is added to the trained DBN
model to find the representation in a given layer.
3) An SVM classifier model is trained with a linear kernel
and grid optimization for the new representation of the
FEMH data, an SVM to find the optimal parameters.
4) The trained SVM model predicts the unlabelled testing
data in the FEMH dataset.
5) Repeating the whole procedure for all the layers of the
DBN model.
V. D ISCUSSION
In this paper we have discussed the process of voice
pathology detection. We showed some notable results achieved
using some machine learning techniques. We also observed
some auxiliary methods used in machine learning algorithms.
Since the deep learning and transfer learning approaches
tent to be more precise in comparison with the traditional
voice recognition methods, we can consider more inves-
tigation in voice detection using deep learning techniques
to achieve more accurate results. Convolutional neural network
showed up to be very effective in extracting important features
from the spectograms of voice recordings, which helps in di-
agnosis. Deep belief network is useful for making the system
more robust by initializing the weights. In the future work we
will further analyze and experiment with optimizing as well
as upgrading the existing algorithms or combining the used
methods, respectively so we can achieve better performance.
Transfer learning and CNN based feature extraction could
potentially provide an additional approach to the automatic
voice pathology detection problem.
ACKNOWLEDGMENT
This work was supported by the Slovak Research and
Development Agency under contract No. APVV-16-0211.
R EFERENCES
[1] S.-H. Fang, Y. Tsao, M.-J. Hsiao, J.-Y. Chen, Y.-H. Lai, F.-C. Lin, and
C.-T. Wang, “Detection of Pathological Voice Using Cepstrum Vectors:
A Deep Learning Approach,” Journal of Voice, vol. 33, no. 5, pp. 634–
641, 2019.
[2] S. R. Schwartz, S. M. Cohen, S. H. Dailey, R. M. Rosenfeld, E. S.
Deutsch, M. B. Gillespie, E. Granieri, E. R. Hapner, C. E. Kimball,
H. J. Krouse et al., “Clinical practice guideline: hoarseness (dysphonia),”
Otolaryngology–Head and Neck Surgery, vol. 141, no. 1_suppl, pp. 1–
31, 2009.
[3] M. R. McNeil, D. Robin, and R. Schmidt, Clinical management of
sensorimotor speech disorders. Thieme New York, 1997.
[4] R. D. Kent, “Research on speech motor control and its disorders: A
review and prospective,” Journal of Communication disorders, vol. 33,
no. 5, pp. 391–428, 2000.
[5] L. E. DeLisi, “Speech disorder in schizophrenia: review of the literature
and exploration of its relation to the uniquely human capacity for
language,” Schizophrenia bulletin, vol. 27, no. 3, pp. 481–496, 2001.
[6] P. Lieberman, E. Kako, J. Friedman, G. Tajchman, L. S. Feldman, and
E. B. Jiminez, “Speech production, syntax comprehension, and cognitive
deficits in parkinson’s disease,” Brain and language, vol. 43, no. 2, pp.
169–189, 1992.
[7] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning
applied to document recognition,” Proceedings of the IEEE, vol. 86,
no. 11, pp. 2278–2324, 1998.
[8] B. Schuller, S. Steidl, A. Batliner, E. Nöth, A. Vinciarelli, F. Burkhardt,
R. v. Son, F. Weninger, F. Eyben, T. Bocklet et al., “The interspeech
2012 speaker trait challenge,” in Thirteenth Annual Conference of the
International Speech Communication Association, 2012.
[9] J. Kim, N. Kumar, A. Tsiartas, M. Li, and S. S. Narayanan, “Auto-
matic intelligibility classification of sentence-level pathological speech,”
Computer speech & language, vol. 29, no. 1, pp. 132–144, 2015.
[10] B. Woldert-Jokisz, “Saarbruecken voice database,” 2007.
[11] G. E. Hinton, S. Osindero, and Y.-W. Teh, “A fast learning algorithm for
deep belief nets,” Neural computation, vol. 18, no. 7, pp. 1527–1554,
2006.
[12] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification
with deep convolutional neural networks,” in Advances in neural infor-
mation processing systems, 2012, pp. 1097–1105.
[13] D. Banerjee, K. Islam, K. Xue, G. Mei, L. Xiao, G. Zhang, R. Xu,
C. Lei, S. Ji, and J. Li, “A deep transfer learning approach for improved
post-traumatic stress disorder diagnosis,” Knowledge and Information
Systems, vol. 60, no. 3, pp. 1693–1724, 2019.
[14] P. Liu, S. Han, Z. Meng, and Y. Tong, “Facial expression recognition via
a boosted deep belief network,” in Proceedings of the IEEE conference
on computer vision and pattern recognition, 2014, pp. 1805–1812.
[15] M. M. U. Chowdhury, F. Hammond, G. Konowicz, C. Xin, H. Wu,
and J. Li, “A few-shot deep learning approach for improved intrusion
detection,” in 2017 IEEE 8th Annual Ubiquitous Computing, Electronics
and Mobile Communication Conference (UEMCON). IEEE, 2017, pp.
456–462.
[16] D. Martínez, E. Lleida, A. Ortega, A. Miguel, and J. Villalba, “Voice
pathology detection on the Saarbrücken voice database with calibration
and fusion of scores using multifocal toolkit,” in Advances in Speech
and Language Technologies for Iberian Languages. Springer, 2012,
pp. 99–109.
[17] N. Souissi and A. Cherif, “Dimensionality reduction for voice disorders
identification system based on mel frequency cepstral coefficients and
support vector machine,” in 2015 7th international conference on
modelling, identification and control (ICMIC). IEEE, 2015, pp. 1–6.
[18] A. Al-nasheri, G. Muhammad, M. Alsulaiman, and Z. Ali, “Investigation
of voice pathology detection and classification on different frequency
regions using correlation functions,” Journal of Voice, vol. 31, no. 1,
pp. 3–15, 2017.
[19] D. Hemmerling, A. Skalski, and J. Gajda, “Voice data mining for
laryngeal pathology assessment,” Computers in biology and medicine,
vol. 69, pp. 270–276, 2016.
[20] P. Harar, J. B. Alonso-Hernandezy, J. Mekyska, Z. Galaz, R. Burget,
and Z. Smekal, “Voice pathology detection using deep learning: a
preliminary study,” in 2017 international conference and workshop on
bioinspired intelligence (IWOBI). IEEE, 2017, pp. 1–4.
[21] K. A. Islam, D. Perez, and J. Li, “A transfer learning approach for
the 2018 FEMH Voice Data Challenge,” in 2018 IEEE International
Conference on Big Data (Big Data). IEEE, 2018, pp. 5252–5257.
[22] S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Trans-
actions on knowledge and data engineering, vol. 22, no. 10, pp. 1345–
1359, 2009.
[23] Z. Dankovičová, D. Sovák, P. Drotár, and L. Vokorokos, “Machine
Learning Approach to Dysphonia Detection,” Applied Sciences, vol. 8,
no. 10, p. 1927, 2018.
[24] H. Kasuya, S. Ogawa, Y. Kikuchi, and S. Ebihara, “An acoustic analysis
of pathological voice and its application to the evaluation of laryngeal
pathology,” Speech communication, vol. 5, no. 2, pp. 171–181, 1986.
[25] A. A. Dibazar, S. Narayanan, and T. W. Berger, “Feature analysis
for automatic detection of pathological speech,” in Proceedings of the
Second Joint 24th Annual Conference and the Annual Fall Meeting of the
Biomedical Engineering Society][Engineering in Medicine and Biology,
vol. 1. IEEE, 2002, pp. 182–183.
[26] M. Eye and E. Infirmary, “Voice disorders database, version. 1.03 (cd-
rom),” Lincoln Park, NJ: Kay Elemetrics Corporation, 1994.
[27] T. A. Mesallam, M. Farahat, K. H. Malki, M. Alsulaiman, Z. Ali,
A. Al-Nasheri, and G. Muhammad, “Development of the Arabic voice
pathology database and its evaluation by using speech features and
machine learning algorithms,” Journal of healthcare engineering, vol.
2017, 2017.
[28] J. R. Orozco-Arroyave, J. D. Arias-Londoño, J. F. Vargas-Bonilla, M. C.
Gonzalez-Rátiva, and E. Nöth, “New Spanish speech corpus database
for the analysis of people suffering from parkinson’s disease.” in LREC,
2014, pp. 342–347.
[29] H. Wu, J. Soraghan, A. Lowit, and G. Di Caterina, “A deep learning
method for pathological voice detection using convolutional deep belief
networks,” Interspeech 2018, 2018.
[30] J. S. Garofolo, L. F. Lamel, W. M. Fisher, J. G. Fiscus, and D. S. Pallett,
“DARPA TIMIT acoustic-phonetic continous speech corpus CD-ROM.
NIST speech disc 1-1.1,” NASA STI/Recon technical report n, vol. 93,
1993.