Speech Recognition using Neural Network
1st Lakshya Jain 2nd Neha Mane
Dept. of Electronics
K.J. Somaiya College of Engineering
Mumbai, India
lakshya.jain@somaiya.edu
1812082
Abstract—A Speech Recognition is a cycle to empower Com-
puters to distinguish and react to human speech sounds. The
project analyzes include extraction procedures applied in speech
acknowledgment, even presence of numerous strategies; the
exactness rate is a central point of interest in speech acknowl-
edgment. In this article we present some notable extraction
procedures, for example, LPC, MFCC, RASTA, PCA, LDA,
and PLP recognize for the most part utilized element extraction
strategy in speech acknowledgment measure.
Index Terms—Speech Recognition, Feature Extraction, LPC,
RASTA, MFCC, PCA, LDA ,PLP
I. I NTRODUCTION
Speech is one of the ancient ways to express ourselves.
Today these speech signals are also used in biometric
recognition technologies and communicating with machine.
These speech signals are slowly timed varying signals (quasi-
stationary). When examined over a sufficiently short period
of time (5-100 msec), its characteristics are fairly stationary.
But, if for a period of time the signal characteristics changes,
it reflects to the different speech sounds being spoken. The
information in speech signal is actually represented by short
term amplitude spectrum of the speech wave form. This
allows us to extract features based on the short term amplitude
spectrum from speech. Speech Recognition is the ability
Fig. 1. Feature Extraction
of machine or program to identify words and phrase from
spoken language and convert them in to machine readable
format. The main intension of speech recognition area is to
evolve techniques and system for speech input to machine.
Speech is the way of communication in human beings, and
it is the dominancy of this medium that motivates research
efforts to allow speech to become a viable human computer
Dept. of Electronics
K.J. Somaiya College of Engineering
Mumbai, India
neha.mane@somaiya.edu
1812088
interaction. The feature extraction is the process of removing
unwanted and redundant information and retains only the
useful information in type of speaker independent automatic
speech recognition.
II. L ITERATURE S URVEY
Ganga Banavath, Sreedhar Potla aimed to describe
an efficient technique or method resulting in effective
speech processing applications. Performance of a classifier
is a function depending on the length of speech sample,
environment etc. They carried out their work using Mel
frequency cepstral coefficients (MFCC) along with Gaussian
Mixture Model (GMM) classifier.
Kishori R. Ghule, R. R. Deshmukh recognized that speech
features extraction and word recognition, these two steps
are followed. After feature extraction feature matching is
performed for word recognition. Their paper describe the
different feature extractions techniques like MFCC, LPC,
LPCC, DWT etc.
P.V. Naresh examined feature extraction techniques applied
in speech recognition, even existence of many techniques; the
accuracy percentage is a key issue in speech recognition.In
their article they have presented some well-known extraction
techniques such as LPC, MFCC, RASTA, PCA, LDA, and
PLP identify mostly used feature extraction technique in
speech recognition process.
Urmila Shrawankar examined that features should describe
each segment in such a characteristic way that other similar
segments can be grouped together by comparing their
features. There are enormous interesting and exceptional
ways to describe the speech signal in terms of parameters.
Anup Vibhute intended to focus on the survey of various
feature extraction techniques in speech processing such as Fast
Fourier Transforms, Linear Predictive Coding, Mel Frequency
Cepstral Coefficients, Discrete Wavelet Transforms, Wavelet
Packet Transforms, Hybrid Algorithm DWPD and their
applications in speech processing.
 III. F EATURE E XTRACTION T ECHNIQUES
A. Linear prediction coding (LPC)
LPC is one of the good signal analysis methods for linear
prediction in speech recognition process. The feature extrac-
tion techniques find out the basic parameters of speech. LPC
is the most powerful method for determining the basic pa-
rameter and computational model of speech. The idea behind
LPC is the Speech sample can be approximated as a linear
combination of past speech samples[1]
B. Mel frequency Cepstral Coefficient (MFCC)
MFCC is most popular feature extraction technique. Fre-
quency bands are placed logarithmically here so it approx-
imates the human system response more closely than any
other system. Due to its advantage of less complexity in
implementation of feature extraction algorithm, only sixteen
coefficients of MFCC corresponding to the Mel scale fre-
quencies of speech Cepstrum are extracted from spoken word
samples in database.[2][3]
Fig. 2. Block Diagram of MFCC
C. Linear prediction cepstral coefficient (LPCC)
The feature extraction is used to demonstrate the speech
signals by finite number of measures of the signals. To obtain
LPCC coefficients the LPC coding is used. LPCC implemented
using autocorrelation method. The main drawback of LPCC is
that the LPCC are highly sensitive to quantization noise. [4]
Fig. 3. Block Diagram of LPCC
D. Discrete Wavelet Transform (DWT)
DWT can be considered as filtering process achieved by
a low pass scaling filter and a high pass wavelet filter. The
transform decomposition separates the lower frequency con-
tents and higher frequency contents of the original signals. The
lower frequency contents provide a sufficient approximation of
the signal while the finer details of the variation are contained
in the higher frequency contents.[3]
E. Wavelet Packet Decomposition(WPD)
The wavelet packet decomposition is the wavelet transform.
In WPD the signal is passed through more filters than discrete
wavelet transform. Wavelet packets are the linier combination
of wavelets. The coefficients in the linear combinations are
computed by a recursive algorithm making each newly com-
puted wavelet packet Coefficient sequence the root of its own
analysis tree.[5]
F. Perceptual Linear Prediction (PLP)
The Perceptual Linear prediction (PLP) technique is devel-
oped by the Hermansky. PLP removes the unwanted informa-
tion of the speech and thus improves speech recognition rate.
PLP is identical to LPC except that its spectral characteristics
have been transformed to match characteristics of human
auditory system.
IV. S PEECH R ECOGNITION T ECHNIQUES
A. Acoustic phonetic approach
This approach of speech recognition is based on finding
speech sound and providing appropriate labels these sounds.
The term acoustic is deals with the different sounds in speech
and phonetic is the study of phonemes in the language. The
basis of acoustic phonetic approach is based on the fact that,
there exist finite and exclusive phonemes in spoken language
and these phonemes are broadly characterized by a set of
acoustic properties that are demonstrated in the speech signal
over time.[6]
B. Pattern recognition approach
Pattern Recognition technique is one of the very important
and actively searched trait or branch of artificial intelligence.
Pattern training and pattern comparison are two steps involves
in pattern recognition approach. Using a well formulated math-
ematical framework and initiates consistent speech pattern
representation for pattern comparison, from a set of labeled
training samples through formal training algorithm is essential
feature of this approach.[7][8]
C. Artificial intelligence approach
To study the thought processes of human and deals with
representing those processes via machines these two basic
ideas involves in Artificial intelligence. Machine behaves like
human being so it is called as Artificial intelligence. AI makes
machine very smarter and useful. The artificial intelligence ap-
proach is the combination of the pattern recognition approach
and acoustic phonetic approach so it is called hybrid approach
of pattern recognition and acoustic phonetic approach and
its recognition procedure is according to how person applies
intelligence on set of measured acoustic features.[9]
 V. M ETHODOLOGY 9) Next we explore the data and compare the waveform,
We followed these steps to implement our model: the spectrogram and the actual audio of one example
from the dataset.
1) We imported the necessary modules and dependencies
such as pathlib and os.
2) We downloaded a portion of the Speech Commands
dataset. The original dataset consists of over 105,000
WAV audio files of people saying thirty different words.
We use a portion of the dataset to save time with data
loading.
3) We select audio files that say the words ”down”, ”go”,
”left”, ”no”, ”right”, ”stop”, ”up” and ”yes” in different
accents,
4) We checked the basic statistics and commands about
the dataset and extracted the audio files into a list and
shuffled it.
5) We then split the files into training, validation and test
sets using a 80:10:10 ratio, respectively.
6) The Audio file was initially a binary file, which we have
to convert into a numerical tensor. We use a function
which returns WAV-encoded audio as a Tensor and the
sample rate.
7) We then examine a few audio waveforms with their
corresponding labels.
Fig. 5. Spectrogram of the Audio files

10) We now build and train a model. For the model, we

use a simple convolutional neural network (CNN), since
we have transformed the audio files into spectrogram
images.

Fig. 6. Training and validation loss curve

Fig. 4. Audio waveforms with their labels
8) We convert the waveform into a spectrogram, which
shows frequency changes over time and can be repre-
sented as a 2D image. This can be done by applying
the short-time Fourier transform (STFT) to convert the
audio into the time-frequency domain.
11) We now run the model on the test set and check its
performance. We observe an accuracy of 90 %.
12) We finally, verify the model’s prediction output using an
input audio file of someone saying ”no.” We can see that
our model very clearly recognized the audio command
as ”no.” Because ’no’ and ’go’ audio commands are very
similar in terms of syllables. Hence the prediction rate
for ’no’ as ’go’ is high. Even humans can make mistakes
differentiating between the two.
 will eventually give better results when provided with more
audio files and more time to train the model.
R EFERENCES
[1] Leena R Mehta 1, S.P.Mahajan 2, Amol S Dabhade” Comparative Study
Of MFCC And LPC For Marathi Isolated Word Recognition System”
Lecturer, Dept. of ECE, CusrowWadia Institute of Technology, Pune,
Maharashtra, India 1 Associate Professor, Dept. of ECE, College of
Engineering , Pune, Maharashtra, India 2 PG Student [SP], Dept. of ECE,
College of Engieering, Pune, Maharashtra, India 3, International Journal
of Advanced Research in Electrical, Electronics and Instrumentation En-
gineering Vol. 2, Issue 6, June 2013
[2] Vimal Krishnan V.R “Features of Wavelet Packet Decomposition and Dis-
crete Wavelet Transform for Malayalam Speech Recognition”, BabuAnto P
School of Information Science and Technology Kannur University, Kerala,
India. 670 567, International Journal of Recent Trends in Engineering, Vol.
1, No. 2, May 2009
[3] Hazrat Ali1,2*, Nasir Ahmad3, Xianwei Zhou2, Khalid Iqbal2 and
Sahibzada Muhammad Ali4 “DWT features performance analysis for auto-
matic speech recognition of Urdu” Ali et al. SpringerPlus a SpringerOpen
Journal 2014.
[4] Nidhi Desai1, Prof.Kinnal Dhameliya2, Prof.Vijayendra Desai3 “ Feature
Extraction and Classification Techniques for Speech Recognition: A Re-
Fig. 7. Confusion Matrix view” 1M.Tech. [Electronics andCommunication] Student, Department Of
Electronics and Communication Engineering, C.G.P.I.T, Bardoli, Gujarat,
International Journal of Emerging Technology and Advanced Engineer-
ing, Website: www.ijetae.com ISSN 2250-2459, ISO 9001:2008 Certified
Journal, Volume 3, Issue 12, December 2013
[5] M.A.Anusuya, S.K.Katti, “Comparison of Different Speech Feature Ex-
traction Techniques with and without Wavelet Transform to Kannada
Speech Recognition”, International Journal of Computer Science and
Information security, Vol.6, No.3, 2010
[6] Santosh K.Gaikward and Bharti W.Gawali, “A Review on Speech Recog-
nition Technique,” International Journal of Computer Applications, vol 10,
No.3, November 2010
[7] Shivanker Dev Dhingra 1, Geeta Nijhawan 2 , Poonam Pandit3, “Isolated
Speech Recognition Using MFCC And DTW”,International Journal of
Advanced Research in Electrical, Electronics and Instrumentation Engi-
neering, Vol. 2, Issue 8, August 2013
[8] M.A.Anusuya, “Speech Recognition by Machine,” International Journal
of Computer Science and Information security, Vol.6, No.3, 2009
[9] Nidhi Srivastava and Dr.Harsh Dev“Speech Recognition using MFCC and
Neural Networks”, International Journal of Modern Engineering Research
(IJMER), march 2007

Fig. 8. Predictions for ”NO”

VI. R ESULTS AND ANALYSIS

We trained the model using the data set which contained
6400 audio files. Our model is giving us good results. We
have reached an accuracy at the highest of 90 %. This accuracy
had been achieved by a Basic Convolution Neural Network,
We have successfully plotted the graph in which it is clear
that the accuracy rate is increasing and most importantly the
loss rate is decreasing constantly. we can experiment more
with our model in terms of increasing the number of epochs ,
finding more optimal learning rates using plot function, more
data augmentation techniques, etc.

VII. C ONCLUSION
We have successfully designed a speech recognition model
which has been trained to identify eight different words being
said by the user. We observe that the model gives an accuracy
of about 90 % on giving around 8000 audio files. This model