See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/269931894

Automatic speech recognition systems: Challenges and recent

implementation trends

Article  in  International Journal of Signal and Imaging Systems Engineering · December 2014

DOI: 10.1504/IJSISE.2014.066600

CITATIONS READS

3 2,426

2 authors, including:

Jamin Atkins
University of the West Indies, St. Augustine
5 PUBLICATIONS   10 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Capacity Building and Research on Smart Grid Technology in the Caribbean Region View project

All content following this page was uploaded by Davinder Sharma on 02 June 2015.

The user has requested enhancement of the downloaded file.

220 Int. J. Signal and Imaging Systems Engineering, Vol. 7, No. 4, 2014

Automatic speech recognition systems: challenges

and recent implementation trends

Davinder Pal Sharma* and Jamin Atkins

VLSI Research Laboratory,
Department of Physics,
The University of the West Indies,
St. Augustine, Trinidad and Tobago
Email: Davinder.Sharma@sta.uwi.edu
Email: Jamin.Atkins@sta.uwi.edu
*Corresponding author

Abstract: Speech recognition is one of the next generation technologies for human-computer
interaction. Speech recognition has been researched since the late 1950s but due to its
computational complexity and limited computing capabilities of the last few decades, its progress
has been impeded. In laboratory settings automatic speech recognition systems (ASR) have
achieved high levels of recognition accuracies, which tend to degrade in real world environments.
This paper analyses the basics of the speech recognition system. Major problems faced by ASR
in real world environments have been discussed with major focus on the techniques used in
the development of noise robust ASR. Throughout the years there have been different
implementation mediums for ASR but Field Programmable Gate Arrays (FPGAs) seems to
provide a unique advantage for the implementation of Digital Signal Processing (DSP) systems
and by extension ASR systems.

Keywords: ASR; automatic speech recognition; DSP; digital signal processing; FPGA; field
programmable gate array; Matlab; FFT; fast Fourier transform.

Reference to this paper should be made as follows: Sharma, D.P. and Atkins, J. (2014)
‘Automatic speech recognition systems: challenges and recent implementation trends’,
Int. J. Signal and Imaging Systems Engineering, Vol. 7, No. 4, pp.220–234.

Biographical notes: Davinder Pal Sharma is a Lecturer of Electronics at the University of the
West Indies, Trinidad and Tobago. He received his PhD from Guru Nanak Dev University, India,
in the year 2004. He has 14 years of teaching and research experience. He is the author of book
Digital Signal Processing and has more than 25 publications in reputed international/national
journals and conferences to his credit. His area of research includes digital signal processing, data
communication, VLSI (very large scale integration) implementation of DSP algorithms, fuel cell
modelling and smart grid system. He is a member of IEEE.

Jamin Atkins received his BSc degree in Computer Science from the University of the West
Indies in 2009. He is currently pursuing PhD in the Department of Physics of the same institution
where his research involves the negation of babble from clean speech. His other academic
interests are: VLSI implementation of speech recognition system and statistical signal processing.
He is a student member of IEEE.

1 Introduction flaws, ASR is an established technology with integration

Automatic Speech Recognition (ASR) has been traditionally
defined as the ability of a computer system to recognise
speech. However, a more recent and accepted description is
given by Jurafsky and Martin (2000) and Rabiner and Schafer
(2007), they define ASR as the designing and action of
systems in mapping acoustic signals to a string of words. The
overall goal of an ASR system is to function accurately
independent of microphone used, speaker’s accent and
acoustic environment. Most of these issues have not been
solved (Raj and Stern, 2005; Huang et al., 2001). Despite its
into mobile phone platforms, automobile stereo systems and
many other devices. It allows hands-free interaction within
environments where key attention must be directed to another
task (Sharma and Atkins, 2010). A typical ASR system is
shown in Figure 1. It consists of a frontend that accepts a
speech waveform as an input and extracts excitation features
also known as observations from it, which represent the
information required to perform recognition. The extracted
features are usually Mel-Frequency Cepstral Coefficients
(MFCCs) but also have other features such as Perceptual
Linear Predictive (PLP) coefficients. The backend attempts to

Copyright © 2014 Inderscience Enterprises Ltd.

 Automatic speech recognition systems
match the speech utterance to previously stored acoustic
models using the language model and dictionary blocks to
constraint the possible decoded outcome to words within the
dictionary, which follow the chosen language model. The
language model is developed by training the system to detect
word sequences which are most likely. The pattern matching
block makes use of the Viterbi algorithm in order to perform
recognition using a set of phone-level statistical-based
acoustic models known as Hidden Markov Models (HMM;
Melnikoff et al., 2002b; Rabiner, 1989). This is done by
calculating the maximum likelihood of the words, which
could have produced the input feature sequence. Recognition
output from the pattern matching block is sent to a decision
rule block responsible for further refining the recognition
output and attempts to maximise the recognition accuracy by
limiting the possible output based on a priori statistics.
Figure 1 Basic speech recognition system (Rodríguez-Andina
et al., 2001)
Rest of the paper is organised as follow: Section 2 describes
the importance of the frontend with emphasis on MFCCs
as the feature of choice; Section 3 discusses the basic
constituent parts of the backend of the speech recognition
system and how they relate to each other. Section 4 indicates
the problems faced by ASR systems while solutions to those
problems are stated in Section 5 with key attention given
to the problem of background noise. An overview of
implementation trends of ASR systems are given in Section 6.
Section 7 discusses Field Programmable Gate Arrays
(FPGAs) based implementations of ASR systems and their
advantages over traditional processing systems. The FPGA-
based implementation results of a Fast Fourier Transform
(FFT) engine, which is required for feature extraction, are
presented in Section 8.
2 The frontend of ASR system
In order for recognition to take place, there is a need to
convert the analog speech signal not only into a digital
format but also into one in which it is possible to
differentiate between the fundamental speech components.
221
Speech can be represented in the phonetic domain though
the use of a set of fundamental components known as
phonemes. For most languages the number of phonemes
ranges from 32 to 64. As such, a Digital Signal Processing
(DSP) frontend is required for analog to digital conversion,
spectral analysis and feature extraction as shown in
Figure 2. The reliability of frontend analysis is critical to the
performance of the speech recognition system, e.g. the frontend
of a speaker independent speech recognition system should be
able to extract features, which are based solely on the message
conveyed by the speech signal with no bearing on the speaker’s
accent, unique voice or acoustic environment.
Figure 2 Speech recognition frontend processing (Huang et al.,
2001)
The frontend is responsible for digital conversion of the
speech signal, environmental noise filtering as well as
feature extraction. In order to perform its function, this
block segments the input signal into frames of 20–30 ms in
length. This length is usually chosen because it allows for
the short time analysis of the speech signal due to the fact
that small segments of the waveform can be considered
stationary for time segments of this length (Rabiner and
Schafer, 2007). The frames are then multiplied by a
windowing function, usually by a Hanning window function
whose equation is given by:
  2 n 
 1      cos   0n N
hn  n     N  (1)
0 otherwise

The result of this multiplication is the transformation of the
window into a bell-shaped curve, as shown in Figure 2. While
the performance of the frontend is paramount to recognition
accuracy, the time taken for frontend processing must be
smaller than the frame overlap used in processing (Gomes-
Cipriano et al., 2004). A thorough review of the algorithms
currently associated with frontend signal processing and
analysis, which include Linear Predictive Coding (LPC),
Mel-Frequency Cepstral Coefficients (MFCCs), Relative
Spectral Analysis (RASTA) as well as different variants of
Perceptual Linear Prediction (PLP), is available in literature
(Rabiner and Schafer, 2007; Huang et al., 2001; Rabiner
222 D.P. Sharma and J. Atkins
and Schafer, 2011; Anusuya and Katti, 2010; Davis and
Mermelstein, 1980).
Of these feature extraction techniques, MFCC and
RASTA-PLP are the most commonly used due to their ability
to provide more robust features in adverse conditions.
2.1 Mel-frequency cepstral coefficients
The aim of the frontend processing system is emulate the
human auditory system. As such, the frontend makes use of
much signal processing in order to match the frequency
response of the eardrum. When sound waves hit the
eardrum, different sections vibrate based on the constituent
frequencies of the incident sound wave (Allen, 1994). The
non-linear sensitivity of the auditory system is modelled by
the use of the Mel-frequency scale, which relates input
frequency to perceived pitch and defined as:
 f 
mels  1125  1   (2)
 700 
Davis and Mermelstein (1980) presented this method of
feature extraction and defined the MFCC as a representation
defined by the real cestrum of a windowed short-time signal
derived from the FFT of the signal, while Zolnay et al.
(2003) show that MFCC representation is beneficial to
speech recognition due the fact that the human perception of
frequency is not linear. The block diagram of frontend of
ASR system is given in Figure 3.
With the exception of the analog to digital converter
(ADC block), all of the other block of the feature extracting
frontend can be implemented in software. The output of the
last five blocks is the result of a computation, which is
performed on the waveform values generated by the analog
to digital converter.
The feature extraction process illustrated in Figure 3
involves the processing of multiple, short overlapping
frames of speech, which is shown in Figure 2. Each frame is
usually a length of 30 ms with a 10 ms overlap usually
sampled at 16 kHz, the minimum sampling frequency
possible since speech may contain components with
frequencies as high as 8 kHz. The abrupt truncation of the
signal into 30 ms segments results in spectral leakage in the
frequency domain. A Hamming window is used to minimise
this leakage before converting to the frequency domain. The
windowed signal is given by
 2 n 
w  n   0.54  0.46 cos   (3)
 N 1 
Figure 3 Block diagram of frontend of an ASR system (Huang et al., 2001) (see online version for colours)
where N represents the total number of samples in the 10 ms
window and n represents the position of the sample under
consideration. The result of the Hamming window is a bell-
shaped transform of the input data.
The FFT block accepts the windowed frame output by
the Hamming block and calculates the Discrete Fourier
Transform (DFT) from the input data. This block is
responsible for the conversion of the input signal from the
time to the frequency domain. The most popular algorithm
for performing the FFT is the Cooley–Tukey algorithms,
which use the divide and conquer approach to recursively
compute the DFT on the input signal. The DFT of the
input signal is of particular importance as the amount
of energy at each frequency is unique to each phoneme.
Early implementation of speech recognition attempted to
recognise speech directly from the DFT of phonemes. The
size of the FFT is performed usually 512 or 1024 points.
One approach to simulating the subjective Mel spectrum
is by the use of a filter bank with a triangular band pass
frequency response and constant Mel-frequency interval
spacing. The filter bank is made up of about 24 filters
(Vu et al., 2010). The Mel filter bank is important as it
has a frequency response similar to that of the human ear. In
this filter bank, more emphasis is given to the lower
frequencies with a log decrease in resolution as the
frequencies increase. The process illustrated in Figure 3
uses a short-time Fourier analysis on pre-windowed (via the
use of the Hamming window) frames resulting in the
calculation of the DFT of each frame λ, via the use of the
FFT. The values are centred on constant Mel-frequency
interval spacing, given in equation (2), which cluster at the
low end of the scale and spread out at larger frequencies. A
Mel-frequency filter bank of M (m = 1,2,…,M) filters can be
defined as:


0 k  f  m  1 ;k  f  m  1

 k  f  m  1
H m k    f  m  1  k  f  m  (4)
 f  m   f  m  1
 f m 1  k
    
f  m   k  f  m  1
 f  m  1  f  m 

where k is the k-th term in the DFT of the input frame.
The DFT is defined as:
N 1
X  k   x  n  e  j 2 n / N
0k  N (5)
n 0
 Automatic speech recognition systems
where x[n] is the n-th sample of N in the frame being
currently processed. The log energy output of each of the M
filters is then calculated as:
 N 1 
S  m  ln   X  k  H m  k 
2
0mM (6)
 k 0 
The log Mel spectrum is then converted back to a time
cepstrum though the use of the Discrete Cosine Transform
(Ahmed et al., 1974), resulting in M MFCC feature vectors
given by:
M 1
c n  S  m cos  n  m  0.5 / M 
m 0
0nM (7)
The value of M is usually between 10 and 15 with 13 being the
most common number (Huang et al., 2001). The delta and
acceleration coefficients of these are also taken, resulting in a
39 dimensional vector for each speech frame. This process is
usually carried out on speech frames of 30 ms with 10 ms
overlap. Huang et al. (2001), Patel and Roa (2010), Staworko
and Rawski (2010) and Gracuarena et al. (2004) provide more
in-depth review of the MFCC computation.
3 The backend of ASR system
The backend of the speech recognition system is responsible
for the decoding of the input feature vectors and performing
the computations necessary for recognition of the speech
pattern initially input into the frontend. That is given a
sequence of observed speech feature vectors O of the form:
O  o1 , o2 , , oT (8)
where oi is the feature vector at time t.
The recogniser aims at calculating the maximum
likelihood given by:
arg max  P  wi | O  (9)
where wi is the i-th vocabulary word. This calculation is
computed through the use of Bayes’ rule, which can be
stated as:
P  O | wi  P  wi 
P  wi | O   (10)
P O 
Given that prior probabilities for P(wi) exist within the
system together with parametric models such as Markov
models, it is possible to solve equation (10) without solving
the more difficult problem of estimating class condition
observation densities P(O|w) (Van Segbroeck and Hamme,
2011). In order to generate the prior probabilities, the
system must be trained on large amounts of labelled data for
the generation of models for matching.
The backend comprises acoustic models for a given
vocabulary, a language modelling module and usually a
confidence scoring module with their arrangement, as shown
223
in Figure 1. The recognition module makes use of a Viterbi
decoder for calculating maximum likelihoods of states from
generated emissions.
3.1 Hidden Markov Models (HMM)
The Hidden Markov Model is a statistically based finite state
automation, which has an associated square transition
probability matrix, defining transition between states A, a
vector of emission probabilities for the generation of
observations b, and an initial state probability vector π
(Rabiner and Juang, 1986). As such the underlying state
transitions can only probabilistically associate with another
observable stochastic process, which is capable of producing
features that can then be observed. An example of an HMM,
in which b is defined by a continuous multivariate Probability
Density Function (PDF), is shown in Figure 4.
Figure 4 Example of an HMM (Huang et al., 2001)
Various constituents of an HMM shown in Figure 4 can be
defined as:
 O= {o1,o2,….,oM}: A set of possible observations,
which can be either discrete output observations. In
terms of speech recognition, the observations are
generally parameterised as10 ms sections of speech.
 Ω = {1,2,…N}: A set of states, which constitute the
model. At every time interval t, the model must be in
one of these states usually denoted s.

A={aij}: An output probability matrix where aij is the
probability of a transition from state i to j also denoted
as:
aij  P ( st  j | st 1  i ) (11)
 B = {bi(k)}: An output probability matrix (in the case of
a discrete probability distribution) where bi(k) is the
probability of emitting observation ok in state i. If
X = X1, X2,…,Xt is the observed output of an HMM, the
state sequence S = s1, s2,…,st is hidden and bi(k) can be
written as:
bi  k   P  X t  ok | st  i  (12)
224 D.P. Sharma and J. Atkins
In the case of speech recognition, the continuous output
probabilities are usually modelled via the use of a Gaussian
Mixture Models (GMMs) with a diagonal covariance for
each mixture. The emission probability is given by:
R n

P  ok | sk   cr  oki  ri ri2  (13)
r 1 i 1
where ok = (ok1,…,okn)T is a deterministic feature vector
with dimension n, cr ranges from 0 to 1 and represents the
weight of the r-th mixture. R denotes the total number of
mixtures and Ψ(o,µ,σ2) is a univariate Gaussian defined as:
1   o   2 
  o;  ;  2
 exp   (14)
2 2  2 2 
 
with means µ and variance σ2. The parameter set for the
diagonal Gaussian Mixture Density of equation (12) is fully
defined by the mean vectors, variance vectors and mixture
weights from all component densities (Kuhne et al., 2011).
All parameters in sk are developed though the training of the
model on an extensive copra.
 π = {πi}: An initial state distribution vector where
 i  P  X t  ok si  i  (15)
In-depth analysis of HMMs and the associated algorithms
are found in the work of Rabiner (1989).
3.2 Acoustic modelling
In the probabilistic approach to speech recognition, given a
sequence of words W = {w1, w2, w3,…,wn} and O, the
acoustic information extracted from the speech signal, the
aim is to find the most probable sequence of words given A
that is maximising P(W|O), which can be found by the use
of Bayes’ formula given in equation (10)
The task of the acoustic model is to compute the
probability P(O|W), that the pronunciation of the word
string W will produce the extracted features A. By modelling
the fundamental sounds of a language, a complete
observation set can be created to cover all possible acoustic
excitations (D’Orta et al., 1987). The study of linguistics has
shown that the phoneme can be considered as the most
fundamental classification for speech (Flanagan, 1972). A
phoneme is formally defined as the smallest contrastive unit
in the sound system of a language. A single phoneme may
have multiple pronunciations, which allow for the use of
HMMs for modelling. Phonemes are best modelled though
the use the left to right or Bakis HMM (as shown in Figure 5),
due to the unidirectional change in their state (Rabiner
et al., 1983). There are currently two methods of modelling
words (Rabiner and Juang, 1993). The first indicates that
there should be as many states as there are phonemes within
a word. This results in the number of states varying from 2
to about 10. The second is that the number of states should
be equal to the average number of observations in a spoken
version of the word. In this case, each state would
correspond to about 10–15 ms for the standard methods of
analysis. Other restrictions may require that each word
model has the same number of states since lower word error
rates are produced when models represent words with a
similar number of sounds. Since words are constituted of
multiple phonemes, they are modelled by concatenating
HMMs together.
Figure 5 Bakis (left to right) HMM used for modelling based
on the number of spoken observations (Rabiner and
Juang, 1993)
3.3 Pattern matching
In the pattern matching step, the results are obtained as the
sequence of HMMs, which maximise the probability of an
observation sequence. While the most obvious computation
of this probability is via the sequencing of all possible
states, the computational complexity is exponential. The
Viterbi algorithm is used in a more efficient manner for
probability computation. This algorithm is similar to the
forward algorithm used in training the HMMs, except is
maximised instead of summing the forward probabilities
(Rabiner, 1989).
3.4 Language modelling
As the size of an ASR system increases, so does the number
of possible recognition output and hence the probability of
erroneous recognition output symbols (Paliwal and Yao,
2010). The language model seeks to refine the output of the
pattern matching section by attempting to limit the number
of invalid output strings by placing a confidence score on
consecutive output words or phonemes. One of the most
commonly used language models is the N-gram model
(Wang and Li, 2009). The N-gram language model
calculates the probability of a word ending an N word
sequence given the last N–1 word, i.e. P(wN|w1w2….wN–1).
The N-gram model presents a confidence score based on the
last N–1 word via the use of estimates. The chain rule of
probability allows for the decomposition to the probability
computation for word sequences. Given a word sequence
W = {w1, w2,…, wN}, P(w1,…,wN) can be calculated as:
P  w1  wN   P  w1  P  w2 | w1   P  wN | wN 1 
N (16)
 P  wk | wk 1 
k 1
Using the Markov assumption that the probability of a word
in the sequence depends only on its predecessor, the bigram
model allows for the word sequence probability computation
using equation (17) (Rapp, 2008):
N
 
P w1N  P  wk | wk 1  (17)
k 1
 Automatic speech recognition systems
It is noted that the language model should contain strings
related to the task for which the recognition system is being
developed, as this would increase the probability of jargon
within the application environment being added to the
language model. Since language is creative, every possible N-
gram cannot be modelled and smoothing techniques must be
used to remove zero probabilities. The ASR system can use
the language model block as a manner of validating output of
the system by defining valid output as any recognised sting
with an N-gram probability above a threshold limit. It should
also be noted that N-gram modelling can also be applied at
the phoneme level for providing a confidence score on a
phone string (Wessel et al., 2001).
4 Challenges in automatic speech recognition
systems
While much work has been done in the field of ASR to
improve recognition accuracies, the problem of providing
acceptable recognition rates under real-world conditions in
real time still proves illusive. Considering the entire speech
recognition system, it is not difficult to say that there are still
many problems to be solved. In the frontend, for example, the
required features for speech recognition should be constant
regardless of sex, accent and age. The development of such
features would not only mean a large reduction in the amount
of training data required for the generation of acceptable
model parameters but a standard frontend capable of
extracting features from a wide range of languages, which
would then result in the possibility of multiple language-
based models. While HMMs together with the use of GMMs
provide an adequate acoustic modelling scheme for speech
from the phonetic to the word levels, the amount of training
data required for acceptable results is proportional to the size
of the recognition vocabulary (Choi et al., 2010b). Provision
of language models capable of reducing perplexity would
increase the capacity of current systems while retaining a
low-error rate. Another problem which may be considered to
be one of the most imperative is that of noise. Currently,
speech recognition systems give acceptable performance
under acoustically controlled laboratory conditions; however,
performance tends to decrease significantly in real-world
environments. This is due to the presence of the non-
stationary noise in the actual operating environments
(Betkowska et al., 2007). The two main sources of noise
affecting ASR are convolution noise due to changes in
microphone and channel conditions and background noise.
5 Methods of creating noise robust systems
Many approaches have been taken towards solving the
noise problem and may be placed in two groups: feature
compensation methods and model adaptation methods (Raj
and Stern, 2005). The two main sources of noise-affecting
ASR as shown in Figure 6 are convolution noise due
to changes in microphone and channel conditions and
225
background noise. The solution to problem of convolution
noise is relatively simple since this form of noise becomes
an additive constant in the log domain and can readily be
removed (Van Segbroeck and Hamme, 2011). Some common
classes of noise are babble, factory, aircraft and car noise
(also known as Volvo noise) (Krishnamurthy and Hansen,
2009; Maithani and Tyagi, 2008; Schuller et al., 2009).
Figure 6 Model of the acoustic environment showing sources
of noise
Noise
n[m]
y[m]
s[m]
Channel h
Channel h +
Clean speech
Noisy Speech 
Each different class of noise is the resultant of different
environmental processes. Car noise, for example, refers to the
noise presents in the interior of a car and may be split into four
groups. First is wind noise, which is produced by turbulence at
the edges of the vehicle and is proportional to the velocity of
the vehicle. Second is driving noise, a result of the wheels,
driving and suspension. This component is highly influenced
by the road surface and wheel type with a rough surface being
more influential than a smooth one. The third group is the
engine, which depends on the load and number of revolutions.
The fourth group results from the rattle and squeaks produced
by interior components of the vehicle (Grimm et al., 2007).
Speech corrupted by Volvo noise tends to be well audible even
below 0 dB. This is due to the fact that most of the energy of
Volvo noise lies above 2000 Hz (Schuller et al., 2009).
Speech babble is one of the most challenging noise
interferences for all speech systems since it occupies the same
feature space as clean speech. Babble is commonly defined as
a noise encountered when a crowd or group of individuals
is speaking together. Babble is an overlap of multiple
conversations and individual speakers. New analysis has
shown that babble is more accurately modelled as a sum of
conversations rather than individual speech streams of
conversations overlapped with each other (Krishnamurthy
and Hansen, 2009). Babble noise is dependent on a number of
factors. These include the number of speakers, the subject as
well as the emotional stress levels of the individuals. An
investigation of babble done by Krishnamurthy and Hansen
seeks to generate a model for the noise. Babble affects
clean speech by increasing the number of Gaussian hops per
feature frame. The possible corruption by babble can then be
calculated by counting Gaussian hops.
Factory and aircraft noise have a few similar properties,
one of them is that they are relatively stationary. The factory
noise is known to fluctuate from moment to moment but
the noise is constant on the average over a long interval of
time (Sacerdote, 1959). Factory noise like aircraft noise is
predominantly due to fans and turbines.
226 D.P. Sharma and J. Atkins
Speech recognition systems are also affected by white and
coloured noises. White noise contains equal energy at all
frequency components with a zero mean value. White noise is
essentially a random process usually considered with a
Gaussian distribution. Coloured noise does not vary complete
randomly as there is a co-variance between samples at different
time-indexes. Since white and coloured noises have different
energy-frequency characteristics, they have different effects on
the speech signal. Kalman and Wiener filtering have proven
very effective in negating these kinds of noise in front-end
speech processing.
5.1 Speech enhancement techniques
Speech enhancement techniques attempt to suppress noise
in the speech signal at the risk of degrading the original
clean signal (Paliwal and Yao, 2010). Both spectral
subtraction and the various filtering techniques belong to
this sub-category of feature compensation.
The method of spectral subtraction (Huang et al., 2001)
involves the assumption of the environment as shown in
Figure 6 that the clean input signal s[m] has been corrupted
by some additive noise n[m] resulting in
y  m  s  m  n  m (18)
Statistical independence of s[m] and n[m] results in the
power spectrum of y[m] being equal to the sum of the
expected values of the individual power spectra given by:
E  Y  f    E  S  f   E  N  f  
2 2 2
(19)
     
The general idea in spectral subtraction (Choi, 2004) is that,
given in periods of speech silence, when the output power
spectrum is assumed due to noise, through the use of a running
noise, power average E[|N(f)|2] can found as given by:
M 1
1
Nf    |Y  f |
2
i
2
(20)
M i 0
Equations (19) and (20) imply that |S(f)| is given by:
2 1 
S  f   Y  f   N  f   Y  f  1 
2 2 2
 SNR  f  
(21)
 
where SNR(f) is the frequency-dependent signal to noise
ratio given by:
Yf
2 N 1
SNR  f   (22)
N f 
2 n0
From equation (20) we can calculate the MFCCs of the cleaned
speech signal and used for recognition. It should be noted that
an additional module would be required for speech detection.
Many novel ways have been developed for the calculation of
SNR(f) (Claes and Van Compernolle, 1996; Morgan and
Hermansky, 1992; Van Compernolle et al., 2007).
Wiener filtering (Neves et al., 2008) attempts to
generate a filtering operation, which results in a linear
estimate of the clean speech. The general approach is taken
to minimise the mean-square value of the error signal,
which is defined as the difference between the desired
response and the filter output. While the Weiner filter is
known for producing acceptable results under conditions
where stationary noise is presented, its reliability rapidly
deteriorates if the inputs become non-stationary. The
following equations explain the process mathematically:

xe  n    h  m y  n  m (23)
m 
  
 
2
E    x  n    h  m  y  n  m   (24)
  m   
The expected output xe[n] is generated via convolution of
the noisy signal with h[n] as stated in equation (23). Using
the Minimum Mean Square Error (MMSE) estimation of
equation (24), an attempt is made to minimise h[n].
5.2 Missing feature techniques
Missing Feature Theory (MFT) attempts to determine the
level of reliability for different spectral regions in the
speech spectrogram (Raj and Stern, 2005). This is achieved
by segmenting the time-frequency spectrogram into cells
and attempting to determine which ones are unreliable or
missing for subsequent processing. Since MFT compensates
for additive noise distortions by locating the unreliable time-
frequency cells, then performing the recognition on the
partial feature vectors. Firstly, a missing feature mask must
be estimated to differentiate between the spectrally reliable
regions (Van Segbroeck and Hamme, 2011). The application
of missing feature techniques to ASR is due to the influence
of two complementary fields of signal analysis with the
majority of mathematical approaches to missing feature
(MF) recognition being developed for the completion of
partially occluded objects in the field of visual pattern
recognition (Raj and Stern, 2005). Raj and Stern point out
that the mathematics developed for missing feature
recognition can be applied to types of signal restoration,
which can prove useful in the identification of degraded
signal components with the potential for restoration though
the use of MF techniques.
The procedure for MF analysis requires 20 ms frame-
based segmentation of the conventional frontend followed
by the computation of the frames power spectrum given by:
2
Z p  m, k   w  n  mL  x  n  e
 j 2  n  mL  k / N
(25)
where Zp(m,k) represents the k-th frequency band of the
power spectrum of the m-th frame of speech, w(m)
represents the desired windowing function (usually Hamming),
x(n) represents the n-th sample of the speech signal,
L represents adjacent inter-frame shift and N represents the
number of samples per frame. A perception-based filter
such as the Mel Filter bank may be applied to emulate the
frequency sensitivity of the human as:
X p  m, k   hk  i  Z p  m, i  (26)
i
 Automatic speech recognition systems
where hk(i) represents the response of the k-th filter in the
filter bank. The MF technique makes use of the additive
noise composition given in equation (18), which shows that
the output power spectrum of the corrupted speech signal is
the sum of power spectrums of the corrupting noise and
clean speech. A hard masking system such as the binary
system (Kim and Loizou, 2010; Srinivasan and Wang,
2007) scores cell as being either fully reliable or not. In
general, the hard masking system is essentially a maximising
operator between the speech signal and corrupting noise,
generating the score based on the high of the two signals.
The soft or fuzzy masking technique attempts to estimate
the probability of each cell being dominated by speech. This
mask mt can be generated via the use of a sigmoid function
(Barker et al., 2000):
 1  degree of parallelism which can be achieved, is based on the
mt    (27)
 1  exp     st  nt     
 
where ρ is the slope and θ is a preset threshold. It has been
observed that soft masking provides better decision than
hard masking but ρ tends to be sensitive to noise type
(Van Segbroeck and Hamme, 2011). Another modification
required for MFT implementation occurs during the evaluation
of the acoustic model. At this state, the probability of being
reliable for each feature should be taken into account when
computing the acoustic score.
6 Implementation trends for ASR systems
Speech recognition has seen a shift in implementation schemes
over the years. Initially, the entire system was implemented in
software resulting in less than real time recognition of speech
(Lee et al., 1990). Currently, much work is being done in the
mapping of ASR systems onto FPGA as an option over
implementations making use of digital signal processors. The
reconfigurable nature of FPGA as well as the relative ease of
subsystem implementation on FPGA makes it an interesting
option for system deployment. FPGA-based implementations
not only reduce the software overhead and system support
requirement, but also increase the real-time factor while
maintaining the level of accuracy of speech recognition
systems. As a result of this, in order for speech recognition to
become viable, the parallel nature of pattern recognition
algorithms must be exploited (Melnikoff et al., 2002a; Ke et al.,
2008).
The SPHINX system developed by Carnegie Mellon
University provides an example of the pure software
approach to speech recognition (Lee et al., 1990). The
system produced accuracies of 96% for a 997-word
vocabulary. The main short fall of the software-based
approach is that the von Neumann architecture of most
CPU-based systems limits the amount of true parallelism
which can be exploited.
Meng et al. (2005) show the complications involved in
optimising the speech recognition algorithms for digital
signal processor. This includes use of low-level languages,
decomposition of the algorithm to identify computationally
227
extensive sections and pre-computation of frequently
use parameters (which may result in additional memory
requirements). Equipped with wider inner bus lines, the ability
to perform numerous simultaneous commands and high speed
sum of product calculations, digital signal processors are
capable devices for signal processing (Heungsuk et al., 2001;
Muzaffar et al., 2005). However, while optimisation techniques
enable the realisation of real-time functionality with the
recognition of a 20 ms frame being accomplished in 3.090 ms a
time much shorter than the 10 ms inter-frame spacing. Such
implementations have achieved recognition accuracies ranging
from 85% to 98% in implementations of small vocabulary
systems. The major pitfall of the DSP hardware approach is the
fact that in order to convert the system to a real-time capable
system, code optimisations must be performed in assembly,
which can prove to be time-consuming task, as well as the
fixed architecture available.
FPGAs, which allow the implementation of RISC soft-core
processors, provide a framework in which the entire speech
recognition system may be mapped onto the FPGA platform
for a cheap embedded solution. The addition of a PROM for
storing the reference HMMs is also easily achieved. The fact
that FPGAs can provide a hardware implementation of
algorithms allows for an RISC processor to be incorporated
into the system design and does not require power in which the
available resources are unused and results in power saving,
board real-estate and overall system cost. As a result smaller
and cheaper systems can be implemented with improved
performance as shown in Figure 7, where all data were
obtained from a laboratory environment.
Many attempts have been made to port Hidden Markov
Model-based speech recognition to FPGA. Notable attempts
include the work of Hauck (1998) and Choi et al. (2010b) in
which full systems were implemented completely with
independent external memories. FPGAs provide an ideal
platform for speech recognition systems since they allow for
the implementation of true parallelism.
The frontend may be implemented in software and run
on a soft-core processor such as the Xilinx Microblaze or
the Altera Nios II platforms (Xilinx Foundation, 2011a;
Xilinx Foundation, 2011b; Xilinx Foundation, 2011c). The
approach taken by Cheng et al. (2009) was to implement a
GMM calculator which is used to calculate the emission
probabilities at each state. It was found that about 65% of the
processing time was spent calculating emission probabilities
with the actual recognition by Viterbi decoding taking up
33% (Miura et al., 2008). A GMM module was proposed to
perform all emission probability calculations in parallel. A
similar approach was also taken by Choi et al. (2010b).
Implementation trends indicate that the gain over software
of hardware implementation of the frontend is miniscule
(Melnikoff et al., 2002b; Gonzalez-Concejero et al., 2006;
Mosleh et al., 2010). It becomes more resource efficient to
implement only the backend in hardware. Most implementations
attempt to improve the efficiency of the emission probability
calculator and Viterbi decoder (Ke et al., 2008; Lim et al.,
2006).
228 D.P. Sharma and J. Atkins
Figure 7 Performance comparison of different ASR implementation technologies (Choi et al., 2010a; Lee et al., 1990; Meng et al., 2005)
(see online version for colours)
7 FPGA-based implementation of ASR systems
FPGA were developed from the advancement of
reprogrammable and reconfigurable hardware such as the
Programmable Logic Device (PLD; Stitt, 2011). In general,
it consists of many basic hardware cells which can be
reprogrammed to implement small combinational or sequential
circuits. These cells called logic blocks are placed on a matrix
of horizontal and vertical pathways as shown in Figure 8,
which can be programmed to interconnect cells resulting in
signal routing throughout the device. On the outside of this
matrixes are rows of Input/Output Blocks (IOB) for
interfacing with other devices.
Figure 8 Block diagram of a typical FPGA architecture
(Stitt, 2011)
There has been a great deal of interest in creation of custom
application-specific reprogrammable systems. In such an
approach, the logic is supplied by the designer and tools are
used to automatically map this circuitry into FPGA logic.
This reduces the cost in terms of money and time for
implementing algorithms in hardware (Sharma, 2009). A
comparison of conventional CPU/DSP processor and FPGA
technologies for the implementation of DSP algorithms
shows that both the CPU and DSP are controlled by an
operating system, which runs algorithms in pseudo parallel
by context switching between process threads with all
threads sharing the same memory and limited multipliers
among other resources. FPGA implementation of an
algorithm allows each subsection to be implemented as an
individual core with pathways connecting each core resulting
in a highly efficient pipeline, without the restrictions of the
one memory Von Neumann architecture. Hauck (1998)
provides an in-depth review of the role of FPGA in
reprogrammable computing and the benefits it offers over
ASICs.
Many implementations of ASR systems on FPGAs have
been successfully attempted. As stated earlier in this section
it is evident that FPGAs provide an optimal platform
for implementation. Current implementations fall into three
major categories:
 Soft-core implementations used for running software-
based ASR.
 Implementation of parts of the speech recognition
system as discussed in Sections 2 and 3.
 Implementation of the entire speech recognition system
(system-on-chip).
The choice of implementation method may be based on the
amount of resources available as the real estate needed for
the implementation of the different categories differ greatly
with the full System-on-Chip (SoC) structure. Floating
point requires the most and the partial implementation requires
the least space. The soft-core implementation, however, has
the advantage that it provides a Reduced Instruction Set
Computing (RISC) based microprocessor which may be used
to run modules different from the speech recognition algorithms.
7.1 Soft-core processor-based implementations
The soft-core implementation approach involves the
implementation of a microprocessor onto the FPGA and
then use microprocessor to run the speech recognition
algorithm usually written in C.
Soft-core processors are generally used to create an
FPGA-based SoC. In these systems, the central-processing
unit core controls the circuit and minimal computations while
other parts of the circuit are responsible for interfacing and
parallel processing. One major advantage of this method is
that, since the soft core is a general computing engine, it may
be used to run another necessary process, a feature which
provides a system with a microprocessor for less intense
computation, while still providing the necessary hardware
for the computationally demanding speech recognition
system. Some popular soft cores include: LEON, OpenRISC,
MicroBlaze, Nios II and Cortex-M1. Table 1 provides a
comparison of available soft cores.
 Automatic speech recognition systems 229

Table 1 Comparison of available soft cores (1-CORE Technologies, 2012)

Word Size Pipeline Cycles Per Hardware Floating Size in Logic

Core Architecture Comments
(bits) depth Instruction Multiplier Point Unit Elements
S1 Core SPARC-v9 64 6 1 YES YES 37,000–60,000 A single core version of
Ultrasparc T1
LEON3 SPARC-v8 32 7 1 YES YES 3500
LEON2 SPARC-v8 32 5 1 YES Extension 5000 Core Unmaintained in
Available favour of LEON3
OpenRISC 1200 OpenRISC 32 5 1 YES NO 6000
1000
MicroBlaze MicroBlaze 32 3,5 1 Optional Optional 1324 Limited to Xilinx Devices
aeMB MicroBlaze 32 3 1 Optional NO 2536 Open-Source clone of
MicroBlaze
OpenFire MicroBlaze 32 3 1 Optional NO 1928 Proprietary Altera Devices
Nios II/f Nios II 32 6 1 YES Optional 1800
NiosII/s Nios II 32 5 1 YES Optional 1170
NiosII/e Nios II 32 0 6 NO Optional 390
LatticeMicro32 LatticeMicro32 32 6 1 Optional NO 1984 Not limited to Lattice
devices.
Cortex-M1 ARMv6 32 3 1 YES NO 2600
DSPuva16 DSPuva16 16 0 4 YES NO 510
PicoBlaze PicoBlaze 8 0 2 NO NO 192 Only on Xilinx Platforms
PacoBlaze PicoBlaze 8 2 NO NO 204 Open-source PicoBlaze
clone
LatticeMicro Latice Micro 8 0 2 NO NO 200

The use of the soft-core for speech processing is best suits
to the designer who may use the speech recognition system
as a secondary module in his design. It allows for taking
advantage of the FPGA parallelism in sections of the
algorithm, which may be time consuming if ran on a general
microprocessor. It provides the best balance between
performance and flexibility of all three methods.
Soft-core-based implementations may be full software or
a hardware-software co-design (Cheng et al., 2009). The full
software-based implementation running on the soft-core does
not take full advantage of the resources and parallel nature of
the FPGA. Most co-designs implement the MFCC feature
extraction via the use of the soft-core with the computationally
demanding GMM computation and Viterbi decoding
implemented in the FPGA fabric (Lim et al., 2006).
7.2 Partial implementations
Partial implementations involve the implementation of small
segments of the speech recognition system onto FPGA. This
method can prove advantageous when only minimum
computational boosts are required to meet performance
goals. Calculations made by Cheng et al. (2009) and Lim
et al. (2006) indicate that the most of the time is spent on the
GMM emission calculations of the speech pattern matching
block. They also indicate that limited computational gains
can be achieved by hardware implementation of the
frontend of the system; Cheng et al. (2009) indicate that
only 6.02% of the timing profile for speech recognition
involves feature extraction, while GMM calculation and
Viterbi decoding require 63.1% and 30.88%, respectively. It
should be obvious that partial ASR systems require a
controlling processor, which feeds input and receives the
output of the system. This controller may be a soft-core on
the same or different FPGA or a traditional microprocessor.
According to Choi et al. (2010b), implementations can also
contain sections, which read and write to a shared RAM
using a valid flag to indicate when processing should be done.
Vu et al. (2010) implement a low power architecture of an
MFCC-based frontend, which consumes little resources and
power but matches performance of microprocessor-based
systems. The flexibility offered by partial system
implementation was also utilised by Veitch et al. (2010) and
also provided an IP (Intellectual Proprietary) core for the
acceleration of HMM-based speech recognition systems. By
decoupling Gaussian calculations from the rest of the
system, results were calculated with minimal communication
between the backend search software and the FPGA-based
Gaussian core.
Designers may choose to implement or improve sections
such as emission probability computation though the GMM
calculator (Miura et al., 2008; Schuster et al., 2006). Partial
implementations seek to accelerate speech-recognition system
implementations, which require minimal speed gains or
whose speed is fairly adequate for its use. The frontend
implementation found in the work of Ahadi et al. (2003)
provides a good example of partial implementation as it
provides facilities for changing the speech enhancement
algorithms used in noisy environment. The FPGA platform
allows for the easy redirection of outputs to newly implemented
cores, which facilitate dynamic hardware choice.
230 D.P. Sharma and J. Atkins

Table 2 Comparison of various FPGA-based implementation methods

Full Implementation Soft-Core Implementation Partial Implementation

 Full parallelism resulting in
extremely high performance.
 Takes full advantage of
underlying hardware.
Advantages
 High resource utilisation.
 High power consumption.
 Flexibility. System
Disadvantages components are not readily
available for use by other
algorithms.
High: entire system implemented
Resource in logic elements.
Utilisation
Low: Requires intricate planning
Ease of
of system state flow and resource
Implementation
management.
 Course with very small foot
print can be used to run ASR
algorithms.
 Cores can be used to run other
tasks.
 Sections of the system can be
developed in software which is a
simpler process.
 Use of floating point numbers is
limited by the core architecture.
 Some cores lack the capability to
use hardware multipliers.
 Limited scope to take advantage
of parallel nature of both
algorithm and FPGA.
Low-high: Determined by choice of
soft-core as well as whether
accelerator cores are also
implemented.
Medium
 Provides acceleration of
computationally intensive
algorithms.
 Best performance gain: footprint
ratio of three techniques.
 Low power consumption.
 Portable.
 Implementation may be used by
other algorithms requiring the
same processing.
 May require data type
conversion for use with driver
system.
 While this method provides a
speed gain, the overall ASR
system may be slower than the
full implementation.
Low-medium: Highly dependent on
the section of core which is to be
implemented.
High

7.3 Full system implementations the low and mid-high end of the FPGA spectrum in terms of
both capability and price are the Spartan-3E and Virtex-5.
There have been many successful attempts to develop full
Using the system generator block, the relevant net list was
system-on-chip implementations of the ASR system. The
developed to implement the circuit on each platform. When
Speech Silicon project (Schuster et al., 2006) discusses the
compared to the output of the Simulink model, the results
design of an FPGA-based system on a chip implementation,
generated by the Virtex-5 showed a great correlation.
which is capable of performing continuous speech recognition
Apart from correlation with theoretical values, the resource
on medium-sized vocabularies. The ability of FPGAs to be usage of the two devices was also compared. The presence
interfaced with many different devices makes it a natural of dedicated multipliers and summers allow for the
candidate for use as a control device powered by speech implementation of high-level logic without the use of CLBs
recognition. Coupled with the wealth of resources available, the to build these systems. The result of this is faster
entire speech chain shown in Figure 1 can be implemented in performance and more efficient implementation.
the FPGA fabric. Choi et al. (2010a, 2010b) provide good Tables 3 and 4 show the utilisation summary for FFT
examples of SoC implementation of ASR. It should be noted
engine implementation on the Virtex-5 and Spartan-3E
that majority of the logic of the implementation is contained in
FPGAs, respectively. The tables were produced by the
the sections of the emission probability calculator and the Viterbi
Xilinx’s design suite (during place and route step) software
decoder. Implementations in the work of Gomes-Cipriano et al.
as part of the output. The values in this table were reported
(2004), Elmisery et al. (2003), Melnikoff et al. (2002a) and
by the Xilinx project navigator after implementation of the
Schuster et al. (2006) show that the clock speed required to
perform the recognition is in the order of 102 MHz and well out system on the identified platform.
of the Giga hertz region of traditional microprocessors. Table 3 Device utilisation summary of Virtex-5 FPGA
Table 2 shows a comparison of the three methods of
implementing ASR technology on FPGA. Device Utilisation Summary Virtex-5
Slice Logic Utilisation Used Available Utilisation
Number of slice registers 3635 32,640 11%
8 Implementation of FFT engine for ASR system Number of Slice LUTs 3253 32,640 9%
Number used as logic 2044 32,640 6%
As a case study, a 1024-point 4-channel FFT engine, which
is used in the frontend of the ASR system has been Number used as Memory 1089 12,480 8%
simulated and implemented on various FPGAs. Using Number of occupied Slices 1154 8160 14%
Simulink and Xilinx DSP system generator, a model of FFT Number with an unused Flip Flop 342 3977 8%
engine shown in Figure 9 was developed and implemented
Number of DSP48Es 10 288 3%
on two different platforms. These platforms selected from
 Automatic speech recognition systems 231

Table 4 Device utilisation summary for Spartan-3E FPGA DSP48 blocks, a dedicated piece of hardware, which allows
for efficient multiplication and accumulation on the Virtex-5. It
Device Utilisation Summary for Spartan 3E1600
should also be noted that the structure of the V5 slice is
Logic Utilisation Used Available Utilisation slightly more complex than that of the S3E, with the Virtex-5
Total Number Slice Registers 4418 29,504 14% being able to implement Boolean equations of six input
Number of 4 input LUTs 4035 29,504 13% variables, while the S3E is limited to four inputs. With
Number of occupied Slices 3101 14,752 21% further optimisation of the implementation, a decrease in
Number of Slices containing only 3101 3101 100% utilisation by 5% is possible, this further solidifies the
related logic premise that algorithms 4–5 times more complex than the
Number of Slices containing 0 3101 0% FFT may be implemented while not consuming the entire
unrelated logic resource set available.
Total Number of 4 input LUTs 4380 29,504 14% Table 5 shows the comparison of the resource utilisation
Number of RAMB16s 7 36 19% for a 1024-point radix-2 implementation of the FFT on various
Number of MULT18X18SIOs 16 36 44% FPGAs. It becomes obvious that specialised hardware such
as the Xilinx DSP48 blocks and the Altera 18  18 blocks
This non-optimal implementation of a very important signal greatly reduce the device latency. The Xilinx devices
processing function consumes 11–14% of the total number of provide the DSP48 blocks, which is capable of performing
slice logic but less than 20% of the total logic implementation complex arithmetic as well as accumulation and greatly
area of the Virtex-5 and Spartan-3E, respectively. reduce the memory requirements and footprint of the
The difference in device utilisation between the Virtex 5 design. The Altera FPGAs however tend to have a shorter
(V5) and the Spartan 3E (S3E) are due the presence of latency than Xilinx-branded counterparts.

Figure 9 Simulink model used for FFT engine simulation on FPGA (see online version for colours)

Table 5 Comparison of typical 1024-point FFT implementation on various FPGA platforms

Spartan 6 Spartan 3A DSP

Slice Logic Utilisation Virtex-5 Virtex-6 Spartan 3E1600 Cyclone III Stratix III
XC6SLX150T XC3D3400A
Number of Slice Registers 3635 2774 3344 2618 4418 4650 4458
Number of Slice LUTs 3235 2744 2567 4144 4035 3857 2480
Flip Flops 1089 3626 3701 3779 4380 155,940 155,904
Maximum Clock Frequency(Hz) 380 395 210 180 66 244 413
Latency(µs) 8.84 18.64 74.17 86.53 834.23 4.19 2.48
18 k Block RAM 5 9 33 17 – 20 12
Number of DSP48Es 16 12 12 12 – – –
18  18 Multipliers – – – – 16 24 10
232 D.P. Sharma and J. Atkins
9 Conclusion
This paper provides an overview of HMM-based ASR system,
which accepts speech signals and produces outputs symbols
that are ideally a unique representation of the input speech
signal. The current problems of speech recognition systems
have been discussed with keen focus on environmental noise as
well as currently proposed solutions. It has been noted that
although these proposals provide some relief, yet they were
highly dependent on the kind of noise present. A side effect of
which could prove detrimental in real-world environments,
where noise source and characteristics may change rapidly.
Highlighted solutions attempt to deal with generalised classes
of noise instead of the individual types. While similarities in
noise properties may result in general stationary/non-stationary
classification, the effect of particular noise groups on the
feature vector may be different, rendering identical treatments
ineffective. ASR system implementations on various targets
have been discussed with a conclusion that FPGAs are the best
solution for such implementations.
Acknowledgements
Authors are thankful to the University of the West Indies for
providing necessary funding through Grant No. CRP.4.
MAR11.4 to carry out research on the project ‘Development
of algorithms and systems for robust speech recognition in
the noisy environments’.
References
1-CORE Technologies (2012) Soft Cores for FPGA. Available
online at: http://www.1-core.com/library/digital/soft-cpu-
cores/soft-cpu-cores.pdf (accessed on 1 November 2013).
Ahadi, S.M., Sheikhzadeh, H., Brennan, R.L. and Freeman, G.H.
(2003) ‘An efficient front-end for automatic speech
recognition’, Proceedings of the 10th IEEE International
Conference on Electronics, Circuits and Systems (ICECS
2003), pp.14–17.
Ahmed, N., Natarajan, T. and Rao, K.R. (1974) ‘Discrete cosine
transform’, IEEE Transactions on Computers, pp.90–93.
Allen, J.S. (1994) ‘How do humans recognise speech’, IEEE
Transactions on Speech and Audio Processing, pp.567–577.
Anusuya, M.A. and Katti, S.K. (2010) ‘Front end analysis of
speech recognition: a review’, International Journal of
Speech Technology, pp.99–145.
Barker, J., Josifovski, L., Cooke, M. and Green, P. (2000) ‘Soft
decisions in missing data techniques for robust automatic
speech recognition’, Proceedings of Interspeech, pp.373–376.
Betkowska, A., Koichi, S. and Sadaoki, F. (2007) ‘Robust speech
recognition using factorial HMMs for home environment’,
EURASIP Journal on Advances in Signal Processing, No. 1,
pp.1–9.
Cheng, O., Abdulla, W. and Salcic, Z. (2009) ‘Speech recognition
system for embedded real-time applications’, EEE International
Symposium on Signal Processing and Information Technology
(ISSPIT), pp.118–122.
Choi, E. (2004) ‘Noise robust front-end for ASR using spectral
subtractions spectral flooring and cumulative distribution
mapping’, Proceedings of the 10th Australian International
Conference on Speech Science & Technology, Sydney,
pp.451–456.
Choi, J., You, K. and Sung, W. (2010a) ‘An FPGA implementation of
speech recognition with weighted finite state transducers’,
Proceedings of 2010 IEEE International Conference on
Acoustics Speech and Signal Processing (ICASSP), IEEE,
pp.1602–1605.
Choi, Y-K., Kinsun, Y., Jungwook, C. and Wonyong, S. (2010b)
‘A real-time FPGA based 20000 – word speech recognizer
with optimized DRAM access’, IEE Transactions on Circuits
and Systems, pp.1–13.
Claes, T. and Van Compernolle, D. (1996) ‘SNR-normalisation for
robust speech recognition’, IEEE International Conference on
Acoustics, Speech, and Signal Processing, pp.331–334.
Davis, S. and Mermelstein, P. (1980) ‘Comparison of parametric
representation for monosyllabic word recognition in
continuously spoken sentences’, IEEE Transactions on
Acoustics, Speech and Signal Processing, pp.357–366.
D’Orta, P., Marco, F., Alessandro, M. and Stefano, S. (1987) ‘An
automatic speech recognition system for the Italian language’,
Proceedings of the 3rd Conference on European Chapter of the
Association for Computational Linguistics, ACM, pp.80–83.
Elmisery, F.A., Khalid, A.H., Salama, A.E. and Hammed, H.F.
(2003) ‘A FPGA-based HMM for a discrete Arabic speech
recognition system’, Proceedings of the 15th International
Conference on Microelectronics, pp.322–325.
Flanagan, J.L. (1972) Speech Analysis, Synthesis and Perception,
Springer, pp.406–426.
Gomes-Cipriano, J.L., Nunes, R.P. and Barone, D.A.C. (2004)
Implementation of Voice Processing Algorithms in FPGA
Hardware, Technological and Exact Sciences Institute, Novo
Hamburgo, Brazil.
Gonzalez-Concejero, C., Rodellar, V., Alvarez-Marquina, A., Martines
de Incaya, E. and Gomez-Vilda, P. (2006) ‘Designing an
independent speaker isolated speech recognition system on an
FPGA’, Research in Microelectronics and Electronics, pp.81–84.
Gracuarena, M., Franco, H., Zheng, J., Vergyri, D. and Stolcke, A.
(2004) ‘Voicing feature integration in SRI’s decipher LVCSR
system’, IEEE International Conference on Acoustics Speech
and Signal Processing, pp.1–4.
Grimm, M., Kroschel, K., Harris, H., Nass, C., Schuller, B., Rigoll,
G. and Moosmayr, T. (2007) ‘On the necessity and feasibility
of detecting a drivers emotional state while driving’, 2nd
International Conference on Affective Computing and
Intelligent Interaction, Lisbon, pp.126–138.
Hauck, S. (1998) ‘The role of FPGAs in reprogrammable systems’,
Proceedings of the IEEE, pp.615–639.
Heungsuk, C., Kim, J., Kim, I., Kwon, Y., Lee, K. and Yang, S.I.
(2001) ‘Realisation of speech recognition using DSP (digital
signal processor)’, IEEE International Symposium on
Industrial Electronics, IEEE, pp.508–512.
Huang, X., Alex. A. and Hsiao-Wuen, H. (2001) Spoken Language
Processing, A guide to Theory, Algorithm and System
Development, Prentice Hall, Upper Saddle River NJ.
Jurafsky, D. and Martin, J.H. (2000) Speech and Language
Processing an Introduction to Natural Language Processing,
Prentice Hall, Upper Saddle River, NJ.
Ke, S., Hou, Y., Huang, Z. and Li, H. (2008) ‘A HMM speech
recognition system based on FPGA’, Congress on Image and
Signal Processing CISP, IEEE, pp.305–309.
 Automatic speech recognition systems
Kim, G. and Loizou, P. (2010) ‘Improving speech intelligibility in
noise using a binary mask that is based on continue spectrum
constraints’, IEEE Signal Processing Letters, pp.1010–1013.
Krishnamurthy, N. and Hansen, J.H.L. (2009) ‘Babble noise:
modeling, analysis and applications’, IEEE Transactions on
Audio, Speech, and Language Proceeding, Vol. 17, No. 7,
September, pp.1394–1407.
Kuhne, M., Togneri, R. and Nordholm, S. (2011) ‘A new evidence
model for missing data speech recognition with applications in
reverberant multi-source environments’, IEEE Transactions on
Audio, Speech and Language Processing, pp.372–384.
Lee, K-F., Hon, H.W. and Ready, R. (1990) ‘An overview of the
SPHINX speech recognition system’, IEEE Transactions on
Acoustics, Speech and Signal Processing, pp.35–45.
Lim, H., You, K. and Sung, W. (2006) ‘Design and implementation
of speech recognition on a softcore based FPGA’, IEEE
International Conference on Acoustics Speech and Signal
Processing, IEEE, pp.1044–1047.
Maithani, S. and Tyagi., R. (2008) ‘Noise characterization and
classification for background estimation’, IEEE – International
Conference on Signal Processing, Communications and
Networking, Chennai, pp.208–213.
Melnikoff, S.J., Quigley, S.F. and Russell, M.J. (2002a)
‘Implementing a simple continuous speech recognition
system on FPGA’, Proceedings of the 10th Annual IEE
Symposium on Field Programmable Custom Computing
Machines, IEEE, pp.275–278.
Melnikoff, S.J., Quigley, S.F. and Russell. M.J. (2002b)
‘Performing speech recognition on multiple parallel files
using continuous hidden Markov models on an FPGA’, IEEE
International Conference on Field-Programmable Technology,
IEEE, pp.399–402.
Meng, Y., Tan, L. and Chinh, P.C. (2005) ‘Speech recognition on
DSP: issues on computational efficiency and performance
analysis’, International Conference on Communications,
Circuits and Systems, IEEE, pp.852–856.
Miura, K., Noguchi, H., Kawaguchi, H. and Yoshimoto, M. (2008)
‘A low memory bandwidth Gaussian mixture model processor
for 20000 word real-time speech recognition FPGA system’,
International Conference on Technology ICECE, pp.341–344.
Morgan, N. and Hermansky, H. (1992) Rasta: Extensions:
Robustness to Additive and Convolutional Noise, ESCS
workshop on Speech Processing in Adverse Conditions.
Mosleh, M., Setayeshi, S., Mehdi Lotfinejad, M. and Mirshekari, A.
(2010) ‘FPGA implementation of a linear systolic array for
speech recognition based on HMM’, International Conference
on Computer and Automation Engineering, pp.75–78.
Muzaffar, F., Mohsin, B., Naz, F. and Jawed, F. (2005) ‘DSP
implementation of voice recognition using dynamic time
warping algorithm’, Student Conference on Engineering
Sciences and Technology, IEEE, pp.1–7.
Neves, C., Veiga, A., Sa, L. and Perdigao, F. (2008) ‘Efficient noise-
robust speech recognition front-end based on ETSI standard’,
ICSP, pp.609–612.
Paliwal, K. and Yao, K. (2010) Robust Speech Recognition
under Noisy Ambient Conditions, Elsevier, Academic Press,
pp.135–162.
Patel, I. and Roa, S. (2010) ‘Speech recognition using hidden Markov
models with MFCC-subband technique’, International
Conference on Recent Trends in Information, Telecommunication
and Computing, IEEE Computer Society, pp.168–172.
Rabiner, L.R. (1989) ‘A tutorial on hidden Markov models and
selected applications in speech recognition’, Proceedings of
the IEEE, pp.257–286.
233
Rabiner, L. and Juang, B-H. (1986) ’An Introduction to hidden
Markov models’, ASSP Magazine, pp.64–16.
Rabiner, L. and Juang, B-H. (1993) Fundamentals of Speech
Recognition, Prentice Hall International, Englewood Cliffs, NJ.
Rabiner, L.R., Levinson, S.E. and Sondhi, M.M. (1983) ‘On the
application of vector quantization and hidden Markov models
to speaker-independent isolated word recognition’, Bell
Systems Technical Journal, pp.1075–1105.
Rabiner, L.R. and Schafer, R.W. (2007) ‘Introduction to digital
speech processing’, Foundations and Trends in Signal
Processing, pp.1–194.
Rabiner, L.R. and Schafer, R.W. (2011) Theory and Applications
of Digital Speech Processing, Pearson Higher Education Inc.,
Upper Saddle River, NJ.
Raj, B. and Stern, R.M. (2005) ‘Missing feature approaches in
speech recognition’, IEEE Signal Processing Magazine,
September, pp.101–116.
Rapp, B. (2008) ‘N-gram language models for Polish language:
basic concepts and applications in automatic speech
recognition systems’, International Multiconference on
Computer Science and Information Technology, pp.321–324.
Rodríguez-Andina, J.J., Fagundes, R.D.R. and Junior, D.B. (2001)
‘A FPGA-based Viterbi algorithm implementation for speech
recognition systems’, IEEE International Conference on
Acoustics, Speech and Signal Processing, Vol. 2, 7–11 May,
pp.1217–1220.
Sacerdote, G. (1959) ‘Statistical measurement of factory noise’,
Noise Control, Vol. 5, No. 6, pp.29–31.
Schuller, B., Wollmer, M., Moosmayr, T. and Rigoll, G. (2009)
‘Recognition of noisy speech: a comparative survey of robust
model architecture and feature enhancement’, EURASIP
Journal on Audio, Speech and Music Processing, February,
pp.1–17.
Schuster, J., Gupta, K., Hoare, R. and Jones, A.K. (2006) ‘Speech
Silicon: an FPGA architecture for real-time hidden Markov-
model-based speech recognition’, EURASIP Journal on
Embedded Systems, pp.1–19.
Sharma, D.P. (2009) ‘On the development of sustainable VLSI
infrastructure for Caribbean countries’, Journal of the
Association of Professional Engineers of Trinidad and Tobago,
Vol. 38, No. 1, October, pp.16–23.
Sharma, D.P. and Atkins, J.M. (2010) ‘FPGA-based embedded
solution for automatic speech recognition’, The 2nd Industrial
Engineering and Management Conference on Fostering
Engineering Networking, Collaboration and Competence,
University of the West Indies, St. Augustine, pp.146–152.
Srinivasan, S. and Wang, D. (2007) ‘Transforming binary
uncertainties for robust speech recognition’, IEEE Transactions
on Audio, Speech and Language Processing, Vol. 15, No. 7,
pp.2130–2140.
Staworko, M. and Rawski, M. (2010) ‘FPGA implementation of
feature extraction algorithm for speaker verification’, 17th
International Conference ‘Mixed Design of Integrated Circuits
and Systems’, pp.557–561.
Stitt, G. (2011) ‘Are field-programmable gate arrays ready for the
mainstream?’, Computing Now, December, pp.58–63.
Van Compernolle, D., Claes, T., Xie, F. and Smolders, J. (2007)
Measuring the Signal-to-Noise Ratio for Noisy Data, Internal
Report, K U Leuven, ESAT.
Van Segbroeck, M. and Hamme, H. (2011) ‘Advances in missing
feature techniques for robust large vocabulary continuous
speech recognition’, IEEE Transactions on Audio, Speech and
Language Processing, pp.123–137.
234 D.P. Sharma and J. Atkins
Veitch, R., Aubert, L-M., Woods, R. and Fischaber, S. (2010)
‘Acceleration of HMM-based speech recognition system by
parallel FPGA Gaussian calculation’, IEEE VI Southern
Programmable Logic Conference (SPL), pp.197–200.
Vu, Ngoc-Vinh, Whittington, J., Ye, H. and Devlin, J. (2010)
‘Implementation of the MFCC front-end for low-cost speech
recognition systems’, Circuits and Systems, pp.2334–2337.
Wang, X. and Li., L. (2009) ‘An N-gram based Chinese syllable
evaluation approach for speech recognition error detection’,
International Conference on Natural Language Processing
and Knowledge Engineering (NLP-KE’2009), pp.1–6.
Wessel, F., Schluter, R., Macherey, K. and Ney, H. (2001)
‘Confidence measures for large vocabulary continuous speech
recognition’, IEEE Transaction on Speech and Audio Processing,
Vol. 9, No. 3, pp.288–298.
View publication stats
Xilinx Foundation (2011a) ‘ML506 development board’, Xilinx
Foundation Web Site. Xilinx Foundation. Available online at:
http://www.xilinx.com/images/boards/ml506/ml506_front.jpg
(accessed on 28 May 2011).
Xilinx Foundation (2011b) ‘Support and documentation’, Xilinx
Company Website, Xilinx. 6 February. Available online at:
http://www.xilinx.com/support/documentation/data_sheets/ds
100.pdf (accessed on 28 May 2011).
Xilinx Foundation (2011c) ‘User guides’, Xilinx Foundation Web
Site, Xilinx, May 17. Available online at: http://www.xilinx.
com/support/documentation/user_guides/ug190.pdf (accessed
on 28 May 2011).
Zolnay, A., Ralf, S. and Hermann, N. (2003) ‘Extraction methods
of voicing feature for robust speech recognition’, Proceedings
of Eurospeech, pp.497–500.