Voice Based Virtual Assistant
(Dr. Subrata Sahana,Assistant Professor in Department of Computer Science Engineering
School of Technology, Sharda University)
Bhawana Sati Karan Rana
Department of Computer Science Department of Computer Science
Engineering Engineering
Sharda University Sharda University
Greater Noida, India Greater Noida, India
Email Id:- Email Id:-
2018012988.bhawana@ug.sharda.ac.in 2018008328.karan@ug.sharda.ac.in
Sameer Kumar
Department of Computer Science
Engineering
Sharda University
Greater Noida, India
Email Id:-
2018011552.sameer@ug.sharda.ac.in
Abstract— This paper is based on voice based virtual
assistant. It is a tool in AI that allows us to fulfill different
purposes just by giving voice commands. The voice assistant we
have developed is a desktop-based built using python modules
and libraries. This assistant is just a basic version that could
perform all the basic tasks which have been mentioned above
but current technology is although good in it is still to be
merged with Machine learning for better classifications.
Hence in this process, we developed a system which recognizes
the voice from the dataset and then the corresponding response
will be done. The process works on three stages like,
preprocessing, feature extraction and classification. Voice
assistants are the great innovation in the field of AI that can
change the way of living of the people in a different manner.
The voice assistant was first introduced on smart phones and
after the popularity it got. It was widely accepted by all.
Initially, the voice assistant was mostly being used in smart
phones and laptops but now it is also coming as home
automation and smart speakers. Many devices are becoming
smarter in their own way to interact with human in an easy
language. The Desktop based voice assistants are the programs
that can recognize human voices and can respond via
integrated voice system. This paper will define the working of a
voice assistant, their main problems and limitations. In this
paper it is described that the method of creating a voice
assistant without using cloud services, which will allow the
expansion of such devices in the future.
Keywords— voice assistant, natural language processing,
python, speech recognition.
Acknowledgement
XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE
Kuhil Saikia
Department of Computer Science
Engineering
Sharda University
Greater Noida, India
Email Id:-
2018003132.kuhil@ug.aharda.ac.in
We would express our sincere gratitude towards our
project guide DR. Subrata Sahana for his extensive support
and expertise on the project, research , development and
execution.
Introduction
This field of virtual assistants having speech
recognition has seen some major advancements or
innovations. This is mainly because of its demand in
devices like smart watches or fitness bands,
speakers, Bluetooth earphones, mobile phones,
laptop or desktop, television, etc. Almost all the
digital devices which are coming nowadays are
coming with voice assistants which help to control
the device with speech recognition only. A new set
of techniques is being developed constantly to
improve the performance of voice automated
search. As the amount of data is increasing
exponentially now known as Big Data the best way
to improve the results of virtual assistants is to
incorporate our assistants with machine learning and
train our devices according to their uses. Other
major techniques that are equally important are
Artificial Intelligence, Internet Of Things, Big Data
access and management, etc. The development of
artificial intelligence is growing day by day that are
able to recognize human voice as a command,
analyse it and provide services to humans. The
voice assistant is gaining lots of popularity today. It
is the era where no longer a human learns how to
communicate with machines, but a machine learns
to communicate with a human, exploring his
actions, asking about his hobbies, habits and trying
become a human's personal assistant. A voice
assistant is a software program that can perform
tasks or provide some kind services for an
individual based on the human verbal commands
i.e. by using human voice commands and the voice
 assistant will respond via synthesized voice. The
Users can ask their assistants’ questions, control
home automation devices, and media playback via
voice, and manage other basic tasks such as email,
to-do lists, open or close any application.
The first virtual assistant was not developed by apple that
was introduced as Siri in the year 2012. The first virtual
assistant was actually introduced in the year 1963 at the
Seattle World’s fair by IBM which was named Shoe Box.
It can easily recognize 1011 words which is equal to the
vocabulary knowledge of a 3 year old student. It can do some
mathematical operations, recognize numbers from 0 to 9 and
was able to recognize 16 spoken words. The Virtual
Assistant Methodology got popular along with the
development of AI and ML. An average person can speak
150 words in a minute and can only type 40 in a minute.
These are the reasons why the IT companies developed
their own personal virtual assistants to make their work flow
easy and smooth.
Literature Survey
The proposed system was based on voice recognition in
commands that convert speech into text. The user gets easier
access to speech than typing on the keyboard. The user spells
commands in the voice so that the user has good quality
microphone. The user must spell the word with the correct
pronunciation that should be understood in the voice
recognition system. But there is some confusion in the
system. For example, the words sun and son have the same
pronunciation so the system finds it difficult to choose which
name. To provide a powerful microphone to be heard in the
system. Voice is a simple and effective way to communicate.
Many people prefer to use speech rather than text. A basic
overview of the proposed system that receives voice input
signals in the form of voice. It moves to the release of the
feature and enters the scanner. The decoder contains two
main models, the acoustic model and the language model.
The decoder will record the input. After all the processing
that goes into the decoder provides the output. The categories
of speech recognition systems are the types of utterances, the
types of platform models, the types of words.
Speech to Text Command Execution
 Speech to Text:- The user will give the input as
a speech and will ask it to run the command.
 Command Execution:- Based on the command
given by the user the system will perform the
necessary execution like e.g.- opening paint or
notepad or Google chrome , etc .
 Text to Speech:- This is the most interesting
part where the machine process the input given
by the user and speaks the command. This
makes the user experience more interactive with
the system.
The accuracy, speed, and contextual abilities of Alexa,
Google Assistant, and Siri are all thanks to Machine
Learning algorithms and servers owned by their developing
companies. all of them add an identical manner, the sole
difference arises in their protocols and data privacy
intricacies. When a user makes missive of invitation, the
request is instantly packaged up and is shipped to the server
of their respective companies for a response i.e. why internet
connectivity is one amongst the fundamental requirements
for Virtual Assistants to function properly. After the
package is distributed to the server the words and tone of
your request are analyzed by a collection of algorithms,
which are then matched with a command that it thinks, you
asked.
The complexity is in respect to the speed of task fulfilment
and understanding of what the user wants within the first try.
Once it knows what it has to do, that’s a basic process of
tapping into a server, third-party computer, or the other
device. Here are the 2 most successful Voice Assistants:
i) Alexa:-
Amazon has designed the Alexa Voice Service
(AVS), Amazon’s cloud-based voice service, to mimic real
conversations, using intuitive voice commands to induce this
service to perform specific tasks. Alexa acts as a “wake up”
word for the Alexa supported devices that alert the service to
start out paying attention to the user commands. Its AVS has
intelligent voice recognition and language understanding
services. it's ready to play music, provide information,
deliver news and sports scores, weather reports, controls
smart home services, and even allow Prime members to
shop for order products from Amazon.
ii) Siri:-
Siri is Apple’s virtual/digital assistant for iOS,
macOS, tvOS, and watchOS devices that use voice
recognition. it's supported the fields of AI, Machine
Learning, and language Processing. And it's supported three
basic components: a conversational interface, personal
context awareness, and repair delegation. It enables the user
to work the mobile device and its apps using natural voice
commands. Siri can facilitate your when the user is out and
about, with sports and entertainment information, phone
calls and messages, getting organized, tips and tricks, read
your last email, text your friends and family, shuffle playlist,
even finding a table for 3 in an exceedingly restaurant and
far more stuff like this.
ML Models explain - 
These are the ML models which are used in it:-
i) KNN  Classifier:-
K-Nearest Neighbors (KNN) is one in every of the
only algorithms employed in Machine Learning for
regression and classification problems. KNN algorithms use
data and classify new data points supported similarity
measures (e.g. distance function). Classification is
completed by a majority vote to its neighbors. As you
increase the quantity of nearest neighbors, the worth of k,
accuracy might increase. It is used for both classification
and regression problems. However, it's more widely
employed in classification problems within the industry. K
nearest neighbors could be a simple algorithm that stores all
available cases and classifies new cases by a majority vote
of its k neighbors. The case being assigned to the category is
commonest amongst its K nearest neighbors measured by a
distance function.
These distance functions will be Euclidean, Manhattan,
Minkowski, and Hamming distances. the primary three
functions are used for continuous function and therefore the
fourth one (Hamming) for categorical variables. If K = 1,
then the case is just assigned to the category of its nearest
neighbor. At times, choosing K seems to be a challenge
while performing KNN modeling. KNN classifier has
disadvantages which are KNN is computationally expensive,
to prevent higher range variables to bias it variable should
be normalized, before Works on pre-processing stage more
should go for kNN like an outlier, noise removal.
ii) CNN Architecture:-
Convolutional neural networks (or ConvNets) are
biologically-inspired variants of MLPs, they need different
types of layers and every different layer works different than
the same old MLP layers. If you're curious about learning
more about ConvNets, a decent course is that the CS231n –
Convolutional Neural Newtorks for Visual Recognition. In
deep learning, a convolutional neural network (CNN, or
ConvNet) could be a class of deep neural networks, most
typically applied to analyzing visual imagery. CNNs are
regularized versions of multilayer perceptrons. Multilayer
perceptrons usually seek advice from fully connected
networks, that is, each neuron in one layer is connected to
any or all neurons within the next layer. The "fully-
connectedness" of those networks makes them susceptible to
overfitting data. Typical ways of regularization include
adding some kind of magnitude measurement of weights to
the loss function. However, CNNs take a unique approach
towards regularization: they profit of the hierarchical pattern
in data and assemble more complex patterns using smaller
and simpler patterns. Therefore, on the dimensions of
connectedness and complexity, CNNs are on the lower
extreme.
iii) SVM (Support Vector Machine):-
It is a classification method. during this algorithm,
we plot each data item as a degree in n-dimensional space
(where n may be a number of features you have) with the
worth of every feature being the worth of a selected
coordinate. For example, if we only had two features like
Height and Hair length of a personal, we’d first plot these
two variables in two-dimensional space where each point
has two coordinates (these co-ordinates are referred to as
Support Vectors).
Interaction of AI voice assistants with people
As technology advances the interaction of people with
devices changes drastically. The internet searches has
become easier after the development of virtual assistants.
Now, the search engines like Google, bing, etc recognize
what you are trying to search in the internet and gives you
the appropriate results. The search engines easily understand
the context and content of the search.
The virtual assistants has made some major growth in the
world of technology. In early days the only way possible to
search in a search engine was through the text but now the
voice has taken over the charge.
The virtual assistants are always listening for the users to
receive a command. The words like hey siri or OK Google
are like the default wakeup call for them. The virtual
assistants are connected to a server where the responses are
stored in the database. So, they need a constant source of
internet to work efficiently. When you give the wakeup call it
immediately connects to the server. The wakeup call is set as
an algorithm for the virtual assistant to connect to the
internet.
They don’t really understand what you are saying they
just hears the command and immediately communicate with
the server and look for possible answers to reply as soon as
possible. NLP is the technology behind it that helps the
virtual assistants to hear the command and process the voice
to text to be understandable to the system to search for
relevant answers in the server database.
They allows us to do various tasks so, that is the primary
reason that people likes using them so much in their day to
day life , especially on their phones.
 In the upper, training data are converted into CNN
descriptors and are incorporated into a voice dataset.
The voice templates in the dataset are indexed by a
KNN structure to speed up the voice matching
procedure.

Future Scope
In future, the proposed system can be used in
making a software application which can be
used in various platforms like educational
institutes, healthcare services and many others.
We will encourage the readers to work more on
Natural Language Processing projects for the
good and development of the society.

The advantages of Virtual Assistants

 It converts the texts to speech.
 It will help you to find the application easily
 It helps to set reminders and makes your daily
life really easy.
Limitations of Virtual Assistants
 The data that is entered should be accurate
otherwise it will not give accurate answers.
 The deaf and dumb users will not be able to
access it.
 The microphone that is used should be of higher
quality for accurate voice input. For ex:- Sun
can be misspelled as son.
Applications of Virtual Assistants
 This can be used by various people to get
counselling session online.
 It makes the operations hands free.
 About 50% of people those who were surveyed
said that it is a better interaction medium and
finds it helpful and easier than other means of
communication used.
Conclusion
The voice based virtual assistant system in this
paper is very fundamental with few features
however the additional and advance feature may
be introduced as future work of this project. In
this paper the design and implementation is
described . With the development of the project
we were able to learn and gain more knowledge
on NLP and AI and ML Models which are the
foundations for future Artificial Intelligence
Models.
References
 Research paper by Ankit Pandey and
Vaibhav Vashist on Smart Voice Based
Virtual Personal Assistant.
 International Journal of Advanced
Science and Technology. Vol. 29, No.
7s(2020),pp. 1651-1654
 Google.co.in

Proposed System
The system proposes a new framework to solve the
aforementioned problems.
This processes based on with substantial extensions.