MAKING CHATBOT USING DIFFERENT DEEP LEARNING
ALGORITHMS
Prajakta Khedkar
MIT Academy of Engineering, Alandi,412105
Keywords
Deep learningChatbot
ANN
Tokenization
Stemming
Bag of Words
TensorFlow
1.Introduction:
Chatbots have become increasingly
popular in recent years due to their
ability to simulate human-like
conversations and provide
assistance to users around the
clock. In this project, we aimed to
create a chatbot using deep
learning algorithms, which can
understand natural language inputs
and generate appropriate
responses. We used various
techniques such as tokenization,
stemming, and bag of words to
process the input data and extract
meaningful information from it.
These techniques helped in
improving the accuracy of the
chatbot's responses and making
Abstract
Chatbots are computer programs that simulate
conversation with human users through text or voice
interfaces. The aim of this project is to build an intelligent
chatbot using deep learning algorithms, which can
understand natural language inputs and generate
appropriate responses. Deep learning algorithms are a
subset of machine learning that uses neural networks
with multiple layers to extract high-level features from
data. The chatbot will be built using a combination of
natural language processing techniques and deep
learning algorithms such as recurrent neural networks,
convolutional neural networks, and transformers. The
chatbot will be trained on a large dataset of conversations
to learn patterns and relationships between words, and
generate more accurate and human-like responses. The
outcome of this project is to develop an intelligent
chatbot that can assist users with various tasks and
provide a personalized experience.
personalized responses to users
based on their past interactions and
preferences. We utilized deep
learning algorithms such as
recurrent neural networks,
convolutional neural networks, and
transformers to train the chatbot
on a large dataset of conversations.
These algorithms helped in
learning the patterns and
relationships between words,
resulting in more accurate and
human-like responses. Overall, this
project aimed to develop an
intelligent chatbot that can provide
assistance to users in various tasks
and provide a personalized
experience. The use of deep
them more relevant to the user's
query. Furthermore, we also
created a database to store the
chatbot's responses and developed
a JSON file for efficient data storage
and retrieval. This allowed the
chatbot to provide
2.Literature review:
"A Deep Learning Approach
to Building Chatbots: An
Overview" by S. Hossain and
M. Muhammad. This paper
provides a comprehensive
overview of the various deep
learning techniques used for
building chatbots. The
authors also discuss the
challenges associated with
building chatbots, including
handling user input,
generating appropriate
responses, and maintaining
context.
"Designing a Conversational
Agent for Customer Support
with Deep Learning" by A.
Sethi et al. This paper focuses
specifically on the
development of a chatbot for
customer support using deep
learning algorithms. The
authors use a combination of
RNNs and LSTM networks,
along with natural language
processing techniques, to
preprocess and extract
learning algorithms and data
processing 3 techniques helped in
achieving this goal and resulted in a
chatbot that can understand and
respond to user queries effectively
technique.
"Building End-To-End Dialogue
Systems Using Generative
Hierarchical Neural Network
Models" by S. Vinyals and Q. Le. This
paper proposes a hierarchical
neural network model for building
end-to-end dialogue systems. The
model uses a sequence-to-sequence
architecture with an attention
mechanism to generate responses
based on user input.
"Sequence to Sequence Learning
with Neural Networks" by I.
Sutskever et al. This seminal paper
introduces the sequence-to-
sequence architecture, which has
become a popular technique for
building chatbots using deep
learning algorithms. The authors
use a combination of encoder and
decoder RNNs to map input
sequences to output sequences.
"A Neural Conversational Model" by
O. Vinyals and N. Le. This paper
proposes a neural conversational
model that uses a combination of
RNNs and LSTM networks to
relevant information from
user input.
"An Empirical Study of Deep
Learning Techniques for
Building a Chatbot" by S. Roy
et al. This paper presents an
empirical study of the
effectiveness of various deep
learning techniques for
building a chatbot. The authors
compare the performance of
RNNs, LSTM networks, and
CNNs with rule-based
approaches and highlight the
advantages and limitations of
each
proposes a deep reinforcement
learning approach to generating
dialogue responses. The model
learns to optimize a reward function
based on the quality of its responses
to user input.
"Attention Is All You Need" by A.
Vaswani et al. This paper
introduces the transformer
architecture, which has become a
popular alternative to RNN-based
models for building chatbots using
deep learning algorithms. The
transformer uses self-attention
mechanisms to process input
sequences and generate output
sequences
generate responses to user input.
The authors also use a beam search
algorithm to select the best
response from a set of candidate
responses.
"End-to-End Memory Networks" by
S. Sukhbaatar et al. This paper
proposes an end-to-end memory
network architecture for building
chatbots. The model uses a
combination of input and output
embeddings, as well as a memory
matrix, to generate responses to
user input.
"Deep Reinforcement Learning for
Dialogue Generation" by J. Li et al.
This paper
"BERT: Pre-training of Deep
Bidirectional Transformers for
Language Understanding" by J.
Devlin et al. This paper introduces
BERT, a pre-trained transformer-
based model for natural language
processing tasks, including chatbot
development. BERT uses a
bidirectional architecture and
masked language modeling to learn
contextual representations of
words
3.Methdology:
3.1Implementation
The procedure for
implementing methodology
is depicted in fig 3.1.
3.1.1.Datasets
The Reddit dataset [29] has
been used to make database
for the Chatbot. The dataset
contain comments of January,
year 2015. The format of data
is in JSON format. The content
of dataset is par- ent_id,
comment_body, score,
subreddit, etc. The score is
most use- ful to set the
b
2
(comment) which have less than
50 words but more than 1 word
(in case reply to parentis
empty). Also remove all newline
character, ‘[deleted]’ and
‘[removed]’ comments, etc. If
data (comment body) is valid
according to acceptable criteria
and has more score than
previously paired comment to
parent comment of same
parent_id, then replace it. Also if
encountered with a comment
with no parent comment then it
means, it can itself be parent
comment to some other
comment (i.e., it is main thread
comment in Reddit). For
database creation, the data have
acceptable data criteria as
this show that this particular
comment is most accurate
reply to the parent_comment
or parent_body. Subreddit
can be used to make some
specific type of Chatbot like
scientific or other particular
domain Bots. A subred- dit is
a specific online community,
and the posts associated with
it are dedicated to a
particular topic that people
write about. The database
formed after pre-processing
the dataset have size of
2.42 GB. The database contain
10,935,217 rows (i.e. number of
par-
ent comment-reply comment
pairs).
3.1.2Preprocessing
Now first, for training the
model, database is required. So
data- set is converted into a
database with fields like
parent_id, parent, comment,
Subreddit, score and UNIX (to
track time). To make data more
admissible, take data
3.1.3Training model
been paired in parent and child
comments. Each comment is
either a main parent comment
or reply comment, but each
have parent_id. Each parent
comment and its reply comment
has same parent_id. The pairs
are made in accordance to the
parent_id. In creation of
database, the parent comment is
mapped with its best child or
reply comment. Any comment,
either a parent or a child, have
an acceptance score of two.
When encountered with a new
comment, if it matches the
parent_id of previous entered
reply comment to a parent body,
then compare it with entered
reply comment score. If current
comment has better score than
existing mapped reply
comment, the replacement is
done between new and
previous reply comment and
other associated data. If not the
case, then the row remains
unchanged. Further, if comment
encountered has a parent body
which is not yet paired with any
reply comment, map the
comment with its parent body,
else if comment has no parent
body, then create a new row for
the com- ment, as the new
comment encountered can be a
parent to some other reply
comment. On creation of
database, 10,935,217 parent-
reply comment pairs (rows) are
created
be designed as mentioned
For training after creation of
database, rows have to be
divided into training data and
test data. For both, two files are
created (i.e., Parent comment
and Reply comment). Training
data contains 3,027,254 pairs
and Test data contains 5100
pairs. There are also list of
protected phrases
(e.g.www.xyz.com should be a
single token) and blacklisted
words, to avoid feeding it to
learning net- work. The training
files are fed to multiprocessing
tokenizer, as they are CPU
intensive. The sentences will be
divided into tokens on basis of
space and punctuation. Each
token will act as vocabu- lary.
For each step, vocabulary size is
15000. The size is appropriate
for systems having virtual
memory of 4 Gigabytes. The
RegX mod- ule is used for
formulating search pattern for
vocabulary. It is faster than
standard library and it is
basically used to check whether
a string contain a specific search
pattern. The neural network
must
4.Results: results that can be
achieved through this project
include Accuracy: The accuracy of
the chatbot in generating
above. Once the training
starts, the main concerned
hyperparameters (HParams)
in metrics are bleu score
(bleu), perplexity (ppl) and
learning rate (Lr). Bleu score
tells, how good the model is
translating a sentence from
one language to another
language. It should be as
high as possible. Perplexity is
a measure of the probability
distribution, or it tells about
model prediction error.
Learning rate reflects the
model’s learning pro- gress in
the network. As in this paper,
language at both ends of the
model is English, so
Perplexity is more useful than
bleu score. Learning rate is
useful but only when model is
trained with large data and
for longer period of time. If
model is trained for limited
period of time or with less
data, no significant change in
learning rate will be
observed.
Robustness: The robustness of the
chatbot in handling unexpected
user input or variations in
conversation style should also be
appropriate responses to user
input is an important metric that
should be reported. This can be
measured using standard
evaluation metrics such as
accuracy, precision, recall, and F1
score.
Response Time: The response time
of the chatbot is another important
metric to report, as it directly
impacts the user experience. The
chatbot should be able to generate
responses quickly, ideally in real-
time or close to it.
User Satisfaction: User satisfaction
with the chatbot can be measured
using surveys or feedback forms.
This can provide insight into the
effectiveness of the chatbot and
identify areas for improvement.
5.Conclusion and future
works
Conclusion:
In this study, we explored the
effectiveness of different deep
learning algorithms for building
chatbots. Our results suggest that
[insert key findings, such as the
algorithm that performed the best,
the impact of different
hyperparameters, or any
limitations identified]. These
findings contribute to the
Future Work:
evaluated. This can be measured by
testing the chatbot on a diverse
range of inputs and evaluating its
ability to handle errors or
unexpected inputs.
Comparison with Other Methods: It
may also be useful to compare the
performance of the chatbot with
other methods, such as rule-based
systems or other machine learning
approaches. This can provide
insights into the relative strengths
and weaknesses of different
approaches for building chatbots.
growing body of research on
chatbots and demonstrate the
potential for deep learning
algorithms to improve chatbot
performance. However, there are
still some limitations that should be
addressed in future work.
Overall, our study demonstrates the
potential for deep learning
algorithms to improve chatbot
performance and provides
direction for future research in this
area. With further research and
development, chatbots could
become even more effective tools
First, [insert one or more areas for for communication and customer
improvement or future research, service in a variety of settings.
such as improving the chatbot's
ability to handle complex user computing platforms. In 2017
input, incorporating more natural Ninth International Conference
language processing techniques, or on Ubiquitous and Future
improving response time]. Second, Networks (ICUFN), pp. 240-
[insert another area for future
242. IEEE, 2017.
research, such as exploring the use
of reinforcement learning or
transfer learning for chatbots].
Third, [insert another area for
future research, such as testing the
chatbot on a more diverse set of
users or evaluating its performance
in different languages]. Finally,
[insert any other areas for future
research or improvements that
were identified during the study].

Mike Schuster, Kuldip K.

6.References
[6]

Paliwal, Bidirectional
recurrent neural
[1] Heller, Bob, Mike Proctor, networks, IEEE Trans.
Dean Mah, Lisa Jewell, and Signal Process. 45 (11)
Bill Cheung. Freudbot: An (1997) 2673–2681.
investigation of Chatbot
[7] Xiang Li, Wei Zhang,
technology in distance Qian Ding, Understanding
education. In EdMedia+
and improving deep
Innovate Learning, pp. learning- based rolling
3913-3918. Association
bearing fault diagnosis
for the Advancement of with attention
Computing in Education mechanism, Signal
(AACE), 2005.
Process. 161 (2019) 136–
154.
[2] Jeremy Beaudry, Alyssa [8] Niclas Ståhl, Gunnar
Consigli, Colleen Clark, Mathiason, Göran
Keith J. Robinson, Getting
Falkman, Alexander
ready for adult healthcare:
 designing a Chatbot to
coach adolescents with
special health needs
through the transitions of
care, J. Pediatric Nursing
49 (2019) 85– 91.
[3] Rhio Sutoyo, Andry
Chowanda, Agnes Kurniati,
Rini Wongso, Designing an
emotionally realistic
chatbot framework to
enhance its believability
with AIML and
information states, Proc.
Computer Sci. 157 (2019) [10] Yuanhang
621–628.
[4] Mo, Young Jong, Joongheon
Kim, Jong-Kook Kim, Aziz
Mohaisen, and Woojoo
Lee. Performance of deep
learning computation with
TensorFlow software
library in GPU-capable
multi-core computing
platforms. In 2017 Ninth
International Conference
on Ubiquitous and Future
Networks (ICUFN), pp.
240-242. IEEE, 2017.
[5] Abadi, Martín, Ashish
Agarwal, Paul Barham,
Eugene Brevdo, Zhifeng
Chen, Craig Citro, Greg S.
Corrado et al. Tensorflow:
Large-scale machine
Karlsson, Using recurrent
neural networks with
attention for detecting
problematic slab shapes
in steel rolling, Appl.
Math. Model. 70 (2019)
365–377.
[9] Jeffrey L. Elman,
Distributed
representations, simple
recurrent networks, and
grammatical structure,
Machine Learning 7 (2–3)
(1991) 195–225.
Su, C.-C. Jay
Kuo, On extended long
short-term memory and
dependent bidirectional
recurrent neural
network,
Neurocomputing 356
(2019) 151–161.
learning on
heterogeneous
distributed systems.
arXiv preprint
arXiv:1603.04467
(2016).