An overview of NLP based Chatbot
Jayshree Ghorpade-Aher Riddhi Kontamwar
Dept. of Computer Engineering
Dept. of Computer Engineering
MITCOE
MITCOE
Pune, India
Pune, India
Email:ridkont@gmail.com
Email: jayshree.aj@gmail.com
Tejal Karpe Saloni Kakkad
Dept. of Computer Engineering
Dept. of Computer Engineering
MITCOE MITCOE
Pune, India
Pune, India
Email:salonikakkad12345@gmail.com
Email:tejal1633@gmail.com
Abstract— A chatbot is a software that simulates a
conversation with a human in a generalized form through
various applications like messaging, websites, mobile
applications, etc. Traditionally, a search engine was used to
answer any type of questions, but with the advancement in
technology the use of chatbot has increased and it allows the
user to simply ask any question in a same manner they would
communicate with humans. The interesting features of this is that
they are self-learning and become intelligent over time using
technologies like Machine Learning and Natural Language
Processing(NLP). This paper provides a brief overview of the use
of NLP in chatbot as well as the working of different algorithms.
It studies various machine learning algorithms used for
developing chatbots.
Keywords— Chatbot, NLP (Natural Language Processing),
Supervised Machine Learning, Entity, Classification.
I. INTRODUCTION
With advancement in World Wide Web, the human-
machine interaction plays vital role in the digital world. In
order to expand and elevate the simplicity of user
communication with any system, interaction between human
and machine is essential.
“Our Differentiation starts with the third dimension of
Chatbots. We call it the Human Dialogue Theory.”
– Riza Berkan.
A chatbot is a software which has been designed to
imitate an intelligent conversation with end user. In the past,
there has been a significant hike in interest in chatbots. Few of
the current scenarios are as follows:
• DOM, The Pizza Bot, Dominos: It helps user to
order dominos pizza of their choice via facebook
messenger account, and DOM allows the user to
customize their pizza hassle free.
• Allo, Google: It helps in various tasks like finding
scores, dinner reservations etc.
• Tay, Microsoft: It is developed by Microsoft on
Twitter. It was capable of interacting with twitter
users.
• KLM Royal Dutch Airlines: This bot helps the users
to get the current updates of their flights, check-in
notifications of Dutch Airlines.
Simran Kukreja
Dept. of Computer Engineering
MITCOE
Pune, India
Email:simrankukreja2911@gmail.com
As chatbots have gained popularity over the last few
years, most of them are rule-based in which output of the
linear conversations is predetermined. For most cases, this
works fine. However, chatbots can do much more than this.
With the introduction of NLP, developers are now able to put
the chat in chatbots .[1]
NLP is a technical process that allows machines to extract
meaning from user text inputs. It aims to understand the intent
of the input instead of just obtaining information about the
intent itself. With NLP, it is possible to train the chatbot on
various conversations it will go through and help to give the
responses. The training part consists of providing the content
examples it might come across. Providing more training set to
the chatbot gives it a wider knowledge base with which it can
interpret and answer the questions.[3]
In this paper, working of chatbot is described thoroughly.
The section II gives the use of NLP in chatbot and section III
explains the steps in NLP. The section IV explains various
supervised algorithms. Lastly, the conclusion is given in
section V.
II. NLP IN CHATBOT
A chatbot is a service or a tool that the user can communicate
with via text messages. The chatbot understands what the user
is trying to say and replies with a coherent, relevant message
or directly completes the desired task by using NLP.
There are two components of NLP:
1. Natural Language Understanding
2. Natural Language Generation
1. Natural Language Understanding:
In NLU, the input gets transformed into the useful
representations in order to analyze various aspects of the
language. NLU has two specific concepts:
Entities: Entities are basically nouns to represent an idea
to chatbot.
Context: When a NLU algorithm considers a sentence, it
doesn’t have the historical data of the user’s conversation.
This implies that, if a same question is asked again and
chatbot has got the response, it won’t recall the previous
inquiry. So, the phases during the conversation of chat are
separately stored. It can either be banners like "Asking about
admission process".
A chatbot must fulfill customer's needs which can be the
same for different queries. For example, "I want to know
about the college reading hall" and "Do you have a reading
hall? I want to know about it", it is the same: details of reading
hall. Hence, all user typing text show a single command which
is identifying the tag: reading hall.
2. Natural Language Generation:
In order to generate the output text, the intermediate
representation requires to be converted back to natural
language format. Hence, it is the process of producing
meaningful phrases and sentences in the form of natural
language from some internal representation.
It involves:
• Text Planning: It includes extracting the relevant
content from knowledge base.
• Sentence Planning: It includes selecting required
words, forming language grammar, setting tone of the
sentence.
• Text Realization: It is the process of mapping the
planned sentence into sentence structure.
III. STEPS OF NLP
Chatbots uses the NLP to convert the user’s text into
structured data which is then used for choosing the relevant
answer [8]. NLP includes the following steps:
Step 1: Lexical Analysis- It identifies and analyzes the
structure of words. The collection of words and phrases in a
language is called a Lexicon. Lexical analysis divides the
whole text into paragraphs, sentences, and words.
i. Sentence Segmentation: The first step is to split
the text into separate sentences.
ii. Word Tokenization: This step breaks a stream
of text into words, phrases or other meaningful
elements called tokens. It separates the words in
order to extract knowledge.
Example: "Education is the process of facilitating learning. A
right to education has been recognized by all Governments."
Sentence Segmentation:
Sentence 1: Education is the process of facilitating learning.
Sentence 2: A right to education has been recognized by all
Governments.
Word Tokenization: Education, is, the, process, of, facilitating,
learning.
Step 2: Syntactic Analysis (Parsing)- It involves analysis of
words in the sentence to check grammar and arrangement of
words in a manner that shows the relationship among the
words. The sentence such as “The food like the girl" is
rejected by English syntactic analyzer.
i. POS Tagging: The part of speech for each word
is determined for each sentence.
ii. Text Lemmatization: It is a linguistic process
that attempts to determine the root of each word.
The lemma of word is its basic form along with
its inflected forms.
Example: "A Computer is a programmable machine."
POS Tagging: A|Article, Computer|Noun, is|Verb,
programmable|Adjective, machine|Noun.
Lemma(root): Compute, Program
Step 3: Semantic Analysis- It draws the exact meaning or the
dictionary meaning from the text. The text is checked for
meaningfulness. It is checked by portraying syntactic
structures and objects in the task domain. The semantic
analyzer ignores sentence described as “The ice-cream is
hot”.
i. Identifying Stopwords: It is a list of empty
words in a terms list (prepositions, determiners,
pronouns etc.) considered to have little semantic
value, and are eliminated when found in
document.
Example: "What is the capital of India?"
Stopwords: is, the, of
Step 4: Discourse Integration- The meaning of any sentence
depends upon the meaning of the sentence just before it. In
addition, it also finds the meaning of its successor sentences.
i. Dependency Parsing: It figures out how all the
words in the sentence are related to each other. It
searches for the subjects, verbs, objects,
common phrases and nouns in the user’s text to
discover related phrases that what users want to
convey.
ii. Detecting N-Grams: It consists in identifying
words that are usually together (compound
words) to be able to process them as a single
conceptual unit.
iii. Co-reference Resolution: It is the task of
finding all expressions that refer to the same
entity in a given text.
Example: "I am a student of Computer Department. Who is
the HOD of this department?"
N-Grams: Computer Department
Co-reference Resolution: In this example, “this” refers to
“Computer Department”.
Step 5: Pragmatic Analysis- During this, what was said is re-
interpreted on what it actually meant. It involves deriving
those aspects of language which require real world
knowledge.
 i. Named Entity Recognition: It looks for
different categories of words, similar to the name
of the specific entity, the user’s address,
whatever information is required.
Example: "Apple CEO Tim Cook introduced two new larger
Iphones at Cupertino, Flint Center Event."
NER: Apple, Flint Center: Name of Organization
Tim Cook: Name of Person
Cupertino: Name of Location
IV. ALGORITHMS AND TECHNIQUES
A. Decision Tree:
Recursive Division stops only when the following conditions
are true:
1. All sample data of a given node belongs to the same
class, that is, value of all flag attribute are same.
2. No remaining backup attribute can be used to divide
the sample further.
3. After split, certain branch does not have any sample
record.
In the decision tree, the first question is the root of the tree.
From there, the chatbot might need additional information in
order to solve the problem, moving the conversation from up
to down. The more specific a query is, the more branches will
be needed to solve it, and the more branches you create, the
more intelligent and helpful the chatbot will be.

Decision trees are how chatbots help customers find
exactly what they are looking for: they follow a step-by-step
process to discover the accurate answer to the customer’s
question in a conversational way. The decision tree algorithm:
ID3 (Iterative Dichotomiser 3) → uses Entropy function and
Information Gain as metrics.[11]
The Information Gain can be calculated as:
...(1)
Entropy can be calculated as:
B. Naïve Bayes Algorithm:
It is a probabilistic classification technique which is based
on Bayes' theorem with an assumption of independence
assumptions among the features. Basically, a Naive Bayes
classifier assumes that the presence of a particular feature in a
class is unrelated to the presence of any other feature. For
example, if a human wants to know which engineering fields
are there in that particular college, the bot will first read the
data and further partition it into two parts like engineering and
non-engineering and accordingly split the data and answer the
human [14].
Naive Bayes model is easy to build. It is particularly
useful for very large data sets. This algorithm is capable of
self-categorizing the databases in an efficient way.

...(2)

Steps to build ID3 decision tree:

Step 1: Decision Tree begins with a single node which

represents all records (as roots) of the entire training set.

Step 2: If the sample records are in the same category,

then the nodes become leaf nodes, and are marked with
the classification the sample records belong to.

Step 3: Otherwise, the algorithm will use measure based

on entropy called as information gain, as the heuristic
information, and select the attribute which can achieve the
best classification from sample backup attribute set. This
attribute become to this node’s “test” or “judge” attribute.
In this version of this algorithm, all descriptive attributes
are classified, namely take discrete values, and continuous Figure 1: Naive Bayes workflow
numeric attributes must be discretizing.

Step 4: Create a branch for certain test attribute’s every Bayes theorem provides a way of calculating the
known value, and divided the sample data into every probability P(c|x) from P(c), P(x) and P(x|c). Refer the
branch according this. equation (3) below:[15]

Step 5: The algorithm does the same way through the

above procedure recursively, to form a sample decision ...(3)
tree for each division. Once an attribute has been selected
as the test attribute of a node, you do not have to consider
the likelihood that test this node’s any descendant nodes’
attribute.
 V. PROPOSED STRUCTURE OF CHATBOT
Figure 2: Workflow of chatbot
The above figure 2 shows the workflow of chatbot. It consists
of the following entities:
1. Input Query: It is the user input which is fed to the system
for further processing.
2. NLU Steps: These are the pre-processing steps used for
extracting information from input query.
3. Classification Algorithm: There are different machine
learning algorithms such as Decision Tree or Naive Bayes’ for
text classification in order to extract the most relevant answer
for the given input query.
4. Retrieval from Database: Based on the query , the
response is retrieved from the database and output is given to
user.
5. NLG: If the relevant answer is not present in the database
then the answer is generated.
6.Output: Response is given to the user.
V. CONCLUSION
With the rising trend of chatbot and along with the passage of
time, new features can be added in the existing platform. This
paper discusses the most commonly used supervised machine
learning algorithms for the implementation of chatbot.
Advancements in the machine learning techniques may be
able to handle complex conversation issues. Chatbots can
reach out to large audience through apps, websites and can be
more effective than humans. It may turn into an information
gathering tool in near future.
REFERENCES
[1] A. M. Rahman, A. A. Mamun and A. Islam,
"Programming challenges of chatbot: Current and future
prospective," 2017 IEEE Region 10 Humanitarian Technology
Conference (R10-HTC), Dhaka, 2017,pp.75-78. doi:
10.1109/R10-HTC.2017.8288910
[2] Chen, H, Machine learning for information retrieval:
Neural networks, symbolic learning, and genetic algorithms. J.
Am. Soc. Inf. Sci., 46: 194-216
[3] B. Setiaji and F. W. Wibowo, "Chatbot Using a
Knowledge in Database: Human-to-Machine Conversation
Modeling," 2016, 7th International Conference on Intelligent
Systems, Modeling and Simulation (ISMS), Bangkok, pp. 72-
77.
[4] Y. Yang, X. Huang, X. Hao, Z. Liu and Z. Chen, "An
Industrial Study of Natural Language Processing Based Test
Case Prioritization," 2017 IEEE International Conference on
Software Testing, Verification and Validation (ICST), Tokyo,
pp. 548-549.
[5] Quote, Riza Berkan founder of exClone Inc.,
https://emerj.com/ai-podcast-interviews/when-will-chatbots-
reach-human-level-sophistication/
[6] A. Gelbukh, "Natural language processing," Fifth
International Conference on Hybrid Intelligent Systems
(HIS'05), Rio de Janeiro, Brazil,2005,pp.1pp.-.doi:
10.1109/ICHIS.2005.79
[7] B. R. Ranoliya, N. Raghuwanshi and S. Singh, "Chatbot
for university related FAQs," 2017 International Conference
on Advances in Computing, Communications and Informatics
(ICACCI), Udupi,2017,pp.1525-1530.doi: 10.1109/ICACCI.
2017.8126057
[8] A. Singh, N. Thakur and A. Sharma, "A review of
supervised machine learning algorithms," 2016 3rd
International Conference on Computing for Sustainable
Global Development (INDIACom), New Delhi, pp.1310-1315.
[9] S. Angra and S. Ahuja, "Machine learning and its
applications: A review," 2017 International Conference on
Big Data Analytics and Computational Intelligence
(ICBDAC), Chirala, 2017, pp.57-60.
[10] R.R.Halde, "Application of machine learning algorithms
for betterment in education system," 2016 International
Conference on Automatic Control and Dynamic Optimization
Techniques(ICACDOT), Pune, 2016, pp. 1110-1114.
[11] Linna Li and Xuemin Zhang, "Study of data mining
algorithm based on decision tree," 2010 International
Conference On Computer Design and Applications,
Qinhuangdao, 2010, pp. V1-155-V1-158.
[12] E. Haller and T. Rebedea, "Designing a Chatbot that
simulates a Historical Figure," 2013 19th International
Conference on Control Systems and Computer Science,
Bucharest, 2013, pp. 582-589
[13] J.Song. et al. , TOLA : Topic-oriented learning
assistance based on cyber-physical system and big data.
Future generation computer systems(2016), http
://dx,doi.org.10.1016/j.future.2016.05.040
[14] F. Qin, X. Tang and Z. Cheng, "Application and research
of multi-label Naïve Bayes Classifier," Proceedings of the
10th World Congress on Intelligent Control and Automation,
Beijing, 2012, pp. 764-768.

[15] M. Niranjan, M. S. Saipreethy and T. G. Kumar, "An

intelligent question answering conversational agent using
Naïve Bayesian classifier," 2012 IEEE International
Conference on Technology Enhanced Education (ICTEE),
Kerala, 2012, pp. 1-5.

[16] Jingnian Chen, Houkuan Huang, Shengfeng Tian, Youli

Qu,Feature selection for text classification with Naïve
Bayes,Expert Systems with Applications, 2009, ISSN 0957-
4174.

[17] Y. Huang and L. Li, "Naive Bayes classification

algorithm based on small sample set," 2011 IEEE
International Conference on Cloud Computing and
Intelligence Systems, Beijing, pp. 34-39.

[18] Sang-Bum Kim, Kyoung-Soo Han, Hae-Chang Rim and

Sung Hyon Myaeng, "Some Effective Techniques for Naive
Bayes Text Classification," in IEEE Transactions on
Knowledge and Data Engineering, vol. 18, no. 11, pp. 1457-
1466, Nov. 2006.

[19] G. M. D'silva, S. Thakare, S. More and J. Kuriakose,

"Real world smart chatbot for customer care using a software
as a service (SaaS) architecture," 2017 International
Conference on I-SMAC (IoT in Social, Mobile, Analytics and
Cloud), Palladam, 2017, pp. 658-664.

[20] A. Augello, G. Pilato, A. Machi and S. Gaglio, "An

Approach to Enhance Chatbot Semantic Power and
Maintainability: Experiences within the FRASI Project," 2012
IEEE Sixth International Conference on Semantic Computing,
Palermo, pp. 186-193.

[21] M. C. Surabhi, "Natural language processing future,"

2013 International Conference on Optical Imaging Sensor and
Security (ICOSS), Coimbatore, 2013, pp. 1-3.

[22] H. N. Io and C. B. Lee, "Chatbots and conversational

agents: A bibliometric analysis," 2017 IEEE International
Conference on Industrial Engineering and Engineering
Management (IEEM), Singapore,2017,pp.215-219. doi:
10.1109/IEEM.2017.8289883