Dynamic Embedding Projection-

Gated for text classication using

Convolutional Neural
Networks(CNN)
 PROBLEM STATEMENT
• The effectiveness of DEP-CNN by comparing it with other up-to-date
methods via extensive experiments on four known benchmark datasets.
• In addition, a visualization experiment shows that DEPG can indeed learn
to acquire more meaningful features for a given text.
• According to the test scores of DEP-CNN on different pre-trained word
vectors, we also observe that our method exhibits high robustness.
• Our work includes how to employ a DEPG layer to other neural
networks and add additional information derived from internal or external
sources to improve the performance of the model.
 SCOPE OF THE PROJECT:
• TEXT classification is highly useful in natural language processing (NLP)
and plays a vital role in various applications, for instance, topic
categorization ,intent detection, sentiment analysis ,and spam filtering .
• Unstructured text-based data are one of the forms of information appearing
in social media, e-mail, and web pages. It often takes much time to extract
useful information from such text because of its unstructured nature .
• Doing so has been prohibitively expensive and challenging because it
takes lots of resources and time to create needed hand-crafted rules.
Recently, a good choice for constructing text data is to use NLP instead of
traditional feature engineering, which makes text classification scalable,
cost-effective, and rapid.
 OBJECTIVE
• we propose a dynamic embedding projectiongated convolutional neural
network (DEP-CNN) for multi-class and multi-label text classification.
• Its dynamic embedding projection gate (DEPG) transforms and carries
word information by using gating units and shortcut connections to
control how much context information is incorporated into each specific
position of a word-embedding matrix in a text.
• To our knowledge, we are the first to apply DEPG over a word-
embedding matrix. The experimental results on four known benchmark
datasets display that DEP-CNN outperforms its recent peers
 ABSTRACT
• Text classification is a fundamental and important area of natural
language processing for assigning a text into at least one predefined
tag or category according to its content.
• Most of the advanced systems are either too simple to get high
accuracy or centered on using complex structures to capture the
genuinely required category information, which requires long time to
converge during their training stage.
 LITERATURE SURVEY
1.Text classification algorithms: A survey
AUTHOR:K. Kowsari, K. J. Meimandi, M. Heidarysafa, S. Mendu, L. E. Barnes,
and D. E. Brown
ABSTRACT:
In Natural Language Processing (NLP), most of the text and documents contain
many words that are redundant for text classification, such as stopwords, mis-
spellings, slangs, and etc. In this section, we briefly explain some techniques and
methods for text cleaning and pre-processing text documents. In many algorithms
like statistical and probabilistic learning methods, noise and unnecessary features
can negatively affect the overall performance. So, the elimination of these features
is extremely important.
2.Baselines and Bigrams: Simple, Good Sentiment and Topic
Classification
AUTHOR: S. I. Wang and D. C. Manning
ABSTRACT:
Variants of Naive Bayes (NB) and Support Vector Machines (SVM) are
often used as baseline methods for text classification, but their performance
varies greatly depending on the model variant, features used and task/
dataset. We show that: (i) the inclusion of word bigram features gives
consistent gains on sentiment analysis tasks; (ii) for short snippet sentiment
tasks, NB actually does better than SVMs (while for longer documents the
opposite result holds); (iii) a simple but novel SVM variant using NB log-
count ratios as feature values consistently performs well across tasks and
datasets. Based on these observations, we identify simple NB and SVM
variants which outperform most published results on sentiment analysis
datasets, sometimes providing a new state-of-the-art performance level.
3. Intent detection and slots prompt in a closed-domain chatbot
AUTHOR: A. Nigam, P. Sahare, and K. Pandya
ABSTRACT: In this paper, we introduce a methodology for predicting intent and
slots of a query for a chatbot that answers career-related queries. We take a multi-
staged approach where both the processes (intent-classification and slot-tagging)
inform each other's decision-making in different stages. The model breaks down
the problem into stages, solving one problem at a time and passing on relevant
results of the current stage to the next, thereby reducing search space for
subsequent stages, and eventually making classification and tagging more viable
after each stage. We also observe that relaxing rules for a fuzzy entity-matching in
slot-tagging after each stage (by maintaining a separate Named Entity Tagger per
stage) helps us improve performance, although at a slight cost of false-positives.
Our model has achieved state-of-the-art performance with F1-score of 77.63% for
intent-classification and 82.24% for slot-tagging on our dataset that we would
publicly release along with the paper.
4. Exploring latent semantic information for textual emotion recognition in blog
articles
AUTHOR: X. Kang, F. Ren, and Y. Wu
ABSTRACT: Understanding people’s emotions through natural language is a challenging
task for intelligent systems based on Internet of Things (IoT). The major difficulty is
caused by the lack of basic knowledge in emotion expressions with respect to a variety of
real world contexts. In this paper, we propose a Bayesian inference method to explore the
latent semantic dimensions as contextual information in natural language and to learn the
knowledge of emotion expressions based on these semantic dimensions. Our method
synchronously infers the latent semantic dimensions as topics in words and predicts the
emotion labels in both word-level and document-level texts. The Bayesian inference results
enable us to visualize the connection between words and emotions with respect to different
semantic dimensions. And by further incorporating a corpus-level hierarchy in the
document emotion distribution assumption, we could balance the document emotion
recognition results and achieve even better word and document emotion predictions.
5.Drifted Twitter spam classification using multiscale detection test on
K-L divergence
AUTHOR: X. Wang, Q. Kang, J. An, and M. Zhou
ABSTRACT: Twitter spam classification is a tough challenge for social media
platforms and cyber security companies. Twitter spam with illegal links may evolve
over time in order to deceive filtering models, causing disastrous loss to both users
and the whole network. We define this distributional evolution as a concept drift
scenario. To build an effective model, we adopt K-L divergence to represent spam
distribution and use a Multiscale Drift Detection Test (MDDT) to localize possible
drifts therein. A base classifier is then retrained based on the detection result to gain
performance improvement. Comprehensive experiments show that K-L divergence
has highly consistent change patterns between features when a drift occurs. Also,
MDDT is proved to be effective in improving final classification result in both
accuracy, recall and f-measure.
ARCHIECTURE
MODULES
 ALGORITHM
Convolutional Neural Networks (CNNs)
CNN's, also known as ConvNets, consist of multiple layers and are mainly used for
image processing and object detection. Yann LeCun developed the first CNN in 1988
when it was called LeNet. It was used for recognizing characters like ZIP codes and
digits.
Free Deep Learning for Beginners Course
Master the Basics of Deep Learning
CNN's are widely used to identify satellite images, process medical images, forecast
time series, and detect anomalies.
How Do CNNs Work?
CNN's have multiple layers that process and extract features from data
SOFTWARE REQUIREMENT
 CONCLUSION
In this article, we introduce an efficient neural network model with
gating units and shortcut connections, named DEP-CNN, for multi-class
and multi-label text classification. Without dependence on lexical
resources or NLP toolkits, its DEPG can selectively control how much
context information is incorporated into each specific position by
converting a word embedding matrix and delivering it. We prove the
effectiveness of DEP-CNN by comparing it with other up-to-date
methods via extensive experiments on four known benchmark datasets.
 REFERENCE
[1] K. Kowsari, K. J. Meimandi, M. Heidarysafa, S. Mendu, L. E. Barnes, and D. E. Brown, “Text
classification algorithms: A survey,” 2019,arXiv:1904.08067. [Online]. Available:
http://arxiv.org/abs/1904.08067
[2] S. I. Wang and D. C. Manning, “Baselines and Bigrams: Simple, Good Sentiment and Topic
Classification,” in Proc. 50th Annu. Meeting Assoc.Computat. Linguistics (ACL), 2012, pp. 90–
94.
[3] A. Nigam, P. Sahare, and K. Pandya, “Intent detection and slots prompt in a closed-domain
chatbot,” in Proc. IEEE 13th Int. Conf. Semantic Computat. (ICSC), Newport Beach, CA, USA,
Jan. 2019, pp. 340–343.
[4] X. Kang, F. Ren, and Y. Wu, “Exploring latent semantic information for textual emotion
recognition in blog articles,” IEEE/CAA J. Automat. Sinica, vol. 5, no. 1, pp. 204–216, Jan. 2018.
[5] X. Wang, Q. Kang, J. An, and M. Zhou, “Drifted Twitter spam classification using multiscale
detection test on K-L divergence,” IEEE Access, vol. 7, pp. 108384–108394, 2019.