Natural Language Processing - Part 1

Abstract
With the recent hype about the “imagenet moment” of Natural Language Processing (NLP)
[you can try to google this if you want to know what this means], we have decided to put
together 2 paper review sessions to go through the journey of how modern NLP reach the
“imagenet moment”. In particular, we are going to talk about traditional methods such as
tf-idf and it’s shortcomings, to what role the idea of embeddings play in overcoming the
weaknesses in traditional methods, to how embeddings enable the “imagenet moment”. This
will be a 2 part paper review series, where we aim to cover basic concepts about language
model and word embeddings in detail in the first part. This will hopefully allow everyone to
understand the inner workings of the more recent state of the art word representation
methods such as ELMo and CoVe in the second part.

We know that research papers can be technical and too time consuming to read, therefore
we have decided to include some blogs and articles that explains the high level concept very
well. We still hope that participants will read the paper, but do at least go through the articles
if you do not read the paper.

First Part:
Here are the original papers:
1. https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-
and-their-compositionality.pdf​ (Main)
2. https://nlp.stanford.edu/pubs/glove.pdf​ (Optional)

We will be covering:
1. What is a language model?
a. https://mchromiak.github.io/articles/2017/Nov/30/Explaining-Neural-Language
-Modeling/#.XKSq6-szby1
b. https://www.cs.bgu.ac.il/~elhadad/nlp18/nlp02.html
2. What are the common tasks in NLP?
a. https://github.com/sebastianruder/NLP-progress
3. What are word embeddings and how to train them?
a. https://towardsdatascience.com/introduction-to-word-embedding-and-word2ve
c-652d0c2060fa
b. http://mccormickml.com/2016/04/19/word2vec-tutorial-the-skip-gram-model/
c. http://mccormickml.com/2017/01/11/word2vec-tutorial-part-2-negative-samplin
g/
d. https://www.analyticsvidhya.com/blog/2017/06/word-embeddings-count-word2
veec/
4. How can the idea of embeddings be used outside of NLP?
a. https://towardsdatascience.com/collaborative-embeddings-for-lipstick-recomm
endations-98eccfa816bd