Term Frequency –

Inverse
Document Frequency
Mr. V. M. Vasava
GPG,Surat
IT Dept.
 Agenda

INTRODUCTION TF-IDF EXAMPLE

TF -IDF

• Feature Extraction: The mapping from textual data to

real valued vector is called feature extraction.

• BOW (Bag of Words): list of unique words in the text corpus.

• TF-IDF : to count the number of times each word appears in a

document.
Introduction about TF-
IDF
• TF-IDF stands for Term Frequency Inverse Document
Frequency of records. It can be defined as the calculation of
how relevant a word in a series or corpus is to a text.
• The meaning increases proportionally to the number of times
in the text a word appears but is compensated by the word
frequency in the corpus (data-set).
• Vectorization is the process of converting words into numbers
is called Vectorization.
Steps of TF-IDF
1. Clean data / Preprocessing — Normalize data( all lower
case), Stemming, lemmatize data ( all words to root words ).
2. Tokenize words with frequency.
3. Find TF for words.
4. Find IDF for words.
5. Vectorize vocab.
TF-IDF

• TF -(Term Frequency) -It is the ratio of the occurrence of the word (w)
in document (d) per the total number of words in the documents.
No. of repetition of words in sentence
Term Frequency = No. of words in sentence
OR

The weight of a term that occurs in a document is simply proportional to

the term frequency.
TF-IDF

Corpus Text Target

Doc1 He is a good boy 1
Doc2 She is a good girl 1
Doc3 boy and girl are good 0

Count words of Document

Remove punctuation or stop words

• Apply stop words and remove punctuation . Corpus become

unique words.

Corpus​​ Text​​
Doc1​​ good boy​​
Doc2​​ good girl​​
Doc3​​ Boy girl good​​
Create the frequency distribution of words
Corpus​​ Text​​ Vocabulary​ Frequency of words​
Doc1​​ good boy​​ good​ 3​
Doc2​​ good girl​​ boy​ 2​
Doc3​​ boy girl good​​ girl​ 2​

TF
Doc1 Doc2 Doc3
good 1/2 1/2 1/3
boy 1/2 0 1/3
girl 0 1/2 1/3
 IDF

• Inverse Document Frequency (IDF)

• IDF calculates the importance of a word in a corpus D.
• it tests how relevant the word is. The key aim of the search is
to locate the appropriate records that fit the demand.
• No. of sentences
• IDF(t)= log No. of sentences containing words
• OR
Term Frequency Inverse Document Frequency (TFIDF)
• TF-IDF is the product of term frequency and inverse document
frequency. It gives more importance to the word that is rare in
the corpus and common in a document.
 TF Calculate IDF
Doc1 Doc2 Doc3 Words IDF
good 1/2 1/2 1/3 good Log(3/3) =0

boy 1/2 0 1/3 boy Log(3/2)=

girl 0 1/2 1/3 girl Log(3/2)=

TF-IDF =TF *IDF

​ Feature1(good)​ Feature2(boy)​ Feature3(girl)​

Doc1​ 0 ½*Log(3/2) 0
Doc2​ 0​ 0 ½*Log(3/2)
Doc3​ 0 1/3*Log(3/2) 1/3*Log(3/2)
Implementation of TF-IDF
Example
Advantages & Disadvantages
• Reflects Word Importance: TF-IDF highlights words that
are important to a specific document in a corpus.
• Reduces Emphasis on Common Words: Commonly
occurring words (e.g., "the," "is," "and") often have high term
frequencies but low importance.
• Handles Variable Document Lengths: TF-IDF accounts for
variations in document lengths by considering the relative
frequency of terms in a document.
• Support text retrieval system like google search, text
classification, keyword extraction.
Disadvantages
• Sparsity
• Out of vocabulary(OOV)
• ordering
Any Questions????