Indian Institute of Technology Kharagpur

Department of Computer Science & Engineering

CS60075
Natural Language Processing
Autumn 2021

Sudeshna Sarkar

Lecture 1A : Introduction
August 11 2021

8/11/2021 CS60075 Natural Language Processing, Autumn 2021, Sudeshna Sarkar, Computer Sc & Engineering Department, IIT Kharagpur 1
 Instructors
Teacher:
Sudeshna Sarkar (sudeshna@cse.iitkgp.ac.in)
Teaching Assistants
Alapan Kuila (alapan.cse@iitkgp.ac.in)
Aniruddha Roy (aniruddha.roy@iitkgp.ac.in)
Prithwish Jana (jprithwish@kgpian.iitkgp.ac.in)
Udit Dharmin Desai (uditdesai2206@iitkgp.ac.in)

8/11/2021 CS60075 Natural Language Processing, Autumn 2021, Sudeshna Sarkar, Computer Sc & Engineering Department, IIT Kharagpur 2
 Class Timing
• Wednesday (11:00--11:55 AM)
• Thursday (12:00--12:55 PM)
• Friday (08:00--08:55 AM)

Microsoft Teams

8/11/2021 CS60075 Natural Language Processing, Autumn 2021, Sudeshna Sarkar, Computer Sc & Engineering Department, IIT Kharagpur 3
 Books & References
• Speech and Language Processing (Daniel Jurafsky & James H. Martin)

8/11/2021 CS60075 Natural Language Processing, Autumn 2021, Sudeshna Sarkar, Computer Sc & Engineering Department, IIT Kharagpur 4
 Evaluation
• 3 class tests : 50 marks
• Assignments: 30 marks
• Project or Video Presentation : 20 marks

8/11/2021 CS60075 Natural Language Processing, Autumn 2021, Sudeshna Sarkar, Computer Sc & Engineering Department, IIT Kharagpur 5
 Information
• Website currently hosted at
https://sites.google.com/view/iitkgp-nlp-aut2021

8/11/2021 CS60075 Natural Language Processing, Autumn 2021, Sudeshna Sarkar, Computer Sc & Engineering Department, IIT Kharagpur 6
 Tentative Schedule – Part 1
Serial Date Time Module Topics
NLP 1 11 Aug Wednesday 11:00 AM Module 1 Introduction
NLP 2 12 Aug Thursday 12:00 PM
NLP 3 13 Aug Friday 8:00 AM
NLP 4 18 Aug Wednesday 11:00 AM Module 2 Language Modeling
Word
NLP 5 20 Aug Friday 8:00 AM Representation
NLP 6 25 Aug Wednesday 11:00 AM Morphology
NLP 7 26 Aug Thursday 12:00 PM
Module 3 ML Machine Learning
NLP 8 27 Aug Friday 8:00 AM Basics
NLP 9 1 Sep Wednesday 11:00 AM
NLP 10 2 Sep Thursday 12:00 PM Class Test 1

8/11/2021 CS60075 Natural Language Processing, Autumn 2021, Sudeshna Sarkar, Computer Sc & Engineering Department, IIT Kharagpur 7
 Tentative Schedule – Part 2
Serial Date Time Module Topics
NLP 11 3 Sep Friday 8:00 AM Module 4 Text Classification
NLP 12 8 Sep Wednesday 11:00 AM POS Tagging
NLP 13 9 Sep Thursday 12:00 PM NER
NLP 14 10 Sep Friday 8:00 AM CRF
NLP 15 15 Sep Wednesday 11:00 AM
NLP 16 16 Sep Thursday 12:00 PM Module 5 Syntax
NLP 17 17 Sep Friday 8:00 AM
NLP 18 22 Sep Wednesday 11:00 AM
NLP 19 23 Sep Thursday 12:00 PM
NLP 20 24 Sep Friday 8:00 AM Module 3 ML RNN
NLP 21 29 Sep Wednesday 11:00 AM
NLP 22 30 Sep Thursday 12:00 PM Class Test 2

8/11/2021 CS60075 Natural Language Processing, Autumn 2021, Sudeshna Sarkar, Computer Sc & Engineering Department, IIT Kharagpur 8
 Tentative Schedule – Part 3
Serial Date Time Module Topics
NLP 23 1 Oct Friday 8:00 AM Module 6 Machine Translation
NLP 24 6 Oct Wednesday 11:00 AM
NLP 25 7 Oct Thursday 12:00 PM
NLP 26 8 Oct Friday 8:00 AM
NLP 27 20 Oct Wednesday 11:00 AM
NLP 28 21 Oct Thursday 12:00 PM Module 7 Transformer
Contextual
NLP 29 22 Oct Friday 8:00 AM Representations
NLP 30 27 Oct Wednesday 11:00 AM
NLP 31 28 Oct Thursday 12:00 PM

8/11/2021 CS60075 Natural Language Processing, Autumn 2021, Sudeshna Sarkar, Computer Sc & Engineering Department, IIT Kharagpur 9
 Tentative Schedule – Part 4
Serial Date Time Module Topics
NLP 32 29 Oct Friday 8:00 AM Module 8 Applications
NLP 33 3 Nov Wednesday 11:00 AM
NLP 34 5 Nov Friday 8:00 AM
NLP 35 10 Nov Wednesday 11:00 AM
NLP 36 11 Nov Thursday 12:00 PM
NLP 37 12 Nov Friday 8:00 AM
NLP 38 17 Nov Wednesday 11:00 AM
NLP 39 18 Nov Thursday 12:00 PM Class Test 3
NLP 40 19 Nov Friday 8:00 AM Conclusion and Discussions

8/11/2021 CS60075 Natural Language Processing, Autumn 2021, Sudeshna Sarkar, Computer Sc & Engineering Department, IIT Kharagpur 10
 Natural Language Processing

Developing systems that Concerns how

allow computers to computational methods
communicate with people can aid the understanding
using natural language. of human language.

8/11/2021 CS60075 Natural Language Processing, Autumn 2021, Sudeshna Sarkar, Computer Sc & Engineering Department, IIT Kharagpur 11
 Automating Language

• Analysis: Language → Representation

• Generation: Representation → Language
• Acquisition: Obtaining the representation and
necessary algorithms, from knowledge and data

8/11/2021 CS60075 Natural Language Processing, Autumn 2021, Sudeshna Sarkar, Computer Sc & Engineering Department, IIT Kharagpur 12
 NLP Goals
Goals can be very ambitious
• True text understanding
• Good quality translation

Goals can be practical

• Question Answering
• Machine Translation services on the Web
• Conversational Agents
• Summarization

Natural language technology not yet perfected

• But still good enough for several useful applications

8/11/2021 CS60075 Natural Language Processing, Autumn 2021, Sudeshna Sarkar, Computer Sc & Engineering Department, IIT Kharagpur 13
 Course Focus

Linguistic Issues Science Goal

• Understand the way language

Modeling Techniques operates
• Probabilistic
• ML Engineering Goal

Engineering Methods • Build systems that analyse and

generate language
Multilinguality

8/11/2021 CS60075 Natural Language Processing, Autumn 2021, Sudeshna Sarkar, Computer Sc & Engineering Department, IIT Kharagpur 14
 What does it mean to “know” a language?

analysis
Natural Language
Representation
generation

8/11/2021 CS60075 Natural Language Processing, Autumn 2021, Sudeshna Sarkar, Computer Sc & Engineering Department, IIT Kharagpur