Xu-Ly-Ngon-Ngu-Tu-Nhien - Kai-Wei-Chang - 01-Intro - (Cuuduongthancong - Com)

Lecture 1:
Introduction
Kai-Wei Chang
CS @ University of Virginia
kw@kwchang.net
Couse webpage: http://kwchang.net/teaching/NLP16
CS6501– Natural Language Processing 1

CuuDuongThanCong.com https://fb.com/tailieudientucntt
Announcements
 Waiting list: Start attending the first few meetings
of the class as if you are registered. Given that
some students will drop the class, some space
will free up.
 We will use Piazza as an online discussion

platform. Please enroll.

Staff
 Instructor: Kai-Wei Chang

 Email: nlp16@kwchang.net
 Office: R412 Rice Hall
 Office hour: 2:00 – 3:00, Tue (after class).
 Additional office hour: 3:00 – 4:00, Thu
 TA: Wasi Ahmad
 Email: wua4nw@virginia.edu
 Office: R432 Rice Hall
 Office hour: 4:00 – 5:00, Mon

This lecture
 Course Overview
 What is NLP? Why it is important?
 What will you learn from this course?
 Course Information
 What are the challenges?
 Key NLP components

What is NLP
 Wiki: Natural language processing (NLP) is
a field of computer science, artificial
intelligence, and computational linguistics
concerned with the interactions between
computers and human (natural) languages.

Go beyond the keyword matching
 Identify the structure and meaning of

words, sentences, texts and conversations
 Deep understanding of broad language
 NLP is all around us
Machine translation
Facebook translation, image credit: Meedan.org

Statistical machine translation
Image credit: Julia Hockenmaier, Intro to NLP

Dialog Systems

Sentiment/Opinion Analysis

Text Classification
www.wired.com
 Other applications?
Question answering
'Watson' computer wins at 'Jeopardy'
credit: ifunny.com
Question answering
 Go beyond search

Natural language instruction
https://youtu.be/KkOCeAtKHIc?t=1m28s

Digital personal assistant
More on natural language instruction
credit: techspot.com
 Semantic parsing – understand tasks

 Entity linking – “my wife” = “Kellie” in the phone
book

Information Extraction
 Unstructured text to database entries
Yoav Artzi: Natural language processing

Language Comprehension
Christopher Robin is alive and well. He is the same

person that you read about in the book, Winnie the Pooh.
As a boy, Chris lived in a pretty home called Cotchfield
Farm. When Chris was three years old, his father wrote
a poem about him. The poem was printed in a magazine
for others to read. Mr. Robin then wrote a book
 Q: who wrote Winnie the Pooh?

 Q: where is Chris lived?

What will you learn from this course
 The NLP Pipeline

 Key components for
understanding text
 NLP systems/applications
 Current techniques & limitation
 Build realistic NLP tools

What’s not covered by this course
 Speech recognition – no signal processing
 Natural language generation
 Details of ML algorithms / theory
 Text mining / information retrieval

This lecture
 Course Overview

Overview
 New course, first time being offered

 Comments are welcomed
 Aimed at first- or second- year PhD students
 Lecture + Seminar
 No course prerequisites, but I assume
 programming experience (for the final project)
 basics of probability calculus, and linear
algebra (HW0)

Grading
 No exam & HW -- hooray

 Lectures & forum
 Participate in discussion (additional credits)
 Review quizzes (25%): 3 quizzes
 Critical review report (10%)
 Paper presentation (15%)
 Final project (50%)

Quizzes
 Format
 Multiple choice questions
 Fill-in-the-blank
 Short answer questions
 Each quiz: ~20 min in class
 Schedule: see course website
 Closed book, Closed notes, Closed laptop

Critical review report
 1 page maximum
 Pick one paper from the suggested list
 Summarize the paper (use you own words)
 Provide detailed comments
 What can be improved
 Potential future directions
 Other related work
 Some students will be selected to present
their critical reviews

Paper presentation
 Each group has 2~3 students

 Picked one paper from the suggested
readings, or your favorite paper
 Cannot be the same as critical review report
 Can be related to your final project
 Register your choice early
 15 min presentation + 2 mins Q&A
 Will be graded by the instructor, TA, other
students
Final Project
 Work in groups (2~3 students)

 Project proposal
 Written report, 2 page maximum
 Project report (35%)
 < 8 pages, ACL format
 Due 2 days before the final presentation
 Project presentation (15%)
 5-min in-class presentation (tentative)

Late Policy
 Credit of 48 hours for all the assignments

 Including proposal and final project
 No accumulation
 No more grace period
 No make-up exam
 unless under emergency situation

Cheating/Plagiarism
 No. Ask if you have concerns

 UVA Honor Code:
http://www.virginia.edu/honor/

Lectures and office hours
 Participation is highly appreciated!

 Ask questions if you are still confusing
 Feedbacks are welcomed
 Lead the discussion in this class
 Enroll Piazza
https://piazza.com/virginia/fall2016/cs6501004

Topics of this class
 Fundamental NLP problems

 Machine learning & statistical approaches
for NLP
 NLP applications
 Recent trend in NLP

What to Read?
 Natural Language Processing

ACL, NAACL, EACL, EMNLP, CoNLL, Coling, TACL
aclweb.org/anthology
 Machine learning
ICML, NIPS, ECML, AISTATS, ICLR, JMLR, MLJ
 Artificial Intelligence
AAAI, IJCAI, UAI, JAIR

Questions?

This lecture
 Course Overview

Challenges – ambiguity
 Word sense ambiguity

 Word sense / meaning ambiguity
Credit: http://stuffsirisaid.com
 PP attachment ambiguity
Credit: Mark Liberman, http://languagelog.ldc.upenn.edu/nll/?p=17711

Challenges -- ambiguity
 Ambiguous headlines:
 Include your children when baking cookies
 Hospitals are Sued by 7 Foot Doctors
 Iraqi Head Seeks Arms
 Safety Experts Say School Bus Passengers

Should Be Belted

 Pronoun reference ambiguity
Credit: http://www.printwand.com/blog/8-catastrophic-examples-of-word-choice-mistakes

Challenges – language is not static
 Language grows and changes

 e.g., cyber lingo
LOL Laugh out loud
G2G Got to go
BFN Bye for now
B4N Bye for now
Idk I don’t know
FWIW For what it’s worth
LUWAMH Love you with all my heart

Challenges--language is compositional
Carefully
Slide

Challenges--language is compositional
小心: 地滑:
Carefully Slide
Careful Landslip
Take Wet Floor
Care Smooth
Caution

Challenges – scale
 Examples:
 Bible (King James version): ~700K
 Penn Tree bank ~1M from Wall street journal
 Newswire collection: 500M+
 Wikipedia: 2.9 billion word (English)
 Web: several billions of words

This lecture
 Course Overview

Part of speech tagging

Syntactic (Constituency) parsing

Syntactic structure => meaning
Image credit: Julia Hockenmaier, Intro to NLP

Dependency Parsing

Semantic analysis
 Word sense disambiguation

 Semantic role labeling
Credit: Ivan Titov

Q: [Chris] = [Mr. Robin] ?
Christopher Robin is alive and well. He is the
same person that you read about in the book,
Winnie the Pooh. As a boy, Chris lived in a
pretty home called Cotchfield Farm. When
Chris was three years old, his father wrote a
poem about him. The poem was printed in a
magazine for others to read. Mr. Robin then
wrote a book
Slide modified from Dan Roth
49
Co-reference Resolution
Christopher Robin is alive and well. He is the

same person that you read about in the book,
Winnie the Pooh. As a boy, Chris lived in a
pretty home called Cotchfield Farm. When
Chris was three years old, his father wrote a
poem about him. The poem was printed in a
magazine for others to read. Mr. Robin then
wrote a book
50
Questions?


Xu-Ly-Ngon-Ngu-Tu-Nhien - Kai-Wei-Chang - 01-Intro - (Cuuduongthancong - Com)

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Xu-Ly-Ngon-Ngu-Tu-Nhien - Kai-Wei-Chang - 01-Intro - (Cuuduongthancong - Com)

Uploaded by

Copyright:

Available Formats

Lecture 1:

Couse webpage: http://kwchang.net/teaching/NLP16

CS6501– Natural Language Processing 1

 We will use Piazza as an online discussion

CS6501– Natural Language Processing 2

 Instructor: Kai-Wei Chang

CS6501– Natural Language Processing 3

CS6501– Natural Language Processing 4

CS6501– Natural Language Processing 5

 Identify the structure and meaning of

Facebook translation, image credit: Meedan.org

CS6501– Natural Language Processing 7

Image credit: Julia Hockenmaier, Intro to NLP

CS6501– Natural Language Processing 8

CS6501– Natural Language Processing 9

CS6501– Natural Language Processing 10

'Watson' computer wins at 'Jeopardy'

CS6501– Natural Language Processing 13

CS6501– Natural Language Processing 14

 Semantic parsing – understand tasks

CS6501– Natural Language Processing 15

 Unstructured text to database entries

Yoav Artzi: Natural language processing

CS6501– Natural Language Processing 16

Christopher Robin is alive and well. He is the same

 Q: who wrote Winnie the Pooh?

CS6501– Natural Language Processing 17

 The NLP Pipeline

 Build realistic NLP tools

CS6501– Natural Language Processing 18

 Speech recognition – no signal processing

 Natural language generation

 Details of ML algorithms / theory

 Text mining / information retrieval

CS6501– Natural Language Processing 19

CS6501– Natural Language Processing 20

 New course, first time being offered

CS6501– Natural Language Processing 21

 No exam & HW -- hooray

CS6501– Natural Language Processing 22

CS6501– Natural Language Processing 23

CS6501– Natural Language Processing 24

 Each group has 2~3 students

 Work in groups (2~3 students)

CS6501– Natural Language Processing 26

 Credit of 48 hours for all the assignments

CS6501– Natural Language Processing 27

 No. Ask if you have concerns

CS6501– Natural Language Processing 28

 Participation is highly appreciated!

CS6501– Natural Language Processing 29

 Fundamental NLP problems

CS6501– Natural Language Processing 30

 Natural Language Processing

CS6501– Natural Language Processing 31

CS6501– Natural Language Processing 32

CS6501– Natural Language Processing 33

 Word sense ambiguity

CS6501– Natural Language Processing 34

 Word sense / meaning ambiguity

Credit: Mark Liberman, http://languagelog.ldc.upenn.edu/nll/?p=17711

CS6501– Natural Language Processing 36