Introduction to Semantics and Pragmatics

NLP tends to focus on:

Syntax
Grammars, parsers, parse trees, dependency structures

Semantics
Subcategorization frames, semantic classes, ontologies, formal semantics

Pragmatics
Pronouns, reference resolution, discourse models
LING 2000 - 2006
NLP 2

Semantics and Pragmatics

High-level Linguistics (the good stuff!) Semantics: the study of meaning that can be determined from a sentence, phrase or word. Pragmatics: the study of meaning, as it depends on context (speaker, situation)
LING 2000 - 2006
NLP

Language to Logic
John went to the book store.
John store1, go(John, store1)

John bought a book.

buy(John,book1)

John gave the book to Mary.

give(John,book1,Mary)

Mary put the book on the table.

put(Mary,book1,table1)
LING 2000 - 2006
NLP 4

Semantics Same event - different sentences

John broke the window with a hammer. John broke the window with the crack. The hammer broke the window. The window broke.

LING 2000 - 2006

NLP

Same event - different syntactic frames

John broke the window with a hammer.
SUBJ VERB OBJ MODIFIER

John broke the window with the crack.

SUBJ VERB OBJ MODIFIER

The hammer broke the window.

SUBJ VERB OBJ

The window broke.

SUBJ VERB
LING 2000 - 2006
NLP 6

Semantics -predicate arguments

break(AGENT, INSTRUMENT, PATIENT)

AGENT PATIENT INSTRUMENT John broke the window with a hammer.

INSTRUMENT PATIENT The hammer broke the window. PATIENT The window broke.
Fillmore 68 - The case for case
LING 2000 - 2006

NLP

AGENT
SUBJ

PATIENT
OBJ

INSTRUMENT
MODIFIER PATIENT

John broke the window with a hammer.

INSTRUMENT

The hammer broke the window.

SUBJ
PATIENT

OBJ

The window broke.

SUBJ
LING 2000 - 2006
NLP 8

Natural Language Processing Applications and Tasks

Machine Translation Question-Answering Information Retrieval Information Extraction

CIS 8590 Fall 2008

NLP

Machine Translation

One of the first applications for computers

bilingual dictionary > word-word translation

Good translation requires understanding!

War and Peace, The Sound and The Fury?

What can we do? Sublanguages.

technical domains, static vocabulary Meteo in Canada, Caterpillar Tractor Manuals, Botanical descriptions, Military Messages
LING 2000 - 2006

NLP

10

Example translation

LING 2000 - 2006

NLP

11

Machine Translation
The Story of the Stone
=The Dream of the Red Chamber (Cao Xueqin 1792)

Issues: (Language Divergences)

Sentence segmentation Zero-anaphora Coding of tense/aspect
Penetrate -> penetrated

Stylistic differences across languages

Bamboo tip plaintain leaf -> bamboos and plantains

Cultural knowledge
Curtain -> curtains of her bed

Machine Translation
Chinese gloss: Dai-yu alone on bed top think-of-with-gratitude Bao-chai again listen to window outside bamboo tip plantain leaf of on-top rain sound sigh drop clear cold penetrate curtain not feeling again fall down tears come

Hawkes translation: As she lay there alone, Dai-yus thoughts turned to Bao-chai Then she listened to the insistent rustle of the rain on the bamboos and plantains outside her window. The coldness penetrated the curtains of her bed. Almost without noticing it she had begun to cry.

Language Families

Babelfish Demo
http://babelfish.yahoo.com/ Old example:

The spirit is willing, but the flesh is weak.

Question Answering
What does door mean? What year was Abraham Lincoln born? How many states were in the United States when Lincoln was born? Was there a military draft during the Hoover administration? What do US scientists think about whether human cloning should be legal?

Modern QA systems
Still in infancy Simple factoid questions beginning to work OK Annual government-sponsored bakeoff called TREC

QA Demo
UIUC QA Demo Qualim QA Demo

Issues in NLP
Ambiguity! World Knowledge its needed for understanding, but computers dont have it

NLP

19

Ambiguity
Computational linguists are obsessed with ambiguity Ambiguity is a fundamental problem of computational linguistics Resolving ambiguity is a crucial goal

Ambiguity
Find at least 5 meanings of this sentence:
I made her duck

Ambiguity
Find at least 5 meanings of this sentence:
I made her duck

I cooked waterfowl for her benefit (to eat) I cooked waterfowl belonging to her I created the (plaster?) duck she owns I caused her to quickly lower her head or body I waved my magic wand and turned her into undifferentiated waterfowl At least one other meaning thats inappropriate for gentle company.

Ambiguity is Pervasive
I caused her to quickly lower her head or body
Lexical category: duck can be a N or V

I cooked waterfowl belonging to her.

Lexical category: her can be a possessive (of her) or dative (for her) pronoun

I made the (plaster) duck statue she owns

Lexical Semantics: make can mean create or cook

Ambiguity is Pervasive
Grammar: Make can be: Transitive: (verb has a noun direct object)
I cooked [waterfowl belonging to her]

Ditransitive: (verb has 2 noun objects)

I made [her] (into) [undifferentiated waterfowl]

Action-transitive (verb has a direct object and another verb) I caused [her] [to move her body]

Ambiguity is Pervasive
Phonetics!
I mate or duck Im eight or duck Eye maid; her duck Aye mate, her duck I maid her duck Im aid her duck I mate her duck Im ate her duck Im ate or duck I mate or duck

Headlines
Police Begin Campaign To Run Down Jaywalkers Iraqi Head Seeks Arms

Teacher Strikes Idle Kids

Miners Refuse To Work After Death Juvenile Court To Try Shooting Defendant

LING 2000 - 2006

NLP

26

Kinds of knowledge needed?

Consider the following interaction with HAL the computer from 2001: A Space Odyssey Dave: Open the pod bay doors, Hal. HAL: Im sorry Dave, Im afraid I cant do that.

Knowledge needed to build HAL?

Speech recognition and synthesis
Dictionaries (how words are pronounced) Phonetics (how to recognize/produce each sound of English)

Natural language understanding

Knowledge of the English words involved
What they mean How they combine (what is a `pod bay door?)

Knowledge of syntactic structure

Im I do, Sorry that afraid Dave Im cant

Whats needed?
Dialog and pragmatic knowledge
open the door is a REQUEST (as opposed to a STATEMENT or information-question) It is polite to respond, even if youre planning to kill someone. It is polite to pretend to want to be cooperative (Im afraid I cant) What is `that in `I cant do that?

Even a system to book airline flights needs much of this kind of knowledge

Eliza: Weizenbaum (1966)

User: You are like my father in some ways. ELIZA: What resemblance do you see User: You are not very aggressive but I think you dont want me to notice that. ELIZA: What makes you think I am not aggressive User: You dont argue with me ELIZA: Why do you think I dont argue with you User: You are afraid of me ELIZA: Does it please you to believe I am afraid of you

Modern conversational agents

Switchboard Receptionists
Implications: need to know how a native or non-native speaker of English is likely to pronounce a random American name. Eek.

Airline and other simple dialogue systems

Lots of fabulously fun pragmatic issues
Grounding Error handling

In general
Now that there are commercial installations Big incentive to use linguistic or any other kind of knowledge if it actually improves performance

Well-understood Semantic Tasks

Named-entity Recognition Coreference Resolution Semantic Role Labeling Sentiment Classification

Entities
Named Entity Tagging: Identify all the proper names in a text Sally went to see Up in the Air at the local theater.

Coreference Resolution: Identify all references (aka mentions) of people, places and things in text, and determine which mentions are coreferential.
John stuck his foot in his mouth.

Semantic Role Labeling

Semantic role labeling is computational task of assigning semantic roles to phrases
B-A0 REL B-A1 I-A1 B-AM I-AM John broke the window with a I-AM hammer.

Sentiment Classification
Given a review (about a movie, hotel, Amazon product, etc.), a sentiment classification system tries to determine what opinions are expressed in the review.

Coarse-level objective: is the review positive, negative, or neutral overall? Fine-grained objective: what are the positive aspects (according to the reviewer), and what are the negative aspects?