Discourse and Pragmatic Processing: Introduction To Natural Language Processing (CSE 5321)

Introduction to Natural Language Processing (CSE 5321)
Lecture 05: Discourse and Pragmatic Processing
Department of Computer Science and Engineering

Adama Science and Technology University
Teshome M. Bekele
2020/21—Sem I
Introduction
Concepts in Discourse Processing
Discourse Segmentation
Applications of Discourse Processing
Reference Resolution
Issues in Discourse Modeling
Pragmatics
• Discourse deals with the properties of the text as a whole that convey meaning by
making connections between component sentences.
• Much of language interpretation is dependent on the preceding discourse/dialogue.
• Discourse imposes meaning and structure on individual sentences (or utterances) that
go well beyond the compositional meaning of sentences in isolation.
• For example:
The meaning of a text is entirely different when we reorder the sentences.

A sentence in isolation is frequently ambiguous, whereas a sentence in a
naturalistic discourse context is rarely ambiguous.
• Important issues that are concerned with discourse processing are context, cohesion,
coherence, and rhetorical structure.
• The most common methods applied for discourse processing are discourse
segmentation and reference resolution.
Department of Computer Science and Engineering, ASTU Lecture 05: Discourse and Pragmatic Processing 2/18
Introduction
Pragmatics
• Discourse processing is required, among others, for:

Natural language understanding
Text summarization
Machine translation
Natural language generation
Introduction
Pragmatics
Issues in Discourse Modeling: Context and Cohesion
• Context includes the participants in:

the acts of communication (speaker, listener, overhearer, reader, writer,
narrator, narratee);
the communication setting and goals, the pragmatic ground-rules;

the subject matter knowledge; and
a host of other components that situate the discourse event.
• Cohesion refers to the linguistic elements or features that signal how to connect
constituents, such as:
connectives (and, in order to, although, etc.);

punctuation;
verb tense; and
syntactic markers.
Introduction
Pragmatics
Issues in Discourse Modeling: Coherence
• Coherence refers to the linguistic conceptual knowledge that inter-relates constituents

in the discourse, such as time, space, causality, goals, and agency.
• Coherence is a property of well-written texts that makes them easier to read and
understand than a sequence of randomly strung sentences.
• Coherence relations
Result: S0 causes S1
Abebe was sleepy. He went to the coffee shop.
Explanation: S1 causes S0.
Abebe went to the coffee shop. He was sleepy.
Parallel: S0 and S1 are parallel.
Abebe drinks coffee. Almaz drinks tea.
Elaboration: S1 is an elaboration of S0.
Abebe likes coffee. He drinks it every day.
Contrast: S1 contrasts S0
Abebe likes coffee. Almaz hates it.
…
Introduction
Pragmatics
Issues in Discourse Modeling: Rhetorical Structure
• Rhetorical Structure specifies the global organization of discourse, such as:

setting + plot + moral;
problem + solution;
compare / contrast;
claim + evidence;
question + answer; and
argue + counter-argue.
• Rhetorical Structure aims to describe “building blocks” of text structure.
Nucleus (central segment of text) vs. satellites (peripheral segments of text)
Binary relations between discourse units

• Compositionality principle defines how to build a tree from binary relations.
Introduction
Pragmatics
Issues in Discourse Modeling: Rhetorical Structure
Example
[No matter how much one wants to stay a non-smoker, A] [the truth is that the pressure
to smoke in junior high is greater than it will be any other time of one’s life B]. [We know
that 3,000 teens start smoking each day C] [although it is a fact that 90% of them once
thought that smoking was something that they’ll never do. D]
Binary relations Tree built from binary relations
JUSTIFICATION
(JUSTIFICATION, A, B)
(JUSTIFICATION, D, B)
(EVIDENCE, C, B)
(CONCESSION, C, D) A B C D
(RESTATEMENT, D, A) JUSTIFICATION CONCESSION
Introduction
Discourse Segmentation Discourse Segments
Reference Resolution Building Discourse Structure
Pragmatics
Discourse Segments
• Discourse Segmentation refers to the task of determining the positions at which topics
change in a stream of text or speech.
• Utterances form discourse segment, and a discourse is made up of embedded

discourse segments.
• Discourse structure:
Discourse can be partitioned into segments, which can be connected in a

limited number of ways.
Speakers use linguistic devices to make this structure explicit (e.g.,
cue phrases, intonation, gesture, etc.).
Listeners comprehend discourse by recognizing this structure.
Introduction
Discourse Segmentation Discourse Segments
Reference Resolution Building Discourse Structure
Pragmatics
Building Discourse Structure
• Building discourse structure involves the following tasks.

Identify discourse segment boundaries.
What exactly is a discourse segment?

Any evidence that humans naturally recognize segment
boundaries?
Do humans agree on segment boundaries?

How to find the boundaries automatically?
Determine relations between segments.
Determine intentions of the segments.
Introduction
Anaphora and Coreference Resolutions
Types of Referential Expressions
Syntactic and Semantic Constraints
Pragmatics
• References in a given text can be anaphoric or coreferential noun phrases.

• The task of reference resolution is to determine which noun phrases refer to each real-
world entity mentioned in the text.
• Anaphora:
An expression α1 is in an anaphoric relation with expression α2 if and only if

the interpretation of α1 depends on α2.
The relation holds within a text.

• Coreference:
Two expressions α1 and α2 are coreferent if and only if
Referent (α1) = Referent (α2).
The expressions can be in the same text or different texts, in the same
language or different language.
Introduction
Pragmatics
• Some expressions are both coreferential and anaphoric.

A bus had to divert to the local hospital when one of the
passengers had a heart attack. It got to the hospital in time
and the man’s life was saved.
Coreferential: {the local hospital, the hospital} , {bus, it},
{one of the passengers, the man}
Anaphoric: {it}
• Some expressions are coreferential but not anaphoric.
Captain Abebe was a good seaman, worthy of the frigate he

commanded. His vessel and he were one. He was the soul of it.
Coreferential: {the frigate, his vessel, it}

Anaphoric: {his vessel, it, he}
• We can see that coreference resolution is a harder task than anaphora resolution.
Introduction
Pragmatics
• Pronouns
Abebe went to Kebede’s party, and parked next to a Toyota car. He went inside
and talked to Kebede for more than an hour. Kebede told him that he recently got
engaged. He also said that he bought it yesterday.
• Nouns
Indefinite Noun Phrases:
I saw a Toyota car today. Some Toyota cars were being unloaded. I saw
this awesome Toyota car today.
Definite Noun Phrases:
I saw a Toyota car today. The car was white and needed to be washed.
• Demonstratives (this, that): capture spatial proximity.
I like this house, better than that.
• “One” Anaphora: evokes a new entity into the discourse whose description is
dependent of this new entity.
I saw no less than 6 Toyota cars today. Now I want one.
Introduction
Pragmatics
Preferences in Pronoun Interpretation

• Recency: Entities introduced in recent utterances are more salient than those
introduced further back.
Abebe has a Toyota car. Kebede has a BMW. Aster likes to drive it.
• Repeated mention: Entities that have been focused on in the prior discourse are more
likely to continue to be focused on in subsequent discourse.
Abebe needed a car to get his new job. He decided that he wanted something
sporty. Kebede went to the Toyota dealership with him. He bought a Corolla.
• Grammatical role: Hierarchy of candidate entities based on their grammatical role.

Abebe went to the Toyota dealership with Kebede. He bought a Corolla.
Kebede went to the Toyota dealership with Abebe. He bought a Hilux.
• Parallelism:
Aster went with Almaz to the Toyota dealership. Abebe went with her to the
Mazda dealership.
Introduction
Pragmatics
Syntactic Constraints
• Number Agreement
Abebe has a new car. It is red.
Abebe has three new cars. They are red.
• Person and Case Agreement

Abebe and Almaz have cars. We love them.
You and I have cars. We love them.
• Gender Agreement
Abebe has a car. It is attractive.
• Syntactic Agreement
* Abebe bought himself a new car.
Abebe bought him a new car.
Introduction
Pragmatics
Semantic Constraints
• Selectional restrictions of the verb on its arguments
Abebe parked his car in the garage. He had driven it around for hours.
Abebe parked his car in the garage. It is incredibly messy, with old bike and car
parts lying around everywhere.
I bought a bottle of wine, sat down on a stone, and drank it.
Introduction
Discourse Segmentation Pragmatic Processing
Reference Resolution Approaches to Pragmatic Processing
Pragmatics
Pragmatic Processing
• Pragmatics is the study of how linguistic properties and contextual factors interact in
the interpretation of utterances, enabling hearers to bridge the gap between sentence
meaning and speaker’s meaning.
• The study of pragmatics comprises interactions between

entities on different levels of the linguistic structure; and
the linguistic and the non-linguistic reality.
• Classical works on pragmatics indentify problems as:
What entities does a given message refer to (discourse referents)?
What background knowledge is needed to understand a given message?

How do the beliefs of speaker and hearer interact in the interpretation of a
message?
What is a relevant answer to a given question?
Introduction
Discourse Segmentation Pragmatic Processing
Reference Resolution Approaches to Pragmatic Processing
Pragmatics
Approaches to Pragmatic Processing
• Problem solutions focus on:

General vs. domain-specific resources and algorithms
User models (beliefs, attitudes, etc.).

The interplay between prosody, syntax, and semantics.
For example, identification of discourse referents in spoken language

requires interplay between phonetic/phonological, morphological,
syntactic, and semantic factors as well as the use of extra linguistic
knowledge.
New means of communication, new types of discourse.
TOC: Course Syllabus
Previous: Semantic Analysis
Current: Discourse and Pragmatic Processing
Next: Disambiguation

Discourse and Pragmatic Processing: Introduction To Natural Language Processing (CSE 5321)

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Discourse and Pragmatic Processing: Introduction To Natural Language Processing (CSE 5321)

Uploaded by

Copyright:

Available Formats

Introduction to Natural Language Processing (CSE 5321)

Lecture 05: Discourse and Pragmatic Processing

Department of Computer Science and Engineering

Concepts in Discourse Processing

• Much of language interpretation is dependent on the preceding discourse/dialogue.

The meaning of a text is entirely different when we reorder the sentences.

Applications of Discourse Processing

• Discourse processing is required, among others, for:

Natural language generation

Issues in Discourse Modeling: Context and Cohesion

• Context includes the participants in:

the communication setting and goals, the pragmatic ground-rules;

a host of other components that situate the discourse event.

connectives (and, in order to, although, etc.);

verb tense; and

Issues in Discourse Modeling: Coherence

• Coherence refers to the linguistic conceptual knowledge that inter-relates constituents

Issues in Discourse Modeling: Rhetorical Structure

• Rhetorical Structure specifies the global organization of discourse, such as:

question + answer; and

Nucleus (central segment of text) vs. satellites (peripheral segments of text)

Binary relations between discourse units

Issues in Discourse Modeling: Rhetorical Structure

Binary relations Tree built from binary relations

• Utterances form discourse segment, and a discourse is made up of embedded

Discourse can be partitioned into segments, which can be connected in a

Listeners comprehend discourse by recognizing this structure.

Building Discourse Structure

• Building discourse structure involves the following tasks.

What exactly is a discourse segment?

Do humans agree on segment boundaries?

Determine intentions of the segments.

Anaphora and Coreference Resolutions

• References in a given text can be anaphoric or coreferential noun phrases.

An expression α1 is in an anaphoric relation with expression α2 if and only if

The relation holds within a text.

Two expressions α1 and α2 are coreferent if and only if

Referent (α1) = Referent (α2).

Anaphora and Coreference Resolutions

• Some expressions are both coreferential and anaphoric.

Coreferential: {the local hospital, the hospital} , {bus, it},

{one of the passengers, the man}

• Some expressions are coreferential but not anaphoric.

Captain Abebe was a good seaman, worthy of the frigate he

Coreferential: {the frigate, his vessel, it}

Types of Referential Expressions

Types of Referential Expressions

Preferences in Pronoun Interpretation

• Grammatical role: Hierarchy of candidate entities based on their grammatical role.

Syntactic and Semantic Constraints

• Person and Case Agreement

Abebe has a car. It is attractive.

Syntactic and Semantic Constraints

I bought a bottle of wine, sat down on a stone, and drank it.

• The study of pragmatics comprises interactions between

the linguistic and the non-linguistic reality.

• Classical works on pragmatics indentify problems as:

What entities does a given message refer to (discourse referents)?

What background knowledge is needed to understand a given message?

Approaches to Pragmatic Processing

• Problem solutions focus on:

User models (beliefs, attitudes, etc.).