Presentation on

Information Extraction ( IE )
By

Santosh S. Peerappagol M.Tech 3rd Sem

(2GI19SCS05)

Under the Guidance of

Dr. V. S. Rajpurohit
Contents
1. O r g a n i z a t i o n P r o f i l e

2. I n t e r n s h i p O b j e c t i v e s a n d S c o p e

3. O v e r v i e w o f I E

4. P r o b l e m D e s c r i p t i o n

5. P r o p o s e d S o l u t i o n

6. I m p l e m e n t a t i o n D e t a i l s

7. P r o g r a m m i n g To o l s

8. A r c h i t e c t u r e D i a g r a m

9. I E A p p l i c a t i o n s

10. O u t p u t s
Organization Profile

1. Z e e l C o d e L a b s , i s a s / w d e v e l o p m e n t c o m p a n y s t a r t e d i n 2 0 1 2

2. T h e y a r e s p e c i a l i z i n g i n
o We b D e v e l o p m e n t
o Application Development
o Networking and Security Applications
3. T h e c o m p a n y u s e s t e c h n o l o g i e s l i k e P y t h o n , H a d o o p , C l o u d e t c
4. C l i e n t s a r e

o Belgaum Urban Development Authority (BUDA)

o Plus Point India Pvt Ltd. (Mumbai)

o Maharashtra Police (Head Quarters)

Internship Objectives and Scope
Objectives

1. To g a i n P r a c t i c a l K n o w l e d g e

2. To w o r k w i t h r e a l t i m e a p p l i c a t i o n

3. U n d e r s t a n d i n g r e a l w o r l d p r o b l e m s a n d f i n d i n g s o l u t i o n s t o t h e m

4. U n d e r s t a n d i n g / L e a r n i n g n e w t e c h n o l o g i e s

Scope

1. H o w t o d e a l w i t h R e a l Ti m e P r o b l e m s

2. To g a i n P r a c t i c a l K n o w l e d g e a n d E x p e r i e n c e s
What is Information Extraction ?

1. I E i s a p o w e r f u l N a t u r a l L a n g u a g e P r o c e s s i n g ( N L P ) c o n c e p t t h a t

will enable us to parse through any piece of text.

2. T h e task of Information Extraction (IE) involves extracting

meaningful information from unstructured text data and presenting

it in a structured format.
What is Information Extraction ? (Contd…)

For example, consider a cricket news article:

We c a n e x t r a c t t h e f o l l o w i n g i n f o r m a t i o n f r o m t h e t e x t
How does information extraction work?

1. We a l l k n o w t h a t s e n t e n c e s a r e m a d e u p o f w o r d s b e l o n g i n g t o

different Parts of Speech (POS).

2. T h e r e are eight different POS in the English language: Noun,

P r o n o u n , Ve r b , A d v e r b , A d j e c t i v e , P r e p o s i t i o n , C o n j u n c t i o n a n d

Intersection.

3. T h e P O S d e t e r m i n e s h o w a s p e c i f i c w o r d f u n c t i o n s i n m e a n i n g i n e a c h

sentence.
How does information extraction work? (Contd…)
For example, take the word “right”. In the sentence,

“The boy was awarded chocolate for giving the right answer ”

“right” is used as an adjective. Whereas, in the sentence,

“ Yo u h a v e t h e r i g h t t o s a y w h a t e v e r y o u w a n t ” ,

“right” is treated as a noun.

4. This show that the POS tag of a word carries a lot of significance
when it comes to understanding the meaning of a sentence. And we can
control it to extract meaningful information from our text.
Problem Description

1. In current digital age, the amount of natural language text is available is increasing every
day.

2. However, the complexity of natural language makes it difficult to access information in

that text.

3. So, NLP is able to build general purpose representations of meaning from unrestricted
text.

4. We focus our efforts on smaller set of “entity relations” like “How many people were
affected by pandemic”, “Who is culprit for murder”, etc.
Problem Description

The goal our project is to;

• How can we build a system that extract structured data ?

• What are the methods for identifying the entities and relationships described in a text?

• Which corpora are appropriate and how do we use them for training and evaluating

our models?
Proposed Solution

For each sentence/paragraph, we find entities based on characteristic of the sentence such as;

 Named Entity Recognition: Such as Date, Time, Org, Loc etc.

 POS Tags: Determines how specific word function in a sentence.

 Hypernyms: Are higher names. A hyponym is a word/phrase whose

semantic field is included within that of
 Hyponyms: Are lower names. another word as hypernym

 Meronyms: Denotes a constituent part of or a member of something.

Example: Finger is a meronym of hand

 Holonyms: Denotes a part of / the whole, which is denoted by a second term.

Example: Face is holonym of eye

 Dependency Parse: Process of analyzing the grammatical structure of a sentence based on the dependencies between
the words in a sentence.
Implementation Details
Tasks were performed are;
1 . Create a set of information templates and their properties.

Killing(Victim, perpetrator, location, Instrument) Diseases(Name, Location, Victims/casualties, Causes)

Disaster(Type, Location, Country, year)

2 . Create a corpus of natural language statements.

3. Implement deeper NLP pipeline to extract the following NLP based features from the natural language
statements.
a. Tokenize the articles into paragraphs and sentences
b. Lemmatize the words to extract lemmas as features
c. Part-of-speech (POS) tag the words to extract POS tag features
d. Perform dependency parsing to get subjects
e. Identify entities using hypernyms, hyponyms, meronyms and holonyms

4. Implement a combination of statistical and heuristic based approach to extract filled information templates from
the corpus of natural language statements.
Programming Tools

Following tools were used;

1 . NLTK-Natural Language Tool Kit.

NLTK is a leading platform for building Python programs to work with human language data. It provides easy
to use interfaces to over 50 corpora and lexical resources such as WordNet, along with a suite of text processing
libraries for classification, tokenization, stemming, tagging, parsing, and semantic reasoning.

2 . Spacy.

spaCy is an open-source software library for advanced natural language processing, written in the programming
languages Python.

3. Wordnet

WordNet is a lexical database of semantic relations between words in more than 200 languages. WordNet links
words into semantic relations including synonyms, hyponyms, and meronyms. The synonyms are grouped into
synsets with short definitions and usage examples.
Architectural Diagram Raw Text

Sentence
Figure shows the architecture for simple information extraction system ; 1 segmentation

1. Raw text of the document is split into sentences . 2 Tokenization

2. Each sentence is further subdivided into words.

Parts of Speech
3 Tagging
3. Each sentence is tagged with POS tags which helps in named entity
detection
Entity
4 Detection
4. We search for entities in each sentences
Relation
5. Finally, we use relation detection to extract relations between entities 5 Detection

Relations
Information Extraction Applications

Information Extraction has many applications, including;

o Business intelligence

o Resume harvesting

o Media analysis

o Sentiment detection (Technique used to interpret and classify emotions in subjective data)

o Patent search

o Email scanning
Outputs
Template 1: Killing (Victim, perpetrator, location, Instrument, Date)

Example:

The bullet responsible for killing Ron Helus from Ventura County during November's mass shooting at the
Borderline Bar & Grill was fired by Ian David Long, authorities said Friday. Ron Helus responded to the scene after
Ian David Long stormed into the Thousand Oaks bar Nov. 7 and sprayed the crowd with gunfire, killing 12 people
with his .45-caliber semi-automatic pistol.

Named Entity Recognition (NER)

Ron Helus PERSON Ventura Country ORG
Ventura Country ORG November DATE
November DATE the Borderline Bar & Grill ORG
the Borderline Bar & Grill ORG Ian David Long PERSON
Ian David Long PERSON Friday DATE
Friday DATE sixth ORDINAL
Ron Helus PERSON Helus NORP
Outputs (Contd…)

Extracted Output

{'Victim': 'Ron Helus', 'perpetrator': 'Ian David Long', 'Location': 'Ventura County Ventura Country Thousand Oaks

the Borderline Bar & Grill', 'Instrument': '.45-caliber semi-automatic pistol', 'Date': 'November 7 November Friday'}