TA 14

12S17004 Fivin Sadesla Tambunan

12S17026 Mika Lestari Valentina Manurung
12S17037 Nita Sophia Winandi Sirait
 EMBEDDINGS FROM LANGUAGE MODELS
REPRESENTATIONS BASED BIDIRECTIONAL LONG
SHORT-TERM MEMORY AND CONDITIONAL
RANDOM FIELDS MODEL FOR
INDONESIAN NAMED-ENTITY RECOGNITION
 What is Named Entity Recognation
1

Background
2

Research Question
3

Outlines
Research Purpose
4

5 Research Method

6 ELMo

BiLSTM-CRF
7
Named Entity Recognition

Task of information extraction that seeks to locate and classify named entities mentioned in
unstructured text into pre-defined categories such as person names, organizations, locations, etc.

NER is used for : • machine translation

• question answering
• information retrieval
• text to speech

Example of Indonesia NER :

Setelah lulus pada 1985 , Jokowi bekerja di BUMN dan ditempatkan di Aceh .
 Background

Semi-Supervised In order for the

Bidirectional Long information obtained
Short Term Memory to be more specific, to
and Conditional increase the number of
Random Fields Model entities, namely, person,
for Named- organization, location,
Entity Recognition date, time, currency,
Using and event entity
Embeddings from
Language Models
Representations,
with F1 score 86.1 %
 1
How to make ELMo model based on
BiLSTM-CRF on Indonesian NER?

2
How does ELMo perform based on BiLSTM-CRF on
Indonesian NER with person, organization, location,
date, time, currency and event entity?
 2
1 Knowing ELMo based on
Knowing ELMo modeling using the BiLSTM-CRF 's performance on
BiLSTM-CRF algorithm in Indonesian Indonesian NER with person,
NER organization, location, date, time,
currency and event entities.
 Research Methodology

1. problem 2. Study of 3. Data 4. Data

statement literature preparation preprocessing

5. Model Analysis

8. Evaluation and
9. Formulation of
discussion of 7. Implementation 6. Design
conclusion
results
 ELMo (Embedding from Language Models)

ELMo is a new type of deep contextualized word representation that can model complex features
(such as syntax and semantics) and changes in words in a language context.
BiLSTM-CRF Model

Conditional Random Field

Bidirectional Long Short-
(CRF) is a statistical mapping
Term Memory (LSTM) learns
method applied to pattern
the output weight of the
recognition and machine
previous time and the input of
learning. In the field of Natural
each sequence at the
Language Processing,
current time. Information in
Conditional Random Field is
the previous time (backward)
applied to Named Entity
and information at the time
Recognition. CRF are include
after (forward) in the
d in a probabilistic model that
sentence sequence can be
is widely used in the
captured simultaneously by
segmentation and labeling
the forward layer and
process of a data sequence.
backward layer with separate
hidden layers in BiLSTM to get
sentence sequence modeling
process.