See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/359710740

DEEP LEARNING BASED AFAN OROMO TEXT SUMMARIZER A MASTER'S

THESIS

Thesis · January 2022

CITATIONS READS

0 204

1 author:

Bulte Belay
Addis Ababa Science and Technology University
2 PUBLICATIONS   0 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Deep Learning View project

All content following this page was uploaded by Bulte Belay on 04 April 2022.

The user has requested enhancement of the downloaded file.

 BULTE BELAY
DEEP LEARNING BASED AFAN OROMO TEXT SUMMARIZER

ADDIS ABABA SCIENCE AND TECHNOLOGY UNIVERSITY

DEEP LEARNING BASED AFAN OROMO TEXT

SUMMARIZER

A MASTER’S THESIS

BY

BULTE BELAY HORDOFA

DEPARTMENT OF SOFTWARE ENGINEERING

COLLEGE OF ELECTRICAL AND MECHANICAL

ENGINEERING
AASTU 2022

FEBRUARY, 2022
 ADDIS ABABA SCIENCE AND TECHNOLOGY UNIVERSITY

DEEP LEARNING BASED AFAN OROMO TEXT

SUMMARIZER

A MASTER’S THESIS

BY

BULTE BELAY HORDOFA

A Thesis Submitted to the Department of Software Engineering in Partial Fulfillment for the Degree of
Master of Science in Software Engineering

To

DEPARTMENT OF SOFTWARE ENGINEERING

FEBRUARY, 2022
Scanned by CamScanner
Scanned by CamScanner
Abstract

The number of useful resources for extracting information from internet documents,
articles, and news has significantly expanded in recent years. These materials are
lengthy and time-consuming to read and comprehend. These materials must be
summarized in order to extract important information and save time. Text
summarization approach can be divided into extractive or abstractive summarizer.
Extractive summarization selects the most important (meaningful) sentences in an
article without any modification. As abstractive text summarization needs a real
knowledge and understanding of the document it is harder than extractive
summarization. Although, there are many deep learning based abstractive summarizer
model for foreign languages, there is no reported research work on abstractive text
summarizer using deep learning for this languages to the best knowledge of the
researcher. Therefore, this study proposes three deep learning models: Seq2Seq model
with LSTM, Seq2Seq model with Bi-LSTM and Seq2Seq model with Hybrid LSTM to
perform a comparison analysis for Afaan Oromoo dataset.

Our experiments result indicates that Seq2Seq model with LSTM model achieves
95.48% of training accuracy and 94.84% of validation accuracy, Seq2Seq model with
Bi-LSTM scores 99.95% of training accuracy and 99.97 % of validation accuracy and
Seq2Seq model with hybrid-LSTM scores 94.37% of training accuracy and 93.92% of
validation accuracy. The model generated summaries are also evaluated for ROUGE-1
and ROUGE-2 metrics. Accordingly, 32.15% rouge-1 recall, 32.85% rouge-1
precision, 32.49% rouge-1 f-measure and 6.5% rouge-2 recall, 7.7% rouge-2 precision
and 7.04% rouge-2 f-measure were obtained by LSTM model. Bi-LSTM achieves
41.77% rouge-1 recall, 31.33% rouge-1 precision, 35.8% rouge-1 f-measure and
16.33% rouge-2 recall, 13.1% rouge-2 precision and 14.18% rouge-2 f-measure. The
third model, hybrid-LSTM model achieves 21.69% rouge-1 recall, 22.27% rouge-1
precision, 21.31% rouge-1 f-measure and 6.39% rouge-2 recall, 7.007% rouge-2
precision and 6.68% rouge-2 f-measure.

Keywords: Automatic summarization, NLP, Extractive summarization, Abstractive

summarization, LSTM, Bi-LSTM, Hybrid LSTM.

iv
Acknowledgements

I would first like to thank and praise the Almighty God for giving me this chance and
guiding me to achieve it. I will praise you!
My special thanks go to my research advisor Dr. Tulu Tilahun. Doctor, you are
encouraging, a good guider and you understands me a lot. You have invested your
invaluable time to finalize this thesis work. Thank you!
I would also like to thank my father Balaay Hordofaa and my mother Wogaayyoo
Hirphaa. Your prayers for me is invaluable. My wife Ayyaantuu Sintaayyoo, a gift
from heaven, your roll in this thesis work is bold. You helped me collecting the dataset.
Thank you and I Love You! My brothers Dalasaa Balaay and Humnawaaq Balaay,
my one and only sister Kitilee Balaay for your unfailing support and encouragement
throughout my years of study.
Finally, I must express my thanks to all contributors on this thesis work.

v
Table of Contents
Declaration .....................................................................................................................ii
Approval Page .............................................................................................................. iii
Abstract ......................................................................................................................... iv
Acknowledgements ........................................................................................................ v
Abbreviations ................................................................................................................ ix
List of Figures ................................................................................................................ x
CHAPTER ONE ............................................................................................................ 1
1 Introduction ............................................................................................................ 1
1.1 Background ..................................................................................................... 1
1.2 Motivation ....................................................................................................... 2
1.3 Statement of the problem ................................................................................ 2
1.4 Research questions .......................................................................................... 3
1.5 Objectives ........................................................................................................ 4
1.5.1 General Objective .................................................................................... 4
1.5.2 Specific Objective .................................................................................... 4
1.6 Methodology ................................................................................................... 4
1.6.1 Literature Review..................................................................................... 4
1.6.2 Data Collection ........................................................................................ 4
1.6.3 Preprocessing ........................................................................................... 5
1.6.4 Scope and Limitation ............................................................................... 6
1.6.5 Organization of Thesis ............................................................................. 6
CHAPTER TWO ........................................................................................................... 7
2 Literature Review ................................................................................................... 7
2.1 Text Summarization ........................................................................................ 7
2.2 Types of Summarization ................................................................................. 7
2.3 Level of Text Summarization .......................................................................... 8
2.4 Approaches to Text Summarization ................................................................ 8
2.4.1 Sentence Scoring ...................................................................................... 9
2.4.2 Surface Level Approach .......................................................................... 9
2.4.3 Lexical Chain Methods .......................................................................... 10
2.4.4 Latent Semantic Analysis ...................................................................... 10
2.4.5 Machine Learning Approach ................................................................. 11
2.5 Basics of deep learning ................................................................................. 12
2.6 Deep learning techniques for text summarization ......................................... 13

vi
 2.7 Evaluation Metrics ........................................................................................ 14
2.8 Afaan Oromoo ............................................................................................... 15
2.8.1 Afaan Oromo writing system ................................................................. 15
2.8.2 Afaan Oromo Punctuation Marks .......................................................... 16
2.8.3 Morphology of Afan Oromo .................................................................. 17
2.9 Related Works ............................................................................................... 18
2.9.1 Non-Ethiopian Language Text Summarization ..................................... 18
2.9.2 Ethiopian Language Text Summarization ............................................. 22
CHAPTER THREE ..................................................................................................... 25
3 Methodology......................................................................................................... 25
3.1 Dataset collection and Preprocessing ............................................................ 25
3.1.1 Dataset Collection .................................................................................. 25
3.1.2 Preprocessing ......................................................................................... 25
3.1.3 Sentence segmentation ........................................................................... 25
3.1.4 Tokenization .......................................................................................... 26
3.1.5 Handling Sentence Length ..................................................................... 26
3.1.6 Stop word removing ............................................................................... 26
3.2 Performance Evaluation Metrics ................................................................... 27
3.3 Tools and Libraries........................................................................................ 27
CHAPTER FOUR ........................................................................................................ 29
4 Experimental Results and Discussion................................................................... 29
4.1 Model Selection............................................................................................. 29
4.1.1 Seq2Seq model with LSTM ................................................................... 29
4.1.2 Seq2Seq model with bi-directional LSTM ............................................ 30
4.1.3 Seq2Seq model with hybrid LSTM ....................................................... 32
4.2 System Architecture ...................................................................................... 34
4.3 Experimental settings .................................................................................... 35
4.4 System Generated Summary Evaluation ....................................................... 42
CHAPTER FIVE ......................................................................................................... 44
5 Conclusion and Recommendation ........................................................................ 44
5.1 Conclusion..................................................................................................... 44
5.2 Recommendation ........................................................................................... 45
References .................................................................................................................... 46
Appendixes .................................................................................................................. 53
Appendix A: Afaan Oromoo Contractions............................................................... 53

vii
Appendix B: Afaan Oromoo stop words .................................................................. 54
Appendix C: Sample output for the models ............................................................. 56

viii
Abbreviations

AE - Auto Encoder
ATS - Automatic Text Summarizer
Bi-LSTM - Bi-directional Long Short Term Memory
CNN - Convolutional Neural Network
DBN - Deep Belief Network
GRU- Gated Recurrent Unit
HRL- Hierarchical Reinforcement Learning
LSTM - Long Short Term Memory
NLP - Natural Language Processing
OOV- Out of Vocabulary
RNN - Recurrent Neural Network
Seq2Seq – Sequence-to-Sequence

ix
List of Figures

Figure 1: Deep learning techniques ............................................................................. 12

Figure 2: Taxonomy of summary evaluation metrics[47] ........................................... 15
Figure 3: The pointer generator mechanism[56] ......................................................... 19
Figure 4: An overview of encoding model [60]........................................................... 20
Figure 5: Abstractive summarization model using rich features [64]......................... 21
Figure 6: Sample collected data ................................................................................... 25
Figure 7: Sample dataset with and without stop words. .............................................. 26
Figure 8: Sample dataset after data preprocessing ....................................................... 27
Figure 9: LSTM model for Abstractive Afaan Oromoo text summarizer ................... 29
Figure 10: Encoder model of Seq2Seq model with LSTM.......................................... 30
Figure 11: Decoder model of Seq2Seq with LSTM model ......................................... 30
Figure 12: Bi-directional LSTM model for abstractive Afaan Oromoo text
summarization. ............................................................................................................. 31
Figure 13: Encoder model of Seq2Seq with bi-directional LSTM .............................. 31
Figure 14: Decoder model of Seq2Seq with bi-directional LSTM .............................. 32
Figure 15: Hybrid LSTM model for abstractive Afaan Oromoo text summarization. 33
Figure 16: Encoder model of Seq2Seq hybrid LSTM ................................................. 33
Figure 17: Decoder model of Seq2Seq with hybrid LSTM ......................................... 34
Figure 18: System Architecture ................................................................................... 34
Figure 19: Splitting data into training and test data ..................................................... 35
Figure 20: Training/Validation Accuracy/Loss for LSTM model ............................... 37
Figure 21: Training/Validation Accuracy/Loss for BI-LSTM model.......................... 38
Figure 22: Training Accuracy and validation accuracy for Hybrid-LSTM model ...... 40

x
Table 1: Experimental setting ...................................................................................... 35
Table 2: Training/Validation Accuracy/Loss for LSTM model .................................. 38
Table 3: Training Accuracy and validation accuracy for BI-LSTM model................. 39
Table 4: Training Accuracy and validation accuracy for Hybrid-LSTM model ......... 40
Table 5: Training and validation accuracy for LSTM, BI-LSTM, and Hybrid-LSTM41
Table 6: ROUGE-1 and ROUGE-2 Value ................................................................... 42

xi
 CHAPTER ONE

1 Introduction

1.1 Background
Nowadays valuable resources for extracting information from online documents,
articles and news has rapidly increased. Those documents are large and takes time to
read and get precise information. To retrieve useful information and save a time those
documents needs to be summarized. Summary is a short and brief overview of a
sentence or paragraphs. Therefore, we can list several reasons about importance of text
summarization: 1) we can retrieve significant information within short period of time
2) we can easily load significant information from long text. Due to the rapidly growth
of automatic text summarization method, it is needed to reviewed and summarized.

Machine learning, NLP, and data mining all include automatic text summarization. It
is becoming a popular research area while data grow and there is a demand to process
it more efficiently. The goal of ATS is to locate the essence of a text set, minimize its
size and covering key concepts and general meaning and avoiding duplication[1].
Automatic summarizing has gotten a lot of attention in the research community and in
the commercial world as a way to reduce information overload and enable users scan a
large number of documents[2].

Based on the number of document, summary context, type of summarizer, function and
genre we can divide text summarization into several categories. According to[3], text
summarization is broadly divided into two based on their approach: abstractive
summarization and extractive summarization. Extractive summarization selects the
most important (meaningful) sentences in an article without any modification.
Abstractive summarization needs real world knowledge. It paraphrases the most
important part of sentences in the article. As abstractive text summarization needs a real
knowledge and understanding of the document it is harder than extractive
summarization[4].

Depending on the input document we can classify it to a single document and multi
document text summarization. A single document text summarization produces
summary of a single input document while multi-document summarization is an

1
extraction of information from multiple texts written about the same topic. The resulting
summary report allows individual users, such as professional information consumers,
to quickly familiarize themselves with information contained in a large cluster of
documents[5].Text summarization is applicable in many enterprise areas like media
monitoring, newsletters, search marketing and SEO, internal document workflow,
financial research, social media marketing, question answering and bots, video
scripting, medical cases, book and literature, email overload, helping disable people
etc.

1.2 Motivation

The Oromo people are a Cushitic ethnic group and nation native to Ethiopia who speak
the Oromo language. They are the largest Ethnic group in Ethiopia and represent 45.5%
of Ethiopian population. The Oromo nation has a single common mother tongue, called
the Oromo language or Afan Oromo which is the third most-widely spoken language
in Africa as a mother tongue, next to Hausa and Arabic[6]. Afan Oromo is now the
official language of the Oromia regional state, a medium of instruction for elementary
and junior secondary schools, and a field of concentration in various Ethiopian
universities at the diploma, bachelor's, master's, and doctoral levels.

On the other hand, a variety of literary works, newspapers, journals, educational

materials, official credentials, and religious texts have been published in this
language[6]. The advancement of modern technology has increased the availability of
digital material on the internet, written by Afan Oromo. Due to this, information
overload may occur which leads the reader absorbing a large quantity of relevant
information from these information.

Generally, the facts listed above and lack of active research on the deep learning based
text summarization for Afaan Oromoo texts are a motivating factor for this work to
come up with modules that can alleviate or minimize these problem.

1.3 Statement of the problem

This days, there is large amount of information produced both in digital and printed
format available in local languages. As locally available data increases the demand for
tools that summarize relevant information will increase. In almost all languages,
including Afan Oromo language any domain of texts are written in detail which lead
2
readers to go through unwanted detail without being interested in it. Reports from
government offices, criminal investigation documents, online news articles, newspaper
text etc. produces large content of information textually.

Different researchers has been done different research works on automatic text
summarization both for foreign and local languages like[7,11] for English,[12,14] for
Amharic and[15,18] for Afaan Oromoo. Most of the research works has been done for
Afan Oromo on extractive type summarization.

The researcher[15], develop Afan Oromo text summarization based on sentence

selection methods using two features; sentence position and term frequency.
Researcher[18], also proposed a generic type of Afan Oromo text summarization. The
researcher has tried to fill the research gap found in the work of[15] by incorporating
the above missed features but, there is a sentence redundancy problem and lack of
coherence and cohesion in this work. Afaan Oromo text summarizer based on queries
has been proposed by another researcher. For sentence extraction from documents, the
researcher employed two methods. The first method is the vector space model (VSM),
which arranges both the sentences in a document and the query in vectors. The angle
between the document vector and the query vector is measured using cosine similarity
techniques. The second technique computes the relevance of a sentence by considering
the position of sentences in the document as additional features.

Now a day’s most of researchers on foreign language text summarization and some for
Amharic language are using the deep neural network techniques which is current state
of the art. The current state of art is also needed to summarize Afan Oromo texts.

1.4 Research questions

This study will try to explore and address the following research questions.
RQ 1: How to determine optimal hyper-parameter values for the deep learning models?

RQ 2: Which model contributes more and less to the performance of the

summarizer?

3
1.5 Objectives
1.5.1 General Objective

The general objective of this study is to explore the development of Abstractive Afaan
Oromoo Text Summarizer using Recurrent Neural Network.

1.5.2 Specific Objective

 Conduct literature review on automatic text summarization in extractive

methods, specifically abstractive approaches.
 Collecting and preparing dataset from different Afan Oromo online news
platform.
 Modeling a text summarizer to train and validate proposed model, to conduct
a comparative analysis based on various settings.
 Evaluate performance of the model.

1.6 Methodology

We followed the following methods to achieve the above objectives.

1.6.1 Literature Review

We have reviewed a number of connected publications and resources. This includes

conference and journal articles, white papers, and text summarizing systems created for
languages other than English. In addition, relevant discussions about the linguistic
nature of the language were held with Afan Oromo Linguistic experts.

1.6.2 Data Collection

As African languages are low-resource languages, working with Afan Oromo is quite
challenging. Data to evaluate our summarizer are collected and prepared from online
Afan Oromo news agencies like Oromia Broadcasting Service, Oromia Broadcasting
Network, Oromia Media Network, Oromia News Network, Fana Afan Oromo, Voice
of America Afan Oromo Service, Ethiopia News Agency, and other online texts in Afan
Oromo Language. Lexicon data like Afaan Oromo stop words and stems gathered from
different sources.

4
1.6.3 Preprocessing

In our case the preprocessing phase words are tokenizing, stop words are removed,
sentence length is handling, etc.

Sentence segmentation

The calculated value of each sentence is used to generate the summary, with the
calculated value of each word being determined by the value of computed each
word[15]. As a result, sentence segmentation is defined as the process of breaking down
text documents into sentences and determining sentence boundaries. Thus, text
documents are broken down into sentences in this system by detecting the boundary of
a phrase, which ends with a period symbol, a question mark, or an exclamatory mark.

Stop word removing

Stop-word removers delete words that do not contribute significantly to the general
meaning of the document but are the most prevalent terms found in all documents[15].
Stop words can be articles, conjunctions, prepositions, and pronouns, or they can be
words that appear in multiple phrases[19]. "Kun" means "this," "fi" means "and," and
"koo" means "mine or my" are some of the Afaan Oromo stop words that cannot modify
the meaning of the document if they are eliminated.

Tokenization

The technique of tokenization is the separation of the input document into individual
words. This is accomplished by recognizing the spaces (), commas (,), and special
symbols that appear between the words. The frequency of each word is computed and
saved in this procedure for later processing. Periods employed as a full stop and used
in abbreviated[15] are one of the issues with sentence boundaries. For example, k.k.f.
(Kan kana fakkaatan), and I.G.M.B (itti gaafatamaa mana barumsaa) etc. The
researcher used a list of abbreviations prepared by previous researchers to solve this
challenge.

Handling Sentence Length

When we summarize texts, we delete too short or too long sentences since they degrade
the summary's quality[19]. As a result, this component uses an array of tokenized texts
to regulate the challenge. So in order to identify too long or too short sentence, we put

5
a limit for length of headlines and text. If length of sentence is greater or less than the
limit it will be removed for the list of columns.

1.6.4 Scope and Limitation

This research focuses on single document summarization for Afaan Oromo texts. It
employs an abstractive summarization even though Afan Oromo is one of the low
resourced language. The scarce of the dataset affected the ability of the system to
predict the summary.

Application of Results

Automatic summarization have many applications and they can be applied to different
domains. Some of the domains are:

 News summarization.
 Newswire generation.
 Twitter, Blog, Email etc. summarization.
 Domain specific summarization etc.

1.6.5 Organization of Thesis

The entire thesis is divided into five Chapters including chapter one. Chapter two is
about literatures review and related works. The Third Chapter deals with the research
methodology. The fourth chapter focuses on experimental results and discussion.
Finally, Chapter Five is about drawing conclusion and recommendation which
summarizes the study.

6
 CHAPTER TWO

2 Literature Review
2.1 Text Summarization

Automatic Text Summarization (ATS) is a natural language processing discipline

(NLP). It became a discipline in 1958 following Luhn’s research into scientific text
summarization[7]. Automatic Text Summarization seeks to condense text documents
by producing a reduced version of the original material that contains the document's
major points, despite the fact that the process results in information loss[20].

Automatic text summarization widely used in the areas like, education, news, health
data etc. The main purpose of summarization is that it is used as a tool to help peoples
to easily get most useful information in the shortest time[21]. A software-generated
automatic summary is a paragraph that is coherent and contains a considerable quantity
of useful information from the source material. Its compression rate is less than a third
of the original document's length. Summarizing a document necessitates that the
summarizer (person or computer) pick, reformulate, and generate a coherent text
incorporating the most informative segments of the content[7].

A well-written abstract allows readers to quickly and properly assess the core content
of a document, determine its relevance to their interests, and so judge whether they need
to read the text in its entirety. Some of the reasons why automatic summarization
needed are: Summaries promote current awareness, facilitate literature searches, reduce
reading time, make the selection process easier, improves the effectiveness of indexing,
enable commercial abstract services to increase the number of texts they can process
etc.

2.2 Types of Summarization

Summaries are classified using a variety of characteristics, including function, type, and
document source. Based on their function, text summarization can be indicative or
informative[22]. The informative summary provides details on the subjects covered in
the source document. It has the appearance of a table of contents. The goal of an
informative summary is to replicate the content of the source material while also
perhaps explaining the reasoning. It might be a single document or a multi-document

7
summary depending on the number of papers to be summarized[23]. A single document
is a summary of a single document; a multi-document is a summary of a set of
documents that are not necessarily heterogeneous and are typically about the same
topic. Text summarization might be extractive or abstractive depending on their type.
Extractive is the assembling of fragments from the original document, whereas
abstractive is the reformulating of the summary. A summary created by rewriting and/or
paraphrasing the text is known as abstractive.

Text summarization can be generic summary or query guided summary based on the
context. A generic summary is a summary of a document that does not take into account
the information demands of the users. A query-based summary, on the other hand, is
one that is led by information demands or user inquiries.

2.3 Level of Text Summarization

Topic identification, interpretation, and creation are all stages of the automatic text
summarizing process[24]. Topic identification is a technique for identifying the most
essential topics in a text or for identifying central or important topics from a document
that has to be summarized[25]. Topic spotting or topic identification and tracking uses
techniques including positional importance, cue phrases, and word counts to
automatically sort a group of documents into classes, categories, or topics from a
specified set. When using the extraction method for text summarization, just the subject
identification stage is performed, and most systems nowadays use this stage[12].

The examination of themes to understand how they are related is achieved solely by
summarization systems based on abstraction, which are opposed to the extraction
summary because the extraction summary does not change the original text[12]. To
build an abstraction, the themes discovered in the first stage are connected, represented
in new terms or words not found in the original text, and conveyed using new concepts.
After sentence scoring is accomplished, whether in extractive or abstractive
summarization, summary generation is used to reconstruct the extracted and linked
material into a coherent, phrased new text[14].

2.4 Approaches to Text Summarization

A number of researchers have proposed two ways for automatic text summarization:
extractive and abstractive approaches[26]. Because of the intricacy of natural language,
8
the abstraction approach necessitates fusing and phrasing notions from original texts,
which is inherently difficult. It also aims to grasp the key points of a text and then
explain them clearly to the reader. Extraction summaries are created by extracting key
text sentences or parts from the original document using statistical analysis of
individual attributes or combinations of two or more elements.

2.4.1 Sentence Scoring

Sentence scoring method uses features like term, cue phrase frequency, sentence
position and sentence length mechanism to extract sentences from the original
document[18]. Weight of a sentence is determined by combining the scores of each
attribute. Even though combining these diverse scores is tough, different literatures
describe various techniques. The main point of sentence scoring is that coefficients
assign several weights to different scores, which are then aggregated[27]. However,
because there is no standard for distributing weight, simple combination, which is a
linear combination in which the restrictions are manually defined through
experimentation, is used.

Sentence score is calculated by:

𝑛
𝑆𝑒𝑛𝑡𝑒𝑛𝑐𝑒 𝑠𝑐𝑜𝑟𝑒 = ∑ P𝑗 𝐶𝑗 (1)
𝑗

In this equation Cj: is the jth constraint coefficient, Pj: is the jth constraint, n: is the
number of parameter.

2.4.2 Surface Level Approach

The surface level approaches for deciding which sections of the text are relevant are
used in most early works of automatic summarization [14]. This method favors the use
of surface features, which are then strategically blended together to build a salience
function. It is then utilized to extract the important data like, a thematic feature is the
presence of statistically significant terms based on term frequencies statistics. A
location feature is the position in text, paragraph, section depth, and specific sections.
A background feature is the occurrence of title or headings terms and a cue feature is
the presence of title or headings terms in the background. The words and phrases that
indicate whether a statement is important or not are referred to as words and phrases.

9
Luhn's[7] automatic text summary is the oldest known. Term frequencies are used to
determine sentence significance. That is, the most frequently occurring words in a
document typically indicate document content, making sentences including them more
significant. Words in a document are not all created equal; those with a lower frequency
and those on a stop word list are excluded, but the remaining terms in the text are
organized alphabetically, and related words are determined by comparing pairs of
words letter by letter. The frequency of words is directly related to their relevance to
the entire document, and after pre-processing, the frequency of words is counted and
the words are arranged in decreasing order. Sentences are scored based on the presence
of those significant terms. The top-scoring sentences are then placed in the summary.

2.4.3 Lexical Chain Methods

A lexical chain is a set of words or concepts that are semantically related[28]. It might
happen between two words or over a sequence of related words. Strong chains can be
built utilizing lexical relations derived from a lexical database, which aid in the
identification of key sentences[29]. In addition, the WorldNet database stores
relationships between words, and words that appear as nouns in WorldNet entries are
picked for text summarization. The lexical chain is one of the most effective approaches
for calculating lexical cohesiveness. To build lexical chains, there are three steps to
follow[27]. The first stage is to make a list of candidate words, then discover a suitable
chain for each one, and finally, if the second requirement is met, insert the applicant
word and build a new chain or update an existing one. The strongest chains among those
produced by the algorithm must be determined in order to obtain the summary for a
certain input text.

2.4.4 Latent Semantic Analysis

Latent Semantic Analysis is a statistical technique for extracting and representing words
from huge corpora of text based on their context-usage meaning[30]. LSA is used to
detect the correlation between the subjects and sentences, terms and sentences of a
series of documents in order to determine the document's effective summary and
dimension reduction[30]. LSA extracts words based on the meaning and similarity of
sentences using information from word usage in context[18]. The LSA model analyses
semantic similarity between textual information fragments for word use, and it is used

10
to increase the performance of information retrieval systems by assessing the
relationships between the pair documents and their terms. The problem of synonym, in
which a different word or term might convey the same semantic meaning, can also be
addressed via latent semantic analysis. It attempts to address the issue of literal term
mismatch by allowing retrieval to be based on concepts rather than terms.

In Latent Semantic Analysis, there are 3 steps. The first step is to generate an input
matrix: the input document is represented in a matrix form in order to do computations,
and a matrix representing the input text is created. The row and column of the matrix
represents word in sentence and documents sentence. The importance of words in
sentence is represented by their intersection[31].

Singular value decomposition is the other techniques for matrix reduction. It’s
technique of algebraic matrix factorization. The sentence selection procedure, which is
usually based on linguistic, mathematical, and statistical methodologies, is the final
step. As different algorithms employ different ways to identify relevant phrases from
the document for summarizing using this result, this step is conducted to construct
output summary using the SVD result[32].

2.4.5 Machine Learning Approach

Machine learning is a field of Artificial Intelligence (AI) that deals with creating
algorithms and strategies that allow machines to learn and make accurate predictions
based on previous observations and thinking. Learning/training and prediction are the
two fundamental tasks involved in machine learning. A set of examples known as
training data is provided to the system. The main goal is to generate an accurate model
from the training data automatically[33].

The prediction phase of machine learning involves mapping a set of inputs to the
associated target values. The basic goal of machine learning is to develop a model that
has strong prediction performance on test data, and good generalization on unknown
data[33].

11
2.5 Basics of deep learning

Deep learning is a form of machine learning that enables computers to learn from
experience and understand the word in terms of a hierarchy of concept. The computers
gather information from experience. Deep learning is referred to deep neural network
because most of deep learning methods use neural network architectures[34]. Deep
learning techniques contains Deep Belief Network[35], Auto encoder[36], Recurrent
Neural Network[37], Recursive Neural Network, Convolutional Neural Network[38],
and Direct Deep Reinforcement Learning[39]. In this paper we have deeply discussed
a deep learning techniques for text summarization.

Figure 1: Deep learning techniques

12
2.6 Deep learning techniques for text summarization

RNN Encoder-Decoder: This architecture is based on sequence-to-sequence model in

which the source document is an input and the summary of the document is an
output[40]. According to[41], recurrent neural network is a deep learning model and
process data in sequential order and a given states input depends on the output of
previous one. In RNN there are a hidden state which is the representation of previous
inputs. In seq2seq framework text summarization is composed of encoder and decoder.
After reading the input sequence from right to left, RNNs generate a sequence of hidden
states[39].

Bidirectional RNN: In Bi-directional RNN, the neurons in regular RNNs splits into
two opposite directions, called forward states and backward states. The two hidden
layers of forward and backward states connected to the same output[42]. Forward
RNNs generate a sequence of hidden states after reading the input sequence from right
to left.

Long Short Term Memory: LSTM consists of input gate, output gate, update gate and
forget gate[43]. Those gates are used for information flow for long period of time.

Input gate: an input gate is the previous state memory cell content. The output of
previous step is used as an input for the current one. Based on the signal from forget
gate, the input subjects to element wise multiplication. Finally, the result of
multiplication will added to the current memory cell.

Forget gate: Forget gate is a sigmoid activation function, which have an inputs like a
bias vector, current input embedding, and memory cell from previous hidden state. In
forget gate a sigmoid activation function in order to decide to remember or not
remember information from previous state. The previous state will remembered if the
sigmoid value is 1 and not remembered if the value is 0.

Memory gate: The main task of memory gate is controlling the impacts of remembered
information on the new information. It is composed of two neural network. The first
one is the same with forget gate structurally but differ in bias. The second network
generate new information using tanh activation by adding the output of element wise
multiplied previous hidden state and adding the result to the old memory.

13
Output gate: The new memory, bias vector and previous hidden states are an input for
the output gate. The amount of the current new memory which is forwarded to the next
long short term memory is controlled by the output gate.

Gated recurrent unit: Long short term memory (LSTM) simplified to gated recurrent
unit (GRU) with two gates. Those gates are reset gate and update gate. GRU does not
have explicit memory[44].

Attention Mechanism: A main problem of encoder-decoder architecture is its failure

to consider all elements of long input[43]. Attention mechanism was introduced to
remember an input that have impact on the summary. Attention mechanism calculates
the of output and an input in order to identify which input word must got attention with
respect to output word.

Beam Search: In beam search a number of beam width or beam size hypothesis is
considered[40]. As the beam size increases the performance of decoder becomes better
at finding shorter sentences.

Word-Embedding: Word-embedding representation of word by word distributional

vector that represents the syntax and semantic features of words[45]. To handle
semantic similarities between words, cosine similarities will calculated by converting
them to vectors. The word embedding form of the words are used as an input to neural
network models. Word2Vec, Glove, Fast Text and BERT are type of word embedding
models.

2.7 Evaluation Metrics

Evaluation is essential for assessing the accomplishments of many ways in a study, as

well as the important indicator of the results' applicability and the method of measuring
one thing's performance[46]. Human annotators are frequently used to judge the quality
of text. Each summary is given a numerical value from a predetermined scale. The most
typical way for measuring summary quality is to evaluate the intrinsic content, which
is frequently done by comparing it to an ideal summary. It determines the number of
ideal sentences in the system generated summary. Instead of comparing the full
sentence, content-based measurements compare the individual words in a sentence.
Their advantage is that they can compare both human and machine extracts with newly
written sentences in human abstracts. Task-based approaches are another important

14
category. They assess how well they succeed while using summaries for a certain
job[47].

Figure 2: Taxonomy of summary evaluation metrics[47]

2.8 Afaan Oromoo

Cushitic is a sub-family of Afro-Asiatic which is Ethiopia's largest family. Among

Cushitic family language, Afaan Oromoo has the largest speaker. Afan Oromo language
is a prominent African language and it is widely spoken throughout Ethiopia, as well
as some regions of neighboring nations[18, 48]. The Oromos, who speak this language,
accounts maximum percent of the population[48]. A handful of individuals of other
ethnicities who come into touch with Oromos speak a second language, such as the
Nilo-Saharan-speaking Kwama and Omotic-speaking Bambassi in northwestern
Oromia[18].

This language is now the official language of Oromia state [18, 41], as well as the
Oromia’s special Zone of neighboring Regions [39]. Furthermore, it is used as a
medium of instruction from basic school through higher education, administration,
politics, and social interaction. Many literatures are published by this language [42].

2.8.1 Afaan Oromo writing system

Afaan Oromo is a phonetic language, meaning it is spoken in the same manner as it is

written[48]. Since 1991, the Latin-based alphabet "Qubee" has been accepted and

15
adopted as the official script of Afaan Oromo[15]. This language's writing system is
based on the Latin writing system, and it shares many of the elements of the Latin
writing system with minor modifications[16]. All letters in the English alphabet are
present in Afaan Oromo, although they are written differently, and there are no skipped
or unpronounced sounds/alphabets, unlike in English or other Latin-based languages
[43]. When a consonant is doubled in a word, the sounds are more stressed, and when
the vowels are doubled, the sounds are stretched or prolonged[48].

This language has 33 characters[15], 26 of which are identical to English letters, and 7
of which are known as "Qubee Dachaa," which are created by combining two consonant
letters[16]. The letters are CH ch, DH dh, SH sh, NY ny, TS ts, PH ph, and ZY zy. The
vowels in Qubee Afaan Orormo are the same as those in English, and they are written
with the same letters. Among them are the letters a, e, I, o, and u. In the English
language, there are two sorts of vowels: short and long vowels, which have different
meanings. Because there are no native words generated from these characters in Afaan
Oromo, P, V, and Z are not included in the basic alphabet[16]. They are, however, used
to signify borrowed terms from other languages in this language. For example,
televiijinii means "TV," poolisii means "Police," while Zeekkara means "Opera." The
alphabets of Afaan Oromo are listed below in both upper and lower case.

2.8.2 Afaan Oromo Punctuation Marks

To make meaning apparent and reading easier, Afaan Oromo punctuation marks are
used in text punctuation patterns, much as they are in other languages that utilize the
latin writing system[48]. The following are a discussion of those punctuations. Tuqaa
(Full stop) punctuation mark is used in abbreviations and at the end of sentences (.).
Mallattoo gaaffii (question mark) (?) is the punctuation mark used at the conclusion of
question sentences or interrogatives. Raajeffannoo (exclamation mark) (!) is a

16
punctuation mark used at the end of exclamatory or command sentences. Qoodduu
(comma) is another punctuation mark that is used to divide parts . These punctuation
marks is used in the same way in both Afaan Oromo and English, with the exception
that in English, the apostrophe is used to express possession. When two vowels came
together, for example, ta'i "sit down" and bu'i "fall down" were employed.

2.8.3 Morphology of Afan Oromo

Morphology is the study of the meaning of discrete units or linguistic morphemes, word
formation, and internal structure of words[18]. In morphology, morphemes are the basic
building blocks, whereas words are the basic building blocks of a language. Words are
a self-contained unit of meaning, but morphemes are the smallest unit of grammatical
analysis and can stand alone or not[48].

The branches of morphology are inflectional and derivational[18]. Inflectional

morphology is the study of processes, not the creation of new words. It is the process
of adding meaning to existing words rather than the creation of new words. Inflectional
morphology is the addition of a distinct form of the same word to a word.

The technique of generating a new word from a root word by adding a bound morpheme
to a stem and changing the word classes is known as derivational morphology[15]. By
modifying the class of the term, derivational morphology alters the lexical meaning of
the stem word in comparison to the other derived words. In this language, suffixes like
–ummaa are utilized for word creation in derivational morphology. In Afaan Oromo,
words can be constructed in a variety of ways[18].

Afan oromo nouns: This language concatenates and affixes person, gender, number,
and possession markers to a stem or singular noun form, and the same is true for
pronouns and determinants[16].

Gender is divided into two categories in Afaan Oromo[18], masculine and feminine, as
in other Afro Asiatic languages. For example, Korbeessa (male goat) is masculine
where Goromti (female goat) is feminine and obboleessa (brother) is masculine
where obboleettti (sister) is feminine.

Suffixes such as -aan, -olii, -olee, oota are used to transform a noun to plural form,
however the use of suffixes varies by dialect[18]. For example Barataa (student) is

17
singular where Barattoota (Students) is plural. Farda (horse) is singular where
Fardeen (horses) is singular.

2.9 Related Works

In this chapter, we covered studies that used several strategies to focus on a single
document in the news domain. We have reviewed some non-Ethiopian works, and then
move on to local works in the field of text summary. A summary is also provided at the
conclusion.

2.9.1 Non-Ethiopian Language Text Summarization

An encoder and a decoder make up a sequence-to-sequence model. The encoder is in

charge of reading the tokens as an input. To generate the encoder hidden states, the
input tokens' vectors, known as word embedding, are read and fed into a bi-directional
LSTM. The decoder receives the final encoder state and feeds it, along with its input,
to a unidirectional LSTM to generate the hidden state of the decoder for the given time
step. The attention distribution over the input tokens is estimated to assist the decoder
in producing the correct outputs. The context vector is then created, which is the
weighted average of attention weights, and the decoder utilizes this weight to generate
the final distribution across all vocabulary terms. Finally, the word with the highest
probability is chosen as the time step's output.

One of the earliest attempts to employ neural networks to beat the performance of
existing standard summary models was a neural attention-based abstractive
summarization model[49] for headline generation inspired by[50] on short texts (one
or two phrases). Authors[51] improved the findings on short text summary creation by
computing the conditional probability of following words using a recurrent decoder,
which is an extension of[49].

Authors[52], proposed an attentional encoder-decoder recurrent neural network for

abstractive summarization. They also provide a new dataset that includes multi-
sentence articles and summaries of on-line CNN and Daily Mail stories that were
initially created for question-answering systems[51]. Their findings also served as
benchmarks for future research.

18
A fundamental flaw with sequence-to-sequence models is that the grammar and
semantic coherence of the decoder output cannot be guaranteed, and the generated
sequence may be repetitious and unreadable in practice. CopyNet[53], which learns to
replicate words and phrases from the input sequence, is one recent attempt to overcome
this issue. Another new approach is the pointer generator method, which acts as a switch
between abstractive and extractive summarization characteristics. Attention is used as
a pointer in the pointer network[54] to pick a member of the input sequence for the
output. This approach is also used by[52] to generate rare out-of-vocabulary words and
named entities. Author[55] Uses the same method for OOVs on summarization and
machine translation tasks, improving performance across various datasets.

However, as shown in Figure 3, [56] teach the model not only to be activated for OOVs
but also to choose freely between pointing to a word in the input sequence or producing
a novel term from vocabulary. The decoder must decide whether it is better to point to
a token in the input sequence or generate a new word from the specified vocabulary at
each time step.

Figure 3: The pointer generator mechanism[56]

Exposure bias is caused by cross-entropy optimization employing ground truth terms.
This problem arises because the model gets access to the actual sequence words during
training, but there is no such supervision during testing, so the model must rely on its
own predictions. Authors [57] and [58] trained sequence generation models using
discrete metrics such as ROUGE and BLEU in a reinforced approach to overcome
exposure bias and optimize at the sequence level rather than the word level. Paulus et

19
al.[59], in their most recent summarization study, use REINFORCE to abstractive
summarization and uses similar techniques.

Kaichun Yao et colleagues[60] use a dual encoding technique to simulate abstractive

text summarization (DEATS) as shown in figure 4. This technique differs from earlier
methods in that it has two encoders: a primary and secondary encoder. The DEATS
model comprises of a primary and secondary encoder, as well as a decoder. The
researcher's main contributions include proposing a dual encoding mechanism (DEM)
that extends the traditional sequence-to-sequence model by adding an additional
encoder to fully utilize document text information, an enhanced repetition avoidance
mechanism (RAM), and finally, experiments on CNN/Daily Mail and DUC2004
datasets.

Figure 4: An overview of encoding model [60]

On Atari games, hierarchical reinforcement learning (HRL) has shown significant
improvements[61]. In most HRL settings, there are two policies: a high-level policy
that operates at a higher resolution to set sub-goals, and a low-level policy that performs
primitive actions at a lower resolution to achieve the high-level policy's sub-goal. To
ensure that the low-level policy achieves the sub-goal, an internal critic is frequently
used. Some attempts have recently been made to apply HRL to practical applications.
[62] proposes a composite task-completion conversation agent to manage the
complexities of dialogue systems, and [63] develops an HRL framework for video
captioning that provides semantic descriptions phrase by phrase.

Viet Nguyen et al[64], proposes an abstractive summarization model using rich features
to overcome the problem of sequence to sequence which is limited in summarizing long

20
text. They have proposed a new seq2seq network by using sentence position and term
frequency as features as shown in figure 5.

Figure 5: Abstractive summarization model using rich features [64]

In this model sentence position is used to enhance the weight of the first sentence
without cutting off the input text, thus the sequence of input words expressed as x = (x11,
x21, . . . , xJk), xjk means the word 𝑥𝑗𝑘 means the word 𝑥𝑗 in the Kth sequence. The term
frequency (TF) is used for model focus on the important words and the frequency of
word is calculated as:

𝑓(𝑥𝑗, 𝑥)
𝑡𝑓(𝑥𝑗, 𝑥) = (2)
max{𝑓(𝑥𝑖, 𝑥)|𝑖 = 1 → 𝐽}

Ziqiang Cao et al.[65], proposed a summarization system called ArrSum. Their main
contribution in automatic text summarization is that they apply the attention mechanism
that simulate human attentive reading behavior for query-focused summarization and
they have proposed a joint neural network model to learn query relevance ranking and
sentence saliency ranking simultaneously.

Romian Paulus et al.[66], introduces a deep reinforced model for abstractive

summarization. They have introduced a neural network model with an intra-attention.
This attends over the input to generate output separately and continuously. They have
also implemented a new training method that combines reinforcement learning and
standard supervised word prediction.

21
Jianmin Zhang et al[67], proposed a neural network approach to abstractive multi-
document summarization. This approach is proposed to extent the neural abstractive
model trained on large scale single document summarization data to multi-document
summarization task.

Evangelia Kantazola[68], proposed a sentence clustering-based extractive

summarization technique for Greek news stories. The main goal of their study is to
assess the impact of three distinct forms of text representations on the summarization
task: Word2Vec embedding, TF-IDF, and LASER embedding. They construct three
different versions of the initial summarizer using these techniques. The researcher
evaluates his work both automatically and manually. System A, which uses Average
Word2Vec vectors to produce sentence embedding, outperforms the other two systems
by providing higher ROUGE-L F-scores, according to the automatic ROUGE results.
Even when using LASER embedding, System C fails to outperform Word2Vec
embedding, resulting in a weak sentence representation.

2.9.2 Ethiopian Language Text Summarization

To construct an Amharic text summarizer, Authors[12] proposed a topic modeling

approach based on probabilistic latent semantic analysis. To investigate the utilization
of the word by idea matrix, the researcher suggests six approaches. There are two
common steps in all of the algorithms. The document's keywords are chosen in the first
stage using a term by concept matrix. The sentences that best contain the keywords are
chosen for inclusion in the summary in the second step. Furthermore, the algorithms
always choose the document's initial sentence for inclusion in the summary. Finally,
they tested the proposed algorithms for precision and recall for summaries with
extraction rates of 20%, 25%, and 30%. The best results obtained were 0.45511 at 20%,
0.48499 at 25%, and 0.52012 at 30%. According to the study, the results gained are
encouraging, and the systems outperformed earlier summarizing efforts.

Kifle et al[14] proposed the graph-based Automatic single-document summary, and

demonstrated how it can be used to successfully generate high-quality extracts from
Amharic texts. The author added two-sentence centrality metrics to the two most
popular graph-based link analysis methods, PageRank and HITS: When discounted
cumulative sum and HITS were combined, the results showed that extractive

22
summaries of higher quality could be obtained. The authors obtained 0.632 F-measure
at a compression rate of 20% and 0.697 F-measure at a compression rate of 30%.

Kamil[69], proposed Amharic single document summarizer system. Natural language

processing techniques and statistical tools were applied by the researcher. To assign
weights to sentences, the author employed surface level statistical features. The
summary was created by extracting the sentences with the highest scores. Title words,
cue phrases, the first sentence of the document (header), words in a header, the first
sentence of a paragraph, paragraph end sentences, and high frequency words are the
seven features employed. Each features has a weight associated with it, which was
determined by training with manual summaries of four news articles, and the weights
are added linearly to generate a sentence's overall score. Five news articles were used
to test the algorithm, with a 38.5 percent extraction rate. Precision is 0.704, recall is
0.58, and F-measure is 0.636 for this system. The title words and keywords are the most
prominent aspects of the tests. The study concluded that the frequency ratings of the
word were skewed due to the lack of a stemmer in the pre-processing of the input
text/news items. The usage of exhaustive stop-words, according to the researcher, could
increase system performance.

Girma et al[48] proposed three approaches for summarizing Afan Oromo news text.
The methods are: Method 1, which does not employ the Afan Oromo stemmer or other
lexicons and instead relies on term frequency and position approaches (synonyms and
abbreviations), Method 2 is a summarizer that uses a combination of term frequency
and position approaches, as well as an Afan Oromo stemmer and lexicons tailored to
various languages (synonyms and abbreviations) and Method 3 features an enhanced
position technique, term frequency, stemmer, and language-specific lexicons
(synonyms and abbreviations). The performance of the summarizers was evaluated
using both subjective and objective methods. The three summarizers, Method 1,
Method 2, and Method 3, have f-measure values of 34 percent, 47 percent, and 81
percent, respectively, according to the results of objective evaluation. Method 3
outscored the other two summarizers (Method 1 and Method 2) by 47 and 34 percent,
respectively.

Fisseha[17] proposed an extractive Afaan Oromo automatic news text summarizer that
combines features such as sentence position, sentence length processor, cue phrases,

23
keyword frequency, and numbers and events that appear in the sentence, such as time,
date, and month. The system was evaluated on the basis of seven experimental
scenarios, which were subjective and objective tests. The subjective evaluation focused
on assessing the structure of the abstract, such as the completeness and non-redundancy
of the references, coherence, and formatting. For objective evaluation, metrics like as
precision, recall, and F-measure are used. Subjective examination yielded an 88 percent
informativeness rating, a 75 % referential integrity and non-redundancy rating, and a
68 percent coherence rating. The system gave 87.47% f-measure and outperformed the
previous work by 26.95% due to the increased features, different methodologies, and
experiment done to this study.

Lamessa[70] used a word embedding to provide a generic automatic text summarizer

for Afaan Oromo text. To develop the summarizer, language-specific lexicons such as
stop words and stemmers are utilized; graph-based PageRank is used to select the
summary of worthy sentences from the document; and cosine similarity is used to
measure the similarities between sentences. The system has been evaluated subjectively
and objectively based on three different experimental scenarios. The subjective
evaluation focuses on the summary's structure, such as its informativeness, coherence,
referential clarity, non-redundancy, and grammar. We used precision, recall, and F-
measure measures in the objective evaluation. Subjective evaluation yielded a score of
83.33 percent informativeness, 78.8% referential integrity and grammar, and 76.66
percent structure and coherence. Using the data acquired, the author were able to attain
0.527 precision, 0.422 recall, and 0.468 F-measure. However, when comparing the
summarizer's overall performance to earlier studies using the same data, the summarizer
outperformed by 0.648 precision, 0.626 recall, and 0.058 F measure.

Summary

From the above discussion, we can see that many works have been done by using
various techniques. Majority of non-Ethiopian languages summarization systems are
developed using deep learning method. But Afan Oromo language summarizations
systems are not deep learning based. In this work we focus on summarizing numerous
Afaan Oromo texts using various deep learning algorithms, and we conduct a
comparison analysis based on various settings.

24
 CHAPTER THREE

3 Methodology

3.1 Dataset collection and Preprocessing

3.1.1 Dataset Collection

Datasets are collected from afan Oromo online news sources manually. We have used
the headlines of the news as a reference summary and the body as text. Data sets like
daily mail, amazon fine tune etc. are used the headlines as the reference summary and
the body as text. The collected data was preprocessed as discussed in section 4.1.2.

Figure 6: Sample collected data

3.1.2 Preprocessing

Real-world texts are imperfect, and sending them to the model directly will result in
inaccuracies. As a result, we clean up all of our texts and turn them into a format that
may be used for prediction tasks.

In our case the preprocessing phase words are tokenized, stop words are removed,
stemming words into verbs and suffixes is done, and sentence length is handled, etc.

3.1.3 Sentence segmentation

The calculated value of each sentence is used to generate the summary, with the
calculated value of each word being determined by the value of computed each
word[27]. As a result, sentence segmentation is defined as the process of breaking down
text documents into sentences and determining sentence boundaries[15]. Thus, text
documents are broken down into sentences in this system by detecting the boundary of
a phrase, which ends with a period symbol, a question mark, or an exclamatory mark.

25
3.1.4 Tokenization

The technique of tokenization is the separation of the input document into individual
words. This is accomplished by recognizing the spaces (), commas (,), and special
symbols that appear between the words. The frequency of each word is computed and
saved in this procedure for later processing. Periods employed as a full stop and used
in abbreviated [18] are one of the issues with sentence boundaries. For example, k.k.f.
(Kan kana fakkaatan), and I.G.M.B (itti gaafatamaa mana barumsaa) etc. The
researcher used a list of abbreviations prepared by previous researchers to solve this
challenge.

3.1.5 Handling Sentence Length

When we summarize texts, we delete too short or too long sentences since they degrade
the summary's quality[19]. As a result, this component uses an array of tokenized texts
to regulate the challenge. So in order to identify too long or too short sentence, we put
a limit for length of headlines and text. If length of sentence is greater or less than the
limit it will be removed for the list of columns.

3.1.6 Stop word removing

Stop-word removers delete words that do not contribute significantly to the general
meaning of the document but are the most prevalent terms found in all documents[15].
Stop words can be articles, conjunctions, prepositions, and pronouns, or they can be
words that appear in multiple phrases[19]. "Kun" means "this," "fi" means "and," and
"koo" means "mine or my" are some of the Afaan Oromo stop words that cannot modify
the meaning of the document if they are eliminated.

Figure 7: Sample dataset with and without stop words.

26
Figure 8: Sample dataset after data preprocessing

3.2 Performance Evaluation Metrics

To evaluate our system generated summary we have used Recall-oriented abstract

evaluation (ROUGE). the ROUGE score was first proposed in[71] and later became a
standard metric for evaluating abstract text abstraction models. The number of
overlapping units (i.e., n-grams, word sequences, and word pairs) between its
computerized machine-generated abstracts and the gold standard (human-written)
abstracts was used to determine the quality of the abstracts[71]. ROUGE-1 (unigram),
ROUGE-2 (bigram), and ROUGE-L (longest common subsequence) are the most
extensively utilized ROUGE measures for single-document abstractive summarization.
We have used pyrouge package to evaluate our models. This package provides
precision, recall and f-measure for the metrics.

𝑛𝑢𝑚𝑏𝑒𝑟_𝑜𝑓_𝑜𝑣𝑒𝑟𝑙𝑎𝑝𝑝𝑖𝑛𝑔_𝑤𝑜𝑟𝑑𝑠
Recall = (3)
𝑡𝑜𝑡𝑎𝑙 𝑤𝑜𝑟𝑑𝑠 𝑖𝑛 𝑟𝑒𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑠𝑢𝑚𝑚𝑎𝑟𝑦

𝑛𝑢𝑚𝑏𝑒𝑟_𝑜𝑓_𝑜𝑣𝑒𝑟𝑙𝑎𝑝𝑝𝑖𝑛𝑔_𝑤𝑜𝑟𝑑𝑠
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 = (4)
𝑡𝑜𝑡𝑎𝑙_𝑤𝑜𝑟𝑑𝑠_𝑖𝑛_𝑠𝑦𝑠𝑡𝑒𝑚_𝑠𝑢𝑚𝑚𝑎𝑟𝑦

2 ∗ 𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 ∗ 𝑟𝑒𝑐𝑎𝑙𝑙
f − measure = (5)
𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 + 𝑟𝑒𝑐𝑎𝑙𝑙

3.3 Tools and Libraries

Kaggle Notebook: Users can use Kaggle to search and publish data sets, study and
construct models in a web-based data-science environment. The kaggle notebook looks
like the well-known Jupyter Notebook, with the added benefit of GPU and TPU access.
Kaggle have a huge libraries for deep learning and other fields. The supportive libraries

27
used are numpy for mathematical functions, pandas for data analysis and manipulation,
seaborn for data visualization, matplotlib and Tensorflow.

The research is implemented using computer with specifications:

 Laptop, Lenovo L540

 Windows 10pro operating system
 Processor, Intel(R) ore(TM) i5-4210M CPU @ 2.60 GHz
 System type, 64-bit operating system, x64-based processor
 Installed RAM, 4.00 GB(3.73 GB usable)
 Storage, 1TB

28
 CHAPTER FOUR

4 Experimental Results and Discussion

4.1 Model Selection

In this study we have used three different deep learning models to perform comparison
analysis for Afaan Oromoo abstractive text summarization. The models are:

 Seq2Seq model with LSTM

 Seq2Seq model with bi-directional LSTM and
 Seq2Seq model with hybrid LSTM

4.1.1 Seq2Seq model with LSTM

In this model our encoder and decoder model have just LSTM. Our model summary
Long Short Term Memory (LSTM) is described in figure 8, figure 9 and figure 10.

Figure 9: LSTM model for Abstractive Afaan Oromoo text summarizer

29
Figure 10: Encoder model of Seq2Seq model with LSTM

Figure 11: Decoder model of Seq2Seq with LSTM model

4.1.2 Seq2Seq model with bi-directional LSTM

In this model both encoder and decoder have a bi-directional LSTM. Model summary
for Bi-directional LSTM, encoder and decoder are shown under figure 11, figure 12
and figure 13.

30
Figure 12: Bi-directional LSTM model for abstractive Afaan Oromoo text
summarization.

Figure 13: Encoder model of Seq2Seq with bi-directional LSTM

31
Figure 14: Decoder model of Seq2Seq with bi-directional LSTM

4.1.3 Seq2Seq model with hybrid LSTM

In this model encoder has bi-directional LSTM and decoder has LSTM. Model
summary for hybrid LSTM, encoder and decoder are shown below in figure 14, figure
15 and figure 16.

32
Figure 15: Hybrid LSTM model for abstractive Afaan Oromoo text summarization.

Figure 16: Encoder model of Seq2Seq hybrid LSTM

33
Figure 17: Decoder model of Seq2Seq with hybrid LSTM

4.2 System Architecture

Architecture of the system is shown in figure 17.

Figure 18: System Architecture

34
4.3 Experimental settings

Table 1: Experimental setting

Models Hyper parameter Value

LSTM, Activation Softmax
Bi-LSTM, Learning_rate 0.001
Hybrid-LSTM 0.002
Optimizer RMSprop
Batch size 32
64
Dropout 0.4

4.3 Training and Validating Models

Our deep learning models needs to be trained by feeding the prepared Afaan Oromoo
dataset in order to evaluate the performance of predicting the summary of the given
text. For this purpose we have collected more than 11000 pairs of text and reference
summary. After data preprocessing the number of our data decreased to 7400 because
we limit the size of our text to 53 and reference summary to 17 for the performance of
our model. 85 percent of this data is used for training the model and 15 percent is used
for testing randomly as shown in figure 19.

Figure 19: Splitting data into training and test data

Using this dataset we have trained the three models (Seq2Seq model with LSTM,
Seq2Seq with bi-directional LSTM and Seq2Seq model with hybrid LSTM) and
evaluate using evaluation set using and we score different accuracy values.

35
We used different scenarios to perform experiments.

a) Checking with learning rate 0.001, batch size 64, epoch 60

b) Checking with learning rate 0.001, batch size 64, epoch 100
c) Checking with learning rate 0.001, batch size 64, epoch 140
d) Checking with learning rate 0.002, batch size 64, epoch 100
e) Checking with learning rate 0.002, batch size 32, epoch 100

Model 1: With our first model as shown in table 2, Seq2Seq model with LSTM we
scored 80.39% of training accuracy and 77.33 % of validation accuracy at epoch 60
with learning rate 0.001. We have trained the model by maximizing the epoch to 100
and 140 and we scored 91.2 % of training accuracy and 90.08% of validation accuracy
and 92.13% of training accuracy and 91.01% of validation accuracy respectively. As
we observe from the results, the accuracy value between epoch 100 and epoch 140 is
very small. This indicates that we do not need to increase the epoch number to get more
values.

Finally, we have replaced the hyper parameters; learning rate to 0.002 and batch
size=64 and 32. We have scored 93.13% training accuracy and 91.94% of validation
accuracy 95.48% training accuracy and 94.84% and validation accuracy of and
respectively.

36
Figure 20: Training/Validation Accuracy/Loss for LSTM model

37
Table 2: Training/Validation Accuracy/Loss for LSTM model

Model Epoch Batch Optimizer Learning Training Validation

Size rate Accuracy Accuracy
60 64 RMSprop 0.001 80.89% 77.33%
LSTM 100 64 RMSprop 0.001 91.2% 90.08%
140 64 RMSprop 0.001 92.13% 91.01%
100 64 RMSprop 0.002 93.13% 91.94%
100 32 RMSprop 0.002 95.48% 94.84%
Model 2: With the second model, Seq2Seq model with bi-directional LSTM we scored
98.78% of training accuracy and 98.33 % of validation accuracy at epoch 60. We have
trained the model by maximizing the epoch to 100 and we score 99.8% of training
accuracy and 99.01% of validation accuracy at epoch 100. The maximum value
99.95% training accuracy and 99.97% validation accuracy was scored at epoch 100,
Learning rate 0.002 and batch size 32 as shown in table 3.

Figure 21: Training/Validation Accuracy/Loss for BI-LSTM model

38
Table 3: Training Accuracy and validation accuracy for BI-LSTM model

Model Epoch Batch Optimizer Learning Training Validation

Size rate Accuracy Accuracy
60 64 RMSprop 0.001 98.78% 98.33%
Bi- 100 64 RMSprop 0.001 99.8% 99.01%
LSTM 100 64 RMSprop 0.002 99.84% 99.88%
100 32 RMSprop 0.002 99.95% 99.97%

Model 3: With the third model, Seq2Seq model with hybrid-LSTM we scored 57.87%
of training accuracy and 57.40% of validation accuracy at epoch 60. We have trained
the model by maximizing the epoch to 100 and we score 59.24 % of training accuracy
and 58.53 % of validation accuracy. From this result we observed that no need to
increase the epoch to get more accuracy. Rather, we have replaced the learning rate by
0.002 and performed 90.34% training accuracy and 89.01% validation accuracy.
Finally, we have replaced the batch size by 32 and learning rate to 0.002 and scored
94.37% and 93.92% of training accuracy and validation accuracy respectively as shown
in table 4.

39
Figure 22: Training Accuracy and validation accuracy for Hybrid-LSTM model
Table 4: Training Accuracy and validation accuracy for Hybrid-LSTM model

Model Epoch Batch Optimizer Learning Training Validation

Size rate Accuracy Accuracy
60 64 RMSprop 0.001 57.87% 57.40%
Hybrid- 100 64 RMSprop 0.001 59.24% 58.53%
LSTM 100 64 RMSprop 0.002 90.34% 89.01%
100 32 RMSprop 0.002 94.37% 93.92%
4.1.2. Discussion

Under section 4.1 we have trained the models at different epoch and batch size in order
to achieve high accuracy values for evaluation set. The accuracy values for all models
are reviewed under table 5 below.

40
Table 5: Training and validation accuracy for LSTM, BI-LSTM, and Hybrid-LSTM

Model Epoch Batch Optimizer Learning Training- Validation-

Size rate Accuracy Accuracy
LSTM 60 64 RMSprop 0.001 80.89% 77.33%
Bi- 60 64 RMSprop 0.001 98.78% 98.33%
LSTM
Hybrid- 60 64 RMSprop 0.001 57.87% 57.40%
LSTM
LSTM 100 64 RMSprop 0.001 91.2% 90.08%
Bi- 100 64 RMSprop 0.001 99.8% 99.01%
LSTM
Hybrid- 100 64 RMSprop 0.001 59.24% 58.53%
LSTM
LSTM 100 64 RMSprop 0.002 93.13% 91.94%
Bi- 100 64 RMSprop 0.002 99.84% 99.88%
LSTM
Hybrid- 100 64 RMSprop 0.002 90.34% 89.01%
LSTM
LSTM 100 32 RMSprop 0.002 95.48% 94.84%
Bi- 100 32 RMSprop 0.002 99.95% 99.97%
LSTM
Hybrid- 100 32 RMSprop 0.002 94.37% 93.92%
LSTM

From the experimental results:

 Seq2Seq model with Bi-LSTM performs well when compared with the other
two models. It out performs Seq2Seq model with LSTM model by 4.47% and
Seq2Seq model with Hybrid-LSTM by 5.58 %.
 Hybrid-LSTM performance is poor at scenario a, and scenario b when compared
with LSTM model and Hybrid LSTM, because its architecture is complex.

41
4.4 System Generated Summary Evaluation

The ROUGE-N (ROUGE-1 and ROUGE-2) measure was performed for our system
predicted summary as shown in table 6. The overlap of unigrams (per word) between
the system and reference summaries is referred to as ROUGE-1 and the overlap of
bigrams between the system and reference summaries is referred to as ROUGE-2.
LSTM, Bi-LSTM and Hybrid-LSTM model generated summaries are evaluated for
ROUGE-1 and ROUGE-2.

For LSTM model we scored 32.15% recall, 32.85% precision and 32.49% f-measure
and 6.5% recall, 7.7% precision and 7.04% f-measure for ROUGE-1 and ROUGE-2
measures respectively. The second model, Bi-LSTM scores 41.77% recall, 31.33%
precision and 35.8% f-measure for ROUGE-1 measure and 16.33% recall, 13.1%
precision and 14.18% f-measure for ROUGE-2 measures. The third model, hybrid-
LSTM scores 21.69% recall, 22.27% precision and 21.31% for ROUGE-1 measure and
6.39% recall, 7.007% precision and 6.68% f-measure for ROUGE-2 measure.

Table 6: ROUGE-1 and ROUGE-2 Value

ROUGE-1 ROUGE-2
r P f r p f
Model
LSTM 32.15% 32.85% 32.49% 6.5% 7.7% 7.04%
Bi-LSTM 41.77% 31.33% 35.8% 16.33% 13.1% 14.18%
Hybrid- 21.69% 22.27% 21.31% 6.39% 7.007% 6.68%
LSTM

So, when we see the f-measures of all three models, LSTM model scores 32.49%,
7.04%, Bi-LSTM model scores 35.8%, 14.18% and Hybrid-LSTM scores 21.31%,
6.68% for ROUGE-1 and ROUGE-2 respectively. This result indicates that Bi-LSTM
model performed well when compared with LSTM model and Hybrid-LSTM model.

We formulated a research question in Chapter one section 1.4. Our study answered the
formulated research questions as follows:

42
RQ 1: How to determine optimal hyper-parameter values for the deep learning models?

There are several algorithms for determining optimal hyper-parameter values for the
models we are using like grid search, random search, manual search, automated hyper-
parameter tuning, and artificial neural network-tuning. To determine an optimal hyper-
parameter value for our model we have used a manual search technique. We choose
some model hyperparameters based on our judgment/experience when utilizing Manual
Search as given under table 1. After that, we train the model, assess its accuracy, and
repeat the procedure. This loop is performed until the accuracy is adequate.

RQ 2: Which model contributes more and less to the performance of the

summarizer?

As our experimental result shows sequence to sequence model with Bi-LSTM performs
better and sequence to sequence model with Hybrid-LSTM performs less as we can
observe from the experimental results reviewed under table 5 and table 6.

43
 CHAPTER FIVE

5 Conclusion and Recommendation

5.1 Conclusion

This thesis had investigated the capability and performance of deep learning techniques
for Afaan Oromoo abstractive text summarization. The researcher tried to perform a
comparison analysis between three deep learning models, Seq2Seq model with LSTM
for Afaan Oromoo text summarizer, Seq2Seq model with Bi-LSTM for Afaan Oromoo
text summarizer and Seq2Seq model with hybrid-LSTM for Afaan Oromoo text
summarizer.

To train the models and evaluate their performance; we have used 7400 pairs of articles
and summaries collected from different online Afaan Oromoo news site. Then, the
models were trained with the prepared dataset. Seq2Seq model with LSTM model
scores 95.48% of training accuracy and 94.84% of validation accuracy. The second
model, Seq2Seq model with Bi-LSTM scores 99.95% of training accuracy and 99.97
% of validation accuracy. The last model, Seq2Seq model with hybrid-LSTM scores
94.37% of training accuracy and 93.92% of validation accuracy.

To evaluate the performance of predicted summary generated by the systems, recall,

precision and f-measure for ROUGE-1 and ROUGE-2 were employed to evaluate the
overlap of unigram and bigram between predicted summary and reference summary.
Accordingly, 32.15% rouge-1 recall, 32.85% rouge-1 precision, 32.49% rouge-1 f-
measure and 6.5% rouge-2 recall, 7.7% rouge-2 precision and 7.04% rouge-2 f-measure
were obtained by LSTM model. Bi-LSTM achieves 41.77% rouge-1 recall, 31.33%
rouge-1 precision, 35.8% rouge-1 f-measure and 16.33% rouge-2 recall, 13.1% rouge-
2 precision and 14.18% rouge-2 f-measure. The third model, hybrid-LSTM model
achieves 21.69% rouge-1 recall, 22.27% rouge-1 precision, 21.31% rouge-1 f-measure
and 6.39% rouge-2 recall, 7.007% rouge-2 precision and 6.68% rouge-2 f-measure.

From this result, we can conclude that Bi-LSTM model performed well as it scores
99.8% of training accuracy and 99.03% of validation accuracy which is highest value
when compared with the other two (LSTM and Hybrid-LSTM) models. The f-measure
score of Bi-LSTM 35.8% and 14.18% for ROUGE-1 and ROUGE-2 respectively is the

44
highest when compared with LSTM and hybrid-LSTM which tells us that the summary
generated by this model is better than the other.

5.2 Recommendation

In this study three deep learning models are trained and validated for Afaan Oromoo
dataset. When compared with some of foreign languages, we prepared and used small
dataset, but it’s recommended to prepare huge dataset to increase the performance of
the models. As the number of dataset increases deep learning models can predict better
summaries for a given text.

For abstractive summarization LSTM, Bi-LSTM and Hybrid LSTM are not the only
deep learning models. There are another deep learning models like RNN Encoder-
Decoder, Bidirectional RNN, Gated recurrent unit and Attention Mechanism.

We recommend the future researchers to:

 Perform experiments for other deep learning models except the models we have
experimented.

45
References

[1] M. Y. Day and C. Y. Chen, “Artificial intelligence for automatic text

summarization,” Proc. - 2018 IEEE 19th Int. Conf. Inf. Reuse Integr. Data Sci.
IRI 2018, pp. 478–484, 2018, doi: 10.1109/IRI.2018.00076.

[2] S. Ou, C. S. G. Khoo, and D. H. Goh, “Design and development of a concept-

based multi-document summarization system for research abstracts,” J. Inf.
Sci., vol. 34, no. 3, pp. 308–326, 2008, doi: 10.1177/0165551507084630.

[3] D. Suleiman and A. A. Awajan, “Deep Learning Based Extractive Text

Summarization: Approaches, Datasets and Evaluation Measures,” 2019 6th Int.
Conf. Soc. Networks Anal. Manag. Secur. SNAMS 2019, pp. 204–210, 2019,
doi: 10.1109/SNAMS.2019.8931813.

[4] Q. A. Al-Radaideh and D. Q. Bataineh, “A Hybrid Approach for Arabic Text

Summarization Using Domain Knowledge and Genetic Algorithms,” Cognit.
Comput., vol. 10, no. 4, pp. 651–669, 2018, doi: 10.1007/s12559-018-9547-z.

[5] S. Biswas, R. Rautray, R. Dash, and R. Dash, “Text Summarization: A

Review,” Proc. - 2nd Int. Conf. Data Sci. Bus. Anal. ICDSBA 2018, pp. 231–
235, 2018, doi: 10.1109/ICDSBA.2018.00048.

[6] Ibrahim Bedane, “The Origin of Afaan Oromo : Mother Language,” Double
Blind Peer Rev. Int. Res. J., vol. 15, no. 12, 2015.

[7] H. P. Luhn, “The Automatic Creation of Literature Abstracts,” IBM J. Res.

Dev., vol. 2, no. 2, pp. 159–165, 2010, doi: 10.1147/rd.22.0159.

[8] A. Padmakumar and A. Saran, “Unsupervised Text Summarization Using

Sentence Embeddings,” 2017.

[9] S. Abdulateef, N. A. Khan, B. Chen, and X. Shang, “Multidocument Arabic

text summarization based on clustering and word2vec to reduce redundancy,”
Inf., vol. 11, no. 2, 2020, doi: 10.3390/info11020059.

[10] G. Rossiello, P. Basile, and G. Semeraro, “Centroid-based Text Summarization

through Compositionality of Word Embeddings,” pp. 12–21, 2017, doi:
10.18653/v1/w17-1003.

46
[11] C. Khatri, G. Singh, and N. Parikh, “Abstractive and Extractive Text
Summarization using Document Context Vector and Recurrent Neural
Networks,” 2018.

[12] E. D. Yirdaw and D. Ejigu, “Topic-based amharic text summarization with

probabilisic latent semantic analysis,” Proc. Int. Conf. Manag. Emergent Digit.
Ecosyst. MEDES 2012, pp. 8–15, 2012, doi: 10.1145/2457276.2457279.

[13] A. Guadie, D. Tesfaye, and T. Kebebew, “Amharic Text Summarization for

News Items Posted on Social Media,” vol. 11, 2021.

[14] K. D. Dessalegn and M. Y. Tachbelie, “Graph-based Automatic Amharic Text

Summarizer,” vol. 8, no. October, 2017, doi: 10.13140/RG.2.2.26236.77445.

[15] A. B. Kabeta, “Query-based Automatic Summarizer for Afaan Oromo,” 2015.

[16] Sisay Abdisa, “Information Extraction Model from Afaan Oromo News Texts,"
MSc Thesis,” 2017.

[17] F. Berhanu, “Afaan Oromo Automatic News Text Summarizer Based on

Sentence Selection Function,” no. November, 2013.

[18] Gammachiis Temesgen Olana, “Afaan Oromo News Text Summarization

Using Sentence Scoring Method,” vol. 4, no. 1, p. 6, 2021.

[19] A. Qaroush, I. Abu Farha, W. Ghanem, M. Washaha, and E. Maali, “An

efficient single document Arabic text summarization using a combination of
statistical and semantic features,” J. King Saud Univ. - Comput. Inf. Sci., no.
xxxx, 2019, doi: 10.1016/j.jksuci.2019.03.010.

[20] J. S. Kallimani, K. G. Srinivasa, and B. Eswara Reddy, “Summarizing news

paper articles: Experiments with ontology-based, customized, extractive text
summary and word scoring,” Cybern. Inf. Technol., vol. 12, no. 2, pp. 34–50,
2012, doi: 10.2478/cait-2012-0011.

[21] V. Gupta, “Hybrid Algorithm for Multilingual Summarization,” pp. 717–727,

2013.

[22] E. Lloret, “Text summarization: an overview,” pp. 10–17, 2008.

[23] J. Romero, “Abstractive Text Summarisation with Neural Networks,” no.

47
 January 2017, 2017.

[24] S. Gholamrezazadeh, “Proceedings of the 2009 2nd International Conference

on Computer Science and Its Applications, CSA 2009,” Proc. 2009 2nd Int.
Conf. Comput. Sci. Its Appl. CSA 2009, 2009.

[25] M. F. Mridha, A. A. Lima, K. Nur, S. C. Das, M. Hasan, and M. M. Kabir, “A

Survey of Automatic Text Summarization: Progress, Process and Challenges,”
IEEE Access, vol. 9, pp. 156043–156070, 2021, doi:
10.1109/ACCESS.2021.3129786.

[26] V. Gupta and G. S. Lehal, “A Survey of Text Summarization Extractive

techniques,” J. Emerg. Technol. Web Intell., vol. 2, no. 3, pp. 258–268, 2010,
doi: 10.4304/jetwi.2.3.258-268.

[27] J. M. Torres Moreno, “Automatic Text Summarization,” Autom. Text Summ.,

vol. 9781848216, pp. 1–348, 2014, doi: 10.1002/9781119004752.

[28] K. M. Tahsin Hassan Rahit, M. Al-Amin, K. T. Hasan, and Z. Ahmed,

“BanglaNet: Towards a WordNet for Bengali language,” GWC 2018 - 9th
Glob. WordNet Conf., vol. 2018-Janua, no. January, 2018.

[29] A. Ashagre, “Automatic Summarization for Amharic Text Using Open Text
Summarizer,” 2013.

[30] J. Guadalupe Ramos, I. Navarro-Alatorre, G. Flores Becerra, and O. Flores-

Sánchez, “A Formal Technique for Text Summarization from Web Pages by
using Latent Semantic Analysis,” Res. Comput. Sci., vol. 148, no. 3, pp. 11–22,
2019, doi: 10.13053/rcs-148-3-1.

[31] M. G. Ozsoy, I. Cicekli, and F. N. Alpaslan, “Text summarization of turkish

texts using latent semantic analysis,” Coling 2010 - 23rd Int. Conf. Comput.
Linguist. Proc. Conf., vol. 2, no. August, pp. 869–876, 2010.

[32] S. S. Lwin and K. T. Nwet, “Extractive Summarization for Myanmar

Language,” 2018 Int. Jt. Symp. Artif. Intell. Nat. Lang. Process. iSAI-NLP 2018
- Proc., pp. 1–6, 2018, doi: 10.1109/iSAI-NLP.2018.8692976.

[33] K. Kaikhah, “Automatic text summarization with neural networks,” 2004 2nd
Int. IEEE Conf. ’Intelligent Syst. - Proc., vol. 1, no. June, pp. 40–44, 2004, doi:

48
 10.1109/is.2004.1344634.

[34] P. P. Shinde and S. Shah, “A Review of Machine Learning and Deep Learning
Applications,” Proc. - 2018 4th Int. Conf. Comput. Commun. Control Autom.
ICCUBEA 2018, pp. 1–6, 2018, doi: 10.1109/ICCUBEA.2018.8697857.

[35] Y. Hua, J. Guo, and H. Zhao, “Deep Belief Networks and deep learning,” Proc.
2015 Int. Conf. Intell. Comput. Internet Things, ICIT 2015, pp. 1–4, 2015, doi:
10.1109/ICAIOT.2015.7111524.

[36] B. It et al., “M Ixed - Curvature V Ariational a Utoencoders,” vol. 1, no. i, pp.

1–44, 2020.

[37] C. Yin, Y. Zhu, J. Fei, and X. He, “A Deep Learning Approach for Intrusion
Detection Using Recurrent Neural Networks,” IEEE Access, vol. 5, pp. 21954–
21961, 2017, doi: 10.1109/ACCESS.2017.2762418.

[38] D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, “Learning a deep

convolutional network for image super-resolution,” Lect. Notes Comput. Sci.
(including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol.
8692, no. September, pp. 184–199, 2014.

[39] Y. Deng, F. Bao, Y. Kong, Z. Ren, and Q. Dai, “Deep Direct Reinforcement
Learning for Financial Signal Representation and Trading,” IEEE Trans.
Neural Networks Learn. Syst., vol. 28, no. 3, pp. 653–664, 2017, doi:
10.1109/TNNLS.2016.2522401.

[40] S. Syed, “Abstractive Summarization of Social Media Posts : A Case Study

Using Deep Learning Master ’ s Thesis Unless otherwise indicated in the text
or references , this thesis is,” no. 115594.

[41] S. Song, H. Huang, and T. Ruan, “Abstractive text summarization using

LSTM-CNN based deep learning,” Multimed. Tools Appl., vol. 78, no. 1, pp.
857–875, 2019, doi: 10.1007/s11042-018-5749-3.

[42] D. Bahdanau, K. H. Cho, and Y. Bengio, “Neural machine translation by

jointly learning to align and translate,” 3rd Int. Conf. Learn. Represent. ICLR
2015 - Conf. Track Proc., no. November, 2015.

[43] K. Lopyrev, “Generating News Headlines with Recurrent Neural Networks,”

49
 pp. 1–9, 2015.

[44] R. Adelia, S. Suyanto, and U. N. Wisesty, “Indonesian abstractive text

summarization using bidirectional gated recurrent unit,” Procedia Comput.
Sci., vol. 157, pp. 581–588, 2019, doi: 10.1016/j.procs.2019.09.017.

[45] C. Sun, L. Lv, G. Tian, Q. Wang, X. Zhang, and L. Guo, “Leverage Label and
Word Embedding for Semantic Sparse Web Service Discovery,” Math. Probl.
Eng., vol. 2020, 2020, doi: 10.1155/2020/5670215.

[46] M. Gambhir and V. Gupta, “Recent automatic text summarization techniques: a

survey,” Artif. Intell. Rev., vol. 47, no. 1, pp. 1–66, 2017, doi: 10.1007/s10462-
016-9475-9.

[47] J. Steinberger and K. Ježek, “Evaluation measures for text summarization,”

Comput. Informatics, vol. 28, no. 2, pp. 251–275, 2009.

[48] G. D. Dinegde and M. Y. Tachbelie, “Afan Oromo News Text Summarizer,”

Int. J. Comput. Appl., vol. 103, no. 4, pp. 975–8887, 2014.

[49] P. Kouris, G. Alexandridis, and A. Stafylopatis, “Abstractive Text

Summarization: Enhancing Sequence-to-Sequence Models Using Word Sense
Disambiguation and Semantic Content Generalization,” Comput. Linguist., vol.
47, no. 4, pp. 813–859, 2021, doi: 10.1162/coli_a_00417.

[50] D. Bahdanau, K. H. Cho, and Y. Bengio, “Neural machine translation by

jointly learning to align and translate,” 3rd Int. Conf. Learn. Represent. ICLR
2015 - Conf. Track Proc., pp. 1–15, 2015.

[51] K. M. Hermann et al., “Teaching Machines to Read and Comprehend NIPS

2015,” Proc. 28th Int. Conf. Neural Inf. Process. Syst., pp. 1693–1701, 2015.

[52] R. Nallapati, B. Zhou, C. dos Santos, Ç. Gulçehre, and B. Xiang, “Abstractive

text summarization using sequence-to-sequence RNNs and beyond,” CoNLL
2016 - 20th SIGNLL Conf. Comput. Nat. Lang. Learn. Proc., pp. 280–290,
2016, doi: 10.18653/v1/k16-1028.

[53] J. Gu, Z. Lu, H. Li, and V. O. K. Li, “Incorporating copying mechanism in

sequence-to-sequence learning,” 54th Annu. Meet. Assoc. Comput. Linguist.
ACL 2016 - Long Pap., vol. 3, pp. 1631–1640, 2016, doi: 10.18653/v1/p16-

50
 1154.

[54] O. Vinyals, M. Fortunato, and N. Jaitly, “[2015-NIPS] Pointer Networks,” Nips

2015, pp. 1–9, 2015.

[55] C. Gulcehre, S. Ahn, R. Nallapati, B. Zhou, and Y. Bengio, “Pointing the

unknown words,” 54th Annu. Meet. Assoc. Comput. Linguist. ACL 2016 - Long
Pap., vol. 1, pp. 140–149, 2016, doi: 10.18653/v1/p16-1014.

[56] A. See, P. J. Liu, and C. D. Manning, “Get to the point: Summarization with
pointer-generator networks,” ACL 2017 - 55th Annu. Meet. Assoc. Comput.
Linguist. Proc. Conf. (Long Pap., vol. 1, pp. 1073–1083, 2017, doi:
10.18653/v1/P17-1099.

[57] M. Ranzato, S. Chopra, M. Auli, and W. Zaremba, “Sequence level training

with recurrent neural networks,” 4th Int. Conf. Learn. Represent. ICLR 2016 -
Conf. Track Proc., pp. 1–16, 2016.

[58] P. Clark, P. Harrison, N. Balasubramanian, and O. Etzioni, Proceedings of the

Joint Workshop on Automatic Knowledge Base Construction and Web-scale
Knowledge Extraction (AKBC-WEKEX). 2012.

[59] R. Paulus, C. Xiong, R. Socher, and P. Alto, “a D Eep R Einforced M Odel,”

pp. 1–13, 2018.

[60] K. Yao, L. Zhang, D. Du, T. Luo, L. Tao, and Y. Wu, “Dual encoding for
abstractive text summarization,” IEEE Trans. Cybern., vol. 50, no. 3, pp. 985–
996, 2020, doi: 10.1109/TCYB.2018.2876317.

[61] T. D. Kulkarni, K. R. Narasimhan, A. Saeedi, and J. B. Tenenbaum,

“Hierarchical deep reinforcement learning: Integrating temporal abstraction
and intrinsic motivation,” Adv. Neural Inf. Process. Syst., pp. 3682–3690,
2016.

[62] B. Peng et al., “Composite task-completion dialogue policy learning via

hierarchical deep reinforcement learning,” EMNLP 2017 - Conf. Empir.
Methods Nat. Lang. Process. Proc., pp. 2231–2240, 2017, doi:
10.18653/v1/d17-1237.

[63] X. Wang, W. Chen, J. Wu, Y. F. Wang, and W. Y. Wang, “Video Captioning

51
 via Hierarchical Reinforcement Learning,” Proc. IEEE Comput. Soc. Conf.
Comput. Vis. Pattern Recognit., pp. 4213–4222, 2018, doi:
10.1109/CVPR.2018.00443.

[64] V. N. Quoc, H. L. T. B, and T. L. Minh, LSTMs with Rich Features, vol. 1.

Springer Singapore, 2020.

[65] Z. Cao, W. Li, S. Li, F. Wei, and Y. Li, “AttSum: Joint learning of focusing
and summarization with neural attention,” COLING 2016 - 26th Int. Conf.
Comput. Linguist. Proc. COLING 2016 Tech. Pap., pp. 547–556, 2016.

[66] R. Paulus, C. Xiong, and R. Socher, “A deep reinforced model for abstractive
summarization,” 6th Int. Conf. Learn. Represent. ICLR 2018 - Conf. Track
Proc., no. i, pp. 1–12, 2018.

[67] J. Zhang, J. Tan, and X. Wan, “Towards a Neural Network Approach to

Abstractive Multi-Document Summarization,” 2018.

[68] E. Kantzola, “Extractive Text Summarization of Greek News Articles Based

on,” 2020.

[69] K. Nuru, “Automatic Amharic NewsText Summarizer,” 2005.

[70] L. T. Fanache, “Afaan Oromo Text Summarization using Word Embedding,”

2020.

[71] C.-Y. Lin, “ROUGE: A Package for Automatic Evaluation of Summaries,”

Jpn. Circ. J., vol. 34, no. 12, pp. 1213–1220, 1971.

52
Appendixes

Appendix A: Afaan Oromoo Contractions

Contractions Long forms

Fkn. Fakkeenya
mm Muummee Ministeera
k.k.f Kan kana fakkaatan
w.k.f Waan kana fakkaatan
Mil. Miiliyoona
Bil. Biiliyoona
Ful. Fulbaana
Onk. Onkoloolessa
Sad. Sadaasa
Mud. Mudde
Ama. Amajji
Gur. Guraandhala
Bit. Bitootessa
Ebl. Eebla
Cam. Caamsaa
Sad. Saadasaa
Wax. Waxabajjii
Ado. Adoolessa
Hag. Hagayya
Hub. Hubadhu/Hubachiisa
W.D. Waaree Dura
W.B. Waaree Booda
Obb. Obbo
Add. Aadde

53
Appendix B: Afaan Oromoo stop words

Iseen hoggas Naaf yommuu Hogguu jechaan tana

Jara eegas Otuu Agarsiiso Iseenis naan yoo
o
ka'uun Adda tahullee eegana Kee otuullee Akka
Aanee hogguus Isheef jechuu Ha isinitti isaaniitii
n
hoggaa yoom jechuun iddoo eegasii Kees ta'eef
Iseef ta'ullee Ishii egasiis jechuu tanaaf yeroo
eega saaniif Illee Akkas Isheef qaba otuma
yommii nu ennaa yookiin Hoo naannoo Na
ta'es Keenyaa Akkasumas tanaafi egasii ituullee kanatti
otumallee jechuunis Yookinimo qabdi Akkam Isee hin
o
naa Ishiif tanaafis narraa yookaan duuba yemmuu
Jala immoo qabna Keen Ishee ta'ee otoo
Karaa erga natti jechoota Moo kanaatti Ituu
qabu Akum Keenna isatti duras yammuu ta'e
ni yoolinimo jechuuf hennas osoo Miti kanaan
o
Keenya tanaafuu isheen itumallee isarraa dura henna
ta'an ol Maqaa kanaafuu Itti isaatiin hangas
Dugda ta'aa Manna ittaanee Kanaafiyyu isaanitti hanga
u
Dudduub waliin oggas Malees kanaafis isinittis ta'a
a
isaanirraa hamma Dhaan wajjin sunniin Dabalatee oggaa
s
Malee kanaafillee ta'uyyuu sanis Ammo Ammo keessan

innasuu ishiitti jedhe Isii Keetiis nuti sii

F Alatti Keessatti jedhu jetti teenya teessan
ta'uu ta'uun saniif Keeti Isin Ammo An
sadii sana nu'is nurraa Irra fullee fuullee

54
nus Keessan Alla sanis Ammo irraa irraan
Keessa nu'i ta'uyyu ta'uyyuu tawullee Isini Isinii
jedha innaa faallaa Amma Si jira jirtutti
Ishiin ergii jedhan fagaatee nurraas Keetis Kiyya
Ini Ala ishiirraa innis jette nutis nutti
Akkuma ergasii Ishiin Keetii Isiin siif siin
isiniinis isinillee kana Kunis nuyis sirraa utuullee
isaaniif garuu Booda utuu siqee kan Bira
sinitti Koos nuy ture garas isaan Kun
isaaf Isiniin tti ka kaa'uun gararraa simmoo
Koo isiniifis nuun too Isaa Koof Ati
Kiyyas jiru Ani silaa Isa silas gara
gajjallaa Ana nuu isiniifillee jirutti Anis nuuf
Isinis isaanii ofiis sunis Lama waggaa
isinirraas kanaaf ofii Booddees Sun waan gubbaa
sitti odoo Booddee kanaa gidduu isaani Lafa
nuyi Boodas giddu isaanis isinirraa Kuniyyuu waa'ee
kanaafi tiyya tiyya irratti Isiniif tokko gama

55
 Appendix C: Sample output for the models

56

View publication stats