Applied Soft Computing 30 (2015) 737–747

Contents lists available at ScienceDirect

Applied Soft Computing
journal homepage: www.elsevier.com/locate/asoc

A framework for multi-document abstractive summarization based
on semantic role labelling
Atif Khan a,∗ , Naomie Salim a , Yogan Jaya Kumar b
a
b

Faculty of Computing, Universiti Teknologi Malaysia, 81310 Skudai, Johor, Malaysia
Faculty of Information and Communication Technology, University Teknikal Malaysia Melaka, 76100 Melaka, Malaysia

a r t i c l e

i n f o

Article history:
Received 18 May 2014
Received in revised form 7 September 2014
Accepted 29 January 2015
Available online 20 February 2015
Keywords:
Abstractive summary
Semantic role labeling
Semantic similarity measure
Language generation
Genetic algorithm

a b s t r a c t
We propose a framework for abstractive summarization of multi-documents, which aims to select contents of summary not from the source document sentences but from the semantic representation of
the source documents. In this framework, contents of the source documents are represented by predicate argument structures by employing semantic role labeling. Content selection for summary is made
by ranking the predicate argument structures based on optimized features, and using language generation for generating sentences from predicate argument structures. Our proposed framework differs from
other abstractive summarization approaches in a few aspects. First, it employs semantic role labeling for
semantic representation of text. Secondly, it analyzes the source text semantically by utilizing semantic
similarity measure in order to cluster semantically similar predicate argument structures across the text;
and finally it ranks the predicate argument structures based on features weighted by genetic algorithm
(GA). Experiment of this study is carried out using DUC-2002, a standard corpus for text summarization.
Results indicate that the proposed approach performs better than other summarization systems.
© 2015 Elsevier B.V. All rights reserved.

1. Introduction
The information on Web is growing at exponential pace. In the
current era of information overload, multi-document summarization is an essential tool that creates a condensed summary while
preserving the important contents of the source documents. The
automatic multi-document summarization of text is a major task in
the field of natural language processing (NLP) and has gained more
consideration in recent years [1]. One of the problems of information overload is that many documents share similar topics, which
creates both difficulties and opportunities for natural language systems. On one hand, the similar information conveyed by several
different documents, causes difficulties for the end users, as they
have to read the same information repeatedly. On the other side,
such redundancy can be used to identify accurate and significant
information for applications such as summarization and question
answering. Thus, summaries that synthesize common information
across many text documents would be useful for users and reduce
their time for finding the key information in the text documents.
Such a summary would significantly help users interested in single

∗ Corresponding author. Tel.: +60 1126799325.
E-mail addresses: atifkhan@icp.edu.pk (A. Khan), naomie@utm.my (N. Salim),
yogan@utem.edu.my (Y. Jaya Kumar).
http://dx.doi.org/10.1016/j.asoc.2015.01.070
1568-4946/© 2015 Elsevier B.V. All rights reserved.

event described in many news documents [1]. In this paper, we propose a framework that will automatically fuse similar information
across multiple documents and use language generation to produce
a concise abstractive summary.
Two approaches are employed to multi-document summarization: extractive and abstractive. Most of the studies have focused
on extractive summarization, using techniques of sentence extraction [2], statistical analysis [3], discourse structures and various
machine learning techniques [4]. On other hand, abstractive summarization is a challenging area and dream of researchers [5],
because it requires deeper analysis of the text and has the capability to synthesize a compressed version of the original sentence or
may compose a novel sentence not present in the original source.
The goal of abstractive summarization is to improve the focus of
summary, reduce its redundancy and keeps a good compression
rate [6].
Past literature shows that there have been a few research efforts
made toward abstractive summarization. Many researchers have
tried to generate abstractive summaries using various methods.
These abstractive methods can be grouped into two categories: Linguistic (Syntactic) based approach and Semantic based approach.
Linguistic (Syntactic) based approach employs syntactic parser
to analyze and represent the text syntactically. Usually, in this
approach, verbs and nouns identified by syntactic parser are used
for text representation and further processed to generate the

The important concepts represented by semantic model are rated based on information density metric and expressed as sentences using the available phrasings stored for each concept in a semantic model. which is time consuming. To the best of our knowledge. semantic representation of source text will be a more suitable representation.8. Section 2. which is a drawback for an automatic summarization system. Since abstractive summarization requires deep analysis of text. [19] introduced a work that combined semantic role labeling with general statistic method (GSM) to determine important sentences for single document extractive summary. aims to treat this limitation by using semantic role labeling (SRL) technique to build semantic representation from the document text automatically. First. this approach is limited to Chinese news. We employ SRL in second phase of the framework. Like other approaches. The major limitation of all the semantic based approaches for abstractive summarization is that they are mostly dependent on human expert to construct domain ontology and rules.e. A multi-document summarization system. This approach has several limitations. A fuzzy ontology based approach [11] was proposed for Chinese news summarization to model uncertain information and hence can better describe the domain knowledge. The semantic model is constructed using knowledge representation based on objects (concepts) organized by ontology. the general statistic method was used to computes the sentence score based on features. the similarities of predicate argument structures (PASs) are computed by comparing them pair wise based on Jiang’s semantic similarity measure [21] and edit distance algorithm. Secondly. On other hand. . Rich semantic graph is an ontology based representation i. first of all. Therefore. Moreover. syntactic analysis does not define who did what to whom (and how. Different semantic representations of text used in the literature are ontology based and template based representation. it employs semantic role labeling to extract predicate argument structure (semantic representation) from the contents of input documents. weighted and optimized by genetic algorithm and the top . The limitation of this framework was that it relies on manually built ontology.4 will describe this phase in detail.6. domain ontology and Chinese dictionary has to be defined by a domain expert which is time consuming. Overview of approach The framework of our proposed approach is illustrated in Fig. / Applied Soft Computing 30 (2015) 737–747 abstractive summary. which exploits semantic role parser.). Secondly. This approach use template for topic representation of document. Finally we end with conclusion in Section 4. graph nodes are the instances of ontology noun and verb classes. 1.1. the sentence scores obtained from both methods are combined to assign the overall score to each sentence in the document and the top ranked sentences are extracted according to 20% compression rate. this method could not handle or capture the information about similarities and differences across multiple documents. in contrast. The drawback of the methodology was that information extraction (IE) rules and generation patterns were written by hand. However. Once the similarity matrix for PASs is obtained. therefore. which was extremely time consuming. The PASs in each cluster are scored based on features. In particular. Khan et al. Framework for multi-document abstractive summarization 2. which is time consuming. [19] employed SRL and semantic similarity measure to compute sentence semantic similarity score. information extraction [16]. In semantic similarity matrix computation phase (as discussed in Section 2. . Secondly.e. a few semantic based approaches have also been proposed for abstractive summarization and are briefly discussed as follows. A framework proposed by [13] generates abstractive summary from a semantic model of a multimodal document. is different from previous abstractive summarization approaches in a few aspects. which is time consuming. They addressed that syntactic analysis is far away from representing the meaning of sentences. this study proposes a framework that will employ SRL for semantic representation of text in order to generate a good abstractive summary. First. On other hand. semantic based approach aims to produce abstractive summary from semantic representation of document text. The evaluation of the framework is given in Section 3. The rest of this paper is organized as follows: Section 2 outlines the proposed framework. human judges were asked to assess the quality of system summary by comparing it with other summary generated by traditional extraction methods. the limitation of this approach was that it also relies on manually built ontology. Secondly. we perform agglomerative hierarchical clustering (HAC) algorithm based on average linkage method to cluster semantically similar predicate argument structures. The abstraction scheme uses a rule based information extraction module.3). it is manually evaluated by humans i. without taking into account their weights. since text features are sensitive to the quality of the generated summary. they employed SRL and semantic similarity measure to compute the sentence similarity score. it analyzes the text semantically by utilizing semantic similarity measure in order to cluster semantically similar predicate argument structures across the text. At first. content selection heuristics and one or more patterns for sentence generation. The methodology proposed in [12] generates short and wellwritten abstractive summaries from clusters of news articles on same event using abstraction schemes. The framework for multi-document abstractive summarization presented in this study. text categorization [17] and sentiment analysis [18]. SRL has been widely applied in text content analysis tasks such as text retrieval [15]. presented in [10] exploits template based method to produce abstractive summary from multiple newswire/newspaper documents depending on the output of the information extraction (IE) system. The number of clusters will depend on compression rate of summary.738 A. Given a document collection that need to be summarized. and might not be applicable to English news. In the area of text summarization. Next. for semantic representation of text in the document collection. our work is different from [19] in the following manner. and finally it ranks the predicate argument structures based on the features weighted and optimized by genetic algorithm. Our work. On other hand. The major limitations of these approaches is the lack of semantic representation of source text. Finally. has not been employed for the semantic representation of text in multidocument abstractive summarization. we split the document collection into sentences in such a way that each sentence is preceded by its corresponding document number and sentence position number. The major limitation observed in this approach was that linguistic patterns and extraction rules for template slots were manually created by humans. Titov and Klementiev [7] made a distinction between syntactic and semantic representation of sentence. We focus on multi-document abstractive summarization while [19] focus on single document extractive summarization. The abstractive approach presented by [14] summarizes a document by creating a Rich Semantic Graph (RSG) for the source document. All the linguistic based approaches [1.9] proposed for the abstractive summarization rely on the syntactic representation of source document. These approaches employ syntactic parser to represent the source text syntactically. SENNA semantic role labeler [20] is employed to extract predicate argument structure from each sentence in the document collection. GISTEXTER. when. 2. semantic role labeling (SRL) technique.

named entity recognition (NER) and chunking (CHK). The extracted predicate argument structure is used as semantic representation for each sentence in the document collection. / Applied Soft Computing 30 (2015) 737–747 739 Table 1 Core arguments and adjunctive arguments. We consider all the complete predicates associated with the single sentence structure in order to avoid loss of important terms contributing to the meaning of sentence. The generated sentences will form the final abstractive summary (as discussed in Section 2. The words in PASs are stemmed to their base form using porter stemming algorithm [24]. 2. etc. followed by removal of stop words. the corresponding two predicate argument structures are obtained as follows: P1: [A0: Tropical Storm Gilbert] [V: form] [ArgM-LOC: in the eastern Caribbean] P2: [A0: Tropical Storm Gilbert] [V: strengthen] [A2: into a hurricane Saturday night] Both predicate argument structures P1 and P2 are associated with a single sentence S and hence the sentence S is represented by a composite (more than one) predicate argument structure. Temporal and Manner [23]. and ArgM-LOC for location. and the actual predicate of the sentence. SENNA POS tagger [20] is employed to label each term of semantic arguments (associated with the predicates). Processing of predicate argument structures Once the predicate argument structures (PASs) are obtained. a massive hurricane attack my home The corresponding simple predicate argument structures P1 and P2 are obtained after applying semantic role labeling to sentences S1 and S2: P1: [AM-TMP: Eventually] [A0: a huge cyclone] [V: hit] [A1: the entrance of my house] P2: [AM-DIS: Finally] [A0: a massive hurricane] [V: attack] [A1: my home] 2. We assume that predicates are complete if they have at least two semantic arguments.6). The POS tags NN stands for noun. The framework uses SENNA [20] tool distributed under open source and non-commercial license. ranked predicate argument structures are selected from each cluster (as described in Section 2. they are split into meaningful words or tokens. As abstractive summarization requires deeper semantic analysis of text. identify the semantic roles of the arguments such as Agent. ArgM-TMP for time as adjunctive arguments for predicate (Verb) V. Finally. This step is required. In this study. and yields a host of natural language processing (NLP) predictions: semantic role labeling (SRL). Fig. the two predicate argument structures P1 and P2 in example 2 are as follows: .2.5). At first.2. we compare only terms of the semantic arguments of the predicates which are labeled as noun (NN) and the rest are ignored as discussed in Section 2. In this study. These phrases are referred to as semantic arguments. we consider A0 for subject. S1: Eventually.A. we decompose the document collection into sentences in such a way that each sentence is preceded by its corresponding document number and sentence position number. The primary task of SRL is to identify what semantic relation a predicate holds with its participants/constituents. We employ SENNA tool in our framework for SRL. Proposed framework. V for verb.3. S2: Finally. Both incomplete predicate argument structures (PASs) and the PASs that are nested in a larger predicate argument structure are ignored. 1. Consider sentence S: Tropical Storm Gilbert formed in the eastern Caribbean and strengthened into a hurricane Saturday night. Consider the following two sentences represented by simple predicate argument structures. a huge cyclone hit the entrance of my house. JJ for adjective and RB for adverb. Next.1. The semantic arguments can be grouped in two categories: core arguments (Arg) and adjunctive arguments (ArgM) as shown in Table 1. part-ofspeech (POS) tags. A sentence containing one predicate is represented by After applying semantic role labeling to sentence S. as semantic arguments of the predicates will be compared based on grammatical roles of the terms. Khan et al. After POS tagging. Core Arguments V A0 A1 A2 A3 A4 A5 Verb Subject Object Indirect Object Start point End point Direction Adjunctive arguments ArgM-DIR Direction ArgM-MNR Manner ArgM-LOC Location ArgM-TMP Temporal marker ArgM-PRP Purpose ArgM-NEG Negation ArgM-REC Reciprocal AM-DIS Discourse marker simple predicate argument structure while a sentence containing more than one complete predicate is represented by a composite predicate argument as illustrated below. Next. A2 for indirect object as core arguments. Patient and Instrument. therefore. A1 for object. POS tags and NER. with part of speech (POS) tags or grammatical roles. Example 2. Semantic role labeling The aim of semantic role labeling (SRL) is to determine the syntactic constituents/arguments of a sentence with respect to the sentence predicates. SENNA semantic role labeler is employed to parse each sentence and properly labels the semantic word phrases. the SimpleNLG realisation engine [22] is employed to generate sentences from the selected predicate argument structures. and the adjunctive arguments of the predicate such as Locative. Example 1. this study employs semantic role labeling to extract predicate argument structure from sentences in the document collection.

2. (3) and (4) for computing possible match/similarity between temporal and location arguments of the predicates. According to this measure. (1). verbs. respectively. vjl )] (5) The semantic similarity computation of the two predicate argument structures discussed in example 2 is depicted in Fig. vjl ) + simloc (vik. C2)). 2. Computation of semantic similarity score of predicate argument structures. A1j ) + sim(A2i .e. vjl ) = (sim(Tmpi . 2. Pj )) (2) simtmp (vik. Jiang’ measure determines ‘cylone’ as a least common subsumer of the given two concepts ‘hurricane’ and ‘cyclone’ as shown below. These named entities are stored for each predicate argument structure (PAS) and is required in later phase for scoring the PAS based on proper noun feature. Computation of semantic similarity matrix The objective of this phase is to build matrix of semantic similarity scores for each pair of predicate argument structure. location. Since Jiang’s measure is based on WordNet. Once the nouns. the noun ‘home’ in the semantic argument A1 of PAS P1 will be compared with both of the nouns ‘entrance’ and ‘house’ in the semantic argument A1 of PAS P2. Therefore. and time as identified in previous steps. In order to compute the similarity score of the given two terms/concepts in the semantic argument ‘A0’ of predicates P1 and P2. Locj )) (4) Eqs. the predicate argument structures obtained in example 2 after further processing will become P1: [A0: hurricane NN] [V:attack] [A1: home (NN)] P2: [AM-TMP: Eventually (RB)] [A0: cyclone (NN)] [VBD: hit] [A1: entrance (NN). Therefore. the temporal and location arguments may not be found in the WordNet. verb. Fig. vjl ) = (sim(Pi . (4). vjl ) + [simarg (vik.e. C2) = IC(C1) + IC(C2) − 2 × IC(lso(C1.3. Given two sentences Si and Sj . vjl ) = simp (vik. and other arguments (time and location) if exist. simtmp (vik. vjl ) is computed using Eq. Jiang’s measure [21] calculates the semantic distance to obtain semantic similarity between any two concepts as follows: Jiangdist (C1. All the PASs associated with the sentence will be included in comparison. simloc (vik. A0j ) + sim(A1i . Similarity between corresponding temporal arguments i. The temporal (time) semantic argument ‘Eventually’ in P2 is skipped from comparison as there is no corresponding temporal argument in P1. vjl ) = (sim(Loci . which are labeled as noun. IC(C2). then determines IC(C1). are extracted. Tmpj )) (3) simloc (vik. verb–verb. This study compares predicate argument structures based on noun–noun. similarity of the predicate argument structures (PASs) is computed pair wise based on acceptable comparisons of noun–noun. and IC(lso (C1. vjl ) is the sum of similarities between the corresponding arguments of the predicates determined using Eq. house (NN)] In next phase. verb–verb. A2j ) (1) simp (vik. In this phase. (2). 2. where simp (vik. the framework extracts only tokens from predicate argument structure. (1)–(5). Jiang and Conrath measure has the closest correlation with human judgment amongst all the semantic similarity measures. vjl ) = sim(A0i . and organization names (civil defense) in the semantic arguments of predicate. let concept C1 = “hurricane” and concept C2 = “cyclone” as shown in Fig. First. < entity < physical entity < process < phenomenon < natural phenomenon < physical phenomenon < atmospheric phenomenon < storm < windstorm < cyclone < entity < physical entity < process < phenomenon < natural phenomenon < physical phenomenon < atmospheric phenomenon < storm < windstorm < cyclone < hurricane . vjl ) is the similarity between predicates (verbs) determined using Eq. C2)) (8) Based on WordNet. / Applied Soft Computing 30 (2015) 737–747 P1: [A0: a massive (JJ) hurricane NN] [V: attack] [A1: my home (NN)] P2: [AM-TMP: Eventually (RB)] [A0: a huge (JJ) cyclone (NN)] [VBD: hit] [A1: the entrance (NN) of my house (NN)] We also employ SENNA NER [20] to identify named entities such as person names (cabral). Jiang’s measure [21] uses WordNet to compute the least common subsumer (lso) of two concepts. location–location and time–time. (5) as follows sim(vik. simarg (vik. the similarity of two concepts is dependent on the information that the two concepts share. (3) and the similarity between corresponding location arguments i. The information content (IC) of concept is achieved by determining the probability of occurrence of a concept in a large text corpus and quantified as follows: IC(C) = − log P(C) (6) where P(C) is the probability of occurrence of concept ‘C’ and is computed as follows: P(C) = Freq (C) N (7) where Freq (C) is the number of occurrences of concept ‘C’ in the taxonomy and N is the maximum number of nouns. Both equations (1) and (2) exploit Jiang’s semantic similarity measure for computing similarity between noun terms in the semantic arguments of the predicate argument structures and the verbs of predicate argument structures. (1)–(4) are combined to give Eq.740 A. therefore we use edit distance algorithm instead of Jiang’s measure in Eqs. vjl ) is calculated using Eq. the similarity score between predicate argument structure (PAS) k of sentence Si (v ik ) and PAS l of sentence Sj (v jl ) is determined using Eq. The similarity score between the two predicate argument structures is computed using Eqs. (5). Khan et al. Jiang’s measure is information content based measure and consider that each concept in the WordNet [26] hold certain information. simarg (vik . Based on experimental results in the literature [25]. this study exploits Jiang’s semantic similarity measure [21] for computing semantic similarity between each pair of PASs. vjl ) + simtmp (vik. location–location and time–time arguments.

0726 + 10.P F10}. Pj ) if i = / j 0 otherwise 2. . The following features are extracted for each predicate argument structure (PAS) in the document collection and hence each PAS is represented by a vector representing the weight of these features P = {P F1. the similarity between concepts C1 = “Hurricane” and C2 = “Cyclone” is computed as follows: Jiangdist = 11. P F3 =  sim(Pi .. j) is the semantic similarity between ith and jth predicate argument structures.5.5. . Update the semantic similarity matrix to represent the pair wise similarity between the newest cluster and the original cluster based on average linkage method c. P F2.1.1]. Where ˛ is constant set to 0.5.1. vjl ) and simloc (vik.4.j . PAS to PAS similarity.e. all the sentences in the document collection are represented by corresponding predicate argument structure (PAS) extracted through SENNA SRL. (8). According to Eq. lso(C1. Once the similarity score for each predicate argument structure (PAS) is achieved. This feature identifies proper nouns as words beginning with a capital letter.6671 = 0.6671 IC(lso(C1.2. So. Length of predicate argument structure. Therefore.5526 In order to normalize the result in the range of [0. Length of PAS is the number of words/terms in the PAS. vjl ) = 0.1.j is constructed from the similarity scores of the predicate argument structure Pi and Pj. vjl ) = 2. (5). this study exploits HAC algorithm based on average linkage method.8571 + [0.4055 The similarity of other concepts/terms is determined in the same manner. At first.05X2. vjl ) are set 0 as there are no temporal and location arguments for comparison in both predicate argument structures. Pseudo code for agglomerative clustering algorithm.1.5. However. Pj )  Max (sim(Pi . Semantic clustering of predicate argument structures Agglomerative hierarchical clustering is well-known technique in the hierarchical clustering method. (5). For each predicate argument structure P. P F5 = Number of proper nouns in the PAS Length of PAS (14) 2. = e−0.1. C2)) = 10. Consider 10 sentences in the document. vjl ) [27]. The score of this feature is computed as the ratio of the number of proper nouns in the PAS over the length of PAS [33]. the score of position feature is 10/10 for the first sentence.4. then the score of this feature is obtained by computing the ratio of sum of similarities of PAS P with all other PASs over the maximum of summary in the document collection [34]. j) in the semantic similarity matrix as semantic similarity between ith and jth clusters. Proper nouns.5. is regarded as important for inclusion in summary generation. Pj )) (12) 2.1. the similarity score of the given two predicate argument structures is computed as follows: sim(vik. complete linkage. The score of this feature is computed as follows: P F4 = Length of document − PAS Position + 1 Length of document (13) 2. average linkage.6671 − 2 × 10.5.6. vjl ) = 0. Text features 2.3. The predicate argument structure containing numerical data such as number of people killed.4055 + 0.05 (optimal value). Based on different measures (Entropy and F-Score and Kendall W test). There are five well-known linkage methods of agglomerative hierarchical clustering (HAC) [29] i. Khan et al. This feature is determined by counting the number of matching contents words in the predicate argument structure (PAS) and the title of a document [33].29 + 1 + 0 + 0] sim(vik. The pseudo code for clustering similar predicate argument structures is given below.5.5. Position of PAS [33] gives importance of the PAS in the text document and is equivalent to position of sentence from which PAS is extracted.6671 According to Eq.5526 sim(vik. the above result will become as  The aim of this phase is to select high ranked predicate argument structures from each cluster based on features weighted and optimized by genetic algorithm (GA). the semantic similarity between P and other predicate argument structures in the document collection is computed using Eq. which is quite old but has been found useful in the range of applications [28]. Pj) refers to semantic similarity score of predicate argument structure Pi and Pj in the Matrix Mi. This phase takes semantic similarity matrix as input from previous phase in which the value at position (i. Title feature. then semantic similarity matrix Mi. assuming that the construction of similarity matrix begins with each predicate argument structure as a single cluster.A.8801 Once the semantic similarity score for each pair of predicate argument structure (PAS) is achieved. The predicate argument structure that contains more proper nouns is considered as significant for inclusion in summary generation. C2) = Cyclone IC(C1) = 11. 9/10 for the second sentence and so on.0726 2. which is the ratio of number of words in the PAS over the number of words in the longest PAS of the document [2]. We use the normalize length of the PAS. single linkage. Input: Semantic Similarity Matrix Output: Clusters of similar predicate argument structures a.j is defined as follows: Mi. Position of predicate argument structure.1. Repeat step 1 and 2 until the compression rate of summary is reached P F2 = Number of words occuring in the PAS Number of words occuring in the longest PAS (11) 2. it was found from the literature studies in [30–32] that average linkage is the most suitable method for document clustering.1. P F1 = Number of title words in PAS Number of words in document title (10) 2. we use a scaling factor e−˛Sim(vik. Merge the two clusters that are most similar b. we consider 20% compression rate of summary.5. The score for this . We consider the value at position (i. Mi. 2. Numerical data. (9) where Msim (Pi. ward and centroid method. Selection of PAS from each cluster based on optimized features IC(C2) = 10. / Applied Soft Computing 30 (2015) 737–747 741 In this study.j = Msim (Pi . simtmp (vik.

The GA procedure is shown in Fig. Frequent semantic term.1. Genetic algorithm for optimal feature weighting In automatic text summarization. we will employ F(x) =  Countmatch (gramn ) S∈{Reference Summaries}gramn ∈S   (21) Count(gramn ) S∈{Reference Summaries}gramn ∈S where n is the length of the n-gram. The predicates argument structure containing time and date information for an event is considered as important for summary generation. 3. Nouns and verbs are considered as frequent semantic terms in the predicate argument structure. The chromosomes are randomly initialized with values between 0 and 1. The score of this feature is computed as ratio of the number of temporal information (time and date) in the PAS over the length of PAS [35].1. nouns and verbs in the predicate argument structure.25 0. and ni is the number of documents in which the term i occurs. P F6 = Number of numerical data in the PAS Length of PAS Initial population of chromosomes (15) Fitness evaluation 2.5. To produce a random population of N chromosomes.  (20) Wi (P)) where k is the number of semantic terms in the PAS P.5.9. Each individual chromosome represents weights of features in the form of (w1 .8. w10 ).77 0. P F9 = Number of frequent semantic terms in the PAS Max(Number of frequent semanticterms) (18) 2.73 0.e.1. (21). 2.71 0. Semantic term weight.51 0. The score of this feature is calculated as the ratio of number of frequent semantic terms in the PAS over the maximum number of frequent semantic terms [23].1. Fitness evaluation. In the first step of GA. This feature is computed as the ratio of sum of weights of all semantic terms in the PAS over the maximum summary of the term weights of PAS in the document collection [33].39 0. We define fitness function F(x) as the average recall obtained with each chromosome when summarization process is applied on multi-documents and is given in Eq.02 0. Countmatch is the number of n-grams shared between system generated summary and reference summaries. Temporal feature.5. Therefore.62 ← Chromosome 1 0.83 0.42 0. The fitness evaluation function in GA determines the best chromosome in the population based on its fitness value. feature weighting is crucial for summary generation.01 0. Frequent terms are most probably related to the topic of document and in this study we consider top 10 as maximum number of frequent semantic terms.52 0. Population = Yes Terminate? (17) Optimized feature weights Fig.5. P F8 = No Number of temporal information in the PAS Length of PAS 2. genetic learning algorithm (GA) to obtain optimal feature weights for ranking predicate argument structures for summary generation.742 A.21 0. a real valued population consisting of 50 chromosomes is randomly produced. and Count is the total number of n-grams in the reference summaries.10.08 0.42 ← Chromosome 2 ← Chromosome N . Some sentences may have more than one predicate argument structure associated with them. Khan et al.5.2. Number of nouns and verbs. represented by a composite predicate argument structure and considered important for summary. w2 .26 0. This particular GA based experiment is evaluated against multi-documents in DUC 2002 data set [38]. the quality of the summary is sensitive to the text features i.32 0.33 0.49 0.74 0.03 0. we have for instance: 0.54 0. The score of this feature [34] is computed as follows: P F7 = Total number of nouns and verbs in the PAS Length of PAS Select generation based on fitness score (16) Perform reproduction using cross over and mutation 2. .53 0. 2.18 The weight of semantic term is calculated as follows: N Wi = Tfi x Idfi = Tfi x log ni (19) where Tfi is the term frequency of the semantic term i in the document. 3. . . In this work.5.87 0.2. The score of important term Wi can be determined by the TF-IDF method [36]. N is the total number of documents.1. 0.5. / Applied Soft Computing 30 (2015) 737–747 feature is calculated as the number of numerical data in the PAS over the length of the PAS [33]. The chromosome having highest fitness value has more chances to survive and continue in next generation. GA is chosen since it is a robust and a well-known adaptive optimization technique used in various fields of research and applications [37].7. Genetic algorithm. i=1 P F10 = k Wi (P) i=1 Max ( k where each row represents the chromosome and each column value correspond to the weight of each feature..22 0.67 0. 2.2. Initial population of chromosomes.2. not all features have same relevance with respect to summary. .45 0..31 0.73 0. We apply TF-IDF method to the predicate argument structures in the document collection and consider the term weights for semantic terms i. .e.

The first step in SimpleNLG engine requires defining the basic syntactic constituents that will construct a sentence. employ SimpleNLG [22] and a simple heuristic rule implemented in SimpleNLG. The different features that can be set for constituents are: subject. / Applied Soft Computing 30 (2015) 737–747 2. the SimpleNLG realiser combines the syntactic constituents using simple grammar rules and realize them as syntactically correct sentence. 2. 4. separating them by a comma (if there exist more than two PASs) and then combine them using connective “and”. Thus. cross over and mutation.5. transformed and realize them as summary sentences. The second step assigns features to the constituents. In the last step.e. This process will continue to produce new generations and evaluate its fitness until maximum limit of generations is reached. the final summary sentences generated from the predicate argument structures will be compressed version of the original source sentences in most cases. and ArgM-LOC for location. 4. S2: Hurricane Gilbert slammed into Kingston on Monday with torrential rains and 115 mph winds. object verb. 2. In this study. These operations describe how GA creates its next generation. the significant advantage of this engine is its robustness i. then Eq. the following source input sentences: (22) k=1 where P Fk (Pi ) is score of feature k for predicate Pi. This operation determines the chromosome that will continue to survive in next generation. After applying SENNA SRL. The obtained optimal feature weights are given in Section 3. SimpleNLG is an English realisation engine which provides simple interfaces to produce syntactical structures and transform them into sentences using simple grammar rules. Abstractive summary generation This phase demonstrates how arguments of the predicates will be combined. the engine will not crash when the input syntactical structures are incomplete or ill-formed but typically will yield an inappropriate output. The top scored predicate argument structures are selected from each cluster and are given as input to the summary generation process in next phase.6. In this study. For instance. the simple heuristic rule implemented in SimpleNLG combine the predicate argument structures that refer to the same subject (entity). etc. The following example demonstrates how we generate abstractive summary from the given source input sentences. P2 and P3 are the top scored predicate argument structures selected from different clusters in previous step. which aim to provide a programming control over the realisation process. The SimpleNLG applies the heuristic rule on the above three predicate argument structure and form the summary sentence that is compression version of the original source sentences. A1 for object.2.e.2.2.5.5.Sum]. The heuristic rule states that if the subjects in the predicate argument structures (PASs) refer to the same entity. we consider specific arguments i.2. we used 100 maximum generations. then merge the predicate argument structures by removing the subject in all PASs except the first one. During summary sentence generation process through SimpleNLG.4.3. to generate summary sentences from PASs. According to the rule stated above. When the process finishes. the subject A0 is identified as repeated in the above example and is eliminated from all predicate argument structures except the first one. P2 and P3 are obtained as follows: P1: [A0: Hurricane Gilbert] [V: claimed] [A1: to be the most intense storm on record] P2: [A0: Hurricane Gilbert] [V: slammed] [A1: into Kingston] [AMTMP: on Monday] P3: [A0: Hurricane Gilbert] [V: ripped] [A1:roofs off homes and buildings] We assume that P1. Summary sentence Hurricane Gilbert claimed to be the most intense storm on record. The realisation engine has excellent coverage of English syntax and morphology. tense. (22) is used to determine the score of predicate argument structures in each cluster. For instance the source sentence “Floods prevented officials from reaching the hotel zone in Cancun and there were no relief efforts under way by late Wednesday”. The process of SimpleNLG engine consists of three main steps as depicted in Fig. Stochastic universal sampling (SUS) method [39] has been used to select parents from the current population. arguments. Khan et al. After applying SENNA SRL. the individual chromosome with the best fitness value is chosen as the optimal feature weights. where M is the number of required selections and Sum is the sum of the scaled fitness values over all the individual chromosomes in the current population.6. 2. The next example demonstrates how the SimpleNLG compresses a single sentence from constituents (semantic arguments) of the sentence predicate.e.1. Process of SimpleNLG engine. The cross over operation produces new chromosomes from the two parent chromosome by exchanging information in the parent chromosomes.5. the corresponding three predicate argument structures P1. A0 for subject. As discussed in Section 2. This method uses M equally spaced steps in the range [0. Wk is the weight of feature k.2. Moreover. Score(Pi ) = 10  Wk xP Fk (Pi ) 743 Initialization of Basic Constituents Set Features of Constituents Combine Constituents into larger Structures using Simple NLG Realiser Fig.A. and at the same time offers API to users. S3: Hurricane Gilbert ripped roofs off homes and buildings. A2 for indirect object as core S1: Hurricane Gilbert claimed to be the most intense storm on record in terms of barometric pressure. ArgM-TMP for time as adjunctive arguments for predicate (Verb) V while the rest of the arguments are ignored. Once the feature weights are obtained. we use two reproduction operations i. Selection. slammed into Kingston on Monday and ripped roofs off homes and buildings. 2. This phase takes top scored predicate argument structures (PASs) from previous phase. Mutation operation is performed by altering gene of chromosome by another to generate a new genetic structure. Termination criteria. Reproduction operations. the corresponding two predicate argument structures P1 and P2 obtained from the above sentence are as follows: .

6 7 and 8. from which this PAS is extracted. Floods prevented officials from reaching the hotel zone in Cancun. 9. Step 9: This step uses realiser object created in step3. 3.setVerb (“prevented”). which takes different constituents stored in object p obtained in steps 5. The output variable will contain the generated sentence i. Lexicon lexicon = Lexicon.setIndirect Object (“from reaching the hotel zone in Cancun”). Khan et al. A1. 8. we can create an NLGFactory object to create SimpleNLG structures. A2) of predicate (V: prevented) in P1 as constituents to form the resultant sentence as shown in the sample code below.setObject (“officials”).realsieSentence(p).1. the summary sentence is generated from more than one predicate argument structures then its probable position in the summary is determined based on the position of the first predicate argument structure. The proposed framework is evaluated with DUC 2002 data set [38]. p.3). 2. Sample Code Listing 1. Results and discussion 3.createClause(). In semantic similarity matrix computation phase (as discussed in Section 2. / Applied Soft Computing 30 (2015) 737–747 P1: [A0: Floods] [V: prevented] [A1: officials] [A2: from reaching the hotel zone in Cancun] P2: [A0: officials] [V: reaching] [A2: the hotel zone in Cancu] Step 8: This step defines the constituent “from reaching the hotel zone in Cancun” along with the specified indirect object feature. Realiser realiser = new Realiser (lexicon).setIndirectObject(“from reaching the hotel zone in Cancun”). (2) fully automatic summarization of multiple newswire/newspaper documents on single subject by generating document extracts and (3) fully automatic summarization of multiple newswire/newspaper documents on single subject by generating document abstracts. single document abstracts.setVerb(“prevented”). Step 6: This step defines the constituent “prevented” along with the specification of verb feature. The position of predicate argument structure (PAS) is determined from the position of source sentence in the document collection. the input source sentence is represented by one representative predicate argument structure P1 as discussed in Section 2. Each document set contains documents. DUC 2002 data set contains 59 document sets produced by the National Institute of Standards and Technology (NIST). This step creates a realiser object that will transform/combine SimpleNLG structures (constituents of sentence) into text. Once the similarity matrix for PASs is obtained.getDefaultlexicon(). Step 5: This step defines the constituent “Floods” along with the specification of subject feature. SPhraseSpec p = nlgFactory.setSubject (“Floods”).setObject(“officials”). they are re-arranged based on the position of predicate argument structures. Experimental settings This section illustrates the evaluation of our proposed framework for abstractive summarization. p. 7. p. p. p. String output=realiser. The SimpleNLG will take the semantic arguments (A0. p. However. which allows us to define the syntactic constituents that will form a sentence. 6. The Document Understanding Conference (DUC) is a standard corpus used in text summarization research. 3.realiseSentence(p). The second predicate argument structure P2 is ignored as it is subsumed by the first predicate argument structure P1. Realiser realiser = new Realiser(lexicon).744 A. NLGFactory nlgFactory = new NLGFactory(lexicon). we perform agglomerative hierarchical clustering algorithm based on average linkage method to cluster semantically similar predicate argument structures as . which contains documents along with their human model summaries. 5.2. and accessed via: Lexicon lexicon = Lexicon. Once we get a lexicon. Simple NLG comes with a built in lexicon that contain information (part of speech) about words.setSubject(“Floods”). The stepwise detail of the above code is as follows: Step 1. The most suitable data set chosen for our work is DUC 2002 as it refers to task3 (multidocument abstractive summarization on single subject) defined only for DUC 2002. The SENNA semantic role labeler is employed to extract predicate argument structure from each sentence in the document collection. and finally the sentence is realized to make the result syntactically and morphologically correct: String output = realiser. Step 3. p. Once the summary sentences are generated from predicate argument structures. 4. Step 4: This step creates a simple NLG construct SPhraseSpec by calling a method of nlgFactory object. Step 2.getDefaultLexicon(). assemble them using simple grammar rules embodied in simple NLG. In case. p.e. Hence. the document collection is segmented into sentences in such a way that each sentence is preceded by its corresponding document number and sentence position number. the similarities of predicate argument structures (PASs) are computed by comparing them pair wise based on Jiang’s semantic similarity measure and edit distance algorithm. the rest of DUC data sets deal with other summarization tasks such as question-answering and update summarization.createClause(). Step 7: This step defines the constituent “officials” along with the specified object feature. Three tasks were evaluated in DUC-2002: (1) fully automatic summarization of single newswire/newspaper document. NLG Factory nlg Factory = new NLG Factory(lexicon). At first. SPhraseSpec p=nlgFactory. and multi-document abstracts/extracts with sets defined by different types of criteria such as event sets and biographical sets.

Previous research studies showed that ROUGE metric was employed for the evaluation of extractive summaries.70 0. using 59 news articles/data sets provided by the Document Understanding Evaluations 2002 (DUC.50 0. Best and Avg) besides the average of human model summaries (Models). The fitness function evaluates the quality of each chromosome by determining its fitness value. The two standard evaluation metrics. In order to allow the fitness value to converge. Results The results of optimal feature weighting using genetic algorithm are depicted in Fig. Table 2 shows the mean coverage score and average precision obtained on DUC 2002 dataset for our proposed framework (AS GA SRL).44 0. It could be observed 3. The training and testing of GA is performed on 59 multi-documents (obtained from DUC 2002) using 10-fold cross validation. we run 100 maximum generations before terminating the GA process. and the average of human model summaries (Models).7118985.614672. Khan et al. Method Mean coverage score Average precision Models AS GA SRL AS SRL Best Avg 0. The obvious advantage of Pyramid metric over the ROUGE is that it can capture different sentences in the summaries that uses different words but express similar meanings [41].24141418.67 systems participating in multi-document abstractive summarization task. ROUGE score is the n-gram exact matches between system summary and human model summaries. Table 2 presents the comparative evaluation of proposed framework and other summarization systems based on mean coverage score and precision measures. 2002). For each data set. .28 0. 3. 0. 0. the task undertaken by other Fig. therefore. the proposed framework without the integration of GA (AS SRL).69 0. 0.41313752. Precision = Number of Model SCUs expressed in Peer Summary Average SCU in the Peer Summary (24) 745 Fig. position. in the context of multi-document abstractive summarization task in DUC 2002. 5. The initial population consists of 50 chromosomes which were randomly initialized with the real values between 0 and 1. Optimal feature weights obtained using genetic algorithm. PAS to PAS similarity. 0. This metric cannot capture semantically equivalent sentences. frequent semantic term.18693995. we setup three comparison models (AS SRL. Mean Covarge Score = Total Peer SCUs Weight Average SCU in the Model Summary (23) where SCUs refers to the summary content units and their weights correspond to number of model (human) summaries they appeared in.60 0. The optimal feature weights obtained are 0. Note that proposed framework without the integration of GA assumes that all features have equal weights. 0. proper nouns. respectively. Pyramid score (Mean Coverage Score) [41] for peer summary or candidate summary is computed as follows.2.121493. 0. On other hand. our framework generates a 100 words summary. Selected chromosomes are then reproduced using cross over and mutation operations in each generation. the best chromosomes from the current population are selected as parents for the next generation. 5. we employ Pyramid evaluation metric for the evaluation of our proposed framework. numerical data. This metric measures the quality of system generated summary by comparing it with human model summaries. TF IDF. temporal feature. The selected PASs are fed to SimpleNLG realisation engine for summary sentence generation. The proposed framework is evaluated in the context of multi-document abstractive summarization task. 6. Comparison of summarization results based on mean coverage score and average precision. 0. the average of automatic systems (Avg) in DUC 2002. In order to distinguish among significant and less-significant features. we find optimize weights for features by running the genetic algorithm (GA). Since our work focus on multi-document abstractive summarization.3. / Applied Soft Computing 30 (2015) 737–747 discussed in Section 2. length. 6 visualizes the summarization results obtained with the proposed framework and other comparison models. Based on fitness value. The precision for peer summary [41] or candidate summary is computed as follows. The individual chromosome with the best fitness value is chosen as the optimal feature weights. The fitness function is the average recall obtained with each chromosome when summarization process is applied to the training corpus.475472.4. Discussion This section discusses the results presented in previous section. To compare the performance of our proposed framework (AS GA SRL). 0.A.17 0. nouns and verbs. Pyramid metric is used for the evaluation of abstractive summaries.351493.134772. 0.75 0. Fig. the best automatic summarization system (Best) in DUC 2002. For comparative evaluation.85 0.21308642 correspond to the weights for title. Table 2 Comparison of multi-document abstractive summarization results in DUC 2002 based on mean coverage score and average precision. Recall-OrientedUnderstudy for Gisting Evaluation (ROUGE) [40] and Pyramid [41] have been widely used in the context of evaluation of text summary. The optimized features weights are then used to score and select the top ranked predicate argument structure from each cluster.

Bhattacharyya. 90–93. An efficient concept-based retrieval model for enhancing text retrieval quality. Lee.J. [28] F. Collobert. 23 (2009) 126–144. Lim. Contreras. 142–149. P. 2012. V. Kang.60) on precision. McLean. Genest. Johansson. The results indicate that the integration of GA in the proposed SRL based framework improves summarization results when compared with the SRL based framework without GA. The automatic creation of literature abstracts. J. 31 (2005) 297–328. Comput. [5] H. in: Proceedings of the 12th European Workshop on Natural Language Generation. R. pp. Commun.4690. MIT Press.F. pp. [25] Y. Fayyad. [2] J. in: ACL HLT 2011. [6] P. D. 2000. M. Semantic argument frequency-based multi-document summarization. McKeown.R. 548–555. and 0. in: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers – vol. McKeown. E. [23] C. D. A Bayesian approach to unsupervised semantic role induction. 703–710. Syst. 12 (2011) 2493–2537. and Retrieval of Information by Computer. Snowball: A Language for Stemming Algorithms. Bae. on precision. and called it AS SRL. Mach. Text Categorization Using Predicate–Argument Structures. D.-M. 2011. K. Tanaka. [18] N. J. SimpleNLG: a realisation engine for practical applications. 2000. we also carried out statistical significance tests (T-Tests) to show the improvement of our proposed framework (AS GA SRL) with other comparison model (AS SRL).M.05) show that there is a significant difference between the results of the proposed framework (AS GA SRL) and the other comparison model (AS SRL). Miller. Knowl. 355–366. pp. that the performance of the framework dropped by 12% (from 0. [22] A. Miyamoto. arxiv:0906. p. Kavukcuoglu. Genest. Kupiec. [20] R. 32 (1989) 220–227. Addressing the variability of natural language expression in sentence similarity with semantic structure of the sentences. Salim. Kato. The low significance values for the T-test (typically less than 0. Jian. G.P. Linguist. [13] C. Discov. respectively. [4] B. WordNet: a lexical database for English. Man Cybern. Hierarchical clustering algorithms for document datasets. A fuzzy ontology and its application to news summarization. ACM 38 (1995) 39–41. we plan to integrate graph with SRL to build a semantic graph for multi-document abstractive summarization. 2008.-E. in: Proceedings of the 2009 Workshop on Language Generation and Summarisation. Dive deeper: deep semantics for sentiment analysis. in: Advances in Knowledge Discovery and Data Mining. Inf. T. Reiter. Conrath. 354–358.A. Barzilay. Shehata. IEEE Trans. IBM J. Weston. Framework for abstractive summarization using textto-text generation. Takumi. M. 35 (2013) 411–434. in: 2012 Seventh International Conference on Computer Engineering & Systems (ICCES). Uysal. Bottou. P. Khan et al. 10 (2005) 141–168. References [1] R. Luhn. Larsen. pp. in: Proceedings of the 22nd International Conference on World Wide Web.W. Lacatusu. [29] S. arxiv:1105. Murtagh. . Marcu. Fattah. SRL–GSM: a hybrid approach based on semantic role labeling and general statistic method for text summarization. Aref. In order to validate the results. I. ACL 2014 (2014) 113. pp. 2012. 64–73. Barzilay. Salim. Springer. T. FFNN. K. Syntax-driven sentence revision for broadcast news summarization. 2012. [15] S. 1997 arxiv:cmp-lg/9709008. Kobayakawa.A. F. in: 2012 IEEE International Conference on Granular Computing (GrC). B: Cybern. 1989. Best and Avg). 15 (2003) 871–882. Karray. we dropped GA component from the proposed framework for multidocument abstractive summarization.44) on mean coverage score and 14% (from 0. 4. Semantic Similarity Based on Corpus Statistics and Lexical Taxonomy.F. pp. 2 (1958) 159–165. Achananuparp. 2. Ren. Titov. Karypis. and then the optimized feature weights were used to select the top ranked predicate argument structures from each cluster for summary generation phase. [16] L. the proposed framework (AS GA SRL) outperforms the comparison models and came second to the average of human model summaries (Models). IEEE Trans. Gemulla. I.70 to 0. Jadhav.-K. We exploited GA in our proposed framework to determine the importance of each feature by assigning them appropriate weights. Kinoshita. ClausIE: clause-based open information extraction. Bandar. L. 71. N. 39–47. Information fusion in the context of multi-document summarization. It could be observed from Table 2.50 to 0. 75. Lapalme. Res.-E. Moawad. G. 2002. pp. pp. Suanmali. A. V. 460–464. J. Appl. P. Zhao. 1. Lapalme. [17] J. Can. Methods of Hierarchical Clustering. in: Document Understanding Conferences. in: Information Retrieval Technology. Gatt. Kuksa. Salton. 2009. We believe that the proposed work aims at the real goals of automatic summarization – controlling the content and structure of the summary. 550–557. Syst. 12–22.S. N. Karypis. Towards a framework for abstractive summarization of multimodal documents. P. Knight. Speech Lang. Generating single and multi-document summaries with GISTEXTER. 2012. J. G. Steinbach. Willett. Li. Binwahlan. Statistics-based summarization-step one: sentence compression. [10] S. pp. In order to investigate the impact of GA on summarization. Pedersen. Data Mining Knowl. F. pp. Karlen. [21] J. [8] R. A trainable summarizer with knowledge acquired from robust NLP techniques. 2009. Z.-H. the performance of the proposed framework (AS GA SRL) drops as compare to the Best system but still better than average of automatic systems (Avg) and came third to the average of human model summaries (Models). [32] A.C. Greenbacker. Harabagiu. J. [30] M. Klementiev. [36] G. 459–464. in: KDD Workshop on Text Mining. Del Corro.R. Kumano. M. 1995. Lee. M. Knowl. Nugues. [34] M. GA. A trainable document summarizer. PNN and GMM based models for automatic text summarization.S. Acknowledgement This research is supported by the Universiti Teknologi Malaysia (UTM). Gur. F. in: Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics. K. [31] Y. 132–138. 68–73.031. Addison-Wesley. in: Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics on Computational Linguistics. MR. T. Comparison of hierarchic agglomerative clustering methods for document retrieval. [12] P.-S. pp. However. 82–93. An approach for measuring semantic similarity between words using multiple information sources. 2011. Res. Natural language processing (almost) from scratch. Fully abstractive approach to guided summarization. in: Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. Kumar. C. / Applied Soft Computing 30 (2015) 737–747 from the results in Table 2 that on mean converge score. our proposed framework shows the feasibility of this new direction for summarization research. in: Advances in Automatic Text Summarization. X. Z. Top-down vs bottom-up methods of linkage for asymmetric agglomerative hierarchical clustering. Experimental results show that the proposed framework outperforms the other comparison models and came second to the average of human model summaries. pp. Analysis. 2009. 2001. pp. in: Proceedings of the National Conference on Artificial Intelligence.-S. Bugdayci. [9] H. Semantic graph reduction approach for abstractive text summarization. A comparison of document clustering techniques. Dev. [14] I. Kamel. pp. [26] G. This proves that integration of GA into the proposed framework for abstractive summarization (AS SRL) improves the summarization results. These results suggest that the summary produced by our proposed framework (AS GA SRL) is more akin to way humans produce summary as compared to other comparison models (AS SRL. Huang. Automatic Text Processing: The Transformation. [7] I. 2009. [33] L. Chen. L. Springer. J. 2005. 2013.A. pp. pp. in: Proceedings of the Workshop on Monolingual Text-ToText Generation. 2009. Aksoy. Conclusion Although fully abstractive summarization is a big challenge. Jiang. F. Data Eng. A. Hu.021 for mean coverage and precision. [11] C.0121. G. F. Fuzzy Logic Based Method for Improving Text Summarization. Persson. Sentence extraction using time features in multi-document summarization. Porter. [35] J. pp. Comput. M. in: 24th International Symposium on Computer and Information Sciences 2009 (ISCIS 2009). Comput.746 A. Elhadad. [27] P. R. Sentence fusion for multidocument news summarization. U. El-Hamdouchi. [19] N.F. 1999.-W. vol. 35 (2005) 859–880. The Paired-Samples T-Test procedure is used to compare the means of two results that represent the same test group and obtained low significance values of 0. In future. 525–526. [24] M. Learn. [3] K. Moreover. A. Yang. J. S. 10 (2010) 166–173. 1999. Sci. 2011.

/ Applied Soft Computing 30 (2015) 737–747 [37] M. Lin. Genetic algorithms: a survey. 2004. Patnaik. [41] A. Passonneau. L. R. Reducing bias and inefficiency in the selection algorithm. [39] J. Evaluating Content Selection in Summarization: The Pyramid Method. 2002. pp. in: Text Summarization Branches Out: Proceedings of the ACL-04 Workshop.-Y. Srinivas. 14–21. 747 [40] C.A. Khan et al. pp. . in: Proceedings of the Second International Conference on Genetic Algorithms. Nenkova. Baker. Computer 27 (1994) 17–26.E. 2004. [38] DUC. 74–81.M. 1987. Rouge: a package for automatic evaluation of summaries. Document Understanding Conference 2002.