This action might not be possible to undo. Are you sure you want to continue?
HTTPS://SITES.GOOGLE.COM/SITE/JOURNALOFCOMPUTING/
WWW.JOURNALOFCOMPUTING.ORG 49
Abstract— As the number of objectoriented software systems increases, it becomes more important for organizations to maintain
those systems effectively. However, currently only a small number of maintainability prediction models are available for object
oriented systems. In this paper, we develop an extreme learning machine (ELM) maintainability prediction model for object
oriented software systems. The model is based on extreme learning machine algorithm for singlehidden layer feedforward neural
networks (SLFNs) which randomly chooses hidden nodes and analytically determines the output weights of SLFNs. The model is
constructed using popular objectoriented metric datasets, collected from different objectoriented systems. Prediction accuracy of
the model is evaluated and compared with commonly used regressionbased models and also with Bayesian network based model
which was earlier developed using the same datasets. Empirical results from the simulation show that our ELM based model pro
duces promising results in terms of prediction accuracy measures that are better than most of the other earlier implemented mod
els on the same datasets.
Index Terms— Software maintainability, Extreme Learning Machines, Bayesian Network, Regression
——————————
——————————
I. INTRODUCTION
Software maintainability is defined as the ease of finding
and correcting errors in the software [1]. It is analogous to
the hardware quality of MeanTimeToRepair, or MTTR.
While there is as yet no way to directly measure or predict
software maintainability, there is a significant body of
knowledge about software attributes that make software eas
ier to maintain. These include modularity, self (internal)
documentation, code readability, and structured coding
techniques. These attributes also improve sustainability, the
ability to make improvements to the software [1].
It is arguable that many objectoriented (OO) software
systems are currently in use. It is also arguable that the
growing popularity of OO programming languages, such as
Java, as well as the increasing number of software develop
ment tools supporting the Unified Modeling Language
(UML), encourages more OO systems to be developed at
present and in the future. Hence it is important that those
systems are maintained effectively and efficiently. A soft
ware maintainability prediction model enables organizations
to predict maintainability of a software system and assists
them to manage and plan their maintenance resources.
In addition, if an accurate maintainability prediction mod
el is available for a software system, a defensive design can
be adopted. This would reduce future maintenance effort of
the system. Maintainability of a software system can be
measured in different ways. Maintainability could be meas
ured as the number of changes made to the code during a
maintenance period or be measured as effort to make those
changes [1][5]. The predictive model is called a maintenance
effort prediction model if maintainability is measured as
effort.
In this research we developed a maintainability prediction
model for an objectoriented software system based on the
recently introduced learning algorithm called extreme learn
ing machine (ELM) for singlehidden layer feedforward
neural networks (SLFNs) which randomly chooses hidden
nodes and analytically determines the output weights of
SLFNs. In theory, this algorithm tends to provide good ge
neralization performance at extremely fast learning speed.
The experimental results, found in literatures, based on a
few artificial and real benchmark function approximation
and classification problems including very large complex
applications, and particularly the empirical results from this
study, demonstrated that the ELM can produce good genera
lization performance in most cases and can learn thousands
of times faster than conventional popular learning algo
rithms for feedforward neural networks.
Despite the importance of software maintenance, It is un
fortunate that little work has been done as regards develop
ing predictive models for software maintainability, particu
larly objectoriented software system, which is evident in the
fewer number of software maintainability prediction mod
els, that are currently available in the literature.
We have developed a maintainability prediction model for
an objectoriented software system based on the recently
introduced learning algorithm called extreme learning ma
chine (ELM). Implementation was carried out on representa
tive datasets related to the target systems. Furthermore, we
performed comparative analysis between our model and the
models presented by Koten and Gray [5], which include
RegressionBased and Bayesian Network Based models, in
terms of their performance measures values, as recommend
ed in the literatures.
Furthermore, the usefulness of the Extreme Learning Ma
chines in the area of software engineering and, in particular,
maintainability prediction for an objectoriented software
Extreme Learning Machine as Maintainability Prediction model for
ObjectOriented Software Systems
S. O. Olatunji*
1
, Z. Rasheed*
2
, K.A. Sattar*
3
, A. M. AlMana*
4
, M. Alshayeb*
5
, E.A. ElSebakhy
#6
*
Information and Computer Science Department, King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia
#
Senior Scientist of Artificial Intelligence & Data Mining in Business and Health Science, MEDai Inc. an Elsevier Company, Millenia Park One,
4901 Vineland Road, Suite 450, Orlando, Florida 32811, USA
JOURNAL OF COMPUTING, VOLUME 2, ISSUE 8, AUGUST 2010, ISSN 21519617
HTTPS://SITES.GOOGLE.COM/SITE/JOURNALOFCOMPUTING/
WWW.JOURNALOFCOMPUTING.ORG 50
system, has been made clearer by describing both the steps
and the use of Extreme Learning Machines as an artificial
intelligence modeling approach for predicting the maintai
nability of objectoriented software system.
The rest of this paper is organized as follows. Section II
contains review of related earlier works. Section III dis
cusses background information that includes software main
tainability, some prediction modeling techniques, and also
describes the main modeling technique used: extreme learn
ing machine (ELM). Section IV presents the OO software
data sets used in our study and their description. It also con
tains data description, data analysis based on skewness, and
types of skewness. Section V presents the research ap
proach. Section VI contains model evaluation, prediction
accuracy measures, empirical results, comparison with other
models and discussions. Section VII concludes the paper and
outlines directions for future work.
II. RELATED WORK
Many object oriented software maintainability prediction
models were developed in the last decade; however, most of
them suffer from low prediction accuracies [9]. In fact, most
of the research work found in the literature related to soft
ware maintainability prediction of object oriented software
products either make use of statistical models or artificial
neural networks, though some recent work have been done
using some other artificial intelligence techniques such as
Bayesian Belief Networks (BBN) [5], and Multivariate
adaptive regression splines (MARS) [9].
Regression techniques have been thoroughly utilized to
predict maintainability of object oriented software systems
[7] [17].
In addition, several types of artificial neural networks
were also employed in predicting the maintainability effort
of object oriented software systems. Ward neural network
and General Regression neural network (GRNN) was used
to predict the maintainability effort for object oriented soft
ware systems using object oriented metrics [13]. On the oth
er hand, BackPropagation MultiLayer Perceptron (BP
MLP) has been used to predict faulty classes in object
oriented software [11]. In the same research work, they used
radial basis function networks (RBF) to predict the type of
fault a faulty class has.
Bayesian Belief Networks (BBN) was suggested as a
novel approach for software quality prediction by Fenton et
al. [12] and Pearson [20]. They build their conjecture based
on Bayesian Belief Networks’ ability in handling uncertain
ties, incorporating expert knowledge, and modeling the
complex relationships among variables. However, a number
of researchers [2][10] have introduced several limitations
with Bayesian Belief Networks when they are applied as a
model for object oriented software quality and maintainabili
ty perdition. Recently, Koten and Gray used a special type
of Bayesian Belief Networks called NaiveBayes classifier
[5] to implement a BayesianBeliefNetworksbased soft
ware maintainability prediction model. Although their re
sults showed that their model give better results than regres
sionbased techniques for some datasets, the model is still
inferior to regressionbased techniques for some other data
sets.
Zhou and Leung used Multivariate Adaptive Regression
Splines (MARS) to predict object oriented software maintai
nability [9]. In their research work, they compared MARS
with multivariate linear regression (MLR), artificial neural
networks (ANN), regression trees (RT), and support vector
machines (SVM). However, their MARSbased technique
does not outperform the compared techniques for all datasets
they used.
III. BACKGROUND
This section will talk about the software maintainability
and its types, previously held models used to predict soft
ware maintainability and the novel approach called Extreme
Learning Machine used in this context.
A. Software Maintainability
Software maintainability is the process of modification of
a software product after delivery to correct faults, to improve
performance or other attributes, or to adapt the product to a
changed environment [6]. Maintaining and enhancing the
reliability of software during maintenance requires that
software engineers understand how various components of a
design interact. People usually think of software mainten
ance as beginning when the product is delivered to the
client. While this is formally true, in fact decisions that af
fect the maintenance of the product are made from the earli
est stage of design.
Software maintenance is classified into four types: correc
tive, adaptive, perfective and preventive [1]. Corrective
maintenance refers to fixing a program. Adaptive mainten
ance refers to modifications that adapt to changes in the data
environment, such as new product codes or new file organi
zation or changes in the hardware of software environments.
Perfective maintenance refers to enhancements: making the
product better, faster, smaller, better documented, cleaner
structured, with more functions or reports. The preventive
maintenance is defined as the work that is done in order to
try to prevent malfunctions or improve maintainability.
When a software system is not designed for maintenance,
it exhibits a lack of stability under change. A modification in
one part of the system has side effects that ripple throughout
the system. Thus, the main challenges in software mainten
ance are to understand existing software and to make
changes without introducing new bugs.
B. Regression Based Models
Regression models are used to predict one variable from
one or more other variables. Regression models provide the
scientist with a powerful tool, allowing predictions about
past, present, or future events to be made with information
about past or present events.
1) Multiple Linear Regression Model
Multiple linear regression attempts to model the relation
ship between two or more explanatory variables and a re
JOURNAL OF COMPUTING, VOLUME 2, ISSUE 8, AUGUST 2010, ISSN 21519617
HTTPS://SITES.GOOGLE.COM/SITE/JOURNALOFCOMPUTING/
WWW.JOURNALOFCOMPUTING.ORG 51
sponse variable by fitting a linear equation to observed data.
Every value of the independent variable x is associated with
a value of the dependent variable y. The regression line for p
explanatory variables
1 2
, ,......,
p
x x x is defined to
be
0 1 1 2 2
....
y p p
x x x u     = + + + + . This line de
scribes how the mean response
y
u changes with the expla
natory variables. The observed values for y vary about their
means
y
u and are assumed to have the same standard devi
ationo . Formally, the model for multiple linear regression
given n observations is
0 1 1 2 2
....
i i i p ip i
y x x x     c = + + + + +
for i = 1,2,….n
where
i
c is notation for model deviation.
One approach to simplifying multiple regression equa
tions is the stepwise procedures. These include forward se
lection, backwards elimination, and stepwise regression.
They add or remove variables oneatatime until some
stopping rule is satisfied.
2) Forward selection
Forward selection starts with an empty model. The varia
ble that has the smallest P value when it is the only predictor
in the regression equation is placed in the model. Each sub
sequent step adds the variable that has the smallest P value
in the presence of the predictors already in the equation.
Variables are added oneatatime as long as their P values
are small enough, typically less than 0.05 or 0.10.
3) Backward elimination
It starts with all of the predictors in the model. The varia
ble that is least significant that is, the one with the largest P
value is removed and the model is refitted. Each subsequent
step removes the least significant variable in the model until
all remaining variables have individual P values smaller than
some value, such as 0.05 or 0.10.
4) Stepwise regression
This approach is similar to forward selection except that
variables are removed from the model if they become non
significant as other predictors are added.
Backwards elimination has an advantage over forward se
lection and stepwise regression because it is possible for a
set of variables to have considerable predictive capability
rather than any individual subset. Forward selection and
stepwise regression will fail to identify them because some
times variables don't predict well individually and Backward
elimination starts with everything in the model, so their joint
predictive capability will be seen.
C. Bayesian Networks
A Bayesian network consists of nodes interconnected by
the directed links forming directed acyclic graph. In this
graph, nodes represent random variables (RVs) and links
correspond to direct probabilistic influences. The RVs cor
respond to important attributes of the modeled system which
exemplifying the system’s behavior. Directed connection
between the two nodes indicates a casual effect between
RVs which associated with these nodes.
The structure of directed acyclic graph states that each
node is independent of all its non descendants conditioned
on its parent nodes. In other words, the Bayesian Network
represents the conditional probability distribution
P(Y/X1,…,Xn) which is used to quantify the strength of
variables Xi on the variable Y, Nodes Xi are called the par
ents of Y and Y is called a child of each Xi. This should be
noted that outcomes of the events for the variables Xi have
an influence on the outcome of the event Y.
D. Extreme Learning Machine
In general, the learning rate of feedforward neural net
works (FFNN) is timeconsuming than required. Due to this
property, FFNN is becoming bottleneck in their applications
limiting the scalability of them. According to [4], there are
two main reasons behind this behavior, one is slow gradient
based learning algorithms used to train neural network (NN)
and the other is the iterative tuning of the parameters of the
networks by these learning algorithms. To overcome these
problems, [2][4] proposes a learning algorithm called ex
treme learning machine (ELM) for single hidden layer feed
forward neural networks (SLFNs) which randomly selected
the input weights and analytically determines the output
weights of SLFNs. It is stated that “In theory, this algorithm
tends to provide the best generalization performance at ex
tremely fast learning speed” [4].
This is extremely good as in the past, it seems that there
exists an unbreakable virtual speed barrier which classic
learning algorithms cannot go break through and therefore
feedforward network implementing them take a very long
time to train itself, independent of the application type
whether simple or complex. Also ELM tends to reach the
minimum training error as well as it considers magnitude of
weights which is opposite to the classic gradientbased
learning algorithms which only intend to reach minimum
training error but do not consider the magnitude of weights.
Also unlike the classic gradientbased learning algorithms
which only work for differentiable activation functions ELM
learning algorithm can be used to train SLFNs with non
differentiable activation functions. According to [4], “Unlike
the traditional classic gradientbased learning algorithms
facing several issues like local minimum, improper learning
rate and overfitting, etc, the ELM tends to reach the solu
tions straightforward without such trivial issues”.
The ELM has several interesting and significant features
different from traditional popular gradientbased learning
algorithms for feed forward neural networks: These include:
The learning speed of ELM is extremely fast. In simula
JOURNAL OF COMPUTING, VOLUME 2, ISSUE 8, AUGUST 2010, ISSN 21519617
HTTPS://SITES.GOOGLE.COM/SITE/JOURNALOFCOMPUTING/
WWW.JOURNALOFCOMPUTING.ORG 52
tions reported by Huang et al. [2], the learning phase of
ELM can be completed in seconds or less than seconds for
many applications. Previously, it seems that there exists a
virtual speed barrier which most (if not all) classic learning
algorithms cannot break through and it is not unusual to take
very long time to train a feedforward network using classic
learning algorithms even for simple applications.
The ELM has better generalization performance than the
gradientbased learning such as back propagation in most
cases.
The traditional classic gradientbased learning algorithms
and some other learning algorithms may face several issues
like local minima, improper learning rate and over fitting,
etc. In order to avoid these issues, some methods such as
weight decay and early stopping methods may need to be
used often in these classical learning algorithms. The ELM
tends to reach the solutions straightforward without such
trivial issues. The ELM learning algorithm looks much
simpler than most learning algorithms for feedforward
neural networks.
Unlike the traditional classic gradientbased learning al
gorithms which only work for differentiable activation func
tions, as easily observed the ELM learning algorithm could
be used to train SLFNs with many nondifferentiable activa
tion functions [3].
1) How Extreme Learning Machine Algorithm
Works
Let us first define the standard SLFN (singlehidden layer
feedforward neural networks). If we have N samples (x
i
, t
i
),
where x
i
= [x
i1
, x
i2
, … , x
in
]
T
e
R
n
and t
i
= [t
i1
, t
i2
, … , t
im
]
T
e
R
n
, then the standard SLFN with N
~
hidden neurons and
activation function g(x) is defended as:
¯
=
= = +
N
i
j i j i i
N j o b x w g
~
1
, ... , 1 , ) . (  ,
where w
i
= [w
i1
, w
i2
, … , w
in
]
T
is the weight vector that con
nects the ith hidden neuron and the input neurons, β
i
= [β
i1
,
β
i2
, … , β
im
]
T
is the weight vector that connects the ith neu
ron and the output neurons, and b
i
is the threshold of the ith
hidden neuron. The “.” in w
i
. x
j
means the inner product of
w
i
and x
j
.
SLFN aims to minimize the difference between o
j
and t
j
.
This can be expressed mathematically as:
¯
=
= = +
N
i
j i j i i
N j t b x w g
~
1
, ... , 1 , ) . ( 
Or, more compactly, as:
H β = T
where
N N
N N N N
N N N
N N N
b x w g b x w g
b x w g b x w g
x x b b w w
~
~ ~
1 1
~ ~ ~
1 1 1
1
~
1
~
1
) . ( ... ) . (
.
.
.
...
.
.
.
) . ( ... ) . (
) ..., , , , ... , , , ... , (
×
+ +
+ +
= H
, β =
m N
T
N
T
×
~
~
1
.
.
.


and T =
m N
T
N
T
T
T
×
~
1
.
.
.
As proposed by Huang and Babri (1998), H is called the
neural network output matrix.
According to Huang et al. (2004), the ELM algorithm
works as follows:
Given a training set
( ) { } N i t x t x N
m
i
n
i i i
..., , 1 , ,  , = e e = R R ,
activation function g(x), and hidden neuron number = N
~
,
do the following:
Assign random value to the input weight w
i
and the bias
b
i
,
i = 1, … , N
~
Find the hidden layer output matrix H.
Find the output weight β as follows:
β = H
†
T
where β, H and T are defined in the same way they were
defined in the SLFN specification above.
JOURNAL OF COMPUTING, VOLUME 2, ISSUE 8, AUGUST 2010, ISSN 21519617
HTTPS://SITES.GOOGLE.COM/SITE/JOURNALOFCOMPUTING/
WWW.JOURNALOFCOMPUTING.ORG 53
IV. DATA DESCRIPTION
In this work, we made use of OO software datasets pub
lished by Li and Henry [7]. The datasets consist of five
C&K metrics: DIT, NOC, RFC, LCOM and WMC, and four
L&H metrics: MPC, DAC, NOM and SIZE2, as well as
SIZE1, which is a traditional lines of code size metric.
Those metric data were collected from a total of 110 classes
in two OO software systems: User Interface Management
System (UIMS) and Quality Evaluation System (QUES).
The codes were written in Classical − Ada
TM
. The UIMS
and QUES datasets contain 39 classes and 71 classes, re
spectively. Li and Henry measure maintainability in
CHANGE metric by counting the number of lines in the
code, which were changed during a threeyear maintenance
period. Neither UIMS nor QUES datasets contain actual
maintenance effort data. The same datasets are also used by
other researchers [5][6][7]. The description of each metric is
shown in Table I.
A. Data Analysis based on Skewness
Definition:
Skewness is a measure of symmetry, or more precisely,
the lack of symmetry. A distribution, or data set, is symme
tric if it looks the same to the left and right of the center
point. Skewness tells us about the direction of variation of
the data set.
Mathematical Expression:
The skewness of random variable X is defined as:
( )
3
3
1
1
N
i
i
X
Skewness
N
u
o
=
÷
=
¯
where X
i
is the random variable,
u
is the mean,
o
is the
standard deviation and N is the total length of random variable.
B. Types of Skewness:
Usually there are three types of skewness; Right, normal
and left skewness [21]. The distribution is said to be right
skewed if the right tail is longer and the mass of the distribu
tion is concentrated on the left. Similarly the distribution is
said to be leftskewed if the left tail is longer and the mass of
the distribution is concentrated on the right. Finally if the
skewness is nearly equal to zero, it is normally distributed
throughout the range. Acceptable range for normality is
skewness lying between 1 to 1. Figure 1 shows the three
types of skewness (right: skew >0, normal: skew ~ 0 and
left: skew <0).
In our experiment, we have generated the skew graphs of
some of our dataset fields. Two of the observations are
shown in figure 2 and figure 3.
V. RESEARCH APPROACH
For each data set, the available data was divided into two
parts. One part was used as a training set, for constructing a
maintainability prediction model. The other part was used
TABLE I
DATASET METRICS DESCRIPTION
Name Description
DIT
Depth of the inheritance tree (=
inheritance level number of the
class, 0 for the root class)
NOC
Number of children (= number of
direct subclasses that the class
has)
MPC
Messagepassing coupling(=
number of send statements de
fined in the class)
RFC
Response for a class (= total of
the number of local methods and
the number of methods called by
local methods in the class)
LCOM
Lack of cohesion of methods (=
number of disjoint sets of local
methods, i.e. number of sets of
local methods that do not interact
with each other, in the class)
DAC
Data abstraction coupling (=
number of abstract data types
defined in the class)
WMC
Weighted method per class (=
sum of McCabe’s cyclomatic
complexity of all local methods
in the class)
NOM
Number of methods (= number
of local methods in the class)
SIZE1
Lines of cod (= number of semi
colons in the class)
SIZE2
Number of properties (= total of
the number of attributes and the
number of local methods in the
class)
b f l h d h
JOURNAL OF COMPUTING, VOLUME 2, ISSUE 8, AUGUST 2010, ISSN 21519617
HTTPS://SITES.GOOGLE.COM/SITE/JOURNALOFCOMPUTING/
WWW.JOURNALOFCOMPUTING.ORG 54
for testing to determine the prediction ability of the devel
oped model. Although there are many different ways to split
a given dataset, we have chosen to use the stratify sampling
approach in breaking the datasets due to its ability to break
data randomly with a resultant balanced division based on
the supplied percentage. The division, for instance could be
70% for training set and 30% for testing set. In this work,
we selected 70% of the data for building the model (internal
validation) and 30% of the data for testing/ validation (ex
ternal validation or crossvalidation criterion). We repeated
both internal and external validation processes for 1000
times to have a fair partition through the entire process oper
ations.
It was ensured that the same percentage of division are
used, for each of the two dataset, in order to maintain the
comparability of prediction accuracy of the two datasets by
using the same proportion of the sample cases for learning
and testing.
We also evaluated and compared our developed model
with other OO software maintainability prediction models,
cited earlier, quantitatively, using the prediction accuracy
measures recommended in the literatures: absolute residual
(Ab.Res.), the magnitude of relative error (MRE) and the
proportion of the predicted values that have MRE less than
or equal to a specified value suggested in the literatures
(pred measures). Details of all these measures of perfor
mance are provided in next section.
VI. MODEL EVALUATION
A. Prediction accuracy measures
We compared the software maintainability prediction
models using the following prediction accuracy measures:
absolute residual (Abs Res), the magnitude of relative error
(MRE) and Pred measures.
The Ab.Res. is the absolute value of residual evaluated
by:
Ab.Res. = abs ( actual value − predicted value )
We used the sum of the absolute residuals (Sum Ab.Res.),
the median of the absolute residuals (Med.Ab.Res.) and the
standard deviation of the absolute residuals (SD Ab.Res.).
The Sum Ab.Res. measures the total residuals over the data
set. The Med.Ab.Res. measures the central tendency of the
residual distribution. The Med.Ab.Res. is chosen to be a
measure of the central tendency because the residual distri
bution is usually skewed in software datasets. The SD
Ab.Res. measures the dispersion of the residual distribution.
MRE is a normalized measure of the discrepancy between
actual values and predicted values given by
MRE = abs ( actual value − predicted value ) / actual val
ue
The Max.MRE measures the maximum relative discre
pancy, which is equivalent to the maximum error relative to
the actual effort in the prediction. The mean of MRE, the
mean magnitude of relative error (MMRE):
1
1
n
i
i
MMRE MRE
n
=
=
¯
According to [14], Pred is a measure of what proportion
of the predicted values have MRE less than or equal to a
specified value, given by:
Pred (q) = k / n
where q is the specified value, k is the number of cases
whose MRE is less than or equal to q and n is the total num
ber of cases in the dataset.
According to [15] and [8], in order for an effort prediction
model to be considered accurate, MMRE < 0.25 and/or ei
ther pred(0.25) > 0.75 or pred(0.30) > 0.70. These are the
suggested criteria in literature as far as effort prediction is
concerned.
B. Empirical Results, Comparison and Discus
sion
As stated earlier, In order to conduct a valid comparison,
our model, ELM, was obtained by training it on exactly the
same training set and evaluated on the same testing set
samples as used in the previous works cited earlier, particu
larly as contained in [5]. Below are tables and figures show
the results of our newly developed model in comparison to
the other earlier models used on the same data set.
1) Results from QUES dataset
Table II shows the values of the prediction accuracy
measures achieved by each of the maintainability prediction
models for the QUES dataset. Recall as quoted earlier that,
in order for an effort prediction model to be considered ac
curate, MMRE < 0.25 and/or either pred(0.25) > 0.75 or
pred(0.30) > 0.70, [15][8]. Hence the closer a model value is
to these baseline values, the better. Since Table II shows that
the Extreme learning machine model has achieved MMRE
value of 0.3502, the pred(0.25) value of 0.368 and the
pred(0.30) value of 0.380. Thus, the ELM is the only one
that is very close to the required value for MMRE which is
4.918, hence it is the best in term of MMRE and also in term
of the sum absolute residuals values. It is very close to satis
fying the criterion of an accurate prediction in term of
MMRE. In term of other measures, it competes favorably
with the other models.
In comparison with the UIMS dataset, the MMRE value
of 0.3502 is better, while the pred(0.25) and pred(0.30) val
ues are poorer. This indicates that the performance of the
extreme learning machine models may vary depending on
the characteristics of the dataset and/or depending on what
prediction accuracy measure is used.
JOURNAL OF COMPUTING, VOLUME 2, ISSUE 8, AUGUST 2010, ISSN 21519617
HTTPS://SITES.GOOGLE.COM/SITE/JOURNALOFCOMPUTING/
WWW.JOURNALOFCOMPUTING.ORG 55
Going by the sum absolute residuals values, we can see
that there is strong evidence that the Extreme learning ma
chine model’s value is significantly lower and thus, better
than those of the other models.
2) Results from UIMS dataset
Table III shows the values of the prediction accuracy
measures achieved by each of the maintainability prediction
models for the UIMS dataset. Table III shows that the Ex
treme learning machine model has achieved the MMRE val
ue of 0.968, the pred(0.25) value of 0.392 and the pred(0.30)
value of 0.450. Those values are one of the best among all
the five models presented in table III. Specifically, in term
of MMRE values, it is the best among all the models and we
can see that there is strong evidence that the Extreme learn
ing machine model’s value is significantly lower and thus,
better than those of the other models. In term of pred(0.25)
and pred(0.30), it is the second best model after Bayesian
network. In addition, it is also the best in the absolute resi
dual measures. The values of the absolute residuals have
again confirmed strong evidence that the differences of the
Extreme learning machine model from the other models are
significant.
Thus, it is concluded that the Extreme learning machine
model is able to predict maintainability of the UIMS dataset
better than the other models presented.
3) Discussion
With the exception of extreme learning machine that has
values closer to satisfying one of the criteria; none of the
maintainability prediction models presented get closer to
satisfying any of the criteria of an accurate prediction model
cited earlier. However, it is reported that prediction accura
cy of software maintenance effort prediction models are
often low and thus, it is very difficult to satisfy the criteria
[16].
Thus, we conclude that extreme learning machine model
presented in this paper can predict maintainability of the OO
software systems reasonably well to an acceptable degree.
This work shows that only extreme learning machine model
has been able to consistently perform better by having val
ues closer to satisfying one of the criteria laid down in litera
ture, MMRE, for both data set. For both QUES and UIMS
datasets, whenever the extreme learning machine model’s
prediction accuracy has not been as good as the other mod
els, it has been reasonably close. In terms of absolute resi
duals, ELM is better than other models for both datasets.
VII. CONCLUSION
An extreme learning machine OO software maintainabili
ty prediction model has been constructed using the OO
software metric data used by Li and Henry [7]. The predic
tion accuracy of the model is evaluated and compared with
TABLE II
PREDICTION ACCURACY FOR THE QUES DATASET
Model Max.
MRE
MMRE
Pred
(0.25)
Pred
(0.30)
Sum
Ab.Res.
Med.
Ab.Res.
SD
Ab.Res.
Bayesian
Network [5]
1.592 0.452 0.391 0.430 686.610 17.560 31.506
Regression
Tree [5]
2.104 0.493 0.352 0.383 615.543 19.809 25.400
Backward
Elimination [5]
1.418 0.403 0.396 0.461 507.984 17.396 19.696
Stepwise
Selection [5]
1.471 0.392 0.422 0.500 498.675 16.726 20.267
Extreme Learn
ing
Machine
1.803 0.3502 0.368 0.380 56.122 28.06 22.405
TABLE III
PREDICTION ACCURACY FOR THE UIMS DATASET
Model Max.
MRE
MMRE Pred
(0.25)
Pred
(0.30)
Sum
Ab.Res.
Med.
Ab.Re
s
SD
Ab.Res.
Bayesian
Network [5]
7.039 0.972 0.446 0.469 362.300 10.550 46.652
Regression
Tree [5]
9.056 1.538 0.200 0.208 532.191 10.988 63.472
Backward
Elimination [5]
11.890 2.586 0.215 0.223 538.702 20.867 53.298
Stepwise
Selection [5]
12.631 2.473 0.177 0.215 500.762 15.749 54.114
Extreme Learning
Machine
4.918 0.968 0.392 0.450 39.625 18.768 16.066
JOURNAL OF COMPUTING, VOLUME 2, ISSUE 8, AUGUST 2010, ISSN 21519617
HTTPS://SITES.GOOGLE.COM/SITE/JOURNALOFCOMPUTING/
WWW.JOURNALOFCOMPUTING.ORG 56
the Bayesian network model, regression tree model and the
multiple linear regression models using the prediction accu
racy measures: the absolute residuals, MRE and pred meas
ures. The results indicate that extreme learning machine
model can predict maintainability of the OO software sys
tems. For both datasets, the extreme learning machine mod
el achieved significantly better prediction accuracy, in term
of MMRE, than the other models, as it was closer to satisfy
ing one of the criteria, a fit, which none of the other models
have been able to achieve. Also, for both QUES and UIMS
datasets, whenever the extreme learning machine model’s
prediction measure accuracy has not been as good as the
best among the models, it has been reasonably competitive
against the best models.
Therefore, we conclude that the prediction accuracy of the
extreme learning machine model is better than, or at least, is
competitive against the Bayesian network model and the
regression based models. These outcomes have confirmed
that extreme learning machine is indeed a useful modeling
technique for software maintainability prediction, although
further studies are required to realize its full potentials as
well as reducing its shortcomings if not totally eradicated.
The results in this paper also suggest that the prediction
accuracy of the extreme learning machine model may vary
depending on the characteristics of dataset and/or the predic
tion accuracy measure used. This provides an interesting
direction for future studies. Another interesting direction
would be using the other variants of extreme learning ma
chine such as Evolutionary extreme learning machine [18]
and Fully complex extreme learning machine [19] for soft
ware effort prediction.
Acknowledgment
The authors would like to thank the anonymous review
ers for their constructive comments. The authors also ac
knowledge the support of King Fahd University of Petro
leum and Minerals in the development of this work.
REFERENCES
[1] Liguo Yu, S.R. Schach, Kai Chen, “Measuring the maintainability of
Open Source Software” IEEE 2005
[2] GuangBin Huang_, QinYu Zhu, CheeKheong Siew, “Extreme
learning machine: Theory and applications”, Neurocomputing 70
(2006) 489–501, ELSEVIER.
[3] G.B. Huang, Q.Y. Zhu, K.Z. Mao, C.K. Siew, P. Saratchandran, N.
Sundararajan, “Can threshold networks be trained directly?”, IEEE
Trans. Circuits Syst. II 53 (3) (2006) 187–191.
[4] G.B. Huang, Q.Y. Zhu, C.K. Siew, “Extreme learning machine: a
new learning scheme of feedforward neural networks”, in: Proceed
ings of International Joint Conference on Neural Networks
(IJCNN2004), 25–29 July, 2004, Budapest, Hungary.
[5] Chikako Van Koten, Andrew Gray,” An Application of Bayesian
Network for Predicting ObjectOriented Software Maintainability”,
The Information Science, Discussion Paper Series, 2006.
[6] Rikard Land, "Measurements of Software Maintainability IEEE
Standard Glossary of Software Engineering Terminology, report IEEE
Std 610.121990, IEEE, 1990.
[7] W. Li and S. Henry. Objectoriented metrics that predict maintainabil
ity, Journal of Systems and Software, 23:111–122, 1993.
[8] S.G. MacDonell. Establishing relationships between specification size
and software process effort in case environment. Information and
Software Technology, 39:35–45, 1997.
[9] Y. Zhou and H. Leung, "Predicting objectoriented software maintai
nability using multivariate adaptive regression splines," The Journal
of Systems and Software, vol. 80, 2007, pp. 13491361
[10] Kamaldeep Kaur, Arvinder Kaur, and Ruchika Malhotra "Alternative
Methods to Rank the Impact of Object Oriented Metrics in Fault Pre
diction Modeling using Neural Networks," Proceedings Of World
Academy Of Science, Engineering And Technology, vol. 13, 2006,
pp. 207212.
[11] S.Kanmani, V. Rhymend Uthariaraj, V. Sankaranarayanan, and P.
Thambidurai, "Object Oriented Software Quality Prediction Using
General Regression Neural Networks," ACM SIGSOFT Software En
gineering Notes, vol. 29, 2004, pp. 15.
[12] N.E. Fenton, P. Krause, and M. Neil, “Software Measurement: Uncer
tainty and Causal Modeling,” IEEE Software, vol. 10, no. 4, 2002, pp.
116122.
[13] M. M. T. Thwin and T.S. Quah,, “Application of Neural Networks
for predicting Software Development faults using Object Oriented
Design Metrics”, Proceedings of the 9th International Conference on
Neural Information Processing, November 2002, pp. 2312 – 2316.
[14] N.E. Fenton and S.L. Pfleeger. Software Metrics:A Rigorous & Prac
tical Approach. PWS Publishing Company, second edition, 1997.
[15] S.D. Conte, H.E. Dunsmore, and V.Y. Shen. Software Engineering
Metrics and Models. Benjamin/Cummings Publishing Company,
1986.
[16] A. De Lucia, E. Pompella, and S. Stefanucci. Assessing effort estima
tion models for corrective maintenance through empirical studies. In
formation and Software Technology, 47:3–15, 2005.
[17] F. Fioravanti and P. Nesi, "Estimation and Prediction Metrics for
Adaptive Maintenance Effort of ObjectOriented Systems'', IEEE
Transactions on Software Engineering, vol. 27, no. 12, 2001, pp.
1062–1084.
[18] QinYu Zhu, A.K. Qin, P.N. Suganthan, GuangBin Huang, Evolutio
nary extreme learning machine, Elsevier, Pattern Recognition 38
(2005) 1759 – 1763
[19] MingBin Li, GuangBin Huang_, P. Saratchandran, N. Sundararajan,
Fully complex extreme learning machine, Elsevier, Neurocomputing
68 (2005) 306–314
[20] N.E. Fenton and M. Neil, “A Critique of Software Defect Prediction
Research,” IEEE Transactions on Software Engineering, vol. 25, no.
5, 1999, pp. 675–689.
[21] PEARSON K., Contributions to the Mathematical Theory of Evolu
tion,II. Skew Variation in Homogeneous Material. Phil. Trans. Roy.
Soc. London (A.) 1895,186,343414.
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue reading from where you left off, or restart the preview.