Comparative Analysis of Machine Learning
Techniques for Fault Prediction
Prabhpahul Singh
Department of Computer Science
Jaypee Institute of Information Technology
NOIDA, India
pahulronaldo@gmail.com
Abstract— The quality of the software can be improved by
determining the faulty portions of the software in the initial
phases of software development. There are various machine
learning techniques in literature that can be used to create fault
prediction models using object oriented metrics. These models
will allow the developers and software practitioners to predict
faulty classes and concentrate the constraint resources in testing
these weaker portions of the software. In this work we analyze
and assess the predictive capability of six machine learning
techniques. The results are validated using seven open source
software.
Keywords— Faultt Prediction; Machine Learning Techniques;
Object-Oriented Metrics; Software Quality
I. INTRODUCTION
The prime aim of software industry is to develop effective
quality software products which fulfills its requirements and
has satisfied customers. However, in order to do so existing
faults in the software should be removed as early as possible.
A fault which is detected in the early phases of software life
cycle can be easily corrected as compared to a fault which was
found in the later phases of software development lifecycle
[1]. It has been ascertained that the cost to correct faults
increases exponentially, if it is detected in the later phases [1].
Thus, various researchers have rigorously developed and
evaluated several software fault prediction models, which are
capable of early detection of faults. Remedial actions can be
effectively taken by software developers to modify the
software product by removal of these faults. Moreover,
software managers will also be able to effectively plan
resource usage by assigning more resources to fault-prone
components of a software. These steps would ensure better
quality software products at optimum costs.
While developing software fault prediction models,
researchers have evaluated a wide range of software metrics.
These metrics include process metrics, procedural metrics or
Object-Oriented (OO) metrics [2]. Recent reviews on software
fault prediction studies have ascertained OO metrics to be
widely used in this domain [2-3]. Therefore, this study uses
Chidamber and Kemrer (CK) metrics suite [4], a popularly
used OO metrics suite for developing defect prediction models
[2-3]. The CK metrics suite contains metrics which represent
various structural properties of an OO software such as its
cohesion, size, reusability etc. In order to develop fault
Ruchika Malhotra
Discipline of Software Engineering
Department of Computer Science & Engineering
Delhi Technological University
Delhi-110042, India
ruchikamalhotra@dce.edu
prediction models, we also require fault data from previous
versions of the software product for training. Thus, we will
develop software fault prediction models using previous defect
data and OO metrics to identify components of a software
which are prone to faults in the upcoming releases of the
software.
A key factor while developing software fault
prediction models is the use of an efficient modeling
technique. These modeling techniques are classification
algorithms which learn from the provided historical fault data
of the software and identify faulty classes in the new versions
on the basis of their learning. Traditionally, statistical
techniques such as logistic regression (LR) were used for
modeling software prediction models. However, various fault
prediction studies in literature have advocated the use Machine
Learning (ML) techniques for this task [2,5]. ML techniques
are capable in extraction of worthy information from complex
or difficult problem scenarios in less time [2]. Therefore, this
study analyzes the applicability of these algorithms in
software fault prediction domain. Furthermore, the results of
ML algorithms are compared with those of tradition statistical
technique, LR. It may also be observed that the results
obtained by ML algorithms vary on different datasets. Thus, it
is important to validate them using datasets from different
domains to confirm their effectiveness. Therefore, this study
analyzes the capability of six ML algorithms viz. Adaboost
(AB), Bagging (BG), Decision Tree (J48), LogitBoost (LB),
Naïve Bayes (NB) and Random Forest (RF) for developing
software fault prediction models. Furthermore, the study
compares the results of these ML algorithms with LR for
developing defect prediction models. The comparison is
performed statistically using Friedman test. This study
explores the following research questions (RQ’s):
RQ1: What is the effectiveness of ML algorithms (AB, BG,
J48, LB, NB and RF) for developing prediction models which
determine faulty classes in a software?
RQ2: What is the comparative performance of ML algorithms
(AB, BG, J48, LB, NB and RF) with the statistical technique
LR for developing prediction models which determine faulty
classes in a software?
The above-mentioned research questions are answered by
analyzing the performance of software fault prediction models
on seven open-source datasets. The performance of the fault
prediction models is assessed using four measures viz.
precision, recall, f-measure and the Area Under the Receiver
Operating Characteristic Curve (AUC). Though, precision and
recall are traditional performance measures, the use of AUC is
supported by various studies [2,6] as it is a robust performance
measure. The results point out the RF method as the best for
developing software fault prediction models.
Section II of this study gives a brief overview of related
literature. The research background and the various ML
algorithms are discussed in Section III and IV respectively.
Section V describes the study’s results followed by threats to
validity in Section VI. Future work and conclusions are
mentioned in Section VII.
II. RELATED LITERATURE
Extensive research has been conducted in the domain of
software defect prediction. A number of review studies
extensively evaluated the conducted studies in this domain on
various parameters. A review study by Radgenovic et al. [7]
assessed 106 studies from 1991 to 2011 to evaluate the
relevance of various software metrics for developing defect
prediction models. According to their survey, Object-oriented
metrics have been widely used in literature studies as
compared to process metrics or traditional metrics extracted
from source code. A review by Catal and Diri [5] of 74 fault
prediction studies ascertained that the use of publicly available
datasets has increased significantly over the years. Moreover,
the review confirmed ML techniques as popular choices for
developing defect prediction models. This finding was
supported by a recent review conducted by Malhotra [2]. She
also confirmed the need for more number of studies which
assessed the comparative performance of ML techniques with
statistical techniques in the domain of software defect
prediction. Certain recent reviews have also investigated
studies which use search-based techniques for developing
software defect prediction models [8-9]. A review by Afzal
and Torkar [8] assessed genetic programming for developing
defect prediction models and another one by Malhotra et al.
[9] analyzed the use of several search-based techniques for
their effectiveness in this domain.
It may be noted that a wide category of techniques has
been evaluated for software defect prediction which includes
statistical, ML and the recently explored search-based
algorithms. Though, statistical techniques have been found
effective in this domain [10-12], the use of ML algorithms has
yielded improved results [2]. Dejaeger et al. [13] assessed the
use of several Bayesian network classifiers along with
statistical and other common ML algorithms for developing
defect prediction models. Several other studies too such as the
one by Gyimothy et al. [14], Pai and Dugan [15], Vandecruys
et al. [16] and Chen et al. [17] evaluated the use of both
statistical and ML algorithms. A recent study by
Tantithamthavorn et al. [18] used 12 model validation
techniques using a statistical and two ML algorithms on 18
datasets.
As pointed by Lessmann et al. [19], there were few
studies in literature which statistically assess the effectiveness
of the developed defect prediction models. However, some of
the recent key studies which use ML algorithms in this domain
have conducted rigorous statistical analysis to determine their
effectiveness. Malhotra [20] evaluated the efficiency of 18 ML
algorithms for developing defect prediction using Friedman
and post-hoc Nemenyi test on several android datasets by
developing both within project and inter-release validation
models. A study by Arar and Ayan [21] also assessed the
applicability of 3 ML and 3 search-based algorithms on four
NASA datasets using Friedman test. Similarly, a study by
Harman et al. [22] evaluated the capability of a hybridized
technique i.e. Genetic Algorithms with Support Vector
Machine on Hadoop datasets using Wilcoxon test. De
Carvalho et al. [6] also evaluated the use of Multi-objective
particle Swarm Optimization along with several ML
algorithms and assessed its capability using Wilcoxon test in
the domain of software defect prediction. Zhou et al. [23],
Arisholm et al. [24], Okutan et al. [25] and Canfora et al. [26]
also used statistical analysis to determine the effectiveness of
ML algorithms in this domain.
III. RESEARCH BACKGROUND
The various software metrics used in the study along with
the dependent variable fault-proneness is discussed in this
section. The details of various datasets used in the study along
with the performance measures is also provided in this section.
A. OO Metrics and Fault-proneness
Fault-proneness is defined as the likelihood that a specific
class would contain faulty code in the upcoming releases of
the software product. It is binary in nature with values as
“fault-prone” or “not fault-prone”.
The metrics belonging to the CK metrics suite are used as
predictors in this study. Another common measure of size
known as Lines of Code (LOC), which counts the number of
lines of source code in a class, is also used as an independent
variable. The CK metrics consists of two measures of
reusability, viz. Number of Children (NOC), which represents
the number of direct subclasses and Depth of Inheritance Tree
(DIT) which represents the position of the class in the tree
hierarchy. The Coupling Between Object (CBO) metric counts
the number of coupled classes for a specific class. Also,
another coupling measure, Response For a Class (RFC)
represents the number of methods which can respond to a
class’s message. The cohesive nature of classes is addressed
by the shared variables between two classes, which is
represented by the Lack of Cohesion among Methods
(LCOM) of a class metric. Weighted Methods per Class
(WMC) estimates the number of methods in a specific class.
B. Datasets Description
In order to develop software fault prediction models, we
use seven open source data sets in this study. The seven data
sets are developed using Java. The description of these data
sets in terms of number of total classes and number of faulty
classes is mentioned in Table I. Some of the data sets are
extracted from the PROMISE [27], while some of them are
collected using a data extraction tool developed by students of
Delhi Technological University named Defect Collection and
Reporting System (DCRS) tool [28]. The DCRS tool is
capable of extracting data from data sets which use GIT as a
version control repository. The tool computes OO metrics with
the aid of CKJM tool [29].
TABLE I. DATA SET DETAILS
Dataset Name Number of Number of
data points classes with
faults
Bsf 75 55
Click 403 85
Zuzel 30 13
Xerxes 441 71
Ivy 614 600
Log4j 351 23
Wspomaganiepi 19 12
C. Perfromance Measures
The study uses four performance measures for evaluating
the developed software fault prediction models. The
definition of these performance measures is as follows:
• Recall (Rec.): It is the ratio of correctly predicted
faulty classes amongst the actually present faulty
classes. It is also known as sensitivity.
• Precision (Prec.): It is the ratio of correctly
identified faulty classes amongst the predicted
faulty classes
• F-measure (FM): It is computed as the harmonic
mean of precision and recall.
• AUC: The AUC is a stable performance measure
as it simultaneously optimizes both recall along
with the percentage of correctly predicted non-
faulty classes (specificity). It achieves an
optimum cut-off which balances both recall and
specificity.
IV. MACHINE LEARNING ALGORITHMS
In this section, we discuss briefly, the various ML techniques
used in the study. We used WEKA as a simulation tool for the
investigated ML algorithms. The default parameter settings of
the WEKA tool were used for each ML technique.
Boosting is the process of aggregating various classifiers
which are weak in nature in order to construct a strong
classifier. Thus, boosting is an ensemble of various classifiers
where each classifier is trained using a slightly different
training data. The AB algorithm uses boosting with
distribution of weights on the training data to yield effective
results [30]. WEKA uses a seed value of 1, a weight threshold
of 100 and decision stump as the base classifier for AB. The
LB algorithm uses AB as an additive model along with cost
function provided by LR [30]. The default parameter settings
for the LB technique in the WEKA tool include a likelihood
threshold of -1.79, decision stump as the base classifier, a
weight threshold of 100, a shrinkage of 1.0, 10 iterations, and
a seed value of 1.
J48 is a decision tree algorithm which uses normalized
information gain as the criteria for splitting [30]. For each of
the predictor variable, the information gain is computed and
the attribute with the highest information gain is designated as
the root node. This process is performed recursively. The
default parameter settings for WEKA were a confidence factor
of 0.25, three folds, a seed value of 1 and a pruned tree.
NB is a Bayesian method which develops a classifier based
on probability. Each of the predictor variables is assumed to
be independent.
Bagging (BG) is another type of ensemble learning method
which creates bootstrap sample from the original data by
repeatedly sampling the dataset. The sampling is done in
accordance with uniform probability distribution. It may be
noted that the size of the bootstrap sample is exactly similar to
that of the original data. The parameters used by the WEKA
tool for the BG technique include 100% as the bag size, REP
tree as the base classifier, a seed value of 1 and ten iterations.
RF is also an ensemble learner which consists of several
decision trees. The individual trees of the forest are
constructed with various subsets of the training data.
However, these subsets are constructed randomly and with
replacement. The outcome of the forest is determined as the
mode of the outputs generated by the individual constituent
trees. A forest of 100 trees, a seed value of 1 and a maximum
depth of 0 is used as parameter settings by the WEKA tool for
RF.
V. RESULTS AND ANALYSIS
This section describes the results of the study along with
answers to each of the investigated RQ.
A. RQ1: What is the effectiveness of ML algorithms (AB, BG,
J48, LB, NB and RF) for developing prediction models
which determine faulty classes in a software?
The fault prediction models in the study are developed
using ten-fold cross validation [31]. This strategy divides the
given data sets into ten disjoint subsets. A model is developed
by providing nine of these subsets as the training data. The
tenth remaining subset is used for validating the developed
model. This process is repeated until the prediction values on
each of the ten subsets are obtained, i.e. till all the subsets are
used at least once for the purpose of validation. As discussed
in section III.C, we evaluate the developed software fault
prediction models using recall, precision, FM and AUC. Table
II states the precision and recall values obtained by the
developed software fault prediction models on all the datasets.
The precision values of all the models ranged from 0.736-
0.975 indicated effective fault prediction models.
 TABLE II. PRECISION AND RECALL VALUES

Techniques Bsf Click wspomaganiepi Ivy Log4j Xerces Zuzel

Prec. Rec. Prec. Rec. Prec. Rec. Prec. Rec. Prec. Rec. Prec. Rec. Prec. Rec.
AB 0.844 0.838 0.781 0.808 0.831 0.833 0.955 0.977 0.924 0.94 0.77 0.836 0.793 0.793
BG 0.818 0.811 0.757 0.796 0.774 0.778 0.955 0.977 0.878 0.937 0.79 0.834 0.758 0.759
J48 0.834 0.824 0.766 0.801 0.831 0.833 0.955 0.977 0.912 0.934 0.788 0.836 0.758 0.759
LR 0.794 0.797 0.774 0.803 0.778 0.778 0.955 0.977 0.943 0.949 0.736 0.832 0.758 0.759
LB 0.822 0.824 0.787 0.811 0.889 0.889 0.955 0.977 0.917 0.937 0.799 0.839 0.793 0.793
NB 0.741 0.595 0.773 0.803 0.808 0.778 0.975 0.579 0.911 0.906 0.792 0.82 0.828 0.828
RF 0.849 0.851 0.752 0.766 0.889 0.889 0.955 0.971 0.919 0.934 0.792 0.82 0.758 0.759

TABLE III. F-MEASURE AND AUC VALUES

Techniques Bsf Click wspomaganiepi Ivy Log4j Xerces Zuzel

FM AUC FM AUC FM AUC FM AUC FM AUC FM AUC FM AUC
AB 0.840 0.854 0.769 0.669 0.829 0.819 0.966 0.789 0.925 0.704 0.772 0.747 0.793 0.796
BG 0.814 0.829 0.753 0.741 0.764 0.813 0.966 0.799 0.907 0.603 0.796 0.795 0.757 0.776
J48 0.828 0.758 0.754 0.556 0.829 0.708 0.966 0.413 0.918 0.603 0.789 0.613 0.757 0.750
LR 0.796 0.801 0.751 0.682 0.778 0.752 0.966 0.797 0.934 0.628 0.766 0.583 0.757 0.773
LB 0.823 0.860 0.769 0.718 0.889 0.868 0.966 0.841 0.920 0.619 0.801 0.764 0.793 0.805
NB 0.615 0.807 0.772 0.673 0.784 0.852 0.712 0.842 0.909 0.604 0.803 0.795 0.827 0.788
RF 0.850 0.847 0.758 0.765 0.889 0.826 0.963 0.849 0.924 0.666 0.803 0.795 0.757 0.815

Similarly, the recall values in majority of the cases ranged
from 0.759-0.977. Only in two cases, the recall values were
below 0.60. Thus, the investigated ML algorithms along with
LR techniques developed effective fault prediction models
with acceptable precision and recall values. Table III states the
FM and AUC values obtained by the developed fault
prediction models. The FM values ranged from 0.615-0.966
for all the developed defect prediction models. Similarly, the
AUC values in majority of the cases ranged from 0.6-0.9.
Only in three specific cases the AUC values were less than
0.6. In fact, in most of the cases, the AUC values were greater
than 0.7. This indicates effectiveness of the ML and the
statistical technique LR for developing software fault
prediction models. The average values of f-measure and AUC
obtained by all the techniques over all the datasets are depicted
in Figure 1.
According to the figure, the average AUC values obtained by
only the J48 technique was less than the LR technique. In all
other cases the average AUC values obtained by the ML
techniques were greater than the LR technique. The average
FM values obtained by the ML techniques were also
competent with the LR technique.
The range of AUC values for all the ML techniques on all
the investigated data sets were 0.669-0.854 for AB, 0.603-
0.829 for BG, 0.413-0.758 for J48, 0.619-0.868 for LB, 0.604-
0.852 for NB and 0.666-0.849 for RF. The range of AUC
values for models developed using the RF techniques was
0.583-0.801, which was lesser than most of the ML
techniques. This indicates the superiority of ML techniques.
B. RQ2: What is the comparative performance of ML
algorithms (AB, BG, J48, LB, NB and RF) with the
statistical technique LR for developing prediction models
which determine faulty classes in a software?
In order to compare the performance of six ML algorithms
with the statistical technique LR, we use Freidman statistical
test on AUC values. This is because literature studies advocate
the use of AUC as a stable performance measure. The test was
conducted at a cut-off of 0.05. We evaluated the following
hypothesis:

Null Hypothesis (H0): The ML techniques (AB, BG, J48, LB,

NB and RF) have similar capability when compared with the
LR technique for developing software fault prediction models,
when assessed using AUC values.
Fig. 1. Average AUC and FM values of techniques
Alternate Hypothesis (H1): The ML techniques (AB, BG, J48,
LB, NB and RF) have better capability when compared with
the LR technique for developing software fault prediction
models, when assessed using AUC values.
As already discussed in RQ1, most of the ML
techniques obtained better average AUC values than the LR
technique. This observation supports the use of ML techniques
in the domain of software fault prediction. Table IV states the
mean ranks obtained by different techniques after application
of the Friedman test. It should be noted that the technique
obtaining the lowest rank is the best. The Friedman results
were significant with a p-value of less than 0.001 and a chi-
square value of 24.698 and six degrees of freedom. Therefore,
we accept H1 hypothesis, which indicate the superiority of
ML algorithms in the domain of software fault prediction.
According to Table IV, the best rank was obtained by the RF
technique, followed by the LB and NB techniques. The LR
technique was poorer than most of the ML techniques except
J48.
TABLE IV. FRIEDMAN RESULTS
Technique Mean Rank using
Friedman
RF 1.86
LB 2.57
NB 3.57
AB 3.86
BG 4.07
LR 5.29
J48 6.79
VI. THREATS TO VALIDITY
This section mentions the various threats to validity
of the study.
The threat to conclusion validity is addressed by
validating the study’s hypothesis with the help of Friedman
test at a cut-off of 0.05. Since, Friedman test is non-parametric
in nature, we do not need to validate any assumptions on the
underlying data before the application of the test. Also, the use
of ten-fold cross validation method reduces validation bias [6],
thereby strengthening the obtained conclusions. The use of
AUC as a performance measure has been advocated by
various studies as it is a stable performance measure [2,6]
which is capable of assessing impartially, the models
developed using imbalanced data. This step also enhances the
conclusion validity of the study. The internal validity concerns
itself with the “causal” effect of OO metrics on defect-
proneness. However, studies which can determine this effect
are difficult to conduct in practice as such studies require
highly controlled experimental environments. As, determining
the “causal” effect was not the aim of our study, we did not
address this threat. The OO metrics used in the study have
been used extensively for developing defect prediction models
in literature [2,3]. Thus, the use of these metrics is valid as
they are accurate representatives of the “constructs” they
represent, reducing the threat to construct validity. As the
study evaluates open source datasets, they can be easily used
by other researchers for replication and further
experimentation. This increases the external validity of the
study. It may also be noted that the evaluated datasets are
representative of several domains, thus increasing the
generalizability of the obtained results. However, all the
investigated datasets are developed in Java language, thus
limiting the applicability of results to only datasets developed
in Java language.
VII. CONCLUSIONS AND FUTURE WORK
This study analyzed the effectiveness of six ML
techniques viz. Random Forests, Logitboost, Naïve Bayes,
Adaboost, Bagging, J48 for developing software fault
prediction models along with the statistical technique logistic
regression (LR). The performance of the developed models
was assessed using precision, recall, AUC and F-measure. The
developed models were also compared using Friedman test to
determine whether ML techniques are better for developing
software fault prediction models as compared to the ones
developed using the LR technique. The conclusions can be
summarized as follows:
• The ML techniques are effective in the domain of
software fault prediction as the majority of AUC
values ranged from 0.6-0.9 and majority of f-measure
values ranged from 0.615-0.966.
• The Friedman results indicate that the Random
Forests techniques is the best ML technique amongst
the investigated techniques. Furthermore, the models
developed using the LR technique are poorer than
most ML techniques except the J48 technique.
The results can help researchers in choosing an efficient
technique for developing software fault prediction models.
Also, they can be used for optimization and effective
allocation of a project’s resources. In future, we would like to
investigate search-based techniques for developing efficient
fault prediction models.
REFERENCES
[1] Y. Singh and R. Malhotra, Object-Oriented Software Engineering.
PHI Learning Pvt. Ltd. New Delhi, 2012.
[2] R. Malhotra, “A systematic Review of Machine Learning
techniques for Software Fault prediction,” Applied Soft Comp., vol.
27, pp. 504-518, 2015.
[3] D. Radjenović, M. Heričko, R. Torkar and A. Živkovič, “Software
fault prediction metrics: A systematic literature
review”, Information and Software Technology, vol. 55, no. 8, pp.
1397-1418, 2013.
[4] S. Chidamber and C. Kemerer, “A Metric Suite for Object-
Oriented design,” IEEE Transactions on Software Engineering,
vol. 20, no. 6, pp. 476-493, 1994.
[5] C. Catal and B. Diri, “A systematic review of software fault
prediction studies”, Expert systems with applications, vol. 36, no.
4, pp. 7346-7354, 2009.
[6] A. B. De Carvalho, A. Pozo and S. R. Vergilio, “A symbolic fault-
prediction model based on multiobjective particle swarm
optimization”, Journal of Systems and Software, vol. 83, no. 5, pp.
868-882, 2015.
[7] D. Radjenović, M. Heričko, R. Torkar and A. Živkovič, “Software
fault prediction metrics: A systematic literature
 review”, Information and Software Technology, vol. 55, no. 8, pp.
1397-1418, 2013.
[8] W. Afzal and R. Torkar, “On the application of genetic
programming for software engineering predictive modeling: A
systematic review”, Expert Systems with Applications, vol. 38, no.
9, pp. 11984-11997, 2011.
[9] R. Malhotra, M. Khanna and R. R. Raje, “On the application of
search-based techniques for software engineering predictive
modeling: A systematic review and future directions”, Swarm and
Evolutionary Computation, vol. 32, pp. 85-109, 2017.
[10] L. C. Briand, J. Wüst, J. W. Daly and D. V. Porter, “Exploring the
relationships between design measures and software quality in
object-oriented systems”, Journal of systems and software, vol. 51,
no. 3, pp. 245-273, 2000.
[11] H. M. Olague, L. H. Etzkorn, S. Gholston and S. Quattlebaum,
“Empirical validation of three software metrics suites to predict
fault-proneness of object-oriented classes developed using highly
iterative or agile software development processes”, IEEE
Transactions on software Engineering, vol. 33, no. 6, 2007.
[12] K. El Emam, S. Benlarbi, N. Goel and S. N. Rai, “The
confounding effect of class size on the validity of object-oriented
metrics”, IEEE Transactions on Software Engineering, vol. 27, no.
7, pp. 630-650, 2001.
[13] K. Dejaeger, T. Verbraken and B. Baesens, “Toward
comprehensible software fault prediction models using bayesian
network classifiers”, IEEE Transactions on Software
Engineering, vol. 39, no. 2, pp. 237-257, 2013.
[14] T. Gyimothy, R. Ferenc and I. Siket, “Empirical validation of
object-oriented metrics on open source software for fault
prediction”, IEEE Transactions on Software engineering, vol. 31,
no. 10, pp. 897-910, 2005.
[15] G. J. Pai and J. B. Dugan, “Empirical analysis of software fault
content and fault proneness using Bayesian methods”, IEEE
Transactions on software Engineering, vol. 33, no. 10, pp. 675-
686, 2007.
[16] O. Vandecruys, D. Martens, B. Baesens, C. Mues, M. De Backer
and R. Haesen, “Mining software repositories for comprehensible
software fault prediction models”, Journal of Systems and
software, vol. 81, no. 5, pp. 823-839, 2008.
[17] N. Chen, S. C. Hoi and X. Xiao, ”Software process evaluation: a
machine learning framework with application to defect
management process”, Empirical Software Engineering, vol. 19,
no. 6, pp. 1531-1564, 2014.
[18] C. Tantithamthavorn, S. McIntosh, A. E. Hassan and K.
Matsumoto, “An empirical comparison of model validation
techniques for defect prediction models”, IEEE Transactions on
Software Engineering, vol. 43, no. 1, pp. 1-18, 2017.
[19] S. Lessmann, B. Baesens, C. Mues, and S. Pietsch, “Benchmarking
classification models for software defect prediction: A proposed
framework and novel findings”, IEEE Transactions on Software
Engineering, vol. 34, no. 4, pp. 485-496, 2008.
[20] R. Malhotra, “ An empirical framework for defect prediction using
machine learning techniques with Android software”, Applied Soft
Computing, vol. 49, pp. 1034-1050.
[21] O. F. Arar and K. Ayan, “Software defect prediction using cost-
sensitive neural network”, Applied Soft Computing, vol. 33, pp.
263-277, 2015.
[22] M. Harman, S. Islam, Y. Jia, L. L. Minku, F. Sarro and K. Srivisut,
“Less is more: Temporal fault predictive performance over
multiple hadoop releases”, In International Symposium on Search
Based Software Engineering, pp. 240-246, 2014.
[23] Y. Zhou, B. Xu and H. Leung, “On the ability of complexity
metrics to predict fault-prone classes in object-oriented
systems”, Journal of Systems and Software, vol. 83, no. 4, pp. 660-
674, 2010.
[24] E. Arisholm, L. C. Briand and E. B. Johannessen, “A systematic
and comprehensive investigation of methods to build and evaluate
fault prediction models”, Journal of Systems and Software, vol. 83,
no. 1, pp. 2-17, 2010.
[25] A. Okutan and O. T. Yıldız, “Software defect prediction using
Bayesian networks”, Empirical Software Engineering, vol. 19, no.
1, pp. 154-181, 2014.
[26] G. Canfora, A. De Lucia, M. Di Penta, R. Oliveto, A. Panichella
and S. Panichella, “Multi-objective cross-project defect
prediction”, Software Testing, Verification and Validation (ICST),
2013 IEEE Sixth International Conference, pp. 252-261, 2013.
[27] https://code.google.com/p/promisedata/
[28] R. Malhotra, K. Nagpal, P., Upmanyu, and N. Pritam, “Defect
Collection and Reporting System for Git based Open Source
Software,” In Proceedings of International Conference on Data
Mining and Intelligent Computing, pp. 1-7, 2014.
[29] http://www.dmst.aueb.gr/dds/sw/ckjm/doc/index.html
[30] I.H. Witten and E. Frank, Data Mining: Practical Machine
Learning Tools and Techniques, Morgan Kaufmann Publishers,
USA 2005
[31] M. Stone, “Cross-validatory choice and assessment of statistical
predictions,” Journal of Royal Statistics and Society, vol. 36, pp.
111-147, 1974.