Software Reliability Growth Models, Tools and Data Sets-
A Review
Anurag Kumar
Amity University, Uttar Pradesh
Anurag.ahaar@gmail.com
ABSTRACT
This paper presents effective and comprehensive review on
parametric and non parametric modeling of software reliability.
Reliability is one of the most important characteristic of software
quality. More than hundreds of software reliability models were
proposed in last few decades. These models were based on various
phases of software development life cycle. Software Reliability
Growth Models based on software testing were explored a lot over
the years. Many authors have proposed or discussed parametric
models with following characteristics such as realistic and
unrealistic assumptions, limitations, applicability, environment
dependability and predictability. On the other hand authors have
discussed the shortcomings of parametric models and proposed
non-parametric models as an alternative to address the software
reliability issues. But, both the modeling techniques have their pros
and cons. In this paper our aim is to give an insight of parametric
and non-parametric based software reliability models. The review
of real life datasets and already proposed tools for software
reliability will help the beginner level researcher to continue.
CCS Concepts
• Software and its engineering → Software organization and
properties → Extra-functional properties → Software
reliability
Keywords
Software reliability growth models; tools for software reliability;
parametric technique; non parametric technique.
1. INTRODUCTION
Computers are used in a variety of significant and multifaceted
areas including Air Traffic Analysis & Control, Nuclear Reactors,
Space Research, Hospitals, Schools and Colleges, Military &
Defense services etc. The increasing dependability of various
fields on software creates a need to make the software field more
accurate and reliable [1]. That’s why software reliability is an issue
to make all the depending fields work properly [2]. Software
reliability is emerging as a crucial field of study.
The aim of the research work is to present effective and
comprehensive review on parametric and non parametric modeling
of software reliability. Various growth models presented by
various researchers during last few decades were analyzed by us.
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee. Request permissions
from Permissions@acm.org.
ISEC '16, February 18-20, 2016, Goa, India
© 2016 ACM. ISBN 978-1-4503-4018-2/16/02…$15.00
DOI: http://dx.doi.org/10.1145/2856636.2856648
80
1.1 Paper Identification
Our review work has the following characteristics, which help to
put it out of the crowed-
 Time Frames: - Our review covers the time frame of
1980-2015. The paper includes various growth models
published during this time frame, whether these were
parametric models or non parametric models.
 Comprehensiveness: - The papers considered for the
review were not collected from a single source. We
collected the papers from various digital libraries
including IEEE Explore, ACM Digital Library and
Science Direct etc.
 Analysis: - We tried our best to provide the new comers
detailed information about the growth models, data sets
and tools of reliability.
1.2 Assessment Criteria
Our way to identifying papers suitable for analysis is spurred by
Kitchenham and Charter's idea of a quality check. Our evaluation
is engaged particularly on identifying just papers reporting
adequate data to permit analysis crosswise over studies. To permit
this, an essential arrangement of data must be accounted for in
papers. Without this it is hard to legitimately comprehend what has
been done in a study and just as hard to satisfactorily contextualize
the discoveries.
Our criteria for inclusion of papers is as follows-
 It must be a new growth model.
 It should have sufficient contextual and model building
data.
 It must not be a review or survey paper.
 It must not be an analysis of already published models.
2. SOFTWARE RELIABILITY
Now let’s consider what we mean by software reliability? If we
talk about the word “reliability” then we know that reliability
means “the quality of being trustworthy or steady”. So for software
to be reliable means, the software should be dependable or can be
trusted by anyone who uses it. The particular definition of software
reliability differs from researcher to researcher. But what comes as
an output from all the definitions proposed till now that software
reliability is the probability that the software system will
accomplish its intentional function under some specific limitations
[3]. Software reliability is directly connected to software failure
because if software reliability is its correct functioning then
software is said to be unreliable if it’s not functioning as it’s
supposed to do or we can say software failure occurred. Let’s take
a look at software error, software faults and software failure. An
error is a mental inaccuracy made by the coder. A fault is the
exemplification of that error in the coding of the software while a
software failure is defined as the happening of an erroneous output
as an outcome of an input value [4].
Figure 1: Software Faults vs. Failure
Various studies in the field of software reliability, has proved that
as the size of software system increases the number of faults made
by coder increases, which causes the increased chances of software
failure or we can say less reliability [5].
3. SOFTWARE RELIABILITY GROWTH
MODELS
Software reliability is a significant part of software industry. It
gives a measure to the customer as well as to the developer about
the faults in the software. So the prediction of the reliability of any
software in this competitive software industry is really essential.
Reliability growth models give an estimate to the number of faults
that may occur in near future after the delivery of the product and
thus the models also provide an indication of when to release the
software. Software Reliability Growth Models (SRGMs) use the
past data gathered during the testing process. This data is about the
number of faults occurred over time during testing of the software.
This fault data is used to apply a SRGM to estimate the reliability
of the software product.
The majority of software reliability growth models have a
parameter that relates to the total number of defects present in the
set of code. If this parameter and current number of defects
discovered is known, then remaining defects in the code can be
predicted.
Knowing the number of residual defects helps to make your mind
up whether or not the code is ready to dispatch and how much
more testing is necessary if the code is not ready to dispatch. It
gives an estimate of the number of failures that the customers will
encounter when operating the software. Two types of data is
required to apply any SRGM- one is the defect data and second
one is the time of the occurrence of the defects in the defect data.
Figure 2: Residual Defects
81
4. PARAMETRIC SOFTWARE
RELIABILITY GROWTH MODELS
There are two categories of SRGMs; first one is parametric
reliability growth models. The concept of parametric growth
models was the first try to predict the reliability of software
products. These SRGMs are supported by a variety of assumptions
made by the researchers. The methodology used in these models is
based on statistical approaches like regression. During the last
some decades, researchers had proposed so many parametric
growth models. Here we are presenting some PSRGMs selected
from literature:
Bev Littlewood et al. in 1981 present a modification [6] to the
originally and widely used first growth model of reliability given
by Jelinski & Moranda [7]. Authors used MLE for parameter
estimation to overcome instability problem of the J&M model.
Yamada et al. in 1983 proposed and analyzed the model of error
detection and found the curve of number of errors detected in the
analyzed data is S-shaped, that’s why it’s called s-shaped growth
model [8, 22]. Yoshihiro Tohma et al. in March 1989 published a
structural approach to find out the residual software faults [9].
Application of both the models was shown and they proved that
the fitness of models to the real data is good enough. George L.
Gentzler et al. in February 1990, presented data stability to be an
application of software reliability models [10]. Sarah Brocklehurst
et al. in 1990 proposed the concept of recalibration of software
reliability growth models [11]. They chose the problem of deciding
that which model will give the better result of reliability priory.
The advantage of the recalibration was its non dependability on the
model used. Karama Kanoun et al. in 1991 published a research to
use the already proposed growth models to get better prediction of
errors or failure and Yoshihiro Tohma et al. gave some new
parameter estimation techniques [12, 13].
The growth model given by P.K. Kapur et al. in 1992 [14] was
using the process of NHP distribution. The assumption made by
the authors was that the errors using this process may lead us to
find out some undetected errors without causing any software
failure. Yamada et al. presented a growth model considering the
test effort needed during the period of testing [15]. They used
Weibull Curve to represent the relationship between test efforts to
the time. The methodology of the model was NHPP [16]. P.
Zeephongsekul, G. Xia, and S. Kumar in 1994 proposed their
reliability growth model [17]. This model was based on the
concept that the flawed resolution of current faults in the system
may lead to the new faults and secondary failure of the software
system in near future. In this paper they also presented a cost
model. Jeff Tian in 1995 tried to combine the already proposed
approaches, time domain approach and input domain approach, of
reliability growth to make his own new approach [18]. This growth
model given by Tian had the strengths of both the approaches and
he proved its functionality on software developed by IBM
Software Solutions Toronto Laboratory. A. Pasquini et al. in 1996
published their investigation about the sensitivity of SRGMs to the
faults and also proposed a relationship between this sensitivity and
the testing phase [19]. This investigation was based on a scenario
in which various models of reliability growth were implemented
on the data of testing stage.
In December 1996, Hou et al. used HGDM growth model to give
the solution of two major problems of testing phase, one of them
was to minimize the number of undetected software faults after
testing and other one was to minimize the number of resources
needed for the testing and found that optimal resource allocation covered all the pros of conventional NHPP but also provide
method is capable of improving the reliability of the software coverage to other components as well. Huang, Chin-Yu, and Wei-
system under consideration [20]. In 1996 Hou et al. published one Chih Huang in 2008 put the concept in which the assumption of
more paper related to HGDM (Hyper-Geometric Distribution ignoring the debugging time was challenged by the authors [35].
Software Reliability Model) [21]. Authors proposed two new They used the queuing models for predicting the reliability of the
sampling techniques to find out the Bayes estimates of current software systems. In the end authors concluded that considering
reliability as well as current fault number. Tian, Jeff, and Joe the fault detection and correction during the testing process will
Palma in 1998 proposed their new approach which they called tree give the more accuracy to the growth models. Norman
based approach by combining the plus points of software reliability Schneidewind in 2008 published a paper in which he compared
growth models and input growth models [23]. Authors various models and tried to modify some [36]. Yamada model was
implemented their approach on five large software systems taking found most accurate in conclusion. Kapil Sharma et al. chose a
them as examples and got better results for their approach. problem of selecting the appropriate model to test the reliability
[37]. The number of reliability growth models were increasing at a
Chen et al. in 2001 took the problem of over estimation of pace of airplane so in 2010 they gave a distance based model for
reliability growth models because of nature of testing process and selecting the appropriate growth model for the reliability testing.
proposed a model to overcome this limitation of growth models This model takes some input as selection criteria and gives an
using code coverage [24]. In September, 2001 Kuo et al. tried to appropriate model on the basis of the input parameters. In 2011
propose one more growth model using NHPP [25]. Pasquini, Huang et al. said that the debugging process during testing and
Alberto, Giuliano Pistolesi, and Antonio Rizzo in 2001 gave new operation differs in speed and number of faults so they gave the
theory about the failure of software systems that faults are not only model for both the duration of software life: testing and operation
cause of component failure but also as a whole of the system and [38]. Then they proposed the concept of multi points that were the
also because of the human errors [26]. So the authors tried to points on time graph when environment changes. Other parametric
propose a model which extends the already implemented methods models [39, 40, 41, 42] were also new findings in the parametric
to find the growth of other system components such as human growth models’ field.
interface.
Table 1 is showing the summary of all the papers considered by us
Lyu et al. in 2002 considered the CBSE approach of software in parametric category-
development and gave model for the reliability of the software
integrated from various components [27]. This was the model for Table 1: Various PSRGMs
the component based software development techniques to optimize
Reference Year of Proposed Methodology
the reliability constraints and testing schedules. Teng, Xiaolin, and
Publication And Outcomes
Hoang Pham in 2002 proposed first SRGM for the N-Version Number
Programming System which was built by considering error
introduction rate and also considered error removal efficiency [28]. [6] 1981 Concept of MLE introduced
for the conventional reliability
Yashwant K. Malaiya, et al. in December, 2002 first time proposed model.
growth model considering testing coverage [29]. Authors
implemented relationship between test coverage and defect [8] 1983 Proposed s-shaped reliability
coverage. Test-coverage parameters were branches and blocks etc. growth model
Huang et al. in 2003 proposed the paper [30] in which they showed [9] 1989 Structural approach to find out
various NHPP based models using weighted mean, geometric the residual software faults
mean and harmonic mean and also gave model of NHPP by
implementing power law transformation. Huang, Chin-Yu, and [10] 1990 Data stability to be an
Chu-Ti Lin in 2007 published a paper in which they said that the application of software
assumption made in various models proposed till now is not well reliability models. Used Goel-
applicable in practical implementations [31]. The assumption Okumoto growth model.
challenged was that the faults detected are removed immediately [11] 1990 concept of recalibration of
but authors put the concept that it’s not practically possible to software reliability growth
remove all the detected faults immediately, especially dependent models
faults (the faults which are linked to each other), because of
[12] 1991 They gave the concept of
constraints related to cost and debugging team.
analyzing and then dividing
Huang, Chin-Yu, et al. in 2007 again presented a research work in trend of the testing data to
which they compared various SRGMs which were sensible to the apply multiple different
effort needed to resolve the faults [32]. The problem the chose was growth models on the parts of
that some S-Shaped models may not work well if the test-effort data.
was fluctuating, so they introduced a new function. [13] 1991 Analyzed 6 methods and
Zachariah, Babu, and R. N. Rattihalli in year 2007 proposed two proved LSS parameter
new growth models [33]. The attempt made by the authors was estimation is best for hyper
based on the hypothesis that the failure rate of the software during geometric distribution
the testing phase is because of the inappropriate selection of the [14] 1992 New growth model based on
input patterns. Wang et al. proposed a new version of NHPP called non-homogeneous Poisson
MANHPP abbreviation for moving average NHPP to include the distribution
advantages of both the approaches time domain approaches and
structure based one [34]. This MANHPP approach not only

82
[15] 1993 Growth model based on and they used delay function
NHPP, presented relation of in SRGM to overcome this
test effort to the time using problem.
weibull curve
[17] 1994 Improper fault resolution may
result in secondary software [32] 2007 They analyzed the SWGMs
failure in near future. They which were sensitive to the
proved this. test effort and introduced the
logistic function.
[18] 1995 New model by the
combination of time domain [33] 2007 Two growth models using
and input domain approaches. random testing and partition
testing.
[19] 1996 Proved the sensitivity of
growth models to the errors [34] 2007 Moving Average was
based on a considered case implemented with
study. conventional NHPP

[20] 1996 Used HGDM to solve [35] 2008 Proposed a model by

OPT/RA. considering the debugging
time and effort using queuing
[21] 1996 Proposed a release policy for models.
HGDM to minimize cost and
meet reliability deadline. [36] 2008 It was the comparison paper
in which 3 reliability models
[22] 1997 Implemented two Gibb’s were compared.
approaches for sampling.
Used NHPP [37] 2010 Proposed a model selection
method based on distance.
[23] 1998 Combined two approaches to
make their own tree base [38] 2011 Presented concept of multi
growth model points in the software life
cycle to improve reliability
[24] 2001 Solved the problem of over growth prediction in testing
estimation of the growth and operation phase
models.
[39] 2012 Proposed SRGM for multi
[25] 2001 Based on methods for release software products.
parametric decomposition,
proposed one more growth [40] 2013 Modified NHPP technique
model based model considering
imperfect fault debugging.
[26] 2001 Model for growth of other
system components as man- [41] 2014 A model to predict the release
machine interfaces and of the open source software
operators. based on generalized software
reliability model.
[27] 2002 Reliability growth model for
CBSE based software [42] 2015 Growth model based on
systems. Bayesian theory but different
2002 First growth model for NVP from previous models in the
[28] manner that it uses all
which considers error removal
efficiency & error- available data rather than only
introduction rate. failure data.

[29] 2002 first time proposed growth

model considering testing
coverage
5. Non Parametric Software Reliability
Growth Models
[30] 2003 Used harmonic, arithmetic Parametric SRGMs models utilize the statistical approaches for
and geometric means to predicting the reliability growth. Major pitfalls of these models are
implement NHPP models. the assumptions made, because these assumptions are based on the
[31] 2007 Proved that detection and past behavior of the software and may not relate to the real
correction processes during problem. That’s why over the years many researchers proposed
testing can’t run non parametric software reliability growth models. NPSRGMs
simultaneously because generally use the machine learning approaches like neural
debugging process takes time networks, genetic algorithm, support vector machines etc. The

83
advantage of these models over parametric based ones is that, there [47] 2007 GP was used for this model and
is no need of assumptions and the self learning behavior of the boosting is used to improve the
machine learning approaches makes these models more accurate. efficiency of this model.
Ariela Sofer et al. in 1991 presented a new growth model of
[48] 2007 NHPP using nonparametric method to
reliability by modifying their previous model [43]. The new model
overcome the problem of wrong
was modified to estimate not only current but also long-term
assumptions.
predictions. Karunanithi et al. in 1992 proposed one more efficient
model for reliability growth called connectionist model of software [49] 2007 NPSRGM based on order statistic
reliability growth [44]. They used neural network for this model approach and authors solved the
and also presented comparison of this model with other five problem of random sampling of faults
models and the outcome was the better accuracy of the 2009 Authors tried to overcome the pitfalls
[50]
connectionist model. of canonical genetic programming.
In September, 1999 Sitte and Renate published a paper in which These problems were random
they compared already published two great approaches of generation and slow convergence for
reliability estimation, which were neural network based and large data sets.
parameter recalibration based [45]. Both the approaches so far was [51] 2009 Author used meta heuristic approach
proved better to each other but Sitte & Renate find out that, neural to predict the reliability of the
networks are more accurate. Cai, Kai-Yuan et al. in 2001 used complex data sets of software. ACO
neural network once again but this time for the prediction of was used to deal with the constraints
dynamic reliability of the products [46]. Authors presented their of the problem.
new findings on the behavior of the neural network for the
prediction of the software reliability. Costa, Eduardo Oliveira, et [52] 2010 Author used hybrid GA and simulated
al. in September 2007 published this paper, in which they proposed annealing to propose thus model
a growth model based on the genetic algorithm or programming which was based on SVM. This
[47]. Also authors showed the use of boosting technique to perk up model also called as GA-SA-SVM
the effectiveness of the models. Wang, Zhiguo, et al. [48] and model. Use of SA with GA deals with
Wilson et al. [49] in 2007 proposed non parametric NHPP and the convergence problem of GA and
analyzed order statistic model by means of non parametric SVM helps to deal with non linear
technique. Not only neural networks [56, 57] but also genetic data.
algorithm and SVM based models [50, 51, 52, 53, 54, 59] were [53] 2011 Authors integrated ARIMA of Box
proposed by various researchers over the years. Wason et al. [55] and Jenkin and SVM to create new
in 2012 tried new approach for NPSRGMs. They used finite state model because ARIMA deals with
automata first time in the history for reliability growth model. linear data and SVM deals with non
Table 2 is showing some non parametric models proposed over the linear one.
decades. This table only has the models chosen by us and no
[54] 2012 Li et al. published this short paper in
modification and optimization attempts of authors are included in
which they used GA based GEP for
this table. the prediction of growth of reliability.
Table 2: Various NPSRGMs [55] 2012 This was the first of its kind attempt
made by the authors. They used first
Reference Year Methodology Used and Outcome
time finite state automata to represent
Number the software system.

1991 Ariela Sofer et al. previously [60] 2013 In this paper authors used GA and
[43]
proposed parametric model but this simulated annealing with the aim of
time they used non parametric maximizing reliability prediction and
strategy to predict the long term minimizing cost.
faults.
[44] 1992 Authors used neural network to make
their model called connectionist 6. YEAR WISE COMPARISON OF
model and published the results that
this is better than five parametric PSRGMS AND NPSRGMS
models. Here we are providing the chart of various models of both the
categories over the years. This is the plot between number of
[45] 1999 This paper proved that neural models considered (both type) and years. Figure 3 showing this
networks work better than parametric plot. X-axis showing the number of models considered by us from
models. both the categories and y-axis is showing the years in the range of
[46] 2001 Major findings: 10 years.
 Approach is better for
smooth trend
 Training output is better
than prediction.

84
 Figure 3: Number of Models vs. Years
15
10
Parametric
5 Models
0 Non Parametric
Models
7. TOOLS USED IN SOFTWARE
RELIABILITY GROWTH MODELS
There are so many tools which are used in software reliability
estimation. Which tool would be more efficient, is normal question
which struck in the mind of any researcher, especially for new
comers in the field. So I have collected the information of some
research tools to help the novice researchers-
Table 3: Tools used in Reliability Estimation
Tools Overview
ESTM (Economic Stop It supports multiple models
Testing Model Tool) concurrently and uses MLE
parameter estimation method. But
ESTM only uses time between data
format only.
ROBUST ROBUST provide the researchers an
(Reliability of Basic and easy way of applying earlier growth
Ultra-reliable Software models for examination of the
sysTem) reliability of the software. It
provides some features to enhance
the SRGMs also and can be used for
statistical methods of early error
detection.
SMERFS SMERFS is a tool used for software
(Statistical Modeling and reliability modeling that enables the
Estimation of Software user to execute an analysis of
Reliability Functions) reliability. This program has gone
through a number of modifications
over time. The currently used
version of this is 5. It has included
11 of the mostly used models of
literature; 6 using time between error
occurrences as input and 5 using the
number of detected faults per unit of
time.
SoRel [62] Two major modules of SoRel are:
(Software Reliability  Trend Tests
Program)  Reliability Growth Models.
It operates on 2 types of data-
intensity of faults and times of inner
faults.
CASRE [61] CASRE is a CASE tool used in
reliability estimation. Main features
of this tool are:
85
 Data Modification
 Failure Data Analysis
 Modeling and Measurement
S-PLUS S-PLUS is one of the tools used for
growth modeling based on tree,
tracking of test cases and SRGMs.
Some models can easily be
simulated on S-PLUS but not all
because of the lots of options
required for the models like,
parameter estimation methods etc,
which require lots of effort to use
this tool.
Weibull++ Weibull++ is the product of
ReliaSoft. It is the standard tool for
Weibull analysis for many industries
across the globe. This tool supports
an all-inclusive array of life span
distributions and tenders an
influential interface especially
designed for the need of research
fellows.
8. DATA SETS USED IN SOFTWARE
RELIABILITY TESTING
Pham [3] stated that there are two frequently used types of failure
data:
 Data based on Time-domain and
 Data based on Interval-domain.
These types of data sets are generally utilized by the experts when
investigating and anticipating reliability. Not all the quality models
can deal with both sorts of data.
The time-space methodology includes recording the times when
fault detected, as showed in Table number 4. First fault occurred at
20 minute, 2nd at 32, 3rd at 42 and so on. A few models may have
need of the time data between various failures to compensate for
the actual time of failure. In the example considered, 3rd column
data 20, 12, 10, 16, 30, 2, 21, 3, 6 and 7 should be used as the time-
domain data set.
Table 4
Failure Record Failure Time Time Between
Number (in Min.) Failures (in Min.)
1 20 20
2 32 12
3 42 10
4 58 16
5 88 30
6 90 2
7 111 21
8 114 3
9 120 6
10 127 7
Second approach is defined by counting the failures happening
during a period of time (e.g. minute, day or a testing session). This
approach is shown in the table number 5. In the table, we recorded
six failures in the first 1-hour interval, four failures in the 2 nd
interval and eight failures in the 3rd and so on.
 Table 5
Time Period (hours) Number of Faults Detected
1 6
2 4
3 8
4 3
The time-domain methodology always offers higher accuracy in
the methods of parameter estimates with current tools but absorbs
more data collection pains than the first one.
9. CONCLUSION AND IMPLICATIONS
FOR FUTURE WORK
Reliability is one of the most imperative characteristics of
software. More than hundreds of software reliability models were
proposed in last few decades. The aim of the study was to provide
the new comers and novice researchers a thorough guide on the
reliability growth models. We reviewed almost 60 models of
collectively of both type- parametric and non parametric
techniques. Our research shows the techniques used by researchers
till now to design the growth models and the pros and cons of the
methodologies used by them. Review of parametric and non
parametric models revealed the limitations of both the models. The
limitation of parametric models is that these models use so many
assumptions made by researchers and these assumptions vary from
researcher to researcher. If we talk about non parametric models
which use the concept of machine learning approaches take much
time in learning and require lots of training data to train the ML
model and also the training data is key factor of accuracy of non
parametric models. These limitations of the models can be
removed in near future. Also we provided the data sets and tools
which can be used in reliability estimation, this information of
tools and data sets are very crucial for the new players in the field.
10. REFERENCES
[1] System Software Reliability by Prof. Hoang Pham
Department of Industrial Engineering Rutgers the State
University of New Jersey, USA.
[2] Pai, Ganesh J. "A survey of software reliability models."
preprint: 1304.4539 (2013).
[3] Pham, Hoang. "Springer Series in Reliability
Engineering." New Jersey, USA (2005).
[4] Oman, Paul, and Jack Hagemeister. "Metrics for
assessing a software system's maintainability." Software
Maintenance, 1992. Proceerdings, Conference on. IEEE,
1992.
[5] Pham, Hoang. Software reliability. John Wiley & Sons,
Inc., 1999.
[6] Littlewood, Bev, and John L. Verrall. "Likelihood
function of a debugging model for computer software
reliability." Reliability, IEEE Transactions on 30.2
(1981): 145-148
[7] Z. Jelinski, P.B. Moranda, "Software reliability
research", in Statistical Computer Performance
Evaluation, ed. W. Freiberger, New York and London:
Academic Press, 1972, pp 465-484.
[8] Yamada, Shigeru, Mitsuru Ohba, and Shunji Osaki. "S-
shaped reliability growth modeling for software error
detection." Reliability, IEEE Transactions on 32.5
(1983): 475-484.
86
[9] Tohma, Yoshihiro, et al. "Structural approach to the
estimation of the number of residual software faults
based on the hyper-geometric distribution." Software
Engineering, IEEE Transactions on 15.3 (1989): 345-
355.
[10] Gentzler Jr, George L., and Nelson M. Andrews. "Data
stability in an application of a software reliability
model." Selected Areas in Communications, IEEE
Journal on 8.2 (1990): 273-275.
[11] Brocklehurst, Sarah, et al. "Recalibrating software
reliability models." Software Engineering, IEEE
Transactions on 16.4 (1990): 458-470.
[12] Kanoun, Karama, Marta Rettelbusch de Bastos Martini,
and Jorge Moreira De Souza. "A method for software
reliability analysis and prediction application to the
TROPICO-R switching system." Software Engineering,
IEEE Transactions on 17.4 (1991): 334-344.
[13] Tohma, Yoshihiro, et al. "The estimation of parameters
of the hyper geometric distribution and its application to
the software reliability growth model." IEEE
Transactions on Software Engineering 5 (1991): 483-
489.
[14] Kapur, P. K., and R. B. Garg. "A software reliability
growth model for an error-removal phenomenon."
Software Engineering Journal 7.4 (1992): 291-294.
[15] Yamada, Shigeru, Jun Hishitani, and Shunji Osaki.
"Software-reliability growth with a Weibull test-effort: a
model and application." Reliability, IEEE Transactions
on 42.1 (1993): 100-106.
[16] As H. Ascher, H. Feingold, Repairable System
Reliability: Modeling, Inference, Misconceptions, and
Their Causes, 1984; Marcel Dekker.
[17] Zeephongsekul, P., G. Xia, and S. Kumar. "Software-
reliability growth model: primary-failures generate
secondary-faults under imperfect debugging."
Reliability, IEEE Transactions on 43.3 (1994): 408-413.
[18] Tian, Jeff. "Integrating time domain and input domain
analyses of software reliability using tree-based models."
Software Engineering, IEEE Transactions on 21.12
(1995): 945-958.
[19] Pasquini, A., A. N. Crespo, and P. Matrella. "Sensitivity
of reliability-growth models to operational profile errors
vs. testing accuracy [software testing]." Reliability, IEEE
Transactions on 45.4 (1996): 531-540.
[20] Hou, Rong-Huei, Sy-Yen Kuo, and Yi-Ping Chang.
"Needed resources for software module test, using the
hyper-geometric software reliability growth model."
Reliability, IEEE Transactions on 45.4 (1996): 541-549.
[21] Hou, Rong-Huei, Sy-Yen Kuo, and Yi-Ping Chang.
"Optimal release policy for hyper-geometric distribution
software-reliability growth model." Reliability, IEEE
Transactions on 45.4 (1996): 646-651.
[22] Kuo, Lynn, et al. "Bayes inference for S-shaped
software-reliability growth models." Reliability, IEEE
Transactions on 46.1 (1997): 76-80.
[23] Tian, Jeff, and Joe Palma. "Analyzing and improving
reliability: A Tree-based Approach." IEEE software 2
(1998): 97-104.
[24] Chen, Mei-Hwa, Michael R. Lyu, and W. Eric Wong.
"Effect of code coverage on software reliability
measurement." Reliability, IEEE Transactions on 50.2
(2001): 165-170.
[25] Kuo, Sy-Yen, Chin-Yu Huang, and Michael R. Lyu.
"Framework for modeling software reliability, using
various testing-efforts and fault-detection rates."
Reliability, IEEE Transactions on 50.3 (2001): 310-320.
[26] Pasquini, Alberto, Giuliano Pistolesi, and Antonio Rizzo.
"Reliability analysis of systems based on software and
human resources." Reliability, IEEE Transactions on
50.4 (2001): 337-345.
[27] Lyu, Michael R., Sampath Rangarajan, and Aad Van
Moorsel. "Optimal allocation of test resources for
software reliability growth modeling in software
development." Reliability, IEEE Transactions on 51.2
(2002): 183-192.
[28] Teng, Xiaolin, and Hoang Pham. "A software-reliability
growth model for N-version programming systems."
Reliability, IEEE Transactions on 51.3 (2002): 311-321.
[29] Malaiya, Yashwant K., et al. "Software reliability growth
with test coverage." Reliability, IEEE Transactions on
51.4 (2002): 420-426.
[30] Huang, Chin-Yu, Michael R. Lyu, and Sy-Yen Kuo. "A
unified scheme of some nonhomogenous poisson process
models for software reliability estimation." Software
Engineering, IEEE Transactions on 29.3 (2003): 261-
269.
[31] Huang, Chin-Yu, and Chu-Ti Lin. "Software reliability
analysis by considering fault dependency and debugging
time lag." Reliability, IEEE Transactions on 55.3 (2006):
436-450.
[32] Huang, Chin-Yu, Sy-Yen Kuo, and Michael R. Lyu. "An
assessment of testing-effort dependent software
reliability growth models." Reliability, IEEE
Transactions on 56.2 (2007): 198-211.
[33] Zachariah, Babu, and R. N. Rattihalli. "Failure size
proportional models and an analysis of failure detection
abilities of software testing strategies." Reliability, IEEE
Transactions on 56.2 (2007): 246-253.
[34] Wang, Wen-Li, Thomas L. Hemminger, and Mei-Huei
Tang. "A moving average non-homogeneous Poisson
process reliability growth model to account for software
with repair and system structures." Reliability, IEEE
Transactions on 56.3 (2007): 411-421.
[35] Huang, Chin-Yu, and Wei-Chih Huang. "Software
reliability analysis and measurement using finite and
infinite server queueing models." Reliability, IEEE
Transactions on 57.1 (2008): 192-203.
[36] Schneidewind, Norman. "Comparison of reliability and
testing models." Reliability, IEEE Transactions on 57.4
(2008): 607-615.
[37] Sharma, Kapil, et al. "Selection of optimal software
reliability growth models using a distance based
approach." Reliability, IEEE Transactions on 59.2
(2010): 266-276.
[38] Huang, Chin-Yu, and Michael R. Lyu. "Estimation and
analysis of some generalized multiple change-point
software reliability models." Reliability, IEEE
Transactions on 60.2 (2011): 498-514.
[39] Kapur, P. K., et al. "Two dimensional multi-release
software reliability modeling and optimal release
planning." Reliability, IEEE Transactions on 61.3
(2012): 758-768.
[40] Yan, He. "NHPP software reliability growth model
incorporating fault detection and debugging." Software
Engineering and Service Science (ICSESS), 2013 4th
IEEE International Conference on. IEEE, 2013.
87
[41] Washizaki, Hironori, Kiyoshi Honda, and Yoshiaki
Fukazawa. "Predicting Release Time for Open Source
Software based on the Generalized Software Reliability
Model."
[42] Wayne, Martin, and Mohammad Modarres. "A Bayesian
Model for Complex System Reliability Growth under
Arbitrary Corrective Actions." Reliability, IEEE
Transactions on 64.1 (2015): 206-220.
[43] Sofer, Ariela, and Douglas R. Miller. "A nonparametric
software-reliability growth model." Reliability, IEEE
Transactions on 40.3 (1991): 329-337.
[44] Karunanithi, Nachimuthu, Darrell Whitley, and
Yashwant K. Malaiya. "Prediction of software reliability
using connectionist models." Software Engineering,
IEEE Transactions on 18.7 (1992): 563-574.
[45] Sitte, Renate. "Comparison of software-reliability-
growth predictions: neural networks vs parametric-
recalibration." Reliability, IEEE Transactions on 48.3
(1999): 285-291.
[46] Cai, Kai-Yuan, et al. "On the neural network approach in
software reliability modeling." Journal of Systems and
Software 58.1 (2001): 47-62.
[47] Costa, Eduardo Oliveira, et al. "Exploring genetic
programming and boosting techniques to model software
reliability." Reliability, IEEE Transactions on 56.3
(2007): 422-434.
[48] Wang, Zhiguo, Jinde Wang, and Xue Liang. "Non-
parametric estimation for NHPP software reliability
models." Journal of Applied Statistics 34.1 (2007): 107-
119.
[49] Wilson, Simon P., and Francisco J. Samaniego.
"Nonparametric analysis of the order-statistic model in
software reliability." Software Engineering, IEEE
Transactions on 33.3 (2007): 198-208.
[50] Zhang, Yongqiang, and Huifang Cheng. "Improved
Genetic Programming Algorithm Applied to Symbolic
Regression and Software Reliability Modeling." Journal
of Software Engineering and Applications 2.05 (2009):
354.
[51] Ayyoub, Belal, and Asim El-Sheikh. "A Model for
System Reliability Optimization Problems Based on Ant
colony Using Index of Criticality Constrain." ICIT 2009
Conference-Bioinformatics and Image Volume:
Bioinformatics and Image. 2009.
[52] Jin, C. "Software reliability prediction based on support
vector regression using a hybrid genetic algorithm and
simulated annealing algorithm." IET software 5.4 (2011):
398-405.
[53] Lo, Jung-Hua. "A study of applying ARIMA and SVM
model to software reliability prediction." Uncertainty
Reasoning and Knowledge Engineering (URKE), 2011
International Conference on. Vol. 1. IEEE, 2011.
[54] LI, JUNG-SHIAN, and MING-WANG GUO. "A Non-
Parametric Software Reliability Modeling Approach by
Using Gene Expression Programming " Journal of
Information Science and Engineering 21 (2005): 1201-
1211.
[55] Wason, Ritika, P. Ahmed, and M. Qasim Rafiq.
"Automata-Based Software Reliability Model: The Key
to Reliable Software." International Journal of Software
Engineering and Its Applications 7.6 (2013): 111-126.
[56] N. Kiran, V. Ravi, Software reliability prediction using
wavelet neural networks, in: iccima, IEEE Computer
Society, 2007, pp. 195–199.
[57] S. Zaidi, S. Danial, B. Usmani, Modeling inter-failure
time series using neural networks, in: IEEE International
Multitopic Conference, 2008, pp. 409–411.
[58] J. Lo, The implementation of artificial neural networks
applying to software reliability modeling, in:
Proceedings of the 21st annual international conference
on Chinese control and decision, IEEE Press, 2009, pp.
4385–4390.
[59] B. Yang, X. Li, M. Xie, F. Tan, A generic data-driven
software reliability model with model mining technique,
Reliability Engineering & System Safety 95 (6) (2010)
671–678.
[60] Arjestan, Mina Ebrahimi, and Seyed Hamidreza
Pasandideh. "Applying Meta-heuristic in a multi-
88
objective reliability problem with weight and space
constraints." (2013).
[61] Lyu, Michael R., and Allen Nikora. "CASRE: a
computer-aided software reliability estimation
tool." Computer-Aided Software Engineering, 1992.
Proceedings., Fifth International Workshop on. IEEE,
1992.
[62] Kanoun, Karama, et al. "SoRel: a tool for reliability
growth analysis and prediction from statistical failure
data." Fault-Tolerant Computing, 1993. FTCS-23. Digest
of Papers., The Twenty-Third International Symposium
on. IEEE, 1993.