Bayesian Graphical Models

for Software Testing

David A Wooff, Michael Goldstein, Frank P.A. Coolen

Presented By
Scott Young
 Applying Detailed Metrics to
Testing
 Can provide insight for performing risk
analysis
 Can provide concrete values to inform the
customer or management, concerning
software quality
 May result in more efficient testing
procedures
 Meaningful Metrics
 Simple metrics concerning testing can inform
concerning the success of the testing process
 They also can provide insight for when the test
process is reaching completion
 More complex testing analysis needs to be done
in order to locate points of inefficiency in testing,
as well as providing more fine-grained
knowledge about test results
What is a Bayesian Graphical
Model?
 Also commonly called a Bayesian Belief Network
 Is a directed graph, with nodes signifying
indeterminate factors
 Bayesian models are most commonly heard of
today in relation to email spam filtering
 They are used to calculate probability based on
pre-defined knowledge and relationships
between components
 Software Actions (SA’s)
 A Software Action is an individual, fine
grained component of the software project
which accomplishes a single task.
 An example of a software action in a
system would be the processing of a credit
card number.
 Specifying Nodes
 A node should be a collection of
operations with the same prior probability
of failure, as well as the same change in
probability of failure given a test covering
that set of operations.
 Factors For Defining
Probabilities
 Level of code complexity
 Reliability comparison with existing code
which has been evaluated
 Maturity of codebase
 Typical reliability of author’s code
 Similarities to existing code
 Updating The Model
 As testing continues, the model must be
update in stepwise fashion to follow
changes to components as they occur.
 The probability of an individual node can
be updated according to multiple criteria
(which are necessarily assumptions) about
remaining defects.
 What The Results Provide
 Tests should be arranged according to the
software action(s) which they provide
coverage for. Tests discovered to be
redundant may be safely removed
 Results demonstrate the perceived
probability (or strength of
belief/confidence) that there are no more
existing faults within each SA
 What Does This Mean For
V&V?
 Software producers can demand a level of
confidence for components from their
testing according to the role of the
software and potential financial impact of
defects in specific components.
 Drawbacks
 Informed knowledge is required in order to
build a reliable model
 This “informed knowledge” still consists of
assumptions of relationships (though an
assumption within an order of magnitude
can still provide useful results)
 The amount of additional work to formally
track every SA may be prohibitive
 Resources
 David A Wooff, Michael Goldstein, Frank P.A.
Coolen, “Bayesian Graphical Models for
Software Testing”. IEEE Transactions on
Software Engineering, May 2002.
 Murray Cumming, “Bayesian Belief Networks”.
http://www.murrayc.com/learning/AI/bbn.shtml
Date unknown.
 Kevin Murphy, “A brief introduction to Bayes’
Rule”.
http://www.ai.mit.edu/~murphyk/Bayes/bayesrul
e.html, Jan 2004.