Software Quality Prediction Based on Information Analysis - A Decision Tree
Approach
Gordan Topić and Dragan Jevtić*
Ericsson Nikola Tesla d.d, Research & Development Center, Krapinska 45, 10000 Zagreb, Croatia
*University of Zagreb, Department of telecommunications, Unska 3, HR-10000 Zagreb, Croatia
E-mail: gordan.topic@etk.ericsson.se, dragan.jevtic@fer.hr
Abstract: The problem analyzed in this paper is related to the
software development in telecommunication industry. It is
evaluated and presented as a decision-making task comprising
non-numerical parameters. The results are focused on
extracting the conclusions incorporated in the conflicting
criteria such as minimum costs, maximum quality and
minimum production time.
ID3 algorithm is used to find the gain of each non-numerical
attribute and to make final decision tree. The motivation of this
job is to find auxiliary tools for resolving complex decision-
making tasks through the introduction of computer supported
and automated prediction in software production. The
obligatory quality limits were related to the fault density
parameter. The model was based on 20 different software
modules coded by different programmers and under different
conditions. Critical attributes were detected and considered in
decision tree analysis.
1. INTRODUCTION
Software development industry employs the most complex
processes to create software products from the basic brain
idea. Every process must be measured and the product tested
at different development phases, to minimize the program
errors and product’s costs of development [1, 2 and 4]. Many
different parameters, not included in the numeric calculations
and not easy to measure, determine software quality.
Consequently, drawing of the conclusions that are key for
implementing corrective actions are not facilitated by these
data. Some of these parameters arise from software testing
and document inspection, while the others have non-
numerical nature. For example, numerical values are not
sufficient to describe a software creator, designer or tester, or
his/her behavior and work. Human behavior and work can be
described in words only rather than numerically and
therefrom workability is expressed. The problem appears
with the use of these non-numerical values for software
quality prediction, especially when there are many
parameters to be met. This paper offers one of the solutions
as a model, which can be used to monitor and improve
software quality management in decision-making.
The method applied in this software quality prediction is
based on Shannon and Weaver information theory and
information gain calculation measured by entropy. The
algorithm used for data classification and introduction of
Decision Trees, also known as the ID3/C4.5 algorithm, is
introduced by Quinlan [5, 7, 9].
The problem in this paper appears in software development
and can be formulated by the question: How to make
software with minimum expenditures, minimum time
consumption and maximum quality? The problem can be
solved only by neglecting less important things that is, in the
swarm of information, crucial for high productivity.
However, the key question is how to distinguish more
important from less important information parameters? Is it
spending time for code programming, or the quantity of tests
and document inspection, or someone’s competence? Other
parameters are relevant for the decisions like stress, which
arises from erroneous estimation of the time to complete a
task. Major important parameters, just like quality, are the
costs.
2. CREATING TRAINING DATASET FROM
DATABASE AND MANAGEMENT
MONITORING
The assumption is that the information is collected during
software development. The recorded and analyzed
information are taken from:
 Software Document Inspections (SDIs)
 Basic and Function Test results (BT, FT)
 planned Cost Precision (CP)
 planned Lead Time Precision (LTP)
 Fault Density (FD)
 volume of the code per module
 management monitoring
 work competence.
This example is based on measuring 20 different modules,
coded by different programmers under different conditions.
Each module has a size or code volume, and every module QUALITY attribute is derived from Fault Density
passes Basic and Function Test. The documents produced measurement and talks about good or poor quality module.
before coding are reviewed by the inspection teams in SDIs. This attribute is the final state of a decision tree with two
The quality of every module is visible in the ratio between different states (good and poor).
the number of trouble reports from function test and code
volume, also known as fault density. ATTRIBUTE VALUES DERIVED
Many coding mistakes are corrected in SDIs (60%), PROGRAMMING_SPEED fast, slow, average LTP (prj. mng)
making it the best way to decrease the number of trouble STRESS high, normal, low Line mng.
INSPECTIONS yes, no Insp. records
reports on a product when it is launched on the market. On
the other side, SDIs need human resources, many man-hours PROGRAMMER expert, novice, learned Comp. mng.
QUALITY good, poor FD
and regular scheduling. Unfortunately, the inspections often
take less time than the good SDIs require. Table 1 shows a Table 2 Attributes with their values
database with the information about few modules; SDIs and
results of Function Testing of the modules. First of all, database with training information has to be
created using discrete attributes and their values for
INSPECTION TESTING describing the modules shown as the records in the database.
mod I-hrs pages mist pg/h volume TRs FD
Table 3 shows the created training database.
M-0 10 8 2 0.8 3347 2 0.6
M-1 4 7 3 1.75 2324 2 0.86
M-2 18 18 9 1 6841 6 0.88 REC PROGRAMMER PROG. INSP. STRESS CLASS
M-3 10 51 10 5.1 4563 3 0.66 No. SPEED FIELD
M-4 4 14 7 3.5 1320 3 2.27 0 expert fast yes high good
Abbreviations: 1 expert slow no normal good
mod name of the module 2 expert average yes low good
I-hrs total time used for software document inspection 3 expert average no low good
pages number of pages of the inspected document 4 expert fast no high poor
mist number of mistakes in the text that produce functional error 5 expert average no high poor
pg/h inspection velocity (pages / hours) 6 expert slow yes normal good
volume code volume in the module (measured in kilo-lines of code) 7 novice fast yes high poor
TRs number of trouble reports (Function Test) 8 novice slow yes normal good
FD fault density (TRs / volume) 9 novice average no normal poor
2.27 exceeded parameter value 10 novice slow no normal poor
11 novice average yes normal good
Table 1: Results of SDIs and Function Test 12 novice fast no normal poor
13 learned fast yes normal good
14 learned average no normal poor
As shown in the table, only M-0 and M-2 modules have good 15 learned slow no high poor
16 learned average no low good
inspection velocity, being within the recommended of one
17 learned average yes normal good
page per hour [11]. Fault Density is a measurement defined 18 learned average no high poor
by quality and project management. In this case the 19 learned average no normal poor
borderline is one trouble report per 1000 non-commented
lines of a program code without the empty lines. Table 3 Training database with discrete values
Consequently, only module M-4 has a higher value of the
fault precision parameter. The table shows that few variables From the data shown in Table 3, a decision tree is created
for a decision tree algorithm can be derived, such as the using ID3 algorithm. As shown, each record has the same
attributes INSPECTION with values (yes/no) and QUALITY structure.
(good/ poor) as a class field, i.e. quality of the module. Other
attributes, like PROGRAMMING_SPEED (fast, slow,
average) could be derived from the project management 3. ID3 ALGORITHM
information, such as Lead Time Precision (LTP), which is the
percentage between the planed and the realized programming Algorithms ID3/C4.5 are introduced by Quinlan for inducing
time in days. Also, the percentage between the planned and Classification Models, also called Decision Trees, from data.
the realized man-hours can be used for the same purpose, The task is to determine a decision tree that, on the basis of
known as Cost Precision (CP). The attribute like STRESS the answers to questions about non-categorical attributes,
(high, normal, low) can be derived from the monitoring of the correctly predicts the value of the category attribute named
line and project managers, and PROGRAMMER (expert, QUALITY. Category attribute takes the values (good/poor).
novice, learned) can be derived from the database of In any case, one of its attributes takes the meaning “failure”,
competence management. Table 2 shows the attributes with i.e. poor quality module [7].
their values. All attributes are discrete. A decision tree consists of the nodes and unidirectional
arcs, which connect the nodes. To make a decision, one has
to start from the root node, ask the questions to determine the
arc to follow, and reach the leaf node. Main points of the ID3
algorithm are:
 each non-leaf node of a decision tree corresponds to
an input attribute, and each arc to a possible value of
that attribute. A leaf node corresponds to the
expected value of the output attribute (QUALITY)
when the input attribute is described by the path
from the root node to that leaf node;
 in a “good” decision tree, each non-leaf node should
be associated to the input attribute which is the best
informative output attribute among the unconsidered
input attributes on the path from the root node to
that node, and
 entropy is used as a measure to determine how
informative a particular input attribute is about the
output attribute for a subset of the training data.
Entropy as a measure of uncertainty in the communication
systems is introduced by Shannon [8].
Consider the source S that can produce n messages {m1, m2,...,
mn}. All messages are produced independently of each other,
and the probability of producing the message mi is pi. For
such a source S with the message distribution probability P =
(p1, p2,..., pn) the entropy H(P) is:
If a set T of the records from the training database are
partitioned into k classes {C1, C2,..., Ck} on the basis of the
output attribute, then the average amount of information
(measured in bits) needed to identify the class of record is
H(PT). PT is distribution probability of the classes, estimated
from the data as:
The notation |Ci| means the number of elements in the set Ci.
For the training database, where the QUALITY is the output
attribute, PT for the entire dataset T is:
where class C1 corresponds to “good” and class C2 to “poor”.
Consequently, the entropy of the set T is:
The set, on the basis of an input attribute X, is partitioned
into the subsets T1,T2,...,Tn. The information needed to
identify the class of an element of T is the weighted average
information needed to identify the class of an element of each
subset:
Quantity of information about the output attribute that can
be obtained by knowing that the value of an input attribute X
is the difference between the information needed to classify
an element of T before knowing the value of X, H(T), and the
information needed after partitioning the dataset T on the
basis of the known value of X, H(X, T). The information gain
is defined, due to the attribute X for set T, as:
Gain(X, T) = H(T) – H(X, T)
In order to decide which attribute to split, the ID3
algorithm computes the information gain for each attribute,
and selects the one with the highest gain. For example, the
information gain for the attribute PROGRAMMER is
calculated. The attribute PROGRAMMER can have three
values: expert, learned and novice [8]. Partitioning on the
basis of the PROGRAMMER will, thus, divide T into three
subsets: |Texpert| = 7, |Tnovice| = 6 and |Tlearned| = 7. Calculation of
the information needed to classify an element of T after this
partitioning is:
The information gain for the attribute PROGRAMMER for
the set T is:
Consequently, the ID3 Algorithm works by recursive
application of the procedure above each subset produced,
until the “pure” nodes are found, or until there are no
attributes left to consider. A pure node contains only one-
class elements [8]. Shortly, the ID3 algorithm for a decision
tree construction can be described as follows:
 a tree starts a single node representing all data;
 if all the samples are of the same class, then the
node becomes leaf labeled with the class label;
 otherwise, select the attribute that separates the
sample into individual classes best, and
 recursion stops when the samples in a node belong
to the same class, or if there is no attribute left on
which to split, or if there are no samples with the
attribute value.
The finished tree is shown in figure 1:
Picture 1: Decision Tree created by ID3 algorithm
4. DECISION TREE ANALYSIS
The most informative attribute being a case in the
PROGRAMMING_SPEED, the attribute is set as a root
node. Therefrom, three branches proceed to the next level,
being the best informative attribute of the remaining
attributes in the training database. The tree has four levels:
PROGRAMMING_SPEED, STRESS, INSPECTION and
the last is the PROGRAMMER. Some conclusions can be
made on the basis of the tree. For example:
 if a novice or a programmer beginner is not
suitable for quality programming, someone else
as an expert programmer can support the novice.
 Slow programming speed creates quality
modules, because there is enough time for the
programming and for problem solving. In the
extremes (high stress and poor programming
without SDIs) the quality is not acceptable.
 High stress in any case creates poor quality
modules, but with good programmers (experts
and trained people) and SDIs support the modules
can be made good.
 Only slow programming and low level of stress
can produce the quality modules without SDIs.
The exception is when a novice creates good
quality software, even in these conditions.
 SDIs are always suitable for reducing the errors
and improving the quality.
Consequently, from this information quality and project
management can draw some useful conclusions before a
project starts. Prior to that, they must have some
information e.g. about the people’s competence, the time
required for project realization, volume of the job, etc.
Basic rules that can be derived from the decision tree
results are:
 if quality is the aim, the novices cannot be
involved in the programming if not mentored by
the experts or trained people, even if the
programming conditions for them are good and
suitable. The problem is in the incompetence and
insufficient experience.
 In any case stress must be controlled and rapidly
reduced. If this is not possible, the SDIs can
improve the quality in the conditions of high
stress and fast programming.
 In any case SDIs must be undertaken. Only in the
conditions of slow programming and low level of
stress, SDIs can be excluded from the process.
 The programming speed should not be too high
and must be controlled, because stress also
depends on it.
5. CONCLUSION
The model of database monitoring described in this
paper can be used to review and analyze any large
database with too many information to be analyzed before
a conclusion can be drawn. Today, all software industries
measure software development and quality of their own
products [4]. Many standards, as ISO 9000/2000, require
monitoring and measuring of the processes and products’
quality. This information can be useful in a process
correction and fault density prediction [1 and2].
The problem is with many different information that
cannot be processed by human brain for making the logical
conclusion. Consequently, the management can use the
method described herein to simplify the decisions about
corrective actions. This method can be applied in a product
quality monitoring, just like the Petri net simulation is
applied in a process analysis and correction. Together,
these models can make a good structure for the control of
the whole software development with product
measurement [3]. The model used in this paper was based
on 20 different software modules. Two parameters with
largest information gain were detected: these are
Programming speed and Stress. These parameters are
qualified as particularly important, and must be observed
and if possible controlled during software production
process. Four basic rules for software development process
are extracted as a result of decision tree analysis.
[6] Osmar R. Zaiane, “Principles of Knowledge Discovery
in Databases”, Chapter 7: “Data Classification”,
University of Alberta, 1999.
[7] “Building Classification Models: ID3 and C4.5”,
UGAI Lectures, Workshop: "Providing and Integrating
Educational Resources for Faculty Teaching Artificial
Intelligence", Temple University in Philadelphia, June
20 - June 25, 1994,
http://yoda.cis.temple.edu:8080/UGAIWWW/
[8] Tutorial: “Decision Trees: ID3”, Monash University,
Faculty of Information Technology, CSE5230 Data
Mining, Semester 2, 2002
[9] Lucas Ballard, “Topics in Machine Learning –
Decision Tree Learning”, Web Project September 18,
2002
[10] G. Topić and D. Jevtić, ”Process Measuring and
Monitoring in Multi-process industry using Petri Nets”,
Proceedings of the 10th International Conference on
Software, Telecommunications and Computer
Networks - SoftCOM 2002, pp. 35-39, Split, Venice,
Ancona, Dubrovnik, Croatia, 2002.
[11] Tom Gilb and D. Graham “Software Inspections”,
Addison Wesley Longman Limited, 1993.

6. REFERENCES
[1] Quality management systems – Requirements,
International Standard ISO 9001, Third edition, 2000-
12-12.
[2] Quality management systems – Guidelines for
performance improvements, International Standard ISO
9004, Second edition, 2000-12-15.
[3] Measurement: 3-day training, Q-Labs’ Presentation
Template: Managing Software Risks with World -
Class Customers, Conducted by Erik Johansson, Johan
Brantestam.
[4] Ericsson Quality Auditing, Ericsson Quality Institute,
LME-Q 038 19-EN-LZU 110 7112-29 Uen Rev A
1995-04-18.
[5] Joseph P. Bigus, Jennifer Bigus, “Constructing
Intelligent Agents Using Java”, Second edition, Wiley
Computer Publishing, John Wiley & Sons, Inc, 2001,
chapter 5:”Learning Systems”