SOFTWARE QUALITY ASSURANCE

LECTURE # 14

METRICS AND MEASUREMENT

26th January, 2022 Dr. Ali Javed

 Contact Information
2

q Course Instructor: Dr. Ali Javed

Associate Professor
Department of Computer Science
U.E.T Taxila
¤ Email: ali.javed@uettaxila.edu.pk
¤ Website: https://fms.uettaxila.edu.pk/Profile/ali.javed

¤ Contact No: +92-51-9047747

¤ Office hours:
n Monday, 09:00 - 11:00, Office # 7 S.E.D

q Course TA: Engr. Farman

Dr. Ali Javed

 Course Information
3

q Course Name: Software Quality Assurance

q Course Code: CS-303

Dr. Ali Javed

 4 Metrics & Measurement
ü Metric vs Measurement
ü The 4 P’s of Measurement
ü Product Metrics
ü Process Metric
ü Project Metrics
ü People Metrics

Dr. Ali Javed

 Measure vs Metric
5

q Measure - quantitative indication of extent, amount, dimension, capacity, or

size of some attribute of a product or process.
ü E.g., Number of errors

q Metric - quantitative measure of degree to which a system, component or

process possesses a given attribute.
ü E.g., Number of errors per KLOC

Dr. Ali Javed

 6 The 4P’s of Software Measurement

With regards to software, we can measure:

ü Product
ü Process
ü Project
ü People

Dr. Ali Javed

 7 Product Metrics

ü Size metrics
ü Defects-based metrics
ü Cost-metrics
ü Time metrics

Dr. Ali Javed

 Measuring the Product
8

q Product refers to the actual software system,

documentation and other deliverables

q We examine the product and measure a number of

aspects:
ü Size
ü Functionality offered
ü Cost
ü Various Quality Attributes

Dr. Ali Javed

 Size Metrics
9

q Size of the software produced

q LOC - Lines Of Code

q KLOC - 1000 Lines Of Code

q Some Common Metrics:

ü Defects/KLOC, Cost/LOC, Documentation Pages/KLOC

q Knowing the size of a system is important for comparing

different systems together
Dr. Ali Javed
 LOC Metrics
10

q Easy to use

q Easy to compute

q Language & programmer dependent

Dr. Ali Javed

 The problems with LOC (1/3)
11

q Same system developed with different programming

languages will give different LOC readings

Video Rental
System

FoxPro Pascal Assembly

2 KLOC 5 KLOC 15 KLOC

Dr. Ali Javed

 The problems with LOC (2/3)
12

q Same system developed by different developers using the

same language will give different LOC readings

Video Rental
System

Developer A Developer B Developer C

2 KLOC 1.2 KLOC 2.5 KLOC

Dr. Ali Javed

 The problems with LOC (3/3)
13

q To calculate LOC you have to wait until the

system is implemented

q This is not adequate when management

requires prediction of cost and effort

q A different approach is sometimes necessary…

Dr. Ali Javed

 Function Points [1]
14

q Instead of measuring size, function points measure the functionality offered by a

system.

q Invented by Albrecht at IBM in 1979, still use today: http://www.ifpug.org

q Function points can be calculated before a system is developed

q They are language and developer independent

Dr. Ali Javed

 Function Points
15

q There are two types of functions

ü Data Functions
ü Transaction Functions

q Data Functions
ü Internal Logic Files
ü External Interface Files

q Transaction Functions
ü External Inputs
ü External Outputs
ü External Inquiries

Dr. Ali Javed

 Function Points
16

q Transaction Functions
ü External Inputs: External Input (EI) is a transaction function in which Data goes “into” the application from
outside the boundary to inside. This data is coming external to the application. Data may come from a
data input screen or another application.
ü External Outputs: External Output (EO) is a transaction function in which data comes “out” of the system.
The data creates reports or output files sent to other applications.
ü External Inquiries: External Inquiry (EQ) is a transaction function with both input and output components
that result in data retrieval.
q Data Functions
ü Internal Logic Files: Internal Logical File (ILF) is a user identifiable group of logically related data or
control information that resides entirely within the application boundary. An ILF has the inherent meaning
that it is internally maintained, it has some logical structure and it is stored in a file.
ü External Interface Files: are responsible for exchanging data with other systems e.g files accessed by the
application but not maintained by it

Dr. Ali Javed

 Function Point Analysis
q These function-point counts are then weighed (multiplied) by
their degree of complexity:
Simple Average Complex
Inputs 2 4 6
Outputs 3 5 7
Files 5 10 15
Inquires 2 4 6
Interfaces 4 7 10

Dr. Ali Javed

 Function Points
18

q The simplest way to calculate a function point count is calculated as follows:

(No. of external inputs x 4) +

(No. of external outputs x 5) +
(No. of logical internal files x 10) +
(No. of external interface files x 7) +
(No. of external enquiries x 4)

Dr. Ali Javed

 Function Points Example

Dr. Ali Javed

 Function Points Example
20

q External Inputs: 1
q External Outputs: 2
q Logical Internal Files: 1
q External Interface Files: 0
q External Enquiries: 1

q Total Functionality is (1x4) + (2x5) + (1x10) + (0x7) +(1x4) = 28

Dr. Ali Javed

 The GSC Function Points Extension
q Reasoning: Original Function Points do not address certain functionality
which systems can offer

q E.g. Distributed functionality, performance optimisation, etc

q The GSC extension involves answering 14 questions about the system and
modifying the original function point count accordingly

Dr. Ali Javed

 The GSC Function Points Extension
8. On-line update
9. Complex Processing
10. Reusability
11. Installation ease
12. Operational Ease
13. Multiple sites
14. Facilitation of Change

Dr. Ali Javed

 Degrees of Influence
ü 0 Not present, or no influence
ü 1 Incidental influence
ü 2 Moderate influence
ü 3 Average influence
ü 4 Significant influence
ü 5 Strong influence throughout

Dr. Ali Javed

 The GSC Function Points Extension
q The analyst/software engineer assigns a value between 0 and
5 to each question
q 0 = not applicable and 5 = essential
q The Value-Adjustment Factor (VAF) is then calculated as:
14
VAF = 0.65 + 0.01å Ci
i =1
You then adjust the original function point count as follows:

FP = FC x VAF

Dr. Ali Javed

 GSC Example

Consider the bug-reporting system for which we already looked at and

suppose the analyst involved answers the GSC questions as follows…
1. Data communications 5 8. On-line update 3

2. Distributed Functions 0 9. Complex Processing 1

3. Performance 1 10. Reusability 0

0 2
4. Heavily used 11. Installation ease
configuration 12. Operational Ease 3

Transaction rate 1 4
5. 13. Multiple sites
Online Data Entry 5 0
6. 14. Facilitation of Change
0
7. End-user Efficiency
Total GSC Score = 25
Dr. Ali Javed
 GSC Example
q As you may remember, when we calculated the function point count for this
system, we got a result of 28.

q If we apply the GSC extension, this count will be modified as follows.

VAF = 0.65 +(0.01 x 25) = 0.9

FC = 28 x 0.9 = 25.2

q Note that the GSC extension can increase or decrease the original count

q In larger systems, the GSC extension will have a much more significant
influence on the Function Point Count.

Dr. Ali Javed

 Defect Density
27

q A rate-metric which describes how many defects occur for each

size/functionality unit of a system

q Can be based on LOC or Function Points

# defects
system _ size
Dr. Ali Javed
 Failure Rate
28

q Rate of defects over time

q May be represented by the λ (lambda) symbol
where,
t1 and t2 are the beginning and ending of a specified
interval of time
R(t) is the reliability function

R(t1 ) - R (t 2 )
l=
(t 2 - t1 ) ´ R(t1 )
Dr. Ali Javed
 Example of Failure Rate
29

q Calculate the failure rate of system X based on a time interval

of 60 days of testing. The probability of failure at time day 0
was calculated to be 0.85 and the probability of failure on day
60 was calculated to be 0.2.

Dr. Ali Javed

 Example of Failure Rate
30

q Calculate the failure rate of system X based on a time interval

of 60 days of testing. The probability of failure at time day 0
was calculated to be 0.85 and the probability of failure on day
60 was calculated to be 0.2.

R(t1 ) - R (t 2 )
l=
(t 2 - t1 ) ´ R(t1 )
0.85 - 0.2
l=
60 ´ 0.85
0.65
=
51
Dr. Ali Javed = 0.013 Failures per day
 Mean Time Between Failure
31

q MTBF refers to the amount of time that elapses between one failure and the next

q MTTF is the time that a system is not failed, or is available. Often referred to as
“uptime”

q MTTR is the expected time to recover a system from failure

q MTTR affects the Availability of a system, if it takes longer to recover system from
failure then the system is going to have a lower Availability

q MTBF = MTTF + MTTR

q Availability = MTBF/(MTBF+MTTR)

Dr. Ali Javed

 Interface Complexity (Henry Fan in/ Fan
out Metric) Information Flow Metric [2]
q Interface Complexity measures complexity as a function of
Fan in and Fan out

q Counts the number of local information flows entering (fan-in)

and exiting (fan-out) each procedure. Thus, for a module
complexity can be defined as

q Complexity = [ procedure length] x [ fan-in x fan-out] ^ 2

q This metric is helpful in measuring coupling between files,

procedure or objects.

Dr. Ali Javed

 Interface Complexity (Henry Fan in/ Fan
out Metric) Information Flow Metric [2]

q To better understand visualize it as graph by taking module (file, procedure

or node) as node and call between them as edges.

q Fan-in of a module is the number of other modules calling to this module

directly.

q Fan-out of a module is the number of other modules immediately called by

this module.

Dr. Ali Javed

 Interface Complexity (Henry Fan in/ Fan
out Metric) Information Flow Metric

q In this example module A has Fan-in = ___ and fan-out = ___

q Consider the length of module A to be 100 LOC then What will be the
Interface Complexity of A?
Dr. Ali Javed
 35 Process Metrics

Dr. Ali Javed

 Why measure the process?
q The process creates the product

q If we can improve the process, we indirectly improve the product

q Through measurement, we can understand, control and improve the process

q We will look briefly at a number of process metrics

ü Effort required in the process

ü Time to produce the product

ü No. of defects found during testing

ü Maturity of the process

Dr. Ali Javed

 Phase-Based Defect Removal Pattern
q An extension of the defect density metric

q Tracks defects at all phases of the lifecycle e.g Requirements,

Design, Implementation, Testing, Maintenance.

q The earlier defects are found, the cheaper they are to fix

q This metric helps you monitor when your defects are being
found

Dr. Ali Javed

 Defect Removal Efficiency (DRE)

q With the help of DRE we can measure

what quantity of software bugs we
identified from the number of software
bugs which we could have identified

q DRE benefits both at project level and

process level

q DRE = E/ (E + D)
ü E: No. of Errors found before delivery
ü D: No. of Defects found after delivery

q Ideally DRE should be 1 (means D=0)

q DRE can also be used within a project

team, to measure the teams efficiency

Dr. Ali Javed

 Other useful process metrics
q Fix response time
ü Average time to fix a defect

q Percent delinquent fixes

ü Fixes which exceed the recommended fix time according to
their severity level

q Fix quality
ü Percentage of fixes which turn out to be defective

Dr. Ali Javed

 40 Project Metrics

Dr. Ali Javed

 Project Metrics
q Describes the project characteristics and execution

ü No of Software developers, testers

ü Staffing patterns over the lifecycle of software
ü Cost and Schedule
ü Productivity

Dr. Ali Javed

 42 People Metrics

Dr. Ali Javed

 Measuring the People
43

q Involves analysis of the people

developing a product

q How fast do they work?

q How much bugs do they produce?

q How many sick-days do they take?

q Very controversial. People do not like

being turned into numbers.
Dr. Ali Javed
 Why measure people?
q People metrics are of interest to
management for:

ü Financial purposes (e.g. Putting Joe

on project A will cost me USD500
per function point)
ü Project management purposes (e.g.
Michele needs to produce 5 function
points per day in order to be on
time)
ü HR problem identification (e.g. On
average, developers produce 5
defects per hour. James produces
10. Why?)

Dr. Ali Javed

 Warning on People Measurement
q People do not like being measured

q In many cases, you will not be able to look at a number and draw
conclusions.

q For example, at face value, Clyde may take a longer time to finish his work
when compared to colleagues. However, further inspection might reveal that
his code is bug free whilst that of his colleagues needs a lot of reworking

q Beware when using people metrics. Only use them as indicators for
potential problems

q You should never take disciplinary action against personnel based simply on
people metrics

Dr. Ali Javed

 Some people metrics…
For individual developers or teams:
q Cost per Function Point
q Mean Time required to develop a Function Point
q Defects produced per hour
q Defects produced per function point

Dr. Ali Javed

 References
47

1. http://groups.engin.umd.umich.edu/CIS/course.des/cis525/js/f00/artan/function
points.htm
2. http://www.whiteboxtest.com/information-flow-metrics.php

Dr. Ali Javed

 For any query Feel Free to ask
48

Dr. Ali Javed