Lecture # 3: Software System Quality

SOFTWARE SYSTEM QUALITY
LECTURE # 3
QUALITY ASSURANCE AS DEALING WITH

DEFECTS
20th October, 2021 Dr. Ali Javed

Contact Information
2
q Instructor: Dr. Ali Javed

Associate Professor
Department of Computer Science
U.E.T Taxila
¤ Website: http://fms.uettaxila.edu.pk/Profile/ali.javed
¤ Email: ali.javed@uettaxila.edu.pk
¤ Contact No: +92-51-9047747
¤ Office hours:
n Monday, 09:00 - 11:00, Office # 7 S.E.D
Dr. Ali Javed

Course Information
3
q Course Name: Software System Quality
q Course Code: SE-5001
Dr. Ali Javed

4 Topics to Cover
q Quality Assurance
q Dealing with Pre/Post Release Defects
q Qualification Scheme for QA as dealing with defects
ü Defect prevention
ü Defect Reduction
ü Defect Containment
Dr. Ali Javed

5 Quality Assurance
Dr. Ali Javed

Quality Assurance [1,2]
6
q Quality assurance can be defined as, ‘the establishment of framework of

organizational procedures and standards that lead to a high-quality software’
q Quality assurance is the function responsible for managing quality. The word
“assurance” means that if the processes are followed, management can be assured
of product quality.
q Classification Scheme for QA as Dealing with Defects
ü Defect Prevention
ü Defect Reduction
ü Defect Containment
Dr. Ali Javed

Dealing with Pre/Post Release Defects
7
q Pre-release
ü Defect prevention
ü Defect Reduction (detection and removal)
ü Several approaches including development methodologies, programming techniques, debugging tools, testing,
beta testing, etc.
q The faults left in the finished software products are often called “Dormant
defects/bugs” which may stay dormant for some time but have the potential of
causing problems to customers and users of the products (a situation which we would
like to avoid)
Dr. Ali Javed

Dealing with Pre/Post Release Defects
8
q Post-release
ü Defect containment
ü Dormant defects (hidden defects)
ü Bug reports
ü Cost factor
ü Reputation factor
q After product release, the failures observed and problems reported by customers and
users also need to be fixed, which in turn, could lead to reduced defects and
improved product quality.
q How ever, one cannot rely on these post-release problem reports and give up pre-
release defect prevention and reduction activities, because the cost of fixing defects
after product release is significantly higher than before product release.
Engr. Ali Javed

9 Defect Prevention [7]
Dr. Ali Javed

Defect Prevention [6,9]
10
1. Defect prevention
q Prevent faults from being injected into the software through error blocking or
error source removal
q Eliminating certain error sources, such as correcting human misconceptions
q We can analyze the reasons behind the missing or incorrect human actions and
deal with the root causes or the error sources instead. This generic approach is
called error source removal.
q The focus of these activities is typically on the people and their conceptual
mistakes, which may lead to the selection and use of inappropriate
development methodologies, languages, algorithms, QA strategies, etc. Such
inappropriate selection may lead to numerous fault injections.
Dr. Ali Javed

Defect Prevention Techniques
11
q Education and training
q Process conformance and standards

enforcement
q Tools/technologies and techniques
q Formal Methods
Dr. Ali Javed
Education and Training
12
q Education and training of software professionals can help them control,

manage, and improve the way they work.
q The elimination of these human misconceptions will help prevent certain types
of fault from being injected into software products.
Dr. Ali Javed

13
The education and training effort for error source

elimination should focus on the following areas:
q Product and domain specific knowledge
ü If the people involved are not familiar with

the product type or application domain,
there is a good chance that wrong solutions
will be implemented
q Knowledge about the specific

development/testing tools used by the
organization
ü Also plays an important role in developing

high- quality software products.
Dr. Ali Javed

14
q General Software Development Knowledge and expertise
ü It plays an important role in developing high quality software products
ü For example lack of expertise with requirements analysis and product

specification usually leads to many problems and rework in design, coding
and testing phases
Dr. Ali Javed

15
q Development Process Knowledge used by the organization
ü If the project personnel do not have a good understanding of the

development process involved, there is a little chance that a process can
be implemented correctly.
ü For example, if the people involved in incremental software development

do not know how the individual development efforts for different
increments fit together, the uncoordinated development may lead to many
interface or interaction problems.
Dr. Ali Javed

Tools/Technologies and Techniques
16
q Appropriate use of software methodologies can also help reduce the chances of fault injections. Many of
the problems with low quality “fat software” could be addressed by disciplined methodologies and return
to essentials for high quality “Lean software”
q A better managed process can also eliminate many systematic problems. For example, not having the
defined process or not following it for software configuration management may lead to inconsistencies or
interface problems among different software components. Therefore, ensuring appropriate process
definition and conformance helps eliminate such error sources.
q Specific software tools can also help reduce the chances of fault injections. For example, Syntax directed
editor that automatically balances out each open parenthesis, “{”, with a close parenthesis, “}”, can help
reduce syntactical problems in programs written in the C language
Dr. Ali Javed

Formal Methods
17
q Formal methods includes formal specification and formal verification
ü Formal specification is concerned with producing an unambiguous set of product

specifications so that the customer requirements, as well as environmental constraints and
design intensions are correctly reflected.
ü Formal verification checks the conformance of software design or code against these
formal specifications, thus ensuring that the software is fault free with respect to its
formal specification.
Dr. Ali Javed

18 Defect Reduction [7]
Dr. Ali Javed

Defect Reduction [6,9]
19
2. Defect reduction through fault detection and removal
q Detect and remove faults once they have been injected
q Two categories:
ü Static Testing
(Inspection of software code, design etc., Reviews, Walkthroughs, desk
checking etc) [4]
ü Dynamic Testing
Test a program by executing test scripts e.g White box testing, black box
testing etc.[5,6]
Dr. Ali Javed

20 Defect Containment [1]
Dr. Ali Javed

Defect Containment [6,10]
21
3. Defect containment through failure prevention and

containment
q Containing the failures to local areas
q Limiting the damage
ü Fault tolerance techniques are used to break the causal relation between
faults and failures so that local faults will not cause global failures, thus
“tolerating” these local faults
ü Failure Containment measures to avoid disastrous consequences, such as

death, personal injury and severe property in case of failures
Dr. Ali Javed

Software Fault Tolerance [1,7]
22
q Fault-tolerance or graceful degradation is the property that enables a system to continue

operating properly in the event of the failure of some of its components. If its operating quality
decreases at all, the decrease is proportional to the severity of the failure.[10]
q Software fault tolerance makes the assumption that the system has unavoidable and
undetectable faults and aims to make the system to operate correctly even in the presence of
faults.
¨ Using a variety of Software methods, faults are detected, and recovery is accomplished
q All fault tolerance activities are based on redundancy
q The process of fault tolerance involves many defined stages, from the phase of errors detection,
through the confinement phase and evaluation of the propagation of erroneous information until
the recovery phase
Dr. Ali Javed

Software Fault Tolerance-Recovery Types
23
q Backward Recovery[7]
ü This recovery type undoes the executed actions and it come

back to the consistent state which already has past, in a way to
continue starting from that point.
ü The backward recovery process demands the establishment of
a point, during the execution of the system, denominated
recovery point, which is responsible for the preservation of
appropriate information for subsequent recovery, in case of
error.
q Forward Recovery[7]
ü The forward recovery acts when an abnormal state is detected

with the suspension of the processing to that more favorable
conditions are found
ü This recovery type ignores part or the whole work, and it
continues with what it is supposed correct.
Dr. Ali Javed

Software Fault Tolerance-Fault Injection for
Fault Tolerance Assessment
24
q Software Fault Injection is the process of

testing software under abnormal
circumstances involving invalid external inputs
q The main objective is to test the fault

tolerance capability through injecting faults
into the system and analyze if the system can
detect and recover from faults as specified
by the system
q The results can lead to either fixing of

individual software bugs or discovery of
design deficiencies in system fault tolerance
Dr. Ali Javed

Software Fault Tolerance Techniques-
Data & Design Diversity
25
q Multiple data representation enviroment:

P Data diverse techniques are used in a multiple data representation environment
P utilize different representations of input data to provide tolerance to software
design faults
q Multiple version software enviroment:

P Design diverse techniques are used in a multiple version software environment
P use the functionality of independently developed software versions to provide
tolerance to software design faults
Dr. Ali Javed

Design Diversity technique
26
q Two or more variants of software developed by different teams but to a common

specification are used.
q These variants are then used in a time or space redundant manner to achieve fault
tolerance.
q Disadvantage of design diversity is the high cost involved in developing multiple

variants of software
Dr. Ali Javed

27
q Popular techniques which are based on the design diversity

concept for fault tolerance in software are:
P Recovery Block
P N-Version Programming
P N-Self-Checking Programming
Dr. Ali Javed

28
q Recovery Blocks
ü Recovery blocks use repeated executions (or

redundancy over time) as the basic mechanism for
fault tolerance.
ü The basic recovery block is related to sequential
systems
ü Several alternatives are proposed for a certain
block of instructions, incase of mistake there will
be activation of alternatives one at a time under
acceptance routine until the correct execution or
error message
Dr. Ali Javed

29
q Recovery Blocks
ü A recovery block consists of a conventional block which is provided with a
means of error detection (an acceptance test), Primary alternate and zero or
more stand-by spares (the additional alternates). A possible syntax for recovery
blocks is as follows:
n ensure acceptance test

n by primary alternate
n else by alternate 2
n else by alternate n
n else error
Dr. Ali Javed

30
q Recovery Blocks
¨ The acceptance test is a programmer-provided, program

specific, error detection mechanism, that provides a check on
the results of program execution
¨ Acceptance tests can include Checks for runtime errors

ü arithmetic, e.g., divide by zero
ü Excessive execution time
ü Mathematical errors
P If primary block fails then alternate block starts one at a

time in case the earlier alternate fails to complete, or fails
the acceptance test.
P However, before an alternate is so entered, the state of the
process is restored to that current just before entry to the
primary alternate.
Dr. Ali Javed

31
q N Version Programming [9]
ü NVP uses parallel redundancy, where N copies each of a different version of

programs fulfilling the same functionality are running in parallel to fulfill the
same requirement and the decision of output correctness is based on
comparisons of all outputs.
ü The decision algorithm (voter) selects the correct output
ü The principle objective of N-Version Programming is to mask the effects of
software flaws in the exits of the module
Dr. Ali Javed

32
q N Version Programming
ü Compared with RcB, NVP operates in parallel mode

ü Voter is the fundamental difference between NVP and RB
ü The decision algorithm in NVP makes sure that local failures in limited number of
these parallel versions will not compromise global execution results
ü Since the entire versions are built to satisfy the same requirement, N version
programming thus requires considerable development effort
Dr. Ali Javed

33
General syntax:
run Version 1, Version 2, ..., Version n
if (Decision Mechanism (Result1, Result2,..., Result n))
return Result
else failure exception
Dr. Ali Javed

34
q N Self Check Programming

P N Self Checking programming is the use of multiple software versions combined with structural variations of
the Recovery Blocks and NVP.
P N Self Checking programming using acceptance tests as shown in Figure.
P Here the versions and the acceptance tests are developed independently from common
requirements. This use of separate acceptance tests for each version is the main difference of this N Self-
Checking model from the Recovery Blocks approach.
P The output is taken from the highest-ranking version that passes its acceptance test.
Dr. Ali Javed

Data Diversity technique
35
q Data diversity, a technique for fault tolerance in software, was introduced by Amman
and Knight.
q While the design diversity approaches to provide fault tolerance rely on multiple
versions of the software written to the same specifications, the data diversity
approach uses only one version of the software.
q This approach relies on the observation that a software sometime fails for certain
values in the input space and this failure could be prevented if there is a minor
change of input data which is acceptable to the software.
Dr. Ali Javed

36
q This technique is economical to implement than the design diversity technique.
q Popular techniques which are based on the data diversity concept for fault tolerance
in software are:
P Retry Blocks
P N-Copy Programming
Dr. Ali Javed

37
q Retry Blocks
P A retry block is a modification of the recovery block scheme that uses data diversity instead of design
diversity.
P Data diversity is a strategy that does not change the algorithm of the system (just retry), but does change the
data that the algorithm processes. It is assumed that there are certain data which will cause the algorithm to
fail, and that if the data were re-expressed in a different or equivalent form the algorithm would function
correctly.
P A retry block executes the single algorithm normally and evaluates the acceptance test.
P If the test passes, the retry block is complete. If the test fails, the algorithm executes again after the data has
been re-expressed.
Dr. Ali Javed

38
General syntax:
ensure Acceptance Test
by Primary Algorithm (Original Input)
else by Primary Algorithm (Re-expressed Input)
else by Primary Algorithm (Re-expressed Input)
...
... [Deadline Expires]
else by Backup Algorithm (Original Input)
q The Retry Block syntax above states that the technique will first attempt to ensure the
AT by using the primary algorithm.
q If the primary algorithm’s result does not pass the AT, then the input data will be re
expressed and the same algorithm attempted until a result passes the AT or the
deadline expires.
q If the deadline expire, the backup algorithm is invoked with the original inputs.
q If this backup algorithm is not successful, an error occurs.
Dr. Ali Javed

39
q N Copy Programming
P N Copy Programming (NCP) is the data
diverse variant of NVP
P The Technique use one or more Data re-
expression algorithm’s (DRA’s) and at least
two copies of a program
P The system inputs are run through the DRA(s)
to re-express the inputs.
P The copies execute in parallel using the re-
expressed data as input.
P A DM examines the results of the copy
executions and selects the “best” result, if
one exists.
Dr. Ali Javed

40
The basic NCP technique consists of an executive, 1 to n DRA, n copies of the program or function,
and a DM. The executive composes the NCP technique operation, which has the general syntax:
run DRA 1, DRA 2, ..., DRA n

run Copy 1(result of DRA 1),
Copy 2(result of DRA 2), ...,
Copy n(result of DRA n)
if (Decision Mechanism (Result 1, Result 2, ...,Result n))
return Result
The NCP syntax above states that the technique first runs the DRA concurrently to
re-express the input data, then executes the n copies concurrently.
The results of the copy executions are provided to the DM, which operates upon the results
to determine if a correct result can be decided.
If one can (i.e., the Decision Mechanism statement above evaluates to TRUE), then it is returned.
If a correct result cannot be determined, then an error occurs.
Dr. Ali Javed

Failure Containment
41
q With all the fault prevention and fault tolerant techniques, unfortunately, we will still
have faults. In that case, can we (a) “prevent accidents” and can we (b) “reduce the
damage of the accidents”?
q We already know that we can not prevent all accidents. But we can analyze the
hazards of an accident and hopefully contain or limit the damage.
Dr. Ali Javed

Fault Tree Analysis
42
q Fault tree analysis (FTA) is a top down, deductive failure analysis in which an
undesired state of a system is analyzed using boolean logic to combine a series of
lower-level events.
q This analysis method is mainly used in the field of safety engineering and Reliability
engineering to determine the probability of a safety accident or a particular system
level (functional) failure.
Dr. Ali Javed

Fault Tree Analysis
43
q list the set of events that cause the “accident” or “failure”

q build an upside-down tree that logically connects the events to the failure
Security
Break in F1
AND
Log-in
Granted Access to
F1
“bug”
Or
Log-in - top event is the “accident”

Password
validation - circles are primary events
Exposed
“bug” - AND/OR are logical conditions
Dr. Ali Javed
Containment
44
q We use the fault-tree to analyze and understand the cause of the

“accident.” We may use it for:
P Accident elimination
P Accident reduction
P Accident control
Dr. Ali Javed

References
45
1. http://www.sqa.net/index.htm
2. Software Engineering by Roger Pressman
3. Software Quality Engineering: Testing, Quality Assurance, and Quantifiable Improvement:
JEFF TIAN, 2005.
4. The Art of Software Testing by Glenford Myers
5. http://en.wikipedia.org/wiki/Black-box_testing
6. http://en.wikipedia.org/wiki/White-box_testing
7. http://www.cs.rutgers.edu/~rmartin/teaching/spring03/cs553/papers01/06.pdf
8. http://www.hillside.net/plop/2009/papers/Process/N-Version%20Programming.pdf
9. http://en.wikipedia.org/wiki/Fault-tolerant_system
Dr. Ali Javed

For any query Feel Free to ask
46
Dr. Ali Javed

Lecture # 3: Software System Quality

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lecture # 3: Software System Quality

Uploaded by

Copyright:

Available Formats

SOFTWARE SYSTEM QUALITY

QUALITY ASSURANCE AS DEALING WITH

20th October, 2021 Dr. Ali Javed

q Instructor: Dr. Ali Javed

Dr. Ali Javed

q Course Name: Software System Quality

q Course Code: SE-5001

Dr. Ali Javed

Dr. Ali Javed

Dr. Ali Javed

q Quality assurance can be defined as, ‘the establishment of framework of

q Classification Scheme for QA as Dealing with Defects

Dr. Ali Javed

Dr. Ali Javed

Engr. Ali Javed

Dr. Ali Javed

q Eliminating certain error sources, such as correcting human misconceptions

Dr. Ali Javed

q Education and training

q Process conformance and standards

q Tools/technologies and techniques

q Education and training of software professionals can help them control,

Dr. Ali Javed

The education and training effort for error source

q Product and domain specific knowledge

ü If the people involved are not familiar with

q Knowledge about the specific

ü Also plays an important role in developing

Dr. Ali Javed

q General Software Development Knowledge and expertise

ü It plays an important role in developing high quality software products

ü For example lack of expertise with requirements analysis and product

Dr. Ali Javed

q Development Process Knowledge used by the organization

ü If the project personnel do not have a good understanding of the

ü For example, if the people involved in incremental software development

Dr. Ali Javed

Dr. Ali Javed

q Formal methods includes formal specification and formal verification

ü Formal specification is concerned with producing an unambiguous set of product

Dr. Ali Javed

Dr. Ali Javed

2. Defect reduction through fault detection and removal

q Detect and remove faults once they have been injected

Dr. Ali Javed

Dr. Ali Javed

3. Defect containment through failure prevention and

q Containing the failures to local areas

q Limiting the damage

ü Failure Containment measures to avoid disastrous consequences, such as

Dr. Ali Javed

q Fault-tolerance or graceful degradation is the property that enables a system to continue

q All fault tolerance activities are based on redundancy

Dr. Ali Javed

ü This recovery type undoes the executed actions and it come

ü The forward recovery acts when an abnormal state is detected

Dr. Ali Javed

q Software Fault Injection is the process of

q The main objective is to test the fault

q The results can lead to either fixing of

Dr. Ali Javed

q Multiple data representation enviroment: