UNIT-2 - Software Testing - 6 Slides Handouts

Unit - II
Testing
The aim of testing is to identify all
Unit II defects in a software product.
Software Testing However, in practice even after
thorough testing:
one cannot guarantee that the software is
error--free.
error
Unit -II: Software Testing 2 Dr. Rabins Porwal
Testing Testing
The input data domain of most Testing does however expose many
software products is very large: errors:
it is not practical to test the testing provides a practical way of
reducing defects in a system
software exhaustively with each
input data value. increases the users' confidence in a
developed system.
Unit -II: Software Testing 3 Dr. Rabins Porwal Unit -II: Software Testing 4 Dr. Rabins Porwal
Testing Testing
Testing is an important development Many engineers have the wrong
phase: impression:
requires the maximum effort among all
development phases.
testing is a secondary activity
In a typical development organization: it is intellectually not as stimulating
maximum number of software engineers can be as the other development activities,
found to be engaged in testing activities. etc.
Compiled by: Dr. Rabins Porwal 1

Unit - II
How do you test a

Testing program?
Testing a software product is in fact:
as much challenging as initial Input test data to the program.
development activities such as Observe the output:
specification, design, and coding.
Check if the program behaved as
Also, testing involves a lot of creative
thinking. expected.
How do you test a How do you test a

system? system?
If the program does not behave
as expected:
note the conditions under which it
failed.
later debug and correct.
Error, Faults, and Failures Error, Faults, and Failures

A failure is a manifestation of an error
A fault is an incorrect state entered
(or defect) or fault (or bug), i.e.,
inability of a system to perform its during program execution:
required function. a variable value is different from what
mere presence of an error may not lead to a it should be.
failure. Error is a measure of the difference A fault may or may not lead to a
between the actual and the ideal. failure.
Fault is a condition that causes a system to fail
in performing its required function.

Unit - II
Test cases and Test suites Test cases and Test suites
Test a software using a set of A test case is a triplet [I, S, O]
carefully designed test cases: I is the data to be input to the system,
S is the state of the system at which the
the set of all test cases is called the data will be input,
test suite O is the expected output of the system.
Verification versus Verification versus

Validation Validation
Verification is the process of determining: Verification is concerned with
whether output of one phase of development phase containment of errors,
conforms to its previous phase. whereas the aim of validation is that
Validation is the process of determining the final product be error free.
whether a fully developed system conforms to
its SRS document.
Design of Test Cases Design of Test Cases

Exhaustive testing of any non-trivial If test cases are selected randomly:
system is impractical: many test cases would not contribute to the
significance of the test suite,
input data domain is extremely large. would not detect errors not already being detected
Design an optimal test suite: by other test cases in the suite.
of reasonable size and Number of test cases in a randomly selected

test suite:
uncovers as many errors as possible. not an indication of effectiveness of testing.

Unit - II

Testing a system using a large number of
The code has a simple programming error:
randomly selected test cases:
If (x>y) max = x;
does not mean that many errors in the system
else max = x;
will be uncovered.
test suite {(x=3, y=2);(x=2, y=3)} can detect the
Consider an example for finding the error,
maximum of two integers x and y.
a larger test suite {(x=3, y=2);(x=4, y=3); (x=5,
y=1)} does not detect the error.

Systematic approaches are required There are essentially two main
to design an optimal test suite: approaches to design test cases:
each test case in the suite should Black-box approach
detect different errors. White-box (or glass-box) approach
Testing Testing
Software products are tested at Unit testing:
three levels: test the functionalities of a single module or
function.
Unit testing Integration testing:
Integration testing test the interfaces among the modules.
System testing:
System testing
test the fully integrated system against its functional
and non-functional requirements.

Unit - II
Unit testing Unit testing

During unit testing, modules are Unit testing reduces debugging
tested in isolation: effort several folds.
If all modules were to be tested Programmers carry out unit testing
together: immediately after they complete
it may not be easy to determine the coding of a module.
which module has the error.
Integration testing System Testing

After different modules of a system System testing involves:
have been coded and unit tested:
modules are integrated in steps validating a fully developed
according to an integration plan system against its requirements.
partially integrated system is tested at
each integration step.
Test Drivers & Test Stubs Test Drivers & Test Stubs
In order to test a single module, a complete
Modules required to provide
environment is needed to provide all that is necessary the necessary environment Driver module
for execution of the module. That is, besides the (which either call or are called
module under test, we will need the following in order by the module under test) are
to be able to test the module: usually not available until they, Module under
The procedure belonging to other modules that the too, have been unit tested; stubs test
module under test calls and drivers are designed to

provide the complete
Non local data structures that the module accesses environment for a module.
module. The Stub module
A procedure to call the functions of the module under role of stub and driver modules
test with appropriate parameters are shown in the figure.

Unit - II
Test Drivers & Test Stubs

Black-box Testing
A stub procedure is a dummy procedure that has
the same I/O parameters as the given procedure
Test cases are designed using only
but has a highly simplified behavior. functional specification of the software:
A driver module contains the non-
non-local data without any knowledge of the internal
structures accessed by the module under test, structure of the software.
and also has the code to call the different For this reason, black-box testing is also
functions of the module with appropriate known as functional testing.
parameter values.
White-box Testing Black-box Testing

Designing white-box test cases:
There are essentially two main
requires knowledge about the internal
structure of software.
approaches to design black box test
white-box testing is also called
cases:
structural testing. Equivalence class partitioning
Boundary value analysis
Equivalence Class Partitioning Why define equivalence

classes?
Input values to a program are Test the code with just one
partitioned into equivalence classes. representative value from each
Partitioning is done such that: equivalence class:
program behaves in similar ways at every as good as testing using any other
input value belonging to an equivalence
values from the equivalence
class.
classes.

Unit - II
Equivalence Class Partitioning Equivalence Class Partitioning
How do you determine the If the input data to the program is

specified by a range of values:
equivalence classes?
e.g. numbers between 1 to 5000.
examine both the input & output data.
one valid and two invalid equivalence
few general guidelines for determining classes are defined.
the equivalence classes can be given 5000
1
Equivalence Class Partitioning Example

If input is an enumerated set of values: A program reads an input value in
e.g. {a, b, c} the range of 1 and 5000:
one equivalence class for valid input values computes the square root of the input
another equivalence classes for invalid input number
values should be defined. SQR
T
Example (cont.) Example (cont.)

There are three equivalence classes: The test suite must include:
the set of negative integers, representatives from each of the three
set of integers in the range of 1 and 5000, equivalence classes:
integers larger than 5000. a possible test suite can be:
{-5, 500, 6000}.
1 5000 1 5000

Unit - II
Boundary Value Analysis Boundary Value Analysis

Some typical programming errors occur:
at boundaries of equivalence classes
Programmers may improperly use <
might be purely due to psychological instead of <=
factors. Boundary value analysis:
Programmers often fail to see:
select test cases at the boundaries of
special processing required at the
different equivalence classes.
boundaries of equivalence classes.
Example Robustness testing

For a function that computes the It is nothing but the extension of boundary value analysis. Here, we would
like to see, what happens when the extreme values are exceeded with a
square root of an integer in the range value slightly greater than the maximum, and a value slightly less than
minimum. It means, we want to go outside the legitimate boundary of input
of 1 and 5000: domain. This extended form of boundary value analysis is called
robustness testing and shown in Fig. 8.6
test cases must include the values: There are four additional test cases which are outside the legitimate input
{0, 1, 5000, 5001} domain. Hence total test cases in robustness testing are 6n+1, where n is
the number of input variables. So, 13 test cases are:
1 5000
(200,99), (200,100), (200,101), (200,200), (200,299), (200,300)
(200,301), (99,200), (100,200), (101,200), (299,200), (300,200), (301,200)
Robustness testing Worst-case testing

If we reject single fault assumption theory of reliability and
400 may like to see what happens when more than one variables
300 have extreme values. In electronic circuits analysis, this is
y 200 called worst case analysis.
100 It is more thorough in the sense that boundary value test

cases are a proper subset of worst case test cases, which
requires more effort. Worst case testing for a function of n
0 100 200 300 400 variables generate 5n test cases as opposed to 4n+1 test
x
cases for boundary value analysis. Our two variables example
Fig. 8.6: Robustness test cases for two variables x will have 52=25 test cases.
and y with range [100, 300] each

Unit - II
Decision Table Based Testing Decision Table Based Testing

Test case design
Useful for describing situations in which a number of combinations of actions are
taken under varying sets of conditions. There are four portions of a decision table: Test cases are identified with decision tables by interpreting conditions as
Conditions stub, Action Stub, Condition entries and Action entries. inputs, and actions as outputs. The rules are then interpreted as test cases.
Condition Entry C1:x,y,z are sides of a N Y
Stub triangle?
True False C2:x = y? -- Y N
C1
True False True False C3:x = z? -- Y N Y N
C2
C4:y = z? -- Y N Y N Y N Y N
True False True False True False ---
C3
a1: Not a triangle X
Action a1 X X X
Stub a2: Scalene X
a2 X X X
a3: Isosceles X X X
a3 X X
a4: Equilateral X
a4
X X X X
a5: Impossible X X
Table 2: Decision table terminology Table 3: Decision table for triangle problem
Decision Table Based Testing Decision Table Based Testing

Conditions F T T T T T T T T T T Example 8.10
C1 : x < y + z ?
C2 : y < x + z ? -- F T T T T T T T T T
Consider the triangle program specified in example 8.3. Identify the
C3 : z < x + y ? -- -- F T T T T T T T T test cases using the decision table of Table 4.
C4 : x = y ? -- -- -- T T T T F F F F
C5 : x = z ? -- -- -- T T F F T T F F
C6 : y = z ? -- -- -- T F T F T F T F
a1 : Not a triangle X X X
a2 : Scalene X
a3 : Isosceles X X X
a4 : Equilateral X
a5 : Impossible X X X
Table 4: Modified decision table

Decision Table Based Testing Cause Effect Graphing Technique

Solution It is a technique that aids in selecting a high-yield set of test
There are eleven functional test cases, three to fail triangle property, three cases. It has a beneficial effect in pointing out incompleteness
impossible cases, one each to get equilateral, scalene triangle cases, and and ambiguities in specifications.
three to get on isosceles triangle. The test cases are given in Table 5. Steps
Test x y z Expected Output 1. Causes & effects in the specifications are identified.
case
1 4 1 2 Not a triangle A cause is a distinct input condition or an equivalence class of input conditions.
2 1 4 2 Not a triangle
An effect is an output condition or a system transformation. Causes and
3 1 2 4 Not a triangle
effects are identified by reading the specification word by word. Each cause
4 5 5 5 Equilateral and effect is assigned a unique number.
5 ? ? ? Impossible
6 ? ? ? Impossible
2. The semantic content of the specification is analysed and transformed into a
boolean graph linking the causes & effects.
7 2 2 3 Isosceles
8 ? ? ? Impossible 3. Constraints are imposed
9 2 3 2 Isosceles
4. graph limited entry decision table
10 3 2 2 Isosceles
11 3 4 5 Scalene Each column in the table represent a test case.
Test cases of triangle problem using decision table

5. The columns in the decision table are converted into test cases.

Unit - II
Cause Effect Graphing Technique Cause Effect Graphing Technique

Myers explained this effectively with following example. The characters in column 1
The basic notation for the graph is shown in fig. 8 must be an A or B. The character in column 2 must be a digit. In this situation, the
file update is made. If the character in column 1 is incorrect, message x is issued. If
the character in column 2 is not a digit, message y is issued.
The causes are

c1: character in column 1 is A
c2: character in column 1 is B
c3: character in column 2 is a digit
and the effects are

e1: update made
e2: message x is issued
e3: message y is issued
Fig.8. 8 : Basic cause effect graph symbols
The E constraint states that it must always be true that is at most

one of c1 or c2 can be 1 (c1 or c2 cannot be 1 simultaneously). The
I constraint states that at least one of c1, c2 and c3 must always be
1 (c1, c2 and c3 cannot be 0 simultaneously). The O constraint
states that one and only one of c1 and c2 must be 1. The
constraint R states that, for c1 to be 1, c2 must be 1 (i.e. it is
impossible for c1 to be 1 and c2 to be 0).
There is a need for a constraint among effects. The M constraint
states that if effect e1 is 1, effect e2 is forced to be 0.
Fig. 9: Sample cause effect graph
Fig. 10: Constraint symbols Fig. 11: Symbol for masks constraint

Unit - II
Example 8.12
Consider the triangle problem specified in the example 8.3. Draw the Cause
effect graph and identify the test cases.
Fig. 12 : Sample cause effect graph with exclusive constraint

Solution
The causes are The cause effect graph is shown in fig. 13 and decision table is shown in table 6.
The test cases for this problem are available in Table 5.
c1: side x is less than sum of sides y and z
Conditions
c2: side y is less than sum of sides x and z C1: x < y + z ? 0 1 1 1 1 1 1 1 1 1 1
c3: side z is less than sum of sides x and y C2: y < x + z ? X 0 1 1 1 1 1 1 1 1 1
c4: side x is equal to side y C3: z < x + y ? X X 0 1 1 1 1 1 1 1 1
c5: side x is equal to side z C4: x = y ? X X X 1 1 1 1 0 0 0 0
c6: side y is equal to side z C5: x = z ? X X X 1 1 0 0 1 1 0 0
C6: y = z ? X X X 1 0 1 0 1 0 1 0
and effects are e1: Not a triangle 1 1 1
e1: Not a triangle e2: Scalene 1
e2: Scalene triangle e3: Isosceles 1 1 1
e3: Isosceles triangle
e4: Equilateral 1
e4: Equilateral triangle e5: Impossible 1 1 1
e5: Impossible stage
Table 6: Decision table
Cause Effect Graphing Technique

The image cannot be display ed. Your computer may not hav e enough memory to open the image, or the image may hav e been corrupted. Restart y our computer, and then open the file again. If the red x still appears, y ou may hav e to delete the image and then insert it again.
White--Box Testing
White
There exist several popular white-box testing
methodologies:
Statement coverage
branch coverage
condition coverage
path coverage
data flow-based testing
mutation testing
Fig. 13: Cause effect graph of triangle problem

Unit - II
Statement Coverage Statement Coverage

Statement coverage methodology: The principal idea:
design test cases so that unless a statement is executed,
every statement in a program is executed at we have no way of knowing if an error exists in that
least once. statement.
Statement coverage criterion Statement coverage criterion

Based on the observation: Observing that a statement behaves properly for
an error in a program can not be discovered: one input value:
unless the part of the program containing the no guarantee that it will behave correctly for all input
error is executed. values.
Euclid's GCD computation

Example algorithm
int f1(int x, int y){ By choosing the test set {(x=3,y=3),(x=4,y=3),
1 while (x != y){ (x=3,y=4)}
2 if (x>y) then all statements are executed at least once.
3 x=x-y;
x=x-
4 else y=y-
y=y-x;
5}
6 return x; }

Unit - II
Branch Coverage Branch Coverage

Test cases are designed such that: Branch testing guarantees statement
different branch conditions coverage:
give true and false values in turn and each branch a stronger testing compared to the statement
is traversed at least once. coverage-based testing.
Stronger testing Example

Test cases form a superset of a weaker int f1(int x,int y){
testing: 1 while (x != y){
discovers at least as many errors as a weaker 2 if (x>y) then
testing 3 x=x-y;
contains at least as many significant test cases as a 4 else y=y-x;
weaker test. 5}
6 return x; }
Example Condition Coverage

Test cases for branch coverage can be: Test cases are designed such that:
{(x=3,y=3),(x=3,y=2), (x=4,y=3), (x=3,y=4)} each component of a composite conditional
expression is
given both true and false values.

Unit - II
Example Branch testing

Branch testing is the simplest condition
Consider the conditional expression
((c1.and.c2).or.c3): testing strategy:
Each of c1, c2, and c3 are exercised at least compound conditions appearing in different
once, branch statements
i.e. given true and false values. are given true and false values.
Branch testing Condition coverage

Condition testing Consider a boolean expression having n
stronger testing than branch testing:
components:
Branch testing
stronger than statement coverage testing. for condition coverage we require 2n test
cases.
Condition coverage Path Coverage

Condition coverage-based testing Design test cases such that:
technique: all linearly independent paths in the program
practical only if n (the number of component are executed at least once.
conditions) is small.

Unit - II
Path coverage-
coverage-based
Linearly independent paths testing
Defined in terms of To understand the path coverage-based testing:
control flow graph (CFG) of a program. we need to learn how to draw control flow graph of
a program.
How to draw Control flow

Control flow graph (CFG)
graph?
A control flow graph (CFG) describes:
the sequence in which different instructions of a Number all the statements of a program.
program get executed. Numbered statements:
the way control flows through the program.
represent nodes of the control flow graph.
How to draw Control flow Example

graph?
int f1(int x,int y){
1 while (x != y){
An edge from one node to another node 2 if (x>y) then
exists: 3 x=x-y;
if execution of the statement representing the 4 else y=y-x;
first node 5}
6 return x; }
can result in transfer of control to the other
node.

Unit - II
How to draw Control flow

Example Control Flow Graph
graph?
1
Sequence:
2 1 a=5; 1
3 4 2 b=a*b-1;
2
5
6
How to draw Control flow How to draw Control flow

graph? graph?
Iteration: 1
Selection:
1 while(a>b){
1 if(a>b) then 1
2 b=b*a; 2
2 c=3; 2 3 3 b=b-1;}
3 else c=5;
4 4 c=b+d; 3
4 c=c*c;
4
Path Independent path

A path through a program: Any path through the program:
a node and edge sequence from the starting node to introducing at least one new node:
a terminal node of the control flow graph. that is not included in any other independent
There may be several terminal nodes for program. paths.

Unit - II
Independent path McCabe's cyclomatic

metric
It is straight forward: An upper bound:
to identify linearly independent paths of simple for the number of linearly independent paths of a
programs. program
Provides a practical way of determining:
For complicated programs:
the maximum number of linearly independent paths
it is not so easy to determine the number of in a program.
independent paths.
McCabe's cyclomatic Example Control Flow

metric Graph
Given a control flow graph G,
1
cyclomatic complexity V(G):
V(G)= E-N+2 2
N is the number of nodes in G 3 4
E is the number of edges in G
5
6
Example Cyclomatic complexity

Cyclomatic complexity = Another way of computing cyclomatic complexity:
inspect control flow graph
7-6+2 = 3. determine number of bounded areas in the graph
V(G) = Total number of bounded areas + 1

Unit - II
Example Control Flow

Bounded area
Graph
Any region enclosed by a nodes and edge
sequence. 1
2
3 4
5
6
Example Cyclomatic complexity

From a visual examination of the CFG: McCabe's metric provides:
the number of bounded areas is 2. a quantitative measure of testing difficulty and the
cyclomatic complexity = 2+1=3. ultimate reliability
Intuitively,
number of bounded areas increases with the number
of decision nodes and loops.
Cyclomatic complexity Cyclomatic complexity

The cyclomatic complexity of a program provides:
The first method of computing V(G) is amenable to a lower bound on the number of test cases to be
automation: designed
you can write a program which determines the to guarantee coverage of all linearly independent
number of nodes and edges of a graph paths.
applies the formula to find V(G).

Unit - II
Cyclomatic complexity Cyclomatic complexity

Defines the number of independent paths in a Knowing the number of test cases required:
program. does not make it any easier to derive the test cases,
Provides a lower bound: only gives an indication of the minimum number of
test cases required.
for the number of test cases for path coverage.
Path testing Path testing

The tester proposes: A dynamic program analyzer is used:
an initial set of test data using his experience and to indicate which parts of the program have been
judgement. tested
the output of the dynamic analysis
used to guide the tester in selecting additional test
cases.
Derivation of Test Cases Example

int f1(int x,int y){
Let us discuss the steps to derive path 1 while (x != y){
coverage-based test cases of a program. 2 if (x>y) then
Draw control flow graph. 3 x=x-y;
Determine V(G). 4 else y=y-x;
5}
Determine the set of linearly independent paths.
6 return x; }
Prepare test cases:
to force execution along each path.

Unit - II
Example Control Flow

Derivation of Test Cases
Diagram
1 Number of independent paths: 3
1, 6 test case (x=1, y=1)
2
1, 2, 3, 5, 1, 6 test case (x=1, y=2)
3 4 1, 2, 4, 5, 1, 6 test case (x=2, y=1)
5
6
An interesting application of
Cyclomatic complexity
cyclomatic complexity
Cyclomatic complexity of a program:
Relationship exists between: also indicates the psychological complexity of a
McCabe's metric program.
the number of errors existing in the code, difficulty level of understanding the program.
the time required to find and correct the errors.
Cyclomatic complexity Data Flow-

Flow-Based Testing
From maintenance perspective,

limit cyclomatic complexity
Selects test paths of a program:
of modules to some reasonable value. according to the locations of
Good software development organizations:
restrict cyclomatic complexity of functions to a
definitions and uses of different
maximum of ten or so. variables in a program.

Unit - II
Data Flow-
Flow-Based Testing Data Flow-
Flow-Based Testing
For a statement numbered S, A variable x is said to be live at
DEF(S) = {x| statement S contains a definition of x}
USES(S)= {x| statement S contains a use of x} statement S1, if
Example: 1: a=b; DEF(1)={a}, USES(1)={b} x is defined at a statement S:
Example: 2: a=a+b; DEF(2) = {a}, there exists a path from S to S1 not
USES(2) = {a, b}
containing any definition of x.
DU Chain Example Definition--use chain (DU chain)

Definition
1 X(){ [x, S, S1],
2 a = 5; /* Defines variable a */
3 While(C1) { S and S1 are statement numbers,
4 if (C2) x in DEF(S)
5 b = a * a;/* Uses variable a */ x in USES(S1), and
6 a = a - 1; /* Defines variable a */ the definition of x in the statement S is live at
7 } statement S1.
8 print(a); } /* Uses variable a */
Data Flow-
Flow-Based Testing Data Flow-
Flow-Based Testing
One simple data flow testing strategy: Data flow testing strategies:
every DU chain in a program be covered useful for selecting test paths of a
at least once. program containing nested if and loop
statements

Unit - II
Data Flow-Based Testing Data Flow-

Flow-Based Testing
1 X(){ [a, 2, 5]: a DU chain.
2 B1; /* Defines variable a */ Assume:
3 While(C1) { DEF(a) = {B1, B2, B3, B4, B5}
4 if (C2)
USE(a) = {B2, B3, B4, B5, B6}
5 if(C4) B4; /*Uses variable a */
There are 25 DU chains.
6 else B5;
However only 5 paths are needed to cover these
7 else if (C3) B2;
chains.
8 else B3; }
9 B6 }
Mutation Testing Mutation Testing

The software is first tested: Each time the program is changed,
using an initial testing method based on white-box
strategies we already discussed. it is called a mutated program
After the initial testing is complete, the change is called a mutant.
mutation testing is taken up.
The idea behind mutation testing:
make a few arbitrary small changes to a program at a
time.

A mutated program: If a mutant remains alive:
tested against the full test suite of the program. even after all test cases have been exhausted,
If there exists at least one test case in the test the test suite is enhanced to kill the mutant.
suite for which: The process of generation and killing of mutants:
a mutant gives an incorrect result, can be automated by predefining a set of primitive
then the mutant is said to be dead. changes that can be applied to the program.

Unit - II

The primitive changes can be: A major disadvantage of mutation
altering an arithmetic operator, testing:
changing the value of a constant, computationally very expensive,
changing a data type, etc. a large number of possible mutants can
be generated.
Integration testing Integration Testing
After different modules of a system have Develop the integration plan by

been coded and unit tested: examining the structure chart :
modules are integrated in steps according to big bang approach
an integration plan
top-down approach
partially integrated system is tested at each
integration step. bottom-up approach
mixed approach
Example Structured Design Big bang Integration Testing

root Big bang approach is the simplest
Valid-
Valid-numbers rms
Valid-
rms
Valid-numbers
integration testing approach:
Get-
Get-good-
good-data Compute-
Compute- Display-
Display-solution
all the modules are simply put together and
solution tested.
this technique is used only for very small
Get-
Get-data Validate-
Validate- systems.
data

Unit - II
Big bang Integration Testing Bottom--up Integration Testing

Bottom
Main problems with this approach: Integrate and test the bottom level
if an error is found: modules first.
it is very difficult to localize the error A disadvantage of bottom-up testing:
the error may potentially belong to any of the
when the system is made up of a large
modules being integrated.
number of small subsystems.
debugging errors found during big bang This extreme case corresponds to the big bang
integration testing are very expensive to fix. approach.
Top--down integration testing

Top Mixed integration testing
Top-down integration testing starts with the main Mixed (or sandwiched) integration
routine:
testing:
and one or two subordinate routines in the system.
uses both top-down and bottom-up
After the top-level skeleton has been tested:
testing approaches.
immediate subordinate modules of the skeleton are
combined with it and tested. Most common approach
Phased versus Incremental

Integration Testing
Integration Testing
In top-down approach: Integration can be incremental or
testing waits till all top-level modules are coded phased.
and unit tested.
In incremental integration testing,
In bottom-up approach:
only one new module is added to the
testing can start only after bottom level
modules are ready. partial system each time.

Unit - II
Phased versus Incremental Integration Phased versus Incremental

Testing Integration Testing
In phased integration, Phased integration requires less number of
a group of related modules are added to integration steps:
the partially integrated system each time. compared to the incremental integration
Big-bang testing: approach.
a degenerate case of the phased However, when failures are detected,
it is easier to debug if using incremental testing
integration testing. since errors are very likely to be in the newly
integrated module.
System Testing System Testing

System testing: System tests are designed to validate
validate a fully developed system a fully developed system:
against its requirements. to assure that it meets its
requirements.
System Testing Alpha testing

There are essentially three main kinds System testing is carried out
of system testing: by the test team within the developing
Alpha Testing organization.
Beta Testing
Acceptance Testing

Unit - II
Beta Testing Acceptance Testing
Beta testing is the system testing: Acceptance testing is the system testing
performed by a select group of friendly performed by the customer
customers. to determine whether he should accept
the delivery of the system.
System Testing System & Performance Testing

During system testing, in addition to The functionality tests test the functionality of
functional tests: the software to check whether it satisfies the
performance tests are also required to be functional requirements as documented in the
performed. SRS document and it has already been discussed
Note: Various kinds of test cases are how to design the functionality test cases by
using black-box approach.
designed by referring to the SRS document.
The performance tests test the conformance of
Broadly, these tests can be classified into the system with the nonfunctional requirements
functionality and performance tests. of the system.
Performance Testing Stress testing

Addresses non-functional requirements. Evaluates system performance
May sometimes involve testing hardware and when stressed for short periods of time.
software together.
There are several categories of performance
Stress testing
testing. also known as endurance testing.
All performance tests can be considered as
black-box tests.

Unit - II
Stress testing Stress Testing

Stress tests are black box tests: If the requirements is to handle a
designed to impose a range of abnormal specified number of users, or devices:
and even illegal input conditions stress testing evaluates system
so as to stress the capabilities of the performance when all users or devices are
software. busy simultaneously.
Stress Testing Stress Testing

If an operating system is supposed to support 15 Stress testing usually involves an element of time
multiprogrammed jobs, or size,
the system is stressed by attempting to run 15 or such as the number of records transferred per unit
more jobs simultaneously. time,
A real-time system might be tested the maximum number of users active at any time,
input data size, etc.
to determine the effect of simultaneous arrival of
several high-priority interrupts. Therefore stress testing may not be applicable to
many types of systems.
Volume Testing Configuration Testing

Addresses handling large amounts of data in Analyze system behavior:
the system: in various hardware and software
whether data structures (e.g. queues, stacks, configurations specified in the requirements
arrays, etc.) are large enough to handle all sometimes systems are built in various
possible situations configurations for different users
Fields, records, and files are stressed to check for instance, a minimal system may serve a
if their size can accommodate all possible data
volumes. single user,
other configurations for additional users.

Unit - II
Compatibility testing
Compatibility Testing
Example
These tests are needed when the system If a system is to communicate with a
interfaces with other systems: large database system to retrieve
check whether the interface functions as information:
required. a compatibility test examines speed and
accuracy of retrieval.
Recovery Testing Maintenance Testing

These tests check response to: Addresses the diagnostic tools and
presence of faults or to the loss of data, procedures:
help find source of problems regarding
power, devices, or services maintenance of the system.
The system is subjected to the loss of It may be required to supply
resources memory maps
diagnostic programs
check if the system recovers properly. traces of transactions,
circuit diagrams, etc.
Maintenance Testing Documentation tests

Verify that: Check that required documents exist
all required artifacts for maintenance exist and are consistent:
they function properly user guides,
maintenance guides,
technical documents

Unit - II
Documentation tests Usability tests

Sometimes requirements specify: All aspects of user interfaces are tested:
format and audience of specific Display screens
documents messages
documents are evaluated for compliance report formats
navigation and selection problems
Environmental test Test Summary Report

These tests check the systems ability to perform at the Generated towards the end of testing phase.
installation site.
Requirements might include tolerance for
Covers each subsystem:
heat a summary of tests which have been applied to
humidity the subsystem.
chemical presence
portability
electrical or magnetic fields
disruption of power, etc.
Test Summary Report Regression Testing
Specifies: Does not belong to either unit test,

how many tests have been applied to a subsystem, integration test, or system test.
how many tests have been successful,
how many have been unsuccessful, and the degree to
Instead, it is a separate dimension to
which they have been unsuccessful, these three forms of testing.
e.g. whether a test was an outright failure
or whether some expected results of the test were actually
observed.

Unit - II
Regression testing Regression testing

Regression testing is the running of test Regression tests assure:
suite: the new systems performance is at least
after each change to the system or after each as good as the old system
bug fix
ensures that no new bug has been introduced
always used during phased system
due to the change or the bug fix. development.
Coding Coding Standards

At the end of the design phase we have:
module structure (e.g. structure chart) of the system
Good software development
module specifications: organizations require their
data structures and algorithms for each module. programmers to:
Objective of coding phase:
transform design into code
adhere to some standard style of
unit test the code. coding
called coding standards.
Coding Standards Coding Standards

Advantage of adhering to a standard
Many software development style of coding:
organizations: it gives a uniform appearance to the
formulate their own coding standards codes written by different engineers,
that suits them most, it enhances code understanding,
require their engineers to follow these encourages good programming
standards rigorously. practices.

Unit - II
Coding Standards Coding guidelines

A coding standard Provide general suggestions
sets out standard ways of doing regarding coding style to be
several things: followed:
the way variables are named,
code is laid out,
leave actual implementation of the
maximum number of source lines guidelines:
allowed per function, etc. to the discretion of the individual
engineers.
Coding Standards and Representative Coding

Guidelines Standards
Good organizations usually develop their Rules for limiting the use of globals:
own coding standards and guidelines: what types of data can be declared global
depending on what best suits their and what can not.
organization. Naming conventions for
We will discuss some representative global variables,
coding standards and guidelines. local variables, and
constant identifiers.
Representative Coding Representative Coding Standards

Standards
Contents of headers for different Header data:
Name of the module,
modules: date on which the module was created,
The headers of different modules author's name,
should be standard for an organization. modification history,
The exact format for header synopsis of the module,
information is usually specified. different functions supported, along with their
input/output parameters,
global variables accessed/modified by the module.

Unit - II
Representative Coding
Representative Coding Guidelines
Standards
Error return conventions and exception Do not use too clever and difficult to
handling mechanisms. understand coding style.
the way error and exception conditions are Code should be easy to understand.
handled should be standard within an
organization. Many inexperienced engineers actually take
For example, when different functions pride:
encounter error conditions in writing cryptic and incomprehensible code.
should either return a 0 or 1 consistently.
Representative Coding Guidelines Representative Coding Guidelines

The side effects of a function call include:
Clever coding can obscure meaning of modification of parameters passed by reference,
the code: modification of global variables,
hampers understanding. I/O operations.
An obscure side effect:
makes later maintenance difficult. one that is not obvious from a casual examination of
Avoid obscure side effects. the code.
Obscure side effects make it difficult to Do not use an identifier (variable name)
understand a piece of code. for multiple purposes.
For example, Programmers often use the same identifier
for multiple purposes.
if a global variable is changed obscurely in a
called module, For example, some programmers use a
temporary loop variable
it becomes difficult for anybody trying to
also for storing the final result.
understand the code.

Unit - II
Example use of a variable for Use of a variable for multiple

multiple purposes purposes
The rationale given by programmers for such
for(i=1;i<100;i++) use:
{..} memory efficiency:
e.g. three variables use up three memory locations,
i=2*p*q;
whereas the same variable used in three different
return(i); ways uses just one memory location.
Use of a variable for multiple Use of a variable for multiple

purposes purposes
There are several things wrong with this Leads to confusion and annoyance
approach:
for anybody trying to understand the
hence should be avoided.
code.
Each variable should be given a name
indicating its purpose: Also makes future maintenance
This is not possible if an identifier is used for difficult.
multiple purposes.
Code should be well-documented. Lengthy functions:

Rules of thumb: usually very difficult to understand
on the average there must be at least one probably do too many different
comment line
for every three source lines.
things.
The length of any function should not exceed
10 source lines.

Unit - II
Representative Coding Guidelines

Code inspection and code
walk throughs
Do not use goto statements. After a module has been coded,
Use of goto statements: code inspection and code walk
through are carried out
make a program unstructured ensures that coding standards are
make it very difficult to understand. followed
helps detect as many errors as
possible before testing.
Code inspection and code

Code Walk Through
walk throughs
Detect as many errors as possible An informal code analysis technique.
during inspection and walkthrough: undertaken after the coding of a module is
complete.
detected errors require less effort for
correction A few members of the development team
much higher effort needed if errors were select some test cases:
to be detected during integration or simulate execution of the code by hand using
system testing. these test cases.
Code Walk Through Code Walk Through

The guidelines should be considered as
Even though an informal technique:
examples:
several guidelines have evolved over the years
rather than accepted as rules to be applied
making this naive but useful analysis technique dogmatically.
more effective.
These guidelines are based on The team performing code walk through
personal experience, common sense, and several
should not be either too big or too small.
subjective factors. Ideally, it should consist of between three to seven
members.

Unit - II
Code Walk Through Code Inspection

Discussion should focus on discovery of In contrast to code walk throughs,
errors: code inspection aims mainly at discovery of
and not on how to fix the discovered errors. commonly made errors.
To foster cooperation: During code inspection:
avoid the feeling among engineers that they the code is examined for the presence of certain kinds
are being evaluated in the code walk of errors,
through meeting,
in contrast to the hand simulation of code execution
managers should not attend the walk
through meetings. done in code walk throughs.
Code Inspection Code Inspection

For instance, consider:
Good software development companies:
classical error of writing a procedure that modifies a formal
parameter collect statistics of errors committed by their
while the calling routine calls the procedure with a constant engineers
actual parameter. identify the types of errors most frequently
It is more likely that such an error will be discovered: committed.
by looking for this kind of mistakes in the code, A list of common errors:
rather than by simply hand simulating execution of the can be used during code inspection to look out for
procedure.
possible errors.
Commonly made errors Code Inspection

Use of uninitialized variables.
Use of incorrect logical operators
Nonterminating loops. or incorrect precedence among operators.
Array indices out of bounds. Improper modification of loop variables.
Incompatible assignments. Comparison of equality of floating point values,
Improper storage allocation and deallocation. etc.
Actual and formal parameter mismatch in procedure Also during code inspection,
calls.
adherence to coding standards is checked.
Jumps into loops.

Unit - II
Software Documentation Software Documentation

When developing a software product we develop Good documents enhance understandability and
various kinds of documents : maintainability of a software product.
In addition to executable files and the source code:
Different types of software documents can be
users' manual,
software requirements specification (SRS) document,
classified into:
design document, test document, internal documentation,
installation manual, etc. external documentation (supporting documents).
All these documents are a vital part of good software
development practice.
Internal Documentation Internal Documentation

Internal documentation: Internal documentation provided through:
use of meaningful variable names,
documentation provided in the source code indentation,
code itself. code structuring,
External documentation: use of enumerated types and constant identifiers,
use of user-defined data types, etc.
documentation other than those present module headers and comments
in the source code.
Internal Documentation Internal Documentation

Good software development organizations: Careful experimentation suggests:
ensure good internal documentation
meaningful variable names
through coding standards and coding
guidelines.
is the most useful internal
Example of unhelpful documentation:
documentation.
a = 10; /* a made 10 */

Unit - II
External Documentation External Documentation

Users' manual, A systematic software development style
Software requirements specification ensures:
document, all external documents are produced in an
Design document, orderly fashion.
Test documents, An important feature of good
Installation instructions, etc. documentation is consistency.
External Documentation
Unless all documents are consistent with each
other,
a lot of confusion is created for somebody trying to
understand the product.
All the documents for a product should be up-to-
date:
Even a few out-of-date documents can create severe
confusion.
Unit -II: Software Testing 219 Dr. Rabins Porwal

UNIT-2 - Software Testing - 6 Slides Handouts

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

UNIT-2 - Software Testing - 6 Slides Handouts

Uploaded by

Copyright:

Available Formats

Unit - II

Unit -II: Software Testing 2 Dr. Rabins Porwal

Compiled by: Dr. Rabins Porwal 1

How do you test a

How do you test a How do you test a

Error, Faults, and Failures Error, Faults, and Failures

Compiled by: Dr. Rabins Porwal 2

Verification versus Verification versus

Design of Test Cases Design of Test Cases

of reasonable size and Number of test cases in a randomly selected

Compiled by: Dr. Rabins Porwal 3

Design of Test Cases Design of Test Cases

Design of Test Cases Design of Test Cases

Compiled by: Dr. Rabins Porwal 4

Unit testing Unit testing

Integration testing System Testing

module under test calls and drivers are designed to

Compiled by: Dr. Rabins Porwal 5

Test Drivers & Test Stubs

White-box Testing Black-box Testing

Equivalence Class Partitioning Why define equivalence

Compiled by: Dr. Rabins Porwal 6

Equivalence Class Partitioning Equivalence Class Partitioning

How do you determine the If the input data to the program is

Equivalence Class Partitioning Example

Example (cont.) Example (cont.)

Compiled by: Dr. Rabins Porwal 7

Boundary Value Analysis Boundary Value Analysis

Example Robustness testing

Robustness testing Worst-case testing

100 It is more thorough in the sense that boundary value test

Compiled by: Dr. Rabins Porwal 8

Decision Table Based Testing Decision Table Based Testing

Decision Table Based Testing Decision Table Based Testing

Table 4: Modified decision table

Decision Table Based Testing Cause Effect Graphing Technique

Test cases of triangle problem using decision table

Compiled by: Dr. Rabins Porwal 9

Cause Effect Graphing Technique Cause Effect Graphing Technique

The causes are

and the effects are

Cause Effect Graphing Technique Cause Effect Graphing Technique

The E constraint states that it must always be true that is at most

Fig. 9: Sample cause effect graph

Cause Effect Graphing Technique Cause Effect Graphing Technique

Compiled by: Dr. Rabins Porwal 10

Cause Effect Graphing Technique Cause Effect Graphing Technique

Fig. 12 : Sample cause effect graph with exclusive constraint

Cause Effect Graphing Technique Cause Effect Graphing Technique

Cause Effect Graphing Technique

Compiled by: Dr. Rabins Porwal 11

Statement Coverage Statement Coverage

Statement coverage criterion Statement coverage criterion

Euclid's GCD computation

Compiled by: Dr. Rabins Porwal 12

Branch Coverage Branch Coverage

Stronger testing Example

Example Condition Coverage

Compiled by: Dr. Rabins Porwal 13

Example Branch testing

Branch testing Condition coverage

Condition coverage Path Coverage

Compiled by: Dr. Rabins Porwal 14