ISE

Validation and Verification

White Box Testing
ISE

Overview
◼ White Box Testing
◼ Test Case Selection Strategies
◼ Control Flow Graph
◼ Basis Path Testing
◼ Loop Testing
◼ Coverage Based Testing
ISE

White Box Testing

◼ Idea is to view the program as a white
box (i.e. clear box)
◼ Select the test cases based on the
internals of the program
◼ Code-based testing or glass box testing
ISE
White Box Testing –
A common view
Test Case Selection Strategies

PROGRAM

…
If (a > b) Then
input Do something output
Else
Do other things
…
ISE
White Box Testing –
Advantages
◼ Can test those implemented
functionality
◼ Uncover types of errors different from
those detected by Black box testing
ISE
White Box Testing –
Disadvantages
◼ Cannot test those specified functions
which are not implemented
◼ Need to know the language syntax
◼ Need to wait until the source code is
ready
ISE

Example – GetExtraHours
public int getExtraHours() “test case inputs”
{ Implication: need
int hours, i; to set the values
hours = 0; of these variables
before calling the
i = 0;
getExtraHours()
while (i < numSession) method
{
if (session[i].status == EXTRA)
hours = hours + session[i].duration;
i++;
}
return hours;
}
ISE

Example – GetExtraHours (Alt)

public int getExtraHours() “test case inputs”
{ Implication: need
int hours, i; to set the values
hours = 0; of these variables
before calling the
for (i=0; i < numSession; i++)
getExtraHours()
{ method
if (session[i].status == EXTRA)
hours = hours + session[i].duration;
}
return hours;
}
ISE

White Box Testing Strategies

◼ Basis Path Testing
◼ Loop Testing
◼ Coverage Based Testing
◼ Statement coverage
◼ Branch coverage
◼ Path coverage
ISE

Control Flow Graph (CFG)

◼ A graph that shows the flow of control
in a program
◼ CFG should have ONE entry point (start)
and ONE exit point (stop)
◼ Basically, all white box test case
selection techniques rely on the control
flow graph
ISE
Control Flow Graph –
GetExtraHours Example
Start 1

2
3
4

Stop 6
ISE

Basis Path Testing

◼ Idea is to identify a set of paths that
can be used to form any path of the
program
◼ Each path in the set is called the basis
path
◼ Each path in the set is independent of
other paths
ISE

Basis Path Testing (cont’d)

◼ McCabe’s Cyclomatic complexity, v(G)
= e – n + 2; or
= number of regions in G
G: the CFG of a program
e: number of edges in G
n: number of nodes in G
ISE

Basis Path Testing (cont’d)

◼ The number of basis paths is always
v(G).
◼ There are many different sets of basis
paths.
◼ i.e. basis paths are non-unique.
◼ It does not work if the program is
poorly structured.
ISE

Basis Path Testing – Steps

1. Draw the CFG
2. Calculate the number of basis paths
3. Identify all basis paths
4. Generate a test case for each of the
basis paths
ISE
Basis Path Testing –
Identifying basis paths
a) Have your CFG ready
b) Imagine a “virtual flow” from the last
node (stop) to the first node (start)
c) Find all the simple circles in CFG
d) Identify a path from start to stop (and
back to start, if needed) that contains
a simple circle for every simple circle
e) Discard the “virtual flow” if necessary
ISE
Basis Path Testing –
Identifying basis paths (cont’d)
◼ Simple circles
1
◼ 1-2-6-1
◼ 2-3-5-2
2
◼ 2-3-4-5-2
3
◼ Basis paths
4 ◼ 1-2-6
5 ◼ 1-2-3-5-2-6
6 ◼ 1-2-3-4-5-2-6
ISE
Basis Path Testing –
GetExtraHours Example
1. Control Flow Graph (see slide 11)
2. Number of basis path
=e–n+2=7–6+2=3
3. Basis Paths (see slide 17)
◼ 1-2-6
◼ 1-2-3-5-2-6
◼ 1-2-3-4-5-2-6
ISE
Basis Path Testing – Example
(cont’d)
4. Test Cases (based on coding in slide 7)
▪ Basis Path 1-2-6
▪ numSession = 0
▪ Basis Path 1-2-3-5-2-6
▪ numSession = 1; and
▪ session[0].status* = NORMAL
▪ Basis Path 1-2-3-4-5-2-6
▪ numSession = 1; and
▪ session[0].status = EXTRA

*Assuming that the status must be either NORMAL or EXTRA

ISE

Loop Testing
◼ Select test cases to exercise various
possibilities that a loop can be executed
◼ Advantages
◼ uncover initialization failures
◼ uncover indexing and incremental failures
◼ uncover boundary failures at loop limits
ISE

Loop Testing - Steps

◼ If a loop can be repeated with a
maximum of n times, identify a test
case that exercises the loop (if possible)
◼ 0, 1, and 2 times
◼ m times (m<n)
◼ n-1, n, and n+1 times
ISE
Loop Testing – GetExtraHours
Example
◼ No upper limit of the loop, so design test case
to loop
◼ 0 time
◼ numSession = 0
◼ 1 time
◼ numSession = 1; …(details of session[…])…
◼ 2 times
◼ numSession = 2; …(details of session[…])…
◼ 5 times (let m be 5)
◼ numSession = 5; …(details of session[…])…
ISE

Loop Testing – Types

◼ Concatenated loops
◼ One loop followed by another loop
◼ Nested loops
◼ One loop inside the other
◼ Unstructured loops
◼ There must be something wrong in the program.
◼ This is the result of bad programming practices.
◼ If the program survives the inspection, set up
another inspection session!!
ISE
Loop Testing – Concatenated
Loops
◼ Loop A (10 times)
◼ 0, 1, 2, 5, 9, 10, 11
Loop A
◼ Loop B (7 times)
◼ 0, 1, 2, 4, 6, 7, 8
◼ For a better testing
◼ Consider all
Loop B combinations
◼ Remove the
impossible
combinations, if any
ISE

Loop Testing – Nested Loops

◼ Same as
concatenated loops
Loop A but testing of “Loop
A” with 0 times
implies “Loop B”
Loop B must be of 0 times
ISE

Coverage Based Testing

◼ Idea is to
1. select an “entity” (statement, branch, or
path) from the program
2. select test cases to cover all of the
possible entities
ISE
Coverage Based Testing
(cont’d)
◼ Statement Coverage (or Node Coverage)
◼ Every statement in the program should be
exercised at least once
◼ Branch Coverage (or Decision Coverage)
◼ Every branch in the program should be exercised
at least once
◼ Path Coverage
◼ Every execution path in the program should be
exercised at least once
ISE
Coverage Based Testing –
GetExtraHours Example
Coverage Test cases* Execution Paths
criteria numSession; session[0], session[1], …
Statement 1; (EXTRA, 3) 1-2-3-4-5-2-6
Branch 1; (NORMAL, 3) 1-2-3-5-2-6
1; (EXTRA, 3) 1-2-3-4-5-2-6
Path 0; 1-2-6
1; (NORMAL, 3) 1-2-3-5-2-6
1; (EXTRA, 3) 1-2-3-4-5-2-6
2; (NORMAL,3), (NORMAL, 6), … 1-2-3-5-2-3-5-2-6
2; (NORMAL,3), (EXTRA, 2), … 1-2-3-5-2-3-4-5-2-6
… …

*Assuming that the status in session[i] must be either NORMAL or EXTRA

ISE

Path Coverage – Issues

◼ There are usually infinite number of
execution paths.
◼ Some paths may be infeasible.
◼ An execution path is infeasible if there are
no test cases in the input domain that can
execute along the path.
◼ There is no automated algorithm to
determine that a given path is feasible.
ISE

Path Coverage – A Challenge

public int doSomething(int a, int b, int c) ◼ Is there any infeasible
{ path in this method?
int d = 0;
if ((a > 1) && (b == 0))
c = c / a;
if ((a == 2) || (c > 1))
c = c+1;
while (a >= 2)
a = a – 2;
d = a+c;
return d;
}
ISE

Different Levels of Testing

◼ Unit Testing
◼ test individual unit
◼ Integration Testing
◼ test two or more units together
◼ System Testing
◼ test the system as a whole
◼ Function/Performance/User Acceptance Testing
◼ Installation Testing
◼ Regression Testing
ISE

Unit Testing
◼ A unit may be
◼ a module (structural approach); or
◼ a method in a class (OO approach)
◼ Test the functionality of each unit
ISE

Integration Testing
◼ Test two or more units integrated
together
◼ Detect faults related to parameters
passing between units to be tested
◼ Need driver modules and stub modules
ISE
Integration Testing – Different
Ways
◼ Non-Incremental
◼ Big-bang integration
◼ Incremental
◼ Bottom-up integration
◼ Top-down integration
◼ Hybrid/Sandwich integration
ISE

Integration Testing - Example

A

B C D

E F G
ISE

Big-bang Integration Testing

◼ After A,B,C,D,E,F,G are unit tested, test
ALL of them in one trial
A
Unit
B
Test
C
D ALL
E
F
G
ISE

Bottom-up Integration Testing

◼ Integrate the units from bottom to top
E
Unit
B E, F, B
Test
F ALL
C G, C
G
D
A
ISE

Top-down Integration Testing

◼ Integrate the units from top to bottom
A
Unit
B A, B, C,D
Test
C ALL
D
E
F
G
ISE

Hybrid Integration Testing

◼ A mix between bottom-up and top-down
◼ One possible way
◼ B-E-F and C-G
◼ ALL
◼ Another possible way
◼ A-B, A-B-E, A-B-E-F
◼ A-B-E-F-C, A-B-E-F-C-G
◼ ALL
ISE

System Testing
◼ Function Testing
◼ test the functions of the software
◼ Performance Testing
◼ test the performance of the software
◼ User Acceptance Testing
◼ test the functions and performance of the
software in the presence of users
ISE

Function Testing
◼ Concentrates on correctness of the
functions provided by the software
◼ Need valid inputs as well as invalid
inputs
ISE

Performance Testing
◼ Concentrates on the performance issues
such as
◼ Volume tests
◼ high volumes of data are involved
◼ Stress tests
◼ design for system that allows multiple users
◼ large number of users are involved
◼ Refer to Pfleeger 2001 (p.402-403) for
other types of performance tests
ISE

Usability Testing
◼ A kind of performance testing
◼ A controlled experiment to address user
interface requirements
◼ Focus on usability parameters
◼ End-users explore the look and feel of
screens, messages, reports, etc. in a
laboratory environment (such as the
one in SCHIL)
ISE

Usability Test
◼ 3 types
◼ Scenario test
◼ Prototype test
◼ Product test
ISE

User Acceptance Testing

◼ Testing done with the presence of users
to demonstrate the software works
according to the users’ requirements
◼ Alpha test
◼ within the developer’s organization
◼ Beta test
◼ outside the developer’s organization
ISE

Installation Testing
◼ Testing after installation of system in its
working environment
◼ Final test before system becomes
operational
ISE

Regression Testing
◼ A retest of a software system or a software
module after it has been modified (see
http://www.testingstandards.co.uk/living_glos
sary.htm)
◼ Examples: bug fixing, adding new functionalities,
…
◼ To ensure that faults have not been
introduced during the modification of the
software
◼ It is performed after the software/program
has been modified
ISE

Keywords
◼ Dynamic V&V ◼ Performance testing
◼ Error ◼ Regression testing
◼ Failure ◼ Static V&V
◼ Fault ◼ System testing
◼ Unit testing
◼ Function testing
◼ Usability testing
◼ Inspection
◼ User acceptance
◼ Installation testing testing
◼ Integration testing ◼ Validation
◼ Verification