Software Engineering

Lecture 10
Testing and Verification
 Overview

Testing a significant component of most life-cycle
models

An integral component of development
– should occur during every phase of life-cycle

Divided into:-
– execution based
– non-execution based

Independence of testing
– responsibility of different people
– originators have blind spots
 Non-Execution Based
Testing: Walkthrough

Group discussion of design documents
– Concentrates on fault finding, not presenting
solution

Two main Techniques
– authors of various design documents present /
justify their proposals to design team

Team respond with support / criticism
– Each member of design team presents list of
unclear items
 Non-Execution Based
Testing: Inspection

More detailed and structured than walkthrough

Five Main Steps
– Overview : General Statement by one individual
– Preparation: Participants try to understand documentation
– Inspection: Detailed presentation of each document

Ensure every branch covered

Faults identified

Report of faults produced by discussion moderator
– Rework: Author responds to faults presented

proposes solution to faults
– Follow-up: Ensure all faults addressed, no new ones
introduced
Testing during Development Phases

Requirements Phase
– Allow various users to experiment with prototypes
– Ensure people consulted are representative of end users

Object-Oriented Analysis Phase
– Walkthrough / Inspection of design documentation

CRC Cards, Initial Class Diagrams

Design Phase
– Walkthrough / Inspection again (personnel restricted to
design team)

Detailed Class Diagrams, Behavioural Diagrams
– Show how new documentation follows from earlier stages
Criteria for Execution based Testing

Correctness
– Satisfies its output requirements

Utility
– Extent to which user’s needs are met

Reliability
– Frequency of product failure

Robustness
– Error handling, adaptability to differing operating conditions

Performance
– response time, memory utilisation
 Implementation

Class testing
– informal - by programmers
– methodical - SQA group

non-execution based

execution based

Black-box (test to specifications)
– choose test data based on specifications

White-box / Glass-box (test to code)
– test data chosen by examining code
 Black-Box Testing

Not possible to test all input permutations
– 2 variables, one takes one of seven values, the
other one of nine

this implies 56 permutations and is a very simple case!

Art to choose manageable sets of data

Equivalence testing and boundary value
analysis

Equivalence Class: set of data where one
element is effectively as good as another
 Example

Program designed to work on between 1 and 20 records

Equivalence classes:
– < 1 record
– 1-20 records
– > 20 records

One test from each class employed

Plus choose values on and on either side of boundaries
of an equivalence class

e.g. Test with the following record numbers:-
– 0, 1, 2, 10, 19, 20, 21
 Black-Box Testing

Can apply similar technique to output
– devise test data to produce output in each equivalence
class and around boundaries

Alternative: Functional Testing
– Test functionality of whole system
– each item of functionality for the system tested

Calculate total grades

Read in next record

Print monthly accounts
 White-box Testing

Idea is to test every
path through the code
– permutations very large
– Not possible to test all
paths

For example, flowchart:-

5 paths for central group of 6
boxes

whole chart is:-
– 51 + 52 + 53 +…..+ 518 !!
– Any reasonably sized
program far larger
 White Box: Structural Testing

Statement coverage
– test data formulated so that every statement is executed
at least once
– Weakness: Consider:-

if ( a > 1 AND b = 0) then x = 9
– where it should read

if ( a > 1 OR b = 0) then x = 9
– test data a = 2, b = 0 will not trap error as statement will
execute correctly

Branch Coverage
– devise test data such that every branch of every
statement executed: more difficult but more errors found
Structural Testing: Path Coverage

Test all paths through code
– but too many!!
– use linear code sequences
– Identify points L from which control flow may jump

control statements (if, do, switch etc), entry and exit points
– linear code sequence begins at an element of L and ends at
an element L
– testing only these paths significantly reduces the total
number

Alternative: All-definition-use-paths
– For every use of a variable, classify as either

definition / assignment of x: x = 3, read(x)

use of x: y = x + 3
Structural Testing: Path Coverage

All-definition-use-paths
– identify paths between variable definition and its use

test these paths
– very efficient technique, many faults found
– far fewer paths than maximum

Infeasible paths (dead code)
– Some code cannot be reached -- indicates fault

Integration / Maintenance
– Each class tested as completed
– Test the remainder still works!!
– Maintenance: additional problem: personnel unfamiliar with
product
 Formal Verification

Problems may remain after
all testing completed

Alternative: formally prove
program correct

Example: sum elements of
an array
– mathematical assertions made
at each point in program
– Loop invariant: True for each
iteration
– Proof depends on semantics of
programming language

Proof mathematical and time
consuming
 Comparison / Summary

Testing must be performed at all phases of
software development

Earlier fault found, easier to correct

Overall:-
– Black Box testing
– White / Glass Box Testing
– Code walkthrough
are equally efficient at finding bugs

Code Walkthroughs less cost effective

Less experiences programmers find Code
Walkthroughs more difficult