Software Testing

Copyright by Hong Xu 1
• ‘Testing can only show the presence of errors,
not their absence.’
• Exhaustive testing, where every possible program execution
sequence is tested, is impossible.
• Automatic test case generation is impossible.

• To convince system developers and customers that the

software is good enough for use.
2 distinct goals:
• To discover faults or defects in the software where the behavior of the
software is incorrect, undesirable or does not conform to its specification –
defect testing
• To demonstrate to the developer and the customer that the software meets its
requirements – validation testing

Copyright by Hong Xu 2
Testing process -1 >> test plans
• Testing phases in the software process (Fig. 4.10)

• Test plans, the link between testing and

development activities, should include [S22.1]:
• The testing process,
• A description of the items to be tested,
• The testing schedule and resource allocation,
• The procedures for managing the testing process,
• The hardware and software requirements, and
• Any testing problems that are likely to arise.

Copyright by Hong Xu 3
Testing process -2 >> component
testing -1
• The testing process (Fig. 4.9, Fig. 23.1)
Component testing, system testing and acceptance testing
Responsible people

• 3 types of component testing [S23.2]

Unit testing
Defect testing

• Test individual functions or methods within an object (p548)

A set of calls to these routines with different input parameters
• Test object classes that have several attributes and methods (p548)
Inheritance

Copyright by Hong Xu 4
Testing process -2 >> component
testing -2
[S23.2.1]

• Test composite components made up of several interacting objects or

functions (Fig. 23.7)
Interface testing, to test that the component interface behaves according to
its specification

Static validation techniques are often more cost-effective.

Compilers, static analyzers, and program inspections
4 types of interfaces between components (p550)
Parameter, shared memory, procedural and message passing
interfaces
3 classes of interface errors (p550)
Interface misuse, interface misunderstanding and timing errors
Some general guidelines (p551), interface test case design

Copyright by Hong Xu 5
Testing process -2 >> system testing
-2.1 integration testing
[S23.1.1]

• 2 distinct system testing phases

Testing vs. software processes

• Integration testing
Component¨ interactions¨(  last PPT)
Integration and testing
Integration strategy
Top-down, bottom-up and a mixture of these
A good rule of thumb is to integrate the components that implement the
most frequently used functionality first.
Problems¨
Localizing and repairing errors
Regression testing

Copyright by Hong Xu 6
Testing process -2 >> system testing
-2.1 integration testing
[S23.1.1]

Off-the shelf, reusable or

• 2 distinct system testing phases newly developed
Testing vs. software processes
Actually work together,
• Integration testing are called correctly and
Component¨ interactions¨(  last PPT)
transfer the right data at the right
Integration and testing time across their interfaces
Integration strategy
Top-down, bottom-up and aThe implementation
mixture of these of system features may be
spreadtheacross
A good rule of thumb is to integrate a number
components ofimplement
that components.
the
Integrating
most frequently used functionality first.and testing a new component can
Problems¨ change the pattern of already tested
Localizing and repairing errorscomponent interactions.
Regression testing

Copyright by Hong Xu 7
Testing process -2 >> system testing
-2.2 release testing
[S23.1.2]

• Release testing
A system release¨

Black-box testing (functional testing) (Fig. 23.4)

Use previous experience of what are likely to be successful defect tests
and testing guidelines (e.g. p544) to help choose test cases.

Scenario-based testing, the best approach

Devote efforts to those parts of the system that receive the most use.

Performance testing¨ [S23.1.3]

Stress testing (p547)

Copyright by Hong Xu 8
Testing process -2 >> system testing
-2.2 release testing
[S23.1.2]

• Release testing Meets the system requirements:

A system release¨ delivers the specified functionality,
performance and dependability, and
does not fail during normal use
Black-box testing (functional testing) (Fig. 23.4)
Use previous experience of what are likely to be successful defect tests
and testing guidelines (e.g. p544) to help choose test cases.

Scenario-based testing, the best approach

Devote efforts to those parts of the system that receive the most use.
Can process its intended load
Performance testing¨ [S23.1.3]
Stress testing (p547)

Copyright by Hong Xu 9
Testing process -3 >> test case
design -1
• A model of the software testing process (Fig.
23.2)
• Test cases, specifications of the inputs to the test and the expected
output from the system plus a statement of what is being tested
Testing policies for choosing a subset of possible test cases
One general testing policy: all program statements should be executed at
least once.

• Test data, the inputs that have been devised to test the system

• Test outputs, which can only be predicted by people who understand

what the system should do.

Copyright by Hong Xu 10
Testing process -3 >> test case
design -2
[S23.3]

• Test case design approaches

In general, start with the highest-level tests from the requirements,
then progressively add more detailed tests using partition and
structural testing (p552).
• Requirements-based testing [S23.3.1]
Consider each requirement and derive a set of tests for it.
• Partition testing (Fig. 23.8) [S23.3.2]
Identify all (input/output) equivalence partitions for a system or component
so that the input data or output results of a program lie within these partitions.
Choose test cases from each of these partitions, which are on the boundaries
of the partitions plus close to the mid-point of the partition.
Guidelines in designing test cases for programs with sequences, arrays or
lists (p556)

Copyright by Hong Xu 11
Testing process -3 >> test case
design -3
• Structural testing (Fig. 23.12) [S23.3.3]
White-box testing
Derive tests from knowledge of the software’s structure and implementation

Path testing [S23.3.4]

Does not test all possible combinations of all paths through a component or
program, BUT to ensure that each independent path¨ is executed at lease
once.
After discovering the number of independent paths through the code by
computing the cyclomatic complexity, next design test cases to execute each
of these paths.

A program flow graph¨, the starting point for path testing

Copyright by Hong Xu 12
Testing process -3 >> test case
design -3
• Structural testing (Fig. 23.12) [S23.3.3]
White-box testing
Derive tests from knowledge of the software’s structure and implementation

Path testing [S23.3.4] one that traverses at least one

new edge
Does not test all possible combinations of all paths in the
through flow graphor
a component
program, BUT to ensure that each independent path¨ is executed at lease
once.
After discovering the number of independent paths through the code by
computing the cyclomatic complexity, next design test cases to execute each
of these paths.
A static program analyzer [22.3]
A program flow graph¨, the starting point A
fordynamic program analyzer
path testing

Copyright by Hong Xu 13
Test automation [S23.4]

• A software testing workbench (p562-563)

Fig23.17 vs. Fig. 23.2

Copyright by Hong Xu 14
TSPi Forms and Instructions

Form Abbreviation Book Book

Reference Page
Test Log LOGTEST 197 468

Defect LOGD 464

Recording Log

Copyright by Hong Xu 15