# SOFTWARE TESTING METHODOLOGY Term:2009-2010 B.Tech III/IT Unit-IV PPT Slides Text Books: 1.

Software Testing Techniques: Boris Beizer 2. Craft of Software Testing: Brain Marrick Semester - I

Prof. N. Prasanna Balaji

Sub Topic No¶s 1 2 3 4 5 6 7 Introduction

Sub Topic name

Lecturer No L1 L2 L3 L4 L5 L6 L7

Slide No¶s

3 22 28 42 51 65 70

Domains & Paths Nice Domains Ugly Domains Domain Testing Domains & Interfaces Testing Domains & Testability

2

Domain Testing - Domains & Paths

Domain Testing Model
D1 D2

INPUT
(x,y,z,«) (1, 2, 3, 4, 5,«)

CLASSIFY
D3 Dn

DO CASE 1

OUTPUT
{ Outcome }

DO CASE 2 DO CASE 3

DO CASE n

ref boris beizer

3

Domain Testing Domain Testing

Views Programs as input data classifiers

Verifies if the classification is correct

ref boris beizer

4

Domain Testing - Domains & Paths

Domain Testing Model Two Views  Based on Specs
 Functional Testing

 Based on Implementation information
 Structural Technique

ref boris beizer

5

Domain Testing - Domains & Paths

Domain Testing Model

 Variables  Simple combinations of two variables  Numbers from -w to +w  Input variables as numbers

 Loop-free Programs

ref boris beizer

6

Domain Testing - Domains & Paths

Domain Testing Model
 Structural Knowledge is not needed. Only Specs.

 For each Input Case,  A Hypothetical path for Functional testing  An Actual path for Structural testing

ref boris beizer

7

Domain Testing - Domains & Paths

Domain

(simplified)

 A Single Connected Set of numbers  No arbitrary discrete sets  Defined by the boundaries
   One or more boundaries Specified by predicates Bugs likely at the boundaries

D1

 One or more variables
ref boris beizer 8

Domain Testing - Domains & Paths

Predicates
 Interpretation  Structural Testing - CFG  Functional Testing - DFG  Specifies the Domain Boundary  n Predicates in Sequence   2n domains
 Or, just two domains
ref boris beizer

A .AND. B .AND. C
9

Domain Testing - Domains & Paths

Paths
 At least One PATH thru the Program  More paths for disconnected domains

ref boris beizer

10

Domain Testing - Domains & Paths

Summary:
 Domain for a loop-free program corresponds to a set of numbers defined over the input vector

 For every domain, there is at least one path thru the routine, along which that domain¶s processing is done

 The set of interpreted predicates traversed on that path (ie., the path¶s predicate expression) defines the domain¶s boundaries.

ref boris beizer

11

Domain Testing - Domains & Paths - Domain Closure Boundary: Closed / Open
X >= MIN
MIN D1 Closed

X <= MAX
MAX

D2

D3

D1

X >= MIN

D2

X < MAX

D3

MIN Closed ref boris beizer

Open

MAX 12

Domain Testing - Domains & Paths - Domain Closure Boundary: Closed / Open

MIN D1 Open

MAX D2 D3

X > MIN

X < MAX

ref boris beizer

13

Domain Testing - Domains & Paths - Domain Dimensionality

 One dimension per variable

 At least one predicate
 Slices thru previously defined Domain

 Slicing Boundary

 N-spaces are cut by Hyperplanes

ref boris beizer

14

Domain Testing - Domains & Paths ± Bug Assumptions  Processing is OK. Domain definition may be wrong.

Boundaries are wrong.   Predicates are wrong.

 Once input vector is set on the right path, it¶s correctly processed.

 More bugs causing domain errors «
ref boris beizer 15

Domain Testing - Domains & Paths ± Bug Assumptions..

 Double-zero representation

 Floating point zero check

 Ambiguous domains
ref boris beizer

defined undefined 16

Domain Testing - Domains & Paths ± Bug Assumptions..

 Over-specified domains
X > 3 .AND. X < 2 .AND. Y > 3

 Boundary errors

Boundary closure

Shifted
ref boris beizer 17

Domain Testing - Domains & Paths - Bug Assumptions

 Boundary errors«.

Tilted

Missing

Extra boundary
ref boris beizer 18

Domain Testing - Domains & Paths - Bug Assumptions

U3

X >= 3

 Closure Reversal

 Faulty Logic Simplification of compound predicates
ref boris beizer 19

Domain Testing - Domains & Paths - Bug Assumptions

Strategy for domain testing in 2-dim 2ref boris beizer 20

Domain Testing - Domains & Paths - Restrictions In General«

1. Coincidental Correctness DT cannot detect Example
F1(x) : +1

 Representative outcome Partition testing Input Equivalence
ref boris beizer D2 F2(x) : -1

D1

21

Domain Testing - Domains & Paths

Domain Testing Model
D1 Function f1

INPUT

CLASSIFY
D2

DO CASE 1

OUTPUT

(x, y, z, «) (1, 2, 3, 4, 5,«)

{ Outcome }

DO CASE 2

f2

D3

DO CASE 3

f3

Dn

f4

DO CASE n

ref boris beizer

22

Domain Testing - Domains & Paths - - Restrictions 3. Simple boundaries & Compound predicates
D1

X
0 16

X >=0 .AND. X <= 16

Y
0

D1 Y = 1 .AND. X >=0 .AND. X <= 16 16

X

ref boris beizer

23

Domain Testing - Domains & Paths - - Restrictions Compound predicates«.  Impact of .OR.

 Concave, Disconnected  Adjacent domains with same function  Example ABC + DEF

 Eliminate compound predicates
ref boris beizer 24

Domain Testing - Domains & Paths - Restrictions

4. Functional Homogeneity of Bugs

 Functional form still Retained

a x + b y >= c

 Bugs are only in a, b, c

ref boris beizer

25

Domain Testing - Domains & Paths - - Restrictions

5. Linear Vector Space
 Linear boundary predicate, Interpreted  Simple relational operators  Conversion to linear vector space
 2-d Polar co-ordinates  Polynomials

 Problems with Non-linear Boundaries
ref boris beizer 26

Domain Testing - Domains & Paths - Restrictions

6. Loop-free Software

Predicate for each iteration

Loop over the entire transaction

Definite loop
ref boris beizer 27

Domain Testing - Domains & Paths ± Nice & Ugly domains

Nice Domains
Requirements  Bugs   ill-defined domains

Before DT
 Analyze specs  Make the Boundary Specs Consistent & Complete
ref boris beizer 28

Domain Testing - Domains & Paths ± Nice domains

Implemented Domains
 Complete, Consistent & Process all inputs

Specified Domains
 Incomplete, Inconsistent

Programmer¶s / Designer¶s Effort
ref boris beizer

29

Domain Testing - Domains & Paths ± Nice domains

Nice Domains
 Linear, Complete, Systematic, Orthogonal, Consistently Closed, & Convex
U1 U2

V1

D11

D12

V2

D21

D22

ref boris beizer

30

Domain Testing - Domains & Paths ± Nice domains

Nice Domains
 Ease of DT

 Fewer Bug Occurrences

ref boris beizer

31

Domain Testing - Domains & Paths ± Nice domains

Nice Domains
Boundaries are

1. Linear 2. Complete

5. Closure consistency

6. Convex 3. Systematic 4. Orthogonal 7. Simply connected

ref boris beizer

32

Domain Testing - Domains & Paths ± Nice domains

1. Linear Boundaries Interpreted linear inequalities
n-dim Hyperplane: n+1 Points n+1 + 1 Test Cases

Non-Linear  Transform
ref boris beizer 33

Domain Testing - Domains & Paths ± Nice domains

2. Complete Boundaries  Span the total number space
 One set of Tests

(-w, +w)

Incomplete«  Reasons  Tests
ref boris beizer 34

Domain Testing - Domains & Paths - Nice domains

3. Systematic Boundaries
 Linear Inequalities differing by a constant
fj (X)  kj or, fj (X)  g (j, c) g (j, c) = j + k * c

 Parallel lines  Identical Sectors in a Circle

 DT  Test a domain

boris beizer for other Domains Tests ref

35

Domain Testing - Domains & Paths - Nice domains

4. Orthogonal Boundaries
 Two boundaries  Parallel to axes
Vj

or, boundary sets

Uj

 DT  Each Set Independently  # Tests E O (n)
ref boris beizer 36

Domain Testing - Domains & Paths - Nice domains

Orthogonal Boundaries
 Tilted sets  transformation  Test cases: O(n)

 Concentric circles with radial lines  Rectangular coordinates  Polar r  aj .AND. r < aj+1 .AND. U  Uj .AND. U < Uj+1
ref boris beizer 37

Domain Testing - Domains & Paths - Nice domains

NonNon-Orthogonal Boundaries
 Test Intersections
O ( n2 ) + O (n)

ref boris beizer

38

Domain Testing - Domains & Paths - Nice domains

5. Closure Consistency
 A Simple pattern in all boundary closures  Example  Same relational operator for systematic boundaries
D3 D1 D2 D4

ref boris beizer

39

Domain Testing - Domains & Paths - Nice domains

6. Convex Domain
 Line joining any two points lies with in the domain  DT  n on-points & 1 off-pt

Concave  ³ But, However, Except, Or « ³ in Specs  Handle with special care
ref boris beizer 40

Domain Testing - Domains & Paths - Nice domains

7. Simply Connected Domain
 In a single piece
D2 D1 D3

D1

D2

 2 complementary domains  D1: Convex   D2: Concave, not-Connected

 Programmer / Designer  Convex part First
ref boris beizer 41

Domain Testing - Domains & Paths ± Ugly Domains Generally,

Programmer / Designer Simplifies

=>

Ugly to Good

possibility of Introduction of more bugs ?!?

ref boris beizer

42

Domain Testing - Domains & Paths ± Ugly Domains Causes

1. Non-linear Boundaries Non-

1. Simplifying the topology

1. Rectifying boundary closures

ref boris beizer

43

Domain Testing - Domains & Paths ± Ugly Domains 1. Non-linear Boundaries Non Transform

ref boris beizer

44

Domain Testing - Domains & Paths ± Ugly Domains 2. Ambiguities  Holes in input vector space.  Missing boundary  Detected by Specification languages & tools.

ref boris beizer

45

Domain Testing - Domains & Paths ± Ugly Domains 2. Contradictions..  Overlapping of  Domain Specs  Closure Specs

D3

D1

D2

ref boris beizer

46

Domain Testing - Domains & Paths ± Ugly Domains 3. Simplifying the Topology«.

 Complexity !
 Concavity, Holes, Disconnectedness

ref boris beizer

47

Domain Testing - Domains & Paths ± Ugly Domains 3. Simplifying the Topology«.  Smoothing out concavity

 Filling in Holes

ref boris beizer

48

Domain Testing - Domains & Paths ± Ugly Domains 3. Simplifying the Topology«.

 Joining the pieces

Correct:
 Connect disconnected boundary segments  Extend boundaries to infinity
ref boris beizer 49

Domain Testing - Domains & Paths ± Ugly Domains 4. Rectifying Boundary Closures.  make closures in one direction for parallel boundaries with closures in both directions  Force a Bounding Hyperplane to belong to the Domain.

Consistent Direction

Inclusion / Exclusion Consistency
ref boris beizer 50

Domain Testing - Domains & Paths ± Domain Testing General DT Strategy 1. Select test points near the boundaries. 2. Define test strategy for each possible bug related to boundary 3. Test points for a domain useful to test its adjacent domain. 4. Run the tests. By post test analysis determine if any boundaries are faulty & if so how? 5. Run enough tests to verify every boundary of every domain
ref boris beizer 51

Domain Testing - Domains & Paths ± Domain Testing DT for Specific Domain Bugs Generally,  Interior point  Exterior point  Epsilon neighborhood  Extreme point  On point

ref boris beizer

52

Domain Testing - Domains & Paths ± Domain Testing DT for Specific Domain Bugs
Off Points

Extreme point

Boundary point

Epsilon neighborhood On Points

Domain D1
ref boris beizer 53

Domain Testing - Domains & Paths ± Domain Testing 1. 1-d Domains

2.

2-d Domains

3.

Equality & inequality Predicates

4.

Random Testing

5.

Testing n-dimensional Domains nref boris beizer 54

Domain Testing - Domains & Paths ± Domain Testing Testing 1-d Domains : 1B x A

Bugs with open boundaries
B x A

Closure Bug

Shift left Bug

B x1 x

A

Shift Right Bug

B x x1

A

ref boris beizer

55

Domain Testing - Domains & Paths ± Domain Testing Testing 1-d Domains : 1B x A

Bugs with open boundaries
B x A

Missing Boundary

x

Extra Boundary

B x

x

A

x x1

C

Bugs with Closed Boundaries : Similar to the above
ref boris beizer 56

Domain Testing - Domains & Paths ± Domain Testing 2. Testing 2-d Domains 2 Closure bug  Boundary Shift : up / down  Tilted Boundary  Extra Boundary  Missing Boundary

ref boris beizer

57

Domain Testing - Domains & Paths ± Domain Testing

2 n : Domains share tests 3 n : no sharing of tests by domains

ref boris beizer

Strategy for domain testing in 2-dim 258

Domain Testing - Domains & Paths ± Domain Testing 3. Equality & Inequality Predicates
 An Equality predicate defines a line in 2-d

a A c d B b c¶ c

To avoid bugs

ref boris beizer

59

Domain Testing - Domains & Paths ± Domain Testing 4. Random Testing  A Point in the center : verifies computation

ref boris beizer

60

Domain Testing - Domains & Paths ± Domain Testing 5. Testing n-Dimensional Domains (strategy) nn-dimensions, p boundary segments  (n+1)*p test cases : n on points & 1 off point  Extreme pt shared : 2 * p points  Equalities over m-dimensions create a subspace of n-m dimensions  Orthogonal domains with consistent boundary closures, orthogonal to the axes & complete boundaries  Independent testing
ref boris beizer 61

Domain Testing - Domains & Paths ± Domain Testing Procedure for solving for values
Simple procedure. Need tools.

L5

1. Identify input variables 2. Identify variables which appear in domain-defining predicates, such as control-flow predicates

ref boris beizer

62

Domain Testing - Domains & Paths ± Domain Testing Procedure 3. Interpret all domain predicates in terms of input variables:  Transform non-linear to linear  Find data flow path 4. Predicate expression with p # predicates. Find # domains : < 2 p

L5

3. Solve inequalities for extreme points 4. Use extreme points to solve for nearby on points «
ref boris beizer 63

Domain Testing
Effectiveness  Cost effective  Bugs on boundaries, extreme points  Hardware logic testing ± tool intensive

L5

ref boris beizer

64

Domain Testing Domains & Interface Testing  Domain  Range

L6

Variable Classify

Domain Function / Routine

Range

ref boris beizer

65

Domain Testing
Domains & Interface Testing
Variable

L6

Routine 1
Domain

Range for Routine1

Domain for Routine2

Routine 2

Range for Routine2

 Span compatibility

ref boris beizer

66

Domain Testing
Interface Range/Domain Compatibility Testing  Test each var independently

L6

 Find an inverse function

 Build a super domain

ref boris beizer

67

Domain Testing
Domains and Testability  Linearizing transformations  Polynomial  Rectangular to polar  Generic & rational => Taylor series

L7

ref boris beizer

68

Domain Testing Domains and Testability Perform transformations for better testing.  Co-ordinate transformations Co-

L7

ref boris beizer

69

Domain Testing Domains and Testability  Canonical Program Form

L7

ref boris beizer

70

Domain Testing ± Questions from the previous year¶s exams
1. Explain Nice & Ugly Domains. 2. What is domain testing? Discuss applications of domain testing 3. Explain the domain boundary bugs for two dimensional domains 4. What is the purpose of Domain Testing? Give its schematic representation 5. Define the following concepts. 1. Domains 2. Domain closure 3. Domain dimensionality 4. Bug assumptions for Domain Testing 6. Explain simple domain boundaries & compound predicates. 7. Classify what can go wrong with boundaries, then define a test strategy for each case in domain testing.
ref boris beizer 71

U4