STM Unit 3 DT

Unit 3 – Domain Testing
Compiled with reference from:

Software Testing Techniques: Boris Beizer
Craft of Software Testing: Brain Marrick
Domain Testing U3
We will see in this part of Unit 3:
• Domains & Paths
• Nice & Ugly Domains
• Domain Testing
• Domains & Interfaces Testing
• Domains & Testability
ref boris beizer 2

Transaction-Flow Testing U3
Contents of the unit
• Domains
• Model, as a set, paths
• Predicates
• closure, dimensionality, assumptions
• Nice & ugly domains
• specified & implemented domains
• Nice domains – properties, testing implications
• Ugly domains, simplification by programmers & testers
• Domain testing
• Bugs & testing for those bugs
• Testing procedure
• Variations, tools, effectiveness
• Domains & Interface Testing
• Range, closure, span
• Interface range, domain compatibility
• Finding the values
• Domains & Testability
• Linearizing, coordinate transformations. Canonical program form
ref boris beizer 3

Domain Testing - Domains & Paths U3
Domain Testing Model
D1
D2
INPUT CLASSIFY DO CASE 1 OUTPUT
D3
(x,y,z,…) Dn { Outcome }
DO CASE 2
(1, 2, 3, 4, 5,…)
DO CASE 3
DO CASE n
4 ref boris beizer

Domain Testing U3
Domain Testing
Views Programs as input data classifiers
Verifies if the classification is correct
5 ref boris beizer

Two Views
• Based on Specs
• Functional Testing
• Based on Implementation information
• Structural Technique
6 ref boris beizer

• Variables
• Simple combinations of two variables
• Numbers from - to +
• Input variables as numbers
• Loop-free Programs
7 ref boris beizer

• Structural Knowledge is not needed. Only Specs.
• For each Input Case,
• A Hypothetical path for Functional testing
• An Actual path for Structural testing
8 ref boris beizer

Domain (simplified)
• A Single Connected Set of numbers D1
• No arbitrary discrete sets
• Defined by the boundaries
• One or more boundaries
• Specified by predicates
• Bugs likely at the boundaries
• One or more variables

9 ref boris beizer
Predicates
• Interpretation
• Structural Testing - CFG
• Functional Testing - DFG
• Specifies the Domain Boundary
• n Predicates in Sequence  2n domains
• Or, just two domains A .AND. B .AND. C
10 ref boris beizer

Paths
• At least One PATH thru the Program
• More paths for disconnected domains
11 ref boris beizer

Summary:
• Domain for a loop-free program corresponds to a set of
numbers defined over the input vector
• For every domain, there is at least one path thru the

routine, along which that domain’s processing is done
• The set of interpreted predicates traversed on that path

(ie., the path’s predicate expression) defines the
domain’s boundaries.
12 ref boris beizer

Domain Testing - Domains & Paths - Domain Closure U3
Boundary: Closed / Open
X >= MIN X <= MAX
MIN MAX
D1 D2 D3
Closed
X >= MIN
D2 X < MAX
D1 D3
MIN Closed Open MAX
13 ref boris beizer

Domain Testing - Domains & Paths - Domain Closure
Boundary: Closed / Open
MIN MAX
D1 D2 D3
Open
X > MIN X < MAX
14 ref boris beizer

Domain Testing - Domains & Paths - Domain Dimensionality U3
• One dimension per variable
• At least one predicate

• Slices thru previously defined Domain
• Slicing Boundary
• N-spaces are cut by Hyperplanes
15 ref boris beizer

Domain Testing - Domains & Paths – Bug Assumptions U3
• Processing is OK. Domain definition may be wrong.
 Boundaries are wrong.
 Predicates are wrong.
• Once input vector is set on the right path, it’s correctly

processed.
• More bugs causing domain errors …

16 ref boris beizer
Domain Testing - Domains & Paths – Bug Assumptions.. U3
• Double-zero representation
• Floating point zero check
• Contradictory domains
• Ambiguous domains defined
undefined
17 ref boris beizer
Domain Testing - Domains & Paths – Bug Assumptions.. U3
• Over-specified domains
X > 3 .AND. X < 2 .AND. Y > 3
• Boundary errors
Boundary closure
Shifted
18 ref boris beizer

Domain Testing - Domains & Paths - Bug Assumptions U3
• Boundary errors….
Tilted
Missing
Extra boundary
19 ref boris beizer

X >= 3
• Closure Reversal
• Faulty Logic
Simplification of compound
predicates
20 ref boris beizer

Strategy for domain testing in 2-dim

21 ref boris beizer
Domain Testing - Domains & Paths - Restrictions U3
In General…
1. Coincidental Correctness
DT cannot detect
Example
F1(x) : +1
• Representative outcome D1
Partition testing
Input Equivalence D2
F2(x) : -1
22 ref boris beizer

Function f1
D1
INPUT CLASSIFY DO CASE 1 OUTPUT
(x, y, z, …) D2 { Outcome }
DO CASE 2
(1, 2, 3, 4, 5,…) f2
D3
DO CASE 3
f3
Dn
f4
DO CASE n
23 ref boris beizer

Domain Testing - Domains & Paths - - Restrictions U3
3. Simple boundaries & Compound predicates
D1
X X >=0 .AND. X <= 16
0 16
D1
Y Y = 1 .AND. X >=0 .AND. X <= 16
0 16
X
24 ref boris beizer

Compound predicates….
• Impact of .OR.
• Concave, Disconnected
• Adjacent domains with same function
• Example
ABC + DEF
• Eliminate compound predicates
25 ref boris beizer

4. Functional Homogeneity of Bugs
• Functional form still Retained
a x + b y >= c
• Bugs are only in a, b, c
26 ref boris beizer

5. Linear Vector Space
• Linear boundary predicate, Interpreted

• Simple relational operators
• Conversion to linear vector space
• 2-d Polar co-ordinates
• Polynomials
• Problems with Non-linear Boundaries
27 ref boris beizer

6. Loop-free Software
Predicate for each iteration
Loop over the entire transaction
Definite loop
28 ref boris beizer

Domain Testing - Domains & Paths – Nice & Ugly domains U3
Nice Domains
Requirements
• Bugs  ill-defined domains
Before DT
• Analyze specs
• Make the Boundary Specs Consistent & Complete
29 ref boris beizer

Domain Testing - Domains & Paths – Nice domains U3
Implemented Domains
• Complete, Consistent & Process all inputs
Specified Domains
• Incomplete, Inconsistent
Programmer’s / Designer’s Effort

30 ref boris beizer
Nice Domains
• Linear, Complete, Systematic, Orthogonal, Consistently

Closed, & Convex
U1 U2
V1 D11 D12
V2 D21 D22
31 ref boris beizer

Nice Domains
Advantages
• Ease of DT
• Fewer Bug Occurrences
32 ref boris beizer

Nice Domains
Boundaries are
1. Linear 5. Closure consistency
1. Complete
6. Convex
2. Systematic
1. Orthogonal 7. Simply connected
33 ref boris beizer

1. Linear Boundaries
Interpreted linear inequalities
n-dim Hyperplane:
n+1 Points
n+1 + 1 Test Cases
Non-Linear
• Transform
34 ref boris beizer
2. Complete Boundaries
• Span the total number space (-, +)
• One set of Tests
Incomplete…
• Reasons
• Tests
35 ref boris beizer

Domain Testing - Domains & Paths - Nice domains U3
3. Systematic Boundaries
• Linear Inequalities differing by a constant
fj (X) ≥ kj or, fj (X) ≥ g (j, c) g (j, c) = j + k * c
• Parallel lines
• Identical Sectors in a Circle
• DT
36 • Test a domain  Tests for other Domains
ref boris beizer
4. Orthogonal Boundaries
• Two boundaries or, boundary sets

Uj
• Parallel to axes
Vj
• DT
• Each Set Independently
• # Tests  O (n)
37 ref boris beizer

Orthogonal Boundaries
• Tilted sets
• transformation
• Test cases: O(n)
• Concentric circles with radial lines
• Rectangular coordinates
• Polar
r ≥ aj .AND. r < aj+1 .AND.
 ≥ j .AND.  < j+1
38 ref boris beizer
Non-Orthogonal Boundaries
• Test Intersections  O ( n2 ) + O (n)
39 ref boris beizer

5. Closure Consistency
• A Simple pattern in all boundary closures
• Example
• Same relational operator for systematic boundaries
D3 D1 D2 D4
40 ref boris beizer

6. Convex Domain
• Line joining any two points lies with in the domain
• DT
• n on-points & 1 off-pt
Concave
• “ But, However, Except, Or … “ in Specs

• Handle with special care
41 ref boris beizer

7. Simply Connected Domain

• In a single piece D2 D1 D3
D2
D1
• 2 complementary domains
• D1: Convex  D2: Concave, not-Connected
• Programmer / Designer
• Convex part First
42 ref boris beizer
Domain Testing - Domains & Paths – Ugly Domains U3
Generally,
From Bad Specs
Programmer / Designer Simplifies => Ugly to

Good
possibility of Introduction of more bugs ?!?
43 ref boris beizer

Causes
1. Non-linear Boundaries
1. Ambiguities & Contradictions
1. Simplifying the topology
1. Rectifying boundary closures
44 ref boris beizer

1. Non-linear Boundaries
• Transform
45 ref boris beizer

2. Ambiguities
• Holes in input vector space.
• Missing boundary
• Detected by Specification languages & tools.
46 ref boris beizer

2. Contradictions..
• Overlapping of
• Domain Specs
• Closure Specs
D3
D1 D2
47 ref boris beizer

3. Simplifying the Topology….
• Complexity !
• Concavity, Holes, Disconnectedness
48 ref boris beizer

• Smoothing out concavity
• Filling in Holes
49 ref boris beizer

• Joining the pieces
Correct:
• Connect disconnected boundary segments
• Extend boundaries to infinity
50 ref boris beizer

4. Rectifying Boundary Closures.
• make closures in one direction for parallel boundaries with

closures in both directions
• Force a Bounding Hyperplane to belong to the Domain.
Consistent Direction Inclusion / Exclusion Consistency

51 ref boris beizer
Domain Testing - Domains & Paths – Domain Testing U3
General DT Strategy
1. Select test points near the boundaries.
2. Define test strategy for each possible bug related to

boundary
3. Test points for a domain useful to test its adjacent domain.
4. Run the tests. By post test analysis determine if any

boundaries are faulty & if so how?
5. Run enough tests to verify every boundary of every domain
52 ref boris beizer

DT for Specific Domain Bugs
Generally,
• Interior point
• Exterior point
• Epsilon neighborhood
• Extreme point
• On point
53 ref boris beizer

DT for Specific Domain Bugs

Off Points
Extreme
point
Boundary point
Epsilon neighborhood
On Points
Domain D1
54 ref boris beizer
1. 1-d Domains
2. 2-d Domains
3. Equality & inequality Predicates
4. Random Testing
5. Testing n-dimensional Domains
55 ref boris beizer

Testing 1-d Domains : Bugs with open boundaries
B A B A
Closure Bug
x x
B A
Shift left Bug
x1 x
B A
Shift Right Bug
x x1
56 ref boris beizer

Testing 1-d Domains : Bugs with open boundaries
B A B A
Missing Boundary x
x x
Extra Boundary B A C
x x
x x1
Bugs with Closed Boundaries : Similar to the above
57 ref boris beizer

2. Testing 2-d Domains
• Closure bug
• Boundary Shift : up / down
• Tilted Boundary
• Extra Boundary
• Missing Boundary
58 ref boris beizer

2 n : Domains share tests

3 n : no sharing of tests by domains
Strategy for domain testing in 2-dim

59 ref boris beizer
3. Equality & Inequality Predicates

• An Equality predicate defines a line in 2-d
a
A
c c’
c
d
B
To avoid bugs
60 ref boris beizer

4. Random Testing
• A Point in the center : verifies computation
61 ref boris beizer

5. Testing n-Dimensional Domains (strategy)
n-dimensions, p boundary segments
• (n+1)*p test cases : n on points & 1 off point
• Extreme pt shared : 2 * p points
• Equalities over m-dimensions create a subspace of n-m

dimensions
• Orthogonal domains with consistent boundary closures,

orthogonal to the axes & complete boundaries
• Independent testing
62 ref boris beizer
Procedure for solving for values
Simple procedure. Need tools.
1. Identify input variables
2. Identify variables which appear in domain-defining

predicates, such as control-flow predicates
63 ref boris beizer

Procedure
3. Interpret all domain predicates in terms of input variables:

• Transform non-linear to linear
• Find data flow path
4. Predicate expression with p # predicates.

Find # domains : < 2 p
3. Solve inequalities for extreme points
4. Use extreme points to solve for nearby on points …
64 ref boris beizer

Effectiveness
• Cost effective
• Bugs on boundaries, extreme points
• Hardware logic testing – tool intensive
65 ref boris beizer

Domains & Interface Testing
• Domain
• Range
Variable Domain Range

Function /
Classify
Routine
66 ref boris beizer

Variable
Range for Domain for Range for
Routine 1 Routine 2
Domain Routine1 Routine2 Routine2
• Span compatibility
67 ref boris beizer

• Closure compatibility
68 ref boris beizer

Interface Range/Domain Compatibility Testing
• Test each var independently
• Find an inverse function
• Build a super domain
69 ref boris beizer

Finding the values / solving inequalities
70 ref boris beizer

Domains and Testability
• Linearizing transformations
• Polynomial
• Rectangular to polar
• Generic & rational => Taylor series
71 ref boris beizer

Perform transformations for better testing.
• Co-ordinate transformations
72 ref boris beizer

• Canonical Program Form
73 ref boris beizer

Domain Testing – Questions from the previous year’s exams U3
1. Explain Nice & Ugly Domains.
2. What is domain testing? Discuss applications of domain testing
3. Explain the domain boundary bugs for two dimensional domains
4. What is the purpose of Domain Testing? Give its schematic

representation
5. Define the following concepts.

1. Domains 2. Domain closure 3. Domain
dimensionality 4. Bug assumptions for Domain Testing
6. Explain simple domain boundaries & compound predicates.
7. Classify what can go wrong with boundaries, then define a test

strategy for each case in domain testing.
74 ref boris beizer
Domain Testing U-3
To Unit 5…Paths, Path Products & Regular Expressions
75 ref boris beizer

STM Unit 3 DT

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

STM Unit 3 DT

Uploaded by

Copyright:

Available Formats

Unit 3 – Domain Testing

Compiled with reference from:

We will see in this part of Unit 3:

• Domains & Paths

• Nice & Ugly Domains

• Domains & Interfaces Testing

• Domains & Testability

ref boris beizer 2

Contents of the unit

ref boris beizer 3

Domain Testing Model

4 ref boris beizer

Views Programs as input data classifiers

Verifies if the classification is correct

5 ref boris beizer

Domain Testing Model

• Based on Implementation information

6 ref boris beizer

Domain Testing Model

• Input variables as numbers

7 ref boris beizer

Domain Testing Model

• Structural Knowledge is not needed. Only Specs.

• For each Input Case,

• A Hypothetical path for Functional testing

• An Actual path for Structural testing

8 ref boris beizer

• A Single Connected Set of numbers D1

• No arbitrary discrete sets

• Defined by the boundaries

• One or more boundaries

• Bugs likely at the boundaries

• One or more variables

• Structural Testing - CFG

• Functional Testing - DFG

• Specifies the Domain Boundary

• n Predicates in Sequence  2n domains

• Or, just two domains A .AND. B .AND. C

10 ref boris beizer

• At least One PATH thru the Program

• More paths for disconnected domains

11 ref boris beizer

• For every domain, there is at least one path thru the

• The set of interpreted predicates traversed on that path

12 ref boris beizer

Boundary: Closed / Open

X >= MIN X <= MAX

MIN Closed Open MAX

13 ref boris beizer

14 ref boris beizer

• One dimension per variable

• At least one predicate

• N-spaces are cut by Hyperplanes

15 ref boris beizer

• Processing is OK. Domain definition may be wrong.

 Boundaries are wrong.

 Predicates are wrong.

• Once input vector is set on the right path, it’s correctly

• More bugs causing domain errors …

• Floating point zero check

• Ambiguous domains defined

18 ref boris beizer

19 ref boris beizer

20 ref boris beizer