Graph Based Coverage

CS 5154
Graph Coverage Criteria
Owolabi Legunsen
The following are modified versions of the publicly-available slides for Chapter 7
in the Ammann and Offutt Book, “Introduction to Software Testing”
(http://www.cs.gmu.edu/~offutt/softwaretest)
Graph Coverage
Four Structures for
Modeling Software
Input Space Graphs Logic Syntax

Applied Applied to
to
Applied
Source FSMs
to
Specs DNF
Source Specs Source Models
Design Use cases Integ Input

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 2
Covering Graphs
• Graphs are the most commonly used structure for testing
• Graphs can come from many sources

– Control flow graphs
– Design structure
– FSMs and statecharts
– Use cases
• Tests usually are intended to “cover” the graph somehow

Why Graph Coverage?
• Some of the most widely-used coverage criteria
• The “R” in the RIPR model

– Graph coverage criteria help create tests that reach different
parts of software

The next two classes
• Today:
– Review of graph concepts
– Coverage criteria defined over generic graphs
• Next class (depending on progress today):

– Apply concepts learned in today’s class to source code
• Not in CS 5154
– Applying graph coverage criteria to design, specs, and use cases

Definition of a Graph
• A set N of nodes, N is not empty
• A set N0 of initial nodes, N0 is not empty
• A set Nf of final nodes, Nf is not empty
• A set E of edges, each edge from one node to another

– ( ni , nj ), i is predecessor, j is successor
N0 = { 1 }
Is this a Yes
Nf = { 1 }
graph? 1
E={}

Example Graphs
1 1 2 3 1
Not a
valid
graph
2 3 4 5 6 7 2 3
4 8 9 10 4
N0 = { 1 } N0 = { 1, 2, 3 } N0 = { }
Write down the Write down the Write down the
initialNand f = {final 4} N 8, final
f = {and
initial 9, 10 } initialNand 4}
f = {final
nodes,
E = {and (1,the2), nodes,
E = { (1,4), and the(2,5), (3,6),
(1,5), nodes,
E = and the
{ (1,2),
edges
(1,3), (2,4), (3, 7),edges
(4, 8), (5,8), (5,9), edges
(1,3), (2,4),
(3,4) }
Introduction to Software Testing, Edition 2 (Ch 07)
(6,2), (6,10), (7,10) (9,6) }
© Ammann & Offutt
(3,4) } 7
Paths in Graphs
• Path : A sequence, p, of nodes [n1, n2, …, nM] s.t. there is
an edge between each pair of nodes in p
• Length of a path : The number of edges in p
– A single node is a path of length 0
• Subpath : A subsequence of nodes in p is a subpath of p
1 2 3
A Few Paths
[ 1, 4, 8 ]
Write down
4 5 6 7 [three
2, 5,paths
9, 6, in
2]
this graph
[ 3, 7, 10 ]
8 9 10
Test Paths and SESE graphs
• Test Path : A path that starts at an initial node and ends
at a final node
• Test paths represent execution of test cases
– Some test paths can be executed by many tests
– Some test paths cannot be executed by any tests
• SESE graphs : All test paths start at a single node and end
at another node
– Single-entry, single-exit
– N0 and Nf have exactly one node
Double-diamond graph
2 5 Four test paths
[1, 2, 4, 5, 7]
1 4 7
[1, 2,down
Write 4, 6, all
7]
3 6 [1,test
the 3, 4, 5, 7]in
paths
[1,graph
this 3, 4, 6, 7]
Visiting and Touring
• Visit : A test path p visits node n if n is in p
A test path p visits edge e if e is in p
• Tour : A test path p tours subpath q if q is a subpath of p
Test path [ 1, 2, 4, 5, 7 ]
Visits nodes ? 1, 2, 4, 5, 7
Visits edges ? (1,2), (2,4), (4, 5), (5, 7)
Tours subpaths ? [1,2,4], [2,4,5], [4,5,7], [1,2,4,5],
[2,4,5,7], [1,2,4,5,7]
(Also, each edge is technically a subpath)

Tests and Test Paths
• path (t) : The test path executed by test t
• path (T) : The set of test paths executed by set of tests T
• Each test executes one and only one test path
– Complete execution from a start node to a final node

Tests and Test Paths (2)
• A location in a graph (node or edge) can be reached from

another location if there is a sequence of edges from the
first location to the second
– Syntactic reach : A subpath exists in the graph
– Semantic reach : A test exists that can execute that subpath

– This distinction (semantic vs syntactic) is important when
applied to source code

Tests and Test Paths (3)
test 1 many-to-one
Test
test 2
Path
test 3
Deterministic software: a test always execute same test path
many-to-many
test 1 Test Path 1
test 2 Test Path 2
test 3 Test Path 3

Non-deterministic software: a test can execute >1 test paths
Testing and Covering Graphs
• We use graphs in testing as follows :
– Develop a model of the software as a graph

– Require tests to visit/tour sets of nodes, edges or sub-paths

Testing and Covering Graphs (2)
• Test Requirements (TR) : Describe properties of test paths
• Test Criterion : Rules that define test requirements
• Satisfaction : Given a set TR of test requirements for a criterion C, a set

of tests T satisfies C on a graph if and only if for every test requirement tr
in TR, there is a test path in path(T) that meets the test requirement tr

Two kinds of graph coverage criteria
1. Structural Coverage Criteria : Defined on a graph just in

terms of nodes and edges
2. Data Flow Coverage Criteria : Requires a graph to be

annotated with references to variables

Node Coverage
• The first (and simplest) two criteria require that each

node and edge in a graph be executed
Node Coverage (NC) : Test set T satisfies node coverage

on graph G iff for every syntactically reachable node n in
N, there is some path p in path(T) such that p visits n.
• This statement is a bit cumbersome, so we abbreviate it in terms of

the set of test requirements
Node Coverage (NC) : TR contains each reachable node

in G.

Edge Coverage
• Edge coverage is slightly stronger than node coverage
Edge Coverage (EC) : TR contains each reachable path of

length up to 1, inclusive, in G.
• The phrase “length up to 1” allows for graphs with one

node and no edges

Node and Edge Coverage
• NC and EC are only different when there is an edge and
another subpath between a pair of nodes (as in an “if-
else” statement)
Node Coverage : ? TR = { 1, 2, 3 }
1 Test Path = [ 1, 2, 3 ]
2 Edge Coverage : ? TR = { (1, 2), (1, 3), (2, 3) }

Test Paths = [ 1, 2, 3 ]
3
[ 1, 3 ]

Paths of Length 1 and 0
• A graph with only one node will not have any edges
• It may seem trivial, but formally, Edge Coverage needs to

require Node Coverage on this graph
• Else, Edge Coverage will not subsume Node Coverage
– So, we define “length up to 1” instead of simply “length 1”
• We have the same issue with graphs that

1
only have one edge – for Edge-Pair
Coverage …
2

Covering Multiple Edges
• Edge-pair coverage requires pairs of edges, or subpaths of
length 2
Edge-Pair Coverage (EPC) : TR contains each reachable
path of length up to 2, inclusive, in G.
• The phrase “length up to 2” is used to include graphs that

have less than 2 edges
1 5
Edge-Pair Coverage : ?
2 4 TR = { [1,4,5], [1,4,6], [2,4,5],
[2,4,6], [3,4,5], [3,4,6] }
3 6
• A logical extension is to require covering all paths …

Covering Multiple Edges
Complete Path Coverage (CPC) :TR contains all paths in G.
Unfortunately, this is impossible if the graph has a loop, so a

weak compromise makes the tester decide which paths:
Specified Path Coverage (SPC) : TR contains a set S of

test paths, where S is supplied as a parameter.

Structural Coverage Example
Node Coverage
TR = { 1, 2, 3, 4, 5, 6, 7 }
Test Paths: [ 1, 2, 3, 4, 7 ] [ 1, 2, 3, 5, 6, 5, 7 ]
1
Edge Coverage
TR = { (1,2), (1, 3), (2, 3), (3, 4), (3, 5), (4, 7), (5, 6), (5, 7),
2 (6, 5) }
Test Paths: [ 1, 2, 3, 4, 7 ] [1, 3, 5, 6, 5, 7 ]
3
Edge-Pair Coverage
TR = { [1,2,3], [1,3,4], [1,3,5], [2,3,4], [2,3,5], [3,4,7],
4 5 [3,5,6], [3,5,7], [5,6,5], [6,5,6], [6,5,7] }
Test Paths: [ 1, 2, 3, 4, 7 ] [ 1, 2, 3, 5, 7 ] [ 1, 3, 4, 7 ]
[ 1, 3, 5, 6, 5, 6, 5, 7 ]
6
7
Complete Path Coverage
Test Paths: [ 1, 2, 3, 4, 7 ] [ 1, 2, 3, 5, 7 ] [ 1, 2, 3, 5, 6, 5, 7 ]
[ 1, 2, 3, 5, 6, 5, 6, 5, 7 ] [ 1, 2, 3, 5, 6, 5, 6, 5, 6, 5, 7 ] …
Handling Loops in Graphs
• If a graph contains a loop, it has an infinite number of paths
• Thus, Complete Path Coverage is not feasible
• SPC is not satisfactory because the results are subjective

and vary with the tester
• Attempts to “deal with” loops:

– 1970s : Execute cycles once ([5, 6, 5] in previous example, informal)
– 1980s : Execute each loop, exactly once (formalized)
– 1990s : Execute loops 0 times, once, more than once (informal description)
– 2000s : Prime paths (touring, sidetrips, and detours)

Simple Paths and Prime Paths
• Simple Path : A path from node ni to nj is simple if no node
appears more than once, except possibly the first and last
nodes are the same
– No internal loops
– A loop is a simple path
• Prime Path : A simple path that does not appear as a proper
subpath of any other simple path
Simple Paths : [1,2,4,1], [1,3,4,1], [2,4,1,2], [2,4,1,3],
1 [3,4,1,2], [3,4,1,3], [4,1,2,4], [4,1,3,4], [1,2,4], [1,3,4],
[2,4,1], [3,4,1], [4,1,2], [4,1,3], [1,2], [1,3], [2,4], [3,4],
2 3 [4,1], [1], [2], [3], [4]
4
Prime Paths : [2,4,1,2], [2,4,1,3], [1,3,4,1], [1,2,4,1],
[3,4,1,2], [4,1,3,4], [4,1,2,4], [3,4,1,3]

Prime Path Coverage
• A simple, elegant and finite criterion that requires loops
to be executed as well as skipped
Prime Path Coverage (PPC) : TR contains each prime path

in G.
• Will tour all paths of length 0, 1, …

• That is, it subsumes node and edge coverage
• PPC almost, but not quite, subsumes EPC …

PPC Does Not Subsume EPC
• If a node n has an edge to itself (self edge), EPC
requires [n, n, m] and [m, n, n]
• Neither [n, n, m] nor [m, n, n] are simple paths
(not prime)
EPC Requirements : ?
1 TR = { [1,2,3], [1,2,2], [2,2,3], [2,2,2] }
2 PPC Requirements : ?
TR = { [1,2,3], [2,2] }
3

Prime Path Example
• The previous example has 38 simple paths
• Only nine prime paths
1
Prime Paths
2 [1, 2, 3, 4, 7]
[1, 2, 3, 5, 7] Execute
[1, 2, 3, 5, 6] loop 0 times
3 [1, 3, 4, 7]
[1, 3, 5, 7] Execute
4 5 [1, 3, 5, 6] loop once
[6, 5, 7]
6 [6, 5, 6] Execute loop
7 [5, 6, 5] more than once

Touring, Sidetrips, and Detours
• Prime paths have no internal loops … test paths might
• Tour : A test path p tours subpath q if q is a subpath of p
• Tour With Sidetrips : A test path p tours subpath q with

sidetrips iff every edge in q is also in p in the same order
• Tour can have a sidetrip if it comes back to the same node
• Tour With Detours : A test path p tours subpath q with

detours iff every node in q is also in p in the same order
• Tour can have a detour from node ni, if it returns to the prime
path at a successor of ni
Sidetrips and Detours Example
1 2 3 4
1 2 3 5 6
Touring the prime path

[1, 2, 3, 5, 6] without 4
sidetrips or detours
1 2 5 6
1 2 3 5 6
3 4
Touring with a
sidetrip 4
1 2 5
1 2 3 5 6
3
Touring with a 4
4
detour

Dealing with Infeasible TRs
• An infeasible test requirement cannot be satisfied
– Unreachable statement (dead code)
– Subpath that can only be toured if a contradiction holds, e.g., if
(X > 0 and X < 0)
• Most test criteria have some infeasible test requirements
• It is usually undecidable whether all test requirements are

feasible

Infeasible TRs and Sidetrips
• When sidetrips are not allowed, many structural criteria

have more infeasible test requirements
• However, always allowing sidetrips weakens the test

criteria
Practical recommendation—Best Effort Touring

– First, satisfy as many test requirements as
possible without sidetrips
– Then, allow sidetrips to try to satisfy
remaining test requirements
Simple path & prime path example
“!” Means “cannot be extended to a simple path”
Simple Len 0 Len 1 Len 2 Len 3
paths [1] [1, 2] [1, 2, 3] [1, 2, 3, 4]
[2] [1, 3] [1, 3, 4] [1, 2, 3, 5]
1 [3] [2, 3] [1, 3, 5] [1, 3, 4, 7] !
[4] [3, 4] [2, 3, 4] [1, 3, 5, 7] !
2 [5] [3, 5] [2, 3, 5] [1, 3, 5, 6] !
[6] [4, 7] ! [3, 4, 7] ! [2, 3, 4, 7] !
[7] ! [5, 7] ! [3, 5, 7] ! [2, 3, 5, 6] !
3 [5, 6] [3, 5, 6] ! [2, 3, 5, 7] !
[6, 5] [5, 6, 5] *
4 5 [6, 5, 7] ! ‘*’ means path
[6, 5, 6] * cycles
6
7 Len 4
[1, 2, 3, 4, 7] !
[1, 2, 3, 5, 7] ! Prime Paths ?
[1, 2, 3, 5, 6] !

Round Trips
• Round-Trip Path : A prime path that starts and ends at the
same node
Simple Round Trip Coverage (SRTC) : TR contains at
least one round-trip path for each reachable node in G
that begins and ends a round-trip path.
Complete Round Trip Coverage (CRTC) : TR contains all

round-trip paths for each reachable node in G.
• The criteria omit nodes & edges that are not in round trips
• They do not subsume edge-pair, edge, or node coverage

Data Flow Criteria
Goal : Ensure that values are computed and used correctly
• Definition (def) : A location where a value for a variable is

stored into memory
• Use : A location where a variable’s value is accessed
Z = X*2 Defs: def (1) = { X }
X = 42
2 5 def (5) = { Z } Fill in
these
1 4 7 def (6) = { Z } sets
3 6 Uses: use (5) = { X }
Z = X-8 use (6) = { X }
The values given in defs should reach at least one, some, or

all possible uses
DU Pairs and DU Paths
• def (n) or def (e) : The set of variables that are defined by node n
or edge e
• use (n) or use (e) : The set of variables that are used by node n or
edge e
• DU pair : A pair of locations (li, lj) such that a variable v is

defined at li and used at lj
• Def-clear : A path from li to lj is def-clear with respect to variable

v if v is not given another value on any of the nodes or edges in
the path
• Reach : If there is a def-clear path from li to lj with respect to v,
the def of v at li reaches the use at lj
• du-path : A simple subpath that is def-clear with respect to v

from a def of v to a use of v
• du (ni, nj, v) – the set of du-paths from ni to nj
• du (ni, v) – the set of du-paths that start at ni
Touring DU-Paths
• A test path p du-tours subpath d with respect to v if p tours

d and d is def-clear with respect to v
• Sidetrips can be used, just as with previous touring
• Three criteria
– Use every def
– Get to every use
– Follow all du-paths

Data Flow Test Criteria
• First, we make sure every def reaches a use
All-defs coverage (ADC) : For each set of du-paths S = du

(n, v),TR contains at least one path d in S.
• Then we make sure that every def reaches all possible

uses
All-uses coverage (AUC) : For each set of du-paths to
uses S = du (ni, nj, v),TR contains at least one path d in S.
• Finally, we cover all the paths between defs and uses
All-du-paths coverage (ADUPC) : For each set S = du (ni,

nj, v),TR contains every path d in S.
Data Flow Testing Example
Z = X*2
2 5
X = 42
1 4 7
3 6
Z = X-8
All-defs for X All-uses for X All-du-paths for X

[ 1, 2, 4, 5 ] [ 1, 2, 4, 5 ] [ 1, 2, 4, 5 ]
[ 1, 2, 4, 6 ] [ 1, 3, 4, 5 ]
[ 1, 2, 4, 6 ]
[ 1, 3, 4, 6 ]

Graph-based criteria subsumption
Complete
Path Coverage
CPC
Prime Path
Coverage
PPC
All-DU-Paths
Coverage
ADUP Edge-Pair
Coverage
EPC
All-uses Complete Round
Coverage Trip Coverage
AUC Edge CRTC
Coverage
EC
All-defs Simple Round
Coverage Trip Coverage
ADC Node SRTC
Coverage
NC
Summary
• Graphs are a very powerful abstraction for designing tests
• The various criteria allow lots of cost / benefit tradeoffs

• These two sections are entirely at the “design abstraction
level” from chapter 2
• Graphs appear in many situations in software
– Next: we will apply these criteria to source code
– Design, specs, and use cases are not covered in CS 5154

Graph Based Coverage

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Graph Based Coverage

Uploaded by

Copyright:

Available Formats

CS 5154

Graph Coverage Criteria

Input Space Graphs Logic Syntax

Source Specs Source Models

Design Use cases Integ Input

• Graphs can come from many sources

• Tests usually are intended to “cover” the graph somehow

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 3

• The “R” in the RIPR model

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 4

• Next class (depending on progress today):

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 5

• A set N0 of initial nodes, N0 is not empty

• A set Nf of final nodes, Nf is not empty

• A set E of edges, each edge from one node to another

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 6

(Also, each edge is technically a subpath)

• path (t) : The test path executed by test t

• path (T) : The set of test paths executed by set of tests T

• Each test executes one and only one test path

– Complete execution from a start node to a final node

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 11

• A location in a graph (node or edge) can be reached from

– Semantic reach : A test exists that can execute that subpath

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 12

test 2 Test Path 2

test 3 Test Path 3

– Develop a model of the software as a graph

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 14

• Test Criterion : Rules that define test requirements

• Satisfaction : Given a set TR of test requirements for a criterion C, a set

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 15

1. Structural Coverage Criteria : Defined on a graph just in

2. Data Flow Coverage Criteria : Requires a graph to be

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 16

• The first (and simplest) two criteria require that each

Node Coverage (NC) : Test set T satisfies node coverage

• This statement is a bit cumbersome, so we abbreviate it in terms of

Node Coverage (NC) : TR contains each reachable node

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 17

Edge Coverage (EC) : TR contains each reachable path of

• The phrase “length up to 1” allows for graphs with one

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 18

2 Edge Coverage : ? TR = { (1, 2), (1, 3), (2, 3) }

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 19

• It may seem trivial, but formally, Edge Coverage needs to

• We have the same issue with graphs that

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 20

• The phrase “length up to 2” is used to include graphs that

• A logical extension is to require covering all paths …

Complete Path Coverage (CPC) :TR contains all paths in G.

Unfortunately, this is impossible if the graph has a loop, so a

Specified Path Coverage (SPC) : TR contains a set S of

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 22

• Thus, Complete Path Coverage is not feasible

• SPC is not satisfactory because the results are subjective

• Attempts to “deal with” loops:

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 25

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 27

Prime Path Coverage (PPC) : TR contains each prime path

• Will tour all paths of length 0, 1, …

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 28

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 29