L9 Design Verification

Pittsburgh, PA 15213-3890
Personal Software ProcessSM

for Engineers: Part II
Design Verification
This material is approved for public release. Distribution is limited by the

Software Engineering Institute to attendees.
Sponsored by the U.S. Department of Defense

© 2006 by Carnegie Mellon University
October 2006 PSP II - Design Verification - 1

Lecture Topics
This lecture provides a survey of several design
verification topics.
• reasons for design verification
• design verification methods
- symbolic execution
- execution tables
- case checking and trace tables
- correctness verification
Practice is required to consistently and effectively use

these verification techniques.
© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 2

Need for Design Verification -1
It is hard to eliminate many design defects just by making
process changes.
Checklists and a structured review process can improve

code-review yield.
Design reviews with checklists are also helpful, but not

sufficient.
A checklist item, “Is module logic correct?” cannot be

confirmed by scanning the specification templates.

Need for Design Verification -2
To improve design-review yield, you must use disciplined
verification methods.
An orderly approach to design verification is essential

because
• many common design defects are caused by
overlooked conditions
• situations that seem unlikely become more likely with
complex systems and high-powered computers
• conditions that were initially impossible may be likely
after a program is modified

Benefits of Design Verification
By following a structured design verification procedure,
you are more likely to
• see overlooked conditions
• identify rarely-exposed risks
• recognize possible future exposures
By recording information during each design review, you

can improve the effectiveness of later design inspections.

Using Design Verification
Design verification methods should be used during
• design
• design reviews
• design inspections
Verifying designs with source-code is time-consuming and

error-prone.
Use design verification methods to focus on the defect

types that cause you the most trouble in test.
Use your data to decide which verification methods are

most effective for you.

Symbolic Execution -1
With symbolic execution
• the program is represented symbolically
• program behavior is examined analytically
Though not always practical, symbolic execution can

produce a comprehensive verification.
The approach is to
• assign algebraic symbols to the program variables
• restate the program as one or more equations using
these symbols
• analyze the behavior of the equations

Symbolic Execution -2
Some questions to ask are
• Does the program converge on a result?
• Does the program behave properly with both normal
and abnormal input values?
• Does the program always produce the desired results?

Symbolic Execution Example -1
Examine the following program segment.
begin
X := X + Y;
Y := X – Y;
X := X – Y;
end
What would be the result for various values of X and Y ?
Note: This simple example merely illustrates how symbolic

execution can reveal non-obvious functionality.

Symbolic Execution Example -2
The program steps for this example are as follows.
# Instruction X Y
Initial values A B
1 X := X + Y A+B
2 Y := X - Y A
3 X := X - Y B
Final values B A
The result for inputs A and B is B and A.

Symbolic Execution: Assessment
Advantages: symbolic proofs
• can be general
• typically involve less work than other methods
• are sometimes the only practical way to do a
comprehensive verification
Disadvantages: symbolic execution

• is hard to use, except for algorithmic and substitution
problems
• proofs are manual and error-prone
• is difficult to use with complex logic

Execution Tables -1
An execution table is an orderly way to trace program
execution.
• It is a manual check of program flow.
• It starts with initial conditions.
• A set of variable values is selected.
• Each execution step is examined.
• Every change in variable values is entered.
• Program behavior is checked against the specification.

Execution Tables -2
The advantages of execution tables are that they
• are simple
• give reliable results
The disadvantages of execution tables are that they

• check only one case at a time
• are time-consuming
• are subject to human error

Execution Table Procedure
To use an execution table
1. identify the key program variables and enter them at
the top of the table
2. enter the principal program steps
3. determine and enter the initial conditions
4. trace the variable values through each program step
5. use additional execution-table copies for each cyclic
loop

Execution Table Exercise
In this exercise, use an execution table to verify the
correctness of the LogIn program.
Check a two-cycle scenario:

1. Valid ID & !Valid PW
2. ID timeout error & Valid PW.
Assume nMax = 3.
Pair up and take 20 minutes to do the verification. Blank

execution tables are in the notebook.
20 minutes

LogIn Execution Table -1
Cycle Function: LogIn
# Instruction Test ID PW Fail n
1 Initialize: n := 0; ID: = !Valid; PW := !Valid; Fail := false. F !Valid !Valid F 0
2 Repeat the main loop until a Valid ID and password or Fail.
3 Get user ID.
4 If no ID response in MaxTime, set Fail := true.
5 Check ID for validity. {CheckID state}
6 Get password.
7 If no password response in MaxTime, set Fail := true.
8 If ID Valid, check PW for validity. {CheckPW state}
9 If PW !Valid or ID !Valid, step the n counter.
10 If n exceeds nMax, set Fail := true.
11 Until ID and PW valid or Fail = true.
12 Otherwise, repeat the main loop
13 If Fail = true, cut off user, else, log in the user. {End state}

LogIn Execution Table Discussion
The program has an error.
Since the design does not require ID to be set to !Valid on

a new cycle, the implementation could result in an error.
In the second cycle, a Valid PW could end the test with ID

= Valid, PW = Valid, and Fail = true.
Depending on implementation, this design omission could

result in an incorrect login.

Cycle 1: Valid ID; !Valid PW Function: LogIn
1 Initialize: n := 0; ID: = !Valid; PW := !Valid; Fail := false. F !Valid !Valid F 0
2 Repeat the main loop until a Valid ID and PW or Fail.
3 Get user ID.
4 If no ID response in MaxTime, set Fail := true. F
5 Check ID for validity. {CheckID state} Valid
6 Get password
7 If no password response in MaxTime, set Fail := true. F
8 If ID Valid, check PW for validity. {CheckPW state} T
9 If PW !Valid or ID !Valid, step the n counter. T 1
10 If n exceeds nMax, set Fail := true. F
11 Until ID and PW valid or Fail = true. F
12 Otherwise, repeat the main loop T
13 If Fail = true, cut off user, else, log in the user. {End state}

Cycle 2: ID Timeout; Valid PW Function: LogIn
1 Initialize: n := 0; ID: = !Valid; PW := !Valid; Fail := false.
2 Repeat the main loop until a Valid ID and password or Fail. Valid !Valid F 1
3 Get user ID.
4 If no ID response in MaxTime, set Fail := true. T T
5 Check ID for validity. {CheckID state}
6 Get password
7 If no password response in MaxTime, set Fail := true. F
8 If ID Valid, check PW for validity. {CheckPW state} T Valid
9 If PW !Valid or ID !Valid, step the n counter. F
10 If n exceeds nMax, set Fail := true. F
11 Until ID and PW valid or Fail = true. T
12 Otherwise, repeat the main loop
13 If Fail = true, cut off user, else, log in the user. {End state} T/F

Case Checking
Case checking involves the following five steps.
1. Examine the program to determine its behavior
categories.
2. For these categories, identify the cases to check.
3. Check the cases to ensure that they cover all possible
situations.
4. Examine program behavior for each case.
5. Once every case has been verified, program behavior
has been verified.

Trace Tables
Trace tables are similar to execution tables, but more
general.
Trace tables examine general program behavior, rather

than verifying individual cases.
Trace tables use

• symbolic execution
• case checking

Trace Table Procedure
The trace table procedure is the same as that for the
execution table, plus three steps.
1. Identify the key program variables and enter them at
the top of the table.
2. Enter the principal program steps.
3. Identify symbolic execution possibilities.
4. Define all of the possible cases and select those
to check.
5. Determine and enter the initial conditions.
6. Trace the variable values through each program step.
7. Use additional trace-table copies for each cyclic loop.
8. For multi-cycle loops, group intermediate steps if
their results are obvious.

Trace Table Example
The ClearSpaces routine clears leading and trailing spaces
from an input string and sets the value of state.
ClearSpaces Input empty → (return empty) and State := 0

(Input; State) or
Input = 1 space → (return empty) and State := 1
or
Input >= 2 spaces → (return empty) and State := 2
or
Input has nonspace characters → (return trimmed
input) and State := 3
The following example omits the Length = 0 test and clears trailing
spaces. It is in the textbook, Table 12.5 (page 264).

Trace Table Cases -1
The symbolic execution parameters are as follows.
• string length: Length > 0
• leading spaces: L
• non-space characters: N
• trailing spaces: T
The possible cases are

• L or N or T = 1 while the others = 0
• L or N or T > 1 while the others = 0
• L or N or T = 0 while the others > 0
• L and N and T > 0

Trace Table Cases -2
A comprehensive test would check all cases.
It would also check for any limitations on the upper limit

values of L, N, and T.
A common trace-table strategy is to pick a specific set of

values for each case and then generalize the result.
The following example for trailing spaces shows the

approach with L and N and T > 0.

Trailing Space Example -1
Cycle 1: L = 1, N = 2, T = 1 ClearSpaces(var Input: string; State: int)
# Instructions Condition Input Length State
Length := length(Input)
1 ‘ AB ‘ 4 0
State := 0
2 repeat
3 if Input[Length] = ‘ ‘
4 Length := Length - 1
5 if State < 2 State:=State+1
6 else State := 3
until State=3 or Length=0

Trailing Space Example -2
In using a trace table, it is essential to determine precisely
what the computer would do for each step.
In this example, the value of Input[Length] is indeterminate

because the last character of the string is at Length - 1.
This is a defect.
After correcting this defect, the trace table is as shown in

the next slide.

Trace Table Example: Cycle 1

Length := length(Input)
1 ‘ AB ‘ 4 0
State := 0
2 repeat
3 if Input[Length-1] = ‘ ‘ true
4 Length := Length - 1 3
5 if State < 2 State:=State+1 true 1
6 else State := 3
7 until State=3 or Length=0 false

Trace Table Example: Cycle 2
1 Length := length(Input)
State := 0
2 repeat ‘ AB ‘ 3 1
3 if Input[Length-1] = ‘ ‘ false
6 else State := 3 3
7 until State=3 or Length=0 true

Trace Table Discussion
First, carefully check the initial conditions to ensure that
they are set by the program.
Second, double-check the trace table for errors and

omissions.
Then, after completing each case, check behavior for

different parameter values.
For example, the State := 3 step for the trace-table case

with T trailing spaces, is shown on the next slide.

Trace Table Example: Cycle T
Cycle T: L = 1, N = 2, T = T ClearSpaces(var Input: string; State: int)
1 Length := length(Input)
State := 0
2 repeat ‘ AB…’ 3 2
3 if Input[Length-1] = ‘ ‘ false
6 else State := 3 3
7 until State=3 or Length=0 true

Verifying Program Correctness
Correctness verification methods treat programs as
mathematical theorems.
These methods are particularly useful during design and

design review.
Because program verification often involves sophisticated

reasoning, it is important to check your work carefully.

Correctness Verification
To verify a program
• identify all of the program cases
• consider each verification question while reviewing
each non-trivial construct
• where the answer is not obvious, use a trace table to
evaluate the conditions that the program must satisfy
While this verification approach works with most design

constructs, only “while loops” are described in this lecture.
Proofs for repeat-until and for-loops are described in the

textbook (pages 271 - 277).

WhileLoop Verification -1
The WhileLoop is assumed to have the following form.
while WhileTest
begin
LoopPart
end

WhileLoop Verification -2
A while-loop is correct if all of the following questions
can be answered with “yes.”
• Question 1: Is loop termination guaranteed for any
argument of WhileTest?
• Question 2: When WhileTest is true, does WhileLoop
= LoopPart followed by WhileLoop?
• Question 3: When WhileTest is false, does
WhileLoop = identity?

WhileLoop Example -1
Assuming n > 0, calculate n!
f=1
i=1
while i != n {i not equal to n}
begin
i=i+1
f=f*i
end

Question 1: Is loop termination guaranteed for any
argument of WhileTest?
Yes, the loop terminates when i = n. Since i steps

inside the loop, it will terminate as long as n > 0.
Question 2: When WhileTest is true, is WhileLoop =

LoopPart followed by WhileLoop?
Yes, the LoopPart increments i and sets f equal to f * i,

so f is still equal to i!
Note: The initial conditions ensure that f = i! initially.

Question 3: When WhileTest is false, does WhileLoop =
identity?
Yes, when WhileTest is false, the loop terminates

without further changes to any variables, so i = n and
f = i!
Hence, we can deduce that f = n! as required.

Another WhileLoop Example -1
Given a value x and an array a[0 .. n-1] that is sorted in
ascending order, verify that the following binary search
algorithm finds the smallest index i such that x ≤ a[i].
i=0
j=n
while i != j
begin
k = (i + j) div 2
if a[k] < x
i=k
else
j=k
end

The informal specification is incomplete.
• What should happen if n = 0 (empty array)?
• What should happen if x > a[n-1]?
We can formalize the specification as:
Precondition  s {1 .. n-1} • a[s-1] ≤ a[s]
(array is sorted)
 k {0 .. i-1} • x>a[k] 
Postcondition  k {i .. n-1} • x ≤ a[k] 
(desired operation result) i≥0
i≤n
So
• if x > a[n-1] then the desired result is i = n
• if n = 0 then the desired result is i = 0

To use case checking, we should consider cases when
• x is equal to the first element of the array
• x is equal to any intermediate element of the array
• x is equal to the final element of the array
• x is not equal to any element
Here, we look at x is not equal to any element.
Let the array a = <2, 3, 5, 7> with n = 4 and x = 4.

The WhileLoop Trace Table
Cycle: 1 - 4: a = <2, 3, 5, 7>, n = 4, x = 4 Function: Binary Search

# Instruction T/F i j k i+j x a[k]
Initialize i, j 0 4 0 4
1 While i <> j T
2 k := (i + j) div 2 2 4 5
3 If a[k] < x, i := k F
4 Else j := k 2
5 While i <> j T
6 k := (i + j) div 2 1 2 3
7 If a[k] < x, i := k T 1
8 Else j := k
9 While i <> j T
10 k := (i + j) div 2 1 3 3
11 If a[k] < x, i := k T 1
12 Else j := k
13 While i <> j T
14 k := (i + j) div 2 1 3 3
15 If a[k] < x, i := k F 1
16 Else j := k

Verification Discussion
The algorithm fails to terminate for this set of inputs.
A trace table will reveal design defects, but it does not

suggest how to fix them.
The algorithm correction is to replace i = k with i = k + 1.
With this change, the next step would be to answer the

three WhileLoop questions, with a trace table if necessary.

Comments: Program Verification
If done correctly, these verification methods can guarantee
program correctness.
Mastery of these verification methods requires skill and

practice.
The loop termination test is critical because endless loops

are hard to identify with other methods.
While you can often answer the verification questions by

inspection, when in doubt, check the results with a trace
table.

UML and Verification -1
Graphical designs can be difficult to verify and often require
special care.
UML diagrams can and should be checked for consistency.
The names of all classes, operations, and attributes should

also be defined and used consistently.
UML state charts can be verified as shown before, but the

state and transition conditions must be explicit.

UML and Verification -2
Execution in UML sequence diagrams can be traced if the
diagrams are sufficiently detailed.
Similarly, every message sent between objects must be

supported by a traversable link defined in a class diagram.
Some UML tools can simulate execution of UML models,

but these tools should be used only after other review
techniques.

Messages to Remember
You will significantly improve your design-review yield by
using disciplined design review methods.
With complex programs, the time spent verifying designs

will be more than repaid by the testing time saved.
Practice using verification techniques and then select the

most effective ones for finding the defects that you most
commonly find in testing.

L9 Design Verification

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

L9 Design Verification

Uploaded by

Copyright:

Available Formats

Pittsburgh, PA 15213-3890

Personal Software ProcessSM

This material is approved for public release. Distribution is limited by the

Sponsored by the U.S. Department of Defense

October 2006 PSP II - Design Verification - 1

Practice is required to consistently and effectively use

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 2

Checklists and a structured review process can improve

Design reviews with checklists are also helpful, but not

A checklist item, “Is module logic correct?” cannot be

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 3

An orderly approach to design verification is essential

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 4

By recording information during each design review, you

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 5

Verifying designs with source-code is time-consuming and

Use design verification methods to focus on the defect

Use your data to decide which verification methods are

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 6

Though not always practical, symbolic execution can

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 7

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 8

What would be the result for various values of X and Y ?

Note: This simple example merely illustrates how symbolic

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 9

The result for inputs A and B is B and A.

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 10

Disadvantages: symbolic execution

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 11

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 12

The disadvantages of execution tables are that they

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 13

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 14

Check a two-cycle scenario:

Pair up and take 20 minutes to do the verification. Blank

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 15

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 16

Since the design does not require ID to be set to !Valid on

In the second cycle, a Valid PW could end the test with ID

Depending on implementation, this design omission could

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 17

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 18

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 19

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 20

Trace tables examine general program behavior, rather

Trace tables use

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 21

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 22

ClearSpaces Input empty → (return empty) and State := 0

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 23

The possible cases are

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 24

It would also check for any limitations on the upper limit

A common trace-table strategy is to pick a specific set of

The following example for trailing spaces shows the

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 25

5 if State < 2 State:=State+1

until State=3 or Length=0

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 26

In this example, the value of Input[Length] is indeterminate

After correcting this defect, the trace table is as shown in

© 2006 by Carnegie Mellon University October 2006 PSP II - Design Verification - 27

# Instructions Condition Input Length State