Loop Invariants and Binary Search

Chapter 4.3.3 and 9.3.1

CSE 2011
-1- Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Outline
Iterative Algorithms, Assertions and Proofs of Correctness
Binary Search: A Case Study

CSE 2011
-2- Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Outline
Iterative Algorithms, Assertions and Proofs of Correctness
Binary Search: A Case Study

CSE 2011
-3- Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Assertions
An assertion is a statement about the
state of the data at a specified point in
your algorithm.
An assertion is not a task for the algorithm
to perform.
You may think of it as a comment that is
added for the benefit of the reader.

CSE 2011
-4- Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Loop Invariants
Binary search can be implemented as an iterative
algorithm (it could also be done recursively).
Loop Invariant: An assertion about the current state
useful for designing, analyzing and proving the
correctness of iterative algorithms.

CSE 2011
-5- Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Other Examples of Assertions

Preconditions: Any assumptions that must be true

about the input instance.
Postconditions: The statement of what must be true
when the algorithm/program returns.
Exit condition: The statement of what must be true to
exit a loop.

CSE 2011
-6- Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Iterative Algorithms

Take one step at a time

towards the final destination

loop (done)
take step
end loop

CSE 2011
-7- Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Establishing Loop Invariant

From the Pre-Conditions on the input instance

we must establish the loop invariant.

CSE 2011
-8- Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Maintain Loop Invariant

Suppose that
qWe start in a safe location (pre-condition)
qIf we are in a safe location, we always step
to another safe location (loop invariant)

Can we be assured that the

computation will always be in a safe
location?
By what principle?

CSE 2011
-9- Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Maintain Loop Invariant
By Induction the computation will
always be in a safe location.

S(0)


i ,S(i )

i ,S(i ) S(i + 1)

CSE 2011
- 10 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Ending The Algorithm
Define Exit Condition Exit

Termination: With sufficient progress,

0 km Exit
the exit condition will be met.

When we exit, we know

q exit condition is true
q loop invariant is true
from these we must establish Exit

the post conditions.

CSE 2011
- 11 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Definition of Correctness
<PreCond> & <code> <PostCond>

If the input meets the preconditions,

then the output must meet the postconditions.

If the input does not meet the preconditions, then

nothing is required.

CSE 2011
- 12 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Outline
Iterative Algorithms, Assertions and Proofs of Correctness
Binary Search: A Case Study

CSE 2011
- 13 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Define Problem: Binary Search
PreConditions
qKey 25
qSorted List

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

PostConditions
qFind key in list (if there).

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

CSE 2011
- 14 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Define Loop Invariant

Maintain a sublist.
If the key is contained in the original list, then the key is
contained in the sublist.

key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

CSE 2011
- 15 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Define Step
Cut sublist in half.
Determine which half the key would be in.
Keep that half.

mid
key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

If key mid, If key > mid,

then key is in then key is in
left half. right half.
CSE 2011
- 16 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Define Step
It is faster not to check if the middle element is the key.
Simply continue.

key 43

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

If key mid, If key > mid,

then key is in then key is in
left half. right half.
CSE 2011
- 17 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Make Progress

The size of the list becomes smaller.

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

79 km

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

75 km

CSE 2011
- 18 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Exit Condition

key 25
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

0 km

If the key is contained in the

original list, If element = key,
then the key is contained in the
return associated
sublist. entry.
Sublist contains one element. Otherwise return
false.
Exit
CSE 2011
- 19 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Running Time

The sublist is of size n, n/2, n/4, n/8,,1

Each step O(1) time.
Total = O(log n)

key 25
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

If key mid, If key > mid,

then key is in then key is in
left half. right half.
CSE 2011
- 20 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Running Time
Binary search can interact poorly with the memory
hierarchy (i.e. caching), because of its random-access
nature.
It is common to abandon binary searching for linear
searching as soon as the size of the remaining span falls
below a small value such as 8 or 16 or even more in
recent computers.

CSE 2011
- 21 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
END OF LECTURE
FEB 13, 2014
CSE 2011
- 22 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 BinarySearch(A[1..n], key )
<precondition>: A[1..n] is sorted in non-decreasing order
<postcondition>: If key is in A[1..n], algorithm returns its location
p = 1, q = n
while q > p
< loop-invariant>: If key is in A[1..n], then key is in A[p..q]
p +q
mid =
2
if key A[mid ]
q = mid
else
p = mid + 1
end
end
if key = A[ p ]
return(p )
else
return("Key not in list")
end

CSE 2011
- 23 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Simple, right?
Although the concept is simple, binary search is
notoriously easy to get wrong.
Why is this?

CSE 2011
- 24 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Boundary Conditions
The basic idea behind binary search is easy to grasp.
It is then easy to write pseudocode that works for a
typical case.
Unfortunately, it is equally easy to write pseudocode that
fails on the boundary conditions.

CSE 2011
- 25 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Boundary Conditions

if key A[mid ] if key < A[mid ]

q = mid q = mid
else else
p = mid + 1
or p = mid + 1
end end

What condition will break the loop invariant?

CSE 2011
- 26 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Boundary Conditions

mid
key 36
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

Code: key A[mid] select right half

Bug!!
CSE 2011
- 27 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Boundary Conditions

if key A[mid ] if key <A[mid ] if key < A[mid ]

q = mid q = mid 1 q = mid
else else else
p = mid + 1 p = mid p = mid + 1
end end end

OK OK Not OK!!

CSE 2011
- 28 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Boundary Conditions

p+q p+q
mid = or mid =
2 2

key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

p +q
Shouldnt matter, right? Select mid =
2
CSE 2011
- 29 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Boundary Conditions

p +q
Select mid =
2
mid
key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

If key mid, If key > mid,

then key is in then key is in
left half. right half.
CSE 2011
- 30 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Boundary Conditions

p +q
Select mid =
mid 2
key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

If key mid, If key > mid,

then key is in then key is in
left half. right half.
CSE 2011
- 31 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Boundary Conditions

No progress
toward goal:
Loops Forever!
Another bug! p +q
Select mid =
mid 2
key 25

3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95

If key mid, If key > mid,

then key is in then key is in
left half. right half.
CSE 2011
- 32 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Boundary Conditions

p +q p +q p +q
mid = mid = mid =
2 2 2
if key A[mid ] if key <A[mid ] if key A[mid ]
q = mid q = mid 1 q = mid
else else else
p = mid + 1 p = mid p = mid + 1
end end end

OK OK Not OK!!

CSE 2011
- 33 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Getting it Right
How many
possible p +q
algorithms? or mid = ?
p +q 2
mid =
How many 2
correct if key A[mid ] or if key <A[mid ] ?
algorithms? q = mid
Probability of else
guessing p = mid + 1
correctly? or q = mid 1
end else
p = mid
end

CSE 2011
- 34 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
Alternative Algorithm: Less Efficient but More Clear
BinarySearch(A[1..n], key )
<precondition>: A[1..n] is sorted in non-decreasing order
<postcondition>: If key is in A[1..n], algorithm returns its location
p = 1, q = n
while q p
< loop-invariant>: If key is in A[1..n], then key is in A[p..q]
p +q
mid =
2
if key <A[mid]
q = mid 1
else if key > A[mid]
p = mid + 1
else
return(mid)
Still (log n ), but with slightly larger constant.
end
end
return("Key not in list")

CSE 2011
- 35 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Card Trick
A volunteer, please.

CSE 2011
- 36 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Pick a Card

Done
Thanks to J. Edmonds for this example.
CSE 2011
- 37 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Loop Invariant:
The selected card is one
of these.

CSE 2011
- 38 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Which
column?

left

CSE 2011
- 39 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Loop Invariant:
The selected card is one
of these.

CSE 2011
- 40 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Selected column is placed
in the middle

CSE 2011
- 41 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 I will rearrange the cards

CSE 2011
- 42 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Relax Loop Invariant:
I will remember the same
about each column.

CSE 2011
- 43 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Which
column?

right

CSE 2011
- 44 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Loop Invariant:
The selected card is one
of these.

CSE 2011
- 45 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Selected column is placed
in the middle

CSE 2011
- 46 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 I will rearrange the cards

CSE 2011
- 47 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Which
column?

left

CSE 2011
- 48 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Loop Invariant:
The selected card is one
of these.

CSE 2011
- 49 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Selected column is placed
in the middle

CSE 2011
- 50 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Here is your
card.

Wow!

CSE 2011
- 51 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Ternary Search

Loop Invariant: selected card in central subset of

cards

Size of subset = n / 3i 1
where
n = total number of cards
i = iteration index

How many iterations are required to guarantee success?

CSE 2011
- 52 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder
 Learning Outcomes
From this lecture, you should be able to:
qUse the loop invariant method to think about iterative algorithms.
qProve that the loop invariant is established.
qProve that the loop invariant is maintained in the typical case.
qProve that the loop invariant is maintained at all boundary
conditions.
qProve that progress is made in the typical case
qProve that progress is guaranteed even near termination, so that
the exit condition is always reached.
qProve that the loop invariant, when combined with the exit
condition, produces the post-condition.
qTrade off efficiency for clear, correct code.
CSE 2011
- 53 - Last Updated: 2014-02-25 9:59 AM
Prof. J. Elder