You are on page 1of 40

# Welcome to CS 136 (Winter 2011)

Huma Zafar

## Instructional Assistants (IAs): John Doucette, David Loker,

Caelyn McAulay, Alejandro Salinger

## Web page (main information source):

http://www.student.cs.uwaterloo.ca/˜cs136/
Lectures: 5 sections, each Tuesday and Thursday

## CS 136 Winter 2011 01: Introduction to CS 136 1

Textbooks: “How to Design Programs” (HtDP) by Felleisen, Flatt,
Findler, Krishnamurthi (http://www.htdp.org) and “C
Programming: A Modern Approach” (CP:AMA) by K. N. King.

## Marking Scheme: 25% assignments (weekly), 30% midterm, 45%

final exam + bonus

## ⇒ Your assignment average and weighted exam average must

both be greater than 50%

Where to begin?

An Idea

Implementation

Space
(Racket, C, ...)
Complexity

## CS 136 Winter 2011 01: Introduction to CS 136 3

An Idea
A collection of items such that:
• We can add an item to the collection
• We keep
track of the order in which items were added 4

## is the only one that can be seen or removed

18

• We can see
if there is at least one item in the collection

## CS 136 Winter 2011 01: Introduction to CS 136 4

This idea can be precisely described as an Abstract Data Type

## • The type of data that is stored

• A set of operations that can be performed on the data
This specification does not depend on any specific programming
language

## CS 136 Winter 2011 01: Introduction to CS 136 5

Our idea is called a Stack

## The type of data is described as a sequence of items

[a1 , a2 , . . . , an ] such that n ≥ 0

## CS 136 Winter 2011 01: Introduction to CS 136 6

We will allow the following five operations:

## • Make an empty stack (make-stack)

• Add an item to the top of a stack (push)
• Remove an item from the top of a stack (pop)
• Look at the item at the top of a stack (peek)
• Check if the stack is empty (stack-empty?)
However, these descriptions are not precise

## CS 136 Winter 2011 01: Introduction to CS 136 7

Specification of Operations
For each ADT operation, we must specify:

## • The parameters (if any)

• Preconditions: What must be true before performing this
operation?

## • Postconditions: Assuming the preconditions have been satisfied,

what will be true after the operation has been performed?

satisfied?

## CS 136 Winter 2011 01: Introduction to CS 136 8

Specification of Stack Operations
make-stack

• Parameters: None
• Pre: True (i.e. there are no preconditions)
• Post: The empty stack S = [] is produced
push

## • Parameters: An item x and a stack S = [a1 , . . . , an ], n ≥ 0

• Pre: True
• Post: The stack S 0 = [x, a1 , . . . , an ] is produced

## CS 136 Winter 2011 01: Introduction to CS 136 9

Implementation: Code to make each operation work as intended

## • The choice of programming language

• How data is organized and stored within the language

;; stack1.ss
#lang racket

## ;; specify the operations that are ’visible’ to the user

(provide make-stack push pop peek stack-empty?)

## (define make-stack (lambda () empty))

(define (push item S) (cons item S))
(define pop rest)
(define peek first)
(define stack-empty? empty?)

## CS 136 Winter 2011 01: Introduction to CS 136 11

Using this Implementation
;; stack-test.ss
#lang racket

## ;; specify the implementation we are using

(require "stack1.ss")

## CS 136 Winter 2011 01: Introduction to CS 136 12

What can go wrong here?
The implementation in stack1.ss works

## However, it is possible to violate the requirements that items can

only be added, removed, or accessed at the top of the stack:

## CS 136 Winter 2011 01: Introduction to CS 136 13

A Better Implementation
;; stack2.ss
#lang racket
(provide make-stack push pop peek stack-empty?)
;; ’hide’ the data using a structure
(define-struct stk (lst))
(define (make-stack) (make-stk empty))
(define (push v s) (make-stk (cons v (stk-lst s))))
(define (pop s) (make-stk (rest (stk-lst s))))
(define (peek s) (first (stk-lst s)))
(define (stack-empty? s) (empty? (stk-lst s)))

## CS 136 Winter 2011 01: Introduction to CS 136 14

Introduction to Algorithm Analysis
(define (len lst)
(cond
[(empty? lst) 0]
[else (+1 (len (rest lst)))]))

Let T (n) be the running time of len when applied to a list of length
n (measured as the number of substitution steps required)

## CS 136 Winter 2011 01: Introduction to CS 136 15

A condensed trace
Consider the case where n =4
(len ’(p q r s))
⇒ . . . ;; simplification of cond
⇒ (+ 1 (len ’(q r s)))
⇒ . . . ;; T(3) steps
⇒ (+ 1 3)
⇒4

steps

## CS 136 Winter 2011 01: Introduction to CS 136 16

We can give the value of T (n) as a recurrence relation

 a n=0
T (n) =
 b + T (n − 1) n > 0

## Here, a and b are constants

Solution: T (n) = a + bn

## CS 136 Winter 2011 01: Introduction to CS 136 17

Another Example
(define (sum-of-divisors n)
(foldr + 0
(filter (lambda (y) (zero? (remainder n y)))
(build-list n (lambda (x) (add1 x))))))

A quicker analysis:

## CS 136 Winter 2011 01: Introduction to CS 136 18

Big-oh Notation
A convenient way to classify the “size” of a function. Common
classifications include:

• O(1) (constant)
• O(log n) (logarithmic)
• O(n) (linear)
• O(n log n) (loglinear)
• O(2n ) (exponential)

## CS 136 Winter 2011 01: Introduction to CS 136 19

Informally, use the leading term and ignore coefficients

## len has running time T (n) = a + bn ∈ O(n)

sum-of-divisors has running time
T (n) = an + bn + cn = (a + b + c)n ∈ O(n)
O(n) is called linear running time

## CS 136 Winter 2011 01: Introduction to CS 136 20

Formal Definition of Big-oh
f (n) is O(g(n)) if there exist positive constants c and
Definition:
n0 such that f (n) ≤ c · g(n) for all n ≥ n0
“c · g(n) eventually dominates f (n)”

## CS 136 Winter 2011 01: Introduction to CS 136 21

Example: f (n) = 3n2 + 2 and g(n) = n2

2
c g(n) = 5n

f(n) = 3n2 + 2

n0 = 1 n

## CS 136 Winter 2011 01: Introduction to CS 136 22

Big-oh Proofs
• 3n3 + 4n2 − 3n + 10 is O(n3 )
• 3n3 + 4n2 − 3n + 10 is O(n9 )
• 2n2 + 1 is not O(n)
• If f (n) ∈ O(g(n)) and g(n) ∈ O(h(n)) then
f (n) ∈ O(h(n))

## CS 136 Winter 2011 01: Introduction to CS 136 23

Back to Algorithm Analysis
From now on, we will use big-oh notation to describe the running
times of functions (don’t worry about ”exact” running times)

## Running times of some common built-in functions:

O(1) (Constant running time): cons, first, rest, empty?, and, or, not,
numeric compuations on bounded numbers, operations on
structures, operations on symbols

comparisons

## CS 136 Winter 2011 01: Introduction to CS 136 24

Addition Rule: If function A takes O(f (n)) time and function B
takes O(g(n)) time, then executing A followed by B takes
O(f (n) + g(n)) time.
For example, build-list, filter, and foldr each take O(n) time

## Therefore sum-of-divisors takes

O(n + n + n) = O(3n) = O(n) time

## CS 136 Winter 2011 01: Introduction to CS 136 25

Multiplication Rule: If function A takes O(f (n)) time, then
executing A O(g(n)) times takes O(f (n) · g(n)) time in total.

## (define (reverse-search lst target)

(cond
[(empty? lst) false]
[(equal? (first lst) target) true]
[else (reverse-search (reverse (rest lst)) target)]))

## CS 136 Winter 2011 01: Introduction to CS 136 26

Best Case vs. Worst Case
But what if target is the first item in lst?

Best Case: For an input of size n, what contents of the input will
give the function its quickest running time?

## • The fewest number of steps

• reverse-search has a best case running time of O(1) when (first
lst) equals target

It is incorrect to say that “the best case running time for a function is
O(1) when n=0” (running times must hold for all n)

## CS 136 Winter 2011 01: Introduction to CS 136 27

Worst Case: For an input of size n, what contents of the input will
give the function its slowest running time?

## • The largest number of steps

• reverse-search has a worst case running time of O(n2 ) when
target is not in lst

Some functions have the same best and worst case running times;
here the contents of the input do not matter

## CS 136 Winter 2011 01: Introduction to CS 136 28

The Tabular Method of Analysis
;; The Insertion Sort algorithm
;; isort: (listof num) → (listof num)
(define (isort alon)
(cond
[(empty? alon) empty]
[else (insert (first alon)
(isort (rest alon)))]))

## CS 136 Winter 2011 01: Introduction to CS 136 29

;; insert: num (listof num) → (listof num)
(define (insert n alon)
(cond
[(empty? alon) (cons n empty)]
[(< n (first alon)) (cons n alon)]
[else (cons (first alon)
(insert n (rest alon)))])))

## CS 136 Winter 2011 01: Introduction to CS 136 30

Example with Local
;; Finding the maximum of a list
;; list-max: (listof num) → num
(define (list-max alon)
(cond
[(empty? (rest alon)) (first alon)]
[else
(local [(define max-rest (list-max (rest alon)))]
(cond
[(> (first alon) max-rest) (first alon)]
[else max-rest]))]))

## CS 136 Winter 2011 01: Introduction to CS 136 31

Example using Recursion Depth
;; Compute the binary representation of a natural number
;; binary-digit-list: nat → (listof nat[≥1])
(define (binary-digit-list n)
(cond
[(zero? n) empty]
[(even? n) (cons 0 (binary-digit-list (quotient n 2)))]
[else (cons 1 (binary-digit-list (quotient n 2))]))

## CS 136 Winter 2011 01: Introduction to CS 136 32

True or False?
• Analysis of an algorithm’s running time depends on the
computer used to run the algorithm

## • If f (n) is O(n2 ), then it is also O(n3 ) and O(2n )

• If Algorithm A has O(n) running time and Algorithm B has
O(n2 ) running time, then A will always perform faster than B
• f (n) = loga n is O(logb n) where a and b are two different
positive constants

## • The best case running time of any function is O(1) which

occurs when n = 0

## CS 136 Winter 2011 01: Introduction to CS 136 33

Order Notation
An alternative definition of Big-O (from a document on the Handouts
web page) is:

## Definition 2: O(g(n)) is the set of functions f (n) such that

there exist constants c and n0 such that for all n ≥ n0 ,
f (n) ≤ c · g(n).
The growth rates of sample members of these sets is shown on the
next slide.

## CS 136 Winter 2011 01: Introduction to CS 136 34

18000

16000 O(log2 n)
O(n)
O(n log2 n)
14000
O(n2)
O(n3)
12000

10000

8000

6000

4000

2000

0
50
1

0
10

15

20

25

30

35

40

45

50

55

60

65
CS 136 Winter 2011 01: Introduction to CS 136 35
Does it really work?
Jon Bentley describes an experiment in Programming Pearls, p. 75.
The problem is to take a list of N real numbers and return the
maximum sum found in any contiguous sublist. For example:

## He describes four algorithms to solve the problem. They are O(n3 ),

O(n2 ), O(n lg n), and O(n). To prove that constant factors don’t
matter much, he deliberately tried to make the constant factors of
the O(n3 ) and O(n) algorithms differ by as much as possible.

## CS 136 Winter 2011 01: Introduction to CS 136 36

O(n) on a TRS-80 hobbyist O(n3 ) on a Cray supercom-
computer puter

## CS 136 Winter 2011 01: Introduction to CS 136 37

O(n3 ) algorithm: Cray-1, finely-tuned Fortran, 3.0n3 nanoseconds
O(n) algorithm: TRS-80, interpreted Basic, 19.5n milliseconds =
19, 500, 000n nanoseconds
n O(n) O(n3 )
10 .2 secs .000003 secs

100

1,000

10,000

100,000

1,000,000

## CS 136 Winter 2011 01: Introduction to CS 136 38

O(n3 ) algorithm: Cray-1, finely-tuned Fortran, 3.0n3 nanoseconds
O(n) algorithm: TRS-80, interpreted Basic, 19.5n milliseconds =
19, 500, 000n nanoseconds
n O(n) O(n3 )
10 .2 secs .000003 secs

1,000 20 secs

10,000

100,000

1,000,000

## CS 136 Winter 2011 01: Introduction to CS 136 39

Goals of this module
You should understand the idea of an ADT, the examples given, and
how we can provide several implementations of an ADT. We will be
using ADTs in lecture and on assignments.

Scheme module.

## You should strengthen your intuition regarding the use of O -notation

in describing the running time of functions.

## We will be asking you to aim for certain running times on

assignments and to provide analyses of your own code.