CS5302

Data Structures and

Algorithms
Lecturer: Lusheng Wang
Office: Y6416
Phone: 2788 9820
E-mail lwang@cs.cityu.edu.hk

Welcome to ask questions at ANY time.

Java Source code:

http://net3.datastructures.net/download.html
The course Website:
http://www.cs.cityu.edu.hk/~lwang/cs5302.html
Text Book: Michael T. Goodrich and Roberto Tamassia, Data
Structurea and Algorithms in Java, John Wiley & Sons, Inc.

Analysis of Algorithm 1
 What We Cover
Analysis of Algorithms: worst case time and space
complexity
Data Structures: stack,queue, linked list, tree,
priority queue, heap, and hash;
Searching algorithms: binary and AVL search trees;
Sorting algorithms: merge sort, quick sort, bucket
sort and radix sort; (Reduce some contents)
Graph: data structure, depth first search and
breadth first search. (add some interesting
contents).

Analysis of Algorithm 2
 Why This Course?
You will be able to evaluate the quality of a
program (Analysis of Algorithms: Running
time and memory space )
You will be able to write fast programs
You will be able to solve new problems
You will be able to give non-trivial methods
to solve problems.
(Your algorithm (program) will be faster than
others.)

Analysis of Algorithm 3
 Course Evaluations
Course work: 40%
Final Exam: 60%
Course Work:
Three assignments

Analysis of Algorithm 4
Data Structures: A systematic way of
organizing and accessing data.
--No single data structure works well for ALL purposes.

Input Algorithm Output

An algorithm is a step-by-step
procedure for
solving a problem in a finite amount of
 Algorithm Descriptions
Nature languages: Chinese, English, etc.
Pseudo-code: codes very close to computer
languages, e.g., C programming language.
Programs: C programs, C++ programs, Java programs.

Goal:
Allow a well-trained programmer to be able to
implement.
Allow an expert to be able to analyze the running
time.

Analysis of Algorithm 6
 An Example of an
Algorithm
Algorithm sorting(X, n)
Input array X of n integers
Output array X sorted in a non-decreasing order

for i 0 to n 1 do
for j i+1 to n do
if (X[i]>X[j]) then
{ s=X[i];
X[i]=X[j];
X[j]=s;
}
return X

Analysis of Algorithm 7
 Analysis of Algorithms
Estimate the running time
Estimate the memory space
required.

Depends on the input size.

Analysis of Algorithm 8
Running Time (3.1)
Most algorithms transform best case
average case
input objects into output worst case
objects. 120

The running time of an 100

algorithm typically grows

Running Time
80
with the input size.
60
Average case time is often
difficult to determine. 40

We focus on the worst 20

case running time. 0

Easier to analyze 1000 2000 3000 4000
Input Size
Crucial to applications such
as games, finance and
robotics
Analysis of Algorithm 9
Experimental Studies
9000
Write a program
implementing the 8000

algorithm 7000

Run the program with 6000

Time (ms)
inputs of varying size 5000
and composition 4000
Use a method like 3000
System.currentTimeMillis() to 2000
get an accurate
1000
measure of the actual
running time 0
0 50 100
Plot the results Input Size

Analysis of Algorithm 10
Limitations of Experiments
It is necessary to implement the
algorithm, which may be difficult
Results may not be indicative of the
running time on other inputs not
included in the experiment.
In order to compare two algorithms,
the same hardware and software
environments must be used

Analysis of Algorithm 11
Theoretical Analysis
Uses a high-level description of the
algorithm instead of an implementation
Characterizes running time as a
function of the input size, n.
Takes into account all possible inputs
Allows us to evaluate the speed of an
algorithm independent of the
hardware/software environment

Analysis of Algorithm 12
Pseudocode (3.2)
High-level description Example: find
of an algorithm max element of
an array
More structured than Algorithm arrayMax(A, n)
English prose Input array A of n integers
Less detailed than a Output maximum element of A
program
Preferred notation for currentMax A[0]
describing algorithms for i 1 to n 1 do
Hides program if A[i] currentMax then
design issues currentMax A[i]
return currentMax

Analysis of Algorithm 13
Pseudocode Details
Control flow Expressions
if then [else ] Assignment
while do (like in Java)
Equality testing
repeat until
for do (like in Java)
Indentation replaces n2 Superscripts and
braces other mathematical
formatting allowed
Method declaration
Algorithm method (arg [, arg])
Input
Output

Analysis of Algorithm 14
Primitive Operations
Basic computations
performed by an algorithm Examples:
Evaluating an
Identifiable in pseudocode expression
Largely independent from Assigning a
the programming language value to a
Exact definition not variable
important (we will see why
Indexing into an
array
later)
Calling a method
Assumed to take a constant Returning from a
amount of time in the RAM method
model

Analysis of Algorithm 15
Counting Primitive
Operations (3.4)
By inspecting the pseudocode, we can determine the
maximum number of primitive operations executed by an
algorithm, as a function of the input size

Algorithm arrayMax(A, n)
currentMax A[0] 2
for (i =1; i<n; i++) 2n
(i=1 once, i<n n times, i++ (n-1) times)
if A[i] currentMax then 2(n 1)
currentMax A[i] 2(n 1)
return currentMax 1
Total 6n
Analysis of Algorithm 16
Estimating Running Time
Algorithm arrayMax executes 6n 1 primitive
operations in the worst case.
Define:
a = Time taken by the fastest primitive operation
b = Time taken by the slowest primitive operation
Let T(n) be worst-case time of arrayMax.
Then
a (8n 2) T(n) b(8n 2)
Hence, the running time T(n) is bounded by
two linear functions
Analysis of Algorithm 17
Growth Rate of Running
Time
Changing the hardware/ software
environment
Affects T(n) by a constant factor, but
Does not alter the growth rate of T(n)
The linear growth rate of the
running time T(n) is an intrinsic
property of algorithm arrayMax

Analysis of Algorithm 18
 n logn n nlogn n2 n3 2n

4 2 4 8 16 64 16
8 3 8 24 64 512 256

16 4 16 64 256 4,096 65,536

32 5 32 160 1,024 32,768 4,294,967,296

64 6 64 384 4,094 262,144 1.84 * 1019

128 7 128 896 16,384 2,097,152 3.40 * 1038

256 8 256 2,048 65,536 16,777,216 1.15 * 1077

512 9 512 4,608 262,144 134,217,728 1.34 * 10154

1024 10 1,024 10,240 1,048,576 1,073,741,824 1.79 * 10308

The Growth Rate of the Six Popular functions

Analysis of Algorithm 19
Big-Oh Notation
To simplify the running time
estimation,
for a function f(n), we ignore the
constants and lower order terms.

Example: 10n3+4n2-4n+5 is O(n3).

Analysis of Algorithm 20
Big-Oh Notation (Formal
Definition)
10,000
Given functions f(n) 3n
and g(n), we say that 2n+10
1,000
f(n) is O(g(n)) if there
are positive constants n

c and n0 such that 100

f(n) cg(n) for n n0

10
Example: 2n 10 is O(n)
2n 10 cn
(c 2) n 10 1
1 10 100 1,000
n 10(c 2)
n
Pick c 3 and n0 10

Analysis of Algorithm 21
Big-Oh Example
1,000,000
n^2
Example: the
100n
function n2 is not 100,000
10n
O(n) 10,000 n
n2 cn
nc 1,000
The above
inequality cannot be 100
satisfied since c
must be a constant 10
n2 is O(n2).
1
1 10 100 1,000
n

Analysis of Algorithm 22
More Big-Oh
Examples
7n-2
7n-2 is O(n)
need c > 0 and n0 1 such that 7n-2 cn for n n0
this is true for c = 7 and n0 = 1
3n3 + 20n2 + 5
3n3 + 20n2 + 5 is O(n3)
need c > 0 and n0 1 such that 3n3 + 20n2 + 5 cn3 for n
n0
this is true
3 log n for
+5 c = 4 and n0 = 21

3 log n + 5 is O(log n)
need c > 0 and n0 1 such that 3 log n + 5 clog n for n n0
this is true for c = 8 and n0 = 2
Analysis of Algorithm 23
Big-Oh and Growth Rate
The big-Oh notation gives an upper bound on the
growth rate of a function
The statement f(n) is O(g(n)) means that the growth
rate of f(n) is no more than the growth rate of g(n)
We can use the big-Oh notation to rank functions
according to their growth rate

Analysis of Algorithm 24
Big-Oh Rules

If f(n) is a polynomial of degree d, then f(n)

is O(nd), i.e.,
1. Drop lower-order terms
2. Drop constant factors
Use the smallest possible class of functions
Say 2n is O(n) instead of 2n is O(n2)
Use the simplest expression of the class
Say 3n 5 is O(n) instead of 3n 5 is O(3n)

Analysis of Algorithm 25
Growth Rate of Running Time

Consider a program with time complexity O(n 2).

For the input of size n, it takes 5 seconds.
If the input size is doubled (2n), then it takes 20 seconds.

Consider a program with time complexity O(n).

For the input of size n, it takes 5 seconds.
If the input size is doubled (2n), then it takes 10 seconds.

Consider a program with time complexity O(n 3).

For the input of size n, it takes 5 seconds.
If the input size is doubled (2n), then it takes 40 seconds.

Analysis of Algorithm 26
Asymptotic Algorithm
Analysis
The asymptotic analysis of an algorithm
determines the running time in big-Oh notation
To perform the asymptotic analysis
We find the worst-case number of primitive
operations executed as a function of the input size
We express this function with big-Oh notation
Example:
We determine that algorithm arrayMax executes at
most 6n 1 primitive operations
We say that algorithm arrayMax runs in O(n) time
Since constant factors and lower-order terms are
eventually dropped anyhow, we can disregard
them when counting primitive operations

Analysis of Algorithm 27
Computing Prefix Averages
We further illustrate 35
asymptotic analysis with X
two algorithms for prefix 30 A
averages 25
The i-th prefix average of
20
an array X is average of the
first (i 1) elements of X: 15
A[i] X[0] X[1] X[i])/(i+1)
10
5
Computing the array A of
prefix averages of another 0
array X has applications to 1 2 3 4 5 6 7
financial analysis

Analysis of Algorithm 28
Prefix Averages
(Quadratic)
The following algorithm computes prefix averages
in quadratic time by applying the definition
Algorithm prefixAverages1(X, n)
Input array X of n integers
Output array A of prefix averages of X #operations
A new array of n integers n
for i 0 to n 1 do n
{ s X[0] n
for j 1 to i do 1 2 (n 1)
s s X[j] 1 2 (n 1)
A[i] s (i 1) } n
return A 1
Analysis of Algorithm 29
Arithmetic Progression
The running time of
prefixAverages1 is
O(1 2 n)
The sum of the first n
integers is n(n 1) 2
There is a simple
visual proof of this fact
Thus, algorithm
prefixAverages1 runs in
O(n2) time

Analysis of Algorithm 30
Prefix Averages (Linear)
The following algorithm computes prefix averages
in linear time by keeping a running sum
Algorithm prefixAverages2(X, n)
Input array X of n integers
Output array A of prefix averages of X #operations
A new array of n integers n
s0 1
for i 0 to n 1 do n
{s s X[i] n
A[i] s (i 1) } n
return A 1
Algorithm prefixAverages2 runs in O(n) time
Analysis of Algorithm 31
Exercise: Give a big-Oh
characterization

Algorithm Ex1(A, n)
Input an array X of n integers
Output the sum of the elements in A

s A[0]
for i 0 to n 1 do
s s A[i]

return s

Analysis of Algorithm 32
Exercise: Give a big-Oh
characterization

Algorithm Ex2(A, n)
Input an array X of n integers
Output the sum of the elements at even cells in A

s A[0]
for i 2 to n 1 by increments of 2 do
s s A[i]
return s

Analysis of Algorithm 33
Exercise: Give a big-Oh
characterization

Algorithm Ex1(A, n)
Input an array X of n integers
Output the sum of the prefix sums A
s0
for i 0 to n 1 do
s s A[0]
for j 1 to i do
s s A[j]
return s

Analysis of Algorithm 34
 Remarks:

In the first tutorial, ask the students to try programs with running time O(n), O(n log n), O(n 2), O(n2log n), O(2n) with various
inputs.
They will get intuitive ideas about those functions.

for (i=1; i<=n; i++)

for (j=1; j<=n; j++)
{ x=x+1; delay(1 second);
}

Analysis of Algorithm 35