Algorithms

Design and Analysis

AKRAM_LOQMAN_773133872 1

AKRAM_LOQMAN_773133872 2
 Course Objectives
This course introduces students to the analysis
and design of computer algorithms. Upon
completion of this course, students will be able
to do the following:
Analyze the asymptotic performance of
algorithms.
Demonstrate a familiarity with major
algorithms and data structures.
Apply important algorithmic design
paradigms and methods of analysis.
Synthesize efficient algorithms in common
engineering design situations. 3
AKRAM_LOQMAN_773133872

4
AKRAM_LOQMAN_773133872
 Course Objectives
In this course we will cover the following
topics:
Understand foundations of algorithms
&& design and analysis various variants
algorithms
Accuracy
Efficiency
Comparing efficiencies
AKRAM_LOQMAN_773133872 5

AKRAM_LOQMAN_773133872 6
 Algorithms
The word algorithm comes from the name of a
Persian mathematician Abu Ja far Mohammed
ibn-i Musa al Khowarizmi.

In computer science, this word refers to a special

method useable by a computer for solution of a
problem. The statement of the problem specifies
in general terms the desired input/output
relationship.

For example, sorting a given sequence of

numbers into nondecreasing order provides fertile
ground for introducing many standard design
techniques and analysis tools. AKRAM_LOQMAN_773133872 7

AKRAM_LOQMAN_773133872 8
 What is a program?
A program is the expression of an
algorithm in a programming language

a set of instructions which the computer

will follow to solve a problem

AKRAM_LOQMAN_773133872 9

AKRAM_LOQMAN_773133872 10
Problem-Solving (Science and Engineering)

Analysis
How does it work?
Breaking a system down to known components
How the components relate to each other
Breaking a process down to known functions

Synthesis
Building tools and toys!
What components are needed
How the components should be put together
Composing functions to form a process
AKRAM_LOQMAN_773133872 11

12
AKRAM_LOQMAN_773133872
Problem Solving Using Computers
Problem:
Strategy:
Algorithm:
Input:
Output:
Step:
Analysis:
Correctness:
Time & Space:
Optimality:
Implementation:
Verification: AKRAM_LOQMAN_773133872 13

AKRAM_LOQMAN_773133872 14
Why need algorithm analysis ?
Writing a working program is not good
enough

The program may be inefficient!

If the program is run on a large data set,

then the running time becomes an issue
AKRAM_LOQMAN_773133872 15

16
AKRAM_LOQMAN_773133872
Analyzing Algorithms and Problems
We analyze algorithms with the intention
of improving them, if possible, and for
choosing among several available for a
problem.

Correctness
Amount of work done, and space used
Optimality, Simplicity

AKRAM_LOQMAN_773133872 17

AKRAM_LOQMAN_773133872 18
 Correctness can be proved!
An algorithm consists of sequences of
steps (operations, instructions,
statements) for transforming inputs
(preconditions) to outputs (postconditions)

Proving
if the preconditions are satisfied,
then the postconditions will be true,
when the algorithm terminates.

AKRAM_LOQMAN_773133872 19

AKRAM_LOQMAN_773133872 20
 Amount of work done
We want a measure of work that tells us
something about the efficiency of the
method used by the algorithm

independent of computer, programming

language, programmer, and other
implementation details.

Usually depending on the size of the input

AKRAM_LOQMAN_773133872 21

AKRAM_LOQMAN_773133872 22
 Amount of work done
Counting passes through loops
Basic Operation
Identify a particular operation
fundamental to the problem
the total number of operations
performed is roughly proportional to
the number of basic operations
Identifying the properties of the inputs
that affect the behavior of the algorithm
AKRAM_LOQMAN_773133872 23

AKRAM_LOQMAN_773133872 24
 Space Usage
If memory cells used by the
algorithms depends on the particular
input,

then worst-case and average-case

analysis can be done.

Time and Space Tradeoff.

AKRAM_LOQMAN_773133872 25

AKRAM_LOQMAN_773133872 26
Optimality the best possible
Each problem has inherent complexity
There is some minimum amount of work
required to solve it.
To analyze the complexity of a problem,
we choose a class of algorithms, based on
which
prove theorems that establish a lower
bound on the number of operations
needed to solve the problem.
Lower bound (for the worst case)
AKRAM_LOQMAN_773133872 27

AKRAM_LOQMAN_773133872 28
 Simplicity

Simplicity in an algorithm is a virtue.

AKRAM_LOQMAN_773133872 29

AKRAM_LOQMAN_773133872 30
 Designing Algorithms
Problem solving using Computer

Algorithm Design Techniques

divide-and-conquer
greedy methods
depth-first search (for graphs)
dynamic programming

AKRAM_LOQMAN_773133872 31

AKRAM_LOQMAN_773133872 32
 The problem of sorting
Input: sequence a1, a2, , an of numbers.

Output: permutation a'1, a'2, , a'n Such

that a'1 a'2 a'n.

Example:
Input: 8 2 4 9 3 6
Output: 2 3 4 6 8 9
AKRAM_LOQMAN_773133872 33

AKRAM_LOQMAN_773133872
34
 Analysis of algorithms
The theoretical study of computer-program performance
and resource usage.

What s more important than performance?

modularity
correctness
maintainability
functionality
robustness
user-friendliness
programmer time
simplicity
extensibility
AKRAM_LOQMAN_773133872 35
reliability

AKRAM_LOQMAN_773133872
36
 Analysis of algorithms
Why study algorithms and performance?

Algorithms help us to understand scalability.

Performance often draws the line between
what is feasible and what is impossible.
Algorithmic mathematics provides a
language for talking about program behavior.
The lessons of program performance
generalize to other computing resources.
Speed is fun!
AKRAM_LOQMAN_773133872 37

AKRAM_LOQMAN_773133872 38
 Running Time
The running time depends on the input: an
already sorted sequence is easier to sort.

Parameterize the running time by the size

of the input, since short sequences are
easier to sort than long ones.

Generally, we seek upper bounds on the

running time, because everybody likes a
guarantee. AKRAM_LOQMAN_773133872 39

40
AKRAM_LOQMAN_773133872
 Kinds of analyses
Worst-case: (usually)
T(n) = maximum time of algorithm on any input of
size n.

Average-case: (sometimes)
T(n) = expected time of algorithm over all inputs of
size n.
Need assumption of statistical distribution of inputs.

Best-case:
Cheat with a slow algorithm that works fast on
some input. AKRAM_LOQMAN_773133872 41

Tn
n

Tn
n

AKRAM_LOQMAN_773133872 42
 Machine-Independent time
The RAM Model
Machine independent algorithm design depends
on a hypothetical computer called Random Acces
Machine (RAM).

Assumptions:
Each simple operation such as +, -, if ...etc takes
exactly one time step.
Loops and subroutines are not considered
simple operations.
Each memory acces takes exactly one time step.
AKRAM_LOQMAN_773133872 43

RAM

Random Access Machine

(RAM).

if

44
AKRAM_LOQMAN_773133872
 Machine-independent time
What is sort s worst-case time?

It depends on the speed of our computer,

relative speed (on the same machine),
absolute speed (on different machines).

BIG IDEA:
Ignore machine-dependent constants.
Look at growth of T (n) as n
Asymptotic Analysis 45
AKRAM_LOQMAN_773133872

n Tn
46

AKRAM_LOQMAN_773133872
 Asymptotic Analysis

Engineering:

Drop low-order terms;

ignore leading constants.

Example:

AKRAM_LOQMAN_773133872 47

AKRAM_LOQMAN_773133872 48
 Asymptotic performance
When n gets large enough, a 2) algorithm
always beats a 3) algorithm.

T(n)

AKRAM_LOQMAN_773133872 n n0 49

n2) n
n3)

T(n)

AKRAM_LOQMAN_773133872 n n0 50