ADVANCED ANALYSIS OF ALGORITHMS

Dr. Javed Iqbal Bangash

Class Policy

 Lecture:
 Every Monday (3:30PM – 6:30PM).
 Attendance:
 Not less than 75%.
 Grading:
 Assignments (5 – 6)
 Quizzes (3 – 4)
 Mid-Term Exam
 Final-Term Exam
Text and Reference Books

 Text Book:
 Foundations of Algorithms Using C++ Pseudocode, Third
Edition, By Richard E. Neapolitan & Kumarss Naimipour.
 Reference Books:
 Introduction to Algorithms, Second Edition, By Thomas H.
Cormen, Charles E. Leiserson, Ronald L. Rivest and Clifford
Stein.
 Computer Algorithms Introduction to Design & Analysis,
Third Edition, By Sara Baase.
Course Outline

 Introduction and Review of Necessary Mathematics

 Algorithms – Design, Efficiency, Analysis & Order
 Divide-and-Conquer
 Dynamic Programming
 The Greedy Approach
 Introduction to Computational Complexity—The Sorting
Problem
 More Computational Complexity—The Searching
Problem
 An Introduction to the Theory of NP
Introduction

 Algorithm is about the techniques for solving problems

using computer.
 Technique does not mean the programming style or a
programming language.
 But rather it is the approach or methodology used to
solve a problem.
 For Example:
 Take a telephone directory where the names are sorted in
ascending order
Introduction (Conti…)

 For Example (Conti…):

 You are looking to find your friend’s telephone number
 First Approach:
 Start looking from page 1, 2, 3 and so on until you find the
required number.
 Is it efficient? Why?
 Second Approach:
 Flip to the middle and tear the book into two
 Throw one part and continue with other part
MUHAMMAD BIN MUSA AL-KHWARIZMI
 Abu Abdullah Muhammad Ibn Musa al-Khwarizmi was born at
Khwarizm (Kheva).
 It has been established from his contributions that he flourished under
Khalifah (Calif) Al-Mamun at Baghdad during 813 to 833 C.E. and died
around 840 C.E.
 Al-Khwarizmi was one of the greatest mathematicians ever lived. He
was the founder of several branches and basic concepts of mathematics.
 Algebra, as he not only initiated the subject in a systematic form but also
developed it to the extent of giving analytical solutions of linear and
quadratic equations.
MUHAMMAD BIN MUSA AL-KHWARIZMI (Conti…)

 The name Algebra is derived from his famous book Al-Jabr wa-al-
Muqabilah.
 He developed in detail trigonometric tables containing the sine
functions, which were later extrapolated to tangent functions.
 He adopted the use of zero, a numeral of fundamental importance,
leading up to the so-called arithmetic of positions and the decimal
system.
 His pioneering work on the system of numerals is well known as
"Algorithm," or "Algorizm” which is so named after his last name.
What is an Algorithm? (Informal)

 A sequence of computational step that transform the

input into the output.
 An algorithm is any well-defined computational
procedure that takes some value, or set of values, as
input and produces some value, or set of values, as
output.
 An algorithm is thus a sequence of computational steps
that transform the input into the output.
What is an Algorithm? (Conti…)

 We can also view an algorithm as a tool for solving a

well-specified computational problem.
 The algorithm describes a specific computational
procedure for achieving that input/output relationship.

Input Algorithm Output

What is an Algorithm? (Conti…)

 Computer Programs = Algorithms + Data Structures

 Algorithms are the ideas behind computer programs.
 An algorithm is the thing which stays the same whether
the program is in Pascal running on a Cray in New York
or is in BASIC running on a Macintosh in Kathmandu.
 Data Structure: An organized method of storing and
Retrieving data
What is an Algorithm? (Formal)

 An algorithm is a finite set of precise instructions for

performing a computation or solving a problem.
 An algorithm for solving a problem “a finite sequence of
unambiguous, executable steps or instructions, which, if
followed would ultimately terminate and give the
solution of the problem”.
 Algorithm must have the following criteria:
 Input: Zero or more quantities is supplied.
 Output: At least one quantity is produced
What is an Algorithm? (Conti…)

 Criteria (Conti…):
 Finiteness: If we trace out the instructions of an algorithm, then
for all cases, the algorithm terminates after a finite number of
steps.
 Unambiguous: Each instruction is clear and unambiguous.
 Executable: Every instruction must be basic so that it can be
carried out.
 Terminates: The execution of an algorithm must lead to an end.
Why Study Algorithms?

 Necessary in any computer programming problem

 Improve algorithm efficiency: run faster, process more data, do
something that would otherwise be impossible
 Solve problems of significantly large sizes
 Technology only improves things by a constant factor
 Compare algorithms
 May be more than one algorithm to solve the same problem
 May be the efficiency of the same algorithm is different for
different sizes of data
Why Study Algorithms? (Conti…)

 Algorithms as a field of study

 Learn about a standard set of algorithms
 New discoveries arise
 Numerous application areas
 Learn techniques of algorithm
 Design and
 Analysis
What is a Problem?

 A problem is a question to which we seek an answer.

 The following are the different examples of different
problems.
 Example 1.1:
 Sort a list S of n numbers in non-decreasing order.
 The answer is the numbers in sorted sequence.
 Example 1.2:
 Determine whether the number x is in the list S of n numbers.
 The answer is yes if x is in S and no if it is not.
Parameters

 A problem may contain variables that are not assigned

specific values in the statement of the problem.
 These variables are called parameters to the given
problem.
 In Example 1.1 there are two parameters: S (the list) and
n (the number of items in S).
 In Example 1.2 there are three parameters: S, n, and the
number x.
Parameters

 A problem may contain variables that are not assigned

specific values in the statement of the problem.
 These variables are called parameters to the given
problem.
 In Example 1.1 there are two parameters: S (the list) and
n (the number of items in S).
 In Example 1.2 there are three parameters: S, n, and the
number x.
Parameters (Conti…)

 It is not necessary in these two examples to make n one

of the parameters.
 because its value is uniquely determined by S.
 However, making n a parameter facilitates our
descriptions of problems.
Instance & Solution

 Because a problem contains parameters, it represents a

class of problems, one for each assignment of values to
the parameters.
 Each specific assignment of values to the parameters is
called an instance of the problem.
 A solution to an instance of a problem is the answer to
the question asked by the problem in that instance.
Instance & Solution (Conti…)

 Example 1.3:
 An instance of the problem in Example 1.1 is:
 S = [10, 7, 11, 5, 13, 8]        and        n = 6.
 The solution to this instance is [5, 7, 8, 10, 11, 13].
 Example 1.4:
 An instance of the problem in Example 1.2 is:
 S = [10, 7, 11, 5, 13, 8], n = 6,        and        x = 5
 The solution to this instance is, "yes, x is in S."
Algorithm

 To produce a computer program that can solve all

instances of a problem, we must specify a general step-
by-step procedure for producing the solution to each
instance.
 This step-by-step procedure is called an algorithm.
 We say that the algorithm solves the problem.
 Algorithm:
 Pseudocodes
 Flowcharts
Writing an Algorithm

 Problem:
 Adding two n-digit numbers
 Input:
 Two positive m-digit decimal numbers (a and b)
am, am-1, …., a1
bm, bm-1, …., b1
 Output:
 The sum c = a + b
cm+1, cm, cm-1, …., c1
How to “derive” the Algorithm

 Adding is something we all know:

 done it a thousand times, know it “by heart”
 How do we give the algorithm?
 A step-by-step instruction to a dumb machine
 Try an example:
3492
8157
 “Imagine you looking at yourself solving it”
Algorithm: Adding two n-digit Numbers

Step 1: Set the value of carry to 0

Step 2: Set the value of i to 1.
Step 3: Repeat steps 4 through 6 until i > m.
Step 4: Add ai and bi to the current value of carry, to get xi.
Step 5: If xi < 10, then
let ci = xi and reset carry = 0.
else (/* i.e. xi  10 */)
let ci = xi - 10 and reset carry = 1.
Step 6: Increase the value of i by 1.
Step 7: Set cm+1 to the value of carry.
Step 8: Print the final answer cm+1, cm, …., c1
Step 9: Stop
REVIEW OF NECESSARY

Mathematics
Mathematical Background
 Sets
 A set is a collection of distinct elements that we wish to treat
as a single object.
 Usually the elements are of the same type
 A set has no inherent order
 {a,b,c} and {b,c,a} are same
 Sequence
 A group of elements in a specified order is called a sequence.
 (a,b,c), (b,c,a), and (a,c,b) are distinct sequences
 Series
 A series is the sum of a sequence.

27
Mathematical Background (Conti…)
 Floor and Ceiling:
 For any real number x,  x  (read “floor of x”) is the largest
integer less than or equal to x.  x(read
 “ceiling of x”) is the
smallest integer greater than or equal to x. e.g.
 2.9  2, and  6.1  7.
 Functions:
 A function f of one variable is a rule or law that associates
with a value x a unique value f(x) e.g. the function f that
associates the square of a real number with a given real
number x is
f(x) = x2

28
Mathematical Background (Conti…)
 Functions (Conti…)
 A function determines the set of ordered pairs. e.g. the
above function determines all the ordered pairs (x, x2). A
graph of a function is the set of all ordered pairs
determined by the function.
 Summations
 Often we need to refer to the sum of like terms. If we need
to refer to the sum of the first seven positive integers, we
simply write:
1+2+3+4+5+6+7

29
Mathematical Background (Conti…)
 Summations (Conti…)
 If we need to refer to the sum of the squares of the first
seven positive integers, we simply write:
12+22+32+42+52+62+72
 It is not satisfactory if we need to refer to the sum of the
first 100 positive integers. We could write:
1+2+3+4+…·+ 100
 However, more concise method is to use Greek letter Σ
(sigma): 100 100


i 1
i 
i 1
i 2

30
Mathematical Background (Conti…)
 Series:
 Arithmetic Series: The sum of consecutive integers.
n
n( n  1)
i 
i 1 2
 Polynomial Series: Sum of the squares of integers.
n
n(n  1)( 2n  1)

i 1
i  2

6
 Power of 2: This is a frequently occurring case of a
geometric series
k

 2 i

i 0
 2 k 1
1

31
Mathematical Background (Conti…)
 Series (Conti…):
 Geometric Series: The general case of geometric series.
n 1
n
1 r

i 0
r 
i

r 1
 Harmonic Series: n


i 1
1
i
 ln( n)  .577

 Arithmetic-Geometric
n Series:
 i
i 1
2 i
 ( k  1) 2 k 1
2

32
Mathematical Background (Conti…)
 Series (Conti…):
 Fibonacci Numbers: Although this is not a summation, the
series occurs frequently in analysis of algorithms.
Fn  Fn 1  Fn  2 for n  2,
F0  0, F1 1.
 Theorems:
 The dictionary defines a theorem as a proposition that sets
forth something to be proved.
 Theorem 1: For all integers n > 0, we have:
n( n  1)
1  2  ....  n 
2

33
Mathematical Background (Conti…)
 Lemmas:
 The dictionary defines a lemma as a subsidiary proposition
employed to prove another proposition.
 Like a theorem, a lemma is a proposition that sets forth
something to be proved.
 Lemmas are then employed to prove theorems.
 Logarithms:
 Logarithms are the one of the mathematical tools used most
in the analysis of algorithms.

34
Mathematical Background (Conti…)
 Definition of Logarithm:
 For b>1 and x>0, logb x (read “log to the base b of x”) is
that real number L such bL = x; that is, logb x is the power
to which b must be raised to get x.
 Some Examples of Logarithms:
 log2 8 = 3 because 23 = 8
 log3 81 = 4 because 34 = 81
 log2 1/16 = -4 because 2-4 = (1/2)4 = (1/16)
 log2 7 ≈ 2.807 because 22.807 ≈ 7

35
Mathematical Background (Conti…)
 Properties of Logarithms:
 loga 1 = 0
 logb ba = a
 loga (xy) = loga x + loga y
 loga x/y = loga x – loga y
 loga xy = y loga x
 loga x = logb x / logb a

36