Chapter 1 - 3 PDF

1
Data Structures and Algorithms
BITS-Pilani K. K. Birla Goa Campus
1
Material for the presentation taken from Cormen, Leiserson, Rivest and
Stein, Introduction to Algorithms, Third Edition; 1
BITS-Pilani K. K. Birla Goa Campus Data Structures and Algorithms
CS F211: Data Structures and Algorithms
Handout Discussion
2
Role of algorithms in computing
I What is an Algorithm?
3
It is a well-defined computational procedure which takes an
input and produces an output.
3
I An algorithm solves a computational problem.
3
I Sorting Problem
3
I Sorting Problem
Input: A sequence of n numbers < a1 , a2 , . . . , an >
3
I Sorting Problem
Output: A permutation < a10 , a20 , . . . , an0 > such that
a10 ≤ a20 ≤ · · · ≤ an0
3
I Sorting Problem
a10 ≤ a20 ≤ · · · ≤ an0
I E.g., Input sequence : < 31, 41, 59, 26, 41, 58 >
3
I Sorting Problem
a10 ≤ a20 ≤ · · · ≤ an0
I Sorting algorithm should give the output:
< 26, 31, 41, 41, 58, 59 >
3
I An algorithm is correct if it halts for every input with the

correct output.
4

correct output.
I We can have algorithms for a wide range of computational
problems.
4

correct output.
problems.
I What is a Data Structure?
4

correct output.
problems.
A data structure is a way to store and organize data in order
to facilitate access and modification.
4

correct output.
problems.
I Goal of this course:
4

correct output.
problems.
I Goal of this course:
Learn techniques of algorithm design and analysis so that we
can design algorithms on our own, show that the algorithm is
correct, and understand how efficient the algorithm is.
4
Which Algorithm should be preferred?
I There can be more than one algorithm that correctly solves a

given computational problem. Which one should we prefer?
5

I Computers have limited memory and only a finite number of
instructions can be excecuted per second. We should prefer
an algorithm that uses these two resources wisely.
5

I Computers have limited memory and only a finite number of
instructions can be excecuted per second. We should prefer
an algorithm that uses these two resources wisely.
I A good algorithm would be efficient in terms of computing
time and memory that is used.
5
Efficiency of Algorithms
I Two algorithms for the sorting problem: Insertion sort and

Merge sort.
6

Merge sort.
I Insertion sort takes roughly c1 n2 time. Merge sort takes
roughly c2 n lg n.
6

Merge sort.
roughly c2 n lg n.
I Let c1 = 2, c2 = 50. (c1 is usually much smaller than c2 .)
6

Merge sort.
roughly c2 n lg n.
I Time taken by the two algorithms for n = 8?
6

Merge sort.
roughly c2 n lg n.
I Time taken by the two algorithms for n = 8?
I Time taken by the two algorithms for n = 1 million ?
6
Importance of efficiency
I Can using a faster computer solve the problem of coming up

with a good algorithm?
7

I Let us compare a faster computer A (running insertion sort)
with a slower computer B (running merge sort).
7

I Suppose computer A executes 10 billion instructions per
second and computer B executes 10 million instructions per
second.
7

second.
I Let n = 10 million, c1 = 2 and c2 = 50.
7

second.
2 × (107 )2
I Time taken by faster computer A:
1010 instructions/second
7

second.
2 × (107 )2
50 × (107 ) lg 107
I Time taken by slower computer B:
7

second.
2 × (107 )2
50 × (107 ) lg 107
I Time taken by slower computer B:
I Faster computer A takes more than 20,000 seconds (5.5
hours), whereas slower computer B takes around 1163 seconds
(< 20 minutes).
7
I Conclusion?
8
I Conclusion?
We should study how to analyse and design efficient
algorithms.
8
Comparison of running times
I For each function f (n) and time t in the following table,

determine the largest size n of a problem that can be solved in
time t, assuming that the algorithm to solve the problem
takes f (n) microseconds.
9
Insertion sort
I Time taken : c1 n2
10
Insertion sort
I Time taken : c1 n2
a10 ≤ a20 ≤ · · · ≤ an0
I Sorting algorithm should give the output:
< 26, 31, 41, 41, 58, 59 >
10
Insertion sort
I Main idea: Works the way people sort a hand of playing cards.
11
Insertion sort for an array of numbers
I Convention used : Array index starts from 1.
12
Insertion sort for an array of numbers
I Convention used : Array index starts from 1.
12
Pseudocode
I We will use pseudocode to describe the insertion sort

algorithm.
13
Pseudocode
I We will use pseudocode to describe the insertion sort

algorithm.
I Pseudocode is used to explain the essence of an algorithm
concisely and clearly.
13
Pseudocode for Insertion sort
14
Correctness of Insertion sort
15
I Loop invariant: The subarray A[1..j − 1] contains elements

originally in A[1..j − 1], but in sorted order.
15
I Loop invariant: The subarray A[1..j − 1] contains elements

I We use loop invariant to argue for the correctness of an
algorithm.
15
Loop invariant: The subarray A[1..j − 1] contains elements

I Initialization,
16

I Initialization, Maintenance,
16

I Initialization, Maintenance, Termination

16
Analysis of algorithms
I We will assume a generic one-processor, random-access

machine (RAM).
17

machine (RAM).
I RAM model contains common operations/instructions (add,
subtract, load, store, conditional branch, subroutine call etc.)
17

machine (RAM).
I RAM model contains common operations/instructions (add,
subtract, load, store, conditional branch, subroutine call etc.)
I Each statement in the pseudocode takes a constant amount
of time.
17
I Running time of an algorithm is expressed as a function of the

size of its input.
18

size of its input.
I Input size : depends on the computational problem
18

size of its input.
I Sorting an array
18

size of its input.
I Sorting an array
I Finding shortest path in a graph
18

size of its input.
I Sorting an array
I Is a given integer prime?
18

size of its input.
I Sorting an array
I Is a given integer prime?
I Running time : time needed for the primitive operations or
“steps” executed by the RAM model for an input of size n.
18
Pseudocode with time costs
19
Summation of time costs of all the steps
20
20
21
I What will be the value of tj if the input sequence is already

sorted?
21

sorted?
21

sorted?
I The running time is of the form an + b, where a and b are

constants.
21

sorted?
I The running time is of the form an + b, where a and b are

constants.
I So, the running time is a linear function of n if the input
sequence is already sorted (best case scenario).
21
Summation of time costs in the worst case
I When will the worst case scenario occur?
22

I What will be the value of tj in the worst case scenario?
22

I What will be the value of tj in the worst case scenario?
22
23
23
23
24
I Worst case running time is of the form an2 + bn + c
24
I Worst case running time is of the form an2 + bn + c

I So, the running time is a quadratic function of n if the input
sequence is sorted in the reverse order (worst case scenario).
24
Running time estimation
I What will the average case look like?
25

j
I tj = ,
2
25

j
I tj = , T (n) will again be a quadratic function of n.
2
25

j
I tj = , T (n) will again be a quadratic function of n.
2
I Very often we will be interested in only the worst-case running
time because it gives the upper bound on the running time for
any input.
25
Order of growth: Simplifying abstractions
I We will make simplifying abstractions because we are only

interested in the rate of growth (order of growth) of the
running time for large values of n.
26

I Suppose T (n) = 5n2 + 100n + 500
26

I Suppose T (n) = 5n2 + 100n + 500
I After the simplifying abstractions, we are left with n2 which is
the factor by which T (n) will increase for large values of n.
26

I Suppose T (n) = 5n2 + 100n + 500
I After the simplifying abstractions, we are left with n2 which is
the factor by which T (n) will increase for large values of n.
I We express the above by saying that insertion sort has a
worst-case running time of Θ(n2 ).
26
Problem
27
Insertion sort
I Main idea : follow Incremental approach
28
Insertion sort
I Main idea : follow Incremental approach
28
The divide-and-conquer approach
I The merge sort algorithm follows the divide-and-conquer

paradigm
29

paradigm
Divide: Divide the n-element sequence to be sorted into two
subsequences of n/2 elements each.
29

paradigm
Conquer: Sort the two subsequences recursively using merge
sort.
29

paradigm
Conquer: Sort the two subsequences recursively using merge
sort.
Combine: Merge the two sorted subsequences to produce the
sorted answer.
29
Combine step in Merge sort
I MERGE (A, p, q, r ) procedure.
30

I The procedure assumes that the subarrays A[p..q] and
A[q + 1..r ] are sorted.
30

I It combines the sorted subarrays such that the combined array
A[p..r ] is also sorted.
30

I Main idea :
30

I Main idea :
L: 2 4 5 9
R: 3 6 7 8
30
31
31
Merge procedure
I Running time of MERGE procedure is Θ(n)
32
Merge sort
I Initial call MERGE -SORT (A, 1, A.length)
33
Merge sort operations
A = [ 5, 2, 4, 7, 1, 3, 6, 2 ]
34
Merge sort operations
A = [ 5, 2, 4, 7, 1, 3, 6, 2 ]
34
Analyzing divide-and-conquer algorithms
I Recurrence equation:
35
Analyzing the merge sort algorithm
I Recurrence for worst-case running time of merge sort:
36
36
Recursion tree for merge sort
37
38
39
I The recursion tree will have lg n + 1 levels.
40

I Worst case running time of merge sort:
T (n) = cn lg n + cn
40

I Worst case running time of merge sort:
T (n) = cn lg n + cn
= Θ(n lg n)
40
Ch 3 : Growth of Functions
I Once input size n is large enough, merge sort (Θ(n lg n)) will
beat insertion sort (Θ(n2 )).
41
I We are interested in studying the asymptotic efficiency of
algorithms (i.e. order of growth when n tends to infinity).
41
I We use different asymptotic notations (e.g. Θ notation) for
describing the efficiency of algrithms.
41
I We use different asymptotic notations (e.g. Θ notation) for
describing the efficiency of algrithms.
I When we say running time T (n) = Θ(n2 ), we mean T (n) is a
function in the set Θ(n2 ).
41
Θ notation
42
Θ notation
42
Θ notation
1 2
I Is 2n − 3n ∈ Θ(n2 )?
43
Θ notation
I Is 1 2 2
2 n − 3n ∈ Θ(n )?
I To check the above we need to find positive constants c1 , c2
and n0 such that:
1
0 < c1 n2 ≤ n2 − 3n ≤ c2 n2 , For n ≥ n0 (1)
2
43
Θ notation
I Is 1 2 2
2 n − 3n ∈ Θ(n )?
and n0 such that:
1
0 < c1 n2 ≤ n2 − 3n ≤ c2 n2 , For n ≥ n0 (1)
2
1 3
0 < c1 ≤ − ≤ c2
2 n
43
Θ notation
I Is 1 2 2
2 n − 3n ∈ Θ(n )?
and n0 such that:
1
0 < c1 n2 ≤ n2 − 3n ≤ c2 n2 , For n ≥ n0 (1)
2
1 3
0 < c1 ≤ − ≤ c2
2 n
I Eqn. 1 will be true for c1 = 1/14, c2 = 1/2 and n0 = 7.
(Other choices were also possible.)
43
Θ notation
I Is 3n2 + 4n − 100 ∈ Θ(n2 )?
44
Θ notation
I Is 3n2 + 4n − 100 ∈ Θ(n2 )?

I Is 3n + 2 ∈ Θ(n2 ) ?
44
Θ notation
I Is 3n2 + 4n − 100 ∈ Θ(n2 )?

I Is 3n + 2 ∈ Θ(n2 ) ?
I Is 2n3 ∈ Θ(n2 ) ?
44
Θ notation
I Is 3n2 + 4n − 100 ∈ Θ(n2 )?

I Is 3n + 2 ∈ Θ(n2 ) ?
I Is 2n3 ∈ Θ(n2 ) ?
I In general, if p(n) is a degree d polynomial, then
p(n) ∈ Θ(nd ).
44
Θ notation
I Is 3n2 + 4n − 100 ∈ Θ(n2 )?

I Is 3n + 2 ∈ Θ(n2 ) ?
I Is 2n3 ∈ Θ(n2 ) ?
I In general, if p(n) is a degree d polynomial, then
p(n) ∈ Θ(nd ).
Caveat: The coefficient of the highest degree term must be positive.
44
Θ notation
I 2n2 + Θ(n) = Θ(n2 )
45
O notation (Big-oh)
I We use O notation to express asymptotic upper bound.
46
O notation (Big-oh)
I We use O notation to express asymptotic upper bound.
46
O notation (Big-oh)
I Is 3n + 2 ∈ O(n2 ) ?
47
O notation (Big-oh)
I Is 3n + 2 ∈ O(n2 ) ?
I Is 2n3 ∈ O(n2 ) ?
47
O notation (Big-oh)
I Is 3n + 2 ∈ O(n2 ) ?
I Is 2n3 ∈ O(n2 ) ?
I Is 2n + 2 ∈ O(n) ?
47
O notation (Big-oh)
I Is 3n + 2 ∈ O(n2 ) ?
I Is 2n3 ∈ O(n2 ) ?
I Is 2n + 2 ∈ O(n) ?
I Is the following statement true?
If f (n) = O(g (n)) then f (n) = Θ(g (n)).
47
O notation (Big-oh)
I Is 3n + 2 ∈ O(n2 ) ?
I Is 2n3 ∈ O(n2 ) ?
I Is 2n + 2 ∈ O(n) ?
I O notation helps to express the running time on every input.
47
O notation (Big-oh)
I Is 3n + 2 ∈ O(n2 ) ?
I Is 2n3 ∈ O(n2 ) ?
I Is 2n + 2 ∈ O(n) ?
I For example, the running time of insertion sort on every input
is O(n2 ).
47
O notation (Big-oh)
I Is 3n + 2 ∈ O(n2 ) ?
I Is 2n3 ∈ O(n2 ) ?
I Is 2n + 2 ∈ O(n) ?
is O(n2 ).
I Can we say, “running time of insertion sort on every input is
O(n3 )”?
47
O notation (Big-oh)
I Is 3n + 2 ∈ O(n2 ) ?
I Is 2n3 ∈ O(n2 ) ?
I Is 2n + 2 ∈ O(n) ?
is O(n2 ).
I Can we say, “running time of insertion sort on every input is
O(n3 )”?
I Is 3n2 = O(n2 − 10n − 20)?
47
Ω-notation
I Provides an asymptotic lower bound.
48
Ω-notation
48
Ω-notation
48
Ω-notation
I Is 3n2 + 2 ∈ Ω(n) ?
49
Ω-notation
I Is 3n2 + 2 ∈ Ω(n) ?
I Is 3n2 + 2 ∈ Ω(n3 ) ?
49
Ω-notation
I It helps us express the best case running time for any input.
50
Ω-notation
I The running time of insertion sort is Ω(n) for any input.
50
Ω-notation
I The running time of insertion sort is Ω(n) for any input.
50
o-notation
I o(g (n)) “little-oh of g of n”
51
o-notation

I We use this notation to denote an upperbound which is not
asymptotically tight
51
o-notation

I Tight bound:2n2 = O(n2 )
51
o-notation

I Tight bound:2n2 = O(n2 ) Loose bound:2n = O(n2 ).
51
o-notation

51
o-notation

I Is 2n2 + 1 = o(n2 ) ?
51
o-notation

I Is 2n2 + 1 = o(n2 ) ?
I Is 2n2 + 1 = o(n3 ) ?
51
ω-notation
I ω(g (n)) “little-omega of g of n”
52
ω-notation

I We use this notation to denote a lowerbound which is not
52
ω-notation

I Tight bound:2n2 = Ω(n2 )
52
ω-notation

I Tight bound:2n2 = Ω(n2 ) Loose bound:2n2 = Ω(n).
52
ω-notation

52
ω-notation

I Is 2n2 + 1 = ω(n2 ) ?
52
ω-notation

I Is 2n2 + 1 = ω(n2 ) ?
I Is 2n2 + 1 = ω(n) ?
52
Comparing Functions
I Is the following True?

f (n) = o(g (n)) if and only if g (n) = ω(f (n))
53
Common Mathematical Functions
I alogc b = b logc a , where a > 0 and b > 0

Let k = logb a, then a = b k
54
Common Mathematical Functions
I alogc b = b logc a , where a > 0 and b > 0

Let k = logb a, then a = b k
I Go through section 3.2 of the textbook.
54

Chapter 1 - 3 PDF

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter 1 - 3 PDF

Uploaded by

Copyright:

Available Formats

1

Data Structures and Algorithms

BITS-Pilani K. K. Birla Goa Campus

I An algorithm is correct if it halts for every input with the

I An algorithm is correct if it halts for every input with the

I An algorithm is correct if it halts for every input with the

I An algorithm is correct if it halts for every input with the

I An algorithm is correct if it halts for every input with the

I An algorithm is correct if it halts for every input with the

I There can be more than one algorithm that correctly solves a

I There can be more than one algorithm that correctly solves a

I There can be more than one algorithm that correctly solves a

I Two algorithms for the sorting problem: Insertion sort and

I Two algorithms for the sorting problem: Insertion sort and

I Two algorithms for the sorting problem: Insertion sort and

I Two algorithms for the sorting problem: Insertion sort and

I Two algorithms for the sorting problem: Insertion sort and

I Can using a faster computer solve the problem of coming up

I Can using a faster computer solve the problem of coming up

I Can using a faster computer solve the problem of coming up

I Can using a faster computer solve the problem of coming up

I Can using a faster computer solve the problem of coming up

I Can using a faster computer solve the problem of coming up

I Can using a faster computer solve the problem of coming up

I For each function f (n) and time t in the following table,

I Convention used : Array index starts from 1.

I Convention used : Array index starts from 1.

I We will use pseudocode to describe the insertion sort

I We will use pseudocode to describe the insertion sort

I Loop invariant: The subarray A[1..j − 1] contains elements

I Loop invariant: The subarray A[1..j − 1] contains elements

Loop invariant: The subarray A[1..j − 1] contains elements

Loop invariant: The subarray A[1..j − 1] contains elements

Loop invariant: The subarray A[1..j − 1] contains elements

I Initialization, Maintenance, Termination

I We will assume a generic one-processor, random-access

I We will assume a generic one-processor, random-access

I We will assume a generic one-processor, random-access

I Running time of an algorithm is expressed as a function of the

I Running time of an algorithm is expressed as a function of the

I Running time of an algorithm is expressed as a function of the

I Running time of an algorithm is expressed as a function of the

I Running time of an algorithm is expressed as a function of the

I Running time of an algorithm is expressed as a function of the

I What will be the value of tj if the input sequence is already

I What will be the value of tj if the input sequence is already

I What will be the value of tj if the input sequence is already

I The running time is of the form an + b, where a and b are

I What will be the value of tj if the input sequence is already

I The running time is of the form an + b, where a and b are

I When will the worst case scenario occur?

I When will the worst case scenario occur?

I When will the worst case scenario occur?

I Worst case running time is of the form an2 + bn + c

I Worst case running time is of the form an2 + bn + c

I What will the average case look like?

I What will the average case look like?

I What will the average case look like?

I What will the average case look like?

I We will make simplifying abstractions because we are only

I We will make simplifying abstractions because we are only

I We will make simplifying abstractions because we are only