Design and Analysis of Algorithms E

FAST-NU, Lahore, Spring 2019

Homework 1
(key)

Step Count Analysis, Asymptotic Notation, Mathematical Induction

Due Thursday January 31, in my office. 5*10+4*10 = 90 pts.

Note: this is a partial solution key for this homework. I have given fully detailed solutions
where I feel the students might have missed some details or that they will learn from my
version of the solution. For other problems, I have simply given the answers that you can
compare with your own answers, and I have added some comments. If you wish to
discuss the details of these problems, ask a question during class or discuss with me in
my office.
Problem 1
Perform a step-count analysis of the following code fragments. The following two steps
are required:
• Express T(n) as a series of terms.
• Show the upper and lower bounds on the series, to find the θ class for T(n).
1. 2.
for (i=n; i>=1; i=i/5)
s++; for (i=1; i<=n; i=i*2)
for (j=1; j<=i; j=j*2)
θ(lgn), because of the following property sum++;
+,-. /
of logs: log ( 𝑥 = +,- ( , the bases of the logs
. θ(lg 0 n); there is a common error students
only differ by constant multiples, hence make when they write θ(lgn0 ) instead of
we can replace log of any base by log base θ(lg 0 n); these are not the same thing.
2 i.e. lg. lgn0 =2lgn= θ(lgn).
3.
4.
for (i=0; i<n; ++i)
{ for (i=n; i>=1; i=i/5)
for ( j=n; j>0; j=j-3) for (j=1;j<=i;++j)
{ sum++; sum++;
} θ(n); here T(n) is a geometric series with ratio
} 1/5, bounded by its largest term, n.

θ(n0 )
5.
//arr is an array of size n+1
for(int k =1; k<=n; k*=2)
for(int j=1; j<=k; j++)
sum = sum + arr [j];
θ(n); again a geometric series bounded by
its largest term.
6. 7.
for (i=1; i<n; i=i*2)
for (j=0;j<n;++j) for( int k=1; k<=n; k++)
for (k=1; k<=n; k=k*2) for(int j=1; j<=pow(2,k); j++)
sum++; sum++;
θ(nlg 0 n); T(n) is a series of lgn terms θ(23 ); geometric series with ratio 2, bounded
where each term is of the form cnlgn by its largest term 23 .

8. 9.

//arr is an array of size n
for(int i=0; i<n; i++){
while(arr[i++]<0 && i<n);
sum++;
} sum++;
θ(n); without the condition in blue above
this algorithm runs infinitely and is
considered incorrect so it has no θ class)
10.
//you may only find the upper bound for this
//part, i.e. the big-Oh class
for(int i=1; i<=n; i++)
for(int j=0; j<n; j=j+i)
O(nlgn)
This generates the following series:
T(n) = cn + cn/2 + cn/3… + cn/n
= c (n+n/2+n/3+…+n/n)
= cn(1+1/2+1/3…+1/n)
This is the Harmonic Series. I explained in
class how it is bounded from above by a
constant multiple of lgn.

//you may only find the upper bound for this

//part, i.e. the big-Oh class
for(int i=1; i<=n; i++)
for(int j=1; j<=i; j++)
for(int k=1; k<=i*i; k++)
sum++;

O(n8 )
Let’s talk about the inner two loops only (i.e. loop 2 and 3). Loop 2 tells us how many
terms are in the series, and Loop 3 tells the actual terms. The outer loop tells us how
many series are there. We can arrange this in the following fashion (each column is one of
the series and there are n such series):
12 + 12 + 12 + 12 + … + 12
+ 22 + 22 + 22 +… + 22
+ 32 + 32 + … +32
+42 + … +42
.
.
.
+n2

It’s easy to see that each of these columns is bounded from above by n3 . There are n
columns (because the outer-most loop iterates n times.) Hence, it’s easy to see that the
whole thing is bounded by n4.

To be more precise, each column is a sum of squares, bounded by the cube of its largest
term. First column bounded by 13, second one by 23, third one by 33, and the last one by n3.

So, T(n) <= 13 + 23 + 33 + … + n3 <= cn4 (obtained simply by replacing each term by the
largest term)

Incidentally, this bound is also tight, i.e. T(n) >= dn4. Can you prove that?

Problem 2
You can discuss these with me.
Give inductive proofs for (1) and (2)
Give direct proof for (3.b.) and inductive proof for (3.a.) and (3.c.)
3(3:;)(3:0)
(1) A set of 𝑛 elements has exactly <
subsets of size 3, for integers 𝑛 ≥ 3.
(2) 3 divides 𝑛? + 2𝑛 for all integers 𝑛 ≥ 1.
(3) Consider the geometric series – where c is a positive real number.
𝑔(𝑛) = 1 + 𝑐 + 𝑐 0 + ⋯ + 𝑐 3
a. Prove that 𝑔(𝑛) = 𝑂(1) if c<1
b. Prove that 𝑔(𝑛) = 𝑂(𝑛) if c=1
c. 𝑔(𝑛) = 𝑂(𝑐 3 ) if c>1
Note: part (3) again makes the same point we made in class that a geometric series
with c >1 or c<1 is bounded from above by constant multiple of its largest term (in fact
this constant multiple is never more than 2).
(1) & (2) are simple review exercises.

Let’s prove (3c) first:

 𝑔(𝑛) = 1 + 𝑐 + 𝑐 0 + ⋯ + 𝑐 3
Base case: 𝑔(0) = 1 ≤ 𝑐, for 𝑐 ≥ 1
Inductive Hypothesis: 𝑔(𝑛) = 1 + 𝑐 + 𝑐 0 + ⋯ + 𝑐 3 ≤ 𝑑. 𝑐 3 for constant 𝑑.
Inductive Step:
Here we need to show that 𝑔(𝑛 + 1) ≤ 𝑑𝑐 3J; for the same constant 𝑑 as in the
inductive hypothesis. The constant cannot be allowed to grow with 𝑛.

Consider,
𝑔(𝑛 + 1) = 1 + 𝑐 + 𝑐 0 + ⋯ + 𝑐 3 + 𝑐 3J; = 𝑔(𝑛) + 𝑐 3J;

By inductive hypothesis,

; ;
𝑔(𝑛 + 1) ≤ 𝑑. 𝑐 3 + 𝑐 3J; = KL + M N 𝑑𝑐 3J;
; ; ; ; L:;
ð 𝑔(𝑛 + 1) ≤ 𝑑𝑐 3J; , if KL + M N ≤ 1, or in other words, M ≥ 1 − L = L
or
L L
ð 𝑑 ≥ L:;, so d can be any constant equal or above the quantity L:;.

For example, if 𝑐 = 5, 𝑠𝑒𝑡 𝑑 = 5/4 etc.

(3a) can be done in a similar way. For (3b), a direct proof that computes the sum suffices.
Problem 3
Find a tight bound (i.e. θ class) for an algorithm for which 𝑇(𝑛) = ∑3YZ[ 𝑘3Y .
By looking at T(n) we notice that it contains exactly 𝑘 terms of the type 3Y . So, there is
one 3; s, two 30 s, three 3? s… and 𝑛 33 s. These terms can be arranged into columns as
below:
31 + 32 + 33 + 34 + … + 3n <= c.3n

+ 32 + 33 + 34 +… + 3n <= c.3n

+ 33 + 34 + … +3n <= c.3n

+34 + … +3n

+ 3n <=c.3n
Note that the each row is a geometric series with ratio 3 (but only the first row is the
complete series.) All these rows are bounded as shown (because of what we know about the
geometric series).

Therefore, T(n) <= c.n. 3n = O(n3n)

Obviously, T(n) >= n3n, because this is just the sum of the last column. So, 𝑇 (𝑛) = Ω(𝑛33 ).

Therefore, 𝑇(𝑛) = θ(𝑛33 )

Problem 4
In each of the following cases indicate whether f = O(g) or f = Ω(g), or both (in which case
f = Θ (g)
f g
;/0
a. 𝑛 𝑛0/? 𝑓 = 𝑂(𝑔)
b. 𝑛! 23 𝑓 = Ω(𝑔)
c. 𝑙𝑔 (𝑛!) 𝑛 𝑙𝑔(𝑛) 𝑓 = 𝜃(𝑔)
3
d. 𝑓 = 𝜃(𝑔)
a 𝑖Y 𝑛YJ;
cZ;

In (a), to show that 𝑓 = 𝑂(𝑔), we must show that there exists a constant c such that 𝑓(𝑛) ≤
𝑐𝑔(𝑛).

We give a direct proof.

Let 𝑛;/0 ≤ 𝑐𝑛0/?
d
3e
ð e ≤𝑐
3f
:;/<
ð 𝑛 ≤𝑐
or, 1/𝑛;/< ≤ 𝑐

For 𝑛 ≥ 1, 1/𝑛;/< ≤ 1, because with increasing 𝑛 the denominator 𝑛;/< increases hence
1/𝑛;/< decreases (i.e. stays below 1). Hence we set c=1. This is a constant for which,
𝑛;/0 ≤ 𝑐𝑛0/? .

ð 𝑛;/0 ≤ 𝑛0/? , for 𝑛 ≥ 1

e
ð 𝑛;/0 = 𝑂(𝑛f )
Solve (b) in a similar way.
(c) is important for us in a future problem, so let me solve it here.
𝑓 (𝑛) = 𝑙 𝑔(𝑛!) = 𝑙 𝑔(1 ∗ 2 ∗ 3 ∗ 4 … ∗ 𝑛) ≤ 𝑙 𝑔(𝑛 ∗ 𝑛 ∗ 𝑛 ∗ 𝑛 … ∗ 𝑛) = 𝑙𝑔𝑛3 = 𝑛𝑙𝑔𝑛
ð 𝑓(𝑛) ≤ 𝑛𝑙𝑔𝑛
ð 𝒇(𝒏) = 𝑶(𝒏𝒍𝒈𝒏)
On the other hand,
 𝑓(𝑛) = 𝑙 𝑔(𝑛!) = 𝑙 𝑔(1 ∗ 2 ∗ 3 ∗ 4 … ∗ 𝑛)
𝑛 𝑛 𝑛
K ∗ K + 1N ∗ K + 2N ∗ … 𝑛N 𝑖𝑔𝑛𝑜𝑟𝑖𝑛𝑔 𝑡ℎ𝑒 𝑓𝑖𝑟𝑠𝑡 ℎ𝑎𝑙𝑓 𝑚𝑢𝑙𝑡𝑖𝑝𝑙𝑒𝑠
≥ 𝑙𝑔 2 2 2
𝑛 𝑛 𝑛 𝑛 𝑛
≥ 𝑙 𝑔 K ∗ ∗ ∗ … N 𝑟𝑒𝑝𝑙𝑎𝑐𝑖𝑛𝑔 𝑒𝑎𝑐ℎ 𝑚𝑢𝑙𝑡𝑖𝑝𝑙𝑒 𝑏𝑦 𝑡ℎ𝑒 𝑠𝑚𝑎𝑙𝑙𝑒𝑠𝑡 𝑜𝑛𝑒
2 2 2 2 2
3
𝑛 0 𝑛 𝑛 𝑛 𝑛 𝑛 𝑛
= lg K N = lg K N = (𝑙𝑔𝑛 − 𝑙𝑔2) = (𝑙𝑔𝑛 − 1) = 𝑙𝑔𝑛 −
2 2 2 2 2 2 2
𝑛 𝑛
≥ 𝑙𝑔𝑛 − 𝑙𝑔𝑛, 𝑓𝑜𝑟 𝑛 ≥ 4 (𝑣𝑒𝑟𝑖𝑓𝑦 𝑡ℎ𝑖𝑠 𝑏𝑦 𝑖𝑛𝑠𝑝𝑒𝑐𝑡𝑖𝑜𝑛)
2 4
3
= 8 𝑙𝑔𝑛
;
ð 𝑓(𝑛) ≥ 8 𝑛𝑙𝑔𝑛, 𝑓𝑜𝑟 𝑛 ≥ 4
ð 𝒇(𝒏) = 𝛀(𝒏𝒍𝒈𝒏)
Hence, 𝒍𝒈(𝒏!) = 𝛉(𝒏𝒍𝒈𝒏) 𝑓𝑜𝑟 𝑛 ≥ 4
For (d): it is like the last column in the picture in problem 1.10, but instead of power 2
we have power k. You should see why f is tightly bounded by g in that case.
Problem 5
During analysis of algorithms, we often make the assumption that the size of the input 𝑛
is an exact power of an integer such as 2, i.e. 𝑛 = 2Y for some 𝑘 ≥ 0.
We say that this is a safe assumption for asymptotic analysis, but why? Why does it not
affect our analysis? After all, in practical scenarios 𝑛 will mostly not be an exact power?
Give a mathematical argument to make your point.
n is always a positive integer. Any positive integer is either a power of 2, i.e. 𝑛 = 2Y for
some 𝑘 ≥ 0, or it lies between two powers of 2, i.e. it is no more than 2 times away from
a power of 2, i.e.
2Y:; < 𝑛 < 2YJ; , 𝑓𝑜𝑟 𝑠𝑜𝑚𝑒 𝑘 ≥ 1. Hence, our assumed value of n is only off by a factor
of 2 at max. Hence, for any positive integer 𝑛, 𝑛 = 𝜃(2Y ) for some 𝒌 ≥ 𝟎 . The
asymptotic analysis remains unaffected.
THE END