How to solve recurrences

The substitution method

The iteration method

A mathematical framework
1. Homogeneous recurrences
2. Inhomogeneous recurrences
3. Asymptotic recurrences

1
The substitution method

Guessing the form of the solution

Using the mathematical induction to show that the solution works

2
The substitution method: an example
We’d like to solve .

 



 








We guess .


 


 




We prove by induction that there exists a constant such that



 



for sufficiently large .





 
















 

 















3
The substitution method: subtleties
We’d like to solve .

 



 









We guess .


 


 




We try to prove by induction that there exists a constant such that


for sufficiently large .


 









 





























We really need to show that .

 





4
 such that



!
.

 "
!






We prove by induction that there exists a constant








 
  
 $




 !
"
for sufficiently large .

!
 

5
 "
!  

"


 



 ! !
  " "




  
 

. 
#
 
  
  



We’d like to solve


!

 
 "




The trick

We guess

 


The iteration method

Expand (iterate) the recurrence

Express the function as a summation of terms dependent only on


and the initial condition.

6
 




%

%


 



 

 '




% 


% 




%
.

 
   





   

 &

7

The iteration method: an example

   


 
 


  




  

   
    
 

    
 
     

We’d like to solve

    

 



. We know that


 
5

4 32
,-
*+ /1 
0
. or exceeds

.
. 
.
9 32
) %
/1

 
& 5

8

( 

 
 
( 

  


  
8
. So we have:
 6
(7  
 
The iteration hits 1 when


  
   

(
 







(





A mathematical framework to solve a series of recurrences

Technique of the characteristic equation

Homogeneous recurrences

Inhomogeneous recurrences

9
Homogeneous recurrences
We are interested in homogeneous linear recurrences with constant
coefficients of the form

 ?
>

>
.
.
.
:
;<48

:
; =4

:
; 4?

@


=
.

This recurrence is
linear (no degree of 2 or higher terms)

homogeneous (the combination equals to zero)

with constant coefficients ( all ’s are constants)
:
(

Example: .
"

"


 B
>

>

>

>
DCB

@
=

=
4

4
A

10
One property of homogeneous recurrences
Given ,
 ?
>

>
.
.
.
:
; 48

:
; =4

:
; ?4

@


=

if and satisfy the equation, then is also a solution


;4

4

E4

4

4
A

,
for any constants and .


Proof.

11
Solving homogeneous recurrences
Given . Guess for an

4 F
 ?


>

>
.
.
.
:
; 48

:
; =4

:
; ?4

;4
@


=
unknown constant .
F

We have
>

>
= 
4 F

4 F

4 F

? 
.
.
.
:

@


?
8

Ignoring solution , the equation is satisfied if and only if

F

It is called the characteristic

>
= 
? 

?
 

 ?
.
.
.
F

@
=
8
G


equation of the recurrence.

Let be the roots of . We conclude that

4

.
.
.
H

;4


H
=

(


G



(
=
(7
for any constants .

(

This is the only solution when the roots are distinct.

I

12
 P
. The characteristic equation is

B
> O
=
B


)M P 
  B
@ NO
H =

 ; =
 +
. P 
 B
B > O
= 

.
>  S

13
.
A 
4 RQ

 B P4  =
H A

= @ B "

and
and
> O
> =
 
A  B B  B
 4 P B
>   
J K L B

and
A NO
4 =  =
"  
 = P4 S
A >  =
4 B
H NO @ RQ

A @ =

whose roots are

4  
"
 8
 = A  =
A   
4

Example

We know
"

We have

We have
F

" A
4
B

So
F
 and thus a

`
h\
V X
l

F

b j U
r
4
Z is a solution to
\
^
`
h\
a
a
V ^a
Z Z _
\ \ `
^ ^ h
X V X
X
a a k

14
a a
^a ^a V U
_ _ f H
4
` `
X ` b j U 
h

. So
h\ _
\]V _ V U V_ Z
f Z WVU
Z
^ @
deX deX X
^ c ^ c h

solution to the recurrence.

h V U
V h  

\]V [ V[ j[ m g
n H
Z WbV U Z WbV U Z 
V
p o 4
q
Y Y Y Y Y Y

It is easy to show
Multiple roots WVU
X
WVU
X
VWU
X
T h h
g i
Multiple roots
In general, if are the distinct roots of the characteristic
.
.
.
H

H/

*
=

B


polynomial and their multiplicities are , then

.
.
.
s

s
=

/



>
=
t
/

4v

;4


 v

H
(

(
7 v
=
(7

15
 . Applying the initial conditions, we obtain
.


"
=
>
 4 
H 
+ B "
 =
 
N
@ 
"
4 
F
 
 


" ;4

. Therefore
>9
F

;4


16

B 
> "
F

The characteristic polynomial is

w 


;w 4
4  
" "
= B  
Example: multiple roots

F 
> 9
S 
 
;S 4 "  9

and

J L 4 B
K 9 F 

 
 B
;4 4 = 
 

"

;4

So
 =


.
Inhomogeneous recurrences
Consider the following form

4 !
 ?
>

>
.
.
.
:
; 48

:
; =4

:
; 4?




=



where is a constant and is a polynomial in of degree .
!



E
G

Example: .
4
"

 =
>

;4

;4

We can transform it to a homogeneous recurrence, .

"

>

>
;4

;S 4

;% 4
=

B
17
Inhomogeneous recurrences: a general form
Consider the following generalization

4 !

4 !
 ?


>

.
.
.

>

.
.
.
:
; 48

:
; =4

:
; ?4

 


 


=




=

B
G


=

B
where is a constant and is a polynomial in of degree .
!

 

E
(

(
G

The characteristic polynomial is

>
=

=
?

yz

N
u
"
.
.
.


x 

.
:

!



=

(
8


18
Inhomogeneous recurrences: an example



@

@


;4

4 
K

>

;4

+
;

H

)M

=


L

Rewrite the recurrence as

=
4 

4 

8
"

 =
>

;4

;4



So and . The characteristic
 =

 

 B

 



!



!

 

=

B

G

polynomial is
B

B
"

"

"

"

"
 



F














All solutions to the recurrence has the form
4 =

4 

4 

4 

;4


B

-




9

19
Substitute it into the original recurrence, which gives
4 

4 
"

"


 



B

-



B

=

. We obtain , and thus .
4 




-

;4

 
5

20
 is a power of 2.

B 
,
*
. So



9
. So


9
 
 "  ,
when

*
F 


 B 
( 
 , B  
 ,  

* *+ 9
( =
 
 "
  B
( 
 F
 
)  

Substitute the original recurrence, we get

= 


> 


21
 
 (
=
, then

"

n is a power of 2).



  >

. This is
( F


 ;

The characteristic polynomial is

  
   

 
(
 


( B 
)
Consider the recurrence
  ;
( 
9
  (
)
{
Change variables

( 

) B 
B ,
 (  B  *
; >  

( (
; (  5


" =
 
( 


Let
;  
 (


;
C
Range transformations
Consider the following recuurence where n is a power of 2.









 


 

B
K

 


+
;

H

)M


L

Let denote .
(


 
;
(

>
B

B
(

( ;
 (

 

 


 

 


;

>
=
(
Let denote .
o

*
; ,
(

B
( ;
 (

 (

 

 =
>

>
 


o

*
; ,

)

*
; ,

)

o
>

=
(

(
,

=
(

We solve this inhomogeneous equation.

22
Master Theorem: a simple version
Let be an eventually nondecreasing function such that
N

|

?
 

 

 






$


8
when is an exact power of . The constants , and




!



8

8

are all integers. is a positive real number.
I


@


We have
?

?
J € €

 

!
5

#
A

?

?
*


 



 

*+

 ,

!
5

A

K € €

/1

?
2
 

!
5

$
A

L

23
Asymptotic recurrences
Consider a functiin such that

N
|


 


 


 


 




for all sufficiently large n, where and are constants, and


*

!



for some . We conclude that
?
 

 

I
5

@



A

?
J € €

 


!
5

#
A

?

?
*


 


 

‚


ƒ


!
5

A
„


K € €

†
… 

‡
/

?
ˆ

!
5

$
A

L

24
Master Theorem
Let and be constants, let be a function, and be


*

 

 
$



defined on the nonnegative integers by the recurrence


 


 


 


 



A


where we interpret be either or . The can be bound



 


asymptotically as follows.
1. If for some constant , then
>
/1

‰
2
 
 

 


Š


@
$
A

.
/1

/
2
 


 

 
5

2. If , then .
/1

/1

/
2

2
 

 



 

 

 


 


*
 ,
5

5
A

3. If for some constant , and if 

/1

‰
N
2
 

‹
 

 

@
$
A

for some constant and all sufficiently large ,


*






#
A

A


then .
 




 
5

A





25
 .
.
 ,
 *


 


 


26

 
&


 
 
 

 


Examples