Order and Rates of Convergence

Numerical Analysis
Math 465/565
1
Saturday, September 14, 13

Speed of convergence
We now have two algorithms which we can compare bisection and the fixed-point method. Which is faster?
Hard to answer : Depends on what interval we start
with, how close to a root we start with, etc. So for any
two particular instances one method might converge in
fewer iterations than the other.
How can we qualify more generally which method
might be better in some sense?
2
Saturday, September 14, 13

Review of bisection
We showed that bisection has the property that
|xk

|b

x
|

a|
2k

If we define the error as ek = |xk

ek |b

a| 2

x
|, we can write
k

Or, bisection behaves like the sequence

k

=2

k
3

Saturday, September 14, 13

How fast does an algorithm converge?

Suppose that an algorithm produces iterates that
converge as
lim xk = x

k!1

If there exists a sequence k that converges to zero and

a positive constant C, such that
ek = |xk

x
| C|

k|

then xk is said to converge with rate

k.
4

Saturday, September 14, 13

Convergence rate - Bisection

|xk

x
| |b

1
a| k
2
1
2k

So the bisection method has a convergence rate of

with |b a| as the asymptotic convergence constant.

rate 2

5
Saturday, September 14, 13

Approximate rate of convergence

log(ek ) =

log(2)k + log(C)

Using the Matlab command polyfit, we can get a

best fit line to the data
ak + b = log(ek )

6
Saturday, September 14, 13

Approximate rate of convergence

log(ek ) =

log(2)k + log(C)

Using the Matlab command polyfit, we can get a

best fit line to the data
ak + b = log(ek )
>> k = 1:49;

6
Saturday, September 14, 13

Approximate rate of convergence

log(ek ) =

log(2)k + log(C)

Using the Matlab command polyfit, we can get a

best fit line to the data
ak + b = log(ek )
>> k = 1:49;
>> ab = polyfit(k,log(e),1)

6
Saturday, September 14, 13

Approximate rate of convergence

log(ek ) =

log(2)k + log(C)

Using the Matlab command polyfit, we can get a

best fit line to the data
ak + b = log(ek )
>> k = 1:49;
>> ab = polyfit(k,log(e),1)
ab =

6
Saturday, September 14, 13

Approximate rate of convergence

log(ek ) =

log(2)k + log(C)

Using the Matlab command polyfit, we can get a

best fit line to the data
ak + b = log(ek )
>> k = 1:49;
>> ab = polyfit(k,log(e),1)
ab =
-0.692013993991203

0.046941635188405

6
Saturday, September 14, 13

Approximate rate of convergence

log(ek ) =

log(2)k + log(C)

Using the Matlab command polyfit, we can get a

best fit line to the data
ak + b = log(ek )
>> k = 1:49;
>> ab = polyfit(k,log(e),1)
ab =
-0.692013993991203

0.046941635188405

>> log(2)

6
Saturday, September 14, 13

Approximate rate of convergence

log(ek ) =

log(2)k + log(C)

Using the Matlab command polyfit, we can get a

best fit line to the data
ak + b = log(ek )
>> k = 1:49;
>> ab = polyfit(k,log(e),1)
ab =
-0.692013993991203

0.046941635188405

>> log(2)
ans =

6
Saturday, September 14, 13

Approximate rate of convergence

log(ek ) =

log(2)k + log(C)

Using the Matlab command polyfit, we can get a

best fit line to the data
ak + b = log(ek )
>> k = 1:49;
>> ab = polyfit(k,log(e),1)
ab =
-0.692013993991203

0.046941635188405

>> log(2)
ans =
0.693147180559945
6
Saturday, September 14, 13

Fixed point method

Suppose over the interval [x0 , x0 + ], a function
g(x) satisfies a Lipschitz condition, and furthermore,
x0 is sufficiently close so that a sequence
xk+1 = g(xk ),
converges to a root x
in [x0
ek = |xk
0

where |g (x)|

k = 0, 1, 2, . . .
, x0 + ]. Then
x
|

< 1.

7
Saturday, September 14, 13

Convergence of fixed point iteration

So the convergence rate is

where |g (x)|

< 1.

8
Saturday, September 14, 13

Estimated rate for fixed point

log(ek ) = log( )k + log ()

Convergence rate

1
k
20

9
Saturday, September 14, 13

Estimated rate for fixed point

log(ek ) = log( )k + log ()
>> ab = polyfit(kvec,log(e),1)

Convergence rate

1
k
20

9
Saturday, September 14, 13

Estimated rate for fixed point

log(ek ) = log( )k + log ()
>> ab = polyfit(kvec,log(e),1)
ab =

Convergence rate

1
k
20

9
Saturday, September 14, 13

Estimated rate for fixed point

log(ek ) = log( )k + log ()
>> ab = polyfit(kvec,log(e),1)
ab =

Convergence rate

-2.9148

0.3012

1
k
20

9
Saturday, September 14, 13

Estimated rate for fixed point

log(ek ) = log( )k + log ()
>> ab = polyfit(kvec,log(e),1)
ab =

Convergence rate

1
k
20

-2.9148

0.3012

>> exp(ab(1))

9
Saturday, September 14, 13

Estimated rate for fixed point

log(ek ) = log( )k + log ()
>> ab = polyfit(kvec,log(e),1)
ab =

Convergence rate

1
k
20

-2.9148

0.3012

>> exp(ab(1))
ans =

9
Saturday, September 14, 13

Estimated rate for fixed point

log(ek ) = log( )k + log ()
>> ab = polyfit(kvec,log(e),1)
ab =

Convergence rate

1
k
20

-2.9148

0.3012

>> exp(ab(1))
ans =
0.0542

9
Saturday, September 14, 13

Estimated rate for fixed point

log(ek ) = log( )k + log ()
>> ab = polyfit(kvec,log(e),1)
ab =

Convergence rate

1
k
20

-2.9148

0.3012

>> exp(ab(1))
ans =
0.0542
>> exp(ab(2))

9
Saturday, September 14, 13

Estimated rate for fixed point

log(ek ) = log( )k + log ()
>> ab = polyfit(kvec,log(e),1)
ab =

Convergence rate

1
k
20

-2.9148

0.3012

>> exp(ab(1))
ans =
0.0542
>> exp(ab(2))
ans =

9
Saturday, September 14, 13

Estimated rate for fixed point

log(ek ) = log( )k + log ()
>> ab = polyfit(kvec,log(e),1)
ab =

Convergence rate

1
k
20

-2.9148

0.3012

>> exp(ab(1))
ans =
0.0542
>> exp(ab(2))
ans =
1.3514

9
Saturday, September 14, 13

Comparison of convergence rates

Bisection 2

Fixed point

Which is better?
10
Saturday, September 14, 13

Order of convergence
Suppose we have that
ek+1
|xk+1 x
|
=

lim
=
lim

k!1 |xk
k!1 ek
x
|
Then the convergence of the sequence xk to x
is said
to be of order .

11
Saturday, September 14, 13

Estimating order of convergence

Again, we can write
log(ek+1 ) = log(ek ) + log()
and use a best-fit-line approach to finding , given a
sequence of errors ek .

Both bisection and the fixed point

method are linear methods
Saturday, September 14, 13

12

Estimating order of convergence

Again, we can write
log(ek+1 ) = log(ek ) + log()
and use a best-fit-line approach to finding , given a
sequence of errors ek .
>> ab = polyfit(log(e(1:end-1)),log(e(2:end)),1)

Both bisection and the fixed point

method are linear methods
Saturday, September 14, 13

12

Estimating order of convergence

Again, we can write
log(ek+1 ) = log(ek ) + log()
and use a best-fit-line approach to finding , given a
sequence of errors ek .
>> ab = polyfit(log(e(1:end-1)),log(e(2:end)),1)
ab =

Both bisection and the fixed point

method are linear methods
Saturday, September 14, 13

12

Estimating order of convergence

Again, we can write
log(ek+1 ) = log(ek ) + log()
and use a best-fit-line approach to finding , given a
sequence of errors ek .
>> ab = polyfit(log(e(1:end-1)),log(e(2:end)),1)
ab =
0.9885

-0.8910

Both bisection and the fixed point

method are linear methods
Saturday, September 14, 13

12