Complete

Numerical Methods
Claus Führer
Lund University
claus@maths.lth.se
Lund, Jan/Feb. 2008
C. Führer: FMN081/FMN041/NUMA12-2008
Unit 0: Preface
• These notes serve as a skeleton for the course. They document together
with the assignments the course outline and course content.
• All references in the notes refer to the textbook by Süli and Mayers if
not otherwise stated.
• The notes are a guide to read the textbook. They are no textbook.
• Please report missprints or your comments to the teacher.
1
Unit 1: Basic Iterative Schemes in 1D
Problem: 0.5
Given: f : R → R
Find: x ∈ R such that f (x) = 0. 0
x is called a root or a zero of f .
-0.5
0 5 10
2
1.1 Existence
Theorem. [1.1]
f : [a, b] ⊂ R → R, continuous and f (a)f (b) < 0,
i.e. they have different signs
Then
There exists ξ ∈ [a, b] with f (ξ) = 0 .
Repeat the definition of multiplicity of a root!

Note, Roots with even multiplicity cannot be detected by this theorem.
3
1.2 Example
10
Double Root
5
-5
Single Root
-10
0 5 10
The algorithm based on this theorem is the bisection method.

(Intervallhalveringsmetoden). See also MATLAB’s fzero .
4
1.3 Fixed Point Formulation
We rewrite the problem
f (x) = 0 ⇔ x = g(x)
x is called a fixed point of g.
Examples:
g(x) = x − f (x)
g(x) = x + αf (x)
5
1.4 Brower’s Fixed Point Theorem
Theorem. [1.2] (Brower’s Fixed Point Theorem)

g : [a, b] ⊂ R → R, continuous and g(x) ∈ [a, b],
i.e. g maps [a, b] into itself
Then
There exists ξ ∈ [a, b] with g(ξ) = ξ
6
1.5 Example
Root-finding problem: 0 = exp(x) − 2x − 1

Fixed-point problem: x = ln(2x + 1)
ln(2x+1)
8 2.5
exp(x)-2x-1
6
2
4
1.5
2
1
0
-2 0.5
0.5 1 1.5 2 2.5 0.5 1 1.5 2 2.5
7
1.6 Fixed Point Iteration
Definition. [1.1] The iteration
xk+1 = g(xk )
is called fixed point iteration or simple iteration.
If it converges, then to a fixed point of g:
ξ = lim xk+1 = lim g(xk ) = g( lim xk ) = g(ξ)

k→∞ k→∞ k→∞
8
1.7 Example
>> x(1)=1;for k=1:17, x(k+1)=log(2*x(k)+1);end

>> x
x =
Columns 1 through 10
1.0000 1.0986 1.1623 1.2013 1.2246 1.2381 1.2460 1.2504 1.2530 1.25
1.2553 1.2558 1.2561 1.2562 1.2563 1.2564 1.2564 1.2564

9
1.8 Contractions
Definition. [1.2] A function g is called a contraction (or contractive) on

[a, b]
if there exists a constant L < 1 such that
|g(x) − g(y)| ≤ L|x − y| ∀x, y ∈ [a, b]
The intervall [x, y] shrinks to [g(x), g(y)].
If g is differentiable, then g 0(x) < 1 for all x ∈ [a, b].

10
1.9 Contraction Mapping Theorem
Theorem. [1.3] g : [a, b] ⊂ R → R, continuous and g(x) ∈ [a, b],

g contractive in [a, b]
Then
• g has a unique fixed point in [a, b]
• the iteration xk+1 = g(xk ) converges
11
1.10 Example
g(x) = ln(2x + 1)
By mean value theorem (medelvärdessats)
|g(x) − g(y)| ≤ |g 0(y)||x − y|
and g 0(x) = 2/(2x + 1). g 0(x) is a decreasing function.

Thus maxx∈[1,2] |g 0(x)| = |g 0(1)| = 2/3
We set L = 2/3.
12
1.11 Speed of convergence
Let’s check the speed of convergence:
|xk+1 − xk | = |g(xk ) − g(xk−1)| ≤ L|xk − xk−1|
Thus
|xk+1 − xk |
≤L
|xk − xk−1|
13
1.12 Example: Speed of convergence
>> diff(x(2:end))./diff(x(1:end-1))
ans =
0.6457 0.6134 0.5946 0.5838 0.5776 0.5740 0.5720 0.5709 0.5702 0.56
0.5696 0.5695 0.5694 0.5694 0.5694 0.5694
14
1.13 Error bounds
We get
|xk − ξ| = |g(xk−1) − g(ξ)|
≤ L|xk−1 − ξ|
≤ L(|xk−1 − xk | + |xk − ξ|)
and consequently:
L
|xk − ξ| ≤ |xk − xk−1|
1−L
This is called an a-posteriori estimate.

15
1.14 Error bounds (Conts.)
Analogously we can derive an a-priori bound
Lk
|xk − ξ| ≤ |x1 − x0|
1−L
How many iterations do we need in our example for an accuracy of

|xk − ξ| ≤ 10−8?
See also Th. 1.4 in the book.
16
1.15 Rate of Convergence
Definition. [1.4] Assume limk→∞ xk = ξ.

• Linear convergence, if
k+1
|xk − ξ| < k and lim = µ with µ ∈ (0, 1)
k→∞ k
• Superlinear convergence if µ = 0.
• Sublinear convergence if µ = 1
• Asymptotic rate of convergence ρ = − log10 µ
17
1.16 Rate of Convergence (Cont.)
ρ large ⇒ fast (linear) convergence

ρ small ⇒ slow (linear) convergence
Check example on overhead 1.13 again.

18
1.17 Roots of a function
Example (p.17): f (x) = ex − x − 2 = 0
3
exp(x)-x-2
2
1
x
2
0
x
1
-1
-4 -2 0 2
19
1.18 Roots of a function Cont.
ex − x − 2 = 0 ⇔ x = ln(x + 2) ⇔ x = ex − 2 ⇒ x = x(ex − x)/2
2 4 4
ln(x+2) exp(x)- x(exp(x)-
1 2 x)/2
x 2
2 2 X
0 2 x
2
0
-1 0
X -2 extra
-2 1 x
x 1 fixpoint
1 -2
-3 -4
-2 -1 0 1 2 -3 -2 -1 0 1 2 -2 -1 0 1 2
20
1.19 Relaxation
We construct now systematically g from f .
A first attempt is
Definition. [1.5] A relaxed version of a fixed point iteration with f is
xk+1 = xk − λf (xk )
with λ 6= 0.
Let ξ fullfill f (ξ) = 0. We select λ such that the relaxed iteration converges
fast to ξ, if x0 is near ξ.
21
1.20 Relaxation (Cont.)
Iteration function: g(x) = x − λf (x)

Contraction is determined by max g 0(x) in a neighborhood of ξ.
g 0(x) = 1 − λf 0(x)
Some (loose) concepts:
• λ should have the same sign as f 0(x), why?
• x0 should be sufficiently near ξ
• λ not too large

22
1.20 Relaxation (Cont.)
Theorem. [1.7]
Let f and f 0 be continuous. f (ξ) = 0 and f 0(ξ) 6= 0.
Then
There exists real numbers λ and δ such that the relaxed iteration converges
to ξ for all x0 ∈ [ξ − δ, ξ + δ].
How to find such a λ, which also guarantees fast convergence?

23
1.21 Newton’s Method
We modify the relaxed iteration to
xk+1 = xk − λ(xk )f (xk )
1
and make the optimal choice λ(x) = f 0 (x) . This leads to
Definition. [1.6]
Newton’s method for solving f (x) = 0 is defined by
f (xk )
xk+1 = xk − 0 .
f (xk )
24
1.22 Newton’s Method (Cont.)
Newton's method
0 x2 x1 x
0
Tangent: y = f 0(xk )(x − xk ) + f (xk )

25
1.23 Newton’s Method Convergence
Definition. [1.7] Assume limk→∞ xk = ξ.
• The iteration converges with order q iff
k+1
|xk − ξ| < k and lim q =µ
k→∞ k
• If q = 2 it converges quadratically
26
1.24 Newton’s Method Convergence (Cont.)
Newton’s method converges locally (as it is a fixed point iteration with a

locally contractive function).
It converges also quadratically:
Theorem. [1.8] f, f 0, f 00 continuous in Iδ := [ξ − δ, ξ + δ], f (ξ) = 0 and
f 00(ξ) 6= 0.
If there exists a constant A with
|f 00(x)|
≤ A ∀x, y ∈ Iδ
|f 0(y)|
and x0 ∈ Ih := [ξ − h, ξ + h] with h = min(1/A, δ), then Newton’s method

converges quadratically to ξ.
27
1.25 Variants of Newton’s Method
Secant Method

xk − xk−1
xk+1 = xk − f (xk )
f (xk ) − f (xk−1)
Simplified Newton’s Method
f (xk )
xk+1 = xk − 0
f (x0)
28
Ö1: Exercise Notes
We consider the problem
f (x) = e−x − x = 0
and solve it first by MATLAB’s function fzero:
[ x , f x , e x c i t , d i a g n ]= f z e r o (@( x ) exp(−x)−x , [ 0 . 1 , 2 ] )
Note here the use of anonymous functions in MATLAB.

(see also http://www.mathworks.com/access/helpdesk/help/techdoc/index.html?/access/helpdesk/help/techdoc/
matlab_prog/f4-70115.html)
29
Ö1.2: Exercise Notes (Cont.)
Here the result (use format long e !):
x = 5.671432904097838e-01
fx = 1.110223024625157e-16
excit = 1
diagn =
intervaliterations: 0
iterations: 7
funcCount: 9
algorithm: ’bisection, interpolation’
message: ’Zero found in the interval [0.1, 2]’
30
We write now our own bisection code

f u n c t i o n [ r o o t , e x i t , i t ]= m y b i s e c ( myfunc , s t a r t , Tol )
% initialization
a=s t a r t ( 1 ) ; b=s t a r t ( 2 ) ; m a x i t =50; i t =0;
f a=f e v a l ( myfunc , a ) ; f b=myfunc ( b ) ;
i f fa ∗ fb > 0
d i s p ( ’ wrong i n t e r v a l t a k e n ’ )
e x i t =0;
return
end
% iteration
f o r i =1: m a x i t
c=(a+b ) / 2 ;
31
f c=f e v a l ( myfunc , c ) ;
i f fa ∗ fc < 0
b=c ;
else
a=c ;
f a=f e v a l ( myfunc , a ) ;
end
i f b−a < Tol
e x i t =1;
r o o t =(a+b ) / 2 ;
i t =i ;
break
end
end
Check in the MATLAB help pages the use of feval !

32
Here the result (use format long e !):
>> [r,e,it]=mybisec(@(x) exp(-x)-x,[0,1],1.e-6)

r = 5.671429634094238e-01
e =1
it =20
33
How could the code mybisec be improved?
• Avoid maxit and determine during the initialization step the required
number of iterations.
• Silently it was assumed that b > a. This should me checked during

initialization.
• The code should handle the case when the zero is exactly hit.
34
We solve the problem now by fixed point iteration and rewrite it as a fixed
point problem:
x = e−x.
The interval [0.1, 1] is mapped into itself and the mapp is contractive with
Lipschitz constant
L = max |g 0(x)| = e−0.1 = 0.905.
We expect slow convergence of fixed point iteration:
xk+1 = e−xk .
35
In MATLAB
x (1)=0.1;
f o r i =1:30
x ( i +1)=exp(−x ( i ) ) ;
end
36
... which results in
>> format compact

>> x(1:2)
ans =
0.100000000000000 0.904837418035960
>> x(30:31)
ans =
0.567143328627341 0.567143268734953
37
The error can be estimated via the a posteriori estimate

L =0.905
e s t=abs ( x ( end)−x ( end −1))∗ L/(1−L )
which gives est=5.705e−07 whereas the exact error is eps=3.053e−07.
38
To detremine the convergence rate may use the following commands:

r a t e=d i f f ( x ( 2 : end ) ) . / d i f f ( x ( 1 : end −1))
Note the MATLAB operations ./ .* .ˆ (and check their meaning).
39
Unit 2: Linear Systems, Vector Spaces
Definition. [–] A set V is called a linear space or a vector space over R if

there are two operations + and · with the following properties
• v1, v2 ∈ V ⇒ v1 + v2 ∈ V
• v1 + v2 = v2 + v1
• v1 + (v2 + v3) = (v1 + v2) + v3
• There is an element 0V ∈ V with 0V + v = v + 0V = v
• There is an element −v ∈ V with v + (−v) = 0V
40
2.1: Vector Spaces (Cont.)
• α∈R⇒α·v ∈V
• (α + β) · v = α · v + β · v
• α · (β · u) = (αβ) · u
• α · (v1 + v2) = α · v1 + α · v2
• 1·v =v
One can then easily show 0 · v = 0 and −1 · v = −v.

The elements v of V are called vectors, the elements of R scalars.
41
2.2: Example of Vector Spaces
In linear Algebra: R, the nullspace and the column space of a matrix
In this course:
• space of all n × m matrices
• space of all polynomials of degree n: P n
• space of all continuous functions C[a, b]
• space of all functions with a continuous first derivative C 1[a, b]
• ....
42
2.3: Example of Vector Spaces
Not a vector space: The set of all polynomials of degree n which have the
property p(2) = 5.
43
2.4: Basis, Coordinates
One describes a certain element in a vector space by giving its coordinates in

a given basis. (Recall these corresponding definitions from linear algebra).
The number of basis vectors determines the dimension of the vector space.
C[a, b] is a vector space with an infinite dimension, P n has finite dimension.
44
2.5: Norms
First we recall properties of the absolute value (absolut belopp) of a real

number:
v if v ≥ 0
v ∈ R 7→ |v| =
−v if v < 0
• |v| ≥ 0 and |v| = 0 ⇔ v = 0
• |λv| = |λ||v|
• |u + v| ≤ |u| + |v|
45
2.6: Vector Norms
We generalize the definition of the absolute value of a real number to norms

of vectors and matrices (later also of functions):
Definition. [2.6] V a linear space , a function k · k : V → R is called a

norm if for all u, v ∈ V and all λ ∈ R:
• kvk ≥ 0 and kvk = 0 ⇔ v = 0 (Positivity)
• kλvk = |λ|kvk (Homogenity)
• ku + vk ≤ kuk + kvk (Triangular inequality)
46
2.7: Examples
Examples for norms in Rn:
• 1-norm n
X
kvk1 = |vi|
i=1
• 2-norm (Euclidean norm)

n
2 1/2
X
kvk2 = vi
i=1
• ∞-norm
max |vi|
i=1:n
47
2.8: Unit Circle
The unit circle is the set of all vectors of norm 1.
1 inf
1
0
2
-1
-1 0 1
48
2.10: Convergence
Theorem. [-]
If dim V < ∞ and if k · kp and k · kq are norms in V, then there exist
constants c, C such that for all v ∈ V
ckvkq ≤ kvkp ≤ Ckvkq
Norms in finite dimensional spaces are equivalent.

Sequences convergent in one norm are convergent in all others.
49
2.11: Matrix norms
A matrix defines a linear map.
Definition. [2.10]
Let k · k be a given vector norm. The corresponding (subordinate) matrix
norm is defined as
kAvk
kAk = max
v∈Rn \{0} kvk
I.e. the largest relative change of a vector, when mapped by A.

50
2.12: Matrix norms

2 1
Example: A =
1 1
2
A
0
-2
-1 0 1 -1 0 1
kAk = 2.6180
51
2.13: How to compute matrix norms
Matrix norms are computed by applying the following formulas:

Pn
1-norm (Th. 2.8): kAk1 = maxj=1:n i=1 |aij | maximal column sum
Pn
∞-norm (Th. 2.7): kAk1 = maxi=1:n j=1 |aij | maximal row sum
p
2-norm (Th. 2.9): kAk2 = maxi=1:n λi(ATA)
where λi(ATA) is the ith eigenvalue of ATA.
52
2.14: Condition of a Problem
A mathematical problem can be viewed as a function mapping in-data to

out-data (solution):
f :D⊂V→W
Condition number is a measure for the

sensitivity of the out-data with respect to
perturbations in in-data.
Scalar case: y = f (x) and ŷ = f (x + δx):
0 1 00
ŷ − y = f (x + δx) − f (x) = f (x)δx + f (x + θδx)(δx)2
2!
53
2.15: Condition of a Problem (Cont.)
Scalar case, relative:
0

ŷ − y xf (x) δx
= + O(δx2)
y f (x) x
absolut (local) condition num- relative (local) condition num-

ber ber
0
xf (x)
Condx = |f 0(x)| condx(f ) =
f (x)
54
2.16: Condition of a Problem (Cont.)
In general
absolut (local) condition num- relative (local) condition num-

ber ber
kxk kf 0(x)k
Condx = kf 0(x)k condx(f ) =
kf (x)k
and we get
krel.output errork ≤ condx(f )krel.input errork
55
2.17: Examples
Example 1: (Summation)

x1
Problem: f : R2 −→ R1 with 7→ x1 + x2
x2
0 T

Jacobian: f (x1, x2) = 1, 1
In 1-norm: Condx1,x2 (f ) = 1
|x1| + |x2|
condx1,x2 (f ) =
|x1 + x2|
Problem if two nearly identical numbers are subtracted.
56
2.18: Examples (Cont.)
Example 2: (Linear Systems)
Problem: f : R −→ R with b 7→ x = A−1b

n
n
f b = A−1
0

Jacobian:
In 1-norm: Condb(f ) = kA−1k
kbkkA−1k kAkkxkkA−1k −1
condb(f ) = −1
≤ = kAkkA k =: κ(A)
kA bk kxk
57
2.19: Examples (Cont.)
The estimate is sharp, i.e. there is a worst case perturbation
Example:
3

10 0 1 0
A= b= δb =
0 10−3 0 10−5
(see the exercises of the current week and Exercise 2.14 in the book.)
58
Unit 3: Systems of nonlinear functions
Example (p. 107)
2 2

f1(x1, x2) x 1 + x2 − 1
F (x) = = =0
f2(x1, x2) 5x21 + 21x22 − 9
1
f1
0.5
f2 p p
0 ξ1 = (− 3/2, 1/2) ξ2 = ( 3/2, 1/2)T
T
-0.5 p p
-1
ξ3 = (− 3/2, −1/2) ξ4 = ( 3/2, −1/2)T
T
-1 -0.5 0 0.5 1
59
3.1 Fixed Point Iteration in Rn
Definition. [4.1] Let g : D ⊂ Rn → Rn, D closed (cf. p. 105), x0 ∈ D

and g(D) ⊂ D.
The iteration
xk+1 = g(xk )
is called fixed point iteration or simple iteration.
and similar to the scalar case we define

Definition. [4.2] Let g : D ⊂ Rn → Rn, D closed (cf. p. 105).
If there is a constant L < 1 such that
kg(x) − g(y)k ≤ Lkx − yk
then g is called contractive.

60
3.2 Contractivity and norms
Contractivity depends on the choice of a norm.

Here a function, which is contractive in one norm, but not in another

3/4 1/3
g(x) = x
0 3/4
It follows
kg(x) − g(y)k = kA(x − y)k ≤ kAkkx − yk
Thus L = kAk.
13
But kAk1 = kAk∞ = 12 and kAk2 = 0.9350.
g is contractive in the 2-norm and dissipative and the others.

61
3.3 Contractivity and norms (Cont.)
We start a fixed point iteration in the last example with x = (1, 1)T and
get the following norms:
2
different norms
1 of fixed point iterates
1.5
2
1
inf
0.5
0
0 1 2 3 4 5 6 7 8 9 10
(see also ”equivalence of norms”)

62
3.4 Fixed Point Theorem in Rn
Theorem. [4.1] Let g : D ⊂ Rn → Rn, D closed, g(D) ⊂ D.

If there is a norm such that g is contractive, then g has a unique fixed
point ξ ∈ D and the fixed point iteration converges.
Let J(x) be the Jacobian (functionalmatrix → flerdim) of g.
If kJ(ξ)k < 1 then fixed point iterations converges in a neighborhood of ξ.

(Th. 4.2)
63
3.5: The Jacobian
Newton’s method requires first derivatives.

We recall the definition (see calculus in several variables)
Definition. [4.3] Let f : D ⊂ Rn → Rn, x ∈ D.

The n × n matrix
 ∂f ∂f1 ∂f1

1
∂x1 ∂x2 ··· ∂xn
∂f2 ∂f2 ∂f2 
···

Jf (x) = 
 ∂x1 ∂x2 ∂xn 
···

 
∂fn ∂fn ∂fn
∂x1 ∂x2 ··· ∂xn
is called the Jacobian or functional matrix of g at x.

64
3.6: Jacobian: Example
2 2

f1(x1, x2) x 1 + x2 − 1
F (x) = = =0
f2(x1, x2) 5x21 + 21x22 − 9
Then
2x1 2x2
JF (x) =
10x1 42x2
65
3.7: Jacobian: Numerical Computation
Often the Jacobian is not analytically available and it has to be computed

numerically.
It can be computed column wise by finite differences
f u n c t i o n [ J ]= j a c o b i a n ( f u n c , x , f x )
% computes t h e J a c o b i a n o f a f u n c t i o n
n=l e n g t h ( x ) ;
i f n a r g i n ==2
f x=f e v a l ( f u n c , x ) ;
end
eps =1. e − 8; % c o u l d be made b e t t e r
x p e r t u r b=x ;
f o r i =1: n
x p e r t u r b ( i )= x p e r t u r b ( i )+eps ;
J ( : , i )=( f e v a l ( f u n c , x p e r t u r b )− f x ) / eps ;
x p e r t u r b ( i )=x ( i ) ;
end ;
66
3.8: Newton’s method in Rn
Newton’s method for systems of equations is a direct generalization of the

scalar case:
Definition. [4.5] The recursion
x(k+1) = x(k) − JF (x(k))−1F (x(k))

with JF (x) being the Jacobian of F is called Newton’s method.
Note, in order to avoid confusion with the i-th component of a vector,

we set now the iteration counter as a superscript x(i) and no longer as a
subscript xi.
67
3.9: Newton’s method: Implementation remarks
For implementing Newton’s method no matrix inverse is computed (this is

to expansive), we solve instead linear equation systems:
JF (x(k))∆x = −F (x(k))
and compute the next iterate by adding the Newton increment δx:
x(k+1) = x(k) + ∆x
Solving linear systems is done in MATLAB with the \-command - not with
the command inv !!!
68
3.10: Homotopy Method
Finding a good starting value x(0) for Newton’s method is a crucial problem.
The homotopy method (continuation method, successive loading method)

can be used to generate a good starting value.
Assume, that F0 is a known function with a known zero x∗, i.e. F (x∗) = 0
We construct a parameter depending function
H(x, s) = sF (x) + (1 − s)F0(x) s ∈ [0, 1]
and note, that H(x, 0) = 0 is the problem with known solution and
H(x, 1) = 0 is the original problem F (x) = 0.
69
3.11: Homotopy Method (Cont.)
An example for F0(x) is just the trivial way
F0(x) := F (x) − F (x∗)
This gives the homotopy function
H(x, s) = F (x) + (s − 1)F (x∗)

for a given vector x∗.
70
3.12: Homotopy Method (Cont.)
As the solution of H(x, s) = 0 depends on s we denote it by x∗(s).
We discretize now the intervall into 0 = s0 < s1 < · · · < sn = 1 and solve
a sequence of nonlinear systems with Newton’s method
H(x, si) = 0
Each iteration is started with the solution x∗(si−1) of the preceeding

problem.
71
3.13: Homotopy Method: Example
Let’s consider the scalar problem arctan(x) = 0 again.
We saw previously that Newton’s method for this problem fails to converge
if started with |x0| > 1.34.
Assume we have an initial guess x∗ = 4 instead. We set up a homotopy

function
H(x, s) = arctan(x) + (s − 1) arctan(4) = 0
and discretize s by selecting si = i ∗ 0.1, i = 0 : 10
72
3.14: Homotopy Method: Example (Cont.)
x0 =4;
homotop=@( x , s ) atan ( x )+( s −1)∗ atan ( 4 ) ;
h o m o t o p d e r i v=@( x ) 1/(1+ x ˆ 2 ) ;
s=l i n s p a c e ( 0 , 1 , 1 1 ) ;
x a s t=z e r o s ( l e n g t h ( s ) , 1 ) ;
x a s t (1)= x0 ;
f o r i =2:11
[ x a s t ( i ) , i t e r , i e r ]= newtonh ( homotop , h o m o t o p d e r i v , . . .
x a s t ( i − 1 ) , 1 5 , 1 . e −8, s ( i ) ) ;
i f ( i e r ==1)
disp ( ’ divergence ’ )
break
end
end
Note, we had to provide newton with an extra parameter s.

73
3.15: Homotopy Method: Example (Cont.)
The resulting homotopy path
Homotopy path x(s)

4
3
x(s)
2
0 s
0 0.5 1
74
Ö 3.1: Exercise Notes (Homework 2)
For the task to find the equilibrium of the truck we write the following Newton solver in
MATLAB
f u n c t i o n [ x , conv ]= newtonRaph ( fun , x0 , t o l , j a k o b )
% initialization
x ( : , 1 ) = x0 ; xk=x0 ; m a x i t =100; conv =0; n=l e n g t h ( x0 ) ;
E=eye ( n ) ; h =1. e − 10;
f o r i =1: m a x i t % newton−l o o p
f x=f e v a l ( fun , xk ) ;
i f n a r g i n==3 % i f c a l l e d w i t h t h r e e arguments . . .
f o r j =1: n % numerical Jacobian
J ac ( : , j )=( f e v a l ( fun , xk+h ∗E ( : , j )) − f x ) / h ;
end
else
J ac=f e v a l ( j a k o b , xk ) % a n a l y t i c J a c o b i a n
end
d e l t a x=−J ac \ f x ; xk=xk+d e l t a x ; x ( : , i )=xk ;
i f norm ( d e l t a x )< t o l % c o n v e r g e n c e t e s t
conv =1;
break
end
end
75
Ö 3.2: Exercise Notes (Cont.)
We also write a wrapper for the truck-file. This eliminates the econd
parameter without the need of chaning truck acc.m
f u n c t i o n a=t r u c k w r a p ( p )
a=t r u c k a c c ( p , z e r o s ( 9 , 1 ) ) ;
and execute in the MATLAB command window:

>> p0 = [ 0 . 5 2 , 2 . 1 6 , 0 . 0 , 0 . 5 2 , 2 . 6 8 , 0 . 0 , − 1.5 , 2 . 7 , 0 . 0 ] ’ ;
>> [ x , conv ]= newtonRaph ( @ t r u c k w r a p , p0 , 1 . e − 12);
The solution is then:

Columns 1 t h r o u g h 8
0 . 5 0 0 0 2 . 0 0 0 0 0 . 0 0 0 0 0 . 5 0 0 0 2 . 9 0 0 0 0 . 0 0 0 0 − 1.4600 2 . 9 0 0 0
Column 9
0.0000
76
Now let’s test the rate of convergence:

>> f o r i =1:m, nx ( i )=norm ( x ( : , i ) ) ; end
>> d i f f ( nx ( 2 : end ) ) . / d i f f ( nx ( 1 : end − 1))
ans =
0.3309 0.1073 0.0430 0.0006 0.0000 0
This indicates superlinear convergence.
77
Computing Newton-fractals:
Here we plot two pictures, one which depicts the fractals according to Task
2 and another where we plot the numbers of iterations for each start value.
The main m-file looks like this:
% s u p p r e s s w arni ngs c o n c e r n i n g n e a r l y s i n g u l a r m a t r i c e s
warning o f f a l l
[ X , Y ] = meshgrid ( − 1 : . 0 0 5 : 1 , − 2 : . 0 1 : 2 ) ;
f o r i =1: s i z e (Y , 1 )
f o r j =1: s i z e (X , 2 )
[ A( i , j ) ,B( i , j )]= n e w t o n f r a c ( [ X( i , j ) ; Y( i , j ) ] ) ;
end
end
% Which−Root−F r a c t a l ( s e e t a s k 2)
f i g u r e ( 1 ) ; p c o l o r (A ) ; shading i n t e r p
%How−many−I t e r a t i o n s f r a c t a l
f i g u r e ( 2 ) ; p c o l o r (B ) ; shading i n t e r p
78
The Newton method is implemeneted for this task so:
f u n c t i o n [ r o o t , i t ]= n e w t o n f r a c ( x0 )
x=x0 ; r 1 = [ 1 ; 0 ] ; r 2 =1/2 ∗ [ − 1; s q r t ( 3 ) ] ; r 3 =1/2∗[ −1; − s q r t ( 3 ) ] ;
f o r i =1:40
i t =i ;
f x =[ x (1)ˆ3 − 3 ∗ x ( 1 ) ∗ x (2)ˆ2 − 1;3 ∗ x ( 1 ) ˆ 2 ∗ x (2) − x ( 2 ) ˆ 3 ] ;
J ac =[3 ∗ x (1)ˆ2 − 3 ∗ x (2)ˆ2 , − 6 ∗ x ( 1 ) ∗ x ( 2 ) ; . . .
6 ∗ x ( 1 ) ∗ x ( 2 ) , 3 ∗ x (1)ˆ2 − 3 ∗ x ( 2 ) ˆ 2 ] ;
dx=−J ac \ f x ; x=x+dx ;
i f abs ( dx ) < 1. e −7
break
end
end
i f norm ( x−r 1 ) < 1 . e −5
r o o t =1;
e l s e i f norm ( x−r 2 ) < 1 . e −5
r o o t =2;
e l s e i f norm ( x−r 3 ) < 1 . e −5
r o o t =3;
else % no r o o t
r o o t =−1;
end
79
And here the results:
80
Unit 4: Polynomial Interpolation
(see course book p. 180)

We denote (as above) by Pn the linear space (vector space) of all polynomials
of (max-) degree n.
Definition. [–] Let (xi, yi), i = 0 : n be n + 1 pairs of real numbers

(typically measurement data)
A polynomial p ∈ Pn interpolates these data points if
p(xk ) = yk k = 0 : n
holds.
We assume in the sequel that the xi are distinct.

81
4.1: Polynomial Interpolation
An Interpolation Polynomial
5
polynomial (6th degree)
4
0
measurements
-1
-2
0 1 2 3 4 5 6
How do we determine such a polynomial?

82
4.2: Vandermonde Approach
Ansats: p(x) = anxn + an−1xn−1 + · · · a1x + a0

Interpolation conditions
p(xj ) = anxni + an−1xn−1
i + · · · a1xi + a0 = yi 0≤i≤n
In matrix form
xn0 xn−1
    
0 ··· x0 1 an y0
xn
 1 xn−1
1 ··· x1  an−1
1    y1 
  ..  =  .. 
  
 ···
xnn xn−1
n ··· xn 1 a0 yn
or V a = y.
V is called a Vandermonde matrix.
83
4.3 Vandermonde approach in MATLAB
polyfit sets up V and solves for a (the coefficients)
Alternatively vander sets up V and a = V \y solves for a.
polyval evaluates the polynomial for given x values.
n + 1 points determine a polynomial of (max-)degree n.
Obs! n is input to polyfit .
84
4.4 Vandermonde approach in MATLAB
Essential steps to generate and plot an interpolation polynomial:
• Computing the coefficients (polyfit, vander etc)
• Generating x-values for ploting, e.g.

xval=[0:0.1:100] or xval=linspace(0,100,1000)
• Evaluating the polynomial, e.g. yval=polyval(coeff,xval)
• Plotting, e.g. plot(xval,yval)
85
4.5 Lagrange Polynomials
We take now another approach to compute the interpolation polynomial
Definition. [–] The polynomials Lni ∈ P n with the property

0 if k 6= i
Lnk(xi) =
1 if k=i
are called Lagrange polynomials.
86
4.6 Lagrange Polynomials (Cont.)
It is easy to check, that

n
Y (x − xi)
Lnk(x) =
(xk − xi)
i=0
i 6= k
The interpolation polynomial p can be written as

n
X
p(x) = yk Lnk(x)
k=0
Check that it indeed fulfills the interpolation conditions!

87
4.7 Lagrange Polynomials: Example
Lagrange polynomials of degree 3:

5
4
1
3
00
measurements
-1
-2
0 1 0.33
2 3 0.66
4 5 61
88
4.8 The vector space P n
We have two ways to express a polynomial

Pn k
Monomial representation p(x) = k=0 ak x
Pn n
Lagrange representation p(x) = k=0 yk Lk (x)
They describe the same polynomial (as the interpolation polynomial is

unique).
89
4.9 The vector space P n (Cont.)
We introduced two bases in P n:
Monomial basis {1, x, x2, · · · , xn}, coordinates ak , k = 0 : n
Lagrange basis {Ln0 (x), Ln1 (x), · · · , Lnn(x)}, coordinates yk , k = 0 : n
It is easy to show, that these really are bases (linear independent elements).
90
4.10 Inner Product Space
Definition. [9.1] Let V be a linear space and (·, ·) : V × V → R a map

with properties
• (v, v) ≥ 0 and (v, v) = 0 ⇔ v = 0,
• (αv, w) = α(v, w) for α ∈ R,
• (v, w) = (w, v)
• (v + w, u) = (v, u) + (v, u)
then V is called an inner product space and (·, ·) an inner product.
91
4.11 Inner Product Space (Examples)
• Rn is an inner product space with the inner product

n
X
(v, u) = viui = v Tu see scalar product
i=0
• P n is an inner product space with the inner product

n
X
(p, q)xi = p(xi)q(xi) pointwise inner product
i=0
• P n is an inner product space with the inner product

Z b
(p, q)2 = p(x)q(x)dx L2-inner product
a
92
4.12 Inner Product Space - Orthogonality
Definition. [9.2] Let V be an inner product space and let two elements
p, q ∈ V have the property (p, q) = 0, then they are called orthogonal. One
writes p⊥q or p = q ⊥.
Lagrange polynomials
Pn form an orthogonal basis with respect to the inner
product (p, q) = i=0 p(xi)q(xi).
93
4.13 Inner Products and Norms
Inner products induce norms, so-called inner-product norms
kvk := (v, v)1/2
Examples:
R 1/2
b
• In P n: kpk2 = a
p(x)2
dx
2 1/2
n
Pn
• In P : kpkxi = i=0 p(xi )
• In Rn: scalar product and (Euclidean-)length of a vector
But kpk∞ := maxx∈[a,b] |p(x)| is not an inner product norm.

94
4.14 Interpolating functions
Let f : R → R be a function, we try to describe it by a polynomial.
Best approximation of a function:
min kf − pk
p∈P n
The result depends on the norm we choose.
95
4.15 Interpolating functions - Example
Example [6.1, p. 183] f (x) = ex in [−1, 1]. We select x0 = −1, x1 =

0, x2 = 1 and compute the interpolation polynomial
(x − x1)(x − x2) 1 An Interpolation Polynomial

L0(x) = = x(x − 1) 35
(x0 − x1)(x0 − x2) 2 4
1 exp(x)
3
2
(x − x0)(x − x2) 2 2
L1(x) = =1−x p(x)
(x1 − x0)(x1 − x2) 1
1
00
measurements
(x − x0)(x − x1) 1 -1
L2(x) = = x(x + 1) 00
-2
1-0.50.33 0.66 161
(x2 − x0)(x2 − x1) 2 -1 2 03 4 0.5 5
Thus p(x) = 21 x(x − 1)e−1 + (1 − x2)e0 + 12 x(x + 1)e1.

96
4.16 Interpolating functions - Example (Cont.)
Error:
0.0835
4
0.0613 exp(x)
2
2
0.04
1 p(x)
1
0.0200 measurements
-1
00
-2
-1 1-0.50.33
2 03 0.66
4 0.5 5 161
97
4.17 Interpolating functions - Error
Theorem. [6.2] Let f : R → R with n + 1 continuous derivatives in [a, b]

and let xi ∈ [a, b], i = 0 : n. Then there exists a ξ = ξ(x) ∈ [a, b] with
1
f (x) − p(x) = f (n+1)(ξ)(x − x0) . . . (x − xn)
(n + 1)!
This gives the estimate
1
|f (x) − p(x)| = Mn+1(f )|(x − x0) . . . (x − xn)|
(n + 1)!
with Mn+1(f ) = maxx∈[a,b] |f (n+1)(x)|.
98
4.18 Interpolating functions - Error (Cont.)
Consequently
1
kf − pk∞ = Mn+1(f ) max |(x − x0) . . . (x − xn)|
(n + 1)! x∈[a,b]
If there is the possibility to select the xi freely, one can minimize the
interpolation error for a given n.
See Chebyshev polynomials.

99
4.19 Chebyshev polynomials
Definition. [8.2] The functions
Tn(x) = cos(n arccos(x)), n = 0, 1, 2, . . .
are called Chebyshev polynomials in [−1, 1].
They are indeed polynomials: Set δ = arccos(x).

The identity
cos(n + 1)δ + cos(n − 1)δ = 2 cos δ cos nδ
gives Tn+1(x) = 2xTn(x) − Tn−1(x). With T0(x) = 1, T1(x) = x it

becomes obvious that Tn(x) ∈ Pn.
100
4.20 Chebyshev Polynomials (Cont.)
0.081
35
3 (6th degree)
polynomial 4
4
0.06
0.513 exp(x) 1
2
2
0.040 2
1 p(x)
1
-0.500
0.02 measurements
-1
-1
00
-2
-1 1-0.50.33
2 03 0.66
4 0.5 5 161
101
4.21 Chebyshev Polynomials-Properties
Lemma. [8.2] • The Ti have integer coefficients.
• The leading coefficient is an = 2n−1.
• T2n is even, T2n+1 is odd.
• |Tn(x)| ≤ 1 for x ∈ [−1, 1] and |Tn(x)| = 1 for xk := cos(kπ/n).
• Tn(1) = 1, Tn(−1) = (−1)n
2k−1

• Tn(x̄k ) = 0 for x̄k = cos 2n π for k = 1, . . . , n
102
4.22 Minimality Property of Chebyshev Polynomials
Theorem. [-] 1. Let P ∈ Pn([−1, 1]) have a leading coefficient an 6= 0.

Then there exists a ξ ∈ [−1, 1] with
|an|
|P (ξ)| ≥ n−1 .
2
2. Let ω ∈ Pn([−1, 1]) have a leading coefficient an = 1. Then the scaled

Chebyshev polynomials Tn/2n−1 have the minimal property
kTn/2n−1k∞ ≤ min kωk∞

ω
Proof: see lecture

103
4.23 Chebyshev Polynomials and Optimal Interpolation
We apply this theorem to the result on the approximation error of polynomial

interpolation and conclude for [a, b] = [−1, 1]:
The approximation error
1
f (x) − p(x) = f (n+1)(ξ)(x − x0) · · · (x − xn)
(n + 1)!
error is minimal if (x − x0) · · · (x − xn) = Tn+1/2n, i.e. if the xi are roots

of the n + 1st Chebyshev polynomial, so-called Chebychev points
In case of [a, b] 6= [−1, 1] we have to consider the map:
x−a
[a, b] → [−1, 1] x 7→ 2 −1
b−a
104
Unit 5: Cubic Splines
Let K = {x0, . . . , xm} be a set of given knots with
a = x0 < x1 < · · · < xm = b
Definition. [11.2] A function s ∈ C 2[a, b] is called a cubic spline on [a, b],

if s is a cubic polynomial si in each interval [xi, xi+1].
It is called a cubic interpolating spline if s(xi) = yi for given values yi.
105
5.1: Cubic Splines
Interpolating cubic splines need two additional conditions to be uniquely

defined
Definition. [11.3] An cubic interpolatory spilne s is called a natural spline

if
s00(x0) = s00(xm) = 0
106
5.2: Cubic Splines - Construction
We construct an interpolating in a different but equivalent way than in the

textbook:
Ansatz for m the piecewise polynomials
si(x) = ai(x − xi)3 + bi(x − xi)2 + ci(x − xi) + di
By fixing the 4m free coefficients ai, bi, ci, di, i = 0 : m − 1 the entire spline
is fixed.
107
5.3: Cubic Splines-Construction
We need 4m conditions to fix the coefficients
(1) si(xi) = yi, for i = 0 : m − 1,
(2) sm−1(xm) = ym, 1 condition
(3) si(xi+1) = si+1(xi+1), for i = 0 : m − 2,
(4) s0i(xi+1) = s0i+1(xi+1), for i = 0 : m − 2,
(5) s00i (xi+1) = s00i+1(xi+1), for i = 0 : m − 2,

These are 4m − 2 conditions. We need two extra.
108
5.4: Cubic Splines-Boundary Conditions
We can define two extra boundary conditions. One has several alternatives:
Natural Spline s000 (x0) = 0 and s00m−1(xm) = 0
End Slope Spline s00(x0) = y00 and s0m−1(xm) = ym

0
Periodic Spline s00(x0) = s0m−1(xm) and s000 (x0) = s00m−1(xm)
Not-a-Knot Spline s000

0 (x 1 ) = s 000
1 (x 1 ) and s 000
m−2 (x m−1 ) = s 000
m−1 (xm−1 )
We consider here natural splines.

MATLAB uses splines with a not-a-knot condition.
109
5.5: Natural Splines Construction
si(x) = ai(x − xi)3 + bi(x − xi)2 + ci(x − xi) + di

Let h = xi+1 − xi equidistant spacing
s0i(xi+1) = 3aih2 + 2bih + ci

s00i (xi+1) = 6aih + 2bi
From Condition (1) we get di = yi .
We introduce new variables for the second derivatives at xi, i.e.
σi := s00(xi) = s00i (xi) = 6ai(xi − xi) + 2bi = 2bi i = 0 : m − 1
110
5.6: Natural Splines Construction (Cont.)
σi
Thus bi = 2 .
From
σi+1 = 6aih + 2bi.
and Condition (5) we get
σi+1 −σi
ai = 6h .
By Condition (3) we get
yi+1 = aih3 + bih2 + cih + yi.
111
... and after inserting the hight-lighted expressions for ai and bi we get
σi+1 − σi 3 σi 2
yi+1 = h + h + cih + yi.
6h 2
yi+1 −yi 2σi +σi+1

From that we get ci: ci = h −h 6 .
Using now Condition (4) gives a relation between ci and ci+1
ci+1 = 3aih2 + 2bih + ci.
112
Inserting now the expressions for ai, bi and ci, using Condition (2) and
simplifying finally gives the central recursion formula:
yi+1 − 2yi + yi−1

σi−1 + 4σi + σi+1 =6
h2
with i = 1, . . . , m − 1.
We consider now natural boundary conditions
σ0 = σm = 0.
113
Finally we rewrite this all a system of linear equations
    
4 1 σ1 y2 − 2y1 + y0
1 4 1  σ2  y3 − 2y2 + y1 
... = 6  ..
    
 1 1  σ3 
 h2 
.. .
...
  
 1     . 
1 4 σm−1 ym − 2ym−1 + ym−2
First, this system is solved and then the coefficients ai, bi, ci, di are deter-
mined by the high-lighted equations.
114
5.10: Splines in MATLAB
Example:
x = 0:10; y = sin (x );
xx = 0 : . 2 5 : 1 0 ;
yy = s p l i n e ( x , y , xx ) ;
p l o t ( x , y , ’ o ’ , xx , yy )
115
5.11: Splines in MATLAB (Cont.)
Example for just computing the σi

A=d i a g (4 ∗ o n e s (m− 1 ,1))+ d i a g ( o n e s (m− 2,1), − 1)+ d i a g ( o n e s (m− 2 ,1) ,+1);
r h s =6/hˆ2 ∗ d i f f ( d i f f ( y ) )
s i g m a=A\ r h s ’
sigma =[0; sigma ; 0 ]
See also ppval, find, unmkpp, mkpp, ppval.

(The suffix or prefix pp stands for piecewise polynomial.)
Note, alternatively can A also be constructed by the command
toeplitz(r).
116
5.12: Minimality Property of Cubic Splines
Let V ⊂ C 2 functions which interpolate the points (xi, yi) with i = 0 : m.
Theorem. [-]
Let s∗ ∈ V be a cubic spline satisfying a natural boundary condition. Then
ks∗00k2 ≤ kf 00k2 ∀f ∈ V.
117
5.13: Minimality Property of Cubic Splines (Cont.)
Proof: Let f ∈ V , then there is a g ∈ C 2 with g(xi ) = 0 such that f (x) = s∗ (x) + g(x).
We then obtain kf 00 k2 ∗ 00 + gk2 = ks∗ 00 k2 + 2(s∗ 00 , g 00 ) + kg 00 k2 with
2 = ks 2 2
Z xm
∗ 00 00 ∗ 00 00
(s , g ) := s (x)g (x)dx.
x0
00
We have to show that (s∗ , g 00 ) = 0:
Integration by parts gives
ixm Z xm 000
∗ 00 00 ∗ 00 0 ∗ 0
(s , g ) = s (x)g (x) − s (x)g (x)dx.
x0 x0
From the natural boundary conditions follows ixm
∗ 00 0
s (x)g (x) = 0.
x0
As s∗ is a piecewise cubic polynomial we get for the last term
Z xm m Z x m
∗ 000 0 X i 0 X
s (x)g (x)dx = αi g (x) = αi (g(xi ) − g(xi−1 )) = 0
x0 i=1 x i−1 i=1
with some constants αi .
118
5.14: The Space of Cubic Splines
S is a linear space, its dimension depends on the number of knots

x0 < x1 < · · · < xm.
We construct a basis for S:
Definition. [11.3] The positive part of (x − a)n, n > 0 is defined by
(x − a)n, x ≥ a

(x − a)n+ =
0, x<a
Note the functions (x − xk )3+ are cubic splines.

119
5.15: The Space of Cubic Splines (Cont.)
The set B := {1, x, x2, (x − x0)3+, . . . , (x − xm−1)3+} is a basis of S and

dim S = m + 3.
This basis is not convenient for computations.
120
5.16: Cubic B-Splines
We extend the grid to

ξ1 = . . . = ξ4 < ξ4+1 < . . . < ξ4+m+1 = . . . = ξ4+m+4
with ξ4+i = xi for i = 0, . . . , m
121
5.17: The Space of Cubic Splines (Cont.)
Definition.
The functions Nik , i = 1 : 4 + m + 3 − k defined recursively as follows are
called B-splines:

 0 if ξi = ξi+1
Ni1(x) := 1 if x ∈ [ξi, ξi+1)
0 else

x−ξi ξi+k −x
and Nik := ξi+k−1 −ξi Ni,k−1 +ξ Ni+1,k−1
i+k −ξi+1
where we use the convention 0/0 = 0 if nodes coincide.
122
5.18: B-Splines Cont.
123
5.19: B-Splines Basis
The B-splines Ni4 are cubic splines.
Theorem. The B-splines Ni4, i = 1, . . . , m + 3 form a basis of S, the

space of cubic splines.
124
5.20: B-Splines Basis (Cont.)
1
Grid: 0,0,0,0,1,2,3,4,5, 5,5,5
0.8
0.6
0.4
0.2
0
0 2 4 5
125
5.21: B-Splines Basis Representation
Any spline s ∈ S has a unique basis representation
m+3
X
s= diNi4
i=1
and in particular
m+3
X
1= Ni4.
i=1
The coefficients di are called de Boor points.
126
5.22: B-Splines Properties
1. Ni4(x) 6= 0 only for x ∈ [ξi, ξi+4]: local support (sv: lokalt stöd)
2. Ni4(x) ≥ 0: non-negative
3. Nie ∈ S if ξi 6= ξi+4: B-splines are splines
This makes that the coefficients have a graphical importance.

127
5.23: B-Splines: Control Polygon
-2
d2
-4
-6
0 1 2 3 4 5
For more see Course FMN100: Numerical Methods in CAGD

128
Ö5 : Exercise Notes (Homework 4)
Here we give two versions of a MATLAB m-file for computing B-splines.
First an m-file which is a straightforward MATLAB implementation of the

definition.
Then a recursive implementation.
129
Ö 5.1 : B-Splines (Version 1)
function y=bspline(x,grid,i,k) for level=2:1:k

% function y=bspline(x,grid,i,k) for j=0:k-level
% x evaluation point denom=(xnode(i+j+level-1)-xnode(i+j));
% xnode grid vector if (denom==0)
% i number of B-spline fac1=0;
% degree (level) of B-spline else
% C. Fuhrer, 2005 fac1=(x-xnode(i+j))/denom;
end
N=zeros(k,k); denom=(xnode(i+j+level)-xnode(i+j+1));
level=1; if (denom==0)
for j=0:k-level fac2=0;
if (xnode(i+j)==xnode(i+j+1)) else
N(j+1,level)=0; fac2=(xnode(i+j+level)-x)/denom;
else end
if (xnode(i+j)<=x & x < xnode(i+j+1)) N(j+1,level)=fac1*N(j+1,level-1)+ ...
N(j+1,level)=1; fac2*N(j+1+1,level-1);
end end
end end
end y=N(1,k);
130
Ö 5.2 : B-Splines (Version 2)
... and here the recursive version

denom1=grid(i+k-1)-grid(i);
function [N] = bspline(x,i,k,grid); if denom1==0
% fact1=0;
% else
if k==1 fact1=(x-grid(i))/denom1;
if grid(i) == grid(i+1) end
N=0; denom2=grid(i+k)-grid(i+1);
else if denom2==0
if x >= grid(i) & x < grid(i+1) fact2=0;
N=1; else
else fact2=(grid(i+k)-x)/denom2;
N=0; end
end N=fact1*bspline(x,i,k-1,grid)+...
end fact2*bspline(x,i+1,k,grid);
else
131
Unit 6: L2 Space
Space of all square integrable functions:

( )
Z b
L2(a, b) := f| f (x)2dx < ∞
a
Norm and inner product in L2
!1/2
Z b Z b
kf k2 = f (x)2dx (f, g)2 = f (x)g(x)dx
a a
132
6.1: Best Approximation
The problem in this unit is
Find a polynomial pn ∈ Pn such that
kf − pnk = min kf − qk2

q∈Pn
p is polynomial of best approximation in 2-norm.

133
6.2: Best Approximation
Example [9.4]: Find the best p ∈ P0 to f (x) = 1 − e−20x
1
0.8
best L 2approximation Alternativ problem:
0.6 Find minp∈P0 kf − pk∞
0.4 best max-norm approximation
with kf k∞ = maxx∈[a,b] |f (x)|.
0.2
0
0 0.5 1
134
6.3: Best Approximation (Cont.)
For simplicity [a, b] = (0, 1)
Ansatz: pn(x) = c0 + c1x + c2x2 + . . . + cnxn.
Reformulation
Minimize
Z 1
E(c0, c1, . . . , cn) = (f (x) − pn(x))2dx
0
Z 1 n
X Z 1 n X
X n Z 1
2 j
= f (x) dx − 2 cj f (x)x dx + cj ck xk+j dx
0 j=0 0 j=0 k=0 0
135
6.4: Best Approximation (Cont.)
Condition for minimum:
∂E(c0, c1, . . . , cn)

=0 i=0:n
ci
Working this out gives
n n n n−1 n n
    
(x , x ) (x , x ) ··· (x , 1) cn (f, x )
(xn−1, xn) (xn−1, xn−1) · · · (xn−1, 1) cn−1 (f, xn−1)
 ..  .  =  .. 
  .   
(1, xn) (1, xn−1) ··· (1, 1) c0 (f, 1)
| {z }
=:M
136
6.5: Hilbert Matrices
The matrix M is called a Hilbert matrix.

In MATLAB M=hilb(n) (in revers order!).
20
1
0 Condition vs Matrix
Dimension
10
Hilbert matrices are extremely ill con- 1
0
ditioned. Therefore this way to com-
pute the best polynomial pn is very
0
sensitive to perturbations. 10
0 5 10 15
137
6.6: Orthogonal Polynomials
If pn is represented as
pn(x) = γ0ϕ0(x) + γ1ϕ1(x) + . . . γnϕn(x)
where the ϕ form a basis of Pn and the γi are real coefficients, then the
system becomes
    
(ϕ0, ϕ0) (ϕ0, ϕ1) · · · (ϕ0, ϕn) γ0 (f, ϕ0)
 (ϕ1, ϕ0) (ϕ1, ϕ1) · · · (ϕ1, ϕn)   γ1   (f, ϕ1) 
 ..  .  =  . 
  .   . 
(ϕ , ϕ ) (ϕn, ϕ1) · · · (ϕn, ϕn) γn (f, ϕn)
| n 0 {z }
=:M
138
6.7: Orthogonal Polynomials (Cont.)
... and if the basis functions satisfy

= 0, if i 6= j
(ϕi, ϕj )
6= 0, if i 6= j
then the system simplifies to

    
(ϕ0, ϕ0) γ0 (f, ϕ0)
 (ϕ1, ϕ1)   γ1   (f, ϕ1) 
 .  =  . 
...

  .   . 
(ϕn, ϕn) γn (f, ϕn)
| {z }
=:M
139
6.8: Orthogonal Polynomials (Cont.)
This motivates the following definition
Definition. [9.4] The (infinite) sequence of polynomials ϕj , j = 0, 1, . . .

is called a system of orthogonal polynomials if ϕj has exact degree j and

= 0, if i 6= j
(ϕi, ϕj )
6= 0, if i 6= j
holds.
Note,
”exact degree” j means ϕj (x) = aj xj + aj−1xj−1 + · · · with aj 6= 0.
140
6.9: Gram–Schmidt Orthogonalisation
Does such an orthogonal system excists?
Answer: We just construct it.
Gram-Schmidt Orthogonalisation
Let ϕ0 = 1.
Ansatz ϕ1 = x − α0ϕ0.
(ϕ0, x)
Orthogonality: (ϕ0, ϕ1) = 0, this gives α0 = .
(ϕ0, ϕ0)
j
Pj−1 (ϕi, xj )
In general: ϕj = x − i=0 αi ϕi with αi = .
(ϕi, ϕi)
141
6.10: Orthogonal Polynomials - Examples
Example 1: R
1
Let (f, g) = 0 f (x)g(x)dx. Then (p. 261 f)
ϕ0(x) = 1
1
ϕ1(x) = x −
2
2 1
ϕ2(x) = x − x +
6
3 3 2 3 1
ϕ3(x) = x − x + x −
2 5 20
..
142
Example 2: R
1
Let (f, g) = −1 f (x)g(x)dx. Then (p. 263 f)
ϕ0(x) = 1
ϕ1(x) = x
1
ϕ2(x) = x2 −
3
3
ϕ3(x) = x3 − x
5
..
These polynomials are called Legendre polynomials.

143
Example 3:
Let Z 1
1
(f, g) = √ f (x)g(x)dx.
1−x 2
−1
Then (p. 263 f) we get (again) the Chebyshev polynomials.
144
6.12: Orthogonal Polynomials - Properties
Orthogonal polynomials and three-term recursions are related to each other

by the following theorem:
Theorem.
Orthogonal polynomials with leading coefficient one satisfy the three-term
recursion
2
ϕk+1(x) = (x − βk+1)ϕk (x) − γk+1 ϕk−1(x)
with ϕ−1(x) := 0, ϕ0(x) := 1 and
(xϕk , ϕk )w 2 (ϕk , ϕk )w
βk+1 := γk+1 :=
(ϕk , ϕk )w (ϕk−1, ϕk−1)w
(The proof is based on Gram-Schmidt orthogonalization again.)

145
6.13: Orthogonal Polynomials - Properties
We note an important property of orthogonal polynomials
Theorem. [9.4] Let ϕj , j = 0, 1, . . . be a system of orthogonal polynomi-

als on [a, b].
Then all roots of ϕj are distinct, real and lie in (a, b).
We will use this result later in the unit about integration.

146
6.14 Finite Dimensional case
We consider now the finite dimensional case and ask for the best
approximation in Rm:
Let V ⊂ Rm be an n-dimensional subspace.
Consider the problem:
Let b ∈ Rm. Find a vector v ∗ ∈ V such that
kv ∗ − bk2 = min kv − bk2

v∈V
where now k.k2 denotes the usual norm in Rm.
147
6.15 Finite Dimensional Case (Cont.)
We apply exactly the same approch as in the initedimensional case:

Pn
Let v1, v2, . . . , vn be a basis of V and v = i=1 αivi. Then the αi are
given by
    
(v1, v1) (v1, v2) · · · (v1, vn) α1 (b, v1)
 (v2, v1) (v2, v2) · · · (v2, vn) 
  α.2  =  (b, .v2) 
   

 ..  .   . 
(v , v ) (vn, v2) · · · (vn, vn) αn (b, vn)
| n 1 {z }
=:M
where (., .) is the scalar product in Rm.

Compare M with the Hilbert matrix approach above.)
148
6.16 Finite Dimensional case (Cont.)
If we form a matrix A which have the basis vectors vi as columns, then M = ATA and
we obtain the so-called normal equations
T T
A Ax = A b
with x = (α1, . . . , αn)T.

Pn
The best approximating vector v ∗ = i=1 α i v i can then be written as v ∗
= Ax or
∗ T −1 T
v = A(A A) A b
| {z }
=:P
Note that P is a projection matrix describing the orthogonal projection of Rm onto V .
149
6.17 Finite Dimensional case: Least Squares Method
An application is the solution of an overdetermined linear system
Ax = b
where A is an m × n matrix (n < m) and b an m × 1 vector.
This system has only a solution if b ∈ R(A), where R(A) denotes the range space of A
(bildrum, kolonnrum (sv.)).
150
6.18 Least Squares Method
We consider the problem

min kv − bk
v∈R(A)
and as all v ∈ R(A) can be written as v = Ax this problems becomes
min kAx − bk
x∈Rn
x is the least squares solution of the overdetermined linear equation system.
The normal equations (see above) give then the solution:

T −1 T
x = (A A) A b
151
6.19 Least Squares Method
This equation is rarely evaluated in the form it is written.

Effective numerical methods will be developed in the course “Numerical linear algebra”.
Applications of the least squares method occur in parameter estimation, statistics and
nearly everywhere where you have to handle a large ammount of measurements.
152
Ö6 : Exercise Notes (Homework 5)
Topic: Best Approximations
We give a symbolic code for orthogonal polynomials and demonstrate the best approxima-
tion by monomials, Legendre and Chebychev polynomials.
153
Ö6.1 : Gram-Schmidt/ Symbolic Toolbox
f u n c t i o n p o l y=o r t p o l y ( x , n , omega , a , b ) ;
% computes o r t h o g o n a l p o y n o m i a l s by a p p l y i n g
% Gram−Schmid o r t h o g o n a l i z a t i o n
% symbolic v a r i a b l e s x , omega
f o r i =0: n
sum=0;
f o r j =0: i −1,
a l p h a=i n t ( omega ∗ p o l y ( j +1)∗ x ˆ i , a , b ) / . . .
i n t ( omega ∗ p o l y ( j +1)∗ p o l y ( j +1) , a , b ) ;
sum=sum+a l p h a ∗ p o l y ( j +1);
end
p o l y ( i +1)=x ˆ i −sum ;
end
154
Ö6.2 : Result
>> syms x
>> omega=1/ s q r t (1− x ˆ 2 ) ;
>> c h e b y=o r t p o l y ( x , 5 , omega , − 1 ,1)
legendre =
[ 1 , x , x ˆ2 − 1/2 , xˆ3 −3/4∗ x , xˆ4+1/8− x ˆ 2 , x ˆ5+5/16 ∗ x −5/4∗ x ˆ 3 ]
155
Ö6.3 : Best approximation monomial basis
Computing the coefficients

f u n c t i o n [ c o e f f ]= monombest ( n )
% computes t h e b e s t a p p r o x i m a t i o n o f fun by a p o l y n o m i a l
% o f max d e g r e e n ( H i l b e r t m a t r i x method )
f o r i =0: n
f o r j =0: n
M( i +1, j +1)=quad (@( x ) x . ˆ ( n− i ) . ∗ x . ˆ ( n−j ) , − 1 , 1 , 1 . e − 9);
end
b ( i +1)=quad (@( x ) atan ( x ) . ∗ x . ˆ ( n− i ) , − 1 , 1 , 1 . e − 9);
end
c o e f f=M\ b ’ ;
156
Ö6.4 : Best approximation monomial basis (2)
and plotting:
x p l o t=l i n s p a c e ( − 1 , 1 , 5 0 0 ) ;
subplot (2 ,1 ,1) , plot ( xplot , y p l o t )
s u b p l o t ( 2 , 1 , 2 ) , p l o t ( x p l o t , atan ( x p l o t )− y p l o t )
157
Ö6.5 : Best approximation Legendre basis (1)
f u n c t i o n y=l e g e n d r e ( x , n )
switch n
case 0
y=1∗ o n e s ( s i z e ( x ) ) ;
case 1
y=x ;
case 2
y =1/2 ∗ (3 ∗ x .ˆ2 − 1);
case 3
y =1/2 ∗ (5 ∗ x .ˆ3 − 3 ∗ x ) ;
case 4
y =1/8 ∗ (35 ∗ x .ˆ4 − 30 ∗ x . ˆ 2 + 3 ) ;
case 5
y =1/8 ∗ (63 ∗ x .ˆ5 − 70 ∗ x .ˆ3+15 ∗ x ) ;
end
158
f u n c t i o n c o e f f=b e s t L e g e n d r e
f o r i =0:5
m( i +1)=quad (@( x ) l e g e n d r e ( x ,5 − i ) . ˆ 2 , − 1 , 1 , 1 . e − 9);
b ( i +1)=quad (@( x ) l e g e n d r e ( x ,5 − i ) . ∗ atan ( x ) , − 1 , 1 , 1 . e − 9);
c o e f f ( i +1)=b ( i +1)/m( i +1);
end
159
x p l o t=l i n p s a c e ( − 1 , 1 , 5 0 0 ) ;
sum=z e r o s ( s i z e ( x p l o t ) ) ;
f o r i =0:5
sum=sum+c o e f f ( i +1)∗ l e g e n d r e ( x p l o t ,5 − i ) ;
end
p l o t ( x p l o t , sum)
160
Ö6.8 : Legendre Best Approximation (with symbolic
toolbox)
omega=x^0;
legendre=ortpoly(x,5,omega,-1,1);
bestpoly=0;
for ii=0:5
i=ii+1;
coeff(i)=int(omega*atan(x)*legendre(i),-1,1)/...
int(omega*legendre(i)*legendre(i),-1,1);
bestpoly=bestpoly+coeff(i)*legendre(i);
end;
cpoly=sym2poly(bestpoly);
xplot=linspace(-1,1,200);
yplot=polyval(cpoly,xplot);
subplot(1,2,1), plot(xplot,yplot,’b’)
subplot(1,2,2), plot(xplot,atan(xplot)-yplot)
161
Ö6.9 : Legendre Best Approximation
-3
x 10
1 1.5
Legen Err
dre or
1
0.5
0.5
0 0
-0.5
-0.5
-1
-1 -1.5
-1 0 1 -1 0 1
162
Ö6.10 : Chebyshev Best Approximation
% Best legendre fit

omega=1/sqrt(1-x^2);eps=1.e-8;
cheby=ortpoly(x,5,omega,-1,1);
bestpoly=0;
for ii=0:n
i=ii+1;c=sym2poly(cheby(i));
denom=quad(@atancheby,-1+eps,1-eps,[],[],c)
coeff(i)=denom/int(omega*cheby(i)*cheby(i),-1,1);
bestpoly=bestpoly+coeff(i)*cheby(i);
end;
cpoly=sym2poly(bestpoly);
xplot=linspace(-1,1,200);
yplot=polyval(cpoly,xplot);
subplot(1,2,1), plot(xplot,yplot,’b’)
subplot(1,2,2), plot(xplot,atan(xplot)-yplot)
163
Ö6.11 : Chebyshev Best Approximation
-3
x 10
1 1
Chebys Err
hev or
0.5 0.5
0 0
-0.5 -0.5
-1 -1
-1 0 1 -1 0 1
164
Unit 7: Numerical Integration, Quadrature
In the last unit we saw the need for computing integrals:
Problem
Compute in an efficient way an accurate approximation to

Z b
f (x)dx
a
We will first discuss methods which operate on [a, b] and then so-called composite methods
which act on small subintervals.
165
7.1: Newton–Cotes Formulae
Let’s denote the exact integral by

Z b
b
Ia(f ) := f (x)dx
a
and an approximation by Iâb (f ).
Consider equidistant nodes a = x0 < x1 < . . . < xn = b
Then we define approximations by (see Eq. (7.2))

b b
Iâ(f ) = Ia(pn)
where pn is the interpolation polynomial of f on the grid.

166
7.2: Newton–Cotes Formulae (Cont.)
This concept leads to the so-called Newton–Cotes formulae:

n
X
b
Iâ(f ) = wk f (xk )
k=0
where k, the number of interpolation points, uniquely defines the weights wk and so the
method.
We know from the interpolation chapter that pn can be expressed by Lagrange polynomials
n
X
pn(x) = f (xk )Lk (x)
k=0
167
7.3: Newton–Cotes Formulae (Cont.)
Thus Z b
wk = Lk (x)dx.
a
Examples:
n = 1 : a = x0 < x 1 = b
Gives
x−b x−a
p1(x) = f (a) + f (b)
a−b b−a
1
= [(b − x)f (a) + (x − a)f (b)]
b−a
168
7.4: Trapezoidal Rule
Rb
Integration a
p1(x)dx gives the
Trapezoidal Rule
b `1 1
Z
´
f (x)dx ≈ (b − a) f (a) + f (b)
a 2 2
169
7.5: Simpson’s Rule
n = 2 : a = x0 < x1 = (b + a)/2 < x2 = b

gives in an analogous way (p. 203):
Simpson’s rule
b „ «
1 4 `b + a´ 1
Z
f (x)dx ≈ (b − a) f (a) + f + f (b)
a 6 6 2 6
170
7.6: Basic Requirements
• One requires that constants are integrated exactly, i.e.

b b
Ia(1) = Iâ(1).
Pn
This has as consequence, that k=0 wk = (b − a), why?
• Further as integrals are monoton
Z b Z b
f >g⇒ f (x)dx > g(x)dx
a a
we require also that the method is monoton

b b
f > g ⇒ Iâ(f ) > Iâ(g)
171
7.7: Basic Requirements (Cont.)
Monotony requires positive weights wk , why?
Newton Cotes formulae are monoton up to n = 8.
172
7.8: Error estimates
The error of an integration formula is defined as

Z b n
X
En(f ) = f (x)dx − wk f (xk )
a k=0
From the formula of the interpolation error we directly get
Theorem. [7.1] Let f ∈ C n+1([a, b]), then

b
Mn+1
Z
|En(f )| ≤ |(x − x0) · · · (x − xn)|dx
(n + 1)! a
with Mn+1 = maxx∈[a,b] |f (n+1)(x)|.
173
7.9: Error estimates (Cont.)
This estimate gives the following estimates (p. 205)
(b−a)3
Trapezoidal Rule: |E1(f )| ≤ 12 M2
(b−a)4
Simpson’s rule: |E2(f )| ≤ 196 M3
The last estimate can be improved (Th. 7.2):

(b−a)5
|E2(f )| ≤ 2880 M4
174
7.10: Composite Formulas
For deriving composite formulas we modify our notation (which deviates a bit from the
book) and introduce some terms, which we will meet in the Runge–Kutta chapter again:
We consider the integral Z 1
1
I0 (f ) = f (x)dx
0
and write the Newton-Cotes formulas
s
X
1
Iˆ0 (f ) = bif (ci).
i=1
We call s the stage number, bi the weights and ci the nodes of the method.
175
To decrease the error, we partition the interval [a,b] into m subintervals a = x0 < · · · <
xm and split the integral into a sum of integrals:
Z b m−1
X Z xi+1 m−1
XZ 1
f (x)dx = f (x)dx = h f (xi + hξ)dξ =
a i=0 xi i=0 0
Each subintegral is then approximated by a Newton Cotes formula.
176
This gives then the following approach

Z b m−1
XX s
f (x)dx ≈ h bj f (xi + cj h)
a i=0 j=1
177
7.13: Example: Composite Trapezoidal Rule
This gives for the composite trapezoidal rule the following

formula in case of m equidistant intervals
Z b
f (x)dx ≈
a
„ «
1 1
x i x i+1 h f (x0) + f (x1) + · · · + f (xm−1) + f (xm)
2 2
Trapezoidal Rule
with h = (b − a)/m.
178
7.14: Example: Composite Simpson Rule
In the same way the composite Simpson rule becomes
Z b
f (x)dx ≈
a
„
h h h
f (x0) + 4f (x0 + ) + 2f (x1) + 4f (x1 + ) + 2f (x2) + · · ·
6 2 2
«
h
· · · + 2f (xm−1) + 4f (xm−1 + ) + f (xm)
2
Note, the book expresses the composite Simpson rule in a slightly different way. There, h
is twice as large as here.
179
7.15: Error of Composite Rules
The global error of composite rules is just the sum of the (local) errors in each subinterval:
Composite Trapezoidal Rule:

3
X m−1 h h3 h2
|E1(f )| ≤ =0 M2 = m M2 = (b − a) M2
i
12 12 12
h 4
Composite Simpson’s rule: |E2(f )| ≤ . . . ≤ (b − a) 2880 M4
where we used the fact nh = b − a.

180
7.16: Order of a method
We observe from the error formulas
• When h → 0 the error decreases as fast as hq .

(q = 2 for the trapezoidal rule, q = 4 for Simpson’s rule)
• The method is exact for polynomials up to degree q − 1.

(Mq is then 0)
Definition.
q is called the order of the method.
For higher order methods larger step sizes h give the same accuracy as small step sizes for
lower order methods.
181
7.17 Order plots
In a loglog plot the order of a method can be visualized by comparing the approximate
solution to the exact solution for different step sizes (see also Homework 5):
182
7.18 Construction of a method with optimal order
The cost of the method is related to the number of f evaluations and this is related to
the number of stages s.
The accuracy of the method is determined by its order.
What is the highest order for a method with s stages?
This will be investigated by the next theorems.

183
7.19 Order Criterion
Theorem. [-] Define a method Ib01 by (ci, bi)si=1 with order k ≥ s,

and set
M (x) := (x − c1)(x − c2) · · · (x − cs) ∈ Ps[0, 1].
The order of Ib1 is at least s + m iff
0
Z 1
M (x)p(x)dx = 0 ∀p ∈ Pm−1[0, 1],
0
i.e. M ⊥Pm−1[0, 1] in L2.
(Proof: see lecture)
184
7.20 Example: 3 stage method of order 4
Consider m = 1, s = 3:
Z 1
0 = (t − c1)(t − c2)(t − c3) · 1dt
0
1 1
= − (c1 + c2 + c3) +
4 3
1
+ (c1c2 + c1c3 + c2c3) − c1c2c3
2
⇒
1
4 − (c1 + c2)/3 + c1c2/2
c3 = 1
3 − (c1 + c2)/2 + c1c2
Thus, there are two degrees of freedom in designing such a method.

185
7.21 Order cannot exceed 2s
Theorem.
A method with s stages has maximal order 2s.
(Proof: Given in the lecture. Consequence of the preceeding theorem.)

186
7.22 Gauß–Legendre Methods (Definition)
Theorem.
There is a method of order 2s. It is uniquely defined by taking cj as the roots of the sth
Legendre polynomial Ps(2t − 1), t ∈ [0, 1].
(Proof: Given in the lecture. Consequence of the orthogonality of the Legendre polynomi-
als.)
187
7.23 Gauß–Legendre Methods (Examples)
• s = 1 gives the midpoint rule

1 1
Ib0 (f ) = f ( )
2
• s = 2 Exercise.
• s = 3 gives a 6th order method
√ √
1 5 1 15 8 1 5 1 15
Ib0 (f ) = f( − )+ f( ) + f( + ).
18 2 10 18 2 18 2 10
These methods are called Gauß-Legendre quadrature rules.
188
Unit 8 : Finite Element Method in 1D
We consider the following boundary value problem

„ «
d du
− p(x) (x) + r(x)u(x) = f (x) (BVP)
dx dx
with the boundary conditions
u(a) = r1 u(b) = r2. (BC)
This problem is called a Sturm-Liouville problem. The boundary conditions are called
Dirichlet boundary conditions or sometimes essential boundary conditions, in mechanics: kinematic boundary
conditions.
189
8.1 : Sturm-Liouville problems
We assume p(x) ≥ c0 > 0 (ellipticity) and r(x) > 0.
The problem (BVP),(BC) will now be reformulated as a variational equation (VE) and a
variational
problem (VP).
We will numerically solve the variational equation (VE) instead.
190
8.2 : Sobolev spaces
We will need the following function spaces
• H 1(a, b) set of all functions v , with the property

Rb
a
v(x)2 + v 0(x)2 + dx < ∞,
• HE1 (a, b) ⊂ H 1(a, b) set of all functions v ∈ H 1(a, b),
with the property v(a) = r1, v(b) = r2,
• H01(a, b) ⊂ H 1(a, b) set of all functions v ∈ H 1(a, b),
with the property v(a) = 0, v(b) = 0.
Note, H 1(a, b) and H01(a, b) are linear spaces. HE1 (a, b) is an affine linear space,
i.e. v, u ∈ HE1 (a, b) ⇒ v − u ∈ H01(a, b).
191
8.3 : Sobolev norm
We use in Sobolev spaces the
Rb
• inner product (u, v)H 1 = a
u(x)v(x) + u0(x)v 0(x)dx and the
1/2
• corresponding norm kukH 1 = (u, v)H 1 .
192
8.4 : Variational Problem, Variational Equation, BVP
The boundary value problem can be stated as a variational problem or as a variational

equation. The relationship between these will be stated in the following theorems.
1
Consider the functional J : HE (a, b) → R
b
1
Z Z
0 2 2
J(w) = p(x)w (x) + r(x)w(x) dx − f (x)w(x)dx
2 a a
and the corresponding variational problem (VP):

1
Find u ∈ HE (a, b) such that
J(u) = min J(w) (VP)

w∈H 1 (a,b)
E
193
8.5 : Bilinear Form and L2 inner product
We introduce the symmetric bilinear form

Z
0 0
A(u, w) = p(x)u (x)w (x) + r(x)u(x)w(x)dx.
a
This and the L2-inner product allow a compact notation for the variational functional J :
1
J(w) = A(w, w) − (f, w)2
2
194
8.6 : Characterization of a Solution, Variational Equation
The following theorem gives a necessary and sufficient condition for a solution of (VP):
Theorem. [14.1]
1 1
u ∈ HE (a, b) minimizes J over HE (a, b) if and only if
1
A(u, v) = (f, v)2 ∀v ∈ H0 (a, b) (VE)
(Variational equation)
(proof given in lecture).

195
8.7 : Weak Solutions of BVP
By multiplying (BVP) by an arbitrary v ∈ H01(a, b), taking integrals and applying

integration by parts one sees, that
Theorem. [14.2]
1
If u ∈ HE (a, b) ∩ C 2(a, b) solves (BVP) then it is also a solution of the variational
equation (VE).
As (VE) might have solutions with smoothness (i.e. not in C 2) one defines
Definition. [14.3]
1
A solution u ∈ HE (a, b) of the variational equation (VE) is called a weak solution of the
BVP.
196
8.8 : Uniqueness
As a consequence of the assumptions p(x) ≥ c0 > 0 and r(x) ≥ 0 we obtain uniqueness:
Theorem. [14.3] The boundary value problem (BVP) has at most one weak solution in
1
HE (a, b).
Summary:
⇐=
(V P ) ⇐⇒ (V E) (BV P )
(=⇒)
Here, the relation in parantheses is only valid if the solution is in C 2.

197
8.9: Discretization - Galerkin method
h 1
We consider a finite dimensional affine linear subspace SE ⊂ HE (a, b),
n
X
h h 1 h
SE = {w ∈ HE (a, b) : w = ψ(x) + wiϕi(x)}
i=1
where ψ is any function fullfilling the boundary conditions and the ϕi are linearly indepen-
dent functions fullfilling the homogeneous boundary conditions.
Find uh ∈ SE
h
such that
h h h
J(u ) = min J(w ) (VP )
wh ∈S h
E
198
8.10: Discretization - Galerkin method (Cont.)
The solution is characterized by the discrete variational equations:
Theorem. The discrete variational equations

h h h h h h
A(u , v ) = (f, v )2 ∀v ∈ S0 (VE )
Pn
with S0h = {v h = i=1 viϕi(x)} have a unique solution.
199
8.11: Discretization - Galerkin method (Cont.)
Note that (VEh) is equivalent to the n equations

h
A(u , ϕj ) = (f, ϕj )2 j = 1, . . . , n
Pn
With uh(x) = ψ(x) + i=1 uiϕi(x) these equations can be written as
X
nKij uj = bj i = 1, . . . , n
j=1
with Kij = A(ϕi, ϕj ) bj = (f, ϕj ) − A(ψ, ϕj ).
200
8.12: Discretization - Stiffness Matrix
... or as linear equation system

0 10 1 0 1
K11 K12 ... K1n u1 b1
B K22 ... C B u.2 C = B b.2 C
K2n C B C B C
B
@symm. ... A @ .. A @ .. A
Knn un bn
| {z } | {z }
stiffness matrix load vector
201
8.13: Discretization - Linear Finite Element
A particular choice of discrete spaces is given by the linear finite element approach (FEM):
Grid: a = x0 < x1 < x2 < . . . < xn = b

Basis functions: ϕi, i = 1, . . . , n
with ϕi linear spline functions (“hat” functions):
8
< (x − xi−1) x ∈ [xi−1, xi]
ϕi(x) := (xi+1 − x) x ∈ [xi, xi+1]
0 else
:
202
8.14: Linear Finite Element (Cont.)
And the function ψ , which fullfils the boundary conditions can be given as:
ψ(x) = Aϕ0(x) + Bϕn(x)
In total:
n−1
X
h
u (x) = ψ(x) + uiϕi(x)
i=1
The choice of linear splines with their support [xi−1, xi+1] has as consequence that
Kij = 0 for |i − j| > 1
K is a symmetric tridiagonal matrix (see spline section).

203
8.15: Linear Finite Element - Galerkin Orthogonality
Theorem. [14.6, Cea’s Lemma]

The discrete solution can be viewed as a projection of the exact solution onto SEh :
• A(u − uh, v h) = 0 ∀v h ∈ S0h

• A(u − uh, u − uh) = minwh∈S h A(u − wh, u − wh)
E
(Proof: see lecture)
204
8.16: Energy Norm Error Estimate
Under the assumptions on p and r we can define a norm and an inner product by A:
• A(u, v) is an inner product

1
• kukA = (A(u, u)) 2 is a norm, the so-called energy norm.
205
8.17: Energy Norm Error Estimate (Cont.)
We apply the theorem above now to estimate the finite element error and to get the order
of the method:
Let u(xi) denote the (unknown) exact solution at the grid points.
Then
Xn−1
h
I (u)(x) = ψ(x) + u(xi)ϕi(x)
i=1
is a linear spline function in SEh which interpolates the exact solution.
It can be inserted as “wh” into the second statement of Cea’s theorem:
h h
ku − u kA ≤ ku − I (u)kA
206
8.17: Energy Norm Error Estimate (Cont.)
The linear spline interpolation error in the energy norm can be estimated (Corollary 14.1):
„ «
h 2 2 h 00 2
ku − I (u)kA ≤ C ku k2
π
with C 2 = P + ( πh )2R and P = maxx∈[a,b] p(x), R = maxx∈[a,b] r(x).

` ´
And finally (Corollary 14.2):
h h 00
ku − u kA ≤ Cku k2
π
Thus, the error is linear in h.
207
Ö7 : Excentric Bending of a rod
Here comes a student’s solution of the bar problem in Assignment 6:
function [xint,u]=galbas(n)
E=208*10^9;I=0.036;a=0.06;b=3;F=180000;
xint=linspace(0,b,n);
h=xint(2)-xint(1);
M=zeros(n-2);
M=diag((-2*E*I/h+h*F)*ones(n-2,1))+ ...
diag(E*I/h*ones(n-3,1),1)+ ...
diag(E*I/h*ones(n-3,1),-1);
for k=1:n-2
hf(k)=a/b*h*F*xint(k+1);
end
u=M\hf’;
208
Ö7.1 : Excentric Bending of a rod
Here what you get, when you call the program with n = 20:
x 10 6-6
-
0
-0.2
-0.4
-0.6
-0.8
-1
0 0.5 1 1.5 2 2.5 3
209
Unit 9: Initial Value Problems
We consider now problems of the type
ẏ(t) = f (t, y(t))

y(t0) = y0 initial value
where f : R × Rn → Rn is called the right-hand side function of the problem.
In rigid body mechanics this problem occurs as equations of motion where n describes the
number of degrees of freedom.
y is called the state vector of the system.

210
9.1: Initial Value Problems - Second order Systems
In mechanics due to Newton Euler’s law initial value problems occur in implicit second order form:
M (y(t))ÿ(t) = f (t, y(t), ẏ(t))

y(t0) = η0 initial value
ẏ(t0) = η1
M (y(t)) is a positive definite (thus invertible) mass matrix:
−1
ÿ(t) = M (y(t)) f (t, y(t), ẏ(t))
211
9.2: Second order Systems (Cont.)
−1
ÿ(t) = M (y(t)) f (t, y(t), ẏ(t))
Transformation to first-order (standard-) form:
Set y1(t) = y(t) (position) and y2(t) = ẏ(t) (velocity), then
ẏ1(t) = y2(t) time derivative of position is velocity
−1
ẏ2(t) = M (y1(t)) f (t, y1(t), y2(t)) time derivative of velocity is acceleration
212
9.3: Second order Systems (Cont.)
ẏ1(t) = y2(t) time derivative of position is velocity
−1
ẏ2(t) = M (y1(t)) f (t, y1(t), y2(t)) time derivative of velocity is acceleration
This gives a system of the type
Ẏ (t) = F (t, Y (t))

Y (t0) = Y0 initial value
where Y (t), Y0 ∈ R2n.
213
9.4: Linear Differential Equation Systems
Often (e.g. in control theory) the differential equation is linear:
ẏ(t) = Ay(t) + Bu(t)
where A is a n × n system matris, B an n × m input matrix and u(t) a known input

function (excitation).
Example: The linearized pendulum
„ « „ «„ «
α(t) 0 1 α(t)
=
α̇(t) − gl 0 α̇(t)
| {z } | {z } | {z }
ẏ(t) A y(t)
214
9.5: Linear Differential Equation Systems - Eigenvalues
The eigenvalues λ of A indicate stability see course Linear Systems
stable unstable
1 400
0 200
-1 0
0 5 10 0 5 10
2 20
0 0
-2 -20
0 5 10 0 5 10
Re(λ) ≤ 0 ⇒ stable,Re(λ) > 0 ⇒ unstable, Im(λ) 6= 0 ⇒ oscillatory

215
9.6: Stability Goal
We want that a numerical method reflects the stability properties of the original (linear)
problem.
216
9.7: Directional field
Scalar differential equations can be illustrated by their directional field, a plot which assigns
to every point a slope.
Directional field of ẏ(t) = −0.5y(t) and one solution curve (red)
-5
0 5 10 15
217
9.8: Euler’s method
We partition the time intervall
t0 < t1 < t2 < · · · < ti < ti+1 < · · · < te

| {z }
hi
and call hi the step size.
We denote
• exact solution at ti by yi
• approximate solution at ti by ui
218
9.9: Euler’s method (Cont.)
We replace in the differential equation the differentiation by a difference quotient and

obtain either
• Euler’s explicit method
ui+1 − ui
= f (ti, ui) ⇒ ui+1 = ui + hif (ti, ui)
hi
• or Euler’s implicit method
ui+1 − ui
= f (ti+1, ui+1) ⇒ ui+1 = ui + hif (ti+1, ui+1)
hi
(The method is called implicit because the unknown ui+1 occurs on both sides.)
219
9.10: Euler’s explicit method
2 h=3
1 h=1.5
0
-1
-2
0 5 10 15
220
9.11: Euler’s implicit method
2 h=3
1
h=1.5
-1
-2
0 5 10 15
221
9.12: Stability behavior of Euler’s method
We consider the so-called linear test equation
ẏ(t) = λy(t)
where λ ∈ C is a system parameter which mimics the eigenvalues of linear systems of

differential equations.
The equation is stable if Real(λ) ≤ 0. In this case the solution is exponentially decaying.
(limt→∞ y(t) = 0).
When is the numerically solution ui also decaying, limi→∞ ui = 0?
222
9.13: Stability behavior of Euler’s method (Cont.)
Explicit Euler discretization of linear test equation:
ui+1 = ui + hλui
This gives ui+1 = (1 + hλ)i+1u0.
hλ
i
The solution is decaying (stable)
if |1 + hλ| ≤ 1
-2 -i
223
Implicit Euler discretization of linear test equation:
ui+1 = ui + hλui+1
` 1
í+1
This gives ui+1 = 1−hλ u0 .
hλ
The solution is decaying (stable)
i
if |1 − hλ| ≥ 1
2
-i
224
Explicit Euler’s instability for fast decaying equations:
10 λ =-5 h=0.41
-5
-10
0 2 4 6 8 10 12 14
225
Facit:
For stable ODEs with a fast decaying solution (Real(λ) << −1 )
or highly oscillatory modes (Im(λ) >> 1 )
the explicit Euler method demands small step sizes.
This makes the method inefficient for these so-called stiff systems.
Alternative: implicit Euler method.
226
9.17: Implementation of implicit methods
Implicit Euler method ui+1 = ui + hif (ti+1, ui+1)
Two ways to solve for ui+1: k is the iteration counter, i the integration step counter
(k+1) (k)
• Fixed point iteration: ui+1 = ui + hif (ti+1, ui+1)
| {z }
(k)
=ϕ(u )
i+1
• Newton iteration:
ui+1 = ui + hif (ti+1, ui+1) ⇔ ui+1 − ui − hif (ti+1, ui+1) = 0

| {z }
=F (ui+1 )
(k) (k) (k+1) (k)

F 0(ui+1)∆ui+1 = −F (ui+1) ui+1 = ui+1 + ∆ui+1
227
9.18: Implementation of implicit methods (Cont.)
These iterations are performed at every integration step!

They are started with explicit Euler method as so-called predictor:
(0)
ui+1 = ui + hif (ti, ui)
When should fixed points iteration and when Newton iteration be used?
The key is contractivity!
Let’s check the linear test equation again: ẏ = λy .
Contractivity: |ϕ0(u)| = |hλ| < 1.
228
9.19: Implementation of implicit methods (Cont.)
If the differential equation is
• nonstiff: explicit Euler or

• nonstiff: implicit Euler with fixed point iteration
• stiff: implicit Euler with Newton iteration
229
9.20: Multistep Methods
Multistep methods are methods, which require starting values from several previous steps.
There are two important families of multistep methods
• Adams methods (explicit: Adams–Bashforth as predictor, implicit: Adams–Moulton as

corrector), used together with fixed point iteration for nonstiff problems,
• BDF methods (implicit), together with Newton iteration used for stiff problems
230
9.21: Adams Methods
For deriving ADAMS methods we transform the ODE
ẏ = f (t, y) with y(t0) = y0
into its integral form Z t

y(t) = y0 + f (τ, y(τ )) dτ
t0
and partition the interval into
t0 < t1 < · · · < ti < ti+1 = ti + hi < · · · < te
231
9.22: Adams Methods (Cont.)
Thus Z tn+1
yn+1 = y(tn+1) = yn + f (τ, y(τ )) dτ
tn
Let un+1−i, i = 1, . . . , k be previously computed values and πkp the unique polynomial
which interpolates
f (tn+1−i, un+1−i), i = 1, . . . , k
The we define a method by
Z tn+1
p
un+1 = un + πk (τ ) dτ
tn
232
Example (2-step method, k = 2):

Using the concept of Lagrange polynomials give
p
πk (t) = f (tn, un)L0(t) + f (tn−1, un−1)L1(t)
t − tn−1 t − tn
= f (tn, un) + f (tn−1, un−1)
tn − tn−1 tn−1 − tn
Integration gives:
`3 1
un+1 = un + h f (tn, un) − f (tn−1, un−1)
2 2
This is the Adams–Bashforth-2 method.
233
Adams–Bashforth methods are explicit, their general form is
k
X
un+1 = un + h βk−if (tn+1−i, un+1−i)
i=1
Adams–Moulton methods are constructed in a similar way.
Here the unknown value f (tn+1, un+1) is taken as an additional interpolation point.
This makes the method implicit.
234
Examples (Adams–Moulton):
k=0: un+1 = un + hf (tn+1, un+1) implicit Euler method

`1 1
´
k=1: un+1 = un + h 2 f (tn+1 , un+1 ) + 2 f (tn , un ) Trapezoidal rule
`5 8 1
´
k=2: un+1 = un + h 12 f (tn+1 , un+1 ) + 12 f (tn , un ) − 12 f (tn−1 , un−1 )
The general form is
k
X
un+1 = un + h β̄k−if (tn+1−i, un+1−i)
i=0
235
9.26: Starting a multistep method
To start a multistep method one applies
• the initial value

• then a one-step method, to get two values
• then a two-step method, to get three values
• and so on ...
236
9.27: Adams Method - Stability
The region of stability shrinks when k increases:

1.5 4
AM-1 AM-0
1 AB-1 AB-2 AM-2
2
0.5 AM-3
AB-3
0 0
-0.5
-2
-1
-1.5 -4
-2 -1 0 -6 -4 -2 0 2
20
5
BDF-3 BDF-6
10
BDF-5
BDF-2
o BDF-1 BDF-4
86
0 0 237
-10
9.28: BDF methods
BDF methods BDF=backward differentiation fomula are constructed directly from the differential
equation. We seek for a polynomial, which statisfies the ODE in one point, tn+1 and
interpolates k previous ones:
Define the kth degree polynomial πk+1 by
πk+1(tn+1−i) = un+1−i, i = 0, . . . , k
π̇k+1(tn+1) = f (tn+1, un+1).
238
9.29: BDF methods (Cont.)
Working out these conditions and using again the Lagrange polynomial approach gives
formulas of the following type:
k
X
αk−iun+1−i = hf (tn+1, un+1)
i=0
Examples:
k=1: un+1 = un + hf (tn+1, un+1) implicit Euler method
k=2: un+1 = 43 un − 31 un−1 + h 23 f (tn+1, un+1) BDF-2
18 9 2 6
k=3: un+1 = 11 un − 11 un−1 + 11 un−2 + h 11 f (tn+1, un+1)
239
1.5 4
AM-1 AM-0
1 AB-1 AB-2 AM-2
2
0.5 AM-3
AB-3
9.30: BDF0 - Stability 0
-0.5
-2
The region of -1
stability shrinks when k increases,
-1.5 -4 but remains unbounded for k ≤ 6:
-2 -1 0 -6 -4 -2 0 2
20
5
BDF-3 BDF-6
10
BDF-5
BDF-2
BDF-4
86o BDF-1
0 0
-10
-5
-20
-2 0 2 4 6 8 -10 0 10 20 30
240
9.31: Multisptep methods in General
The general form of a linear multistep method reads
k
X k
X
αk−iun+1−i − hn βk−if (tn+1−i, un+1−i) = 0.
i=0 i=0
For starting a multistep method k starting values
u0, . . . , uk−1 are required.

241
9.32: Global Error
The quantity of interest is the global error of the method
at a given time point tn
en := y(tn) − un,
with n = tn/h.
If for exact starting values en = O(h), then the method is said to be convergent. More
precisely, a method is convergent of order p, if
p
en = O(h ).
242
9.33: Local Residual
To make a statement about the behavior of the global error, we have to introduce and
study first the local residual:
Definition. Let y be a differentiable function, then the quantity
k
X k
X
l(y, tn, h) := αk−iy(tn−i) − h βk−iẏ(tn−i)
i=0 i=0
is called the local residual of the method.
243
9.34: Example
The local residual of the two-step implicit Adams method is defined
by
„ «
5 8 1
l(y, t + h, h) = y(t + h) − y(t) − h ẏ(t + h) + ẏ(t) − ẏ(t − h) .
12 12 12
244
9.35: Example (Cont.)
Taylor expansion leads to

1 2 1 3 (3) 1 4 (4)
l(y, t, h) = hẏ(t) + h ÿ(t) + h y (t) + h y (t) + . . .
2 6 24
5 5 2 5 3 (3) 5 4 (4)
− hẏ(t) − h ÿ(t) − h y (t) − h y (t) + . . .
12 12 24 72
8
− hẏ(t)
12
1 1 2 1 3 (3) 1 4 (4)
+ hẏ(t) − h ÿ(t) + h y (t) − h y (t) + . . .
12 12 24 72
1 4 (4)
= − h y (t) + . . .
24
Thus the implicit two step Adams method has the order of consistency
3.
245
9.36: Order of Consistency
Conditions for higher order of consistency are given by the following theorem:
Theorem. A linear multistep method has the order of consistency p if the following
p + 1 conditions on its coefficients are met:
k
X
αi = 0
i=0
k
X
iαi − βi = 0
i=0
k
X 1 j 1 j−1
i αi − i βi = 0 with j = 2, . . . , p.
i=0
j! (j − 1)!
246
9.37: Asymptotic Form of Local Residual
The local residual of a method with order of consistency p takes the
form
p+1 (p+1) p+2
l(y, t, h) = cp+1h y (t) + O(h ).
Adams–Bashforth methods have order of consistency k, Adams–Moulton
methods have order of consistency k + 1, and BDF methods have order
of consistency k
247
9.38: Global Error Propagation
Consider (for simplicity) the linear differential equation ẏ = Ay.

The difference of
k
X k
X
αk−iy(tn−i) − hn βk−iAy(tn−i) = l(y, tn, h)
i=0 i=0
and
k
X k
X
αk−iun−i − hn βk−iAun−i = 0.
i=0 i=0
gives a recursion for the global error:
k
X k
X
αk−ien−i − hn βk−iAen−i = l(y, tn, h).
i=0 i=0
248
9.39: Global Error Propagation (Cont.)
By introducing the vector
0 1
en
B en−1 C
En := B . C ∈ Rkny
@ .. A
en−k+1
this recursion formula can be written in one-step form as
En+1 = Φn(h)En + Mn
249
with
0 1
−1 −1 −1 −1
−Ak Ak−1 −Ak Ak−2 · · · −Ak A1 −Ak A0
B
B I 0 ··· 0 0 C
C
Φn(h) := B 0 I ··· 0 0 C,
B C
B
@ ... ... ... C
A
0 0 ··· I 0
Ai := (αiI − hβiA) and
−A−1
0 1
k l(y, tn , h)
B 0 C
Mn := B
@ ... C.
A
0
.
250
From this formula we see how the global error of a multistep method is built up. There is in
every step a (local) contribution Mn, which is of the size of the local residual. Therefore,
a main task is to control the integration in such a way that this contribution is kept small.
The effect of these local residuals on the global error is influenced by Φn(h). The local
effects can be damped or amplified depending on the properties of the propagation matrix
Φn(h). This leads to the discussion of the stability properties of the method and its
relation to the stability of the problem.
251
9.42: Stability
The stability requirement is

n
kΦ(h) k < C
with C being independent of n, which is equivalent to
All eigenvalues of Φ(h) are within the unit circle and those on its boundary are
simple.
252
9.43: Organisation of a Multistep code
see Flowcharts (separate file).
253
9.44: Runge–Kutta Methods
Runge–Kutta methods are one-step methods, i.e. they have the generic form
un+1 := un + hφh(tn, un)
with a method dependent increment function φh. In contrast to multistep methods, the
transition from one step to the next is based on data of the most recent step only.
254
9.45: Runge–Kutta Methods: Basic Scheme
The basic construction scheme is
U1 = un
i−1
X
Ui = un + h aij f (tn + cj h, Uj ) i = 2, . . . , s
j=1
s
X
un+1 = un + h bif (tn + cih, Ui).
i=1
s is called the number of stages.
255
9.46: Runge–Kutta Methods: Example
By taking s = 2, a21 = 1/2, b1 = 0, b2 = 1, c1 = 0, and c2 = 1/2 the following

scheme is obtained
U1 = un
h
U2 = un + f (tn, U1)
2
h
un+1 = un + hf (tn + , U2)
2
For this method the increment function reads
“ h h ”
φh(t, u) := f t + , u + f (t, u) .
2 2
256
9.47: Runge–Kutta Methods: Stage derivatives
Normally, Runge–Kutta methods are written in an equivalent form by substituting ki :=

f (tn + cih, Ui)
k1 = f (tn, un)
i−1
X
ki = f (tn + cih, un + h aij kj ) i = 2, . . . , s
j=1
s
X
un+1 = un + h biki.
i=1
257
9.48: Runge–Kutta Methods: Butcher Tableau
The coefficients of a Runge–Kutta method in a compact form

Butcher tableau:
c1
c2 a21
c3 a31 a32 c A
... ... ... ... or
bT
cs as1 as2 ··· as,s−1
b1 b2 ··· bs−1 bs
with A = (aij ) and aij = 0 for j ≥ i.
258
9.49: Butcher Tableau - RK4
The classical 4-stage Runge–Kutta method reads in this notation
0
1 1
2 2
1 1
2 0 2 .
1 0 0 1
1 2 2 1
6 6 6 6
259
9.50: Order of a Runge–Kutta Method
The global error of a Runge–Kutta method at tn is defined in the same way as for multistep
methods
en := y(tn) − un.
with n = tn/h. A Runge–Kutta method has order p if en = O(hp).
260
9.51: Embedded Runge-Kutta Methods
The local error can be estimated by comparing the result after one step with two methods
with different order:
c1
c2 a21
c3 a31 a32
... ... ... ...
cs as1 as2 ··· as,s−1
bp1 bp2 ··· bps−1 bps
bp+1
1 bp+1
2 ··· bp+1
s−1 bp+1
s
(Butcher Tableau of an embedded pair of two RK-methods).
261
9.52: Runge–Kutta–Fehlberg 2(3)
One method of low order in that class is the RKF2(3) method
0
1 1
1 1 1
2 4 4
1 1
2 2 0
1 1 4
6 6 6
It uses 3 stages.
262
9.53: Stability of RK4(5)
The stability plot of the RK45 method (MATLAB’s ode45)
DOPRI4
2
-2 DOPRI5
-4
-4 -2 0 2
263
Ö8 : MATLAB ODE example
Let’s simulate with MATLAB the pendulum.
ODE:
g
α̈(t) = − sin(α(t))
l
ODE in first order form
α̇1(t) = α2(t)
g
α̇2(t) = − sin(α1(t))
l
Initial values
α1 = π/2 and α2 = 0
264
Ö8.1 : Pendulum -right hand side function
In MATLAB define first the right-hand side function:
function ydot=pendel(t,y)
%
g=9.81;
l=1;
%
alpha1=y(1);
alpha2=y(2);
%
ydot(1)=alpha2;
ydot(2)=-g/l*sin(alpha1);
ydot=ydot’;
265
Ö8.2 : Pendulum Simulation
We may use ode15s to integrate this problem
>> ode15s(@pendel,[0 10],[pi/2,0]);
ode15s is a multistep method with variable step sizes and order.
266
Ö8.3 : Pendulum Simulation (Cont.)
The blue curve depicts α(t), the green its derivative and the circles indicate the step sizes:
-5
0 5 10
267
Ö8.4 : Implicit Euler Code
function [t,y]=impeul(odefun,tspan,y0,n)
t0=tspan(1); te=tspan(2); t=linspace(t0,te,n);h=t(2)-t(1);
y(:,1)=y0;
for i=1:n-1 % integrator loop
% predict
ypred=y(:,i)+h*feval(odefun,t(i),y(:,i));
y(:,i+1)=ypred;
% corrector med fixpunktiteration
for j=1:2 % we use a fixed number of iterations
% and do not check convergence
y(:,i+1)=y(:,i)+h*feval(odefun,t(i+1),y(:,i+1));
end
end
268
Ö8.5 : Implicit Euler Damping
We see the damping effect of implicit Euler if we take few steps:
2
n=200
1
-1
-2
0 5 10
269
Ö8.6 : Implicit Euler Damping
... and here with 10 times more steps
2
n=2000
1
-1
-2
0 5 10
270
10.1: Outlook
This course is a basic appetizer for continued work and education in numerical analysis
and applied mathematics.
There are courses on
• Numerical Methods for ODE/PDE

• Finite Volume Methods
• Numerical Methods in Computer Graphics
• Numerical Linear Algebra
• Simulation Tools
and hopefully soon a course on Numerical Optimization...
271
11.1: Appendix
Here we attach for complete course documentation
• the six home assignments

• the final exam preparing project
• the ODE code flowchart
272
Claus Führer, 2008-01-16
FMN081/FMN041/NUMA12: Computational Methods in Physic

Numerical Analysis, Matematikcentrum
Assignment 1
Last hand-in day: 23 January 2008 at 13:00h
Goal: Making practical experiments with fixed point and Newton iteration
Task 1 Consider the problem of finding all zeros of the function f (x) =
2x − tan x. Transform this function into the two different fixed point
problems
1
x = tan x or x = arctan(2x).
2
Try to perform fixed point iteration with these functions, discuss con-
vergence and estimate the error after 10 iteration steps (a posteriori
estimate) in the convergent case.
Construct another fixed point problem, which has as fixed points the
roots of the function f . Discuss even for this third alternative the
convergence properties as above.
Task 2 Write in MATLAB three functions, one which performs one step
of Newton-iteration with exact derivative, one which does the same,
but with a finite-difference approximation of the derivative and one
for the secant method. Solve with these functions the problem f (x) =
arctan(x) = 0 by calling them in MATLAB within an appropriate
for-loop.
Task 3 Study by experiments the convergence behavior for these three

methods, when applied to the function above. What is the order of
convergence? Determine for each method the convergence interval, i.e.
an interval around the solution, where you can observe convergence
for any x0 .
Task 4 Read Sec. 1.7 in the course book. It contains two examples, which
demonstrate interesting phenomena observed when performing fixed
point iteration and Newton iteration. Program fixed point iteration
for the logistic equation (1.33) and demonstrate the phenomena by
your computations. Be prepared to present your results in class on
Tuesday, January 25.
Lycka till!
1
FMN041/FMN081: Computational Methods in Physics

NUMA12 Numerical Approximations
Assignment 2
Last hand-in day: 30 January 2008 at 13:00h
Goal: Matlab experience with Newton iteration in Rn , mechanical example,
fractal structure of Newton convergence regions, a property of the condition
number
Task 1 We consider the mathematical model of the truck (see figure).
p p
8 5
p p
9
6
p7
loading area (body 5) cabin (body 4)
p
2
chassis (body 2)
p
3
p rear wheel (body 1) p

1 4
front wheel (body 3)
reference frame (body 0)
The task is to compute an equilibrium position for the truck starting

from an initial position given by the coordinates
p(0) = (0.52, 2.16, 0.0, 0.52, 2.68, 0.0, −1.5, 2.7, 0.0)T .
Download the MATLAB file truck acc.m which evaluates the right
hand side of the differential equation describing the motion of the
truck. That m-file will also need spridamp.m and relat.m . Compu-
te the equilibrium position of the truck by using Newton’s method.
Discuss the rate of convergence.
1
Task 2 We will construct now a fractal figure using Newton’s method. Con-
sider the roots of the equation system
3
x1 − 3x1 x22 − 1

F (x) = =0
3x21 x2 − x32
1. Write down Newton’s method applied specifically to this problem.

2. Write an m-file called rootz31.m that given an initial vector x(0) ∈
R2 applies a fixed number k = 15 of iterations of Newton’s met-
hod. Use this file to find√the three zeros of F : √
xb = (1, 0)T , xg = (− 12 , 23 )T and xr = (− 12 , − 23 )T .
3. Study now the dependance of Newton’s method on initial vectors
x(0) . Write an m-file called newtonfractal.m following the steps
• Use the meshgrid command to set up a grid of 1002 points in
the set G = [−1, 1] × [−2, 2]. You obtain two matrices X and
Y where a specific grid point is defined as pij = (Xi1 , Y1j )T .
• Build a matrix A such that aij = 1 if Newton’s method
started with pij converges to the first root, xb , and aij = 2 if
it converges to the second root, xg , and aij = 3 if it converges
to the third root xr . If the process diverges you set aij = 0.
• Visualize the resulting matrix, A, to observe the resulting
pattern. Use for this end the MATLAB command pcolor
and shading.
• Why is this figure called a fractal? Explain the term local
convergencefrom the figure.
Task 3 Let A be a symmetric matrix. Show that its condition number
with respect to the 2-norm is |λ max |
|λmin | , where λmax is the largest
eigenvalue of A and λmin the smallest.
Lycka till!
2

Assignment 3
Last hand-in day: 6 February 2008 at 13:00h
Goal: To get experience with Polynomial interpolation, Chebyshev polyno-
mials, Runge’s phenomenon
Task 1 This task is just to get experience with simple data interpolation
and extrapolation. In the table below you find three columns with
energy data from my house in Södra Sandby. Construct the unique
polynomial which interpolates the energy consumption and another
which interpolates the temperature. Plot the curves. Evaluate these
polynomials to see, which energy I will consume on Tuesday, February,
6 and find out which will be the average temperature at that day.
Day Temperature (C) Energy (kwh)

080127 -1.9 109.26
080128 -3.7 92.4
080129 -5.77 115.33
080130 2.53 107.77
080131 4.32 61.14
Solve this task with three methods
• by using MATLAB’s commands polyfit and polyval,

• by using Vandermonde’s approach
• by using Lagrange polynomials.
Note, you get in all three cases the same polynomial but in totally
different representations. I.e. your plots should be identical in all three
cases (if you made things correctly).
Task 2 When interpolating a function with a polynomial using xi , i = 0 : n

as interpolation points the error has the form
1
|f (x) − p(x)| = |f (n+1) | |(x − x0 )(x − x1 ) · · · (x − xn )|.
(n + 1)!
Therefor, the polynomial ωn (x) = (x−x0 )(x−x1 ) · · · (x−xn ) influences

the size of the interpolation error.
Put n distinct (!) interpolation points in different locations in the
1
interval [−1, 1] and plot ωn (x). Set n = 5 and try out different choices
of interpolation points. Can you recommend an optimal one? Test even
the case n = 15.
Task 3 Select in Task 2 the interpolation points as Chebyshev points and

compare the resulting curve with your previous results.
Task 4 Now, we visualize the error. For this end interpolate the function
1
f (x) =
1 + 25x2
on an equidistant grid in the interval [−1, 1] with n = 3, 9, 15. Con-
struct also an interpolation polynomial on a grid with Chebyshev
points.
Task 4 Proof the property of the Lagrange polynomials Li ∈ P n

n
X
Li (x) = 1 ∀ x
i=0
Lycka till!
2
FMN081: Computational Methods in Mechanics

Assignment 4
Goal: To get experience with splines
Task 1 Write a program coeff=cubspline(xint,yint), which takes as in-

put the x- and y-values of m + 1 interpolation points and returns a
m × 4 coefficient matrix of the natural spline which interpolates the
data. The i-th row of this matrix contains the coefficients ai , bi , ci , di
of the cubic subpolynomial si of the spline.
The program may be written for equidistant node-points xi (h con-
stant.)
Task 2 Write a program yval=cubsplineval(coeff,xint,xval), which
evaluates the spline at xval. Test both programs on a simple test
problem of your choice.
Task 3 We will now discuss a wheel profile function, that is based the stan-
dard profile S1002, which is defined sectionwise by polynomials up to
degree 7. The profile and its sections ar shown in Fig. 1. The polyno-
mials are defined by
Section A: F (s) = aA − bA s
Section B: F (s) = aB − bB s + cB s2 − dB s3 + eB s4 − fB s5 + gB s6 − hB s7 + iB s8
2 3 4 5 6 7
Section C: F (s) = −aC −pbC s − cC s − dC s − eC s − fC s − gC s − hC s
Section D: F (s) 2
= aD − bD − (s + cD ) 2
Section E: F (s) = −aE −pbE s

Section F: F (s) = aF + pb2F − (s + cF )2
Section G: F (s) = aG + pb2G − (s + cG )2
Section H: F (s) = aH + b2H − (s + cH )2
and
A B C D
a 1.364323640 0.0 4.320221063 10+3 16.446
b 0.066666667 3.358537058 10−2 1.038384026 10+3 13.
c − 1.565681624 10−3 1.065501873 10+2 26.210665
d − 2.810427944 10−5 6.051367875 10+0 −
e − 5.844240864 10−8 2.054332446 10−1 −
f − 1.562379023 10−8 4.169739389 10−3 −
g − 5.309217349 10−15 4.687195829 10−5 −
h − 5.957839843 10−12 2.252755540 10−7 −
i − 2.646656573 10−13 − −
ξmin 32.15796 −26 −35 −38.426669071
ξmax 60 32.15796 −26 −35
1
E F G H
a 93.576667419 8.834924130 16. 9.519259302
b 2.747477419 20 12. 20.5
c − 58.558326413 55. 49.5
ξmin −39.764473993 −49.662510381 −62.764705882 −70.0
ξmax −38.426669071 −39.764473993 −49.662510381 −62.764705882
5
H G F D C B B A
0 E
-5
-10
-15
-20
The S1002 wheel profile
-25 and its sectors of
polynomial describtion
-30
-80 -60 -40 -20 0 20 40 60
Figure 1.
Describe this wheel profile by means of a natural cubic spline. For this
end, download the file s1002.m, which contains the above description
of the s1002 wheel standard and generate from this data, which you
then use to generate an interpolating spline with your programs in
Task 1 and 2. Plot the resulting curve.
Task 4 Write a program y=bspline(x,xint,i) which computes the i-th

cubic B-spline function y = Ni4 (x). Use this program to plot the curve
given by the deBoor points d = [1, 2, −2, 4, −5, 0, 2, 2] and the node
vektor xint = [0, 0, 0, 0, 1, 2, 3, 4, 5, 5, 5, 5].
Lycka till!
2

Assignment 5
Goal: To get experience with Polynomial interpolation, Chebyshev polyno-
mials, Runge’s phenomenon
Task 1 Compute and plot the polynomial p5 which best approximates the
function f (x) = arctan(x) in the interval [−1, 1]. Make three different
approaches
1. Use a monomial basis for this task, set up a Hilbert matrix
Rand
1
solve for the coefficients. Use the inner product (f, g) =
−1 (x)g(x)dx.
f
2. Use Legendre polynomials as a basis and the same inner product.
R1 1
3. Use the inner product (f, g) = −1 √1−x 2
f (x)g(x)dx instead and
use Chebychev polynomials as a basis.
To compute integrals of nonpolynomial expressions use MATLABs
command quad, i.e. don’t make symbolic computations. Study the
influence of different integration tolerances (TOL).
Task 3 Write two MATLAB programs mysimpson and mygauss3, which
compute
Rb an Simpson and 3-stage Gauss approximation to the integral
a f (x)dx for a given function and a given number of steps.
Task 4 Aplly these methods to compute the arclength of the logarithmic

spiral, which is given in parametric form by
x(t) = 3 cos(t) exp(−0.04t)

y(t) = 3 sin(t) exp(−0.04t)
Rbp
with t ∈ [0, 100]. (The arclength is defined by s = a x0 (t)2 + y 0 (t)2 .)
Task 5 Take a nonpolynomial function of your choice of which you know its
exact integral. Use this function to determine the approximation error
of your Simpson and Gauss3 method. Compare their performance by
making a effort/precision plot. Plot the stepsize versus the error in a
loglog diagram and discuss the order of the methods.
Lycka till!
1

Assignment 6
Goal: To solve a simple boundary value problem with Galerkin’s method
and test a multistep method for an initial value problem.
Task 1 Consider a steel rod with a quadratic cross section, which is loaded
eccentrically by a force F , see figure. The differential equation for the
displacement of its central line (dotted line) is given by
a
EIw00 (x) + F w(x) = Fx
b
where E is Young’s modulus 208 GPa, the cross section I = 0.036 m4 ,
the length b = 3m and the height a = 0.06m. The force F = 180 kN.
The rod is supported at its both ends, which leads to the boundary
conditions
w(0) = w(b) = 0.
The origin of the coordinate system is assumed in the left support.
• Compute numerically the bending w(x) by using Galerkin’s method

together with a linear spline space. Choose an appropriate size of
the discretization parameter h.
• The exact solution is
x sin(λx)
w(x) = a − .
b sin(λb)
p
with λ = F/(EI). Decrease h successively and make a plot of
the error versus the step size. Which order of the error do you
observe?
• Is the stiffness matrix M of the problem positive or negative
definite. Give your answer for a specific h.
• The bending moment is defined as EIw00 (x). Plot the bending
moment and determine its extremal values.
1
Task 2 We compute once more (see Assignment 2) the steady state position
of the 2D-truck model. Now, we compute it by solving the its differ-
ential equation. We take as initial conditions p(0) from Assginment 2
and v(0) = 0 (9 components). The differential equation is given by
the files trurhs.m and the corresponding subprograms, which can be
downloaded as a zip-file from the web.
• Solve this task with MATLAB’s ODE-solver ode15S.

• Write a 3-step Adams–Moulton method myAdams3 and solve the
same task with your code and constant step size. Use fixed-point
iteration for the corrector iteration and start your code with three
steps of the Euler method.
Lycka till!
2

Assignment 7 (Exam Preparation)

Last hand-in day: – no hand in –
Goal: This practical project is intended as a practical preparation for the
final exam. In contrast to the homework it is recommended that you work
on it individually. Clearly, you can discuss your answers and problems with
everybody, you may use also all help you find from books, lecture notes etc..
But work out your own solution. In case you become stuck with MATLAB,
note that for the exam, it is not MATLAB, which maters, it is the under-
standing of the assignment problems and of solution methods, which will be
the topic of the exam. Think also about possible alternative solution tech-
niques, if any.
You need not to hand in anything, but don’t forget to bring your answers to
the final exam, where we pose questions which are related to all homeworks
and to this project in particular.
There will be consulting hours even for the project (see the web page for
dates).
Scenario: Consider the pendulum equation from the lecture with the same
initial conditions. This time we place an obstacle (ett hinder, sv.) at an
angle αobst = 23 π, see picture. The obstacle will be hit at an unknown
time tobst . Let’s say that it has then an angular velocity (also unknown)
α̇obst . When this obstacle is hit, we have to restart integration of the
problem with new initial conditions (αobst , −α̇obst )T .
Your task is to integrate the pendulum numerically over many periods
and to show how the obstacle influences its trajectory. Furthermore
we are interested in tobst , (αobst , and α̇obst . The integration should be
performed with an BDF 2-step method. To start integration use an
implicit Euler step first.
Organization of your work:
1. Write a MATLAB code, which performs a single step of the im-
plicit Euler method.
2. Write a MATLAB code performing a single step of BDF-2. (coef-
ficients of the method and of its predictor, see at the end of this
project page)
1
α
α obst
3. Combine these programs and solve the problem without the ob-
stacle. Check your solution by comparing it to the results from
the lecture notes.
4. Write a MATLAB program, which computes an interpolation
polynomial, for three consecutive solution points un+1 , un , un−1 .
5. Call this MATLAB function after every integration step, so that
you have in a typical step tn → tn+1 the coefficients of a polyno-
mial p(t) available. Note, that this polynomial is a vector valued
function p(t) = (pα (t), pα̇ (t))T with two components one interpo-
lating angles and another interpolating angular velocities.
6. Check if the integration passed the obstacle. For this end, you
have to check if the function pα (t) − αobst = 0 has a solution in
[tn , tn+1 ]. If you find a solution of this nonlinear equation, you
call it tobst . It is the time, when the obstacle is hit.
7. Compute αobst and α̇obst by evaluating the polynomial.
8. Restart your integration by setting the initial time to tobst and
by setting new initial conditions (as given above).
9. Perform your integration over 5 sec and with an appropriate fixed
step size h.
10. Make a plot of your result. A phase plot (α versus α̇) is quite
instructive.
Note, this exercise tries to put the content of the several chapters of the
course in a common application oriented environment. Here you should just
check to what extend you understood the material of the course and that
you can handle practical problems. In the theoretical exam you will be asked
a couple of questions around this problem and the other home assignments.
This exercise may appear hard as it consists of the combination of many
different steps. There will be help as usual. So just try.
2
Lycka till!
3
Flow Chart 1.1: Generic Integrator Call
Initialization:
a) model data: mass, gravity, etc
b) integrator control data: initial stepsize, tolerance, etc
tout=tstart:Dtout:tend
ODE solver:
on return: x(tout), error code see Flowchart 1.2
Error ?
No Yes
Save Data Error Handling
Break
Postprocessing
Flow Chart 1.2: Generic Single Step Integrator Call
Yes First Step? No

Initialization: h=h0
while t < tout
ODEstep solver:
on return: x(tout), error code
see Flowchart 1.3
Error ?
No Yes
Step size too small?
t=t+h Yes No
Error handling Redo step
with new
Break step size
Flow Chart 1.3:
Generic Single Step Integrator Organisation
Predict
new Jacobian needed?
Yes No
Compute Jacobian
Jacnew = 1 Jacnew = 0
Corrector iteration (Newton iteration)

return: solution or error code
Yes convergence ? No
Estimate Error Y Jacnew = 0 N
Y Error < Tol Redo the step
N Redo the step
with the same
with h=h/2
Accept step Reject step step size
increase and require a
decrease and require a
step size step size new Jacobian
new Jacobian

Complete

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Complete

Uploaded by

Copyright:

Available Formats

Numerical Methods

Lund, Jan/Feb. 2008

• Please report missprints or your comments to the teacher.

Repeat the definition of multiplicity of a root!

The algorithm based on this theorem is the bisection method.

We rewrite the problem

x is called a fixed point of g.

Theorem. [1.2] (Brower’s Fixed Point Theorem)

Root-finding problem: 0 = exp(x) − 2x − 1

Definition. [1.1] The iteration

is called fixed point iteration or simple iteration.

If it converges, then to a fixed point of g:

ξ = lim xk+1 = lim g(xk ) = g( lim xk ) = g(ξ)

>> x(1)=1;for k=1:17, x(k+1)=log(2*x(k)+1);end

1.2553 1.2558 1.2561 1.2562 1.2563 1.2564 1.2564 1.2564

Definition. [1.2] A function g is called a contraction (or contractive) on

|g(x) − g(y)| ≤ L|x − y| ∀x, y ∈ [a, b]

The intervall [x, y] shrinks to [g(x), g(y)].

If g is differentiable, then g 0(x) < 1 for all x ∈ [a, b].

Theorem. [1.3] g : [a, b] ⊂ R → R, continuous and g(x) ∈ [a, b],

• the iteration xk+1 = g(xk ) converges

By mean value theorem (medelvärdessats)

|g(x) − g(y)| ≤ |g 0(y)||x − y|

and g 0(x) = 2/(2x + 1). g 0(x) is a decreasing function.

Let’s check the speed of convergence:

|xk+1 − xk | = |g(xk ) − g(xk−1)| ≤ L|xk − xk−1|

0.5696 0.5695 0.5694 0.5694 0.5694 0.5694

This is called an a-posteriori estimate.

Analogously we can derive an a-priori bound

How many iterations do we need in our example for an accuracy of

Definition. [1.4] Assume limk→∞ xk = ξ.

• Asymptotic rate of convergence ρ = − log10 µ

ρ large ⇒ fast (linear) convergence

Check example on overhead 1.13 again.

Example (p.17): f (x) = ex − x − 2 = 0

ex − x − 2 = 0 ⇔ x = ln(x + 2) ⇔ x = ex − 2 ⇒ x = x(ex − x)/2

We construct now systematically g from f .

Definition. [1.5] A relaxed version of a fixed point iteration with f is

Iteration function: g(x) = x − λf (x)

Some (loose) concepts:

• λ should have the same sign as f 0(x), why?

• x0 should be sufficiently near ξ

• λ not too large

How to find such a λ, which also guarantees fast convergence?

We modify the relaxed iteration to

xk+1 = xk − λ(xk )f (xk )

Tangent: y = f 0(xk )(x − xk ) + f (xk )

Definition. [1.7] Assume limk→∞ xk = ξ.

• The iteration converges with order q iff

Newton’s method converges locally (as it is a fixed point iteration with a

and x0 ∈ Ih := [ξ − h, ξ + h] with h = min(1/A, δ), then Newton’s method

Simplified Newton’s Method

We consider the problem

and solve it first by MATLAB’s function fzero:

Note here the use of anonymous functions in MATLAB.

Here the result (use format long e !):

We write now our own bisection code

Check in the MATLAB help pages the use of feval !

Here the result (use format long e !):

>> [r,e,it]=mybisec(@(x) exp(-x)-x,[0,1],1.e-6)

How could the code mybisec be improved?

• Silently it was assumed that b > a. This should me checked during

L = max |g 0(x)| = e−0.1 = 0.905.

We expect slow convergence of fixed point iteration: