UNIVERSITY OF BAHRAIN

COLLEGE OF ENGINEERING
DEPARTMENT OF ELECTRICAL ENGINEERING

EENG 205: Numerical Analysis

Semester II, 2019/2020
Tentative schedule

Week Topic

1-2 Solution of nonlinear equations.

3-4 Solution of linear systems.

5-6 Interpolation and curve fitting.

7-8 Numerical differentiation and integration.

9-10 Solution of differential equations.

11-12 Solution of differential equations.

13-14 Other related topics.

Instructors Dr. Jawad Talaq

Textbook
Numerical Methods Using MATLAB, John H. Mathews and Kurtis D. Fink

References
Applied Numerical Methods with MATLAB for Engineers and Scientists, Steven C.
Chapra
Exams
Test I: 20% 26/3/2020 at 4:00-5:00 pm, Hall 15
Test II: 20% 14/5/2020 at 3:00-4:00 pm, Hall 15
Final 40%
H/W(assignments
+matlab programs
+ Quizzes) 20%
 1

Solution of nonlinear equations

Fixed point iteration
Bracketing methods
- Bisection method
- False position method (regula falsi method)
Newton-Raphson method
Secant method

Fixed point iteration

x  f x 
x k 1  f x k 

Condition or convergence
f x   1

Example

x  f x   0.6 x 2  0.4
has two fixed points at 2 and -1/3.
f   1.2 x
At the vicinity of the fixed point x=2
f x   1
The solution will not converge to this fixed point.

At the vicinity of the fixed point x= -1/3

f x   1
The solution will converge to this fixed point.

f x   1
Divergence
Fixed point at x = 2

f x   1
Convergence
Fixed point at x = – 1/3 yx
y  f x 

Fixed point iteration

 2

Bracketing methods
Bisection method
f x   0
Find x
Select two values of x (x0 and x1) that gives opposite signs of f (f0 and f1). The
solution must be in the interval between x0 and x1. Select x2 as the mean value of x0
and x1.
x 2  0.5  x 0  x1 
Find f2. Depending on the sign of f2, select either x0 or x1 which has the function, f,
opposite to the sign of f2. Start tee next iteration by choosing xk between x2 and the
selected x.
Example
f x   x 2  2 x  2 x  2  0
x 0  1.0  f 0  1.0000
x1  1.5  f1  .8005
x 2  0.5  x 0  x1   1.25  f 2  0.1736
x3  0.5  x 2  x1   1.375  f 3  0.2954
x 4  0.5  x 2  x3   1.3125  f 4  0.0564
x5  0.5  x 2  x 4   1.28125  f 5  0.0597
x 6  0.5  x5  x 4   1.29687  f 6  0.001973

f1
f2
f0
x1
x2
x0

f
 3

False position method (regula falsi method)

Connect the line between f0 and f1 to obtain the intersection point with the x axis (x2).
f1  f 0 f 0 0  f0
 1  f
x1  x 0 x1  x 2 x 2  x 0
giving f0
x1  x 0
x2  x0   f0 x1
f1  f 0 x
x0 x2
or f1
x  x0
x 2  x1  1  f1
f1  f 0

Example
f x   x 2  2 x  2 x  2  0
x 0  1.0  f 0  1.0000
x1  1.5  f1  .8005
x  x0
x2  x0  1  f 0  1.2777  f 2  -0.07279
f1  f 0
x 2  x1
x 3  x1   f 1  1.29622  f 3  -0.00437
f 2  f1
x 3  x1
x 4  x1   f 1  1.29734  f 4  -0.0002597
f 3  f1
x 4  x1
x 5  x1   f 1  1.297402  f 5  -1.5408e - 5
f 4  f1
x 5  x1
x 6  x1   f 1  1.297406  f 6  -9.1414e - 7
f 5  f1

f
 4

Newton-Raphson method

It needs a single point to start. It requires the evaluation of the derivative of the
function at each iteration.

f x   0

Start at x0 and obtain f0.

f0
x1  x 0 
f 0

In general
fk
x k 1  x k 
f k

where k is the iteration number and f k is the derivative of f with respect to x (df/dx).

Example
f x   x 2  2 x  2 x  2  0
df 1
f  2x  2
dx x
x 0  1.0  f 0  1.0000 and f 0  3.0000
f0
x1  x 0   1.3333  f 1  0.13504 and f 1  3.8006
f 0
f1
x 2  x1   1.29780  f 2  0.00147 and f 2  3.71780
f 1
f2
x3  x 2   1.29740  f 3  1.8284e - 7 and f 3  3.7168
f 2
f3
x 4  x3   1.297406  f 4  2.6645e - 15
f 3

f
 5

Secant method

Similar to the false position method. It requires two points to start. No need to have
two values of opposite signs. The starting points can be close to each other. It is not
required to evaluate the derivative.

Selecting x0 and x1. Then x2 is

x1  x 0
x 2  x1   f1
f1  f 0

In general
x k  x k 1
x k 1  x k   fk
f k  f k 1

Example
f x   x 2  2 x  2 x  2  0

x 0  1.0  f 0  1.0000
x1  1.1  f 1  0.6876
x1  x 0
x 2  x1   f 1  1.32012  f 2  0.085029
f1  f 0
x 2  x1
x3  x 2   f 2  1.29589  f 3  -0.005608
f 2  f1
x3  x 2
x 4  x3   f 3  1.297395  f 4  -3.9780e - 5
f3  f2

f
 6

EENG205: Assignment #1

Solution of nonlinear equations

Q1. Given the function

g  x   4  4 x 
1 2
x
2
a- Sketch the graph of g(x) and y = x for x  [0,6] and show that points P1 at x=2 and P2
at x=4 are fixed points.
b- Compute four values of fixed point iteration (x1, x2, x3, x4) with x0=1.8, x0=2.2, x0=3.8,
x0=4.2 and x0=6.2.
c- Can a solution converge to P1? Why?
d- Can a solution converge to P2? Why?

Q2. Use four values of fixed point iteration to solve the following problems.
3
a- Finding 2 12 and 12 .
b- x  cos(x) .
c- x  10 sin( x) .
d- x2  ex .

Q3. Use four iterations of the bisection method to solve the following problems.

a- e x  2  x  0 . Start at [a,b] = [-2.4,-1.6].

b- cos( x)  1  x  0 . Start at [a,b] = [0.8,1.6].
c- ln( x)  5  x . Start at [a,b] = [3.2,4.0].
d- x  10 sin( x)  0 . Start at [a,b] = [1.0,3.0].
2

Q4. Repeat Q3. using regula falsi method.

Q5. Use four iterations of Newton Raphson method to solve the following problems.

a- x 3  3x  2  0 . Start at x0 = 2.1.
b- x  e  x  0 . Start at x0 = 0.
c- x 2  2 x  e  x  0 . Start at x0 = 0.
d- x  cos( x)  2 sin( x)  0 . Start at x0 = 0.5.

Q6. Use four iterations of the secant method to solve the following problems.

a- x 2  x  3  0 . Start at x0 = 1.0 and x1 = 2.0.

x
b- x  e  0 . Start at x0 = 2.0 and x1 = 1.0.
2

c- cos( x)  sin( x)  0 . Start at x0 = 0.0 and x1 = 0.5.

d- cos( x )  sin( x)  0 . Start at x0 = 0.0 and x1 = 0.5.
2
 7

Solution of linear systems

Equivalent systems
Lower and upper triangular systems
Back and forward substitutions
Gaussian elimination
Triangular factorization
Iterative methods
Jacobi iteration
Gauss Seidel iteration
Newton Raphson iteration
Equivalent systems
The following two nxn dimensional systems are equivalent if the solution set x is the
same for both systems.
A 1 x 1  b1
A2x2  b2
If the solution set are equal (x1 = x2), then both systems are equivalent.
Example
The following two systems are equivalent because they have the same solution set.
A1  2 4 ; b1  8 
5 2  6

A2  1 2 ; b2   4 
3  2   2

System 1
A 1 x 1  b1
2 x1  4 x 2  8
5x1  2 x 2  6
Solution is x1=0.5 and x2=1.75.
x 1  0.50
1.75 

System 2
A2x2  b2
1x1  2 x 2  4
3x1  2 x 2  2
Solution is x1=0.5 and x2=1.75.
x 2  0.50
1.75 

Lower and upper triangular systems

Lower triangular system
All elements above the diagonal are zeroes
a11 0 0 
A  L  a 21 a 22 0 
a 31 a 32 a 33 

Upper triangular system

All elements below the diagonal are zeroes
a11 a12 a13 
A  U  0 a 22 a 23 
0 0 a 33 
 8

Back substitution
If a system is transformed to an upper triangular system, then back substitution is used
to determine the solution.
a11 a12 a13   x1  b1 
0 a 22 a 23   x 2   b2 
0 0 a 33   x 3  b3 
x3  b3 / a33
x 2  b2  a 23 x3  / a 22
x1  b1  a12 x 2  a13 x3  / a11

Forward substitution
If a system is transformed to a lower triangular system, then forward substitution is
used to determine the solution.
a11 0 0   x1  b1 
a 21 a 22 0   x 2   b2 
a 31 a 32 a 33   x 3  b3 
x1  b1 / a11
x 2  b2  a 21 x1  / a 22
x3  b3  a31 x1  a32 x 2  / a33

Gaussian elimination
A system can be transformed to an upper triangular system by Gauss elimination.
Construct the augmented matrix containing matrices A and b
a11 a12 a13 b1  a11 a12 a13 b1 
a 21 a 22 a 23 b2  a 21 a 22 a 23 b2 
a 31 a 32 a 33 b3  a 31 a 32 a 33 b3 

Eliminate the elements of the first column starting from the second row.
a11 a12 a13 b1  a11 a12 a13 b1 
0 a 22 a 23 b2  a 21 a 22 a 23 b2 
0 a 32 a 33 b3  a 31 a 32 a 33 b3 
a 21 a 31
a 22  a 22  a12 a 32  a 32  a12
a11 a11
a 21 a 31
a 23  a 23  a13 a 33  a 33  a13
a11 a11
a 21 a 31
b2  b2  b1 b 3  b3  b1
a11 a11

Eliminate the elements of the second column starting from the third row.
a11 a12 a13 b1  a11 a12 a13 b1 
0 a 22 a 23 b2  a 21 a 22 a 23 b2 
0 0  a 31 a 32 a 33 b3 
aˆ 33 bˆ3 
a 32
aˆ 33  a 33  a 23
a 22
a
bˆ3  b3  32 b2
a 22
Then the system can be solved using back substitution.
Example
2 1 4 1 
Solve x from Ax = b; A  4 3 6  ; b  0  using Gauss elimination and back
1 1 3 2
substitution.
The augmented matrix is
 9

2 1 4 1
4 3 6 0
1 1 3 2

Gauss elimination on the first column

2 1 4 1 
0 1 2 2 
0 0 .5 1 1.5

Gauss elimination on the second column

2 1 4 1 
0 1 2 2 
0 0 2 2.5

Using back substitution:

x3  2.5 / 2  1.25
x 2   2  2 x3  / 1  0.5
x1  1  x 2  4 x3  / 2  2.25

Triangular factorization
Matrix A is transformed to a product of lower and upper triangular forms.
A  LU

1 0 0
L  m 21 1 0 
m31 m32 1 

u11 u12 u13 

U  0 u 22 u 23 
0 0 u 33 

The elements of L and U are obtained as follows

1 0 0 a11 a12 a13 
A  0 1 0  a 21 a 22 a 23 
0 0 1 a 31 a 32 a 33 
Perform Gauss elimination on the first column
 
 
1 0 0
a11 a12 a13 
a
A   21 1 0   0 a 22 a 23 
 a11  0 a 33 
 
a 32
a
 31 0 1
 a11 

Perform Gauss elimination on the second column

 
 
1 0 0
a11 a12 a13 
a
A   21 1 0   0 a 22 a 23 
 a11  0 aˆ 33 
a a 32   0
 31 1
 a11 a 22 

Solving Ax=b using triangular factorization

LUx = b
Let Ux = y
Solve Ly = b by forward substitution.
Solve Ux = y by back substitution.
 10

Example
2 1 4 1 
Perform triangular factorization on A  4 3 6 and solve Ax = b; b  0  .
1 1 3 2
1 0 0  2 1 4
A  0 1 0   4 3 6
0 0 1 1 1 3
1 0 0  2 1 4
A  2 1 0   0 1  2
0.5 0 1 0 0.5 1 
1 0 0  2 1 4
A  2 1 0   0 1  2
0.5 0.5 1 0 0 2 
1 0 0
L  2 1 0
0.5 0.5 1
2 1 4
U  0 1  2
0 0 2 

Let Ux = y
Solving Ly = b by forward substitution
1 
y   2
 2.5
Solving Ux = y by back substitution
 2.25 
x   0 .5 
 1.25

Iterative methods
Jacobi iteration
Consider the linear system Ax = b
a11 a12 a13   x1  b1 
a 21 a 22 a 23    x 2   b2 
a 31 a 32 a 33   x 3  b3 
a11 x1  a12 x 2  a13 x3  b1
a 21 x1  a 22 x 2  a 23 x3  b2
a31 x1  a32 x 2  a33 x3  b3

x1 (k  1) 
1
b1  a12 x 2 (k )  a13 x 3 (k ) 
a11

x 2 (k  1) 
1
b2  a 21 x1 (k )  a 23 x 3 (k ) 
a 22

x 3 (k  1) 
1
b3  a 31 x1 (k )  a 32 x 2 (k ) 
a 33
where k is the iteration number.
Gauss Seidel iteration
Same as Jacobi except that the latest values are used in each iteration.
x1 (k  1) 
1
b1  a12 x 2 (k )  a13 x 3 (k ) 
a11

x 2 (k  1) 
1
b2  a 21 x1 (k  1)  a 23 x 3 (k ) 
a 22

x 3 (k  1) 
1
b3  a 31 x1 (k  1)  a 32 x 2 (k  1) 
a 33
 11

Example
Use Jacobi and Gauss Seidel iteration methods to solve the system Ax = b.
Start at x = [0 0 0]T.
2 1 4 1 
A  4 3 6  ; b  0  .
1 1 3 2

Jacobi iteration Gauss Seidel iteration

x1 (k  1)  1  x 2 (k )  4 x 3 (k )  1  x 2 (k )  4 x3 (k )
1 1
x1 (k  1) 
2 2
x 2 (k  1)  0  4 x1 (k )  6 x 3 (k )  x 2 (k  1)  0  4 x1 (k  1)  6 x 3 (k ) 
1 1
3 3

x 3 (k  1)  2  x1 (k )  x 2 (k )  x 3 (k  1)  2  x1 (k  1)  x 2 (k  1) 
1 1
3 3

Jacobi iteration Gauss Seidel iteration

Iteration x1 x2 x3 x1 x2 x3
0 0 0 0 0 0 0
1 0.5 0 2/3 0.5000 -0.6667 0.7222
2 -0.8333 -2.000 0.5000 -0.6111 -0.6296 1.0802
3 0.5000 0.1111 1.6111 -1.3457 -0.3663 1.2373
4 -2.7778 -3.8889 0.4630 -1.7915 -0.0860 1.2925
5 1.5185 2.7778 2.8889 -2.0420 0.1377 1.3014

20 Divergence -2.2500 0.5001 1.2500

Gauss Seidel for nonlinear systems of n equations

x1 (k  1)  f1 ( x1 (k ), x 2 (k ), x3 (k ), ....... , x n (k ))
x 2 (k  1)  f 2 ( x1 (k  1), x 2 (k ), x3 (k ), ....... , x n (k ))
x3 (k  1)  f 3 ( x1 (k  1), x 2 (k  1), x3 (k ), ....... , x n (k ))
Always use the latest values o x in the functions.
Newton-Raphson for nonlinear systems of n equations
It requires the evaluation of the derivatives of the functions at each iteration. The
derivatives are arranged in a matrix known as the Jacobian matrix

f x   0 is a vector of n functions o n variables.

 f 1 f 1 f 1 
 x x x 3
....... 
f x   f 2 f 2 f 2 
1 2
J  .......  is the Jacobian matrix
x  x1 x 2 x 3 
 
......................................
 

Start at x0 and obtain f0 and J.

x1  x 0  J 1 f 0

In general
x k 1  x k  J 1 f k
 12

Example
Solve the following nonlinear system using Gauss Seidel and Newton Raphson
methods. Start at x1 = x2 = 1.

x1  2  x1 x 2
x 2  x12  0.5 x1

Gauss Seidel
Iteration x1 x2
0 1 1
1 1 0.5
2 1.5 1.5
3 -0.25 0.1875
4 2.0469 3.1663
5 Divergence

Newton Raphson
Let
f1  2  x1 x 2  x1  0
f 2  x12  0.5 x1  x 2  0

f 1 f 1
 x2 1 ;   x1
x1 x 2
f 2 f 2
 2 x1  0.5 ;  1
x1 x1

Iteration 1

f  1 0 
f
 f 2   0.5

J   2  1 
1.5  1

x 1  x 0  J 1  f   1.1429 
 0.7143

Iteration 2

f   1    0.0408
f
 f 2   0.0204

J   - 1.7143 - 1.1429
 1.7857 - 1.0000

x 2  x1  J 1  f   1.1475 
 0.7430

Iteration 3

f   1   - 0.1338.10 3
f
 f 2   0.0217

J   - 1.7430 - 1.1475
 1.7950 - 1.0000

x 3  x 2  J 1  f   1.1475 
 0.7430
 13

EENG205: Assignment #2

Solution of linear systems

Q1. Show that the following systems AX=B and UX=Y are equivalent and solve for X using back
substitution.

 2 4  6  4   2 4  6  4 

a- A  1 5
   
3 ; B   10 and U  0 3 6 ; Y   12


1 3 2   5  0 0 3   3 
 5 2  1  1  5 2  1    1
b- A 
       
 1 0 3; B   5  and U   0 0.4 2.8; Y   4.8
 3 1 6   17   0 0  10  10
Q2. Use Gaussian elimination and back substitution to determine the parabola and cubic
parameters A, B, C and D for the following systems.

a- y  A  Bx  Cx 2 that passes through (1,4), (2,7) and(3,14).

b- y  A  Bx  Cx 2  Dx 3 that passes through (0,0), (1,1), (2,2) and(3,2).
Q3. Find the triangular factorization of the following matrices and solve the system AX=B using
forward and back substitutions after factorization.

 1 1 0 4 1 
 5 2  1 1   2 1 0  0 
A   1 0 
3; B  2
5
a- b- A   ;B 
 5 2 1 2 1 
 3 1 6  0    
 3 0 2 6  2
Q4. Use Jacobi iteration method to solve AX=B for X for the following linear systems up to 5
iterations.
1  5  1  8 
4  1 15 
a- A  ;B  b- A  4
 1 
 1 ; B   13
1 5  9 2  1  6  2
Q5. Repeat Q4 using Gauss-Seidel iteration method.

Q6. Use four iterations of Seidel iteration method to solve the following nonlinear system. Use x 1 =
– 0.3 and x2 = – 1.3 for staring.

x1  g1 ( x1 , x 2 ) 
1
7
 3 x1  3 x12  x13  x 2
1

x 2  g 2 ( x1 , x 2 )   x1  2 x 2  x 22  2
2

Q7. Use four iterations of Newton Raphson method to solve the following nonlinear system. Use x 1
= 1.2 and x2 = 1.2 for staring.

f1 ( x1 , x2 )  x12  x2  0.2  0
f 2 ( x1 , x2 )  x22  x1  0.3  0
Q8. Repeat Q7 using x1 = – 0.2 and x2 = – 0.2 for staring
 14

Interpolation and curve fitting

Taylor series expansion
Lagrange polynomials
Newton polynomials
Taylor series expansion

f k   x 0 
f x     x  x 0 k
k 0 k!!

Example

Evaluate Taylor series expansion for the following functions. Determine the
approximate function values at x = 1 using the first few terms.

a- f x   e x b- f x   sinx  c- f x   cosx 

a- f x   e x

Choosing x0 = 0
df x  d k f x 
f 0  x 0   e 0  1 ; f 1 x 0    ex 1; f k   x 0   k
 ex 1
dx 0 dx 0

f x  
x  00  x  01  x  02  x  03  x  04  
0! 1! 2! 3! 4!
x2 x3 x4
f x   1  x     
2! 3! 4!
e1  2.708333 using the first five terms

b- f x   sinx 

Choosing x0 = 0
df x  d 2 f x 
f 0  x 0   sinx 0   0 ; f 1 x 0    cosx  0  1 ; f 2   x 0     sin x  0  0 ;
dx dx 2
d 3 f x  d 4 f x 
f 3 x 0     cosx  0  1 ; f 4   x 0    sin x  0  0
dx 3 dx 4

f x   0 
 x  0 0  1
x  01  0  x  02  1
x  03  0
 x  0 4  
0! 1! 2! 3! 4!
x3 x5 x7
f x   x     
3! 5! 7!
sin1  0.841468 using the first four terms
c- f x   cosx 

Choosing x0 = 0
df x  d 2 f x 
f 0  x 0   cosx 0   1 ; f 1 x 0     sinx  0  0 ; f 2   x 0     cosx  0  1 ;
dx dx 2
d 3 f x  d 4 f x 
f 3 x 0   3
 sin x  0  0 ; f 4  x 0    cos x  0  1
dx dx 4

f x   1 
 x  0 0  0
x  01  1 x  02  0
x  03  1
 x  0 4  
0! 1! 2! 3! 4!
x2 x4 x6
f x   1     
2! 4! 6!
cos1  0.540277 using the first four terms
 15

Lagrange polynomials
A function f(x) is approximated to Lagrange polynomial PN(x) of degree N.
N
PN x    y k  L N ,k x 
k 0
 y 0 L N ,0 x   y1 L N ,1 x   y 2 L N , 2 x          y N L N , N x 

 jj 0k x  x j 
N

L N ,k x  
 jj 0k x k  x j 
N

x  x 0   x  x1  x  x 2       x  x k 1   x  x k 1       x  x N 

x k  x 0   x k  x1   x k  x 2       x k  x k 1   x k  x k 1      x k  x N 

LN,k(x) is the Lagrange coefficient polynomial.

Example
If y = f(x) = cos(x) over [0.0,1.2]
a- Use two nodes x0 = 0.2 and x1 = 1.0 to construct Lagrange linear polynomial.
b- Use three nodes x0 = 0.0, x1 = 0.6 and x2 = 1.2 to construct Lagrange quadratic
polynomial.
c- Use four nodes x0 = 0.0, x1 = 0.4, x2 = 0.8 and x3 = 1.2 to construct Lagrange
cubic polynomial.

a- x0 = 0.2, x1 = 1.0
y0 = cos(x0) = 0.980067
y1 = cos(x1) = 0.540302
P1 x   y 0 L1,0 x   y1 L1,1 x 
x  x1  x  1.0
L1,0  x   
x 0  x1  0.2  1.0
L1,1 x  
x  0.2  x  0.2
x1  x 0  1.0  0.2
P1 x   0.980067
x  1.0  0.540302 x  0.2
0.2  1.0 1.0  0.2
b- x0 = 0.0, x1 = 0.6, x2 = 1.2
y0 = cos(x0) = 1.0
y1 = cos(x1) = 0.825336
y2 = cos(x2) = 0.362358
P2 x   y 0 L2,0 x   y1 L2,1 x   y 2 L2, 2 x 
x  x1   x  x 2  x  0.6  x  1.2
L 2, 0  x   
x 0  x1   x 0  x 2  0.0  0.6  0.0  1.2
x  x 0   x  x 2   x  0 . 0    x  1 .2 
L 2,1 x   
x1  x 0   x1  x 2  0.6  0.0  0.6  1.2
x  x 0   x  x1   x  0 . 0    x  0 .6 
L 2, 2  x   
x 2  x 0   x 2  x1  1.2  0.0  1.2  0.6
P2 x   1.0
x  0.6 x  1.2  0.825336 x  0.0 x  1.2  0.362358 x  0.0 x  0.6
0.0  0.6 0.0  1.2 0.6  0.0 0.6  1.2 1.2  0.0 1.2  0.6
c- x0 = 0.0, x1 = 0.4, x2 = 0.8, x3 = 1.2
y0 = cos(x0) = 1.0
y1 = cos(x1) = 0.921061
y2 = cos(x1) = 0.696707
y3 = cos(x2) = 0.362358
P3 x   y 0 L3,0 x   y1 L3,1 x   y 2 L3, 2 x   y 3 L3,3 x 
 16

x  x1  x  x 2   x  x 3  x  0.4  x  0.8  x  1.2

L3,0 x   
x 0  x1  x 0  x 2   x 0  x 3  0.0  0.4  0.0  0.8  0.0  1.2
x  x 0   x  x 2   x  x 3   x  0 .0    x  0 . 8    x  1 .2 
L3,1 x   
x1  x 0   x1  x 2   x1  x 3  0.4  0.0  0.0  0.8  0.0  1.2
x  x 0   x  x1   x  x 3   x  0 .0    x  0 .4    x  1 .2 
L3, 2 x   
x 2  x 0   x 2  x1  x 2  x 3  0.8  0.0  0.8  0.4  0.8  1.2
x  x 0   x  x1   x  x 2  x  0.0  x  0.4  x  0.8
L3,3 x   
x 3  x 0   x 3  x1   x 3  x 2  1.2  0.0  1.2  0.4  1.2  0.8

P2(x)
P3(x)

P1(x)

P3(x) - f(x)

P2(x) - f(x)

P1(x) - f(x)

Newton polynomials
 17

Newton polynomials
Having N nodes from x0 to xN-1, then Newton polynomial approximation of f(x) is:
PN x   a 0  a1  x  x 0   a 2  x  x 0  x  x1       a N  x  x 0  x  x1    x  x N 1 

The coefficients a k are functions of the divided differences.

a k  f x 0 , x1 , x 2 ,  , x k 

The following divided differences are defined

f x k   f x k 

f x k   f x k 1 
f x k 1 , x k  
x k  x k 1

f x k 1 , x k   f x k  2 , x k 1 
f x k  2 , x k 1 , x k  
x k  x k 2

f x k  2 , x k 1 , x k   f x k 3 , x k  2 , x k 1 
f x k 3 , x k  2 , x k 1 , x k  
x k  x k 3

Example

Find Newton polynomial approximation P1 x  , P2 x  and P3 x  for the function

f x   x 3  4 x using the nodes x0 = 1, x1 = 2, x2 = 3 and x3 = 4.

The divided differences are

f x 0   f x 0   3

f x1   f x1   0

f x 2   f x 2   15

f x3   f x3   48

f x1   f x 0 
f x 0 , x1   3
x1  x 0

f x 2   f x1 
f x1 , x 2    15
x 2  x1

f x 3   f x 2 
f x 2 , x 3    33
x3  x 2

f x1 , x 2   f x 0 , x1 
f x 0 , x1 , x 2   6
x2  x0

f x 2 , x 3   f x1 , x 2 
f x1 , x 2 , x 3   9
x 3  x1

f x1 , x 2 , x 3   f x 0 , x1 , x 2 
f x 0 , x1 , x 2 , x 3   1
x3  x0
 18

The coefficients are

a 0  f x 0   3

a1  f x0 , x1   3

f x1 , x 2   f x 0 , x1 
a 2  f x 0 , x1 , x 2   6
x2  x0

f x1 , x 2 , x 3   f x 0 , x1 , x 2 
a 3  f x 0 , x1 , x 2 , x 3   1
x3  x0

Newton polynomials are

P1 x   a 0  a1  x  x 0 

P1 x   3  3  x  1

P2 x   a 0  a1  x  x0   a 2  x  x0  x  x1 

P2 x   3  3  x  1  6  x  1 x  2

P3 x   a 0  a1  x  x 0   a 2  x  x0  x  x1   a3  x  x 0  x  x1  x  x 2 

P3 x   3  3  x  1  6  x  1 x  2  1 x  1 x  2 x  3

P3(x)

f(x)

P2(x)

P1(x)

Newton polynomials
 19

Curve fitting
Least square fit
Splines
The least square fit
Having N measurements, (xk,yk), we need to find a function that best fits the
measurement points. For the least square fit, we minimize the following index
Minimize J
2
1 N
J   y k  f x k 
2 k 1
y

The function f(x) may be linear, quadratic etc.

The linear fit
f x   a  b  x
1 x
 N  N 
a  
N  xk   y k 
Curve fitting
k 1   k 1 
    
b   N N
2  N 
 x k  x k   x k y k 
 k 1 k 1   k 1 

The quadratic fit

f x   a  b  x  cx 2
1
 N N  N 
N  xk  x k2   y k 
 k 1 k 1   k 1 
a     
  N
3 N 
N N
b    x k  x k2  xk    x k y k 
   k 1 k 1 k 1   k 1 
c     
N 2 N N
4  N

 x k  x k3  xk   x k y k 
2

 k 1 k 1 k 1   k 1 

The exponential fit

f x   ae b x
ln f x   ln a  b  x
ln y k  ln a  b  x k
1
 N  N 
N
ln a  
 xk   ln y k 
k 1   k 1 
    
b   N N
2 N 
 x k  x k   x k ln y k 
 k 1 k 1   k 1 

Example
Find the linear, quadratic and exponential fit functions that best fit the following five
points.
(0.0,1.5), (1.0,2.5), (2.0,3.5), (3.0,5.0) and (4.0,7.5)
x1 = 0.0, x2 = 1.0, x3= 2.0, x4 = 3.0, x5 = 4.0
y1 = 1.5, y2 = 2.5, y3= 3.5, y4 = 5.0, y5 = 7.5
N=5
 x  10.00  x 2  30.00  x 3  100.00  x 4  354.00
 y  20.00  xy  54.50  x 2 y  181.50  ln y  6.198859  x ln y  16.309742
 20

The linear fit

f x   a  b  x
1

a   
N  x k   y k  1
   5 10  20.0  1.10

b      10 30 54.5  1.45
 x k  x k2   x k y k 
f x   1.10  1.45  x

The quadratic fit

f x   a  b  x  cx 2
1
N

 x k  x k2   y k 
  1
a       5 10 30   20.00 1.60 
b    x k  x k2  x k3    x k y k   10 30 100    54.50   0.45
c      30 100 354 181.50  0.25
 x k2  x k3  x k4   x k2 y k 
   
 
f x  1.60  0.45  x  0.25  x 2

The exponential fit

f x   ae b x
ln f x   ln a  b  x
ln y k  ln a  b  x k
1
N
ln a     x k   ln y k  1
   5 10   6.198859   0.457367

b      10 30 16.309742 0.391202
 x k  x k2   x k ln y k 
ln a  0.457367
a  e 0.457367  1.579909
f x   1.579909 e 0.391202  x

exponential

quadratic

linear

Least square curve fitting

 21

Spline functions
Linear spline
Quadratic spline
Cubic spline
Linear splines

Piecewise linear functions are constructed between adjacent points to form linear
splines.
y
S 0  y 0  d 0 x  x 0  S0
S1 S3
S1  y1  d1 x  x1 
S2
y0
S 2  y 2  d 2 x  x 2 
..............
S k  y k  d k x  x k  x0 x1 x2 x3 x
S k 1  y k 1  d k 1 x  x k 1  Liner splines

where d k is the slope of the line between points at k and k+1.

 y k 1  y k 
dk 
x k 1  x k 
Disadvantage: First derivatives are not continuous

Quadratic splines

Piecewise quadratic functions are constructed between adjacent points to form

quadratic splines. y
S0 S1 S3
S 0  s 0,0  s 0,1 x  x 0   s 0, 2 x  x 0 2 S2
y0
S1  s1,0  s1,1 x  x1   s1, 2 x  x1 2
..............
S k  s k ,0  s k ,1 x  x k   s k , 2 x  x k 2 x0 x1 x2 x3 x
Quadratic splines
where s k ,0 , s k ,1 and s k ,2 are constants determined from the quadratic polynomials using
the data points and some other conditions at the nodes such as equality of first
derivatives.
Disadvantage: Second derivatives are not continuous.

Cubic splines

Piecewise cubic functions are constructed between adjacent points to form cubic
splines. y
S 0  s 0,0  s 0,1 x  x 0   s 0, 2 x  x 0 2  s 0,3 x  x 0 3 S0 S1 S3
y0 S2
S1  s1,0  s1,1 x  x1   s1, 2 x  x1 2  s1,3 x  x1 3
..............
S k  s k ,0  s k ,1 x  x k   s k , 2 x  x k 2  s k ,3 x  x k 3 x0 x1 x2 x3 x
Cubic splines

Advantages: First derivatives are smooth and continuous.

Second derivatives are continuous.
 22

The constants s k ,0 , s k ,1 and s k ,2 are determined as follows

The following linear matrix equation is solved for m1 to mN-1.

2h0  h1  h1  m1  v1 

 h1 2 h1  h2  h2   m 2  v 2 
 h2 2h2  h3  h3   m 3  v 3 
 h3 2h3  h4  h4   m 4  v 4 
   
     
     
 2h N  2  h N 1  m N 1  v N 1 
 hN 2

where

hk  x k 1  x k

y k 1  y k y  yk
dk   k 1
x k 1  x k hk

u k  6  d k  d k 1 

v1  u1  h0 m0

v N 1  u N 1  h N 1 m N

vk  u k for k  1 and k  N  1

m0 and m1 depend on the spline type.

m0  m1  0 for the natural spline

s k ,0  y k

hk 2m k  m k 11 
s k ,1  d k 
6
mk
s k ,2 
2
m k 1  m k
s k ,3 
6h k

Example

Given the points (0.0,0.0), (1.0,0.5), (2.0,2.0), (3.0,1.5), construct natural cubic
splines.
x0 = 0.0, x1 = 1.0, x2= 2.0, x3 = 3.0
y0 = 0.0, y1 = 0.5, y2= 2.0, y3 = 1.5

h0 = h1 = h2 = 1.0
d0 = 0.5 d1 = 1.5 d2 = – 0.5
u1 = 6.0 u2 = –12.0
Assuming natural splines
m0 = 0.0 m3 = 0.0
v1 = u1 – h0m0 = 6.0
v2 = u2 – h2m3 = – 12.0
 23

Solve
2h0  h1  h1   m1   v1 
 h1 2 h1  h2   m 2  v 2 
4 1   m1    6.0
1 4 m 2   12.0
1
m1   4 1   6 . 0    2 .4 
m 2  1 4  12.0  3.6

h0 2m 0  m1 
s 0,0  y 0  0.0 s 0,1  d 0   0.5  2.4 / 6  0.1
6
m0 m  m0
s 0, 2   0.0 s 0,3  1  2.4 / 6  0.4
2 6h1
h1 2m1  m 2 
s1,0  y1  0.5 s1,1  d 1   1.5  1.2 / 6  1.3
6
m1 m  m1
s1, 2   1.2 s1,3  2  6.0 / 6  1.0
2 6h 2

h2 2m 2  m3 
s 2,0  y 2  2.0 s 2,1  d 2   0.5  7.2 / 6  0.7
6
m2 m  m2
s 2, 2   1.8 s 2,3  3  3.6 / 6  0.6
2 6h3
S k  s k ,0  s k ,1 x  x k   s k , 2 x  x k 2  s k ,3 x  x k 3
S 0  0.1 x  0  0.4 x  0 3
S1  0.5  1.3 x  1  1.2 x  12  1 x  13
S 2  2.0  0.7 x  2  1.8 x  22  0.6 x  23

S 0 S1
S2

S1
S 0

S0 S 2

S1
S 2

Cubic splines
 24

Example

Given the points (0.0,0.0), (1.0,0.5), (2.0,2.0), (3.0,1.5), construct quadratic splines
that have continuous first derivatives at the nodes and zero first derivative at the end
point.
x0 = 0.0, x1 = 1.0, x2= 2.0, x3 = 3.0
y0 = 0.0, y1 = 0.5, y2= 2.0, y3 = 1.5
S 0  s 0,0  s 0,1 x  x 0   s 0, 2 x  x 0 2
S1  s1,0  s1,1 x  x1   s1, 2 x  x1 2
S 2  s 2,0  s 2,1 x  x 2   s 2, 2 x  x 2 2

We have nine parameters to be determined. We need nine independent equations.

Spline S0
y 0  S 0  s 0, 0 at node (x0,y0) ------ (1)
y1  S 0  s 0,0  s 0,1 x1  x 0   s 0, 2 x1  x 0 2 at node (x1,y1) ------ (2)
Spline S1
y1  S1  s1,0 at node (x1,y1) ------ (3)
y 2  S1  s1,0  s1,1 x 2  x1   s1, 2 x 2  x1 2 at node (x2,y2) ------ (4)
Spline S2
y 2  S 2  s 2, 0 at node (x2,y2) ------ (5)
y 3  S 2  s 2,0  s 2,1 x 3  x 2   s 2, 2 x 3  x 2  2
at node (x3,y3) ------ (6)
Node (x1,y1)
S 0  S1
s 0,1  2s 0, 2 x1  x 0   s1,1 at node (x1,y1) ------ (7)
Node (x2,y2)
S1  S 2
s1,1  2s1, 2 x 2  x1   s 2,1 at node (x2,y2) ------ (8)
Node (x3,y3)
S 2  0
s 2,1  2s 2, 2 x3  x 2   0 at node (x2,y2) ------ (9)
Note that
x1  x0   1
x1  x 0 2  1
x 2  x1   1
x 2  x1 2  1
x 3  x 2   1
 x 3  x 2 2  1
 25

Rearranging the nine equations in matrx form

s 
1 0 0 0 0 0 0 0 0   0, 0   y 0 
1 1 1 0 0 0 0 0 0   s 0,1   y1 
0 0 0 1 0 0 0 0 0   s 0, 2   y1 
0 0 0 1 1 1 0 0 0   s1,0   y 2 
0 0 0 0 0 0 1 0 0    s1,1    y 2 
0 0 0 0 0 0 1 1 1   s1, 2   y 3 
0 1 2 0 1 0 0 0 0   s  0 
0 0 0 0 1 2 0 1 0   s 2 , 0  0 
0
 0 0 0 0 0 0 1 2   2,1  0 
 s 2, 2 

Solving these equations, we obtain

s0,0 = + 0.0 s0,1 = – 3.0 s0,2 = + 3.5
s1,0 = + 0.5 s1,1 = + 4.0 s1,2 = – 2.5
s2,0 = + 2.0 s2,1 = – 1.0 s2,2 = + 0.5
The quadratic splines are
S 0  3.0x  x 0   3.5x  x 0 2
S1  0.5  4.0x  x1   2.5x  x1 2
S 2  2.0  x  x 2   0.5x  x 2 2

Note that the first derivatives are continuous but not smooth. Compare this with the
cubic splines that have continuous and smooth first derivatives. Therefore, cubic
splines are preferable.

S 0

S1

S 0 S1
S2
S 2

S0
S 2

S1

Quadratic splines
 26

EENG205: Assignment #3

Interpolation and curve fitting

Q1. Use Taylor series expansion to expand the following functions up to degree 5.

c- f ( x)  ( x  2)1/ 2 using x0  2
d- f ( x)  1 / ( x  1) using x0  1

f ( x)  e  x / 2 using x0  0
2
e-

Q2. Find Lagrange polynomial functions for the following data points.
a. (1, 2) and (4, 5).
b. (1, 2), (4, 5) and (6, 2). Interpolate to find f(5).
c. (1, 2), (4, 5), (6, 2) and (10, 6). Interpolate to find f(8).
d. (1, 2), (4, 5), (6, 2), (10, 6) and (12, 3).
Q3. Repeat Q2 using Newton polynomial functions.

Q4. Design a linear spline for the data points given in Q2 b and c. Interpolate to find f(5) and f(8)
in parts b and c respectively. Compare the result with Q2.
Q5. Repeat Q4 using a quadratic spline.

Q6. Repeat Q4 using a cubic spline.

Q7. Construct linear functions that best fits the following data points using the least square method
of curve fitting.

a- (0.1, 2), (0.3, 4), (0.6, 8) and (1.2, 30). Interpolate to find f(0.9).
b- (0.1, 2), (0.3, 4), (0.6, 8), (1.2, 30) and (1.5, 32). Interpolate to find f(0.9).

Q8. Repeat Q7 using quadratic functions.

Q9. Repeat Q7 part a using the following function

a2x
f ( x)  a0  a1e

Hint: Use 4 iterations of Newton Raphson iterative method starting at a0 = 1, a1 = 1 & a2 = 0.

 27

Numerical differentiation
Central difference formulas

Central difference formula of order O (h2)

f x  h   f x  h 
f x   f(x)
2h f(x)
f’(x)
Error bound
E = Round-off error + Truncation error
h h
E  erond  etrunc x-h x x+h x
2
Formula of order O (h )

Mh 2
E 
h 6

Optimal step (hbest)

1/ 3
 3 
hbest   
M 

where  is the maximum round-off error and M is the maximum value of the absolute
of the third derivative of f(x).

Central difference formula of order O (h4)

 f  x  2h   8 f  x  h   8 f  x  h   f  x  2h 
f x  
12h

Error bound
f(x)
E = Round-off error + Truncation error f(x)
E  erond  etrunc

3 Mh 4 h h h h
E  x-2h x-h x x+h x+2h
2h 30
4
Formula of order O (h )
Optimal step (hbest)
1/ 5
 45 
hbest   
 4M 

where  is the maximum round-off error and M is the maximum value of the absolute
of the fifth derivative of f(x).
 28

Example
Assume f(x) = cos(x). Approximate the derivative at x = 0.8 using both formulas and
a step h = 0.01. Make error analysis assuming computer round-off error  = 0.5x10-9.
Central difference formula of order O (h2)
f(x+h) = cos(0.8 + 0.01) = 0.6894984329
f(x–h) = cos(0.8 – 0.01) = 0.7038453156
f x  h   f x  h 
f x   = – 0.7173441350
2h

M = maximum value of the absolute of the third derivative = max(|sin(x)|) = 1.0

 Mh 2
E  = 1.6716666666x10-5
h 6

1/ 3
 3 
hbest    = 0.0011447142
M 

Central difference formula of order O (h4)

f(x+2h) = cos(0.8 + 0.02) = 0.6822212072
f(x+h) = cos(0.8 + 0.01) = 0.6894984329
f(x–h) = cos(0.8 – 0.01) = 0.7038453156
f(x–2h) = cos(0.8 – 0.02) = 0.7109135380
 f  x  2h   8 f  x  h   8 f  x  h   f  x  2h 
f x   = – 0.7173560906
12h

M = maximum value of the absolute of the fifth derivative = max(|-sin(x)|) = 1.0

3 Mh 4
E  = 7.5333333333x10-8
2h 30

1/ 5
 45 
hbest    = 0.0223884746
 4M 

Central difference formula of order O (h2) Central difference formula of order O (h4)
 29

Numerical integration
Qudrature formulas
Composite trapezoidal rule
Composite Simpson rule

Qudrature formulas

Rule Truncation error

h 3 2 
x f x dx  2  f 0  f1 
x 1 h
Trapezoidal rule: f max
0 12
h 5 4 
f x dx   f 0  4 f1  f 2 
x2 h
Simpson’s rule: x 0 3 90
f max

3h 5 4 
x f x dx  8  f 0  3 f1  3 f 2  f 3 
x3 3h
Simpson’s 3/8 rule: f max
0 80
8h 7 6 
f x dx  7 f 0  32 f1  12 f 2  32 f 3  7 f 4 
x4 2h
Boole’s rule: x 0 45 945
f max

where k  is
f max the maximum value of the absolute of the kth derivative of f(x).

Composite trapezoidal rule

Interval [a,b] is divided into M subintervals
M 1
f x dx   f a   f b   h f x k 

b h
a 2 k 1

h
1
b  a  f(x)
M
f(x)
Truncation error
b  a 2 2 
E trunc  h f max h
12
2  is the maximum value
where f max
x0 x1 xk xM-1 xM
a b
of the absolute of the second h = (b-a)/M
derivative of f(x). Composite trapezoidal rule

Composite Simpson’s rule

Interval [a,b] is divided to 2M subintervals
M 1 M
f x dx   f a   f b   2h f x 2k   4h
  f x 2k 1 
b h
a 3 3 k 1 3 k 1

h
1
b  a 
2M f(x)
f(x)
Truncation error
b  a 4 4 
E trunc  h f max
180 h
4  is the maximum value
where f max x0 x1 xk x2M-1 x2M
a b
of the absolute of the fourth h = (b-a)/(2M)
derivative of f(x). Composite Simpson’s rule
 30

Example
Consider f(x) = 1 + e-xsin(4x) over the interval [0,1]. Apply the quadrature formulas.

The derivative of the function may be evaluated by MATLAB if f(x) is determined for
the vector x defined by x = 0, 0+T,0+2T, ....., 1. T is the sampling interval. T = 0.01 is
suitable. Then using the diff command in MATLAB
f = 1 + exp(-x).*sin(4*x)
f(1) = 1/T*diff (f)
f(2) = 1/T*diff (f(1))
f(3) = 1/T*diff (f(2))
f(4) = 1/T*diff (f(3))
f(5) = 1/T*diff (f(4))
f(6) = 1/T*diff (f(5))
k  = max(abs(f(k)))
f max

Trapezoidal rule:

h = 1:

0 f x dx  2  f 0  f1   2 1.00  0.721587 = 0.860794

1 h 1

h 3 2 
E trunc  f max = 1/12*13.38 = 1.115
12

Simpson’s rule:

h = 1/2:

0 f x dx  3  f 0  4 f1  f 2   1.0  6.206067  0.721588 = 1.321276

1 h 0.5
3
h 5 4  h5
E trunc  f max = 257 = 0.089
90 90

Simpson’s 3/8 rule:

h = 1/3:

0 f x dx  8  f 0  3 f1  3 f 2  f 3  = 1.314397
1 3h

3h 5 4 
E trunc  f max = 0.0396
80

Boole’s rule:

h = 1/4:

x f x dx  45 7 f 0  32 f1  12 f 2  32 f 3  7 f 4  = 1.308592
x 4 2h
0

8h 7 6  8h 7
E trunc  f max = 4765 = 0.00246
945 945
 31

Example
Consider f(x) = 2 + sin(2 x ) over the interval [1,6]. Apply the composite rules using
11 sample points.

Composite trapezoidal rule

M = 10
h
1
b  a  = (6-1)/10 = 1/2.
M
x0 = 1 = a, x1 = 1.5, x2 = 2, ........ , x10 = 6 = b
M 1
f x dx   f a   f b   h f x k   1 / 2  f 1  f 6  1  f x1   f x 2   ...  f x9 

b h
a 2 k 1 2 2
= 8.1938545652
df

1
cos 2 x  ; d2 f

1
 
cos 2 x 
1
 
sin 2 x
dx x dx 2
2x 
3/ 2 x

2  = max  d f 
2
f max  = 0.7012487490
 dx 2 
 
b  a 2 2 
E trunc  h f max = 0.0730467447
12

Composite Simpson’s rule

2M = 10, M = 5
h
1
b  a  = (6-1)/10 = 1/2.
2M
x0 = 1 = a, x1 = 1.5, x2 = 2, ........ , x10 = 6 = b
2h M 1 M

a f x dx  3  f a   f b   f x 2 k    f x 2k 1 
b h 4h
3 k 1 3 k 1


1
 f a   f b   1  f x 2   f x 4   f x 6   f x8   2  f x1   f x3   f x5   f x 7   f x9 
6 3 3

= 8.1830154940
The derivative of the function may be evaluated by MATLAB if f(x) is determined for
the vector x defined by x = 1, 1+T,1+2T, ....., 6. T is the sampling interval. T = 0.01 is
suitable. Then using the diff command in MATLAB
T = 0.01
x = 1:T:6
f = 2 + sin(2*sqrt(x))
f(1) = 1/T*diff (f)
f(2) = 1/T*diff (f(1))
f(3) = 1/T*diff (f(2))
f(4) = 1/T*diff (f(4))
4  = max(abs(f(4))) = 2.8819225672
f max

b  a 4 4 
E trunc  h f max = 0.0050033378
180
 32

EENG205: Assignment #4

Numerical differentiation and integration

Q1. Use the central difference formula of order O(h2) to approximate f (2.3) for f  e x with
h = 0.1, 0.01 and 0.001. Compute the bounds on the truncation error for each case.

Q2. Repeat Q1 using the central difference formula of order O(h4).

Q3. Determine the best step size (hbest) using the central difference formula of order O(h2) to
approximate f (3) for f  ln( x) assuming a round-off error bound (  ) of 0.5x10-6.

Q4. Repeat Q3 using the central difference formula of order O(h4).

Q5. Consider integration of f(x) over the interval [0, 1]. Apply the various quadrature formulas to
approximate the integral of the following functions.
a- f ( x)  sin ( π x)
b- f ( x)  1  e  x cos (4 x)
c- f ( x)  sin ( x )

Q6. Approximate the following integrals using composite trapezoidal and Simpson’s rules using
M=10 (11 sample points).
1
2 1
a-  (1  x ) dx
1
4
1
b- x
dx
0.25
4
2 x
c- x e dx
0

Q7. Determine the integer M and interval h using the composite trapezoidal rule for the following
integrals such that the truncation error is bounded by 5x10 -9.
 /6 3 2
1 x
a-  cos( x)dx b-  (5  x) dx c-  xe dx
 / 6 2 0

Q8. Repeat Q7 using composite Simpson rule.

 33

Solution of differential equations

Initial value problems
- Euler’s method
- Heun’s method
- Taylor series method
- Runge-Kutta methods
- Predictor-Corrector methods
Boundary value problems

Initial value problems

Euler’s method
Given
t M  t0
 f t , y  yt 0   y 0 and
dy
y  over [t0, tM] with h
dt M

Then Euler’s method is

y k 1  y k  h  f t k , y k 

Heun’s method
Given
t M  t0
 f t , y  yt 0   y 0 and
dy
y  over [t0, tM] with h
dt M

Then Heun’s method is

p k 1  y k  h  f t k , y k 

  f t k , y k   f t k 1 , p k 1 
h
y k 1  y k 
2
Taylor series method
Given
t M  t0
 f t , y  yt 0   y 0 and
dy
y  over [t0, tM] with h
dt M

Then Taylor method of order N is

h 2 f 2  h 3 f 3 h 4 f 4  h N f N 
y k 1  y k  hf 1      
2! 3! 4! N!
dky
where f k  
dt k

Runge-Kutta methods
Given
t M  t0
 f t , y  yt 0   y 0 and
dy
y  over [t0, tM] with h
dt M
Runge-Kutta method of order N = 4 (RK4) is
y k 1  y k 
h
 f1  2 f 2  2 f 3  f 4 
6
where
 h 
f 1  f t k , y k 
h
f 2  f  t k  , y k  f1 
 2 2 
 h 
f 4  f t k  h, y k  hf 3 
h
f3  f tk  , yk  f 2 
 2 2 
 34

Runge-Kutta-Fehlberg (RKF45) is
16h 6656h 28561h 9h 2h
y k 1  y k  f1  f3  f4  f5  f6
135 12825 56430 50 55

where
f 1  f t k , y k 
 h h 
f 2  f  t k  , y k  f1 
 4 4 
 3h 3h 9h 
f3  f tk  , yk  f1  f2 
 8 32 32 
 12h 1932h 7200h 7296h 
f 4  f tk  , yk  f1  f2  f3 
 13 2197 2197 2197 
 439h 3680h 845h 
f 5  f  t k  h, y k  f 1  8hf 2  f3  f4 
 216 513 4104 
 h 8h 3544h 1859h 11h 
f 6  f tk  , yk  f  2hf 2  f3  f4  f5 
 2 27 2565 4104 40 

Optimal step size control of RKF45

Instead of keeping the step size (h) constant, it can be controlled using a specified
error tolerance (tol).

25h 1408h 2197h h

p k 1  y k  f1  f3  f4  f5
216 2565 4101 5
1
 hk   tol  4
hk 1    hk
2y  p 
 k 1 k 1 

Predictor-Corrector methods

Adams-Bashforth-Moulton method

Predictor

p k 1  y k 
h
 9 f k 3  37 f k 2  59 f k 1  55 f k 
24
Corrector

y k 1  y k 
h
 f k 2  5 f k 1  19 f k  9 f k 1 
24
where
f k  f t k , y k 
f k 1  f t k 1 , y k 1 
f k 2  f t k 2 , y k 2 
f k 3  f t k 3 , y k 3 

and
f k 1  f t k 1 , p k 1 
 35

Example
Given
ty
 f t , y   over [0, 3] with yt 0   y 0  1.0
dy
y 
dt 2
Use the initial value problem (IVP) methods to solve the problem using a step size of
0.25.

t M  t0
t0 = 0, tM = 3.0, h   0..25 , y0 = 1.0
M
tM  t0
M  12
h
t  t 0 t1 t 2 t 3 t 4 t 5 t 6 t 7 t 8 t 9 t10 t11 t12 

t  0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 2.25 2.50 2.75 3.00

Euler’s method
tk  yk
y k 1  y k  h  f t k , y k   y k  0.25   y k  0.125  t k  y k 
2
Heun’s method
tk  yk
p k 1  y k  h  f t k , y k   y k  0.25   y k  0.125  t k  y k 
2
  f t k , y k   f t k 1 , p k 1   y k  0.125  0.5t k  y k   0.5t k 1  p k 1 
h
y k 1  y k 
2
Taylor series method
Using Taylor series of order N = 4
h 2 f 2  h 3 f 3 h 4 f 4 
y k 1  y k  hf 1   
2! 3! 4!

f 1 
dy t y
 f 
dt 2
d2y
f  2   2  0 . 5  0 .5
dy
dt dt
d3y d2y
f 3  3  0.5 2
dt dt
d4y d3y
f  4   4  0 . 5 3
dt dt

Runge-Kutta method (RK4)

y k 1  y k 
h
 f1  2 f 2  2 f 3  f 4 
6
where
 h 
f 1  f t k , y k 
h
f 2  f  t k  , y k  f1 
 2 2 
 h 
f 4  f t k  h, y k  hf 3 
h
f3  f tk  , yk  f 2 
 2 2 
 36

t y Euler Heun Taylor RK4

0.00 y0 1.000000000000000 1.000000000000000 1.000000000000000 1.000000000000000
0.25 y1 0.875000000000000 0.898437500000000 0.897491455078125 0.897491455078125
0.50 y2 0.796875000000000 0.838073730468750 0.836403668237229 0.836403668237229
0.75 y3 0.759765625000000 0.814080715179443 0.811869582423750 0.811869582423750
1.00 y4 0.758544921875000 0.822196256369352 0.819594033650794 0.819594033650794
1.25 y5 0.788726806640625 0.858657632576069 0.855786551933871 0.855786551933871
1.50 y6 0.846385955810547 0.920143066258561 0.917102058308097 0.917102058308097
1.75 y7 0.928087711334229 1.003720050681386 1.000588530114770 1.000588530114770
2.00 y8 1.030826747417450 1.106799732242161 1.103640815765855 1.103640815765855
2.25 y9 1.151973403990269 1.227096638620033 1.223959876405184 1.223959876405184
2.50 y10 1.289226728491485 1.362593126281747 1.359516816790557 1.359516816790557
2.75 y11 1.440573387430050 1.511507994295605 1.508521142649650 1.508521142649650
3.00 y12 1.604251714001293 1.672268776214089 1.669392747887015 1.669392747887015

Heun RK4
Taylor
Euler

Solution of differential equations

 37

Adams-Bashforth-Moulton Predictor-Corrector method

Predictor
p k 1  y k 
h
 9 f k 3  37 f k 2  59 f k 1  55 f k 
24
Corrector
y k 1  y k 
h
 f k 2  5 f k 1  19 f k  9 f k 1 
24
where
f k  f t k , y k 
f k 1  f t k 1 , y k 1 
f k 2  f t k 2 , y k 2 
f k 3  f t k 3 , y k 3 
f k 1  f t k 1 , p k 1 

Note that we need four previous values of the function (f(k-3), f(k-2), f(k-1) and f(k))
to start the predictor and the corrector.
e.g., we need f0, f1, f2 and f3 to start evaluating p4 and y4. This means that y0, y1, y2 and
y3 must be available to start the method. Another method must be initially used to
determine these four values.
If we use RK4 method, then we have
y0 = 1.000000000000000
y1 = 0.897491455078125
y2 = 0.836403668237229
y3 = 0.811869582423750
t0  y0 0 1
f 0  f t 0 , y 0    = – 0.500000
2 2
t  y1
f 1  f t1 , y1   1 = – 0.323745727539063
2
t  y2
f 2  f t 2 , y 2   2 = – 0.168201834118615
2
t  y3
f 3  f t 3 , y 3   3 = – 0.030934791211875
2
p4  y3 
h
 9 f 0  37 f1  59 f 2  55 f 3  = 0.819618569688331
24
t  p4
f 4  f t 4 , p 4   4 = 0.090190715155834
2
y 4  y3 
h
 f1  5 f 2  19 f 3  9 f 4  = 0.819590612074072
24
Continue to obtain the following results

t y y t y y t y y
0.00 y0 1.000000000000000 1.25 y5 0.855783825051667 2.50 y10 1.359515320346189
0.25 y1 0.897491455078125 1.50 y6 0.917099586993677 2.75 y11 1.508519822032162
0.50 y2 0.836403668237229 1.75 y7 1.000586345051961 3.00 y12 1.669391582426505
0.75 y3 0.811869582423750 2.00 y8 1.103638895690600
1.00 y4 0.819590612074072 2.25 y9 1.223958180300587
 38

Boundary value problems

A boundary value problem has the following differential form and boundary
conditions
x  f (t , x, x) for a  t  b
x0  x(a) and xf  x(b)
Furthermore, a linear boundary value problem has the following differential form
x(t )  p(t ) x(t )  q(t ) x(t )  r (t )
Boundary value problems can be solved numerically. The finite difference method is
commonly used to approximate the solutions of such problems.

Finite difference method

Consider the following linear boundary value problem

x(t )  p(t ) x(t )  q(t ) x(t )  r (t ) for at b
having the initial and final boundary values of x0 and xf. x0  x(a) and xf  x(b) .
Partition the time to N subintervals of h, so that t = t0, t1, t2, t3, ……., tN. with t 0  a
and t N  b .
N  ( a  b) / h
Then the values xj for j = 1, 2, 3, ……., N-1, are determined as follows
 h
 2



2 h
2
 

2
  p j  1 x j 1  2  h q j x j   p j  1 x j 1  h rj

This can be arranged in a matrix form of AX = B that can be solve for X.

 2 h 
 2  h q1 2 p1  1 0 0 0 ............ 0 
 
 h p  1 2  h 2 q h
p2  1 0 0 ............ 0 
 2 2 2
2 
 h h 
 0  p3  1 2  h 2 q3 p3  1 0 .............. 0 
 2 2 
 .................................................................................... 
A 
 .................................................................................... 
 h h 
 0  p N 3  1 2  h 2q N 3 p N 3  1 0 
 2 2 
 h h 
 0 0  pN 2  1 2  h2qN 2 pN 2 1 
 2 2 
 h 
 0 0 0  p N 1  1 2  h 2 q N 1 
2
 39

 h 2 r  e 
1 0
   x1 
 h 2 r2  x 
   2 
 h 2 r3   x3 
   
B   .......  X   ....... 
 .......   ....... 
   
 h 2 rN  2  xN 2 
  x 
 h 2 rN 1  e N   N 1 
 
where
h   h 
e0   p1  1  x0 and e N    p N 1  1  xf
2   2 

Example
Use the finite difference method to numerically compute x(t) using step sizes (h) of
1.0, 0.5 and 0.2 for the following boundary value problem.
2t 2
x (t )  x (t )  x(t )  1 x(0)  1.25 and x(4)  0.95
2
1 t 1 t2

Solution

a- h = 1.0
2t 2
p (t )  q (t )  r (t )  1.0
1 t2 1 t2
t0 = 0.0 tf = 4.0
x0 = 1.25 xf = -0.95
N = (tf – t0)/h = 4

t p q r B X
0.0000 1.2500
1.0000 1.0000 -1.0000 1 0.8750 0.5214
2.0000 0.8000 -0.4000 1 -1.0000 -0.7071
3.0000 0.6000 -0.2000 1 -1.6650 -1.4357
4.0000 -0.9500

A=
1.0000 -0.5000 0
-1.4000 1.6000 -0.6000
0 -1.3000 1.8000

b- h = 0.5
2t 2
p (t )  q (t )  r (t )  1.0
1 t2 1 t2
t0 = 0.0 tf = 4.0
x0 = 1.25 xf = -0.95
N = (tf – t0)/h = 8
 40

t p q r B X
0.0000 1.2500
0.5000 0.8000 -1.6000 1 1.2500 1.2601
1.0000 1.0000 -1.0000 1 -0.2500 0.9576
1.5000 0.9231 -0.6154 1 -0.2500 0.4677
2.0000 0.8000 -0.4000 1 -0.2500 -0.0848
2.5000 0.6897 -0.2759 1 -0.2500 -0.5903
3.0000 0.6000 -0.2000 1 -0.2500 -0.9553
3.5000 0.5283 -0.1509 1 -1.0745 -1.0987
4.0000 -0.9500

A=
1.6000 -0.8000 0 0 0 0 0
-1.2500 1.7500 -0.7500 0 0 0 0
0 -1.2308 1.8462 -0.7692 0 0 0
0 0 -1.2000 1.9000 -0.8000 0 0
0 0 0 -1.1724 1.9310 -0.8276 0
0 0 0 0 -1.1500 1.9500 -0.8500
0 0 0 0 0 -1.1321 1.9623

c- h = 0.2
2t 2
p (t )  q (t )  r (t )  1.0
1 t2 1 t2
t0 = 0.0 tf = 4.0
x0 = 1.25 xf = -0.95
N = (tf – t0)/h = 20

t p q r B X
0.0000 1.2500
0.2000 0.3846 -1.9231 1 1.2581 1.3145
0.4000 0.6897 -1.7241 1 -0.0400 1.3206
0.6000 0.8824 -1.4706 1 -0.0400 1.2728
0.8000 0.9756 -1.2195 1 -0.0400 1.1774
1.0000 1.0000 -1.0000 1 -0.0400 1.0421
1.2000 0.9836 -0.8197 1 -0.0400 0.8749
1.4000 0.9459 -0.6757 1 -0.0400 0.6837
1.6000 0.8989 -0.5618 1 -0.0400 0.4764
1.8000 0.8491 -0.4717 1 -0.0400 0.2603
2.0000 0.8000 -0.4000 1 -0.0400 0.0424
2.2000 0.7534 -0.3425 1 -0.0400 -0.1706
2.4000 0.7101 -0.2959 1 -0.0400 -0.3726
2.6000 0.6701 -0.2577 1 -0.0400 -0.5576
2.8000 0.6335 -0.2262 1 -0.0400 -0.7201
3.0000 0.6000 -0.2000 1 -0.0400 -0.8550
3.2000 0.5694 -0.1779 1 -0.0400 -0.9572
3.4000 0.5414 -0.1592 1 -0.0400 -1.0222
3.6000 0.5158 -0.1433 1 -0.0400 -1.0455
3.8000 0.4922 -0.1295 1 -0.9432 -1.0227
4.0000 -0.9500
 41

A(1,1:2) = 1.9231 -0.9615

A(2,1:3) = -1.0690 1.9310 -0.9310
A(3,2:4) = -1.0882 1.9412 -0.9118
A(4,3:5) = -1.0976 1.9512 -0.9024
A(5,4:6) = -1.1000 1.9600 -0.9000
A(6,5:7) = -1.0984 1.9672 -0.9016
A(7,6:8) = -1.0946 1.9730 -0.9054
A(8,7:9) = -1.0899 1.9775 -0.9101
A(9,8:10) = -1.0849 1.9811 -0.9151
A(10,9:11) = -1.0800 1.9840 -0.9200
A(11,10:12) = -1.0753 1.9863 -0.9247
A(12,11:13) = -1.0710 1.9882 -0.9290
A(13,12:14) = -1.0670 1.9897 -0.9330
A(14,13:15) = -1.0633 1.9910 -0.9367
A(15,14:16) = -1.0600 1.9920 -0.9400
A(16,15:17) = -1.0569 1.9929 -0.9431
A(17,16:18) = -1.0541 1.9936 -0.9459
A(18,17:19) = -1.0516 1.9943 -0.9484
A(19,18:19) = -1.0492 1.9948

h = 0.2
h = 0.5
h = 1.0

Fig. 1 x(t) for different step intervals (h) using finite difference method
 42

EENG205: Assignment 5

Solution of differential equations

Q1. Use Euler’s method to approximate the solution of the following initial value problems for
time t = [0, 0.4] using time step intervals (h) of 0.2 and 0.1. Determine the final global error
for both time step intervals using the exact solution. Plot the approximated and the exact
solutions on the same graph to examine the accuracy of the solutions.

a- y  t 2  y with y (0)  1.0 and yexact  e t  t 2  2t  2

b- y   2ty 2 with y (0)  1.0 and yexact  1 /(1  t 2 )

Q2. Repeat Q1 using Heun’s method.

Q3. Repeat Q1 using Taylor series method of order N = 4.

Q4. Repeat Q1 using Runge-Kutta method of order N = 4 (RK4).

Q5. Repeat Q1 using Runge-Kutta-Fehlberg method (RKF45).

Q6. Repeat Q1 using Adms-Bashforth-Moulton predictor-corrector method.

Q7. Use the finite difference method to approximate the solution of the following linear boundary
value problems. Plot the approximated solutions and the exact solution on the same graph to
compare the accuracy of the solutions.

a- x  2 x  x  t 2  1 for t  [0,1] with x(0)  5.0 and x(1)  10.0

xexact  t 2  4t  5
Use time step intervals of 0.5, 0.25 and 0.2.

b- x  (1 / t ) x  (1  1 /( 4t 2 )) x  0 for t  [1, 6] with x(1)  1.0 and x(6)  0.0

0.291384 cos(t )+1.001299 sin (t )
xexact 
t
Use time step intervals of 2.5, 1.25 and 1.0.