Lecture 14

Review

Kang ZHOU

1
 Numerical
differentiation
Linear algebraic and integration
equations 0.5

0.45

0.4

0.35

0.3

0.25

UV Signal (a.u.)
0.2

0.15

0.1

0.05

4.5 5 5.5

Time (min)

What you have

learnt in
'Numerical
Nonlinear algebraic
analysis’: equations Differential equations
2
 𝐴𝐴𝐴𝐴 = 𝑏𝑏 Linear algebraic equations

𝐴𝐴 𝑏𝑏 𝑝𝑝 is the only
solution to
𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 𝐴𝐴𝐴𝐴 = 𝑏𝑏
At least one No
Is there any No Is 𝑟𝑟 < 𝑛𝑛 true?
All the solutions can be
inconsistency? solution to 𝐹𝐹
Yes described as 𝑝𝑝 + 𝑤𝑤
Yes 𝐴𝐴𝐴𝐴 = 𝑏𝑏 (𝑝𝑝) −𝐼𝐼

No solution

Find the best approximated solutions

𝐴𝐴′ 𝐴𝐴𝐴𝐴 = 𝐴𝐴𝐴𝑏𝑏

𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅: 𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟 𝑟𝑟𝑟𝑟𝑟𝑟 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓 3

 How to find all the vectors in 𝑁𝑁(𝐴𝐴) ?
Formula Examples Conditions
1 0 0 2 5
𝐼𝐼 𝐹𝐹 𝑟𝑟 = 3 0 1 0 3 6
𝑟𝑟 < 𝑚𝑚, 𝑟𝑟 < 𝑛𝑛
0 0 1 4 7
0 0 𝑚𝑚 = 4, 𝑛𝑛 = 5 0 0 0 0 0

1 0 0 2 5
row 𝐼𝐼 𝐹𝐹 𝑟𝑟 = 3 0 1 0 3 6 𝑟𝑟 = 𝑚𝑚 < 𝑛𝑛
𝑚𝑚 = 3, 𝑛𝑛 = 5 0 0 1 4 7
elimination*
𝐴𝐴 𝑅𝑅
𝑚𝑚 × 𝑛𝑛 1 0 0
𝐼𝐼 𝑟𝑟 = 3 0 1 0
𝑟𝑟 = 𝑛𝑛 < 𝑚𝑚
0 0 1
[m, n] = size(A);
0 𝑚𝑚 = 4, 𝑛𝑛 = 3 0 0 0
r = rank(A);
r is the same to # of 1 0 0
independent columns of A
𝐼𝐼 𝑟𝑟 = 3 0 1 0 𝑟𝑟 = 𝑛𝑛 = 𝑚𝑚
𝑚𝑚 = 3, 𝑛𝑛 = 3 0 0 1 4
* exchange of columns may be needed
 All the rules of rref have been specified here
function A = re4(A) function A = rearrange(A,pi,pj)
pi = 1; [m,n] = size(A); threshold = 1e-10; upper = A(1:pi-1,:);
for pj=1:n [m,n] = size(A);
pivot = A(pi,pj); lower = A(pi:m,:);
if abs(pivot) < threshold column = A(pi:m,pj);
A = rearrange(A,pi,pj); column_abs = abs(column);
end [value,index] = max(column_abs);
pivot = A(pi,pj); memory = lower(1,:);
if abs(pivot) < threshold lower(1,:) = lower(index,:);
continue lower(index,:) = memory;
end A = [upper;lower];
the_row = A(pi,:)/pivot; end
A(pi,:) = the_row;
for i = 1:m
if i ~= pi
A(i,:) = A(i,:) - A(i,pj)*the_row
end
end
if pi == m
break
end
pi = pi + 1;
end

end
5
 Iterated algorithms

Iterated algorithms:
Jacobi algorithm
Gauss-Seidel algorithm

Termination condition:
norm(e)/norm(b) < Tol or other conditions

6
 Non-linear algebraic equations
Iterated methods for single equation:
Bisection algorithm
Secant algorithm
Newton-Raphson algorithm

Iterated method for system of equations:

Newton-Raphson algorithm

7
 The three algorithms

5 9 9

2
f(x) = 0.3*x + sqrt(x) - 5
8 8
4
g(x) = 0

7 7
3

6 6

5 5

1
f(x)

f(x)

f(x)
4 4

f(x)
0

3 3

-1

2 2

-2
1 1

-3
0 0

-4 -1 -1

1 1.5 2 2.5 3 3.5 4 4.5 5 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3

x x x

8
 Numerical Differentiation and integration
Both are needed to solve differential equations
They are also useful in engineering applications, such
as calculating reaction rate and peak area.

Numerical
Differentiations

Comment on other formulas that are not in this table 9

 Numerical integration
We have only discussed one idea - fit data points by
using polynomial function and use its integral for
approximation.

Linear function The trapezoidal rule

𝑦𝑦1 + 4𝑦𝑦2 + 𝑦𝑦3
Quadratic function 1/3 Simpson's rule 3
ℎ
𝑦𝑦1 + 3𝑦𝑦2 + 3𝑦𝑦3 + 𝑦𝑦4
Cubic function 3/8 Simpson's rule 4
ℎ

Generically applicable! These formulas are only applicable to evenly

spaced points!!!

Multiple integral Comment on importance of linear regression

10
 ODEs: Initial value problems
Numerical methods:
𝑂𝑂(ℎ) Euler algorithm - 'box' rule
𝑂𝑂(ℎ2) Runge Kutta 2nd order algorithm - trapezoidal rule
𝑂𝑂(ℎ4) Runge Kutta 4th order algorithm - 1/3 Simpson's rule
They start as Euler algorithm and are refined iteratively.
System of 1st order ODEs
Higher order ODE should be converted into system of 1st
order ODEs

11
 ODEs: Initial value problems
How do I know if a system of ODEs are stiff?

When should I use implicit methods?

You have to solve system of equations simultaneously

12
 ODEs: Boundary value problems
Shooting method
Convert BVP into IVP by guessing initial
conditions and check if boundary condition is
met
Finite difference method
Consider value of y at all the grid points to be
unknowns and establish enough algebraic
equations to calculate the values.
We only handle linear ODE(s) in this module
when finite difference method is used.
13
 How should I prepare for the exam?
Use tutorial questions and homework questions

They have similar difficulty as exam questions

Use the examples in lecture notes

You are expected to understand all the codes in

the lecture notes

14
 Comments on exam
Comments in m-file

Use your time wisely. Balance checking your results and

solving all the problems

You should know how to use functions

Do NOT violate the restrictions. Read them

carefully.

15
 General methodologies of numerical
methods
Approximation (get things started)
Euler method, Vector projection
Represent scattered data by using simple functions
Represent a function by using scattered data
Improve approximation accuracy (get the job done)
Use segments (developing generic model could be difficult)
Reduce step size
Use refinement (Runge-Kutta methods)
16