Review of Numerical Methods: Dr. Radhakant Padhi

Lecture – 16
Review of Numerical Methods
Dr. Radhakant Padhi

Asst. Professor
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
Linear Equations: Solution Technique
AX = b A is nonsingular, b ≠ 0
Problem: X =?
Motivation: X = A X + B U
X C : controlled state
⎡ Xc ⎤
where X =⎢ ⎥ X N : uncontrolled state
⎣XN ⎦
dim ( X C ) = dim (U ) = m
⎡ X C ⎤ ⎡ A1 ⎤ ⎡ B1 ⎤
⎢ X ⎥ = ⎢ A ⎥ X + ⎢ B ⎥ U
⎣ N ⎦ ⎣ 2⎦ ⎣ 2⎦
ADVANCED CONTROL SYSTEM DESIGN 2
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
Motivation: Continued
0
X c = A1 X + B1U
Which gives
U = − B1−1 ( A1 X )
Note: B1 is square
This will be the controller necessary to maintain X c

at steady state

Solution Technique: Direct Inversion of A
−1
X =A b
−1
z Computation of A Involves too many
computations, roughly n × n ! number of
2
operations (very inefficient for large n).
z This approach also suffers from the problem of

sensitivity (ill-conditioning) ,when A → 0
z Round off errors may lead to large inaccuracies

Solution Technique: Gauss Elimination
o Do row operations to reduce the A matrix to a upper
triangular form
o Solve the variable from down to top
⎡2 1 0⎤ ⎡ X 1 ⎤ ⎡1 ⎤
Example: ⎢1 2 1 ⎥⎥ ⎢ X ⎥ = ⎢2⎥
⎢ ⎢ 2⎥ ⎢ ⎥
⎢⎣ 0 1 1 ⎥⎦ ⎢⎣ X 3 ⎥⎦ ⎢⎣ 4 ⎥⎦
Solution Steps:
Step-I: Multiply row-1 with -1/2 and add to the row-2.
row-3 keep unchanged, since a31=0.
⎡2 1 0⎤ ⎡ x1 ⎤ ⎡ 1 ⎤
⎢0 3 2 1⎥ ⎢ x ⎥ = ⎢3 2⎥
⎢ ⎥ ⎢ 2⎥ ⎢ ⎥
⎢⎣0 1 1⎥⎦ ⎢⎣ x3 ⎥⎦ ⎢⎣ 4 ⎥⎦

Solution Technique: Gauss Elimination
Step-II: Multiply row-2 with -2/3 and add to row-3
⎡2 1 0⎤ ⎡ x1 ⎤ ⎡ 1 ⎤
⎢0 3 2 1 ⎥ ⎢ x ⎥ = ⎢3 2⎥
⎢ ⎥ ⎢ 2⎥ ⎢ ⎥
⎢⎣0 0 1 3⎥⎦ ⎢⎣ x3 ⎥⎦ ⎢⎣ 3 ⎥⎦
Upper Triangle Matrix
⎡ x1 ⎤ ⎡ 3 ⎤
Final Solution ⎢ x ⎥ = ⎢−5⎥
⎢ 2⎥ ⎢ ⎥
⎢⎣ x3 ⎥⎦ ⎢⎣ 9 ⎥⎦

Gauss Elimination
o The total number of operations needed is ( 2 / 3) n3
which is far lesser than computing
adj ( A ) (which requires 2
−1
A = n × n ! operations )
A
o The Gauss elimination method will encounter

potential problems when the pivot elements i.e..
diagonal elements become zero, or very close to
zero at any stage of elimination.
o In such cases the order of equations can be changed
by exchanging rows and the procedure can be
continued
Nonlinear Algebraic Equations
Problem: F (X ) = 0 X =?
Motivation: Finding the forced equilibrium condition for a nonlinear
system to get an appropriate operating point for linearization
X = f ( X , U )
⎡ X C ⎤ ⎡ f C ( X ,U ) ⎤ dim ( X C ) = dim ( f C ) = dim (U )
⎢ X ⎥ = ⎢ f ( X ,U ) ⎥
⎣ N⎦ ⎣ N ⎦
0
X C = f C ( X 0 , U 0 )
Solve for U 0 from f C (U ) = 0

Newton-Raphson Method: Scalar Case
f ( x) = 0
Using Taylor series expansion
⎡ df ⎤
f ( xk + Δxk ) ≈ f ( xk ) + ⎢ ⎥ Δxk + " + (higher order terms)
⎣ dx ⎦ xk
0
⎡ df ⎤
⎢⎣ dx ⎥⎦ x Δxk
= − f ( xk)
k
f ( xk )
xk +1 = xk −
f ′( xk )
From the above equation with an initial guess
we can iteratively solve for x with Δx < tolerence
Newton-Raphson Method: Scalar Case
f(x)
f ( xi )
f '( xi ) = A
xi − xi +1
f(x i)
[x f (x )]
i, i
f ( xi )
xi +1 = xi −
f ′( xi )
f(x i+1 )
θ B
C X
x i+2 x i+1 xi

Newton-Raphson Method: Multi Variable Case
F(X ) = 0
Ak

⎡ ∂F ⎤
F ( X k + ΔX k ) ≈ F ( X k ) + ⎢ ⎥ ΔX k
⎣ ∂X ⎦ k
Ak ΔX k = − F ( X k )
Solve for ΔX k = − Ak−1 F ( X k )
Update X k +1 = X k + ΔX k

Newton-Raphson Method: Algorithm
o Start with guess value x1
o Solve for Δxk
o Update x k + 1 = x k + Δ x k ( k = 1, 2,... )
o Continue until convergence
Convergence Condition
1. Relative Error
( )
∈ak xki+1 − xki / xki+1 < tol, ∀k
2. Absolute Error
f ( xk ) < tol
Example: N-R Method
Question : Find a root of the following equation
f ( x ) = x3 − 0.165 x 2 + 3.993x10− 4 = 0
Solution : f ′ ( x ) = 3 x 2 − 0.33 x. Let x0 = 0.02. Then
f ( x0 ) 3.413x10−4
x1 = x0 − = 0.02 − = 0.08320
f ' ( x0 ) −5.4x10 −3
f ( x1 ) −1.670x10−4
x2 = x1 − = 0.08320 − = 0.05824
f ′ ( x1 ) −6.689x10 −3
f ( x2 ) 3.717x10−5
x3 = x2 − = 0.05284 − = 0.06235
f ' ( x2 ) −9.043x10 −3

Newton-Raphson Method: Advantages
z If it converges, it converges fast!
It has “Quadratic convergence” property, i.e.
ek +1 = c ek 2 , where ek ( − xk )
x *
c is a constant
x* is the actual root
z Problem: It requires good initial guess in general

to converge to the right solution.
Newton-Raphson Method: Limitations
z Non-convergence at Inflection points
f ( x ) = ( x − 1) = 0
For a function f(x) the points 3
where the concavity changes
from up-to-down or down-
Inflection point x=1
to-up are called inflection
points.

z Root Jumping
Cases where f (x) is
oscillating and has a
number of roots
Initial Guess near

to one root may
produce another Produced root x=0
root
Example:
f ( x) = sin( x) = 0

z Oscillations around local minima or maxima
Results may oscillate about the local maximum or minimum
without converging on a root but converging on the local
maximum or minimum. Eventually, it may lead to division to
a number close to zero and may diverge.
f(x) has no real roots
f (x) = x2 + 2

z Division by zero
If f '(xi ) ≈ 0 at some xi , xi+1 becomes very large value
f ( x ) = x3 - 0.03x 2 + 2.4x10-6 = 0
f ′( x) = 3x2 − 0.06 x
Even after several

iterations there is no
convergence!

N-R Method Drawbacks
zf'(x*) is unbounded
If the derivative of f(x) is unbounded at the
root then Newton-Raphson method will not
converge.
Exercise: Verify for f(x)=√x

⎛ df ⎞
Numerical Differentiation ⎜ ⎟
⎝ dx ⎠
Technique Definition Numerical Error

Name Approximation
Forward ⎡ f ( x + Δx) − f ( x) ⎤ f ( x0 + Δ x ) − f ( x0 )
difference lim ⎢ ⎥ O ( Δx )
Δx→0
⎣ Δx ⎦ Δx
Backward ⎡ f ( x) − f ( x −Δx) ⎤ f ( x0 ) − f ( x0 − Δx )
difference lim ⎢ ⎥ O ( Δx )
Δx→0
⎣ Δ x ⎦ Δx
f ( x0 + Δx) − f ( x0 −Δx)
O ( Δx2 )
Central ⎡ f ( x + Δx) − f ( x −Δx) ⎤
lim ⎢ ⎥
difference Δx→0
⎣ 2Δx ⎦ 2Δx

Numerical Integration
Trapezoidal Rule:
f3
f2
f1
I1 I2 ----- In-1 In
Note: Numerical Differentiation is “Error Amplifying’’.

where as Numerical Integration is “Error Smoothing’’.

Numerical Integration
o Trapezoidal Rule:
I ≈ I1 + I2 +"+ In-1 + In
1 1
= Δx ( f0 + f1 ) + Δx ( f1 + f2 ) +"
2 2
1 1
+ Δx ( fn−2 + fn−1 ) + Δx ( fn−1 + fn )
2 2
Δx
= [ f0 + 2 f1 + 2 f2 +"+ 2 fn−1 + fn ]
2
⎛ f0 fn ⎞
I ≈ ⎜ + f1 + " + f n −1 + ⎟ Δx
⎝ 2 2 ⎠
Ordinary Differential Equation (ODE)
⎛ d x (t ) d 2 x (t ) d n x (t ) ⎞
f ⎜ x, , ," n ⎟ = 0
⎝ dt dt 2
dt ⎠
z Ordinary: only one independent variable
z Differential Equation: unknown functions
enter into the equation through its derivatives
z Order: highest derivative in f
z Degree: exponent of the highest derivative
4
⎛ d x (t ) ⎞
3
E x a m p le : ⎜ ⎟ − x (t ) = 0
⎝ dt
3
⎠
d e g re e = 4 ; o rd e r = 3
What Is Solution of ODE ??
z A problem involving ODE is not completely
specified by its equation
ODE has to be supplemented with boundary
conditions.
z Initial value problem: x is given at some starting
value ti , and it is desired to find at some final points tf
or at some discrete list of points.
z Two point boundary value problem: Boundary
conditions are specified at more than one t ; typically
some of the conditions will be specified at ti and some
at tf .

Numerical Solution to Initial Value Problem
dx(t )
= f (t , x(t )); x(t0 ) = x0
dt
zA numerical solution to this problem generates
sequence of values for the independent variable
t1,t2,…tn and a corresponding sequence of values
of the dependent variable x1,x2….,xn so that each
xn approximates solution at tn
xn ≈ x (tn) n=0,1,2….n.

Basic Concepts of Numerical Methods to Solve
ODEs
xn +1 − xn
≈ slope of tangent
Δt
We can calculate the
tangent slope at any point.
In fact the differential
equation
dx(t )
= f ( t, x(t )) defines the
dt
tangent slope = f ( t, x(t ))

Euler’s Method
dx x(0) = b
z Solve = f (t , x) with
dt
z At start of time step
x n +1 − x n
≈ f (t n , xn ) Forward difference
Δt
Rearranging
x n +1 = x n + Δ t f n
Start with initial conditions t0=0; x0=b

Euler Integration: Useful Comments
• Euler integration has error of the order of ( Δt )

2
• Small step size Δt may be needed for good

accuracy. This is in conflict with the
computational load advantage.
• Lesser computational load

Runge-Kutta Fourth Order Method
Δt
xi +1 = xi + ( k1 + 2k2 + 2k3 + k4 )
6
z where
k1 = f ( ti , xi )
⎛ 1 1 ⎞
k 2 = f ⎜ ti + Δt , xi + k1Δt ⎟
⎝ 2 2 ⎠
⎛ 1 1 ⎞
k
In each step the derivative is calculated 3 = f t
⎜ i + Δ t , xi + k Δ t ⎟
⎝ ⎠
2
2 2
at four points, once at the initial point,
twice at trial mid points and once at k 4 = f ( ti + Δt , xi + k3 Δt )
trial end point

Runge-Kutta Algorithm
z Error θ ({Δt}5 )
z The method uses a 4th order power series

approximation to come up with this
algorithm. Hence, the algorithm is called
RK-4 method


Review of Numerical Methods: Dr. Radhakant Padhi

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Review of Numerical Methods: Dr. Radhakant Padhi

Uploaded by

Copyright:

Available Formats

Lecture – 16

Review of Numerical Methods

Dr. Radhakant Padhi

This will be the controller necessary to maintain X c

ADVANCED CONTROL SYSTEM DESIGN 3

operations (very inefficient for large n).

z This approach also suffers from the problem of

z Round off errors may lead to large inaccuracies

ADVANCED CONTROL SYSTEM DESIGN 4

ADVANCED CONTROL SYSTEM DESIGN 5

Step-II: Multiply row-2 with -2/3 and add to row-3

ADVANCED CONTROL SYSTEM DESIGN 6

o The Gauss elimination method will encounter

Solve for U 0 from f C (U ) = 0

ADVANCED CONTROL SYSTEM DESIGN 8

ADVANCED CONTROL SYSTEM DESIGN 10

Solve for ΔX k = − Ak−1 F ( X k )

ADVANCED CONTROL SYSTEM DESIGN 11

ADVANCED CONTROL SYSTEM DESIGN 13

z Problem: It requires good initial guess in general

ADVANCED CONTROL SYSTEM DESIGN 15

Initial Guess near

ADVANCED CONTROL SYSTEM DESIGN 16

f(x) has no real roots

ADVANCED CONTROL SYSTEM DESIGN 17

Even after several

ADVANCED CONTROL SYSTEM DESIGN 18

Exercise: Verify for f(x)=√x

ADVANCED CONTROL SYSTEM DESIGN 19

Technique Definition Numerical Error

ADVANCED CONTROL SYSTEM DESIGN 20

Note: Numerical Differentiation is “Error Amplifying’’.

ADVANCED CONTROL SYSTEM DESIGN 21

ADVANCED CONTROL SYSTEM DESIGN 24

ADVANCED CONTROL SYSTEM DESIGN 25

ADVANCED CONTROL SYSTEM DESIGN 26

ADVANCED CONTROL SYSTEM DESIGN 27

• Euler integration has error of the order of ( Δt )

• Small step size Δt may be needed for good

• Lesser computational load

ADVANCED CONTROL SYSTEM DESIGN 28

ADVANCED CONTROL SYSTEM DESIGN 29

z The method uses a 4th order power series

ADVANCED CONTROL SYSTEM DESIGN 30

You might also like