Numerical Methods

Introduction to Numerical Methods and Taylor Series

1
 Lecture 1
Advanced Numerical Methods

 What are NUMERICAL METHODS?

 Why do we need them?

2
Numerical Methods
Numerical Methods:
Algorithms that are used to obtain numerical
solutions of a mathematical problem.
Why do we need them?
1. No analytical solution exists,
2. An analytical solution is difficult to obtain
or not practical.

3
What do we need?
Basic Needs in the Numerical Methods:
 Practical:
Can be computed in a reasonable amount of time.
 Accurate:
 Good approximate to the true value,
 Information about the approximation error
(Bounds, error order,… ).

4
Outlines of the Course
 Taylor Theorem  Solution for linear
 Number System of Equations
Representation  Least Squares curve
 Solution of nonlinear fitting
Equations  Solution of ordinary
 Interpolation differential equations
 Numerical  Solution of Partial
Differentiation differential equations
 Numerical Integration

5
Solution of Nonlinear Equations
 Some simple equations can be solved analytically:
x2  4x  3  0
 4  4 2  4 (1 )( 3 )
Analytic solution roots 
2 (1 )
x   1 and x  3

 Many other equations have no analytical solution:

x 9  2 x 2  5  0 
x  No analytic solution
x  e 

6
Methods for Solving Nonlinear Equations

o Bisection Method

o Newton-Raphson Method

o Secant Method

7
Solution of Systems of Linear Equations

x1  x2  3
x1  2 x2  5
We can solve it as :
x1  3  x2 , 3  x2  2 x2  5
 x2  2, x1  3  2  1
What to do if we have
1000 equations in 1000 unknowns.
8
Cramer’s Rule is Not Practical
Cramer' s Rule can be used to solve the system :
3 1 1 3
5 2 1 5
x1   1, x2  2
1 1 1 1
1 2 1 2

But Cramer' s Rule is not practical for large problems.

To solve N equations with N unknowns, we need (N  1)(N  1)N!
multiplications.
To solve a 30 by 30 system, 2.3  1035 multiplications are needed.
A super computer needs more than 10 20 years to compute this.
9
Methods for Solving Systems of Linear
Equations
o Naive Gaussian Elimination

o Gaussian Elimination with Scaled

Partial Pivoting

o Algorithm for Tri-diagonal

Equations

10
Curve Fitting
 Given a set of data:

x 0 1 2
y 0.5 10.3 21.3

 Select a curve that best fits the data. One

choice is to find the curve so that the sum
of the square of the error is minimized.

11
Interpolation
 Given a set of data:

xi 0 1 2
yi 0.5 10.3 15.3

 Find a polynomial P(x) whose graph

passes through all tabulated points.

yi  P ( xi ) if xi is in the table
12
Methods for Curve Fitting
o Least Squares
o Linear Regression
o Nonlinear Least Squares Problems

o Interpolation
o Newton Polynomial Interpolation
o Lagrange Interpolation

13
Integration
 Some functions can be integrated
analytically:

3 3
1 2 9 1
1 xdx  2 x 1  2  2  4
But many functions have no analytical solutions :
a
 x2
e
0
dx  ?

14
Methods for Numerical Integration
o Upper and Lower Sums

o Trapezoid Method

o Romberg Method

o Gauss Quadrature

15
Solution of Ordinary Differential Equations
A solution to the differential equation :
x(t )  3 x (t )  3x(t )  0
x (0)  1; x(0)  0
is a function x(t) that satisfies the equations.

* Analytical solutions are available for

special cases only.
16
Solution of Partial Differential Equations
Partial Differential Equations are more
difficult to solve than ordinary differential
equations:

2 2
 u  u
2
 2
20
x t
u (0, t )  u (1, t )  0, u ( x,0)  sin(x)

17
Summary
Topics Covered in the Course
 Numerical Methods:
Algorithms that are
 Solution of Nonlinear Equations
used to obtain  Solution of Linear Equations
numerical solution of a
 Curve Fitting
mathematical problem.
 We need them when  Least Squares
No analytical solution  Interpolation
exists or it is difficult  Numerical Integration
to obtain it.
 Numerical Differentiation
 Solution of Ordinary Differential
Equations
 Solution of Partial Differential
Equations

18
Number Representation and Accuracy
 Number Representation
 Normalized Floating Point Representation
 Significant Digits
 Accuracy and Precision
 Rounding and Chopping

19
Representing Real Numbers
 You are familiar with the decimal system:

312.45  3 10 2  1101  2 100  4 10 1  5 10 2

 Decimal System: Base = 10 , Digits (0,1,…,9)

 Standard Representations:

 3 1 2 . 4 5
sign integral fraction
part part
20
Normalized Floating Point Representation
 Normalized Floating Point Representation:

 d . f1 f 2 f 3 f 4  10 n
sign mantissa exponent

d  0,  n : signed exponent

 Scientific Notation: Exactly one non-zero digit appears

before decimal point.
 Advantage: Efficient in representing very small or very
large numbers.

21
Binary System

 Binary System: Base = 2, Digits {0,1}

 1. f1 f 2 f 3 f 4  2  n
sign mantissa signed exponent

(1.101) 2  (1  1  2 1  0  2  2  1  2 3 )10  (1.625)10

22
Fact
 Numbers that have a finite expansion in one numbering
system may have an infinite expansion in another
numbering system:

(1 .1)10  (1 .0001100110 01100 ...) 2

 You can never represent 1.1 exactly in binary system.

23
IEEE 754 Floating-Point Standard
 Single Precision (32-bit representation)
 1-bit Sign + 8-bit Exponent + 23-bit Fraction

S Exponent8 Fraction23

 Double Precision (64-bit representation)

 1-bit Sign + 11-bit Exponent + 52-bit Fraction

S Exponent11 Fraction52
(continued)

24
Significant Digits

 Significant digits are those digits that can be

used with confidence.

 Single-Precision: 7 Significant Digits

1.175494… × 10-38 to 3.402823… × 1038

 Double-Precision: 15 Significant Digits

2.2250738… × 10-308 to 1.7976931… × 10308

25
Remarks

 Numbers that can be exactly represented are called

machine numbers.

 Difference between machine numbers is not uniform

 Sum of machine numbers is not necessarily a machine

number

26
Calculator Example
 Suppose you want to compute:
3.578 * 2.139
using a calculator with two-digit fractions

3.57 * 2.13 = 7.60

True answer: 7.653342

27
Significant Digits - Example
48.9

28
Accuracy and Precision

 Accuracy is related to the closeness to the true

value.

 Precision is related to the closeness to other

estimated values.

29
30
Rounding and Chopping

 Rounding: Replace the number by the nearest

machine number.

 Chopping: Throw all extra digits.

31
Error Definitions – True Error
Can be computed if the true value is known:

Absolute True Error

Et  true value  approximation
Absolute Percent Relative Error
true value  approximation
t  *100
true value

32
Error Definitions – Estimated Error
When the true value is not known:

Estimated Absolute Error

Ea  current estimate  previous estimate
Estimated Absolute Percent Relative Error
current estimate  previous estimate
a  *100
current estimate

33
Notation
We say that the estimate is correct to n
decimal digits if:
n
Error  10

We say that the estimate is correct to n

decimal digits rounded if:
1 n
Error   10
2

34
Summary
 Number Representation
Numbers that have a finite expansion in one numbering system
may have an infinite expansion in another numbering system.

 Normalized Floating Point Representation

 Efficient in representing very small or very large numbers,
 Difference between machine numbers is not uniform,
 Representation error depends on the number of bits used in
the mantissa.

35
 Taylor Theorem

 Motivation
 Taylor Theorem
 Examples

36
Motivation
 We can easily compute expressions like:
3 10 2
2( x  4)
But, How do you compute 4.1, sin(0.6) ?

Can we use the definition

b
a
to compute sin(0.6)?
0.6
Is this a practical way?

37
Remark

 In this course, all angles are assumed to

be in radian unless you are told otherwise.

38
Taylor Series
The Taylor series expansion of f ( x ) about a :
(2) ( 3)
' f ( a ) 2 f (a )
f (a )  f (a ) ( x  a )  ( x  a)  ( x  a ) 3  ...
2! 3!
or

1 (k )
Taylor Series   f (a ) ( x  a )k
k 0 k!
If the series converge, we can write :
∞
1 (k )
f ( x)  ∑k! f (a ) ( x  a ) k
k 0

39
Maclaurin Series
 Maclaurin series is a special case of Taylor
series with the center of expansion a = 0.
The Maclaurin series expansion of f ( x ) :
( 2) ( 3)
' f ( 0 ) 2 f ( 0) 3
f ( 0)  f ( 0 ) x  x  x  ...
2! 3!
If the series converge, we can write :
∞
1 (k )
f ( x )  ∑ f ( 0) x k
k 0
k!
40
Maclaurin Series – Example 1
Obtain Maclaurin series expansion of f ( x )  e x
f ( x)  e x f ( 0)  1
f ' ( x)  e x f ' ( 0)  1
f ( 2) ( x )  e x f ( 2 ) ( 0)  1
f (k ) ( x)  e x f ( k ) (0)  1 for k  1
∞ ∞ k 2 3
1 x x x
e x  ∑ f ( k ) ( 0) x k  ∑  1 x    ...
k 0
k! k 0
k! 2! 3!
The series converges for x  ∞.
41
Taylor Series
3

Example 1
2.5
exp(x)
1+x+0.5x 2
2

1+x

1.5

1
1

0.5

0
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

42
Maclaurin Series – Example 2
Obtain Maclauri n series expansion of f ( x )  sin( x ) :
f ( x )  sin( x ) f ( 0)  0
f ' ( x )  cos( x ) f ' ( 0)  1
f ( 2 ) ( x )   sin( x ) f ( 2 ) ( 0)  0
f ( 3) ( x )   cos( x ) f ( 3) (0)  1
∞
f ( k ) ( 0) k x3 x5 x7
sin( x )  ∑ x  x     ....
k 0
k! 3! 5! 7!
The series converges for x  ∞.
43
4

3
x

1 x-x 3/3!+x 5/5!

0 sin(x)

-1
x-x 3/3!

-2

-3

-4
-4 -3 -2 -1 0 1 2 3 4

44
Maclaurin Series – Example 3
Obtain Maclaurin series expansion of : f ( x)  cos( x)
f ( x )  cos( x ) f ( 0)  1
f ' ( x )   sin( x ) f ' ( 0)  0
f ( 2 ) ( x )   cos( x ) f ( 2 ) ( 0 )  1
f ( 3) ( x )  sin( x ) f ( 3) (0)  0
∞
f ( k ) ( 0) x 2
x 4
x 6
cos( x )  ∑ ( x ) k  1     ....
k 0
k! 2! 4! 6!
The series converges for x  ∞.
45
Maclaurin Series – Example 4
1
Obtain Maclaurin series expansion of f(x) 
1 x
1
f ( x)  f ( 0)  1
1 x
1
f ' ( x)  f ' ( 0)  1
1  x 2
2
f (2) ( x )  f ( 2 ) ( 0)  2
1  x 3
6
f ( 3) ( x )  f ( 3) (0)  6
1  x 4
1
Maclaurin Series Expansion of :  1  x  x 2  x 3  ...
1 x
Series converges for | x |  1
46
Example 4 - Remarks

 Can we apply the series for x≥1??

 How many terms are needed to get a good

approximation???

These questions will be answered using

Taylor’s Theorem.

47
Taylor Series – Example 5
1
Obtain Taylor series expansion of f(x)  at a  1
x
1
f ( x)  f (1)  1
x
1
f ' ( x)  2 f ' (1)  1
x
2
f (2) ( x)  3 f ( 2 ) (1)  2
x
6
f ( 3) ( x )  4 f ( 3) (1)  6
x
Taylor Series Expansion ( a  1) : 1  ( x  1)  ( x  1) 2  ( x  1) 3  ...
48
Taylor Series – Example 6
Obtain Taylor series expansion of f(x)  ln( x ) at ( a  1)

1 ( 2) 1 ( 3) 2
f ( x )  ln( x ) , f ' ( x )  , f ( x )  2 , f ( x )  3
x x x
f (1)  0, f ' (1)  1, f ( 2) (1)  1 f ( 3) (1)  2

1 1
Taylor Series Expansion : ( x  1)  ( x  1) 2  ( x  1) 3  ...
2 3

49
Convergence of Taylor Series
 The Taylor series converges fast (few terms
are needed) when x is near the point of
expansion. If |x-a| is large then more terms
are needed to get a good approximation.

x-x 3/3!+x 5/5!

sin(x)

x-x 3/3!

few terms more terms 50