SE301: Numerical Methods

Topic 1:
Int d ti n to Introduction t Numerical N i l methods th d and nd Taylor T l Series S i

Lectures 1-4:

SE301_Topic1

Lecture 1
Introduction to Numerical Methods

Why do we need them?

What are

NUMERICAL METHODS?

Topics covered in SE301.

Reading Assignment: pages 3-10 of text book

SE301_Topic1 2

N Numerical ri l M Methods th d
Numerical Methods:
Algorithms that are used to obtain numerical solutions of a mathematical problem.

Why do we need them? 1 No analytical solution exists, 1. exists

2. An analytical solution is difficult to obtain p or not practical.

SE301_Topic1

Wh t d What do we need d
Basic Needs in the Numerical Methods: Practical:
can be computed p in a reasonable amount of time.

Accurate:

Good approximate to the true value I f Information ti about b t the th approximation i ti error (Bounds, error order, )

SE301_Topic1

O tli Outlines of f the th Course C r

Taylor y Theorem Number Representation Solution of nonlinear Equations Interpolation Numerical Differentiation Numerical Integration

Solution of linear Equations Least Squares curve fitti fitting Solution of ordinary differential equations Solution of Partial differential equations

SE301_Topic1

S l ti n of Solution fN Nonlinear nlin r Eq Equations ti n

Some simple equations can be solved analytically

x2 + 4x + 3 = 0 Analytic solution roots = 4 4 2 4 (1 )( 3 ) 2 (1 )

x = 1 and

x = 3

Many other equations have no analytical solution

x9 2x2 + 5 = 0 N analytic l ti No x x = e
SE301_Topic1

solution l ti

Methods for solving Nonlinear E Equations i

o o o

Bisection Method Newton-Raphson Method Secant Method

SE301_Topic1

Solution of Systems of Li r E Linear Equations ti

x1 + x 2 = 3 x1 + 2 x 2 = 5 We can solve it as x1 = 3 x 2 , 3 x2 + 2 x2 = 5 x 2 = 2, x1 = 3 2 = 1 What to do if we have 1000 equations in 1000 unknowns
SE301_Topic1 8

Cr Cramers r R Rule l i is not t pr practical ti l

Cramer' s Rule can be used to solve the system 3 1 5 2 x1 = = 1, 1 1 1 2 1 3 1 5 x2 = =2 1 1 1 2

But Cramer' s Rule is not practical for large problems. To solve N equations in N unknowns we need (N + 1)(N 1)N! multiplications. To solve a 30 by 30 system, 2.3 1035 multiplications are needed. A super computer needs more than 10 20 years to compute.
SE301_Topic1 9

Methods for solving Systems of Linear E Equations i

o o o

Naive Gaussian Elimination Gaussian Elimination with Scaled Partial pivoting Algorithm for Tri-diagonal Equations

SE301_Topic1

10

C r Fitti Curve Fitting

Given a set of data

x y 0 0.5 1 10.3 2 21.3

Select a curve that best fit the data. One choice is find the curve so that the sum of the square of the error is minimized.
11

SE301_Topic1

I t rp l ti Interpolation

Given a set of data

xi yi 0 0.5 1 10.3 2 15.3

find a polynomial P(x) whose graph passes through all tabulated points. points

yi = P ( xi ) if f xi is in the table
SE301_Topic1 12

M th d f Methods for rC Curve r Fitti Fitting

o

Least Squares
o o

Linear Regression Nonlinear least Squares Problems Newton e to polynomial po y o a interpolation te po at o Lagrange interpolation

Interpolation
o o

SE301_Topic1

13

I t r ti Integration

Some functions can be integrated analytically

1 2 9 1 xdx = 2 x = 2 2 = 4 1 1 But many functions have no analytical solutions

e
0

x2

dx = ?
14

SE301_Topic1

M h d f Methods for N Numerical i lI Integration i

o o o o

Upper and Lower Sums Trapezoid Method Romberg Method Gauss Quadrature

SE301_Topic1

15

Solution of Ordinary Differential Equations

A solution to the differential equation (t ) + 3 x (t ) + 3 x(t ) = 0 x (0) = 1; x(0) = 0 x is a function x(t) ( ) that satisfies the equations q *A Analytical l ti l solutions l ti are available il bl for f special cases only
SE301_Topic1 16

Solution of Partial Differential E ti Equations

Partial Differential Equations are more difficult to solve than ordinary differential equations

u
2 2

x t u (0, t ) = u (1, t ) = 0, u ( x,0) = sin( i (x)

2
SE301_Topic1 17

u
2

+2=0

S Summary r

Numerical Methods: Algorithms that are used to obtain numerical solution of a mathematical problem. We need them when No analytical solution exist or it is difficult to obtain.

Topics Covered in the Course

Solution of nonlinear Equations Solution of linear Equations Curve fitting

Least Squares Interpolation

Numerical N me ical Integration Integ ation Numerical Differentiation Solution of ordinary differential equations Solution of Partial differential q equations
18

SE301_Topic1

Lecture 2

Number Representation and accurcy

Number Representation Normalized Floating Point Representation

Significant Digits Accuracy and Precision Rounding and Chopping

Reading assignment: Chapter 3

SE301_Topic1 19

R pr Representing ti R Real lN Numbers b r

You are familiar with the decimal system y

312.45 = 3 10 2 + 1101 + 2 100 + 4 10 1 + 5 10 2

Decimal System Base =10 , Digits(0,1,9) Standard Representations

3 1 2 . sign integral part

SE301_Topic1

4 5 fraction part
20

Normalized Floating Point R Representation i

Normalized Floating g Point Representation p

0. d1 d 2 d 3 d 4 10 n sign i d1 0,

mantissa i n : integer

exponent

No integral part, Advantage

Efficient in representing very small or very large numbers

SE301_Topic1

21

C l l t rE Calculator Example pl

suppose you want to compute 3.578 * 2.139 using a calculator with two two-digit digit fractions

3 57 3.57

2 13 = 7 2.13 7.60 60 7.653342

22

True answer
SE301_Topic1

Bi Binary System S

Binary System
sign i mantissa ti

Base=2, Digits{0,1}
exponent t

0. 1 b2 b3 b4 2 n

b1 0 b1 = 1
(0.101) 2 = (1 2 + 0 2 + 1 2 )10 = (0.625)10
1 2 3

SE301_Topic1

23

7-Bit Representation
( i 1 bit, (sign: bi M Mantissa i 3bits,exponent 3bi 3 bits) bi )

SE301_Topic1

24

Fact

Number that have finite expansion p in one numbering g system may have an infinite expansion in another numbering system

(0.1)10 = (0.000110011001100...) )2

You can never represent 0.1 exactly in any computer

SE301_Topic1

25

R Representation i
Hypothetical Machine (real computers use 23 bit mantissa) Mantissa a t ssa 2 b bits ts e exponent po e t 2 b bit t sign sg 1b bit t

Possible machine numbers .25 .3125 .375 .4375 .5 .625 .75 .875 1
SE301_Topic1

1.25 1.5

1.75
26

1 08 0.8

R Representation i
Gap near zero

0.6 0.4 0.2 0 -0 0.2 2 -0.4 -0.6 -0.8 -1 0

SE301_Topic1

0.2

0.4

0.6

0.8

1.2

1.4

1.6

1.8

27

R Remarks rk

Numbers that can be exactly represented are called machine numbers Difference between machine numbers is not uniform sum of machine numbers is not necessarily a machine number 0 25 + .3125 0.25 3125 =0.5625 0 5625 (not ( t a machine hi number) b )

SE301_Topic1

28

Si ifi t Di Significant Digits it

Significant g digits g are those digits g that can be used with confidence.

SE301_Topic1

29

48 9 48.9

SE301_Topic1

30

A r and Accuracy d Pr Precision ii

Accuracy is related to closeness to the true value Precision is related to the closeness to other estimated values

SE301_Topic1

31

SE301_Topic1

32

R Rounding di and d Chopping Ch ppi

Rounding: Replace the number by the nearest machine number Ch Chopping: i Th Throw all ll extra di digits i

SE301_Topic1

33

Fi 3.7 Fig 37

SE301_Topic1

34

Error Definitions
T Error True E

can be computed if the true value is known

Absolute True Error Et = true value approximation Absolute Percent Relative Error true value approximation t = *100 true value
SE301_Topic1 35

Error Definitions
Estimated error
When the true value is not known

EstimatedAbsoluteError Ea = current estimate prevoiusestimate EstimatedAbsolutePercent RelativeError current estimate prevoiusestimate a = *100 current estimate
SE301_Topic1 36

Notation
We say y the estimate is correct to n decimal digits if n

Error 10

We say the estimate is correct to n decimal digits rounded if 1

Error

10

SE301_Topic1

37

S Summary r

Number Representation
Number that have finite expansion in one numbering system may have an infinite expansion in another numbering system.

Normalized Floating Point Representation

Efficient in representing very small or very large numbers Difference between machine numbers is not uniform Representation error depends on the number of bits used in the mantissa.

SE301_Topic1

38

Lectures 3 3-4 4

Taylor Theorem

Motivation T l Th Taylor Theorem Examples

Reading assignment: Chapter 4

SE301_Topic1 39

M ti ti Motivation

We can easily W il compute expressions i like lik 3 10 2 2( x + 4)

4.1, sin(0.6) ?

But, How do you compute

We can use the h definition d fi i i to compute sin(0.6) ? is this a practicalway?

SE301_Topic1

b
0.6

40

T l rS Taylor Series ri
Th Taylor The T l series i expansion i of f f ( x) about b t x0 f ( 2) ( x0 ) f (3) ( x0 ) 2 f ( x0 ) + f ( x0 ) ( x x0 ) + ( x x0 ) + ( x x0 )3 + ... 2! 3! or
'

Taylor Series =

1 (k ) k f ( x ) ( x x ) k! 0 0 k =0

if the series converge we can write f ( x) = 1 (k ) k! f ( x0 ) ( x x0 ) k k =0

41

SE301_Topic1

Taylor Series
E Example l 1
Obtain Taylor series expansion of f ( x) = e x about x = 0

f ( x) = e x f ' ( x) = e x f ( 2 ) ( x) = e x f ( k ) ( x) = e x

f ( 0) = 1 f ' ( 0) = 1 f ( 2 ) ( 0) = 1 f ( k ) (0) = 1 for k 1

k 1 x e x = f ( k ) ( x0 ) ( x x0 ) k = k = 0 k! k = 0 k!

The series converges for x <

SE301_Topic1 42

Taylor Series
3

E Example l 1
2.5 exp(x) p( ) 2 1+x+0.5x 2 1+x 1.5

0.5

0 -1 1
SE301_Topic1

-0 0.8 8

-0 0.6 6

-0 0.4 4

-0 0.2 2

02 0.2

04 0.4

06 0.6

08 0.8

1
43

Taylor Series
E Example l 2
Obtain Taylor series expansion of f ( x) = sin( x) about x = 0

f ( x) = sin( ( x) f ' ( x) = cos( x) f ( 2) ( x) = sin( x) f (3) ( x) = cos( x) sin( x) =

f (0) = 0 f ' (0) = 1 f ( 2) (0) = 0 f (3) (0) = 1

k =0
SE301_Topic1

3 5 7 f ( k ) ( x0 ) x x x ( x x0 ) k = x + + .... 3! 5! 7! k!

The h series i converges for f x <

44

4 3 2 1 0 -1 -2 -3 -4 -4 4
SE301_Topic1

x-x 3/3!+x 5/5!

sin(x)

x-x 3/3!

-3 3

-2 2

-1 1

4
45

Convergence of Taylor Series

(Observations, Example 1)

The Taylor Th T l series i converges fast f (few (f terms are needed) when x is near the point of expansion. p If | |x-c| | is large g then more terms are needed to get good approximation.

SE301_Topic1

46

Taylor Series
E Example l 3
Obtain Taylor series expansion of f(x) = 1 f ( x) = 1 x 1 f ' ( x) = (1 x )2 2 f ( 2 ) ( x) = (1 x )3 6 f ( 3) ( x ) = (1 x )4 Taylor Series Expansion of
SE301_Topic1

1 about x = 0 1 x

f (0) = 1 f ( 0) = 1 f (0) = 2 f (0) = 6 1 = 1 + x + x 2 + x 3 + .... 1 x

47

Example 3 remarks k

C Can we apply l Taylor T l series i for f x>1?? 1?? How many terms are needed to get good approximation???

These questions will be answered using Taylor Theorem

SE301_Topic1

48

T l r Th Taylor Theorem r
If a function f(x) posses continuous derivatives of orders 1,2,..., (n + 1) in a closed interval [a, b] then for any c [a, b]
( (n+1) ) terms Truncated Taylor Series

f ( x) = where

k =0

f ( k ) (c ) ( x c) k k!

+ En+1
Reminder

f ( n+1) ( ) ( x c) n+1 and is between x and c. En+1 = (n + 1)!

SE301_Topic1 49

Taylor Theorem
We can apply Taylor thorem for 1 f(x) = with point of expansion c = 0 if | x | < 1 1 x if [a, b] includes x = 1 then the function and its derivatives are not defined. defined Taylor Theorem is not applicable.
SE301_Topic1 50

Err r T Error Term r

T get an idea To id about b the h approximat i ion i error We can derive an upper bound on ( ) n +1 En+1 = ( x c) (n + 1) )! for all values of between x and c. f
SE301_Topic1 51

( n +1)

Error Term for

E Example l 4
How large is the error if we replaced f ( x) = e x by the first 4 terms (n = 3) of its Taylor series expansion about x = 0 when x = 0.2 ?
f ( k ) ( x) = e x f ( k ) ( ) e 0.2 for k 1 f ( n+1) ( ) En+1 = ( x c) n+1 (n + 1)! e 0 .2 (0.2)n+1 E4 8.14268E 05 En+1 (n + 1)!
SE301_Topic1 52

Alternative form of Taylor Theorem

L f ( x) have Let h continuous ti d i ti s of derivative f orders 1,2,...(n + 1) on an interval [a,b] , and x [a,b] and x + h [a,b] then f ( x + h) = f

( ) n+1 h where is between x and x + h En+1 = (n + 1)!

SE301_Topic1 53

k =0 ( n +1)

f ( k ) ( x) k h + En+1 k!

Taylor Theorem
Alternative forms
n ( n +1) ( ) f ( k ) (c ) f k ( x c) + ( x c) n+1 k! (n + 1)!

f ( x) =

k =0

where is between x and c x x + h, c x

f ( x + h) =

k =0
SE301_Topic1

f ( k ) ( x) k f ( n+1) ( ) n+1 h + h k! (n + 1)!

54

where h is i between b x and d x+h

M Mean V Value l Th Theorem r

If f(x) f( ) is i a continuous ti f ti on a closed function l d interval i t l [a, [ b] and its derivative is defined on the open interval (a, b) then there exist [a, b] df( ) f(b) f(a) = dx (b-a) Proof : Use Taylor y Theorem n = 0, x = a, x + h = b df( ) f(b) = f(a) + (b-a) dx
SE301_Topic1 55

Alt r ti Series Alternating S ri Th Theorem r

Consider the alternating series S = a1 a2 + a3 a4 + a a a a The series converges 1 2 3 4 then and If f and lim a = 0 S S n an +1 n n S n : partial sum s m (sum (s m of the first n terms) an +1 : First omitted term
SE301_Topic1 56

Alternating Series E Example pl 5

1 1 1 sin(1) can be computed using sin(1) = 1 + + 3! 5! 7! This is a convergent alternating series since a1 a2 a3 a4 and lim an = 0
n

Then 1 1 sin(1) 1 3! 5! 1 1 1 sin(1) 1 + 3! 5! 7!

SE301_Topic1 57

E Example pl 6
Obtain the Taylor series expansion of f(x) = e 2x +1 about c = 0.5 (the center of expansion) How large can the error be when (n + 1) terms are used to app approximat o ate e 2x +1 w with t x = 1?

SE301_Topic1

58

E Example l 6
Obtain Taylor series expansion of f ( x) = e 2 x +1 , c = 0.5
f ( x) = e 2 x +1 f ' ( x) = 2e 2 x +1 f ( 2) ( x) = 4e 2 x +1 f ( k ) ( x) = 2 k e 2 x +1 e
2 x +1

f (0.5) = e 2 f ' (0.5) = 2e 2 f ( 2) (0.5) = 4e 2 f ( k ) (0.5) = 2 k e 2

k =0 2

f ( k ) (0.5) ( x 0.5) k k!
2 2

k ( x 0.5) 2 k 2 ( x 0.5) = e + 2e ( x 0.5) + 4e + ... + 2 e + ... 2! k!

SE301_Topic1 59

Example 6 Error term

f ( k ) ( x) = 2 k e 2 x +1 f ( n+1) ( ) Error = ( x 0.5) n+1 (n + 1)! Error = 2 n+1 e 2 +1 Error 2
n +1

( x 0.5) n+1 (n + 1)!

( x 0.5) n+1 max e 2 +1 (n + 1)! [0.5,1]

n +1 ( x 0 . 5 ) Error 2 n+1 e3 (n + 1)!

SE301_Topic1 60

R Remark rk

In this course all angles are assumed to be in radian unless you are told otherwise

SE301_Topic1

61

M l ri series Maclurine ri
Find Maclurine series expansion of cos (x)

Maclurine series is a special case of Taylor series with the center of expansion c = 0

SE301_Topic1

62

Taylor Series
E Example l 7
Obt i Maclurine Obtain M l i series i expansion i of f f ( x) = cos( ( x)

f ( x) = cos( x) f ' ( x) = sin( x) f ( 2 ) ( x) = cos( x) f ( 3) ( x) = sin( x) cos( x) =

k =0

f ( 0) = 1 f ' (0) = 0 f ( 2 ) (0) = 1 f ( 3 ) ( 0) = 0

2 4 6 f ( k ) ( 0) x x x ( x) k = 1 + + .... k! 2! 4! 6!

The series converges for x <

SE301_Topic1 63