You are on page 1of 34

Numerical Methods

INTRODUCTION TO NUMERICAL METHODS 1.0 Introduction Numerical methods are techniques by which mathematical problems are formulated so that they can be solved with arithmetic operations, however, they involve large numbers of tedious arithmetic calculations. With the availability of fast, digital computers, numerical methods of solving engineering problems became more popular. In pre-computer era, engineers solved problems in the following manner: Solutions were derived for some problems using analytical, or exact methods. Graphical solutions were used to characterize the behavior of systems. Calculators and slide rules were used to implement numerical methods manually.

With this, significant amount of energy were expended on the solution technique itself, rather than on the problem definition and interpretation. With fast and efficient computers, numerical methods can be implemented and solutions obtained within a short span of time.

Figure 1. 1 Process of problem solving with (a) depicting the process pre-computer era and (b) when computers are made available. The size of the box is proportional to the time spent for each process.

Introduction to Numerical Methods

Page 1

Numerical Methods
Reasons why an engineer should study numerical methods:

Numerical methods are extremely powerful problem-solving tools. In the field, there are occasions which an engineer will use an available prepackaged or canned computer programs that involve numerical methods. However, not all problems can be solved by these canned programs. Studying numerical methods can increase the capabilities of these canned programs by customizing it according to the specifications of the problem. Numerical methods are an efficient vehicle for learning to use computers. Numerical methods provide a vehicle for you to reinforce your understanding of mathematics.

The following subject areas are covered in the study of numerical methods: Roots of equations these problems are concerned with the value of a variable or a parameter that satisfies a single nonlinear equation. These problems are especially valuable in engineering design contexts where it is often impossible to explicitly solve design equations for parameters.

Figure 1. 2 Roots of equation.

Systems of linear algebraic equations these problems are similar in spirit to roots of equations in the sense that they are concerned with values that satisfy equations. However, in contrast to satisfying a single equation, a set of values is sought that simultaneously satisfies a set of linear algebraic equations.

Figure 1. 3. Systems of linear algebraic equations.

Introduction to Numerical Methods

Page 2

Numerical Methods
Optimization these problems involve determining a value or values of an independent variable that correspond to a best or optimal value of a function. Thus, optimization involves identifying maxima and minima.

Figure 1. 4. Optimization

Curve fitting the techniques developed for this purpose can be divided into two general categories: regression and interpolation. Regression is employed where there is a significant degree of error associated with the data, and the strategy is to derive a single curve that represents the general trend. Interpolation is used when the objective is to determine intermediate values between relatively error-free data, and the strategy is to fit a curve directly through the data points and use the curve to predict the intermediate values.

Figure 1. 5. Curve fitting.

Integration and differentiation - numerical integration is the determination of area under the curve, while differentiation is finding the slope of the curve at various points.

Figure 1. 6. Integration.

Introduction to Numerical Methods

Page 3

Numerical Methods

Figure 1. 7. Differentiation.

Ordinary and partial differential equations many physical laws are couched in terms of the rate of change of the quantity rather than the quantity itself. If the quantity is one-dimensional, an ordinary differential equation arises; a partial differential equation is obtained when the quantity is multidimensional.

1.1 Mathematical Modeling and Engineering Problem Solving A mathematical model can be broadly defined as the formulation or equation that expresses the essential features of a physical system or process in mathematical terms. In a very general sense, it can be represented as a functional relationship of the form Dependent independent forcing f , parameters, variable variables functions (1.1)

where the dependent variable is a characteristic that usually reflects the behavior or state of the system; the independent variables are usually dimensions, such as space and time, along which the systems behavior is being determined; the parameters are reflective of the systems properties or composition; and the forcing functions are external influences acting upon the system. The actual mathematical expression of Equation 1.1 can range from simple algebraic relationship to large complicated sets of differential equations. Mathematical models of the physical world are characterized by the following: It describes a natural process or system in mathematical terms. It represents an idealization and simplification of reality. It yields reproducible results and consequently can be used for predictive purposes.

Introduction to Numerical Methods

Page 4

Numerical Methods
Example 1.1 Construct a mathematical model for a falling parachutist and solve it analytically. The schematic is shown below. Assume that the mass of the parachutist is 68.1 kg and the drag coefficient is 12.5 kg/s and that the parachutist is initially at rest.

Answer: A free-falling body is acted upon by gravity, and by the Newtons second law of motion, its acceleration, or the rate of change of its velocityv, is given as dv F dt m

where F is the net force acting on the body and m is its mass. The net force acting on the body consists of the pull of gravity FD taken positive and the drag force FU acting opposite the direction of the motion, or F FD FU The pull of gravity or the weight is the force exerted due to the mass of the body, or FD mg where g 9.8 m/s . The drag force FU is assumed to be directly proportional to the velocity of the body, or FU cv where c is the drag coefficient in kg/s. Relating these to the acceleration of the body gives dv mg cv dt m
Introduction to Numerical Methods Page 5

Numerical Methods
or dv c g v dt m which is a first-order linear differential equation. Its general analytical solution, assuming the body is initially at rest is given as vt gm ct 1 exp c m

Inserting the given parameters, the analytical model for the falling parachutist is vt 53.391 exp0.18355t Solving for values of the velocity for different times generates the following table.

Plotting the values gives the following graph

Introduction to Numerical Methods

Page 6

Numerical Methods
In the case of Example 1.1, the resulting differential equation can be solved analytically using elementary methods. However, there are mathematical models which cannot be solved analytically. For these cases, a numerical solution can be formulated. Example 1.2 Formulate a numerical solution for the mathematical model of Example 1.1. Answer: To recall, the differential equation that describes the motion of the falling parachutist is c dv g v m dt

The derivative dv/dt can be approximated as (see the figure below the equation) dv v vt vt dt t t t

where vt is the value of the veolocity at time t and vt is the value of the velocity at some late time t . This equation is called the finite divided difference approximation of the derivative. This is substituted to the mathematical model derived for the falling parachutist. Thus, vt vt c g vt t t m
Introduction to Numerical Methods Page 7

Numerical Methods
Rearranging, vt vt g c vt t t m

which gives an idea on how to (numerically) compute the value of the velocity vt at time t using the previous value of the velocity vt at time t . If one is given an initial value for velocity at some time t , the value of the velocity for the later time t can be computed. Using this, the following table can be generated.

The plot of the numerical solution, together with the exact solution is shown below.

Introduction to Numerical Methods

Page 8

Numerical Methods
From Example 1.2, the following ideas can be deduced: The approach used to compute for subsequent values of the velocity was New value Old value slope step size This approach is formally called the Eulers method. The numerical solution of the differential equation is the approximation of the exact solution. This is because a finite value for the time interval or the step size is used. The approximation can be improved if the step size is made small; however this results to a increasing the number of computations. In numerical methods, a trade-off between accuracy of the solution and the computational effort exists. Thus, one has to compute more values to achieve more accurate solution.

Drill Problems 1.1 1. Rather than the linear relationship for the drag force employed in modeling the motion of a falling parachutist of Example 1.1, use the second-order relationship FU cv where c here is the second-order drag coefficient. (a) Obtain an analytical solution for the velocity of the body. Assume that the jumper has zero velocity at t 0. Assuming that the mass of the jumper is 68.1 kg and the drag coefficient as 0.225 kg/ m, compute the velocity for the first ten seconds starting from t 0 with two seconds interval. (b) Formulate a numerical solution to the differential equation and compute for the velocity of the body for the same instances as that of (a). (c) Recompute the numerical solution for (b), reducing the time intervals to one seconds. Compare the results with that of (b) in terms of their differences with the solution of (a). 2. The amount of a uniformly distributed radioactive contaminant contained in a closed reactor is measured by its concentration c (in becquerel per liter or Bq/L). The contaminant decreases at a decay rate proportional to its concentration. Use Eulers method to numerically solve the model for t 0 up to t 1d (d days, with the decay constant k 0.2 d , step size t 0.1 and at t 0, the concentration of the contaminant is 10 Bq/L. 3. A storage tank as shown in the figure below contains a liquid at depth y where y 0 when the tank is half full. Liquid is withdrawn at a constant flow rate Q to meet demands. The contents are resupplied at a sinusoidal rate 3Q sin t. A conservation model for this system is written as dAy 3Q sin t Q dx

Introduction to Numerical Methods

Page 9

Numerical Methods

or, since the surface area A is constant, dy Q Q 3 sin t dx A A

Use Eulers method to solve for the depth y from t 0 to 10 d with a step size of 0.5 d . The parameter values are A 1200 m and Q 500 m /d. Assume that the initial condition is y 0. 4. Suppose that a spherical droplet of liquid evaporates at a rate that is proportional to its surface area, dV kA dt

where V = volume (mm ), t = time (min), k = evaporation rate (mm/min), and A = surface area (mm ). Use Eulers method to compute the volume of the droplet from t 0 to 10 min using a step size of 0.25 min. Assume that k 0.1 mm/min and that the droplet initially has a radius of 3 mm. What is the radius of the droplet after ten minutes? 5. Newtons law of cooling says that the temperature of a body changes at a rate proportional to the difference between its temperature and that of the surrounding medium (the ambient temperature) dT kT T dt

where T = temperature of the body (), t = time (min), k = proportionality constant (per min), and T = the ambient temperature (). Suppose that a cup of coffee originally has a temperature of 68. Use Eulers method to compute the temperature from t 0 to 10 min using a step size of 1 min if T 21 and k 0.1/min.

Introduction to Numerical Methods

Page 10

Numerical Methods
1.2 Programming and Software In the previous article, a model for the free-falling motion of a parachutist was developed. It took the form of the differential equation dv c g v dt m (1.2)

Also, a solution to this equation was developed by using a simple numerical approach called the Eulers method, v v dv t dt (1.3)

Given an initial condition, this equation can be repeatedly used to compute the velocity as a function of time. But accurate solutions require very small step sizes and hence a lot of computational effort. With the aid of a computer, the computations can be performed easily. Computer programs are set of instructions that direct the computer to perform a certain task. Computer programming can be narrowed down into the following topics: Simple information representation (contants, variables, and type declarations) Advanced information representation (data structure, arrays, and records) Mathematical formulas (assignment, priority rules, and intrinsic functions) Input and output Logical representation (sequence, selection, and repetition) Modular programming (functions and subroutines)

In the early days of computers, programmers usually did not pay much attention to whether their programs were clear and easy to understand. Today, it is recognized that there are many benefits to writing organized, well-structured code. Thus, structured programming came into place. Structured programming is a set of rules that prescribe good style habits for the programmer. A key idea behind structured programming is that any numerical algorithm can be composed using the three fundamental control structures: sequence, selection, and repetition. To demonstrate the idea of structured programming, flowcharts and pseudocodes are used. A flowchart is a visual or graphical representation of an algorithm. The flowchart employs a series of blocks and arrows, each of which represents a particular operation or step in the algorithm. The arrows represent the sequence in which the operations are implemented. See Figure 1.8.

Introduction to Numerical Methods

Page 11

Numerical Methods

Figure 1. 8. Symbols used in flowcharts.

Another approach used to express algorithms is called the pseudocode. This technique uses code-like statements in place of the graphical symbols of flowcharts. As stated before, structured programming requires effective use of the following control structures: Sequence. The sequence structure expresses the trivial idea that unless you direct it otherwise, the computer code is to be implemented one instruction at a time.

Figure 1. 9. Sequence structure of computer programs.

Introduction to Numerical Methods

Page 12

Numerical Methods
Selection. The selection structure provides a means to split the programs flow into branches based on the outcome of a logical condition.

Figure 1. 10. Selection structure.

If there are multiple alternatives, the following selection structures can be used.

Introduction to Numerical Methods

Page 13

Numerical Methods

Figure 1. 11.Supplementary selection or branches construct.

Repetition. Repetition provides a means to implement instructions repeatedly. The resulting constructs are called loops. There are two types of loops, depending on how they are terminated. The first one is called a decision loop because it terminates based on the result of a logical condition. The second one is called a count-controlled loopperforms a specified number of repetitions or iterations.

Introduction to Numerical Methods

Page 14

Numerical Methods

Figure 1. 12. A decision loop structure.

Figure 1. 13. A count-controlled structure.

Aside from structured programming, modular programming is also being practiced in the development of computer programs. Modular programming refers to the practice of dividing the computer program into modules, called procedures, designed to specifically perform a certain task. There are two types of procedures commonly employed: Functions which return a single result; Subroutines which return several results.
Page 15

Introduction to Numerical Methods

Numerical Methods
Modular programming has a number of advantages: The use of small, self-contained units makes the underlying logic easier to device and to understand for both the developer and the user. Development, as well as maintenance and modification of the program is facilitated. Increases the ease with which a program can be debugged and tested.

Example 1.3 Develop a pseudocode for a computer program that will compute for the roots of a quadratic equation ax bx c 0. The algorithm is as follows: 1. Prompts the user for coefficients a, b and c. 2. Implements the quadratic formula, guarding against all eventualities. 3. Displays the solution. 4. Allows the user the option to return to step 1 and repeat the process. Example 1.4 Develop a pseudocode for a computer program that will implement the Eulers algorithm as applied to solving the differential equation describing the motion of the parachutist of Example 1.1. In this course, Microsoft Excel VBA and the Mathscript mode of LabVIEW will be used to implement the algorithms developed. See Figures 1.14 and 1.15 for implementation of pseudocodes in Excel and Mathscript, respectively. Drill Problems 1.2 1. Write a well-structured pseudocode to implement the flowchart depicted in the figure below.

Introduction to Numerical Methods

Page 16

Numerical Methods
2. Develop a well-structured function procedure that is passed a numeric grade from 0 to 100 and returns a letter grade according to the following scheme:

3. The cosine function can be evaluated by the following infinite series: x x x cos x 1 2! 4! 6!

Write an algorithm to implement this formula so that it computes and prints out the values of cos x as each term in the series is added. In other words, compute and print in sequence the values for cos x 1 x cos x 1 2 x x cos x 1 2! 4!

up to the order term n of your choosing. Write the algorithm as a well-structured pseudocode. 4. The divide and average method, an old-time method for approximating the square root of any positive number a can be formulated as x x 2 Write a well-structured pseudocode for the implementation of this algorithm. It should do the following: Allow for a user input a.Check whether the user input is valid. Display 0 when the input is 0 or not valid. Set a value of tolerance ( 10 ). Implement the formula. Repeat the calculation until the answer is correct to six decimal places. Display the answer. 5. Write a well-structured pseudocode that will compute the factorial of an input integer. Make sure that the algorithm includes checks on valid values, as well as a correct output for 0!.

Introduction to Numerical Methods

Page 17

Numerical Methods

Figure 1. 14. Pseudocode implementation in Excel VBA.

Introduction to Numerical Methods

Page 18

Numerical Methods

Figure 1. 15. Pseudocode implementation in Mathscript.

Introduction to Numerical Methods

Page 19

Numerical Methods
1.3. Approximations and Round-Off Errors This section covers basic topics related to the identification, quantification, and minimization of these errors. Two major forms of errors are discussed in this section and the next: round-off errors is due to the fact that computers can represent only quantities with a finite number of digits; and truncation error is the discrepancy introduced by the fact that numerical methods may employ approximations to represent exact mathematical operations and quantities. Whenever numbers are employed in computations, there should be an assurance that it can be used with confidence. The concept of significant figure, or digit, has been developed to formally designate the reliability of a numerical value. The significant digits correspond to the number of certain digits plus one estimated digit. The concept of significant figures has two important implications for the study of numerical methods: Numerical methods yield approximate results. Therefore, there should a criteria to specify the confidence in the approximate result. Computers can only represent finite number of digits for a given number. Quantities such as e, , and 7 has infinite number of significant digits. Therefore, the computer will omit most of these significant digits which will result in round-off errors.

The errors associated with both calculations and measurements can be characterized with regard to their accuracy and precision. Accuracy refers to how closely a computed or measured value agrees with the true value. Precision refers to how closely individual computed or measured values agree with each other. Figure 1.16 explains the point.

Figure 1. 16. Illustration of accuracy and precision.

Introduction to Numerical Methods

Page 20

Numerical Methods
Inaccuracy (also called bias) is defined as the systematic deviation from the true value. Imprecision (also called uncertainty) refers to the magnitude of the scatter of the measurements or computed values. Numerical methods should ensure that there is no bias nor uncertainty with the results for it to be considered a valid solution. Numerical errors arise from the use of approximations to represent exact mathematical operations and quantities. These include round-off and truncation errors. For both types, the following relationships hold true: True value approximation error Thus, error is defined as the discrepancy between the true and approximated values, or E True approximate (1.4)

where E is called the true or exact error. Errors are normalize to take into account the scale or magnitude of the problem being considered. Thus, the true fractional relative error as a percentage is given as e E 100% True value (1.5)

Example 1.5 The measured values of a bridge and a rivet are 9999 cm and 9 cm, respectively. If the true values are 10000 cm and 10 cm respectively, compute the true error and the true percent relative error for each case. Answers: For the bridge, E 1 cm, e 0.01% For the rivet, E 1 cm, e 10% In most cases, the true value will never be known, unless there is an analytic solution to the problem. An alternative for this is to normalize the error using the best available estimate of the true value, that is e approximate error 100% approximate value (1.6)

One of the challenges of numerical methods is to determine error estimates in the absence of knowledge regarding the true value. That is why numerical methods employ different ways of determining the approximate error depending on the algorithms used.

Introduction to Numerical Methods

Page 21

Numerical Methods
Errors can be positive or negative, however, signs of the error may not play a role in the computation. Therefore it is often useful to employ the absolute values of the errors and compare it with a certain tolerance, or |e | e (1.7)

If this relationship holds, the results are assumed to be within the prespecified acceptable level e . It is also convenient to relate these errors to the number of significant figures in the approximation. Thus, it can be shown that e 0.5 10 % where n is the number of significant figures (at least). In the following example, the Taylor series expansion for e will be used to demonstrate errors. Example 1.5 The Taylor series expansion for e is x x x 2 3! n! (1.8)

e 1 x

Using this expansion, determine the value of e. correct to three significant figures. Answer: Adding successive terms of the Taylor series expansion, the following table can be generated.

After six terms, e. 1.648697917 which is correct to five significant figures.

Introduction to Numerical Methods

Page 22

Numerical Methods
Round-off Errors. Round-off errors result from the following situations: Irrational numbers such as , e, and 7 cannot be expressed by a fixed number of significant figures; hence, some of these significant figures are dropped. Computers use base-2 number representation; they cannot precisely represent certain base-10 numbers.

The discrepancy introduced by this omission of significant figures is called round-off error. Computer Representation of Numbers. The base-10 number system is the number system that humans are most familiar with. However, since computers are built on devices that acquires only two states (on / off), computers represent numbers using a binary, or base-2 system. Numbers in computers are represented in two forms: integer and floating-point representation. Integers may be in signed or unsigned magnitude format. If the format is unsigned, all the bits representing the number determine the magnitude of the number. If the format is signed, the most significant bit (left-most bit) represents the sign of the number, with a 0 for a positive number and 1 for a negative. The remaining bits determine the magnitude of the number.

Figure 1. 17. Weights of each binary bit position for an unsigned 8-bit binary number.

Figure 1. 18. Weights of each binary bit position for a signed 8-bit binary number.

Example 1.6 What is the number 10000101 (base-2) in decimal if it is written as (a) unsigned integer; (b) signed integer? Answers: (a) 133; (b) 123

Introduction to Numerical Methods

Page 23

Numerical Methods
On the other hand, fractional quantities are typically expressed in computers using floating-point form. In this approach, the number is expressed as a fractional part, called a mantissa or significand, and an integer part, called an exponent or characteristic, as in m b , where m is the mantissa, b is the base of the number system being used, and c is the exponent. The mantissa holds a value that is 1 m1 b

Thus, for a base-10 number, the mantissa is between 0.1 and 1.

Figure 1. 19. Number representation using floating-point.

For computer systems, IEEE Standard 754 defines the standard for representing floating-point numbers. In this standard, Single precision floating-point numbers, which uses 32 bits, has one sign bit, 8-bit biased exponent and 24-bit mantissa (but only 23 bits are presented); Double precision floating-point numbers, which uses 64 bits, has one sign bit, 11-bit biased exponent and a 53-bit mantissa (but only 52 bits are presented).

Example 1.7 Represent the decimal numbers 12 and 0.75 into single precision floating-point numbers. Answers:

Example 1.8 Create a hypothetical floating-point number set for a machine that stores information using 7-bit words. Employ the first bit for the sign of the number, the next three for the sign and magnitude of the exponent, and the last three for the magnitude of the mantissa.

Introduction to Numerical Methods

Page 24

Numerical Methods
The following are some important points about floating-point numbers that are significant in relation to computer round-off errors as observed from Example 1.8: 1. There is a limited range of quantities that may be represented. 2. There are only a finite number of quantities that can be represented within the range. 3. The interval between numbers increases as the numbers grow in magnitude.

Figure 1. 20. Problems associated with floating-point numbers.

Arithmetic Manipulations of Computer Numbers. Aside from the limitations of a computers number system, the actual arithmetic manipulations involving these numbers can also result in round-off error. Addition and subtraction when two floating-point numbers are added or subtracted, the mantissa of the number with the smaller exponent is modified so that the exponents are the same. For subtraction, the sign of the subtrahend is complemented. The results are then added and the results are chopped off to the specified number of mantissa digits. Multiplication and division when two floating-point numbers are multiplied (or divided), the mantissas of the numbers are multiplied (divided) and the exponents added (or subtracted). Most computers hold intermediate results in a double-length register (since multiplication will yield results double the number of digits). The results are then chopped. Large computations Certain methods require extremely large numbers of arithmetic manipulations to arrive at their final results. In addition, these computations are often
Page 25

Introduction to Numerical Methods

Numerical Methods
interdependent. Consequently, even though an individual round-off error could be small, the cumulative effect over the course of a large computation can be significant. Drill Problems 1.3 1. If |x| 1 it is known that

For x 0.1, evaluate this series correct to seven significant figures. How many terms are required to achieve such accuracy? 2. Evaluate e using two approaches x x 2 3!

1 1 x x x 1x

e 1 x

and e 1 1 e 1 x !

and compare with the true value of 6.737947 10 . Use 20 terms to evaluate the series and compute the true and relative errors as terms are added. 3. The infinite series

fn

1 k

converges on a value of fn as n approaches infinity. Evaluate this infinite series in single and double precision to calculate fn for n 10000 by computing the sum from k 1 to 10000 and in reverse order. Also evaluate the true and approximate relative errors for each case. 4. Determine the roots of a quadratic equation having a 1, b 3000.001, and c 3. Use single and double precision arithmetic. Evaluate the true relative error and determine the cause of the discrepancy between the solutions. The true roots for the equation are x 0.001 and x 3000. 5. For 100 equally spaced values of x between 10 to 10. , evaluate the expressions y x 4 x 3 and y 1

x 4 x 3

Note that both expressions are mathematically equivalent. Using plotting tools, graph the function in the given interval. Which function is better in terms of resisting loss of significance?
Introduction to Numerical Methods Page 26

Numerical Methods
1.4. Truncation Errors and the Taylor Series Truncation errors are those that result from using an approximation in place of an exact mathematical procedure. In order to gain insight into the properties of such errors, the Taylor series will be used. Taylor Series. The Taylor series provides a means to predict a function value at one point in terms of the function value and its derivatives at another point. In particular, Taylors theorem states that any smooth function can be approximated by a polynomial. The Taylors theorem states that, if the function f and its first n 1 derivatives are continuous on an interval containing a and x, then the value of the function at x is given by
fx fa f ax a f a f a f a x a x a x a R 2! 3! n!

(1.9)

where the remainder R is defined as

x t t dt f n!

(1.10)

Equation 1.9 is called the Taylor series expansion of fx. If the remainder R is omitted, Equation 1.9 is the Taylor polynomial approximation to fx. The remainder R has another form called the derivative or the Lagrange form of the remainder, which can be derived from the integral mean-value theorem. R f x a n 1! (1.11)

where is a point between the interval a and x such that the function fx, takes the average value f. Figure 1.21 depicts how Taylor series approximates the function. The zero-order approximation, involves the first term of the series, fx fx (1.12)

Indicates that the value of the f at the new point is the same as its value at the old point. However, better approximation will result if more terms of the Taylor series are added, that is, the first-order approximation, fx fx f xx x takes the value of f closer to the true values. Subsequent addition of terms,
Introduction to Numerical Methods Page 27

(1.13)

Numerical Methods
fx fx f xx x f x x x 2! (1.14)

fx fx f xx x

f x f x x x x x 2! 3!

(1.15)

and so on results in better approximations, resulting in the complete Taylor series fx fx f xx x f x f x x x x x 2! 3!

f x x x R n!

(1.16)

where R is the remainder term that accounts for all the terms above n up to infinity.

Figure 1. 21. Taylor series approximation.

It is often convenient to simplify the Taylor series by defining a step size h x x and expressing 1.16 as f x f x f x fx fx f xh h h h R 2! 3! n!

(1.17)

Introduction to Numerical Methods

Page 28

Numerical Methods
where the remainder term is now expressed as f R h n 1! (1.18)

Example 1.9 Use zero- through fourth-order Taylor series expansions to approximate the function fx 0.1x 0.15x 0.5x 0.25x 1.2 from x 0 with h 1. Answers: Zero-order approximation : f1 1.2 First-order approximation f1 0.95 Second-order approximation: f1 0.45 Third-order approximation: f1 0.3 Fourth-order approximation: f1 0.2 Note that the true value of the function at x 1 is 0.2, that is, the fourth-order approximation is the true value. For an nth degree polynomial, the n th-order Taylor series approximation yields the exact value of the polynomial at the point. For other differentiable and continuous functions, such as exponentials and sinusoids, a finite number of term will not yield an exact result. Thus, truncation errors are incurred, in the form of the remainder term R . Although the above is true, the practical value of the Taylor series expansions is that, in most cases, the inclusion of only a few terms will result in an approximation that is close enough to the true value for practical purposes. The assessment of how many terms are required to get close enough is based on the remainder term of the expansion. However, the relationship of 1.18 has two major drawbacks: The number is not known exactly but merely lies between x and x . The n 1th derivative of fx must be known, but fx is basically unknown.

Despite this, the remainder term is still useful for gaining insight into truncation errors, because h can be chosen and hence the number of terms to be included in the expansion can be controlled. It can be shown that the error is proportional to the step size h raised to the n 1th power, or R Oh (1.19)

Introduction to Numerical Methods

Page 29

Numerical Methods
Example 1.10 Use Taylor series expansions with n 0 to 6 to approximate fx cos x at x , on the basis of the value of fx and its derivatives at x .

Answer: The table below gives subsequent results of computations.

The Remainder Term For the Taylor Series Expansion. If the Taylor series expansion was truncated after the zero-order approximation, the remainder term R is R f x h f x f x h h 2 3! (1.20)

A graphical depiction of such truncation is shown in Figure 1.22.

Figure 1. 22. Graphical interpretation of zero-order truncation.

Introduction to Numerical Methods

Page 30

Numerical Methods
Note that in this truncation, the lower-order derivatives usually account for the greater share in the remainder term than the higher-order derivatives, although the results are still inexact because the secondand higher-order terms are neglected. According to the derivative mean-value theorem, if a function fx and its first derivative are continuous over an interval from x to x , then there exists a point on the function that has a slope, f , that is parallel to the line joining fx and fx . The point marks the x value where this slope occurs.

Figure 1. 23. Graphical interpretation of the derivative mean-value theorem.

Thus the zero-order remainder term, with this theorem can be written as R f h (1.21)

The higher-order remainder term can be derived in the same manner, for example, the first-order remainder term as f R h 2 (1.22)

Using the Taylor Series To Estimate Truncation Errors. The Taylor series approach can be used to numerically solve the problem of Example 1.1 (and hence reinforce the idea of the Eulers method presented in Example 1.2). The Taylor series expansion of the velocity as a function of time gives vt vt v t t t v t t t R 2 (1.23)

Introduction to Numerical Methods

Page 31

Numerical Methods
Truncating the series after the first derivative, vt vt v t t t R (1.24)

with which v t vt vt R t t t t (1.25)

where the first term on the right of 1.25 depicts the first-order approximation term while the second one is the truncation error. Thus, the algorithm used to estimate the derivative of v t before now has an estimate of the truncation error associated with this approximation, which, according to 1.22. is R v t t t t 2 or R Ot t t t which implies that the error is a function of the step size h t t . Example 1.11 Investigate the effect of increasing the value of m for the function fx x to the truncation error when Taylor series approximation is used to estimate the function. Note that as m increases, the function becomes more nonlinear. Also, for a particular value of m, say m 4, observe how the first-order truncation error changes as the step size is varied. Note that Equation 1.25 is called the finite divided difference and can be generally represented as f x or f x f O h h (1.29) fx fx Ox x x x (1.28) (1.27) (1.26)

where f is called the first forward difference and h is called the step size. This is one of many numerical methods developed from the Taylor series to approximate derivatives numerically. The others are called backward and centered difference approximations. Moreover, more accurate approximations of the first
Introduction to Numerical Methods Page 32

Numerical Methods
derivative can be developed by using higher-order terms of the Taylor series. Finally, all the above versions can also be developed for second, third, and higher derivatives.

Figure 1. 24. Evaluation of first derivative using (a) forward; (b) backward; and (c) centered finite difference.

The Taylor series can be expanded backward to calculate a previous value on the basis of the present value, as in fx fx f x h f x h 2 (1.30)

Truncating this equation after the first derivative and rearranging yields f x fx fx f O h O h h h (1.31)

where the error Oh and f is called the first backward difference

Introduction to Numerical Methods

Page 33

Numerical Methods
A third way to approximate the first derivative is to subtract 1.31 from the forward Taylor series expansion which will give the centered difference representation of the first derivative f x fx fx O h 2h (1.32)

Example 1.12 Use forward and backward difference approximation of Oh and a centered difference approximation of Oh to estimate the first derivative of fx 0.1x 0.15x 0.5x 0.25x 1.2 at x 0.5 using a step size (a) h 0.5 and (b) h 0.25. Compare the true error in both cases. Answers: (a) For h 0.5 Forward difference Backward difference Centered difference (b) For h 0.25 Forward difference Backward difference Centered difference True error 26.5% 21.7% 2.4% True error 58.9% 39.7% 9.6%

Total Numerical Error. The total numerical error is the summation of the truncation and round-off errors. From the previous discussions, the following are observed: the only way to minimize round-off error is to increase the number of significant figures of the computer. truncation error can be reduced by decreasing the step size.

Introduction to Numerical Methods

Page 34