You are on page 1of 34

Numerical Methods

INTRODUCTION TO NUMERICAL METHODS 1.0 Introduction Numerical methods are techniques by which mathematical problems are formulated so that they can be solved with arithmetic operations, however, they involve large numbers of tedious arithmetic calculations. With the availability of fast, digital computers, numerical methods of solving engineering problems became more popular. In pre-computer era, engineers solved problems in the following manner: • • • Solutions were derived for some problems using analytical, or exact methods. Graphical solutions were used to characterize the behavior of systems. Calculators and slide rules were used to implement numerical methods manually.

With this, significant amount of energy were expended on the solution technique itself, rather than on the problem definition and interpretation. With fast and efficient computers, numerical methods can be implemented and solutions obtained within a short span of time.

Figure 1. 1 Process of problem solving with (a) depicting the process pre-computer era and (b) when computers are made available. The size of the box is proportional to the time spent for each process.

Introduction to Numerical Methods

Page 1

Numerical Methods
Reasons why an engineer should study numerical methods:

• •

• •

Numerical methods are extremely powerful problem-solving tools. In the field, there are occasions which an engineer will use an available prepackaged or “canned” computer programs that involve numerical methods. However, not all problems can be solved by these “canned” programs. Studying numerical methods can increase the capabilities of these “canned” programs by customizing it according to the specifications of the problem. Numerical methods are an efficient vehicle for learning to use computers. Numerical methods provide a vehicle for you to reinforce your understanding of mathematics.

The following subject areas are covered in the study of numerical methods: • Roots of equations – these problems are concerned with the value of a variable or a parameter that satisfies a single nonlinear equation. These problems are especially valuable in engineering design contexts where it is often impossible to explicitly solve design equations for parameters.

Figure 1. 2 Roots of equation.

Systems of linear algebraic equations – these problems are similar in spirit to roots of equations in the sense that they are concerned with values that satisfy equations. However, in contrast to satisfying a single equation, a set of values is sought that simultaneously satisfies a set of linear algebraic equations.

Figure 1. 3. Systems of linear algebraic equations.

Introduction to Numerical Methods

Page 2

Numerical Methods
• Optimization – these problems involve determining a value or values of an independent variable that correspond to a “best” or optimal value of a function. Thus, optimization involves identifying maxima and minima.

Figure 1. 4. Optimization

Curve fitting – the techniques developed for this purpose can be divided into two general categories: regression and interpolation. Regression is employed where there is a significant degree of error associated with the data, and the strategy is to derive a single curve that represents the general trend. Interpolation is used when the objective is to determine intermediate values between relatively error-free data, and the strategy is to fit a curve directly through the data points and use the curve to predict the intermediate values.

Figure 1. 5. Curve fitting.

Integration and differentiation - numerical integration is the determination of area under the curve, while differentiation is finding the slope of the curve at various points.

Figure 1. 6. Integration.

Introduction to Numerical Methods

Page 3

The actual mathematical expression of Equation 1. In a very general sense. it can be represented as a functional relationship of the form Dependent independent forcing ൌ fቀ . the independent variables are usually dimensions. the parameters are reflective of the system’s properties or composition. • Ordinary and partial differential equations – many physical laws are couched in terms of the rate of change of the quantity rather than the quantity itself. Differentiation.1) where the dependent variable is a characteristic that usually reflects the behavior or state of the system. Mathematical models of the physical world are characterized by the following: • • • It describes a natural process or system in mathematical terms. such as space and time.1 Mathematical Modeling and Engineering Problem Solving A mathematical model can be broadly defined as the formulation or equation that expresses the essential features of a physical system or process in mathematical terms. If the quantity is one-dimensional.Numerical Methods Figure 1. a partial differential equation is obtained when the quantity is multidimensional. along which the system’s behavior is being determined. Introduction to Numerical Methods Page 4 . parameters. an ordinary differential equation arises. and the forcing functions are external influences acting upon the system.1 can range from simple algebraic relationship to large complicated sets of differential equations. It yields reproducible results and consequently can be used for predictive purposes. ቁ variable variables functions (1. 7. 1. It represents an idealization and simplification of reality.

or the rate of change of its velocityv. or FD ൌ mg where g ൌ 9. or FU ൌ cv where c is the drag coefficient in kg/s. The drag force FU is assumed to be directly proportional to the velocity of the body.1 Construct a mathematical model for a falling parachutist and solve it analytically.Numerical Methods Example 1.5 kg/s and that the parachutist is initially at rest. Answer: A free-falling body is acted upon by gravity. The schematic is shown below. Relating these to the acceleration of the body gives dv mg െ cv ൌ dt m Introduction to Numerical Methods Page 5 . The net force acting on the body consists of the pull of gravity FD taken positive and the drag force FU acting opposite the direction of the motion. or F ൌ FD െ FU The pull of gravity or the weight is the force exerted due to the mass of the body.1 kg and the drag coefficient is 12. is given as dv F ൌ dt m where F is the net force acting on the body and m is its mass. its acceleration.8 m/s ଶ. and by the Newton’s second law of motion. Assume that the mass of the parachutist is 68.

Numerical Methods or dv c ൌgെ v dt m which is a first-order linear differential equation.39ሺ1 െ expሺെ0. Its general analytical solution.18355tሻሻ Solving for values of the velocity for different times generates the following table. assuming the body is initially at rest is given as vሺtሻ ൌ gm ct ൬1 െ exp ൬െ ൰൰ c m Inserting the given parameters. the analytical model for the falling parachutist is vሺtሻ ൌ 53. Plotting the values gives the following graph Introduction to Numerical Methods Page 6 .

This is substituted to the mathematical model derived for the falling parachutist. However. vሺt ୧ାଵ ሻ െ vሺt ୧ ሻ c ൌ g െ vሺt ୧ ሻ t ୧ାଵ െ t ୧ m Introduction to Numerical Methods Page 7 . the differential equation that describes the motion of the falling parachutist is c dv ൌgെ v m dt The derivative dv/dt can be approximated as (see the figure below the equation) dv Δv vሺt ୧ାଵ ሻ െ vሺt ୧ ሻ ؆ ൌ dt Δt t ୧ାଵ െ t ୧ where vሺt ୧ ሻ is the value of the veolocity at time t ୧ and vሺt ୧ାଵ ሻ is the value of the velocity at some late time t ୧ାଵ . Answer: To recall.Numerical Methods In the case of Example 1. the resulting differential equation can be solved analytically using elementary methods. a numerical solution can be formulated. Thus. Example 1.1.1.2 Formulate a numerical solution for the mathematical model of Example 1. For these cases. This equation is called the finite divided difference approximation of the derivative. there are mathematical models which cannot be solved analytically.

The plot of the numerical solution. the following table can be generated. Introduction to Numerical Methods Page 8 . vሺt ୧ାଵ ሻ ൌ vሺt ୧ ሻ ൅ ቂg െ c vሺt ሻቃ ሺt ୧ାଵ െ t ୧ ሻ m ୧ which gives an idea on how to (numerically) compute the value of the velocity vሺt ୧ାଵ ሻ at time t ୧ାଵ using the previous value of the velocity vሺt ୧ ሻ at time t ୧ . If one is given an initial value for velocity at some time t ୧ .Numerical Methods Rearranging. Using this. together with the exact solution is shown below. the value of the velocity for the later time t ୧ାଵ can be computed.

In numerical methods. Compare the results with that of (b) in terms of their differences with the solution of (a). The contents are resupplied at a sinusoidal rate 3Q sinଶ t. the following ideas can be deduced: • The approach used to compute for subsequent values of the velocity was New value ൌ Old value ൅ slope ൈ step size This approach is formally called the Euler’s method. Use Euler’s method to numerically solve the model for t ൌ 0 up to t ൌ 1d (d ൌ daysሻ. A conservation model for this system is written as dሺAyሻ ൌ 3Q sinଶ t െ Q dx Introduction to Numerical Methods Page 9 . (c) Recompute the numerical solution for (b). Rather than the linear relationship for the drag force employed in modeling the motion of a falling parachutist of Example 1. a trade-off between accuracy of the solution and the computational effort exists. with the decay constant k ൌ 0. Thus. step size Δt ൌ 0. Liquid is withdrawn at a constant flow rate Q to meet demands. however this results to a increasing the number of computations.1. (b) Formulate a numerical solution to the differential equation and compute for the velocity of the body for the same instances as that of (a).Numerical Methods From Example 1. • Drill Problems 1. 2. reducing the time intervals to one seconds. use the second-order relationship FU ൌ cv ଶ where c here is the second-order drag coefficient. one has to compute more values to achieve more accurate solution. A storage tank as shown in the figure below contains a liquid at depth y where y ൌ 0 when the tank is half full.2. The approximation can be improved if the step size is made small. • The numerical solution of the differential equation is the approximation of the exact solution.1 kg and the drag coefficient as 0. Assuming that the mass of the jumper is 68.1 and at t ൌ 0. The contaminant decreases at a decay rate proportional to its concentration. Assume that the jumper has zero velocity at t ൌ 0. 3.225 kg/ m.2 dିଵ . This is because a finite value for the time interval or the step size is used. compute the velocity for the first ten seconds starting from t ൌ 0 with two seconds interval. the concentration of the contaminant is 10 Bq/L. The amount of a uniformly distributed radioactive contaminant contained in a closed reactor is measured by its concentration c (in becquerel per liter or Bq/L).1 1. (a) Obtain an analytical solution for the velocity of the body.

and Tୟ = the ambient temperature (Ԩ). Use Euler’s method to compute the volume of the droplet from t ൌ 0 to 10 min using a step size of 0. k = evaporation rate (mm/min).5 d . Assume that k ൌ 0. Newton’s law of cooling says that the temperature of a body changes at a rate proportional to the difference between its temperature and that of the surrounding medium (the ambient temperature) dT ൌ െkሺT െ Tୟ ሻ dt where T = temperature of the body (Ԩ). Use Euler’s method to compute the temperature from t ൌ 0 to 10 min using a step size of 1 min if Tୟ ൌ 21Ԩ and k ൌ 0. since the surface area A is constant. dy Q Q ൌ 3 sinଶ t െ dx A A Use Euler’s method to solve for the depth y from t ൌ 0 to 10 d with a step size of 0. Suppose that a cup of coffee originally has a temperature of 68Ԩ. The parameter values are A ൌ 1200 mଶ and Q ൌ 500 mଷ /d.25 min.1 mm/min and that the droplet initially has a radius of 3 mm. t = time (min). Assume that the initial condition is y ൌ 0. and A = surface area (mmଶ ). dV ൌ െkA dt where V = volume (mmଷ ). Introduction to Numerical Methods Page 10 . What is the radius of the droplet after ten minutes? 5. Suppose that a spherical droplet of liquid evaporates at a rate that is proportional to its surface area.1/min.Numerical Methods or. k = proportionality constant (per min). t = time (min). 4.

A flowchart is a visual or graphical representation of an algorithm. and type declarations) Advanced information representation (data structure. The flowchart employs a series of blocks and arrows. and repetition) Modular programming (functions and subroutines) In the early days of computers. selection. selection. it is recognized that there are many benefits to writing organized. a solution to this equation was developed by using a simple numerical approach called the Euler’s method. priority rules. With the aid of a computer.Numerical Methods 1. Thus.8.2) Also. Computer programming can be narrowed down into the following topics: • • • • • • Simple information representation (contants. But accurate solutions require very small step sizes and hence a lot of computational effort. and repetition. structured programming came into place. the computations can be performed easily. Structured programming is a set of rules that prescribe good style habits for the programmer. arrays. Introduction to Numerical Methods Page 11 . and intrinsic functions) Input and output Logical representation (sequence. variables.3) Given an initial condition. The arrows represent the sequence in which the operations are implemented. Today. each of which represents a particular operation or step in the algorithm. flowcharts and pseudocodes are used. this equation can be repeatedly used to compute the velocity as a function of time. and records) Mathematical formulas (assignment. well-structured code. A key idea behind structured programming is that any numerical algorithm can be composed using the three fundamental control structures: sequence.2 Programming and Software In the previous article. It took the form of the differential equation dv c ൌgെ v dt m (1. a model for the free-falling motion of a parachutist was developed. programmers usually did not pay much attention to whether their programs were clear and easy to understand. To demonstrate the idea of structured programming. See Figure 1. Computer programs are set of instructions that direct the computer to perform a certain task. v୧ାଵ ൌ v୧ ൅ dv୧ Δt dt (1.

The sequence structure expresses the trivial idea that unless you direct it otherwise. Introduction to Numerical Methods Page 12 . 9. As stated before. Another approach used to express algorithms is called the pseudocode. structured programming requires effective use of the following control structures: • Sequence. 8. Sequence structure of computer programs. the computer code is to be implemented one instruction at a time. This technique uses code-like statements in place of the graphical symbols of flowcharts. Symbols used in flowcharts.Numerical Methods Figure 1. Figure 1.

Introduction to Numerical Methods Page 13 . Figure 1. 10. If there are multiple alternatives. the following selection structures can be used. Selection structure.Numerical Methods • Selection. The selection structure provides a means to split the program’s flow into branches based on the outcome of a logical condition.

Numerical Methods Figure 1.Supplementary selection or branches construct. 11. The resulting constructs are called loops. Introduction to Numerical Methods Page 14 . • Repetition. The first one is called a decision loop because it terminates based on the result of a logical condition. There are two types of loops. depending on how they are terminated. Repetition provides a means to implement instructions repeatedly. The second one is called a count-controlled loopperforms a specified number of repetitions or iterations.

Modular programming refers to the practice of dividing the computer program into modules. There are two types of procedures commonly employed: • • Functions – which return a single result. A count-controlled structure.Numerical Methods Figure 1. Aside from structured programming. designed to specifically perform a certain task. A decision loop structure. called procedures. Page 15 Introduction to Numerical Methods . Subroutines – which return several results. 12. 13. modular programming is also being practiced in the development of computer programs. Figure 1.

1. respectively. The algorithm is as follows: 1. Displays the solution.4 Develop a pseudocode for a computer program that will implement the Euler’s algorithm as applied to solving the differential equation describing the motion of the parachutist of Example 1.15 for implementation of pseudocodes in Excel and Mathscript. Drill Problems 1. Increases the ease with which a program can be debugged and tested. Microsoft Excel VBA and the Mathscript mode of LabVIEW will be used to implement the algorithms developed. Prompts the user for coefficients a. Development. Implements the quadratic formula. as well as maintenance and modification of the program is facilitated.2 1. Allows the user the option to return to step 1 and repeat the process. Example 1.3 Develop a pseudocode for a computer program that will compute for the roots of a quadratic equation ax ଶ ൅ bx ൅ c ൌ 0. b and c. 3. 2. Example 1.14 and 1. Write a well-structured pseudocode to implement the flowchart depicted in the figure below. guarding against all eventualities. See Figures 1. self-contained units makes the underlying logic easier to device and to understand for both the developer and the user.Numerical Methods Modular programming has a number of advantages: • • • The use of small. 4. In this course. Introduction to Numerical Methods Page 16 .

Write the algorithm as a well-structured pseudocode. as well as a correct output for 0!. • Set a value of tolerance (ൌ 10ିହ ). The cosine function can be evaluated by the following infinite series: xଶ xସ x଺ cos x ൌ 1 െ ൅ െ ൅ ‫ڮ‬ 2! 4! 6! Write an algorithm to implement this formula so that it computes and prints out the values of cos x as each term in the series is added. compute and print in sequence the values for cos x ൌ 1 xଶ cos x ൌ 1 െ 2 xଶ xସ cos x ൌ 1 െ ൅ 2! 4! up to the order term n of your choosing. Make sure that the algorithm includes checks on valid values.Numerical Methods 2. Write a well-structured pseudocode that will compute the factorial of an input integer. Display 0 when the input is 0 or not valid. The “divide and average” method. 5. • Implement the formula.Check whether the user input is valid. • Display the answer. Introduction to Numerical Methods Page 17 . Repeat the calculation until the answer is correct to six decimal places. In other words. Develop a well-structured function procedure that is passed a numeric grade from 0 to 100 and returns a letter grade according to the following scheme: 3. 4. It should do the following: • Allow for a user input a. an old-time method for approximating the square root of any positive number a can be formulated as ୟ x୧ ൅ ୶ ౟ x୧ାଵ ൌ 2 Write a well-structured pseudocode for the implementation of this algorithm.

Pseudocode implementation in Excel VBA.Numerical Methods Figure 1. Introduction to Numerical Methods Page 18 . 14.

Numerical Methods Figure 1. Pseudocode implementation in Mathscript. 15. Introduction to Numerical Methods Page 19 .

Quantities such as e. The errors associated with both calculations and measurements can be characterized with regard to their accuracy and precision. The concept of significant figure. there should be an assurance that it can be used with confidence. Introduction to Numerical Methods Page 20 . has been developed to formally designate the reliability of a numerical value. the computer will omit most of these significant digits which will result in round-off errors.3.Numerical Methods 1. Illustration of accuracy and precision. The concept of significant figures has two important implications for the study of numerical methods: • • Numerical methods yield approximate results. Approximations and Round-Off Errors This section covers basic topics related to the identification. The significant digits correspond to the number of certain digits plus one estimated digit. Accuracy refers to how closely a computed or measured value agrees with the true value. Therefore. there should a criteria to specify the confidence in the approximate result. π. Figure 1. quantification. Two major forms of errors are discussed in this section and the next: round-off errors is due to the fact that computers can represent only quantities with a finite number of digits. Precision refers to how closely individual computed or measured values agree with each other. or digit. 16. and minimization of these errors. and √7 has infinite number of significant digits. Whenever numbers are employed in computations. Therefore. Computers can only represent finite number of digits for a given number. Figure 1. and truncation error is the discrepancy introduced by the fact that numerical methods may employ approximations to represent exact mathematical operations and quantities.16 explains the point.

Errors are normalize to take into account the scale or magnitude of the problem being considered. If the true values are 10000 cm and 10 cm respectively. Numerical errors arise from the use of approximations to represent exact mathematical operations and quantities.6) One of the challenges of numerical methods is to determine error estimates in the absence of knowledge regarding the true value.5 The measured values of a bridge and a rivet are 9999 cm and 9 cm. These include round-off and truncation errors.5) Example 1. E୲ ൌ 1 cm. An alternative for this is to normalize the error using the best available estimate of the true value. or E୲ ൌ True െ approximate (1. Introduction to Numerical Methods Page 21 .4) where E୲ is called the true or exact error. the following relationships hold true: True value ൌ approximation ൅ error Thus. the true fractional relative error as a percentage is given as e୲ ൌ E୲ ൈ 100% True value (1. Answers: For the bridge. the true value will never be known. respectively. e୲ ൌ 0.01% For the rivet. that is eୟ ൌ approximate error ൈ 100% approximate value (1. e୲ ൌ 10% In most cases. E୲ ൌ 1 cm. That is why numerical methods employ different ways of determining the approximate error depending on the algorithms used. Thus. error is defined as the discrepancy between the true and approximated values.Numerical Methods Inaccuracy (also called bias) is defined as the systematic deviation from the true value. Numerical methods should ensure that there is no bias nor uncertainty with the results for it to be considered a valid solution. unless there is an analytic solution to the problem. compute the true error and the true percent relative error for each case. For both types. Imprecision (also called uncertainty) refers to the magnitude of the scatter of the measurements or computed values.

Therefore it is often useful to employ the absolute values of the errors and compare it with a certain tolerance. or |eୟ | ൏ eୱ (1.7) If this relationship holds. In the following example. it can be shown that eୱ ൌ ሺ0. It is also convenient to relate these errors to the number of significant figures in the approximation.648697917 which is correct to five significant figures. Thus. signs of the error may not play a role in the computation.5 The Taylor series expansion for e୶ is xଶ xଷ x୬ ൅ ൅ ‫ڮ‬൅ 2 3! n! (1. e଴. After six terms. the following table can be generated. Answer: Adding successive terms of the Taylor series expansion.ହ ൌ 1. however. Example 1.5 ൈ 10ଶି୬ ሻ% where n is the number of significant figures (at least).Numerical Methods Errors can be positive or negative. the results are assumed to be within the prespecified acceptable level eୱ . Introduction to Numerical Methods Page 22 . the Taylor series expansion for e୶ will be used to demonstrate errors. determine the value of e଴.ହ correct to three significant figures.8) e୶ ൌ 1 ൅ x ൅ Using this expansion.

the most significant bit (left-most bit) represents the sign of the number. some of these significant figures are dropped. all the bits representing the number determine the magnitude of the number.Numerical Methods Round-off Errors. 18. The base-10 number system is the number system that humans are most familiar with. they cannot precisely represent certain base-10 numbers. Round-off errors result from the following situations: • • Irrational numbers such as π. (b) signed integer? Answers: (a) ൅133. The remaining bits determine the magnitude of the number. and √7 cannot be expressed by a fixed number of significant figures. with a 0 for a positive number and 1 for a negative. or base-2 system. Computers use base-2 number representation. Weights of each binary bit position for a signed 8-bit binary number. Integers may be in signed or unsigned magnitude format. Figure 1. since computers are built on devices that acquires only two states (on / off). computers represent numbers using a binary. If the format is signed.6 What is the number 10000101 (base-2) in decimal if it is written as (a) unsigned integer. However. The discrepancy introduced by this omission of significant figures is called round-off error. Figure 1. hence. Numbers in computers are represented in two forms: integer and floating-point representation. Example 1. e. (b) െ123 Introduction to Numerical Methods Page 23 . Weights of each binary bit position for an unsigned 8-bit binary number. Computer Representation of Numbers. If the format is unsigned. 17.

• • Single precision floating-point numbers. Answers: Example 1. Introduction to Numerical Methods Page 24 . as in m ൈ bୡ . 19.Numerical Methods On the other hand. fractional quantities are typically expressed in computers using floating-point form. where m is the mantissa. the next three for the sign and magnitude of the exponent. and the last three for the magnitude of the mantissa. The mantissa holds a value that is 1 ൑m൑1 b Thus. 11-bit biased exponent and a 53-bit mantissa (but only 52 bits are presented). In this standard.8 Create a hypothetical floating-point number set for a machine that stores information using 7-bit words. which uses 64 bits. Example 1. has one sign bit. IEEE Standard 754 defines the standard for representing floating-point numbers. Double precision floating-point numbers. For computer systems. In this approach. for a base-10 number. has one sign bit. called a mantissa or significand.7 Represent the decimal numbers ൅12 and െ0. 8-bit biased exponent and 24-bit mantissa (but only 23 bits are presented). and an integer part. b is the base of the number system being used. the number is expressed as a fractional part. Number representation using floating-point.1 and 1. Figure 1. Employ the first bit for the sign of the number. and c is the exponent.75 into single precision floating-point numbers. called an exponent or characteristic. which uses 32 bits. the mantissa is between 0.

Figure 1. these computations are often Page 25 • • Introduction to Numerical Methods . Most computers hold intermediate results in a double-length register (since multiplication will yield results double the number of digits). the mantissas of the numbers are multiplied (divided) and the exponents added (or subtracted). There are only a finite number of quantities that can be represented within the range. the sign of the subtrahend is complemented.Numerical Methods The following are some important points about floating-point numbers that are significant in relation to computer round-off errors as observed from Example 1. The results are then chopped. There is a limited range of quantities that may be represented.8: 1. Arithmetic Manipulations of Computer Numbers. Aside from the limitations of a computer’s number system. the mantissa of the number with the smaller exponent is modified so that the exponents are the same. Multiplication and division – when two floating-point numbers are multiplied (or divided). 3. 20. The results are then added and the results are chopped off to the specified number of mantissa digits. Problems associated with floating-point numbers. • Addition and subtraction – when two floating-point numbers are added or subtracted. The interval between numbers increases as the numbers grow in magnitude. the actual arithmetic manipulations involving these numbers can also result in round-off error. For subtraction. Large computations – Certain methods require extremely large numbers of arithmetic manipulations to arrive at their final results. In addition. 2.

Evaluate this infinite series in single and ଽ଴ double precision to calculate fሺnሻ for n ൌ 10000 by computing the sum from k ൌ 1 to 10000 and in reverse order. Evaluate eିହ using two approaches xଶ xଷ െ ൅‫ڮ‬ 2 3! 1 ൌ 1 ൅ x ൅ xଶ ൅ xଷ ൅ ‫ڮ‬ 1െx eି୶ ൌ 1 െ x ൅ and eି୶ ൌ 1 1 ൌ మ ୶ ୶ ୶య e 1 ൅ x ൅ ଶ ൅ ଷ! ൅ ‫ڮ‬ and compare with the true value of 6. Which function is better in terms of resisting loss of significance? Introduction to Numerical Methods Page 26 . Use 20 terms to evaluate the series and compute the true and relative errors as terms are added. Evaluate the true relative error and determine the cause of the discrepancy between the solutions.ସ . Drill Problems 1.001 and xଶ ൌ െ3000.001. Also evaluate the true and approximate relative errors for each case. How many terms are required to achieve such accuracy? 2. graph the function in the given interval. Consequently.737947 ൈ 10ିଷ . b ൌ 3000. The true roots for the equation are xଵ ൌ െ0. 3. evaluate this series correct to seven significant figures. 5. Determine the roots of a quadratic equation having a ൌ 1.3 1. If |x| ൏ 1 it is known that For x ൌ 0.1. For 100 equally spaced values of x between 10ିଽ to 10ି଻.Numerical Methods interdependent. The infinite series ୬ fሺnሻ ൌ ෍ ୩ୀଵ ஠ర 1 kସ converges on a value of fሺnሻ ൌ as n approaches infinity. evaluate the expressions y ൌ √x ൅ 4 െ √x ൅ 3 and yൌ 1 √x ൅ 4 ൅ √x ൅ 3 Note that both expressions are mathematically equivalent. the cumulative effect over the course of a large computation can be significant. Use single and double precision arithmetic. Using plotting tools. and c ൌ 3. 4. even though an individual round-off error could be small.

9) where the remainder R ୬ is defined as R୬ ൌ න ଴ x െ tሻ୬ ሺ୬ାଵሻ ሺtሻ dt f n! (1.12) Indicates that the value of the f at the new point is the same as its value at the old point.21 depicts how Taylor series approximates the function. fሺx୧ାଵ ሻ ؆ fሺx୧ ሻ (1. fሺx୧ାଵ ሻ ؆ fሺx୧ ሻ ൅ f ᇱ ሺxሻሺx୧ାଵ െ x୧ ሻ takes the value of f closer to the true values. Taylor’s theorem states that any smooth function can be approximated by a polynomial. In order to gain insight into the properties of such errors. involves the first term of the series. Figure 1.9 is called the Taylor series expansion of fሺxሻ. The zero-order approximation. takes the average value fሺξሻ. Truncation Errors and the Taylor Series Truncation errors are those that result from using an approximation in place of an exact mathematical procedure. In particular. that is. if the function f and its first n ൅ 1 derivatives are continuous on an interval containing a and x. However. The remainder R ୬ has another form called the derivative or the Lagrange form of the remainder. Taylor Series. The Taylor series provides a means to predict a function value at one point in terms of the function value and its derivatives at another point. If the remainder R ୬ is omitted. Equation 1.13) . then the value of the function at x is given by fሺxሻ ൌ fሺaሻ ൅ f ᇱ ሺaሻሺx െ aሻ ൅ f ᇱᇱ ሺaሻ f ሺଷሻ ሺaሻ f ሺ୬ሻ ሺaሻ ሺx െ aሻଶ ൅ ሺx െ aሻଷ ൅ ‫ ڮ‬൅ ሺx െ aሻ୬ ൅ R ୬ 2! 3! n! ୶ሺ (1. the Taylor series will be used.10) Equation 1.Numerical Methods 1. better approximation will result if more terms of the Taylor series are added. The Taylor’s theorem states that.4. Subsequent addition of terms. which can be derived from the integral mean-value theorem. the first-order approximation. Introduction to Numerical Methods Page 27 (1.11) where ξ is a point between the interval a and x such that the function fሺxሻ.9 is the Taylor polynomial approximation to fሺxሻ. R୬ ൌ f ሺ୬ାଵሻ ሺξሻ ሺx െ aሻ୬ାଵ ሺn ൅ 1ሻ! (1.

16) where R ୬ is the remainder term that accounts for all the terms above n up to infinity.17) Introduction to Numerical Methods Page 28 . Taylor series approximation. 21.15) and so on results in better approximations.16 as f ᇱᇱ ሺx୧ ሻ ଶ f ᇱᇱᇱ ሺx୧ ሻ ଷ f ሺ୬ሻ ሺx୧ ሻ ୬ fሺx୧ାଵ ሻ ൌ fሺx୧ ሻ ൅ f xሻh ൅ h ൅ h ൅ ‫ڮ‬൅ h ൅ R୬ 2! 3! n! ᇱሺ (1.14) fሺx୧ାଵ ሻ ؆ fሺx୧ ሻ ൅ f ᇱ ሺxሻሺx୧ାଵ െ x୧ ሻ ൅ f ᇱᇱ ሺx୧ ሻ f ᇱᇱᇱ ሺx୧ ሻ ሺx୧ାଵ െ x୧ ሻଶ ൅ ሺx୧ାଵ െ x୧ ሻଷ 2! 3! (1.Numerical Methods fሺx୧ାଵ ሻ ؆ fሺx୧ ሻ ൅ f ᇱ ሺxሻሺx୧ାଵ െ x୧ ሻ ൅ f ᇱᇱ ሺx୧ ሻ ሺx୧ାଵ െ x୧ ሻଶ 2! (1. Figure 1. It is often convenient to simplify the Taylor series by defining a step size h ൌ x୧ାଵ െ x୧ and expressing 1. resulting in the complete Taylor series fሺx୧ାଵ ሻ ൌ fሺx୧ ሻ ൅ f ᇱ ሺxሻሺx୧ାଵ െ x୧ ሻ ൅ f ᇱᇱ ሺx୧ ሻ f ᇱᇱᇱ ሺx୧ ሻ ሺx୧ାଵ െ x୧ ሻଶ ൅ ሺx୧ାଵ െ x୧ ሻଷ ൅ ‫ڮ‬ 2! 3! f ሺ୬ሻ ሺx୧ ሻ ሺx୧ାଵ െ x୧ ሻ୬ ൅ R ୬ ൅ n! (1.

such as exponentials and sinusoids.5x ଶ െ 0.Numerical Methods where the remainder term is now expressed as f ሺ୬ାଵሻ ሺξሻ ୬ାଵ R୬ ൌ h ሺn ൅ 1ሻ! (1.9 Use zero. However.15x ଷ െ 0. The ሺn ൅ 1ሻth derivative of fሺxሻ must be known.19) Introduction to Numerical Methods Page 29 . in the form of the remainder term R ୬ . the n th-order Taylor series approximation yields the exact value of the polynomial at the point. The assessment of how many terms are required to get “close enough” is based on the remainder term of the expansion.2 Note that the true value of the function at x ൌ 1 is 0.3 Fourth-order approximation: fሺ1ሻ ؆ 0. the practical value of the Taylor series expansions is that.2.2 from x୧ ൌ 0 with h ൌ 1.95 Second-order approximation: fሺ1ሻ ؆ 0.18) Example 1. Answers: Zero-order approximation : fሺ1ሻ ؆ 1.2 First-order approximation fሺ1ሻ ؆ 0. the inclusion of only a few terms will result in an approximation that is close enough to the true value for practical purposes. but fሺxሻ is basically unknown.through fourth-order Taylor series expansions to approximate the function fሺxሻ ൌ െ0. because h can be chosen and hence the number of terms to be included in the expansion can be controlled. or R ୬ ൌ Oሺh୬ାଵ ሻ (1.1x ସ െ 0. that is. Thus. the fourth-order approximation is the true value. Despite this.18 has two major drawbacks: • • The number ξ is not known exactly but merely lies between x୧ and x୧ାଵ . in most cases.45 Third-order approximation: fሺ1ሻ ؆ 0. It can be shown that the error is proportional to the step size h raised to the ሺn ൅ 1ሻth power.25x ൅ 1. For other differentiable and continuous functions. For an nth degree polynomial. Although the above is true. a finite number of term will not yield an exact result. truncation errors are incurred. the relationship of 1. the remainder term is still useful for gaining insight into truncation errors.

Introduction to Numerical Methods Page 30 . the remainder term R ୬ is R ଴ ൌ f ᇱ ሺx ୧ ሻh ൅ f ᇱᇱ ሺx୧ ሻ ଶ f ᇱᇱᇱ ሺx୧ ሻ ଷ h ൅ h ൅‫ڮ‬ 2 3! (1. ஠ Answer: The table below gives subsequent results of computations.22. on the basis of the value of fሺxሻ and its derivatives at x୧ ൌ ସ .10 ஠ Use Taylor series expansions with n ൌ 0 to 6 to approximate fሺxሻ ൌ cos x at x୧ାଵ ൌ ଷ . 22. The Remainder Term For the Taylor Series Expansion. If the Taylor series expansion was truncated after the zero-order approximation.20) A graphical depiction of such truncation is shown in Figure 1. Figure 1.Numerical Methods Example 1. Graphical interpretation of zero-order truncation.

the first-order remainder term as f ᇱᇱ ሺξሻ ଶ Rଵ ൌ h 2 (1.22) Using the Taylor Series To Estimate Truncation Errors.21) The higher-order remainder term can be derived in the same manner.2). for example. According to the derivative mean-value theorem. then there exists a point on the function that has a slope.Numerical Methods Note that in this truncation. The point ξ marks the x value where this slope occurs. if a function fሺxሻ and its first derivative are continuous over an interval from x୧ to x୧ାଵ .1 (and hence reinforce the idea of the Euler’s method presented in Example 1. 23. with this theorem can be written as R ଴ ൌ f ᇱ ሺξሻh (1. although the results are still inexact because the secondand higher-order terms are neglected. that is parallel to the line joining fሺx୧ ሻ and fሺx୧ାଵ ሻ. Figure 1.23) Introduction to Numerical Methods Page 31 . the lower-order derivatives usually account for the greater share in the remainder term than the higher-order derivatives. The Taylor series expansion of the velocity as a function of time gives vሺt ୧ାଵ ሻ ൌ vሺt ୧ ሻ ൅ v ᇱ ሺt ୧ ሻ ‫ ڄ‬ሺt ୧ାଵ െ t ୧ ሻ ൅ v ᇱᇱ ሺt ୧ ሻ ‫ ڄ‬ሺt ୧ାଵ െ t ୧ ሻଶ ൅ ‫ ڮ‬൅ R ୬ 2 (1. Graphical interpretation of the derivative mean-value theorem. Thus the zero-order remainder term. The Taylor series approach can be used to numerically solve the problem of Example 1. f ᇱ ሺξሻ.

27) (1. Also. Note that as m increases. Thus.25 is called the finite divided difference and can be generally represented as f ᇱ ሺx ୧ ሻ ൌ or f ᇱ ሺx ୧ ሻ ൌ Δf୧ ൅ O ሺh ሻ h (1. vሺt ୧ାଵ ሻ ൌ vሺt ୧ ሻ ൅ v ᇱ ሺt ୧ ሻ ‫ ڄ‬ሺt ୧ାଵ െ t ୧ ሻ ൅ Rଵ (1.29) fሺx୧ାଵ ሻ െ fሺx୧ ሻ ൅ Oሺx୧ାଵ െ x୧ ሻ x୧ାଵ െ x୧ (1. say m ൌ 4. is Rଵ v ᇱᇱ ሺξሻ ሺt ୧ାଵ െ t ୧ ሻ ൌ t ୧ାଵ െ t ୧ 2 or Rଵ ൌ Oሺt ୧ାଵ െ t ୧ ሻ t ୧ାଵ െ t ୧ which implies that the error is a function of the step size h ൌ t ୧ାଵ െ t ୧ . The others are called backward and centered difference approximations. which.11 Investigate the effect of increasing the value of m for the function fሺxሻ ൌ x ୫ to the truncation error when Taylor series approximation is used to estimate the function.25 depicts the first-order approximation term while the second one is the truncation error. Example 1.22.28) (1. for a particular value of m.Numerical Methods Truncating the series after the first derivative. the function becomes more nonlinear.26) where Δf୧ is called the first forward difference and h is called the step size. the algorithm used to estimate the derivative of v ᇱ ሺt ୧ ሻ before now has an estimate of the truncation error associated with this approximation. Moreover. This is one of many numerical methods developed from the Taylor series to approximate derivatives numerically.25) where the first term on the right of 1.24) with which v ᇱ ሺt ୧ ሻ ൌ vሺt ୧ାଵ ሻ െ vሺt ୧ ሻ Rଵ െ t ୧ାଵ െ t ୧ t ୧ାଵ െ t ୧ (1. according to 1. Note that Equation 1. observe how the first-order truncation error changes as the step size is varied. more accurate approximations of the first Introduction to Numerical Methods Page 32 .

30) Truncating this equation after the first derivative and rearranging yields f ᇱ ሺx ୧ ሻ ൌ fሺx୧ ሻ െ fሺx୧ିଵ ሻ ‫׏‬fଵ ൅ O ሺh ሻ ൌ ൅ O ሺh ሻ h h (1. The Taylor series can be expanded backward to calculate a previous value on the basis of the present value. third. (b) backward. Finally. 24. as in fሺx୧ିଵ ሻ ൌ fሺx୧ ሻ െ f ᇱ ሺx୧ ሻ ‫ ڄ‬h ൅ f ᇱᇱ ሺx୧ ሻ ଶ ‫ڄ‬h െ‫ڮ‬ 2 (1. Evaluation of first derivative using (a) forward. Figure 1.Numerical Methods derivative can be developed by using higher-order terms of the Taylor series. and higher derivatives. all the above versions can also be developed for second. and (c) centered finite difference.31) where the error Oሺhሻ and ‫׏‬fଵ is called the first backward difference Introduction to Numerical Methods Page 33 .

The total numerical error is the summation of the truncation and round-off errors.5 Forward difference Backward difference Centered difference (b) For h ൌ 0. Introduction to Numerical Methods Page 34 .6% Total Numerical Error. the following are observed: • • the only way to minimize round-off error is to increase the number of significant figures of the computer.Numerical Methods A third way to approximate the first derivative is to subtract 1. Answers: (a) For h ൌ 0.5 using a step size (a) h ൌ 0.12 Use forward and backward difference approximation of Oሺhሻ and a centered difference approximation of Oሺhଶ ሻ to estimate the first derivative of fሺxሻ ൌ െ0.5 and (b) h ൌ 0.25 Forward difference Backward difference Centered difference True error 26.1x ସ െ 0.4% True error 58.9% 39.15x ଷ െ 0. Compare the true error in both cases.7% 2.5x ଶ െ 0.2 at x ൌ 0.32) Example 1.25.31 from the forward Taylor series expansion which will give the centered difference representation of the first derivative f ᇱ ሺx ୧ ሻ ൌ fሺx୧ାଵ ሻ െ fሺx୧ିଵ ሻ െ O ሺh ଶ ሻ 2h (1. truncation error can be reduced by decreasing the step size. From the previous discussions.5% 21.25x ൅ 1.7% 9.