Chapter 3 Numerical calculus Calculus is at the heart of describing physical phenomena. As soon as we talk bout motion, we must invoke differentiation and integration, For example, the velocity and the acceleration of a particle are the first-order and second-order time derivatives of the corresponding position vector, and the distance traveled by a particle is the integral ofthe corresponding speed over the elapsed time. In this chapter, we introduce some basic computational methods for dealing \ith numerical differentiation and integration, and numerical schemes for search- ing for the roots of an equation and the extremes of function, We stay at a basic level, using the simple but practical schemes frequently employed in computa- tional physics and scientific computing. Some of the subjects will be reexamined later in other chapters to a greater depth. Some problems introduced here, such as searching for the global minimum or maximum of a multivariable function, are considered unsolved and are still under intensive research, Several interesting examples are given to illustrate how to apply these methods in studying important problems in physics and related fields. 3.1 Numerical differentiation One basie tool that we will often use in this book is the Taylor expansion of a function f(x) around a point xo: we 2 20 ay) boot sx) 4 BD FE) = feo) + (x = 40) Fa) + ‘The above expansion can be generalized to describe a multivariable function Je, y,...) around the point (x0...) PY.) = Sra. 300 IE Ha) flo, 010) (x0) a FO - WF b00. 30+ Fea Yor.) O30 is = wy 9) OP C8020 RIO Oso ame) 62) 4950 Numerical calculus where the subscript indices denote partial derivatives, for example, fay = #yfaray The first-order derivative ofa single-variable function f(x) around a point x; is defined from the limit fist Ax) = fxd FG) = jim, Ar G3) ifit exists, Now if we divide the space int diserete points x, with evenly spaced intervals x01 — x; = A and label the function at the lattice points as fi = fs). wwe obtain the simplest expression forthe first-order derivative f= Ft sow, 6a) i ‘We have used the notation O(h) for a term on the order off. Similar notation will be used throughout this book. The above formula is referred to as the wo- ‘point formula for the first-order derivative and can easily be derived by taking the Taylor expansion of f.1 around x,. The accuracy can be improved if we expand fis and fi around x; and take the difference Sis ~ finn = 2hfi + OO) es) ‘After a simple rearrangement, we have fi +00) 66) 7 whichis commonly known as the three-point formula forthe first-order derivative ‘The accuracy of the expression increases toa higher oder inh if more point are used. For example, a five-point formula can be derived by including the expan- sions of fis2 and 2 around x, If we use the combinations fan fama 2 Pou oo and far ae to cancel the f°? terms, we have Tyna Bhics 4 8figs 8» ‘We can, of course, make the accuracy even higher by including more points, but inmany cases this is not good practice. For real problems, the derivatives at points close to the boundaries are important and need to be calculated accurately. The errors in the derivatives of the boundary points will accumulate in other points ‘when the scheme is used to integrate an equation, The more points involved in the expressions of the derivatives, the more difficulties we encounter in obtaining accurate derivatives at the boundaries. Another way to increase the accuracy is by decreasing the interval , This is very practical on vector computers. The algorithms for first-order or second-order derivatives arc usually fully vectorized, s0.a vector processor can calculate many points in just one computer clock eyele.3.1 Numerical differentiation Approximate expressions forthe second-order derivative can be obtained with different combinations of f,.The three-point formula forthe second-order deriva- tive is given by the combination fia - 2h + fin ‘withthe Taylor expansions of fis: around x;..Note that the third-order term with FP vanishes because of the cancellation in the combination. The above equation ives the second-order derivative as 7 Similarly, we can combine the expansions of fiy2 and fi4; around x; and f; to cancel the f, /2?, ff, and /-° terms; then we have 1 ar as the five-point formula for the second-order derivative. The difficulty in dealing with the points around the boundaries still remains, We can use the interpolation formulas that we developed inthe Chapter 2 to extrapolate the derivatives to the ‘boundary points. The following program shows an example of calculating the first-order and second-order derivatives with the three-point formulas. of + 006) 6.19) fe + 00k, en fia +16 fins =30fi + 16fisr = fis) + OOH) B.12) 11 Ao example of evaluating the derivatives with the static final int n= 100, m public static vold main(String axgv(I) ¢ Gouble(] x = new double{n+i]7 doubie(} £ = new double{neal; double(} £1 = new double(net] Gouble(] £2 = new double(ntt] 11 nasign constants, data points, and function double & = Math.Pr/(2%m)7 for (int ind; Leens ++4) xi 11 calouiat eOrderDerivative(h, f, K) condOrderberivative(h, £, 11 output the result in every m data points for (int 420; Leen; teem) { double afi = £1(1)-math-cos(e(1): double af2 = £2{i}+nath.sin(x(t) Syetem.out.printia( =" + x(4] Syetem.out -printia(te'Ge =" + SCN) System.out-printia(*Erzor in f(x): * + £1); system.out-printin(*f''(x) = * + £2041); System out -printia(*ercor in £1¢ Ge): "+ 462)7 5152 Numerical calculus Syatem.out-printia(); > > 1/ Method for the lst-order derivative with the 2-point 1/ formula. Extrapolations are made at the boundaris public state double[] flretorderberivative double hy double £(1, int &) ( int a= f-leagth-1; doublet] y = new double(aetly doublet] x1 = new doubletk+1]; doublet} £1 = new doubletke1} Goublet] fr = new doubletk+t]; 1 Bvaluate the derivative at nonboundary points for (int tet; deny +44) yO) = (eCbead “£01128 11 tagrange-extzapolate the boundary pointe for (int int; Len(ken); +44) ( salina} = bey su-a) = yttly gel-a) = yia-tiy > yO) = aderen(0, x21, £107 yial = aiteen(0, 32, f2)7 > 11 Method for the 2nd-order derivative with the 3-point 11 formla, Extrapolations are made at the boundaris public static double(] secondorderDerivative(double h, Goublet] £, int x) { Ant a= E.length-ty double) y = new double(net] Gouble(} x1 = new double(kei]: Gouble(] £1 = new double(k+a] Gouble(] x = new double(k+2]; 1/ wvatuate the derivative at noaboundary points for (int det; Leap +41) ( yUS) = (£C462]-248 [4] 4£14-219 /(048) 7 > xtrapolate the boundary points for (int tet; Len(men); +44) sa t-ay = hey ada) = ytd) feelin} = ytmils > y{0] » aivken(0, x, £1)7 yin] = aiteen(0, x1, fr)7 > public static double aitken(double x, double x11, double £110) (+4) 1 tagrangs3.1 Numerical differentiation Table 3.1, Derivatives obtained in the example x £ ar £ af ° 0.999959 0,000 04 0.000008 0.000004 aplo 0.951017 0.000039 =0,309087 0.000070 ais o.808 98s 0.000032 0.587736 0.000049 3/10 0.587 762 0.000023 =0309013 0.000008 2n/S 0309003 0.000014 0,951 055 0.000001 afd 0.000004 0.000004 0.999 980 0.000020 We have taken a simple function f(r) = sin.x, given at 101 diserete points with evenly spaced intervals in the region (0, 1/2]. The Lagrange interpolation is applied to extrapolate the derivatives at the boundary points. The numerical results are summarized in Table 3.1, together with their errors. Note that the extrapolated data are of the same order of accuracy as other calculated values for f’ and f” at both x =0 and x = x/2 because the three-point Lagrange interpolation scheme is accurate to a quadratic behavior. The functions sinx and cos ate well approximated by a linear or a quadratic curve at those two points In practice, we may encounter two problems with the formulas used above, The first problem is that we may not have the data given at uniform data points. One solution to such a problem is to perform an interpolation of the data first and then apply the above formulas to the function at the uniform data points generated from the interpolation. This approach can be tedious and has errors from two sources, the interpolation and the formulas above. The easiest solution to the problem is to adopt formulas that are suitable for nonuniform data points. Ife use the Taylor expansion Ses Goer mF") +00) G13) = fo) +01 OF) + and a combination of fi-1, fi, and fis: to cancel the second-order terms, we obtain That th) row. es) whore hy = x41 —x) and fis the larger of Yj-i| and [yl This is the three- point formula forthe first-order derivative inthe ease of nonuniform data points Note that the accuracy here is the same as for the uniform data points. This is a better choice than interpolating the data first because the formula can be implemented in almost the same manner as in the case of the uniform data points, The following method returns the first-order derivative for such @ situation 5354 Numerical calculus 11 Wathod to calculate the 1st-order derivative with the 17 nonuniform 3-point forma. Extrapolations are made 17 st the boundaries. public static double[] sirstorderberivative (Gouble (1, double (1, ine ®) ( Ant a= x-lesgth-1; doublet} y = new doubletned}; Gouble(] x1 = new doubleteen]y doublet) £1 = new doublets Gouble() xr = new double (ket Gouble(] fr = new doubletk+t]; 11 caloulate the derivative at the field points Gouble RO = x(2]-x(017 double ad = h0*n0; for (int Set; fens 444) ¢ double B= x(4+21-x1417 double B Gouble c= heho* (m+n); YUL = (aO*e(dea}4ote(4)-a¥e(4-23)/07 aoa > 11 tagrange-extrapolate the boundary points for (int int; ten(ked)) #63) { zati-1) = xt6)-xt017 Ata} = vis we(i-a) = xtad-xta-t1 fete} = ytmefly > y(0] * aieken(0, 92, £1); yin] = aitnen(0, xx, fr)? > public static double aitken(double x, double x11, double £110) (++) ‘The corresponding three-point formula for the second-order derivative ean be derived with a combination of f-1, fi, and fi+1 to have the first-order terms in the Taylor expansion removed, and we have Ahi fi ~ (he 4 aif hh jv Tai + mad Note tha the accuracy here is an order lower than inthe ease of uniform data points ‘because the third-order terms in the Taylor expansion are not canceled completely atthe same time. However, the reality is that the third-order terms are partially canceled depending on how close the data points are to being uniform. We need to be careful in applying the three-point formula above. If higher accuracy is needed, we can resort toa corresponding five-point formula instead. The second problem is that the accuracy cannot be controlled during evalua- tion with the three-point or five-point formulas. Ifthe function f(x) is available Oth, aus)3.1 Numerical differentiation 55 continuously, that is, for any given x in the region of interest, the accuracy in the ‘numerical evaluation of the derivatives can be systematically improved to any desired level through an adaptive scheme. The basic strategy here is to apply the combination of fx +h)= fix =H) A = G18) and A(h/2) to remove the next nonzero term in the Taylor expansion and to repeat this process over and over. For example, Au(h) = 4068/2) = -3 f°) + O00") G17) Then we ean use the difference inthe evaluations of f"(x) from A,(h) and Ax(/2) to set up a rough criterion for the desired accuracy. The following program is such an implementation, 11 Ae example of evaluating 1st-order derivative through 11 the adaptive scheme. public class beriv3 { Public static void main(String argvil) ¢ double del = 10-6; Ant n= 10, m= 107 Gouble B= Math.PI/(2*m)7 // evatuate the derivative and output the remult for (int Leo; Loony +41) double x = bei; Gouble a = (£(ceB)-€(-8)) /(29007 double £1 = fizetorderDerivatives(x, hy a, del, ky mir double afi = f1-math.cos(x); Syetem.out.printin(t =" + 307 Syetem.out -printia(e'(e) =" + £1); System.out-printia(*Erzor in f(x): " + aft); carry out Int-order derivative through the 11 aaapeive public static double firstorderbertvatives (double *, double b, double 4, double del, int step, Ant maxstep) ( Bena double a1 = (£(een)-£ (en) 1/ (20m) Af (step >= maxaten) ( System.out.printia (*Not converged after * return 41) > e Lf ((u*heMath.abe(d-a1)) < del) return (4ta-a)/3756 Numerical calculus return flrstorderDerivatives(x, hy 41, dol, step, maxstep) > > public state double £(doubie x) ¢ return Math.sin(x); > > Running the above program we obtain the first-order derivative of the function accurate to the order of the tolerance set in the program. In fact, we can analytically work out a recursion by combining more A,(h/2*) for k = 0, 1...... For example, using the combination Ay(h) — 204 (h/2) + 6464/4) = 45") + OOH, G8) We cancel the fourth-order terms in the Taylor expansion. Note that the fifth- order term is also canceled automatically because of the symmetry in A(H/2") This process can be repeated analytically; this scheme is called the Richardson extrapolation. The derivation and implementation of the Richardson extrapolation are left as an exercise for the reader, A similar adaptive scheme can be devised forthe second-order derivative. For ‘example, if we define the function fO+M) ~2fto) + fe-h) 20) = Ath) = G19) and then use A2(h) and A3(h/2) to remove the next nonzero terms in the Taylor expansion, we have Aalh) — 48x(8/2) = 3") + O08"). 20) Based on the above equation and further addition ofthe terms involving A2(h/2*), ‘we can come up with exactly the same adaptive scheme and the Richardson extrapolation for the second-order derivative. 3.2 Numerical integration Let us turn to numerical integration. In general, we want to obtain the numerical value of an integral, defined in the region (a, 6], [roe oan We can divide the region (a, 5] into n slices, evenly spaced with an interval h. If ‘we label the data points as x, withé = 0, 1, ....n,we ean write the entire integral asa summation of integrals, with each over an individual slice, [roa = rr "pends 22)