666) IPH POjUel Numerical Method: Using MATLAB Third Edition John H. Mathews: California State Universi Fullerton Kurtis D. Fink Northwest Missoun State Universit Hagskoten i Vestioic Bubliotwket - Borre Prentice Hall, Upper Saddle River, NI O7458 Contents 2a 22 23 24 25 ul Preface vii Preliminaries Review of Calculus 2 Binary Numbers 13 ror Analysis 24 ‘The Solution of Nonlinear Equations fx=0 40 eration for Solving x = g(x) 4 Bracketing Methods for Locating a Roct $1 Initia Approsimation and Convergeme Ctiteia 02 Newion-Raphson and Scan Meshads 70 ‘Althe’s Process and Stffensen's and Malie’s Metiogs (Uptoraly 0 The Solution of Linear Systems AX =B 101 Invevvction ‘a Veer and Maircns 207costes 32 Properies of Vectors and Matrices /09 35 UpperiangularLinet Systems 120 3A Gauisian Elimination and Photing 125 35 ThangularFactoriason [47 36 Meratve Methods for Linear Systems 150 7 teraunn tor Nontisear Systems; Seidel and Neoton's Method (Options!) 167 Yoterpolation and Polynomial Approximation 186 4.1 Tavlor Series und Caleulation of Functions 187 42 Inodetion to Interpolation 199 43 Lagrange Approximation 206, 4 Newioe Polymouials 229 45 Chebyshev Potwmomsials(Ontional) 230 4 Padé Approximations 243 Curve Fitting 252 5.1 Leaseaquares Lice 253 52 CurveFitung 263 53 Interpolation by Spine Funstions 279 $4 ounerSeries and THgonomeisc Polynomials 297 Numerical Differentiation 370 61 Approximating The Desvatve 317 62 Numerica Differentiation Formulas 329 Numerical Integration 342 11 nraduction Quadrature 543 12 Composite Tapezoidl ana Simpsoe's Rule 354 73 Recunive Rules and Romberg Integration 268 TA Acaptive Quadrature 382 75 Gauss-Legendee Integration (Optional) 389 contests 8 81 on 92. 9a 94 95 vo 98 99 he 2 to 102 103 11 a U2 3 ne Numerical Opti Minimization of ¢ Funesion 400 Solution of Differential Equations 426 Invoduetion o Differential Equations 427 Evier's Method 423 Heun's Method 443 Taylor Series Method 450 Runge-Kutta Methods 458 redictor-Corrector Methods 474 Systems of Differertia! Eqeations 487 Boundary Velue Problems 497 Finite-tiffereace Method 505 Soiution of Partial Differential Kquations = 5/4 Hypesboiic Equations 516 Parabolic Equasions 526 Elliptic Equations 53 igenvalues and Eigenvectors 555, Homogeneous Systems: The Bigenvalue Problem $56 Power Method 568 Jaeobi's Methed 987 Figenvaluecif Symmetric Matriows S04 Appendix: An Introduction to MATLAB 608 Some Suggested References for Reports 616 Bibliography and References 619 Answers to Selected Exercises 631 Index 655Preface ‘This book srovides a fundamental introduction to numerica: analvsis suitable for ur Other exits 23 ~ 0 ch tha, whenever x = $,0-~ br zal <3 implies ht |f(2) - LI < +. When the h-increment notation x — 29 + his used, equation (1 Becomes @ Jim flan +h) = b a Ske. 1.1 REVIRW OF CALCULK'S 4 Definition 1.2. Assume that f(x) is defined on 4 set 8 of re nurobers and ley € S Then f is sid 1 be continuous af x = x0 if % li £00) = Foo. ‘the function fis said w be continuous on $ if itis continuous at each point x € 5. ‘The novaion C7¢S) stats fore set of all functions such that f and ts nest n derivatives are continuous on S. When S is an interval, say [a,b then the notation (C*la.6] is used, As an example, consider the funetion f(x) = x" on the inter: val {21,1}. Clearly, fle) and f"(2) — (4/22 are continuous ont FH, i], while F°(8) = 4/97 not comtinuous atx = 0. a Definition 1.3. Suppose that (x9) said to have the limit. and we write is an infinite sequence, ‘Then the sequeue ie m lima Mt gre any € > 0, Were exists a postive mleger A = (e) such that > A implies that Ix. — Ll <€ a When a sequence has 4 init, we say that it is a convergent sequence. Another commonly used notation is "x— > Las n > 20.” Equation (4) is equivalent 10 6) imn L) ‘Thus we can view the sequence {én}. = (% ~ L)yuy 88 an error sequence. The folowing theorem relales the concepts of continuity and convergeat sequence. ‘Theorem 1, Assume that f(x) is defined on the Set 9 and xp € 8. Une following statements are equivalemt: (@) The function Fis comtinvous atx © COE fm xy — x9, then lien Frq) = Fx ‘Theorem 1.2 (Intermediate Value Theorem), Assume that f € Cla, df and £16 any aumber between f(a) and f(b). Then theve exits number, with © « (a, ‘neh that fe) Example Lt. The function f(4) ~ costx— 1) ws cominsous over [0,1], nd the constant L = 08 € (cox(0), cost). The solution to f(x) = 0.8 over [0 1} c1 = 0.356499. Simutary, f(a) ic comuinuous over(1,2.5} and Z = O38 € (c06(2.8),20s(1}). The sottion to f(x) =08 over [1,25] iscz — 11643802. These ewo eases are shown in Higure 1.2. =4 Cuan 1 Preuinananses Figure 1.2. The intermediat: va einen applied to the fetion f(a) = conte ~ 1) over (0.1) and ‘ver the Interval {1,23 a wfasay » 6) 2» 0 ee Fey) Wigore 1.3. The extreme viv i theorem applied 0 the fant 5 fi) a3 459.5 O65" 00 05 10 15 20 25 30 averse imenal 0.31 ‘Theorem 1.3 (Extreme Value Theorem for a Continuous Function), Assume that F © Cla,b). Then there exists « lower bound Mj, an upper bound Mz, and two numbers xy. € fe, BP suck that a M Fs) = 08) = fle) — Mz whenever © La, OL We sometimes expres thie by writing 8 My= Fed = ming FOO) and Ma F0r2) = pees LO) Differentiable Functions Definition 14. Assume that f(x) is defined on an open interval containing xp, Then {is said tobe differentiable at xp if oO Jimn, £@. tt, Se Sec. 1.1 REVIEW OF CaLcttUS 5 ‘existe, When this limit exist, iti denoted by f(x) and is called the derivative of f aro, An equivalent way fo express tis imi :s 10 use the h-increment notation: tim L29+M) = fo a 7% any = St ‘A faut that has evivaive at ca puint ina se: 5 iy sad co be alifferencdable ‘on S, Note that, the number mi = {"(rn) is the slope of the targent line to the graph of the fmction y = f(x) atthe point (x9, f(21). a ‘Theorem LA. If /1x) is differentiable at 1k follows from Theorem 1.3 that, if a function is differentile on a closed tnterva! Ja, 0), nen as exteme values occur at te en points ofthe interval ar at the ‘rtical points (solutions of #"(x) =D) ia the open interval (a,b xo then £2) is continuous atx Example 12. The function fe) = US? 65.59 1 $9.85 | 38s ditereniableon (0.31 The solutions wo f(x) = 45e? ~ 18x + $9.5 = Oare x) = 0.54985 and xy = 24060 "he maxsmear ai mioimum values of J on (0.31 re ming 70h F9) TOD TD 390 and mart f(0). (3). fay) (e3}} = wax[35, 20, 50.1048, 2.11850) 5010838. Theorem 1.5 (Rolle’s Theorem). Assume that f € Ca, bh and that £"(x) exists for 5 € (a,b). W Sta) = fb) = 0, then thers exices 2 number, with © © (2.6) ch that fo) = 0. ‘Theorem 1.6 (Mean Value Theorem). Assume that f-€ Cla, 6] and that /"¢x) exists forall x ¢ (a,b). Then there exists a mrmber c, with c € (a,b), such that an ‘Geometrically. the Mean Value Theorem says that there is at least one number € (@.6) such thatthe slope ofthe tangent ine tothe graph of y = J (x) atthe point (e./4€); equal the slope ofthe secant ine duvugh the points (a. f(a)) and (b, /{B)). ‘Bxample 1.3. The tunction #4) = vin(x):s continous on the closed steal 1.1, 2.1) and differentiable on te open interval (0.1.2.1). Thus. by the Mean Value Theavern there {sa aumber¢ such that, sg) Tiel) JOA) _Onescu9 ~ vo9yass Fle) = HEY LR ese 381088 fe Solution to fe) = cose) = 0.381688 in the intel (0.1.2.1) is ¢ = 1.791 “he graphs of 2). he secant line y = 0.381688" + O99K33, and the tangent Hine = O-se16Sue 4 0.474218 are shows a Figure 1 .6 Hap. PRELIMINARIES mo) 10 SO), We) $100) a Ca ee ees Figure 1.6 The mean value theorem applied to fle) = site) over che inervat (0.1.2.1. Theorem 1.7 (Generalized Rolle’s Theorem). Assuune thet f «© Cla, } and that LR) LCs ons FMLA) exit over (a, B) and. x9, 23, --1.%q € fab). EFC) for j =0,1,.-.yn:then there exists a numberc, with € (a,b), suck that f(c) Integrals Theorem 18 (First Fundamental Theorem), If fis continuous ver a, 6} and F ibary anidervatnec fon [a 0) hen a2) [[fodx=FO)— Fle) where FO) = £0. ‘Theorem 1.9 (Second Fundamental Theorem). If fis continuous over (a, b) and xe (@,b),then Pars ay ral far = Fee) ‘Example £4. The function f(x) = cos(x) satisfies the hypotheses of Theorem 1.9 over interval (0, 1/2], hus by Che chain ale af? ayn?) = 2) Eff conned = Dec : ‘Theorem 1.10 (Mean Value Theorem for Integrals). Assume that f € C{a, Then there exists a number c, with ¢ € (a,b), such that a raf foe- sto ‘The value fc) is the average value off over the interval (a, Bl See. 1.1 REVIEW OF CALCULUS 7 06 oa ee Figure LE The moan vane i » : theorem or mera pid to oo f 7 Ff) = sina) + FainGey ver tne oo os to ts 20 as Ae asat? = sin(x) + | sin(x) sats the hypotheses of The Example 15. The function £0 costs) = ‘orem 1.10 over the interval (0, 2.5]. An aniderivative of f(x) is F(x) Yee 32) The average value ofthe function f(2) aver the interval (0.2 5) 5 ‘There are three solutions to the equation f(c) — 0.749496 over the interval {0,25} 1 = 0:440566,é2 = 1.268010, and cs = 1.873583. The area of the rectangle with base 6 — a = 23 and height (cj) = 0.749896 is f(c))(b — a) = 1.873740. The area ‘of the rectangle has the same numerical valve asthe integral of f(x) taken over the iter- val (0,25. A comparison ofthe area under the curve y = f(x) and that ofthe rectangle can be acem in Figure 1. . ‘Theorem 1.11 (Weighted Integral Mean Value Theorem), Assume that f.g € C{a, 6] and g(x) > O for x ¢ (a, b]. Then there exists a number c, with © € (a,b), such that . . «ag [ford so [aeras Raample 16 The fonetons #(<) ~ sin) and gf) — 2 eat the hypseses ‘heoren .1 over the interval (0, x/2). Ts thee exists number uch kat KO wsindaddx _ 1.14159 Par 7 ass 70808! since) ‘ore = sin!(0.883631) = 1.08356. .Series Definition 1.8. Let (an), he 2 sequence. Then TX ay i€ an infinite series, The nih partial sum is Sy = 7 sequence [Su)$, convergor to 44, The infinite series converges if and only if the lim S. dati, Ifa series does not converge, we say tha it diverges. ‘ Example 1.7. Consider te infinite sequence {aq}, asia suas ‘Therefore, the sum ofthe infinite series is lin, Sm = im, ‘Theorem 1.12 (Taylor's Theorem}. Assume that f € C""[a, b} and tet 19 © a, A]. Then, for every x € (a,b), there exists a number © = c(x) (the value of ¢ ‘pends on the value of x) tat lies between xo and. such that 6 fa) = Pate) + Rs) whee an Poe) — Ye aa! and they 1 rate) Lege aay! ‘ 0» FO ce) Example 18. The function f(x) = sin(x) satisfies the hypotheses of Theorem 1.12.1 ‘Taylor polynomial Fy(x) of depreem = 9 expanded about «= = is obtained by evaluat Sec, 11 Revinw or Catctus y 10 00 0s -10 Figure 16 The graph of f(2) = sin(x) and the Taylor polrmomial Ptr) =x — 2/3! +28/5!~ 2/11 429/9 far sinter, FO =0. Ss) = cust LH =1 fix) = sin), F100, Fx) — costed, 40) — Fa) mcoste), f= 8 Au) = A graph of hath f ana Pacer the interval [fr] ie chown in Figure 1 6 . ‘Corothary Li. If Py(x) isthe Taylor polynomial of degree « given in Theorem 1.12, a9 Peay) — F449) for k=O, A, ay Evaluation of a Polynomial have the form Let the polynomial P(x) of dog 120) PO) gk ag tt aa? Haye +910 CHAR. PreLacnantes Horner's method or synthetic division is «technique for evaluating polynvanias, 1. ‘canbe thought of as nested multiplication. For example, afifth-degree polynomial can bbe wnt sn the nesced multipleston form Psi) (Ulasx + ag)x + as)e ag) —ay)x + a, Theorem 1.13 (Horner's Method for Polynomial Evaluation). Assume that P(x) is the polynornal given n equation (20) and.x = c 4 a number for which P(c) isto be evaluated. Sei be = ay and sompute en batch. fork= ae then by = Pe). Moreover if @ 2045) = baa! + bya ose ba? bar tb, then en Peay = (x e043) + Rov where Qo(x) is the quotient polynomial of degree n — 1 and Ro = bo remainder. Proof, Subsiuting the right side of equation (22) for Qo(x) and bs for Ro in equa- ‘ion (23) yielde P(e) is the PE) = ON a bg ot Ba One + BY) +O ra = bes = Cbynt eb 4 (be — cbs)? (by — ebabe + (bo ~ cb1) ‘The numbers by ate daternined by comparing the cooficients of «tin equations (20) and (24), as shown in Tale 1.1. ‘The value Pc) = py is easly obtained by substimting x and using the Fact that Ry into equation (22) es Poo fe = A300) = Ro — hy : ‘The recursive forma for by given i (21) is easy to implement with a computer A simple algorithm is, b(n) = ain fork =n 1-10 Bek) = atk) +6 #B(k = Ns end SBC.1.1 REVIEW OF CaLcuLs n ‘able 11 Coeficiens by for Horner's Method CCompaieg (20) and (24) [Solving for by n= be arta daci mete | bunt mapa the aembemcbnss | by matey: somby 2 ooh Table 1.2 Homer's Table forthe Symetic Division Process Trpot ] ay dt Geez sae a Xby xb, abet bs xb by on When Homer’s method is performed by hand, i is easier to write the coefficients of P(x) 00 @ five and perfor tie valculation by = ay + CBee} below ay 10 a comma, ‘The format for this procedure is ilustred in Table 1.2, Example 19. Use synthetic division (Homer's method) to find P(3) forthe polyaomicl Pla) = 08 ~ 6x4 + Be? — Ret 4 dr ~ 40. Tae] a ee 8 ee eeu I= PQ) =o Ourpt be Therefore, P(3) = 17 .12 CHAR. PRELMINARIES ixercises for Review of Calculus, Lota) Find 6 = ling oa in 4 Man He Then determine fee = [Leland nad (by Find L = Eimy-o(2 nd fing ye 6 1)/(4n-+2n41), Then determine (eq) = [La Let faghgy Be a sequence such that itp an = 2 (2) Find Hing inte) () Find lms ec inte3) 3, Finc the number(s) ¢ refered 1a inthe intermediate value theorem for each function cover the indicate interva and forthe goven value of 1 (4) fea) = 12 + 2a v3 one [1,0] ving (0) fix) = VaT $x over [6 8 using L 4. Find the opper and lower bounds referred to inthe exteme value theorem fr esch function over the indicated interval fae p Lever! 1.2 (by fx) = c0s"(x) ~ sin(x) over 0,245 5. Find the nuraber(s¢ referred to in Roles theorem foreach function over thei cated interval, @ fo fi 6, Find the number(s) refered to in the mean value theorem for each function over the Incated inter fay fads Ye overt0.41 to) fare A overt0.11 7. Apply the gonerslized Rolle’ A. Apply dhe Sex fandamental theorem of calculus to each function over the indicated interval ta) 00) = 26° over ty.) theorem to Fr) — ete = Her = 2) over 0.3] +i 9%. Apply he second fundamenal dheore oF calculus tu each Fame everio}e ta) ff Pcouerde o 10, Find the nurher(e ¢ refered tithe mean wale Hever fo inteals foe each fametion, over the indicaced terval fa) fx) = 60? over |= 34, th) 2) = costa) over 0.34/21 11, Find the sus of exch sequence o series. 12 See. 12 Bivary NumaeRs r o (|. © Limo 12, Find the Taylor polynomial of degree n = 4 foreach fonction expanded about tn ‘given value of x, @) fo-Jeen! @) fader bate tet hx Oi yaa at 13. Given tat 7(2) = sincx) and x) = x —29/3t4 0 /3t— 27/777, Show PHO) = FO) Fork 1.2.09 14, Use sythetic division Homer’ method) t find Pt). (@) Payextex?— 138-2 Tend ©) Pape eat a Bat Bem Ad 18, Find the average area ofall ccies centered atthe origin with radi between 1 and 3 16. Aszume thats polyomil. P(x). has » real roo inthe interval fa. Show tha ‘PO™Mx) asa least one eal roti the interval a,b 17, Assume that f,f', and f” are defined on the interval a.]; f(a) = f(b) = 0: ane (F(e) > Ofer ¢ (a,b). Show tat there ist number d « (a,b) such that f"(d) <0 inary Numbers Human beings do ariuuncvi using he devil ase 10) number system Most com ‘puters do arithmetic using te binary (base 2) suber system, I may seem otherwise Since comunceation with the Computer (inpuvoupat sin base 10 numbers. Thi Uamnpateny oes aot wean that te computer ses base 10. Un fat, i converts inpet to base 2 (or perhaps base 16) then performs base 2 arithmetic, and finaly ransete the answer ilo bese 10 before it displays a result. Some experimentation i require to verify this. One compurer with nine dacinal digits of accuracy gave the answer a Ye ou = a809.98087 Here the nent was to ad the number f repeatedly :00, Otis. The matbemtic answer s exactly 10. QU. One goa 1 0 unerstan he reason forthe computer's ap arenily awed calculation, Ai the end ofthis section, it willbe shown how somethin islost when the compter wales the decimal fraction jy into a binary umber.14 Cart PRELIMINARIES ‘Binary Numbers Base 10 numbers ar used for most mathematical purposes. For illustration, the number 1563 is expressible in expanded form ss 1563 = (1 x 10°) ~ (5 x 10) + (6 x 1049 + (3 x 10%). In general. Jet N denote a positive integer; them the digits ap, a)... ak exist so that 1V has the base 10 expansion N (an TO) Capa LOY + + (ay « 10!) + (ay x 10°, where te digits ay are chosen from (0,1. .., 8.9}. Thus Vis expressed in decimal ey ak °°+220 dy, (deeamal) Fics understood that 10s the base, then (2) is writen as N= agai -azaag For example, we undersiard that 1563 = 1563. Using powers of 2, dhe number 1563 can be writen 1963 = (1 21°) 4 (129) +028) 0x27) + 0x 2%) o Hx 2) HME x PIF OKA) +2) er) ‘Thiscan be vesied by performing the calculation 1308 1024+ 912+ 10484241. Tn general. let N denote a postive integer; the digits bo, by, .., by exist so that NV has the base 2 expansion Baby RE RP A ey eM, Where eaclt digit by is euher a0 oF 1. Thus‘ is expressed in binary notation as o Nx byhyn i -PabrPogey (inary). ‘Using the notation (5) and the result in (3) yields 1563 100001101 Ive Remarks. The word "two" will always be used as a subseript atthe end ofa binary numer. This will enzble the reader to distinguish binary nomibers from the ordinary ase 10 usage. Thus 11 means one hundred eleven, whereas [lve stands for seven, See. 1.2 BINARY NowBERS w Ics usually the cate thatthe binary repretentation for Nill require more digits than the decimal representation. This is due 1o the fact that powers of 2 grow much ‘more slowly than do powers of 10 ‘An efficient algorithm for tning the hace 2 representation of te integer W’ can be ts ten te senes diverges Proof. The sumamaton formula for ite geometic series is 2 ea rt sy SaretertorP4.. tert =U 2 for #1 To exablsh (12), observe tat ax Ir <1 Spies oe tin 9°17 0. “Taking te imi asm + owe (14) and (15910 et (1- Jim”) By eaaion (15) Seton Lhe ii ahve exis 1 ‘When | 2 1, the sequence ir?!) does not converge. Hence the sequence (Sy) 1 (14) does not end to 2 limit. Therefore, (13) is established, ° ‘Equacon (12) in Theorem 1.14 represents an efficient way to conver an infinite repeating decimal into fraction, Heample 13. Binary Fractions Binary (base 2) fractions can be expressed as suis iuvolving negative powers wf 2 If Risa real aumber that lies in the cange 0 < R <1, there exist digits di, di, dys = $0 that as) Ra XD YD ty KIM Hoy where dy € (0, 1). We usually express the quantity on the right side of (16) in the binary fraction woxation an R=0Aid2--- dy 0018 CHART PraLiwisantes ‘There are many real numbers whose binary representation requires infinitely many igic, The fraction 7/10 can be expressed as 0.7 in baze 10, yet its base 2 representa tion requires infinitely many digits a a8) Pence ‘he binary fraction in (18) iso repeating fresion where the group of four digits 0110 is repeated forever. ‘An efficient algorithm for finding base 2 representations can now be developed. If both sides of (16) are multiplied by 2, the result is 19) WR dy ede PN dyed, ‘The quamity in paremtheses on the right side of (19) is @ positive number and is less ‘han 1. Therefore dy is the integer part of 2R, denoted dy = int (2R), To continte the rocess take the fractional part of (19) and write 0) Fy = teaches = ag 2H dy where frac(2R) isthe fractional part of the reat number 2K. Multiplication of both sides of (20) by 2 results in an BA Sa (ds AE hy IM Hd, ‘Now take the integer part of (21) and obtain ds = int(2Fi). ‘The process is continued, possibly ad infnivum (if & has an infinite nonrepeating bbase 2 representation), and two sequences fdr] and {4} arc cocutsively generated de = int 2Fe. fi = Sachi), where dy = int(2R) and F, = fac(2R), The binary decimal representation oF Ris then given by the convergent geometric series R= Lay Example 1.12. The binary decimal representation of 7/10 given in (18) was found using Let R= 7/10 0.7, hen he ftdes 223 2Ratd ty = ich a) 1 Fy hala) 08 2h 08 By = foe. 08 als Fre fiecil.) 06 hela int(.2) = 1 Fy 2 fre.) 2 02 2i=04 inu0.4) = 0 1 = fra) = 04 re =08 aus) ~0 Ky fa0t0.8) = 08 tele ian) Fre fraes1.6) © 06 SEC. 1.2 BINARY NUMBERS. 9 Note that 2F2 = 1 3.4. Ths 7/1 Geometric series can be used to find the base 10 rational number th umber represents. 2F The patterns Figs will occur for 10 yo . a binary Example 1.13. Find the base 10 rational number that the binary number OO spre sents. In expended form, Oi me = (0x 2°) Hx 2) OXIDE IE mi+ Patt Binary Shifting 11a rational number that equivalent ton infnite repeating binazy expansion isto be ‘can he helpful. For example let She given hy a 5 = 0.000007 Ts ‘Muinplyine both sides of (23) by 2° wal shift the binary point five places tothe reht snd 325 has the form on 325 = 0.17000 no, Sinifrs. muliplying both side of 23) by 2"° will shift she binary point ten places to the right nd 10245 has the form, as 10245 — 11000.TT00 0. ‘The mult of naively taking the differences herween the Ieft- and right-hand sides of 24) und (25) is 9925 = 1100099 0F 9925 = 24, since 110009 = 24, Therefore. Scientific Notation A scklan! way to present @ real nurnber, sinlung ue decimal pount and supplying an appropnace power ot 10. 9.9000747 = 7.47 10°, 35.4159265 ~ 3.14159265 » 10. 9.700,000.000 = 9.7 x 10" {Inchemisty_ an important conscan is Avogadro's number. which is 6.02252 x 10%. is cue number of atoms in the gram atomic weight of an element. In computer science, 1a = 1028 x 108 called slentfic notation, is obtained by ‘or example.2 CHART PRELISONARIES ‘Table 1.3 Decimal Equivalent fora Set of Binary Numbers wich 4-Bit Manisa and Exponent ofa 3B de pone: Manisa I net 01000imy | 0005 a8 aaalitiliit 0100490 | oo7as125 | 0.140625 | 0.28125, 2 | 4s] 9 vite | Gu7ei2s | 0.3625 | vs123 ase sie lee 0.4019 | 0.as937s | 0.171875 | o2437s 25 | 3s | O.1i0hee | 008375. | 0.1875 | oa7s jars fis fa | 6 | 2 0.110% | otois62s | 0.203128 | oauers | osizs| 162s | aas | 6s | 12 0.1110} | ot0s375 | o2is7s | oas7s | oss |i7s | 3s | 7 | o1nttey | onr7s7s | 0705 | oases | os] rans | ars | 7s | is Machine Numbers Computers use a nomaied toting pint binary representation fora numbers Instead, the computer stores a binary approximation o 26) xg ‘The numberq isthe mantissa and tsa fnite binary expression saistying the inequal- ity 1/2 |p~AV/pl = 70344210"! > 109/2tbe approximation Faytees wit the me answer p = 0'544987104184 to five significant digi. The graphs of f(x) ~ o* andy = Far) and the area under the curve for O = 4 5 4/2are shown in igure 7 Ste 14 ERROR Anatvsis a Round-Off Error ‘computer's representation of real numbers is lime to the lined precision of the sa, Thue values are vometimes not stored exactly by: a computer’ repreeen taton THs is called round-off error. In the yreveding section the real curaber 1. O.008TTiwo was truncated when 1! was Stored in @ compute. The actual num hort is stared in the compar may undereo chopping or rounding af the last digt Therefore. singe the computer hardware works with oaly a limited numberof digits im rachie cunbers, ounwhing errors are murodueed and propagated in successive com Chopping off Versus Rounding Of CGonsser any real number p that is expressed ina normalized decimat form: 4 = 20didady cde -p... X10" there | gy ¢ Gand < dy < Oforj > 1. Supyce that fis the maxim aomier .dcimal digits carried in he loatng- point computations of a computer: den the rel fuer pS FePFESEMIeD DY ftp Pe WHEN Ys gIVEA Dy B0didods.. dy ¥ 10° 1 Flex? where |< dz ¢ Yand 0 < dj = Ofor 1 < j x k. The number Flguep) is calied Inechoppedflazing-point representation ot inthis case the kth gt Of ejg( P) srs with the dah digit of p. An altemative k-digit eprescnation is the rounded Boating point represemation Fla P shich given by 4 Slaea(P) ~ £0drdady 14 10? here Fs = Gm Od, ¢ Ofor 1 L.itfollows that x? +1 < 20 forx > 1. Therefore, f(x) = O(@(2)).30 Cary eeLaarvaRis The big Oh notation provides a useful way of describing the rate of growth of a function in terms of well known elementary functions (x".!", a, Jog, £, st). The rate of convergence of tequences can be described in a similar manner, Definition 1.10. Let 4352, and (yal, be two sequences. The sequence (rq) is said to be of order big On of {yx}. dented xq ~ O(yq) if there exist constants C ane such dia & Val € Clyel whenever m 2 1 a Example 1.21. =f whenever 2 1 : tien 2 fonction F(A) is approximated by a Function p(k) and the error bound is known to be 41)". This leads tothe following defimion Definition 1.11. Assume that f(A) is approximated by the funetion p(k) and that {here exis a teal venstant Bf > O anna punitive integer n 9 ha % 17! rst sl ‘We say that p(h} approximates /(h) with order of approximation O(k") and write 10) Sth) = pth) + O(n"). ‘When relation (9) is rewritten in the form | (A) ~ p(h)i < MIN" we see that the notation O(5") sands in place ofthe error bound Hi"). The following results show lan to apply the defiitinn to simple combinations of two Functions ‘Theorem 1.15. Assume that fh) = poh) + OU"). gh) = yh) + OH), and info. ni}. Then ay Fin) + ah) = ph) | ath) + O18"), a J hygth) = pihyath) + OF") and Over a a AR hoary It ic instructive o consider p(x) t0 be the mth Taylor polynvnsial approximation ‘of f(r: then the remainder term is simply designated O(H"*"), which stands for the presence ot omitied terms staring with the power "The remainder term converses to 7ero with the same rapidity that WT"! converges to vera ac h approaches zen © expressed inthe relationship Provided that g(h) #0 and g(h) #0. ty we MBM wy LOMO pat aay Oth) se MBE we LOO yt “wn wer iy Sr. 1.3 BRROR AwaLysis a for suficiently small A, Heace the notation 0 (h"~!) stands im place ofthe quantity Ai’ *1 where M is. conetamt or “hehaves ike a constant” Theorem 1-16 (Taylor's Theorem). Assume that f € C** "fa, ). IF both x9 and say fhe la, bl. then os FO) A el as soon = nh 5 om, & The following example illustrates the above theorems. Te computations use the addition properties (i) OH?) + OA?) = O(AP), (b) BUH") + OU) = OtH'), ‘shore # — min{p.q}, and the mobiplicative property (ii) OUKP)O (he) = Ara) wheres = p tg Example 1.22. Consider the Taylor polynomial expansions Poe ca sare eM out amt conet Bik out Deere te ode of pronation fr dcr susan pr Forte sum we have A secon owes CA eae 2h EO 4 F400, Since Oth") 4 Sy = O(h*) and O¢H4) + Oth®) = OU), this reduces to As conthy 2m COU, and the order of approximation is Oh"). Penna (1+ae8 4 pom! ~(11-8-¥)( ’ a(n + On} OG5) eee 7 Hata + 04h) + Oh) + OHV O1H"). +ha2 CHART PReLiannantes Since OOOH) = Oth) and she & eae eae - Ty FO + 04 +04 7 the preceding equation i simples fo yield H ey ones Pronth) = 1h Ka aunty 5 +a and the order of approximations OCH) . Order of Convergence of a Sequence ‘Numerical approximations are oflen arrived at by computing a sequence of appro nations thar get closer and closer to the desires answer. The definition of big Ob fox sequences was given ia Definition 1-10, and the defiition of order of convergence for a sequence ie analogs that given foe fetions ix Definition 133, enminon 5.12 Suppose IMA ye 5p aE Pay 8 a seqUERKE vt Nima fa = 0. We say that {201% converses to x will the order of conve: gence (ra) there exits aconstat R > 0 such that SK form saffciently large This is indicated by writing xy = x + Ol'y), oF xy > x with onder of conve sence Ora) * Example 123, Let x, = cos(n)/n? and ry = 1/n?; te ity-oo Fg = O with at rate of comvergence (1/42), This follows immediately fom the elation Leostey| <1 for all m . Propagation of Error [Lecus investigate how ervor might be propagated in successive computations. Cun ser the addition of to numbers p and q (the tre vais) with the approximate veluca F nd 9. which contain eros ¢p and, respectively. Stating with p= ji-+ © and a= a +e, the sums 61 p+a P+ D+ ep +) P+ elt Gre) ‘eae, for addition, the ero in the sum isthe sum of the errors in the addends SECTS ERROR ANALYSIS. 3 ‘The propagation af error in multiplication is more complicated. The produc: is an 14 B+ NB 6) = BO Bey b Fn + Ceo ene. Band fare larger than I in absolute value, erin Peg and Jey show una ere isa possiblity of magnification ofthe original error €» and cy. Insights are gained if \we look a the relative error. Rearrange the terms in (17) 10 get oo) 24 — BG = Bea + dep ~ toby Swppote il py £ Oa y # 0; then we van vide (18) by py 0 obain dhe eave eon the prove py Pa Bey tote ia ha (9) Ry Pa pa” Pa Farhermet, sippoce that and ave god appesimanions for p and 9: thew Fip%1,8/@ ~ |, and RyRy = (€p/p)(¢q/q) ~ 0 (Ry and Rg are the relative exors iin app numadions Bani qj. Then maxing these suosinsions io (9) yes Simpler elaticnship @o Rog = Ry + Ry OG “This chours that the relive eror in the prodoct a is approximately the sum « relative errors inthe approximations B and 9. ‘Often an inal error will be propagated in a sequence of calcvlations. A quality ‘huts desiranle for any numerical process is that a small errr in the initial conditions. wil prodce sh changes in the final result. An algorithm with this feature is called stable, alrerwise its called unstable. Wheaevet possible we shall cose meibods that are stable. ‘he following definition is used to describe the propagation oferor. Letiniion 1.13, Suppose that « represents an miual error ana ¢(n) represents the ‘rowth of he enw aterm steps. I lef © ne. the growth of errr is sai t be linear. If c/ayt © Ke, the growth of error i called exponential. IF K > 1, the exponent cron grows without bound 4s + os, and if 0 < K <1, the exponential erroe Alcsnishes to 7210 86 9 a he nect two examples showy how an inital errr can propagive i either a stele ‘or an unstable fashion. In the first example, the algorithms are introduced. Each _lgoritim recarsvely generates the sare sequence. Then. inthe second example, small ‘anges will be made to the initial conditions and the propagation of error will he onalyaed,34 CHARI PRELIMNARIES ‘Table 14 The Sequence {5} = (1/3" andthe Approximations [rs (pe an (gal SHC. LSBRROK ANALYSIS s Tohle 1.5 The Bror Sequences (x4 ~ ras (te ~ Palsand ta — a 0 ‘oano000000 | e.9999600000 | 10000000000 | .0900000000 ete “010000800000 “0000000000 ‘e.0000000000 : bors oom Comporsss 1 siasssss | osm | osama | qss:sm0 ea Saporsss3 90001333 fl O11) | 0.1111066667 01110933330 01110666667 ‘4 (0.0000014815 ‘0,0000192593 0,0001348 148 | : omens o.o83t fotos. 3) foams | anss6 | om | omvoam : Gomeme commer cowmioes ooinesse | consis | ooiansnse | oonsurr ‘ Secon Saat 26 Sonsous) 4) romain | | : cor oe oe 3] sbeomsuns | vam | enn | amen tf Copomnest oo 3990 fonzenioe2 1p econ sre | oootsirees | commas 3 | enw Sonate 6{ qycomnnracr | conser { oonnsinas | coax oo esis 7] gdyceemsnas | eomusrner | nesnerses | nonce —|__seomoneg 2| cer=oomsoass | coooszas | covotsames | -ooxssosss 9| elar=oonsenss | oocmsoeos: | commasss | ~oosecuies 10| seb =o.000018351 | oommiess4s | -oommnsis | -o29s2280688 Example 1.24. Show that the following thee schemes can be used with infnte-precision aithmedc fo recursively generate the trms inthe sequence (1/3*}2q aww foc = 1,2, 21) po= ber 0 le go= 191 and n= F 4? form = 2,3, ec ana AI". Thincanbe vented des ean ’) jporei(ne4) {G65° = (4-2)a-(f-}ecabsnen ans (will generate the desired sequence. In 21e) the difference pet 3 Sewing A = 1 and B Sling A = 1 and B = 0 genortes the requited sequence . ample 25. Generate approximations to the sequence (r=) = (1/3") using the 229 0.99986 aul ry = rant formate for n= 2, 3, H) post, pi = 0.33332, and py for n= 2,3, 0.33332, and an 2) tn (22) the initial error in rq is 0.00004, and in (226) and (22c) the initial errors in py, sn gy are 0.000013. Investigate the propagation oferor for each scheme, Table 1. gives the rt ten numerical appronimason: for cach sequence, and Table LS escathe emor in each formula, The ero for (r) is stable and decreases in an exponenial ‘rane. Lne errr Yor {pa} 18 stable. Ine ero Tot (gy 8 unstable and grows a a expo- nomial rte, Although the error for [pq is sible, the terms fy > 0.85 > 20, 50 that the aor eventually dominates and te terms past pg have no sigmificant dis. Figures 18, 19, and 1.10 chow the erors in irq] [pals ad (gy Feepetivly. : Lanoowors} 2.000010 0000s . ie ge 2 + ‘ 5 10 Figure 1.8 A stable decreasing enor sequence (xg ~ rl ‘Uncertainty in Data Daa from real-world problems contin uncerainty or eros. This typeof eror is forred\o 2 noice, It wil affect the accuracy of any mumerial computation tha is bed on the data, Aa improvement of precision isnot accomplished by performing succes ‘Sve computations using noisy data. Hence, if you start with data with significant digit of aecuracy then the rest ofa compaton should be eared in d significant sigs of accuracy. For example, suppose that the dta py = 4.152 and p2 = 0.07931 bot have tour sigsneanc agus of accuracy, hen wis tempting to report al the digas that appear on your ealulator Vie. oy + py = 4.23131), This is en oversight because Ane 03 . 02 on . 3 7 6 3 0 Figure 1.10 An unstable increasing eror sequence (4 — dl SECIS ERROR ANALYSIS x you should not report conclusions from noisy data that have more significant digits ‘han the orginal data. The proper answer in tis situation 18 p+ p2 = 4.231, Exercises for Error Analysis 1 Find the error F and relative errr Ry. Also determine the numberof significant ‘igs nce approximaton, G@) v2 7198189, 7 27-718 ©) y= 98,350,5 wo: Complete he ftlowing computation [Pe sen [ (e548) ame ‘State what type of eerie present in thie siuaticn. Compare your answer with the teu value p = 0,2553074606. 3. (a) Consider the data p) = 1.414 and p> = (209125, which have four significant Aigits of accuracy. Determine the proper anewer forthe sum py + pa and the product py (b) Consider the data py = 31.415 and pp = 0.027182, which have dive significant ign of scenracy. Determine the proper answer forthe sum pi + pa and the product pip 4. Complete the following computation and state what type of error is preset in tis fay 8 (E-+0.00001) —in(§) _ 070711388222 — o707LO67811 aD “como 20.0005) — Inc) _ .69s172IMMDS —a¢ast4ETINNG ‘O.0008 ‘0.90005 5. Somedmes the ls of significance er can beavoided by rearranging termi the function using a known identity from tigonomety or algebra. Find an equivalent formula forthe following function that avoids a loss of significance. Ge) ba(e 1 1) InGs) for large « @) JP FT~ 2 forlargex (© costa) ~ sina) for © 2/4 Freeman, Oe 6. Polynomial Evauation, Let P(x) = x—33+3x~1, OC) and Roxy = (2 = 1)" (2) Use fourdigitrounding arithmetic and compute P(2.72), Q(2.72), and R272) Inthe computation of P(x), assume that (2.72)? = 20.12 and (2.72)? = 7.368 (343-13B CHAR. PRELIMINARIES () Use foursisit rounding arithmetic and compute P(O.975), (0.975). and (0.975). In the computation of P(x), assusse that (0.975)> = 0.9268 and (0975)? = 0.9706. 1. Use thee-igit rounding arithmetic 1 compere the following gums (sum in the given ©) Lint 8 Dinas de pean of ev forthe flowing (a) The sum of tie numbers: B+ enh+G+eq)t Fre tat (8) The quodent of two numbers: (©) The product of dee mambers: PU EP HepGree rer 9. Given the Taylor polynomial expansions Lthe +i? +0064) aie +00, Deter te order of approximation fr thelr sur and produc. 10. Given the Taylor polynomisl expansions conth) = and singh Determine the order af appronimatin fo her um and pe 11, Given the Taylor polynomial expansions Boe cot 1 Fy Ee oy snd » sin(hy =h— 0) =a Determine the order of approximation fr ther sur and princ. SBC.1.3 ERROR ANALYSIS » 12, Improving the Quadratic Formal. Assumethata 4 and 6?--dec > Oandconsider the equation ax?-+bx-+¢ = O. The roots can be computed with the quadratic formalas © =b+ Va Tae wpb Vee 2a Ce gaara ‘Show that mpese roos can he calculated wit che equivalent formulas ne x C n= amd o ers ers ‘Hint, Raiomalize the numerator in), Remark in the cases when |b] VOF—Aac, ‘one must proceed with caution 1 avoid loss of procision due to catastrophic can- cellaton. If 6 > 0, then 1 should be computed with formula (i) and x2 should be computed using (). However if < 0, then xy should be computed asg () aid 42 should be computed using (i. 13, Use the appropriate formula forx1 and x2 mentioned in Exercise 12 to find the roots ofthe following quadrats equations (a) x2 1.000001 +1 =0 (b) x? ~ 10,000.00" +1 = (©) 29 ~ 100,000.000015 + 1 (a) x? ~ 1,000,000,0000015 + 1 = Algorithms and Programs 1. Use the results of Exercises 12 and 13 to construct an algorithm and MATLAB pro- gram that will accurately compute the rots of 2 quadatie equation in all stations, ineluding the woublesome ones when bl = v6? = fae 2, Follow Example 1.25 and generate the frst ten numerical approximations for each ofthe following tree difference equations. In euch ease a small itil ear ie in ‘woduced. If there were no inital eer, then eich of the difference equations would _enerate the sequence (1/2"]z. Produce ourpuranalogous1o lables I and 1.9 anc Figures 1.8, 1.9, and 1.10. @ © po © phe Pract, form =1,2, = 0497, a6 Pa = Ppt ~ Parte form = 2.3, ent anos form = 24,The Solution of Nonlinear Equations f(x) =0 Consider the physical problem that involves a spherical ball of radius r that is sub- tnesged to adept din water (see Figure 2.1), Assume thatthe bal is constructed from « varizy of longleaf pine that has a density of p = 0,638 and tht its radius measures r= 10cm, How much ofthe ball wil be submerged when itis placed in water? ‘The mass My of water displaced when a sphere is submerged to « depth is y= {m2 nde OD ‘and the mass of the balls My = 4xr3p/3. Applying Archimedes’ law My = Mo. prodiucns the following equation that must he solved: x! =r +47, ~ a a Tg 24 mp 1 1 ‘sphere of radius r that is t0 be sub- ot ee Qt StC.2.1 ITERATION FOR SOLVING = (2) = 304g? {In our ase (with r = 10 and p = 0.638) this equation becomes a2 0. 3 ‘The staph of the cubic polynomial y = 2552 — 30d? +d is shown in Figure 2: from it one can se thatthe solution lies near the value d = 12 ‘The gual ofthis chapter is o develop a variety of rains for fining aur approximations for the roots of an equation, For example, the bisection method « be applied to obtain the tree Toots dy = ~8.17607212, dy = 11,86150151 4; = 26.31487061. The frst root di ia nota feasible solution for thie problem, be: d cannot be negative. The third root di is larger than the diameter of the sphere ¢ isnot the desired solution. The root d) = 11.8615015I lies inthe interval [0, 20 4s the proper solution. Its magnitude is reaconable because alinle mare than on ofthe sphere must be submerged. Iteration for Solving x = g(x) ‘A fundamental principle in computer science is iteranon. As the name sugge process is repeated unl an answer 1s achieved. erative techniques are used t 1.0 Finear and nonlinear systems of equations sed eo ‘of differential equations. In ths section we study the process of iteration sing rey substation, ‘A rule or function g(x) for computing successive terms is needed, together starting value po, Then a sequence of values (ps} is obtained using the iterativ42 CHAR2 THF SOLUTION OF NONLINEAR EQUATIONS (3) = eet = giped. The sequence asthe peters po (starting value PL = (0) p= stp) ao ‘Wat can we Jeara from a unending sequence of numbers? W une numbers ter to lint, we feel thar something has been achieved. But what ifthe numbers diveree or ae periodic? The next example addresses this situation, Example 2.1. The iterative rule po = 1 and pigs = 1001 pi fork = 0,1... produces divergent Sequence, ihe Mt 109 Lermns ok es follows: ‘P1 = 1.001 pp, = (1.001)(1.000000) = 1.001000, P= 1.001 p) = (1.001)(1,001000) — 1.002001 _P3 = 1.001 p2 = (1.001)(1.002001) = 1.003003. pica = 1.001 pp = (1.001), 1.104012) = 1.108216. “The process can be continued indefinely, and it is easily shown that lintgn. Ds = =. In Chapter 9 we will see that the sequence [pj is @ numerical solution tothe ditferenial equation y= 0.001. The gluten is known tobe yx) = e202, Indeed, if we compare ‘the 1000 term sn the sequence with »(100), we se ut pian = 1.108116 ~ 1.195171 2 = 1W, . In dis sectin we are concerned with the types of tuncuons g(x) that procuce convergent sequences 174). 1 Fixed Points Definition 2.1 (Fixed Point. A fred point of a funtion g(1) is areal aumbsr such bot 2 = 6) o fixed pointe ofa function y — g(x) ae the point of intersection Definition 22 (Fixed-point Iteration). The serstion pest = tips) for n = 0. 1... is called fived point iteration a 90.2.1. [HeRsTION FoR SLING 4 = 4K * Thesrem 21. Assume that is continoes function and tha! (pa 9s x sequence seveate by fn pint raion. Hime p= Pesca P va hxed pane gs). Froaf {Vite Pa = Po te biBieeae Pa) = P11 follows from this result the fonts of g, and the elation Pay (Ps) that (2) iP) = 6 (i, Po) = im.) = fi Pat = Cea een : Example22. Consider the convergent iteration pos0S and pis ze ‘Te first t0 terms are obtained bythe cal pre S800 =o, p09931 95829 9 10703, 034609 9 567560 95080 4. 9, 586507 pw “The sequenet is converging. and further clcultions reveat that ig Pe = 0567143 ‘Taus we have found an approximation forthe fixed poin ofthe function y ‘Tre followicg two theorems establish conitions for the exinence of a fied point ‘andthe convergence ofthe fxed-poimt iteration process toa fixed paint “Theorem 22. assume that ¢ € Cla, b}. 2) Uf the range of the mapping y = g(x) satisties y € (a,b! for all x ¢ fab], then g hae a Bred pointin (2,5) (4) Furthérmore.cuppose that g(x) is defined over (a, b) and that apositise constant A < J exists with |g'(e)i < K <1 for sll x © (a,b) then g has 2 urique fixed pint Pin ab)4° Cnar.2. Twe SOLUTION OF NONLINEAR EQUATIONS (x Proof of (3}. If gla) = a or g(b) = b, the assertion is true. Otherwise, the values ‘of g(a) and g(b) must satisty g(a) € (a,d] and g(4) € [a, 6). The function f(x) = gC) has the property that say Now apply Theorem 1.2, the Intermediate Value Theorem, to f(s), with the constant 1. = 0, and conclude that there exists a number P with P € (a, 6) so that f(P) ‘Therefore, P = g(P) and P is the desired fixed point of g(x). Proof of(4), Now we must show tha this solution is unique, By way of contradic~ tion, let us make the addtional assumption that there exist two fixed points Py and Pa, Now appiy Theorem 1.6, the Mean Value Theorem, and conclude that there exists @ number d ¢ (a, b) so that ~ 610) > 0 sla) <0 and fd) 6 gtd) Next, use the facts that g(Pi) = Pj and g(Ps) = P to simplify the right side of equation (5) and obtain re RA Bur this contradicts the hypothesis in (4) chat la"(x)| = 1 over (a,b), 60 it is not possible for two fixed points to exist, Therefore, g(x) has a unique fixed point P in a, 6] under the conditions given in (4), . ed) Example 23. Apply Theorem 22 to rigorously show that g(x) = cos(x) hes a unique Fixed poin in (0, 1} Clearly, ¢ € C{0, 1). Secondly, g(x) = cos(+) isa decreasing function on 10, 1, thus its range on (0. 1s 051). 1] ¢ (0, 1]. Thus condition (3) of Theorem 2.2 is satisfied and sg ha a fixed point in [0,1]. Filly. if ¢ (0, Ie them [sx)] = | sin(x)! = sints) = Sin(1) < 0.8415 < 1. Thus K = sin(1) < 1, condition (4) of Theorem 2.2 is satisfied, and {as 2 unique hxed point n 10, 1. . ‘We can now state a theorem that can be used to determine whether the ixed-point iteration process given in (1) will prose a convergent or divergent sequence Theorem 2.3 (Fised-point Theoren). Assuie that (i) g. 4” © Cla, b} Gi) K is postive constant i) 70 € (a, Band (iv) #(x) € fa, 6] forall x € Ta. bl (6) WF [g's K <1 forall x © {a,b} then the iteration py = (Py) wil ‘converge tothe unique fixed point P € {a,b}. In this ease, P is Said 10 be an auractive fixed point (Ty Af Leo! > 1 for all x € (a,b), then the iteration po = 2¢px-1) will not converge to P.Inthis case, P is aid wo bea repelling fixed poin and he iteration ceahibits local divergence. Sue. 2.1 ITERATION FOR SOLVING = gtx) 4s We-p|—4=-—! pd —e} a ™ % Figure 23. The retationship among P, po, Ps |P — pol and |P — pil ‘Remark 1, Ris assumed that po # P in statement (7). Remark? Because gis continuo: on an interval containing itis petmissile w use the simpler criterion g'(P)| < K <1 and |g'(P)! > 1in (6) and (7), espectvely Prodf, We fist show that te pots (pu) alle in (a. 2). Starting with po, we apply Tacorem 1.6, the Mean Valve Theo, These en a vale € a. 9) 50 hat VP ~ pil = 1g) — gtpo)1 = le"{eo)l po? = IP po KU pol <1 = po @) Therefore, pr is mo further foour F thas po was, an it follows that pi € (a,b) (See Figure 2.3). In general, suppose that p»1 € (a,b): then le P) = ere = fe DP ~ pe DI [ea DIP ~ Peal SKIP = penil

1, and «sy P= pal s Ue Pol for all n> 1 Sec.2.1 ITERATION FoR SoLVING x 0 ence when I< (P), Figure 25 (b) Divergent oscilla aon when g'(P) <1 Graphical Interpretation of Fixed-point Iteration Since we seek a fixed point P t0 g(x) i is necesary thatthe graph of the curve y= g(a) and ie line y — 4 intersect at he point (P, P). Two supe type of ‘converter iteration, monotone and oscillating ar illustrated in Figure 2a) and (), respectively "To visualize the process, sat at py on the Aan nd have verily Wo the point (po, pd) = (po, 8(po)) on the curve y = g(x). Then move horizontally from (po, P1) to the point (py, pi) onthe line y = x. Finally, move vertically downward to p) on the x-ans. Tho rooutsion ces = 6(pr) is wed to constact the pot (Ps Prt) 00 the graph, then a horizontal motion locates (py 41. Py) on the line y = x, and then a vertical moverentends up at py. onthe x-ats. The situation is shown in Figure 2448 Cuiae 2 THe SoLUTION oF NontiNeaK EQUATIONS fix) ~0 If |g'(P)] > 2, then the iteration py 1 = g(pn) produces a sequence that diverges wey from P, The two simple types of divergent iteration, monotone aud oscillating, ae ilustrated in Figure 2.5(a) and (b), respectively. Example24. Consider the iteration p,1 = g(ps) when the function g(x) = +x~27/4 'sused. The ied points can be ound by solving the easton x = (x). The tw slutons (oa piven ebarev — -Pand r~9 The drain ofthe fimston sq) ~ 12/2, fn there are only two cases consider. 2 casei: 20s Sa ih 2 owes tenet 2 Tanaris = daira Covet P: then get slg, me=t ‘Since ig < 4 on [1,3], by Theo- oi Sin ig (21> $on{-3, —1.by The- cram 28 tere il comers 3 |r ‘Theorem 2.3 does not state what will happen when @/(P) = 1. The next example ins been specially consrucied su that the sequence {pe} converges whenever po > and it diverges if we choose po < P. 8(Px) when the function x) = 2¢2— 1/2 Example 25. Consider the iteration py exists. The derivativeis e(2) = 1/(e— 1) forx 2 1 is used. Only one fixed point and g'(2) = 1, a0 Theorem ?3 dace not apply. There ser wo cxses to concider when the starting value lies othe left or right of P = 2. i Case i: Str with py = 15, Case i: Stax with py tener ey = ALOIS den ge 87 pas te7teest P= 240789513. ps Lares 37309914 ps =0.53890832 5aasen84 ps= 2 -awurtony"? 2B, Pe =z, Since pg lies ooece tne domain of ‘Tm sequence 1s converging 10 slowly £13), the tem pe canna be computed (o the vulue P = 2; inded, Phono = ‘oosssria. Ste. 2.1. IrBRaTio FoR Sotvina 4 = 5(2) ” Absolute and Relative Error Considerations In Example 2.5, case (i), the sequence converges Slowly, and after 1000 iterations the three consecutive terms are roo = 2.00398714, pico = 2.00398317, and proms = 2.00397921 This should not be disturbing; after all, we could compute few thousand more terms and find « better approximation! But what about a criterion for stopping the iteration? Notice that if we use the difference between consecutive terms, lero Proce! ~ 12.00398317 2.00397921| ~ 0.000396. Yet the absolute error in the approximation poop ie known ta he |? ~ presa| = |2.00000000 ~ 2.003987141 = 0.00398714. This is about 1000 times larger than |pico1 — pion! and it shows that closeness of consecutive terms does not guarantee that accuracy has been achieved. But itis usually {We only criterion available and is often used to terminate an iterative procedure. Program 2.1 (Fixed-Point Iteration), To approximate a solution to the equation | + = ie) sarin wit te iat 055 py a ‘function Oe,pyorr,P]-fixpt (@,p0, tol mast) f laput ~ 6 ie she sceration function input ae a string “4° ~ pO is the initial guess for the fixed point i tol is the tolerance % ~ maxi is the maximum nunber of iterations Yourput ~ k io tho nunber of iterations that vere carried out = p 4a the approximation to the fixed point x = err is the error in the approximation % ~ P contains the sequence {pn} PAD p05 for ke2:maxt POO=feval (g,PQ1)); erreabe(P(i)-P(z-2)) relerr-err/(abe(P(R))+eps); PPP: Af (arrctot) | (relerretol) ,breaksand rimim number of ‘iterations excceded’) ‘Remark. When using the user-defined function fixpt, it is necessary to input the Mofile g.m as a string: "(see MATLAR Appendix)50 CHAP. 2 THE SOLUTION OF NONLINEAR EQUATIONS. f(x) = 0 Exercises for Iteration for Solving x = g(x) 1. Deteemine rigoroly if ech fnetion has & nique ied point on the given interval (llow Exampie 23) (a) 60) =1~7/40010.1) ) 9s) =2-F on 0,1] (@) ex) = Ix on f05.5.21 2. laveigate the nature ofthe ined pon iteration when aya, (@) Solve g(x) = + and show that P = 2and P = 4 are ined point. {b) Use me staring vale py = 1 and.eompute pi, po, and ps (©) Use the sting vale po ~ 3.8 and compute ps, ps, ad ps (@) Find the errs Ei and relative ers Re forthe vals pin pats) ad (6) (©) What conclusions can be deawn from Theorem 2.3? 4 raph (2) Be ne y = x, andthe gWven xed point # onthe same coordinate system. Using the given starting value po. compute and pa, Consist figures similar to Figures 24 and2.5. Based on your graph, determine geometrically if fixed point tration converges. (@) £02) =6-+2)"?, P=3,and po =7 2s.P 23P (8) ghey 42842, Pe Dando = 4, Let g(x) = x* +1 ~4, Can Bxed-point iteration be used to find the solution(s fo the 22 equation x= a(x)? Why? 5, Let gx) = xeostx), Solve x = g(2) and find all he fixed points of ¢ (there ae in finitely many) Can fxe-pin iertion be weed to fnd te olin) othe equation x= g(a)? Why? 6 Suppose that (x) and («are defined and comtinuouson (a6), Pos pa € (a0) and p; ~ a(po) and Ps ~ gps). Also czar tht there exit 3sorataat sch tage) = K. Show that [p> ~ pti < K'py ~ po. Hint. Use the Mean Value ‘neorem 7. Suppose that g() ang) ase continous on (4,6) and Maa jy’G)| = Lou dis interval. Ifthe fixed point P and ihe inital approximations py and ps lien the itera (2.2), the show that p, = gpa) implies that Ei] =P pil = IP ~ po! = IEol once sateen (1) of Theorem 23s etablished (local divergence) 8 Let 2) = ~0 00014? + x and ao = and consider fixed-point iteration, (a) Show that po > pa >> Pa > Past >> (©) Show mat py > O fora SEC.2.2 BRACKETING METHODS FOR LOCATING 4 ROor st (©Since the sequence {9} is decreasing and bounded below, it has limit. What is the init? 9. Let g(x) = 0.5x + 1.5 und pp = 4, and consider fixed-point teravon, (@) Show that the fixed point is P = 3 (©) Show that |P ~ py = |P — py/2 for (©) Show that 1# — pal = 1" — poi/2" torn 10, Let g() = 1/2, and consider fixed-point iteration (a) Find the quantity p11 pal/|pvatl (©) Discuss what will happen if only the relative error stopping criterion were use in Program 2, 11, For fixed point iteration, aiseuse why iti an advantage to have (P) Pe y i Algorithms and Programs 1, Use Program 2.1 to approximate the fixed points if any) ofeach function. Answers cane a yr of eae Zaeion ai tne that clearly shows any fixed points. (a) gle) =x ~3e8 28 2 ©) g(a) = corcsinte)) (©) etx) sintx + 0.19) er @ a Bracketing Methods for Locating a Root Consider afaniliae topic of interest. Suppose that you save money by making eeu monthly deposits P andthe annual interest rate is 1; then te toal amount A after depositsis (Digs See = +P (i+ 5) +P(i+5) +. sees ‘The fistterm on the right side of equation (1 isthe last payment, Then the nextlast payment, which has eared one period of interest, contibutes P (1+ 45). Te second- ‘from-last payment has earned two periods of interest snd contributes P (1+ 4)*, and soon. Finally the last payment, which hac earned interest for N 1 periods, connvtes (14 4)""" coward the total. Recall chat the formula for the sum of the N terms of a geometric series is 22 Cuar2 THESOLUTION oF NontaNean EQUATIONS fis) =O ‘We can write (1) in the form, e(ia(ep)a(- and se the sobstttion x= (1+ /12)n (2) t0 obtain a+" ‘This canbe simplited to obtain the annuity equation, P ry" ® se ((4d)"=) ‘te following example uses the annuity-due equation and requires @ sequence ot repeated calculations t find an answer A Example 2.6, You save $250 per smo fr 20 years and desie dha he total value of all payments and interest is $250, 000 atthe end ofthe 20 years. What interest rate 7 is Deeded to achieve your goal? If we hold N = 240 fixed, then A is a function of I alone: that A = ACD), We wil tart with two guesses, fo = 01? and fy = 013, and perform sequence ofcaleuations to narrow down te final answer. Starting with fp = 0.12 yields 1) 250 oy Since this value is litle short ofthe goal, we next ry fy = 0.13: a0 gia (+82) viare(('=2) -!) =a ‘This is lite high, so we ty the value inthe mide fy = 0.125: 250 0.125)" 40.125) aisha ((1+ ) -1) “This i wpain high ad we conclude that de desived rate Hes iu the interval (0.12, 0.12 164, 623. The next uss isthe midpoint Is = 0.1225: 250 0.1225) a = 1) =255, 803. nee) = ras (1+ 8) —) asa This is high and the interval is now narrowed wo [0.12, 0.1225]. Our las calculation uses the midpoint approximation fg = 0.12125: 5 1) =2sisie Sec.2.2 BRackETING Merilops Fok Locarina a Root s (ofa (ofa) Pa éfle) ee Nite a > fo) ~——_+ fbn ero (2 11 7.a) and fey nave “opposite signs then squeeze from the right. (80 fee) and fb) have onposite sins then squeeze from the left ‘igure 2.6 The decision process forthe bisection process unter iterations can be done to obtain as many significant digits as required The purpose ofthis example was to find the valuc of { dat produced a specified level L of the Funetion value, tha is to find a solution to AC) = L. Ie is stand practice 10 place the constant Zon the lft and solve the equation A(Z) — . Definition 23 (Root of an Equation, Zero of a Function). Assume that f(x) 18 a continuous function. Any number r for which f(r) = Qis called araot ofthe equation I(x) = 0. Also, we say ris azero ofthe function f(x). 4 For example, the equation 2x? + $x — 3 = 0 has two real roots ry = 0.5 and r2 = 3, whereas the corresponding function f (x) = 2x7-+5x—3 = (2x—1)(r+3) hhas two real eros. 7) = 0.5 and re = —3 ‘The Bisection Method of Bolzano In this section we develop our fist bracketing medhod for using a ze ura continuous function. We must start with an initial inerval fa, b], where f(a) and f(b) have ‘opposite signs. Since the graph y = f(x) of a continuous function is unbroken, it will crosethe x-axis at zero.x = r that lies eomewherein the interval (sce Figure 2.6). The bisection method systematically moves the end points ofthe interval closer and closer together until we obiain an interval of arbitrarily small width dhat brackets the zero. ‘The decision sep for this process of intorval halving is fist to choose the midpointS4CHAR.2 THE SOLUTION oF NONLINEAR EQUATIONS f(x) = 0 (¢= (@-+0)/2 and then to analyze the three possbiltes that might aise: a If Flay and fc} nave opposte signs. & zero Hes in (a. c 6) If Fle) and f(b) have opposite sign, 42270 lis in [e, 5) 6) If Fle) =O. then the zer0 is either ease(4)or(5) occurs, we have found an interval haf as wide asthe original intervel that contains the root, and we are "squeezing down on it” (see Figure 2.6). To wontinue the process, selabel the uew salle inteval fu b] and tepeat the process until the interval is as small as desired, Since the bisection process involves sequences of nestcd intervals and their midpoints, we will use the foliowing notation to keep wack sof the details i the proves jag, Bo} the starung interval and cy = 85% is the midposat. 1-6 ]is the second interval, which brackets the zero, ané i its mudpoint; (7) helinterval (a1, by] is Hallas wide a Lao, Boh tier ariv:ng atthe nth anierval ft, By Which Drackers r and "as Mudpoint 0 the interval [ac boa] i constructed, which also brackets r and is half {8 wide (09, Dal Tt is Heft as an exercise forthe reader to show thatthe sequence of left end points is imereasing and the sequence oP right end puis 5 dcvieasing, hat i, 8 Says eager Se Sb se Sh ho were og = BH, an Flor P Fag < 0.160 lana OF Wdgets Beet = Lens Bad f08 all (9) Laat. bye tT ‘Theorem 2.4 (Biseetion Theorem). Assume the: f € C[a, b] and that there exists a vumberr © fu.d] such that f(r) = 0. IF fa) and £6) have opposite signs’ and ‘er} Fepresents the sequence of midpoints generated by the bisection process of (8) and (9), then for n=0, 1, U0) rae < and therefore the sequetce (¢4]%iq converses 0 the zero x =r; that 1, wy Jim cy =r. Proof Since both the 7era sed the midpoint cx He in the interval [ap Be) the die- tance between cy and r cannot be greater than half the width ofthis interval (see Fig- ure 2.7). Thus «2 W-ols for all n Sec. 2.2 BRACKETING METWODS FOR LOCATING A ROOT s Figure’27 The root rand midooint cy of fs. by) forthe bisection meted (Observe that the successive interval widths form the pattern Itis eft as an exercise forthe reader o use mathernatical inductioe and show thet oy ees Combining (12) and (13) results in aay for all n [Now an argument similar 0 the one given in Theorem 2.3 can be used to show that 114) implies thet the sequence (cy converges tar aod the proof ofthe theorem is complete : Example 2.7. Te funciion n(x) = x sinix) occurs mn We study of undamped forced ‘oscillations. Find the valve of x that ies in the interval (0.21. where tae function takes on the value x) = 1 (the function sin(x) is evalvaed in radians) ‘We use the bisection method o finda zeroof the function f(a) — sina) 1, Stating with ap = Oand by = 2, we compute #0) = -1.000000 and f12 sias9s 50 a root of f(1) — 6 Nes in the interval [0.2]. At the midpoint

0, break,end naxiei+round( (Log (b-a)-log(delta)) /1og(2)) for ket:maxt exCaeh) /25 yerfeval (fc); Af youn yarye: if bra < delta, break, end end c=(aeb) /25 errsabs(b-a) yerfeval (fc),60 Ciiap,2 THE SOLUTION OF NONLINEAR EQUATIONS f(x) = 0 | Program 2.3 (False Position or Regula Falsi Method), To approximate & root of | the equation <2) = 0 in the interval (a. 61. Proceed with the method only if f(x) | iscontinuous and f(a) and f(b) have opposite sign. j function (c,err,yc]=regula(f,a,b,delta,epstlon,maxt) ‘input - f is the function input as a string *f° % 0"), break, for kel:maxt xrybe(o-a) /(yb-ya) cxbrdx; acec-a; ye=teval(t,c); Af yors0 break: elseif ybeyooo axruin(abs(ex) Af abe (dx) *-__ In Exercises 1 and 2, find an approximation forthe interest rate that wll yield the total annaity value A st 240 monthly paymens P are made. Use the wo staring values for 7 and compute the next three approximations using the bisection medhod 1. P =$275, A =$250,000, fy = 0.11, = 0.12 2, P $325, A =$400, 000, lo = 0.13, = 0.14 3. Foreach function, find an interval (a,b so that f(a) and f(b) have opposite signs (@) suy—e 2-4 () for) = costed + 1x (©) fa) =Iny—S+2 (Joys ~ oe $28 tn Exereincs 4 throiagh 7 start wi co.crrea. andes fs, dy] and use the false posit metal to cumple lay, bo] = [-2.4, -1.6] {4 69] = 108, 6} « 8, ap, 9] = (9.24.0) 1. x2 — 108 +23 = 0, ap, bo] = (6.0, 638] ‘8 Denote the intervals that arise in the bisection method by (a9, hs), [01 i}, (ey. (2) Show that ay = a) +++ S dy 3. then fap) f(09) <0. Thus, on the imirval (a9, 25) the bisection method will converge f0 one ofthe hee zeros. If ag < 1 and bo > 3 are selected such that cy = S542 isnot equal to 1,2, oF 3 for any n > 1, then the Disestion method will over converge to which <70(5)° Why? 14, 15, Tf polynomial, f(0), has an old nomber of real rer in the interal [g, Po), and cach of the zeros is of odd multiplicity, then f(a) (ba) < 0, and the bisection into emerge one oe Hm aid ch (snot equal to any ofthe eros of F(x) for any n > f, then the bisection ‘method will never converge to which zero(s)? Why? Algorithms and Programs ia apyronimation aceurew 10 vine pace} fin te inate cate Fit wi Yield oa annuity valu of $50, 000i 240 monthly payments of $300 are made 2. Consider spherical ball of radius ¢ ~ 15 om that i constructed fom a variety of ute oak that has a density of p = 0.710. How much ofthe dal (accurate to 8 decimal places) wil be submerged when itis placed in water? 3. Modify Programs 22 and 23 to ouput « matrix analogous to Tables 2.1 and 22 respectively (ie, the fist row ofthe matix would be [029 cp & feo) 4. Use your prgrunm from Problem 3 « approximate the Hee smallest poste Toots ofx = tan) (accurate to 8 decimal places). 5 A unit phere is cut into two Segments by a plane. One segment has thes times the Yolume ofthe other. Determine the distance x of te plane rom the centr ofthe sphere acct 1 ema places 3 Initial Approximation and Convergence Criteria ‘The bracketing methods depend on finding an interval [a,b] so that f(a) and (6) have ‘opposite signs, Onve de interval ius been found, oo quatter bow lage, re iesations ‘will proceed until a root is found. Hence these methods are called globally convergent. However if f(r) = O has several roots in fa, b), then a different starting inferval mist be used to find cach root, I is not easy to locate these smaller intervals on which f(s) changes sign. Tn Section 2.4 we develop the Newton-Raphson method and the secant method for solving f() — 0. Both of these methods require that a close approximation to the root See.2.3 INITIAL APPROXIAEATION AND CONURROPNCR CRITERIA a be givento guarantee convergence. Hence these methods ae called Locally converges ‘They usually converge more rapidly than do global anes. Some hybrid algorithms start with a globally convergent method and switgh fo a locally convergent method when ‘the iteration gets close f0 a root Ifthe computation of roots is one part of a larger project, then a leiarely pace is suggested and the first thing to do is graph the function. We can view the graph F(X) and make decisions based on what it 10OKS like (concavity, slope, oscillatory ‘behavior. local extrema. inflection points, etc). But more important, i the enordinates ‘of points on the graph are available, they can be analyzed and the approximate location ofsvots determined. These approximations can then be used as starung values in our root-finding algorithms. ‘We mast proceed carefully. Computer software packages use graphics software of vatying sophistication. Sappose thet a compulcr is used w yrapit y = fx) om (a, 2) ‘Typically, the interval is partitioned into N + 1 equally spaced points: @ = xo < x) Oand X41 C48 not Necessary r0 USE the difference quotients. and it will suffice to check to see ifthe differences ve ~ v1.1 and yy) — yy change sign, In this ease, xe is the approximate root. Unfortunately, ‘we cant guarantee tha this starting value will produce « vowvergent sequence. IT de graph of y = f(x) basa local minimum (or maximum) that i extremely close to zero, then itis possible that x, will be reported as an approximate root when f(x4) * 0, although xx may not be closc to 8 root.64 CHAP 2. THESOLUTION OF NONLINEAR EQUATIONS f(s) = 0 “Table?.3._Finting Appeonimate Lovatone for Roots Funcom values | __ Differences im y : Siu tags 4 % murm | imymose “O98 13 e201 1463 | changes sign 4a.) “06 1028 a9 m3 Hiss =0183 _| fangs sign ear, oo 1.000 “0368 03 037 0381 a | 0230 mos) 09 oI 06 | cng sign ear, 12 | 0.088 oss gure 210 The graph ofthe cx bie polynomial y Port! eample 29, Find the approximate location ofthe rnt of 1 — «2 = eg T= onthe itera [-1.2, 1.2] Forillasration, choose N = 8 and lok at Table 23 “Te ee shcise orctakieratnmse ~1,05,~0.3, at 09, Becasne f(x) changes sign on the inerval [=1.2.~03]. the value ~108 is an approximate roe; indesd FC.) = ~0.210. ‘Although te slope changes sign near ~0.3, we Gd that /(-0.3) ~ 1.183; hence =0.3is not neara oo, Finally, the slope changes sign nar 0.9 and/(09} = 0,019,500 {5 an approximate root se Figure 210) . INITIAL APPROXIMATION AND CONVERGENCE CRITERIA 6 Figure 2.11 (2) The horizontal comergence band for locating a solution to Ie) * Figure 2.11 (b)The vertical convergence band for locating a solution to Fx) Checking for Convergence _Agraph can be wed to ae the approximate locaton ofa rot, but an algorithm must be ‘sed fo compute a value py that Is an accepable computer solution. Heaton i often ‘zed to produce a sequence (ps) that converges oa root p, anda termination eitrion cr rtegy mt he Geigned ahead of time an tha the compmter wil top hen an Accurate approximation is reached. Since the gosl ito sve f(x) = Othe final value Ps Should have te propery that |/ pe} <€ ‘The usercan saply a tolerance value « forthe sizeof fp and shen an iterative process proces points Py = (pu, J(pe) unl the ast point Pies inthe horizontal ‘and bounded by the lines y te and y= ~e, as alown in Foguce 2.110). This esiterion is Useful ifthe user is tying to Solve A(z) = L by applying a oot-fnding6 CAP.2. THE SOLUTION OF NONLINEAR EQUATIONS f(x algorithm tthe function f(x) = hla) ~L. “Another termination cierion involves te abscissis, and we can try tod the sequence (74) in cnverging I we dew the vera lines x = p+ 8am! + fon each side of x = p, we could decide o stop the eration when the pst between these two verti lines, as show in Figure 211 (0). The later witcriom is often deste ut itt implement Because i volves che unknown solution p. We adap this idea and terminate further calculatios when the consecutive trates py and py are suflcentl close rif they agree with significant ist Sometimes the user ofan algorithm will be satisfied if ry ~ 75-1 and other ies when (Pq) * 0. Correct logical reasoning is required to understand the cons gpences, If we require that jem p) = 5 and IF Pn ~ «. the put Px wil Tocated inthe retangulr region about the solution (p, 0), sb shown in Figure 2.12( If we stipulate that py ~ pl = 8 oF if(py)| < e:the point Py could be loca anywhere in the region formed bythe io of the horizontal and ver} spe shovin in Figure 2.12(b) ‘The size of the tolerances 8 and are crcil. Tf the to races are chosen too small ilraton may continue forever. They shouldbe chose ston 100 mes larger than 10M where Af isthe umber of decimal digit in he computer's Noting point numbers, The closeness ofthe abscissas is checke® th on: of te entena IPe~pe-il <8 (estimate forthe absolute eror or 21pm — Prot! <4 (estimate forthe relative error). ial ¥ Ipe-1l “The closeuess uf tu urdinte is usually checked by Lf (Pa}i <€ ‘Tronhlesome Functions [A computer solution to f(a) = 0 will almost always he in esrnr de sso and/or instability in the calculations. If the graph y = f(x) is steep nev (p.0), wen the root-finding problem is well conditioned (Le. a solution with sex significant digits is easy to obtain’. Ifthe graph v = f(x) is shallow neae (0). then the root-finding problem ill conditioned (.., the computed root may have ya few siguificen digits). This oeurs when /¢«) ls a muldple ro ap. This scussee further in the next section ‘SEC.2.3 INITIAL APPROXIMATION AND CONVERGENCE CRITERIA, o Figore 212. (e) The rectangular region defined by [x ~ p) < 6 ANDIy| <. yan) Figare212_ (&) The unbounded region defined by [x ~ p| < 8 OR [yl <«68 Cuar.2 THE SoLUTION oF NoNLiNeaK Equarions f(x) —0 [Program 2.4 (Approximate Location of Roots). To roughly estimate the loca. tions ofthe roots of the equatton f(x) = 0 over the interval {a,b}, by using the equally spaced sample points (xs, /()) and the following criteria: © Gr0ON <0.0r i) Lye) <€ and Oe — DOH ~ 9) <0. ‘That is, either f(x4-1) and f (4x) have opposite signs or | f(x] is small and the slope of the curve y = f(s) changes sign near (xa, f(a). = approot C,epettou) = £ ip the object fumction caved ac an M-file aanod fm ~ K is the vector of abscissas = epsilon is the tolerance 4 Output - Rie the vector of approxiaate roots Yet: yrange = nax(¥)-ain(); spsaion2 = yrangerepsilon; nelength(X): Xturs)=x60) 5 Yonts)=¥@); for ke2:n, Af YOCDHYG)<0, weet; Row end fe CVO) Veet) de C#) YO) if (abs(¥(K)) < epsilon2) & (s<=0), ment Rem =X) ; end ond (e-1)#X009/25 Example 2.10, Use approot to find approximate locations for the roots of f(e) = sincos(x?) in the imterval [-2, 2). Furst save f as an M-file named fm. Since the results ‘willbe used a iil approximations for aroot finding algorithm, we will construct X20 that the approximations will be accurate to 4 decimal places. 001 >>approot (X,0.00001) 1.9076 -1.0765 -1.1626 4.1625 1.6765 1.9075 See 22. TWITTAL AppRoxtwaTio aNn CONVERGENCE CRITERIA o Comparing the results with the graph of J, we now have good initial approximations for ‘ane of cert Snting algorithms . Exercises for Initial Approximation In Exercises | through 6 use a computer or graphics calculator to graphically determine ‘he appronicnate location of the ot: of Ge) — 0 in the given interval. In each case, 0 such tbat the sequence {4}? defined by te iteration _ fn Fon yal appronizuaton py © Lp — 8. p+ 81 1) defined hy foros for o p B(P1) = Pea will comenge to p for Remark The function ¢ fey 6 aly Fm is called the Newion-Raphson eration function. Since 1(p) 0, its easy to see ‘that g(p) = p. Thus the Newion-Raphson iteration for finding the root of the equation (2) — Os accomplished by finding a fixed point ofthe function g(a), derstanding why 7g needs to be close to p of why the continuity of "¢x) is essential (Our analysis starts with the Taylor polynomial of degree n = 1 and its remainder term: The Sain Pigare 2.19 aes wo ee a we Fox ~ po}? o Feo) = Spo + F (pode poy + LOS ‘wher lies somewhere between pp and.x, Substiuting x = p into equation (6) and using the fat that fp) = O produces ie ad a 0 Spo) + F (perp ~ py) + HAE POP TT ny is close enough to the last term on the right side of (7) will be small com: ‘ered to the sum of the lirst two terms. Hence i: can be neglected and we can use the approximation 0) US J (po) + Swot? ~ Bod. So’sing for p in equation (8). we get p * po ~ f(Pod/f"(Po)- This i used to define the rex approximation p) tothe root Sow 0 1 = po Le, 2 Pe PF) When ps is used place of pc in equation (9) the general rule (4) established. For ‘most applications this is ull that needs to be understood. However, to fully comprehend72 CHab.2. Tae SOLUTION OF NONLINEAR EQUATIONS f(x) =: what is happening, we need to consider the fixed-point iteration function and apply Theorem 2.2 i our situation. The key isin the analysis of g(x): POF) =fOs"w _ fos" Cor Fon? By hypothesis, (p) = 0: thus g’(p) — 0. Since g'(p) = Oand g(x) is continuous it ‘possible to find a8 > 0 so tha the hypothesis [g"(x)| < 1 of Theorem 2.2is stisied on (p — 5, p +8). Therefore, a sufficient condition for po to initialize a convergent sequence {pa)j2y. which convergesto a root of f(s) ~0,is that py € (p 8, +8) and that 8 be chosen o that Wie fo) FOF dale G9) AL forall ve Gp pas) + Corollary 22 (Newton's eration for Finding Square Koots). Assume that A > U isa real miner and et pp > Ohe an inital approximation ta VA Define the sequence (paleo using the recursive rule 4 mat be for k= 1, 2, an a Thon the eequonce (pi }fiy converges to V/H tha i iby nce Pk — VA Outi of Prof. Stax with he Sncion f(x) = ~ Avan otc tht he oof thecquton © — A= Oare/A. Now ute f(s) andthe derivative f'n formula (3) and wate down the Newton Raphson eration formula a fo This fucmuta can be simplified to obtain 03) ao ‘When g(x) in (13) is used to define the recursive iteration in (4), the results formula (11D, Ie can be proved that the sequence that is generated in (11) will converge for any starting value py > 0. The details arc left forthe exercises . ‘An important point of Corollary 2.2 is the fact shat the iteration function 4s involved only the arithmetic operations +, ~, x, and /. If g(x) had involved the cal: culation ofa square roo, we would be caught in the circular reasoning that being able vw calculate the square root would permit you to recursively define a sequance that will converge to VA. For this reason, f(x) =x? ~ A was chosen, because it involved onls the arithmetic operations. 'S8C:2.4 NBWION-RAPHSON AND SECANT METHODS Example 211, Use Newton's squareroot algorithm ind 5 ‘Suing with pp-~ 2 and using formas (11, compu = 2482 was p= PEHSEB pasitttns Mm zzz a6 1 _ a associat pa = 23898 Sra2seosrs ae Further iterations produce pe ~ 2.236007978 lor k > 9, so we see that convergence accurate 10 nine decimal places has been achieved. . Now fetus tum w a fasniliat problem oun elementary physics aid see why de termining the location ofa root isan important task. Suppose that a projectile i fired from the origin with an angle of elevation by and inital velocity wp. In clementary the height y — 9G} ad de ise eFC ls vest fa neghevied ad tance traveled « = (0), measured in fee, obey the rules ay ty 6? amd ‘where the horizontal and vertical components ofthe initial velocity are v< = v9 cas(bo) and », = vo in(bp), tespectively. The mathematical model expressed by tne rules in (14) is easy 10 work with, but tends 10 give too high an altitude and too long a range for the projectte’s path. If we make the additional assumpyion thatthe air resistance is proportional w the velocity, te equations of moti: become as yas) = ten 4220 (1-8) 004 and a6 rerth=ty (1%), ‘where C = ni/k and & 18 the coetictent of aur resistance and m 1s the mass of the rrojectile A larger value of C will resait in a higher maximnm alftude and a longer ‘ange fo the projectile. The graph of a fight path ofa projectile when ar resistance is considered is shown ts Figure 2.14. This mproved unodel is more wealistic, but requires the use ofa root finding algorithm for solving f(1) = 0 to determine the elapsed time until the projectile hits the ground. The elementary model in (14) does not require & sophisticwted procedure to Gnd the elapsed time.14 CHar.2 THE SOLUTION oF NONLINEAR EQUATIONS (2 300} os 200 | a Pgare 214 Pat of a ysicuite 2) Wad GO 300TH * ae wesunce cone Table 24 Finding the Time Whee te Height f(t) 1s Zero 7 [en pam] To — ormrrio zane 874200041 ~n.onse7s 0.03050700| aairdct 0.900001 =a.09000100| - 74217466 ‘0000000 ‘.ove000 Bxanuple 212, A projet is red wil an angle of elevation ty = 45°. vy = vy 160 sec, and C = 10, Find the elapsed time un rapt and find the range. Using formulas (15) and (18, the equations of motion are y = f(0) = 4800(t 19) 3201 and x — v(t) ~ 16001 ~ e110) Since f(8) = 83.220972 and £9) — ~31.534367, we will se the ial guess py = 8. The derivative #(@) = 4€00°#!9 — 2320, and is Yale f'(po) = J (8) = 108322025 used in formula (8) wo get 3.2009 /200 Tie mone = a 797731010. ‘A sorumary ofthe calculation i ven in Table 2.4, ‘The value pg has eight decimel places of accuracy, and the time until impact is ¢ 8.74217466 seconde. The range can now be computed using r(t) and we get 718.1421 1800)= 1000 (1 -€ 84) we gszapyosoer . ‘The Division-by-Zero Error One obvious pitfall of the Newton-Raphcon method is the possibilty of division by zero in formula (4), which would occut if #"(p4-1) = O. Program 2.5 has a procedure SEC.2.4 NEWTON-RAPHSON AND SECANT METHODS B {o check for this situation, but what use isthe last calculated approximation p—1 in this case? It is quite possible that f (pp) is sufficiently clase to zero and that px ian acceptable approximation tothe root, We now investigate this situation and will, uncover an interesting fact, that is, how fast the iteration converges. Definition 2.4 (Order of « Root). Assume thet fx) and its denivatives (x) . Fe) are defined and continuous on an interval about x = p. We say that Fee) = O has arootof order M atx = p and only if a Jo) erp and J°P(p) £0. £0 A ot of order AY = 1 is often called a simple root, and if M > 1, itis called mulipie root. A root of vider M = 2 is sometines culled 4 double root, ad 50 09, ‘The next esult wil illuminate these concepts, a Lemuma 2.1. If dhe equation f(x) = Oh u root of onder Af at x = then there exists continuous function (x) so that F(x) can be expressed as the product as) Fee) =e pMats), where hip) £0. Example 2.3. The function f(x) = x ~ 3x +2 has a simple root at p = —2 and a double root at p = 1. This can be verified by considering the detivatives f(x) = 3x? — 3 and f"(2) = 6x. At the value p = ~2, we have f(-2) = O and f° in Definition 24; hence p = ~2 sa simple oot. For the value p ~ 1, we have £0) =0. 71) =0. and {"() = 6, 50M = 2 in Definition 2.4: hence p = 1 is double oot, Also, note that f(x) has the factorization f(x) = (x + 2)¢x— 18 . Speed of Convergence ‘The distinguishing property we seck i the following. If pis a simple oot of f(x) [Newton's method will converge rapidly, and dhe stumber of accurate decinal places (oughly) doubles with each iteration. On the other hand, if p is a multiple root, the error in each successive approximation is a fraction of the previous error. To make this precise, we deGine the onder af convergence. This is a measure of how sapidly a sequence comverges. Deftntiin 25 (Order of Convergence). Assume that (pylg converges to p and Set Ey = p— py torn & 0. If two positive constants A # Oand R > Oexist, and a76 Chat? THESOLCTION OF NONLINEAR EQUATIONS fx ‘Table 2.6 Newton's Method Converges Quidraically ot « Simple Root SBC.2.4 NEWTON-RAPHSON AND SECANT METHODS n ‘Table 26 Newton's Method Converpes Linealy at» Double Root then the sequence is said 0 converge 10 p with order of convergence K. The num- ber A is called the asymptotic error constant. The cases RK = 1.2 are given special consideration 20) IF R= 1. the convergence of {7} is called near. en It R = 2, the convergence of [paz called quadratic, a 1f R islarge, the sequence {py} converges rapidly to: thats, relation (19) implies that for large values of » we have the approximation \Fa-.i1 = AlFf. Foc example suppose that R = 2 and |Fn| ~ 10"; then we would expect that |Eysi| © A x 10"! ‘Some sequences converge ata rate that is not an integer and We will see thatthe order of convergence ofthe secant method is R = (1 + /5)/2~ 1.618033989, Example 2.14 (Quadratic Convergence at a Simple Root). Start with pp 1nd use Newton-Raphson iteration 10 find the root p = ~2 of the polynomial fx) = Sx 42, The iteration formula tor computing {74} 15 2phy-2 Bhs 3 2 Pe = etre sing Formula (21) to check for quadratic convergence, We get the valucsin Table25. A detailed look atthe rate of convergence in Example 214 will reveal thatthe error in each suecessve iteration is proportional to the square ofthe error in the previous iteration. That is, Ip pussi® Alp ~ pels where A * 2/3, To check this, we use 0.,000008589 and |p — py \- pal and it is easy to see that lp — psi = 0.000008589 ~ 0.000008621 ~ - (Exot ° 2 aon000000 ‘0323808528 ‘0400000000 "01476190475 0 | 7200000000 |" —0.096969097 ~0:200000000 o.sts15isis 2a era (gare | aceon | creme (| fitom | chemo | caimoms | Sua 2 | =2003s9011 | oonsssve22 | oonssseott 064202613 7 Laszssomo | -omisscon | -naszxsonn assis 7 1 | timer | Cateaen | Coonne | SSnoythe ¢ | Smee | $ | immom | stasis | tamen | oman Example 2.15 (Linear Convergence at » Double Root). Stan with pp = 1.2 and use NNewton-Rephson iteration to find the double root p = I of the polynomial f(x) = x? — Using formula (20) to check for linear convergence, we get the values in Table 2.6. Notice thatthe Newion-Raphson metho is converging to the double root, bu at slow ruc, The values of /(p4) in Example 2.15 0 wo 220 faster than he Vales of #'(pe). 50 the quotient f(70)/f" (ps) in formula (4) is defined when pe * p ‘The sequcnce is converging linearly and the eror is decreasing by a factor of approx. imately 1/2 with each successive teiaion, The following theorem surmmataes the performance of Newton's method on simple and double roots. ‘Theorem 2.6 (Convergence Rate for Newton-Raphson Iteration). Aseume that [Newton-Raphson iteration produces a sequence (9, that converges tothe r00t p of te function (x). IF pis simple roct, convergence s quadratic and 3) lEneil Brune for ifficiently large. Mp isa oot fends Bf caverns a ad 24) JE nil Hat \En| for m sufficiently large. Pitfalls ‘The diyision-by-7ero enor was easy to anticipate but there are ther difcuties that se nt 0 easy to spot Suppose thatthe function ie fa) = x2 ~ dr +; then the sequence (px) of real numbers generated by formula (4) will wander back and forh ‘rom left 10 right and not converge. A simple analysis of te situation reveals that $2) > O and has no rea root.TR Cup? THE SOLUTION OF NOMLINGAR EQUATIONS fis} y asa Figure 215 (a) Newion-Raphson iteration for f(x) = xe can proauce a arvergent sequence Sometimes the initial approximation pp is too far away from the desited root and s small and the tangent line tothe curve y F'00 #2) is nearly horizontal. For example, f f(x) = cost) and we seek the root p = /2 and start with py = 3, calenlation reveals that py = —4.01525255, p ~ —-4,85265757,..... and (pal will converge toa different root ~3x/2 ~ ~4.71238898, ‘Suppose that f(x} 1s positive and monotone decreasing on the unbounded interval (a.00) and po > a: then the sequence (} might diverge to 400. For example, if fle) = xe and po = 2.0, then 19.723549434, 339333333, py and (m} diverges slowly 19 40 (see Figure 2.15(a)). This particular fanetion has ‘another surprising problem, The valve of f (2) goes to zero rapidly asx gets large, for example, 7(p1s) = U.0OXKQOSI6, and its possible that pis could be mistaken for root. For this reason we designed stopping criterion in Program 2.5 to involve the relative enor 2p, pal/(Ipel+10~), and when k = 15, this value is 0.106817, so the tolerance 8 = 10 § wall help guara against reporting a tlse root. Another phenomenon, eseling, accu when the tems in the sequence py} tend to repeat or almost repeat, Forexample, if f(z) = x—x~3 and the inital approximation pa = 1.961538, py = ~1.147176, pg = -0.006579, P= 1.961818, py = ~1.147430, and we are stuck in a cycle where pes ~ px for k = 0 1, ... (see Figure 2.15(b). But if the starting value py is culicietly close to the root p ~ 1.671699881, then (px) Sec. 2.6 NEWTON-RAPHSON AND SECANT METHODS ” faces pe Po 1 os 20 =} = scan produce a eyeic sequence Fo arcane) Figure 218 (©) Newton-Raphinn iteration for fx) = atctan(x) can produce a divergent oscillating sequence. conveiges. If py — 2, the sequence converges: pi — 1.72727272, pz = 1.67369173. ps = 1,671702570, and py = 1.671699881 ‘When |g'(x)/ > 1 on an interval containing the root p, there isa chance of di- vvergent oscillation. Por cxample, let f(r) = arctan(s); then the Newton-Rephson iteration function is g(x) = x — (1 +2*)aretan(x), and g(x) = —2x arctan(x). Irthe starting value po = 1.45 is chosen, then 1889109054, pi = -1,950263297, pr = 1.845951751, ps te cee Figure 2 15(¢)). Rut ifthe starting value is sufficiently close tothe root p = 0,80 Cuan? Tue SOLUTION oF NONLINEAR EQUATIONS f(x) Figure 216. The geomet contistion af fr these. cat method. 0.5, then a convergent sequence results. If p pi = ~0.079559514, pp = 0.000335302, py = 900000000, “The stations shove point tothe fact that we must be hones in reporting an answer. ‘Sometimes the sequence does not converge. Itis not always the case that after V ‘erations a solution s found. The user ofa root-inding algorithm needs to be warned ofthe situation when @ root is not found. If there is other information concerning the context of the problem, then itis less likely that an erroneous root will be found. Someumes f(x) has a detlaie interval in which a root is meaningful. If knowledge of the behavior of the function or an “accurate” graph is available, then itis easier to choose po ‘The Secant Method ‘The Newion-Raphson algoritnm requies the evaluation of two Functions per iteration, ft pe-1)and £1. Traditionally the calculation of derivatives of elementary func: sions could involve considerable effort. But, with modern computer algebra software packages, this has becoue iss of an issue. Still many Functions have uonelemeatary forms (integrals, sums, ec.) and itis desirable to have @ method that convertes almost as fast as Newton's method yet involves only evaluations of f(x) and not of f(x) ‘The secant method will require only one evaluation of (x) per step and ata simple root has an onder of convergence R = 1.618033989. Iris almost as fast as Newton's ‘method, which has order 2 ‘The formula involved in the secant method is the came one that was usod in the regula falsi method, except that the logical decisions regarding how to define each succeeding term are different, Two intial points (po. f (po)) and (pi. F(pi)) near the point (p,0) are needed, as shown in Figure 2.16. Define pe to be the abscissa Se. 2.4 NEWION-RAPHSUN AND SECANI METHODS, a ‘ble 2.7 Convergence ofthe Secant Method ata Simple Root mn * aaron Za. 5 soomony | ~ Seanonaoo | oor : oeume's | ago | oanaarres 2 ors, | atossivas | osrra0012 3 acute | ooarisosi | goaeusia | Gaaxanas 4 | =zonistioe | soowssscr | oamstiose | osnseainis 3 | “Bommazsey | oooomasis | womens? | onions? © | “zone | coomneoze | oooonn2 | =Poninoono | Sonne | oomooato ‘ofthe point of intersection of the line through these two points and the x-axis: then Figure 2.16 shows that pz will be closer top then to either pp or pr. The equation selaing pa, paid pv is fount by considering de slope 0s) mo Le and m= ©_L, Di= Po PP The values of m in (25) are the slope ofthe secant line through the fist two approxi ‘mations and the slope ofthe line through (p\, f(p1)) and (pz, 0), respectively. Set the right hand sides equal in 25) and golve for ps = g(t, po) and get _ £o00N.0.= po) F(p1) ~ f(p0) ‘he general term is given bythe Cwo-point tration formula Flpv(Pe= ps 26) p2= alpi. Po) on Secale on SEP =~ Top Tepid Example 216 (Secant Method at a Simple Roo), Star with pp = —26 sn Pin 7 4and ase the secant mediog to find hero p= =2 of he polynomial function fa—0 342 Tm this cae the iteration formula (27) is (P= 3 P+ 2 = Pat) Fi Pin —3PL+3pe-1 ‘isan be algebraically manipulated to obtain Piper + Papas ~2 oo Pret 4 * Fit PERI * ‘The sequence of iterates is given ia Table 2.7 . oa Pat = (PL PAD Pe (Pe Pe2 CuAR2 THESOLU (ON OF NONLINEAR EQUATIONS (2) =0 ‘There is a relationship between the secant method and Newton's method. For a polynomial function f(x), the secant method two-pount formula pk+i = §(Dk. Pk-) will reduce to Newton's one-point formula psi = gC) if ps is replaced by >. Indeed, if we replace px by Ps in (29), then the right side becomes the same asthe Fight side of (2) in Example 2.14. Proofs about he rate of convergence ofthe secant method can be found in advanced texts on numerical analysis, Let us state that the eror terms satisfy the relationship | Fen) re 27) G0) WBesal * [Eel ‘where the order of convergence is R = (1 + /5)/2 1.618 and the reation in (30) is valid only at simple roots. ‘To check this, we make use of Example 2.16 and the specific values and AFD 2S 2H = (2/3) = 0.778351205. ‘Combine hese and itis easy to see that lp = pst = 0.000022537 ~ 0,000021246 = Alp — pel!*"* Accelerated Convergence We comld hope that there. are rat-finding techniques that converge faster than Finearly when p isa root of order M. Our final result shows that a modification can be made to [Newton's method so that convergence becomes quadratic a a multiple root ‘Theorem 2.7 (Acceleration of Newton-Raphson Iteration). Suppose that the ‘Newion-Raphson algorithm produces x sequence that converges linearly to the root a= pof order Mf > 1. Then the Newton Raphson iteration formula en pe ‘ill produce a sequence (pg that converges quadratically tp. Sec.2.4 NewroN-RAPHION AND SECANT METHODS 8 ‘Table 28 Acceleration of Convergece ata Double Root ial ‘ n win on i iE? 7 Taanonoon | —oua9ae5e4 | ——aaanonoe 1 ocowoses | —o.nwosasis | —o.cosososos | oxesriasr8 2 owns? | “0000887 | “econonsae? a 100000000 omcceeo cxnon000 ‘Table29 Comparison ofthe Speed of Convergence Special Relation berween Method conseraons successive ear ems Bisection Ext © HER egal es Bet. AIL Secant method ‘Maltpe rot Bua © AIEG| ‘newom-penon naiape rt agi ~ aicel Seca met Simple root Beat © Ale SIE Newton Raphton ‘Simple rot Bess © AEE Accelerated Mapa rot Bens © AEE Newton-Rapbson ‘Example 2.17 (Acceleration of Convergence at a Double Root). Start with po = 1.2 spf essed Nevin Rapson tation to fn the double oot 9 = 1 of #0) a 3x42, Since Af — 2, de acceleration formula (31) becomes ‘and we obtain the values in Table 28 . “Table 2.9 compares the speed of convergence of the various root-finding methods that we have studied so far. The value ofthe constant A is different for each method.BA CAR, 2 TH:SOLUIION OF NONLINEAR EQUATIONS f(x) = ‘wen one mital approximation py and Using the iteration, | fee) Function [p0,eF¥ ky) -covton(E af pO delta, pollen maxi) Ainput ~ f is the object function input as # string '{” + d¢ is the derivative of f soput ax string ’af? 1 po sv the taivtel appronionrion £0 8 rere sf 7 = oita s= the tolerance for pO epsilon 6 te tolerance for the tuiction values y = atrt ia the anriaur numer of itarations = p0 is the Nevton-Rapheon approrieation to the zero + ie in the error extiaace for po and p3 Port (> pay? B Let f(x) 207 = 2e = 1 Start with py = 2.6 and py 9. Let fin) =A? 4 = 3. Slat wih py = 17 a ) and") do nt change to approximations (pe) #°(p) and fx) * 1"(p). Now use pat (a 0g 1. Lat 2) — 23 2-42. Set with y= LS and pi = —1.82 ; 1. Cube-root algorithm. Start with f(x) = x3 — A. where A is any real number. und Fae ea erive the recursive formula 419, Suppose mat A 1s a positive real number. (2) Show that A has the eepemeentation A = 672", where 1/4 £0,000062, and E3 = 0.000000, Estimate the asymptotic error constant A and the onder of convergence R of the sequence generate by the iterative method. Algorithms and Programs 1. Modify Programs 25 and 26 to display an appropriate eror message whet (i vision by zero oscurs in (4) or (27, respectively, or (i) the maximum aorber erations, maxi, exceeded 2 1s ofen tmuuctve wpa tetris We sequen genie by (4) atl (Ge. the second column of Table 24). Modify Programs 25 and 26 to display Sequences generated by () and (27), respectively ‘3. Modily Program 29 to use Newton's square-root algorithm approtmate eich the following square rots o 10 decimal places. (a). Start with pp = 3 and approximate V8 (©) Start wih po = 10 and approximate 5. {© Stat with p= andepprosimate VE 4. Mesify Program 25 tn xe the eube-oot algorithm in Exercise 11 1 approxi: cach ofthe folowing cube rots to 10 decimal places. (a) Start win pp = 2 and approximate 7°” (Stan with 7 = 6 nd approximate 200° (©) Start with pp = ~2 and approximate (~7)? Sec.2.4 NEWTON-RAPHSON AND SecaNr MerHons ry ‘5, Modity Program 2.5 0 use the accelerated Newton-Reptison algorithm in. Theo- rem 2.7 to find the root p of onder Mf ofeach ofthe folowing funchons (a) fla) = (x= 2)°. M=5, p = 2; start with po 0) fle) w sine, a 0; lar with po = 1 (©) (02) = Ue) Inte), M =2, p = Ly star with pp = 2 ‘6 Modify Program 2. to use Halley's method in Execcise 2 wo find the simple 2210 of Tha) 50 ~ Se4 2, sing pp = ~2.4, 7. Suppose thatthe equations of motion fata projectile sxe = FA) = 96001 — 615) — 408 arty = 24000 68, (@)_ Find the clapsed time und impact accurate 10 decal plas (h) in the range senate 18 decimal places 8. (a) Find she point on the parabola y 110 decimal places, (@)_ Find the point on the graph of y = sin(x — sin¢x)) that is closest tothe point 2-1,0.3) wocurate to 10 decimal places. that is eloset tothe point (3,1) accurate (6) Find the value of» a which the minimum vertical distance berween the graphs of fe) = x? +2 and g(x) = (4/5) — sin(x) oocurs accurate 1010 decimal pices, ‘9. An oper-top box is consiructed from 2 rectangular piece of sheet met measuring 10, by L6iaches. Squares of what size (accurate 6 0,008000000 inv} sll be a fon the comers if the volume of the box isto be 100 cubic inches? 10, A catenary isthe curve formed by a hanging able. Assume thatthe loiest point i (0,0); then the formula fr the catenary is y = C coshix/C) ~ C, To determine the sotenary tat goes thuough (La, D) we nwsi salve the equation b Ceusha/C) ~€ fore. (Show that the catenary through (2:10, 6) is y= 9.1889 cosh(x/9.1889) 9.1889. (©) Find the eatenary that parses trough ( >99 CHAt.2 THE SOLUTION OF NONLINEAR EQUAHIONS fC) 2.5 Aitken’s Process and Steffensen’s and Muller’s Methods (Optional) In Section 2.4 we saw that Newton's method converged slowly at a multiple root and the sequence of trates {pz} exhibited linear convergence. Theorem 2.7 showed how to speed up convergence, ht it depends on knowing the order af the root in advance Aitken’s Process ‘A technique ealled Aitken’s A? process can be used to speed up convergence of any Sequence that 1s Lnearly convergent. In order to proceed, we will need a definition, Definition 26, Given the sequence {pu define the forward difference Ape by my APe= Poti — pr for n>. Higher powers AK p, are defined recursively hy e Ahr Mea) Theorem 2.8 (Aitken’s Acceleration), Assume tbat the sequence {pq}%9 con verges linearly to the limit p and that p — py #0 forall n 2 0, WF there exists @ real nomber A with JA] < 1 such that a then the sequence (gp}$2 defined by (apn Opn nt Prva 2st + Po “ QP converges top faster than {pa} in the sense that oy a jee =o. ola pe Proof. We will show how to derive formula (4) and will leave the proof of (5) as an cxorste Sings the termi (3) ae approaching imi, we can write 6 EPs and ZOE A whem n is lage Pes P= Past ‘The reltions in (6) imply that o (= Past? © (O = Pas) (P~ Pa) See.2.$ ArrkEN's PROCESS AND STEFFENEEN'S AND MULLER’S Meroe 9 ‘Table2.10 Linealy Convergent Sequence (p4) Ee . ™ Fuamnp | eae 1 | csvessacco | casose7se | —oxecei6co9 2} ossszaoai2 | -o.nn1so1079 | -0.5s6119357 3 | osrs7030%5 | o.o1zssse0s | —o 57300209 4 | oSeonssea8 | -o.con0neees | 05396551 5 | os7it72149 | —ooowozssse | a seoisssss 6 | nscasaur | -oomasmas | —0 sensi ‘Table 211 Desived Sequence [ae) Using ‘Aiken's Process 7 ‘oser9e989 | o00015s6%9 2 | osenvsa | nownsss2 3 | Oserise36e 000016074 4 | oseriassss 005163 s Oo seriauos | onnnni62 6 | _ossniass2s | _aonnonas3s When both sides of (7) are expanded and the terms p? are canceled, the resus 2 Pae2Pn— Fj Tal gy for n=0, 1, bi PO pest — 2m + Oe The formula in (8 is used to define the term dq. I can be rearranged algebraically to obixin formula (4), which has less error propagation When computer calculations are mode . Example 2.18. Show that the sequence (p,.) in Example 2.2 exhibits linese convergence, and show that ihe sequence (ge) obtanea By astken's A? process converges faster. The sequence {9} was obteined by fixed-point iteration using the function ex) e-F and gariing with po = 0:5. After convergence has been achieved, the limit is P= 0.567143290. The values py aad yp ate given in Tables 2.10 and 2.11. Fos Husaliv, Whe valve of quis given by the calculation aan e Pi — 2p +P (-Woo1291 488 96530660 — OSETIA = 0,567298980. .92 CuAR? THE SoLUTION oF NONLINEAR EQUATIONS f(s) = 0 op s00 Figure 2417 The starting approximations po, ps, and py for Muller's method, and the Aifferences ho andy Altnougn the sequence {gq} mn able 2.11 converges linearly, it converges taster than Line} in the sense af Theowem 9-8 and neva Aitken’s method gives m hatter {improvement than this. When Aitken’s process is combined with fixed-point iteration the result is called Seffensen’s accelerasion. The details are given In Program 2.7 and inthe exercises. Mauller’s Method Muller's method is a generalization of the secant method, in the sense that it doe not require the derivative of the function. It isan iterative method that requires thre. starting points (po, f(po))s (Pi. f(p1)). and (po, f(p2)). A parabola is construct that passes through the three points; then the quadratic formula 1s used to find a 10: ff the quadratic for the next approximation Tt hac heen paved that near « simp oot Muller's method converges faster than the secant method and almost as fast a "Newton s method. The method can be used 10 1nd real or complex zeros of & tuncto and can he programmed to use complex arithmetic Without loss of generality, we assume tbat p2 is the best approximation to th uot and wonsider the parabola dough the dave stating values, shown in Figure 2.1 ‘Make the change of variable o and use the difference (1) ho=po-p2 and hy =pi~ pr Consider the quadratic polynomial involving the variable ¢: ay a +or+e. Sec. 25 AITKEN's PROCESS AND STEPFENSEN'S AND MULLER’S METHODS 93 Each point is used to obtain an equation involving a,b, and c: At ho: ah} + bho +6 = fos hy: akb+bhky +e= fi. =0: a +50 +e= fe 2) ‘From the third equation in (12), we see that «ay ‘ Substituting (13) nc the fist two equations in (12) and using the definition e andy = fi ereuls in he fneat system ahh +0h0= fo ak +h fone ay, Solving the linear system for a and b results in ne oh certo hg hoki = fg eo yh — heh ‘The quadratic formula is used to fiod the roots 1 1.22 08 (11) 16) Formula (16) i equivalent to the standard formula for the rots of a quadeatic and is better inthis case because we know that = fy To cnsure stability of the method, we choose the root in (J6) that has the smallest stsolote value. IF ~ 0, use the positive sign with the square root, and if B= 0, se ‘he negative sign. Then ps is shown in Figure 2.17 and is given by an P= pte ‘To update te iterates. choose pn and ps ta be the two values selected from amone ‘po, pty ps that lie closest to py (ie. throw out the one thats farthest away). Then re- place /Athith ps. Alihougi lt of auxiliary calculations are done in Malles’s method, ‘only requites one function evalustion per iteration. If Muller's method is used to find the real roots of (x) = 0, itis possible that ‘onc my encounter complex approximations, because the roots of the quadratic in (16) ‘might be complex (nonzero imaginary components) In these cases the imaginary com ponents will hve a small magnitude and can be set equal to zer0 so that the calculations Proceed with real numbers.94 Cuan 2 THE SoLUTION oF NONLINEAR EQUATIONS f(x) = 0 ‘Table 212 Comparison of Convergences near a Simple Root SeC.2.5 ARTREN'S PROCESS AND STERFENSEN'S AND MULLER's MeTKoDs 9S ‘Table 213 Comparison of Convergence Neat a Double Root Seca ‘Mall's News Seensen Seat Maters Newt's Stetensen t eed ested ‘ethos with Newion k smebod eho rch with Newton 3} 2 nanan | 2 eonnnnti —] 2 anno |= aan 2] heoonnon0 Taop00000 7 20600006 aa 1 | =2acoee00 | -2scon00000 | 2ovsis07s | —2076i9047 1 ¥ 200000000 ¥ 300000000 1103030303 iago30303 2 | “2howssines | “2aoooooom | poossasort | —aemsosort 2 | 1iseseiss8 1 200000000 1092356817 1082386807 3 aemens2 ioaszesze7 | 2000009509 o9es618i43 3 Vomsra7at anna Tomeannaia |p 6toness | -2onisiio% | -2oo0ssane2 | —2.000000000 2oomoioe2 ’ 10093854 103839922 rowzsma | ogeee4so23 5 | crows | CZeeomuzia = auaeet 3 1eozes0156 00002740 toosesssis | ogsazzi213 6 | —zon0000022 | 200000000 —Soo0002388 6 | toro aspesonais Loos32ss7s | 0999999193 7 | =om0oi00 | “2eccencono 1 | grapes? ose! 1.03607 0999993397 3 | toorsszi2e 1000000000 Hovossa0ss | c.eegeeno8 ’ ootsa757 Looosieans | Oseiae8 Comparison of Methods aa Steffensen's method can be used together withthe Newton Raphion fixed poiat func tion g(x) = x ~ f(2)/f"(x)._ In the next two examples we look at the roots of the polynomial fr) = x° — 3x + 2. ‘The Newton-Raphson function is g(x) = (x3 —25/Gx2 = 3). When this fmction is used in Program 2.7, we get the calculs- tions under the Reading Steffensen with Newton in Tables 2.12 and 2.13, For example, staring with pp = ~2.4, we woule compute as) pi = eo) = ~2.076190476, and a9) pa = a(p1) = 2003596011 ‘Then Aitken's improvement will give ps = ~1.982618143 Example 2.19 (Convergence near a Simple Root). This ic = comparison of method! forthe funetion f(1) = 2° — 3x + 2 near the simple root p = ~2. ‘Newton method andthe secant method for tus function were given in Examples 2.14 and 2.16. respectively. Table 2.12 provides a summary of calculations forthe methods. = Example 2.20 (Comvergence near a Double Root). This s «comparison ofthe aicdnes for the function f(x) = x? — 3x + 2 near the double root p = 1. Table 2.13 provides summary of calculations. 7. ‘Newton's method is the best choice for finding a simple root (see Table 2.12). AT double root, cither Muller's method or Steffensen’s method with the Newton Raphso' {ormula is a good choice (see Table 2.13). Note inthe Aitken’s acceleration formula (¢ that division by zero can occur as the sequence {p4) converges. In this case, the la ‘calculated approximation to zero ehould be used as the approximation tothe zero of J with the Newton-Raphson formula, is stored in a matrix Q that has maxct rows and ‘ee columns. The fst column of J contains the initial approximation to the root, and the terms ps, pe,» Pak, -- Generated hy Aitken's acceleration method (4) ‘The second and thid columns of Q contain the terms generated by Newton's method. ‘he stopping criteria inthe program are based on the difference between conseculve terms from the frst column of O. [ Promram 27 (Steffensen’s Acceleration). To quickly finda soktlon ofthe fixed | point equation x — g(x) given an initia! approximation po: where i is aseumed | that both g(x) and gc) are continuous, g"(x) < 1, and that ordinary fixed-point | eration converges slowly (linearly) 0p. function (p,Gl=ster# (f,af,p0,delva, epsilon wax) Minput — f i» the object function input aa @ string "£° i = af is the derivative of £ input as a string ‘df? x = pO ig the initial approximation to a zero of t % Getta ie the tolerance for pO % % ysilon is the tolerance for the function values = maxi is the maxisun number of iterations Yourput - p 42 the Stoffongen approximation to the zero * ~ Q is the matrix containing the Steffensen sequence Initialize the matrix R Rezeros(aaxt,3): RCL. Depo;9% CHAP.2 THESOLUTION OF NONLINEAR EQUATIONS f(x) = C for kel:maxi for j=2:3 ‘Denominator in Nevton-Raphson method is calculated ardenonsfeval (Af Re, -1)); Halculate Nevton-Raphson approximations if nrdenon==0 ‘division by zero in Newton-Rapheon method? brenk ele Rove, $)“R Ue, f-1)~foval RG, }-1))/medenan: end ‘WDenoeinator in Aitken’s Acceleration process calculate ‘aadenoz-R(x,3)-2+R(ie,2)4R (ke, 1) wealculate Aitkea’s Acceleration approximations Af aadenoa==0 Daiwinian hy zara in Aitkan’s Accaleration? break else Rk, 1)*R(K, 1) (RC. 2)-R(e.1))°2/aadenom; and end Wend program if division by zero occurres rf (ardencme=0) | (aadenom==0) break ond ustopping cricaria are evaluated erreabs(R(k, 1)-RG+1,19); relerr=err/(abs(R(x+i,1))“de1v8) yofaval (£ RU: 102 af (erzedelta) | (xelerr 0 dtacesare (b-2-48a%c) ; lee disco; end Wand the smallest root of (17) are dlacedisc: end 2ec/ feds P ‘Sort she entries of P to tind the t¥o closest to p Af abe(p-P(Q)) I and shrinks the vector when le|< 1. This is shown by using equation (10) Cbs tds tetehyl 2 AeXI = ay eR dt tad"? = loll. An important relationship exists between the dot product and norm of 2 vector. I ot sides of equation (10) are squared and equation (9) is used, with ¥ being replaced with. we have ay WP aaa eta x, If X and ¥ are postion vectors that lacate the to points (ej #3, aay) and (i 72..-.. 9») in N-cimensional space, then the displacement vector fom to Y 1s given byte ference a Y—X — (isplacement from position X to positon ¥). Notice that if a particle starts th: position X andl muves through the aisplacement ¥ ~ X, its new position is ¥. This can be obtained by the followin vector sum: CHa gt yexXs+@-4), Using equations (10) and (13), we cen write down the formula for the distance beaween two points in space. ay WX, hi \when the distance between points is computed using formula (15), we say that the points lie in N-dimensionat Euclidean space. (1-10? + 02-0) ++ GW am?)104 Char.J. THe SOLUTION oF Livear SyaTEMS AX = B Example 34. Let X = (2.-3,5,-1) and ¥ above are now ilostraed for vectors in 4-spoce. (6, 1,2, ~4). The concepts mentioned sum X+Y=@,-2.7.-3) Difference KAY =(-4,-4.3,3) Scalar multiple 3X = 6,-9,15,—3) Length Wx 41.9825 1 DY? = 39!?? Dot product X-¥=12-3410+4=23 Displacement from X to ¥ ¥-x=(4,4,-3,-3) Distance from X to ¥ (Y-XY— Ub + 1649491 — sol? is sometinncs useful ts ite vectors as eulamns istead of vows, Fur example, 6) X=] P| and Y= : cxw + dy By clowning ¢ and d appropriately in equation (17), we have the suum LX + 1Y, the difference LX — 1Y, and the scalar multiple cX + OY. We use the superscript" for transpose to indicate that a row vector should be converted toa column vector, and 4 1 08) Gta P| and [77 | = Grate, ‘The set of vectors has a zero element 0, which is defined by as) 0=(0,0,...,0) Theorem 3.1 (Vector Algebra). Suppose that X, ¥, and Z are N-dimansional vee tors and a and bare scalars (real numbers). The following properties of vector addition and scalar multiplication hol: (0) ¥4X=KeyY commutative property @) 0+¥=x+0 additive identity See.3.1 lntnopuction To VecToRS AnD Manices 2) X-X=x4+(-x)=0 additive inverse @3 (LLY) I Z=X |W |Z) associative propery 04) (a+ b)X = aX +bX distributive property for scalars 25) a(X4¥)= aX +a¥ distributive property for vectors (26) abX) = (eb) X associative property for scalars ‘Matrices and Two-dimensional Arrays ‘A matrix is a rectangular array of numbers that is arranged systematically in rows and wolumns, A matrix having M rows and N colurans is called an M >N (read “M by N”) rates. The capital letter A denotes a matrix, and the lowercase subscripted leter 0, denotes one of the numbers forming the matrix. We write on Azlawe forlsisM1sisM, where aus is the number in location (i,j) Ge, stored inthe ith ena an jth enum of the matrix), We refer to a; as the element in location (, j). In expanded form we. wate ax am y+ aw o rows | ain ain ay |e an ane omy omy wl The rows of the M x N matrix A are N-dimensional vectors; 9) vy “The row vectors in (29) can also be viewed as 1 N matrices. Here we nave sficed the MN matrix A into M pieces (submatrices) that are 1 x N matrices. Ti this vase we could express A as an M 2» 1 mautix consisting ofthe 1 > N row snatrices V;; thats, (ayia. escdin) for 1, 2s eens Mt (30) ={Vi We v Vue106 CHan3 THE SoLUHON oF LiveaR Systems AX: Similarly, the columns of the M x N matrix A are M x | matrices: an ayy ‘aw aay any oy On eee al Cael oun amy aun, Inthis case we could express A asa 1 x N matrix consisting of the M x 1 column smamices Cj: 2 aaler Cr €) Oy] Example 3.2, Identify the row and column matrices associated withthe 4 x 3 matrix The fourcow matricesare Vj =[-2 4 9},V2=[5 -7 1,¥s=[0 -3 8} and V4=[-4 6 —3]. The dee column matrices are ysl i “2 s 5 1 i ad Cs=| 3 -4 -5 Let A = layly cy and B = [bijiyyy be two matrices of the same dimension. ‘The two matrices Aand Bare said w be equal if aud only if each corresponding clement is the same; that is, G3) A=B ifand only if ay ‘The sum of the two Af x N matriogs A and B is computed clement by element, using the definition on A+B lai + bike. for < conte of ll zeros G9) 0 (Ohraw ; F i £ i : : forthe matrices 12 tans Aa]o7 3) md ae] = 3-4 oe Using formula (37), we obtain 24 6 9 a4 10) and =| 3-12 6 8. 721 ‘The linear combination 2A ~ 3B is now found “246 4-9] [4 =! za-ap—f ada wy i2}—ln 22 ee | ‘Theorem 3.2 (Matrix Addition). Suppose that A, B, and C are M x N matrices and p and are scalars, The following properties of mattix addition and scalar multi- plication bot 40) B+A~AtB ['7 34 Wa5]- tha ‘When an attempt is made to form the product BA, we discover that the dimensions ze rot compatible in this order beeauze the cas of Mase three-dimasional vetory ao dhe columns ofA are two-dimensional vectors. Hence the dot product of the jth row of B and the kth column of A is pot defined . IF it happens that AB = BA, we say that A and B commute, Most often, even when AB and BA are both defined, the products are not necescarly the same, ‘We now discuss how to use matrices to represent a linear system of equations ‘he hinear equations in (3) can be written as a matrix product. The coefficients ay; sem stored in a matrix A (called the coefficient matrix) of dimension M ~ Nand the ‘unknowns x, are stored in a matrix X of dimension N x 1. The constants by are stored ‘ma mamnx B of dimension Af x |, 111s conventional to use column matrices for both X and H and write an aa ay aw Pay Ph a a 2 ay {fre | |b = : : =|; |=2 ee aaa aun | | ay |] =| by amy ava amy + aun} {ew} [ow The matrix muliplication AY = R in (8) ig reminiscent of the dot product for ordinary vectors, because each element by in B is the result obtained by taking the dot product of row & in mamx A withthe column matnx X Example 3.6, Express the system of linear equations (5) in Example 34 as a matrix product. Use matsix mulipicaion w vesify ist [3 33} is he sedation of (5). v.12 0.200 y.av0)] [x1 2.3] % 0375 0500 o40o| |x| =|4% 0300 0300 900) [3] [28 ‘To verify that (4 3. 3]' isthe solution of (5), we must show that A(4 3. 3] [23 48 29): 0125 0.200 a400] [4] [05406412] [23 0373 0500 o600||3]=|15+15+18)=]as]. 0500 0.300 0.000113] }20+09+00! [29 2152 Cuan.3 The SoLUTION oF LiNkAR SYETEME A ‘Some Special Matrices ‘The Mx N’ matrix whose elen sion M + N and is denoted by sls ar all zero is called the zero matrix of dimen- 119) 0=(0lve¥. ‘When the dimension is clear, we use 0 9 denote the 2ero matrix ‘The Mentey matrix of order N is the square mawix given by 1 when ~ j, fe Ty =Wilven where &, i Med 0 whens 4. is the amiplcative identity, as illustrated in ube next exemple. Example 37, Let A bea? x 3 matrs, Then [2A = Aly = A. Multiplication of A on the left by F2 results in [fan en an] o | Ae au +0 anr0 on t0]_ 4 a) ‘Mulnpiication of A on the right by f3 reswtsin jt ag [t ee) feo) bean eo oro a ‘Some properties of matrix multiplication are given inthe following theorem. ‘Theorem 3.3 (Matrix Maltiptication). Suppove that c i¢ a scalar and that A. # and C are matrices such that the indicated sus and products are defined: then 02 (ABYC = CBO) associativity of matrix multiplication (3) [A= AI~A Identity matrix (14) ABE C)=ABTAC — Seftdisuibasive property (15) (A+ BIC—AC+BC sigh distributive propery (16) $(AB) = (cAVB = A(eB) scalar associative property The Inverse of a Nonsingular Matrix Ie concept of an inverse applies ro matrices, hu special auention must be Kiven.\1 NN snatrix A is called nonsingular or invertible if there exists an N x N wats such that un AB=BA SEC 37 PROPERTIES OF VECTORS Axo MAtRICES us Ino such matrix & can be found, A is sad to be singular. When B can be found and (17) holds, we usually write B= A“! and use dhe familiar elation as) AA 2 ACA if A is nonsingular Its easy to show that at most one matrix B can be found that satisfies relation (17) ‘Suppose that C 1s also an inverse of A (te, AC = CA = 1), Then properties (12) and (13) can he weed en abe CAC = (BANC = BAC B. Determinants ‘The deworminant of a square mateix A isa scalar quantity (cal number) and is denoted by det( A) or [Ai IEA iva N > N matrix ieee [eee tera its castomary to write jan aio ay OWA) Set er te Although he mation or determinant may Took fike a matin, its properties are oun pletely differen. For one, the determinant is a scalar quantity (real number). The definition of det(A) found in most Linear algebra textbooks isnot tractable for compu tation shen NV > 3, We will review how to wompute detevaiaants using the cofactor expansion method. Evaluation of higher-order determinants is done using Gaussian elimination and is meuioned inthe body of Program 3. MA — fay] isa se Late, we define det A) = 211. WA = ai heme here > 2 then let Af he the determinant of the N_ 1x N — 1 submatrix of A obtained by deleting the ith row and jth column of 4. The determinant Mf, is said to be the minor of a;;. The cofactor A,, of 2, is defined ws Ay, — (TIM, Then the determinant of an N x matrix A is given by 9 deta = Si, (ih v0» expansion)(CHAP. 3 THE SOLUTION OF LINEAR SYSTEMS AX = B w 2) deta) = ay ij (ith column expansion). a Appling fom 1, with = 3.40939 manic -[e ai) nits — ans, Te flowing expe lasts owt ase formulas (19) and (20) to recursively reduce the calculation of the determinant of en ‘AV x N maunx fo the calcUlation of a number of 2 x 2 determinants, Wwe see that det A Example 38, Use forms (19) wih / = 1 nd formula 20) with j= 2 tcl be Serna fe mui ael4 3 AL. L7-6 9} {sng ora (39) with = J we ebxin ee aa-al§ Jolt roft J (5-6) ON-364 74+ A= 35 =n Ung Sl 2) wth j = 2. cnn exay=-ai4 real s-col 2 tf =n. i The following theorem gives Sufficient conditions For the existence and uniqueness. ‘of solutions ofthe linear system AX = B for square coefficient matrices. ‘Theorem 3.4, Assume that A is an N > N mauix. The following statements are squivalent (2)) Given any N x 1 matrix B, the linear system AX = B has a unique solution (22) ‘The matrix A is nonsingular (Le., A~! exists). (23) The system of equations Aa’ = O nas the unique sotution (20 desta) #0, SEC.3.2 PROPERTIES OF VECTORS AND MarRices us ‘Theorems 3.3 and 3.4 help relate matrix algebra to ordinary algebra. IF state ment (21) is tue, then statement (22) together with properties (12) and (13) give the following line of reazoning: (2) AX=B implies AT'AK=A™'B, whichimplies X= A~)B. Example 39. Use the inverse mauix aif a ot oL5 andthe reasoning in (28) solve the ner systems AX = 3 1p] _ pe of QE}-B) [4 1P] 22) fos) Ler Rd 5 Ly” wz] Remark. In practice we never numerically calculate the inverse of a nonsingular rmatix ofthe determinant ofa square matrix, These concepts are used as theortceh “ool” to establish the existence and uniqueness of solutions of as a means 10 alge- bratcally express the solution of linear system (asin Example 39), 2. Using (25), we get xeate 1 Plane Rotations Suppose that Aisa 3x3 mauis awd = [x yz} ia3xt maxis; men ne product AU is another 3 1 matrix. This is an example ofa linear transformation, snd spplcations are found in the area of computer graphics. The matrix {i equivalent to the positional vecun B= (a, » <), whinh ecpreseus the eaves of point thre-dimensional space. Consider tee special mates: eco o 4 26) Re(a)=]0 costa) sina) |, 0 sina) — costa) cos(6) 0 | RO) eign e =sin(B) 0+ costs) cos(y) —sinty) 0 28) Re) =| sin(y)—costy) 0 o CeaM6 Cnar.d THeSoLUTION oF Linear SyStEMS AX = B ‘Table 31 Coordinates ofthe Vertices ofa Cube under Successive Rotations @ VERY WR RODE way ‘Ca, 00000, 67 ‘Donon, 6.9000, doo (0.0 {orm ox0ne7 9) {Denes O77 03335507 wo Caran. 0.70707. {0612372 0707107. 0353553) 0,01) (0.000000, 0.000000, 1)" (0.500000, 0.000000, 0.86025)" key {0.00000 1412814. 0) {0.00000 1.414214, 0.00000) ony {ornon o70707 (uanzsm2, 0707107 051272) wun Cornnion onuncr, ay (eoatasig etviiin1 2199197 ay {ocweene, 249 1) {0 Seen, 1.81214, 0866025" ‘Mhese matnces Re(a), Ry(B), and Kp(y) are used to rotate pomts about the and z-axes though the angles , A, andy, respectively ‘The inverkes are Ry (a), Ry(—A), and R,(y) and they rotate space about the x-, y- and zaxes through the angles —a, —p, and —y, respectively. The next example ilustates the situation, and further investigations are lef forthe reader. Example 2:10. A unit cube is situatd inthe fist ostant with one vertex atthe origin. Fir, rotate the cube through an angle /4 about the z-axis; then rotate this image through ‘an angle =/6 about the y-axis. Find the images ofall eight vertices ofthe cube. “The fist mtion ie piven hy the transformation con(§) sinc) 0] fs (aa cox i f oO TLE 0.707107 0.707107 0.000000] [= 0.707107 0.707107 0.000000 || y s.o0000 v.00 F-avaUNN | {Z| ven ‘Then the second rotation is given by : cost) 8 se mee yon gio |y . sin) 0 cost), 0.886025 0.000000 0.509000 ~| 2.000000 1.000000 0 00non0 | ¥ 0.500000 0.000000 0.866025, ‘The composition of the two rotations is : 0612372 0612372 0.500000) [x wen, (2)a(Z)e=| ororicr ar07107 0.0000 |s ~0.353553 0.353553 0.866025 [2 SEC.3.2 PROPERTIES OF VECTORS AND MATRICES 7 r if jag A © » © igure 82 (a) The onginal staring cube. (b) V = K.(7/4)U- Kotaton about the z-axis. (€) W = Ry(/6)¥. Rotation about he y-axis, [Nusnerical computations fr the coordinates of the vertices of the starting cube are gwen in able 3.1 (as positional vectors), and the images of these cubes are shown in Figure 3.2(8) trough () . MATLAB ‘The MATLAB functions det (A) and inv(A) calculate the determinant and inverse (fA is invertible), respectively, ofa square matrix A. ‘Example 3.1. Use MATLAB to solve the linear system in Example 36, Use the inverse matrix method deserted in 2. First we verify that A is nonsingular by showing that det(A) 0 (Theorem 3.4). 0,128 0.200 0.400;0.376 0.600 0.600;0.500 0.200 0.0001; soaee (a) 0.0175 ollowmng the reasoning in (25), the solution of AX = Bis X = AM'a. setesay(a)af2.2 48 2.0)? © ¢.0000 3.0000 3.0000 ‘We can check our solution by verifying that AX = B. >oveant 2.3000 44,2000 2.9000 .N18 Cixp 3. THE SOLUTION OF LINEAR SYSTEMS AX = B Exercises for Properties of Vectors and Matrices ‘The reader is encouraged to carry out the following exercises by hand and with MATLAB, 1. Find AB and BA forthe following matsees afd eb a] 2. Find AB and BL forthe following matrices ; af Beefs °. 3-2 3. Le A Rand be pety 3 ' a-B ae eis fo) Fed AyCand A080 (@) Find (ABy’ and BA’, 4. We use the notation A? = AAA. Find A? and B for the following matrices: anes aoe a-[$ oq} e-[ ss ae asa 5. Find the determinant ofthe following mates it exes ofiq ia! © [34 00. 6 Show that R,(a)R,(—a) Rea); (sce formula (26)). 7. (a) Show that Ryla)R, (A) = costa) ° sins) sin(B)sin(a) —cos(a) — os (f) sina) ~cos(a)sin(B) sinfad —cos(B)costa) (cee formulas (26) and (27). Sec.3.2_ PROPERILES OF VECTORS AND MATRICES 49 () Show that R, (A)Ry (a 0 costa) = sina) ox{A)sin() situ) cust sin(By =sin(a) c0s(2)sinfa) cos(B) costa) 8. IfA and B are nonsingular Nx. matrices and C = AB, show that C~' Hint, Use the associative property of matrix multiplication. 9. Prove statements (13) and (10) of Theorem 5.3 10, Let Abe an M x M mattis and X aa 8 1 main (a) How many muliplcatons ae needed to calculate AX (b) How many adltions are needed to calculate AX? 11, Let A be an M x N matrix, and let B and C be N x P matrices, Prove the left Aisributive law for matic meltiplcation: A(B +0) ~ AB | AC. 12, Let A and B be M x W matrices, and let C be @.N x P matrix. Prove the right N matrix, Example 3.16. Express the following system in sagmented matrix form and fied an equivalent upper-tiangular system and the Solution. stdin t bau = Dey +Oxg +415 +3nq = 28 Au t2n+2ns a= 20 Ait adnan 6 The augmented matrix is prone Fo 2 1 4/13 minz | 20 4 3428 m= 4221/2 mie 3 L313 2] 6128 CHAP.3. THE SOLUTION OF LINEAR SYSTEMS. AX = B ‘The first row is used to eliminate elements in the first column below the diagonal We refer tothe first row as the pivotal row and the element ay; = Lis called the pivotal clement. The values may are the multiples of row 1 that ae to be subtracted from row k for k= 2.3.4. The result aftr elimination ix 121 4 B Genes ie 0 “6 -2 -15]-32 Oia ecla|itas “The second row is sed to eliminate elements in the second column that lie below the diagonal. The second row isthe pivotal row and the values mz are the multiples of row 2 that are tobe subtracted from row & for = 3, 4. The result after elimination ic Tae eas (Olea ees ee pt | 0 0 =$ -75|-35 my=-t9 0 0 93 325) 483 Finally, the multiple mgs = ~1.9 of the third row is subtracted from the fourth row. rt the rests the upper-riangular system img :| B 0-4 2-5) 2 ay 0 0 -5 -75|-35 0 0 0 -9]-18 ‘The hack-substtution algorithm can be use to salve (11), and we get ‘The process described above is called Gaussian elimination and must be modified so that it can he used in most circumstances. If ayy = 0, row k cannot be used to gag im “ oe ee af) a ass ay lla] | al, axa far a ag a TPT, fap. ‘Then we will consuuct an equivalent upper-iiangular system UX = ¥: ay ay a3 ny | fe aye] 0 of of Bll] | oy ux=|° ay ay {]+3] =| aina | av. fo oo otf] fot.) i Step 1. Store the coeficients in the augmented matrix. The superscript on af!” means that this isthe fist ime that a number is stored in loestion (r, ¢) 4 ait) 1 | af) af aff ofa | fan on le oP oP on a | alas rou aa Gy | Suet ahi ak ol om ahaa | Step 2. If necessary, switch rows so that a{') # 0; then eliminate x; in rows 2 through NV. In this process, m,y is the multiple of row 1 that is subtracted from row 1. forr=2:N van al ath Po, fore=2:N 41 ity (Ps end130 Car. 3. The SOLUTION oF LivgaR SveTEMS AX = B The new elements are written a{? to indicate that this is the second time that number has been stored inthe matrix at location (r, ¢), The result after step 2is “0 09 | ah on a Sin | SN4 Or amen iueaw:| eohel Oa ay ms ay | Sarat 0 of a oh [oP | Siep 3, KE necessary, swatch the second row with some row below it so that 28) o£ 0- then elininate x2 in rows 3 through Nn this process. myy is the malt pile of row 2 that is subtracted from row r for =3: “The new elements ae witten af to indicate that this is Ue itd the Usa ber has been stored in the matrix at location (r, c). The result after step 3 is oy eames em (ae ot? ay tn | tna oO 2 | 2 OZ ug oh | Fra 21 oe 0a es ON | oa °) > | 9, 00 ayy vay | ais Step p+ This i the general sep. Wnecessary, switch mou p with come row beneath itso that lf) 4 0; then eliminate x» in rows p+ | through N. Here mp is the multiple of row p that is subtracted tom for. Ske 3.4 GaUssiaN ELmawaTiOn AND PivorINe at force=pHiinet aff? al? mp wai end end ‘The final result after xy.-1 has been eliminated from row N is Ie Ne it ai aig aty | athe Ge eee aa may ay | a my 2 | Qo eg ee ew ae 0 0 6 i | gar “The wpper-cangulasiaton rocen is now complete Since Ais nonsingular, when row operations ae serfomed the scesive mies are also nonsingular. This guarantes that o® #0 for alk in the construction process ence back subttutioncat be wed to sol UX = ¥ for X, andthe theorem is prove Pivoting to Avoid ayy = 0 aif) = 0, row p cannot be used to eliminate the elements in column p below the main diagonal. tis necessary to find row k, where a{2) # O and k > p, and then in: teschange row p and row & so that a nonzero pit eletent 1s obtained. This process s called pivoting. and the criterion for deciding which row to choose i called a pivots strategy. The trivial pivoting strategy is as follows. If a?) +4 0, do not switch rows. aff? = 0, locate the first row below p in which a{?? #0 and switch rows k and p Xn lesan anew clement af! #0, which nonzero pivot element Pivoting to Reduce Frrar HHecamse the computer utes fixed-precision arithmetic, itis possible that a small error will be introduced each time that an arithmetic operation is performed. The following example illustrates how the use of the trivial pivoting strategy in Gaussian elimination can lead to significant eror in the solution of «linear system of equations.132 CHaP.3. THE SOLUTION OF LINEAR SYSTEMS AX = B Example 3.17, The values 11 — sz — 1.000 are the solutions to Lasae1 + 5.281 6.414 ” 2A, 1Axy — 1.2102 = 22.93. Use four-digit avithetic (tee Exercises G and 7 in Section 1.3) and Gaussian elimination with aval pivoting to find a approximate solution othe system, ‘The multiple may = 24,14/1.133 = 21.31 of row 1 is tobe subtracted from row 2 t0 ‘obtain the upper tangular system. Using four digits in the caleulatons, we obtain the new coefficients 2 210213165281) = =1210~ 1125 = -1137 B= 293-2364) = 2295-1367 -1138 ‘The computed upper-tringula system is LIM +578 KALE 113.92 = -1138. Back substitution i used to compute x2 = ~113.8/(-113.7) = 1.001, and.x1 = (6.414 5.281(1.001))/(1.133) = (6.414 ~ 5.286) /(1.133) = 0.9956. . ‘The error in the solution of the linear system (12) is due to the magnitude of the ‘multiplier mai = 2131, In the next example the magnitude of dhe multiplier may is reduced by fist interchanging the first and second equations in the linear system (12) nd then using the trivial pivoting strategy in Gaussian elimination to solve the system. Example 3:18. Use four-digit arithmetic and Gaussian elimination with trivial piveting te sole the linear system 21d ~1.210r = 22.93 Liga + 9.28129 = 6.414 1.133/24.14 = 0.04693 isthe multiple of row 1 that i 1 be subir from tow? The new eeeticiete are a) = 5.281 0,04603( 1.210) = 5.281 4 0.05679 = 8.338 12) — 6.414 —0.0469%02.08) 641M 1076 — 8398 The computed upper triangular system is 2.tde| ~ 1.210 = 2293 5 38Req = 5.338 Bock substitution is used to compute sz = $.338/5.338 — 1.000, and xj = (22.93 1.210(1,000)) (24.14) = 1.000, . Ste. 3.4 GAUSSIAN ELIMINATION AND PIVOTING 1% ‘The purpose of a pivoting sumegy is to move the entry of greatest magnitude tc the main diagonal and then use ito eliminate the remaining entries in the colums. 1 there is more than one nonzero element in column p that lies on or below the mait ‘iagonal, then there is a choice to determine which rows to interchange. The parta. ivoting strategy itustrated in Example 2.18, is the most conimnon one and ie used ir Program 3.2. To reduce the propagation of error, itis suggested that one check the ‘magnitude of all the elements m columm p that lie on of below the main diagonal cate row Fin which the element that has the largest abealute value His, that i, lap} — mmaxtappl lapsuple +++ la ipls Kemps sud then switch row p with row k if k > p. Now, each of tne muplirs niry tor P= p~1,....1V will be less than or equal to 1 in absolute value. This process will usually keep the relative magnitudes of the elements of the matrix U in Theorem 3.9 i sane as Uiose in Uhe origina cueMiciewt mnaliix A. Usually, te ebwive of the larger pivot element in partial pivoting wll result in a smaller erzor being propagated, In Section 3.5 we will find that it takes a total of (4N 4 9N? — 7N)/6 arithmetic ‘operations to solve an Nx system, When N = 20, dhe ial aunuber of ariduuetic ‘operations that must be performed is 5910, and the propagation of error in the compu: ‘ations could result in an erroneous answer. The technique of sealed partial pivoting or equlibrating can be used to forther reduce the effect of error propagation. In scaled Partial pivoting we search all the elements in column p that lie on or below the main diagonal forthe one that is largest relative to the enuies in its row. First search rows ‘through NV forthe largest element in magnitude in each row, s4y (13) 5) maxtlayplelarpeiles- lara ll for r =p, ply “The pivotal row k ts determined by finding (4) ‘etal om [2 Uapzigl lanl Sp! Spei Se Now interchange wow p and A, wiless p ~ A Again, this pivoting process is desigued to keep the relative magnitudes of the elements in the matrix U' in Theorem 3.9 the same as those inthe original coefficient matrix A. Mi conditioning ‘A matrix A is called ill conditioned if thore exists a matt B for hich small pertar bations in the coefficients of A or B will produce lage changes in X = A~'B. The system AX = B is suid to be ill conditioned when A is ill conditioned. In this case, ‘tumerieal methods for computing an approximate solution are prone to have more ‘One cxrcumstance involving ill conditioning accurs when A is “nearly singular” and the determinant of A is close to 2er0. TM conditioning ean also acer in systemeTM Cua 3 THESOLUTION OF LINEAR SYSTEMS AX “ os Se 08) os ii ra2yed oo > deli 21 4:2043;6224;-313 21; >> Bel13 28 20 6)"; >> AvgeTAtow UIA. 9 1Me SULUTION OF LINEAR SYSTEMS AN — og 413 4328 422120 31326 (In the following MATLAB display, a is the element of greatest magnitude in te fi coluun of A and 9 isthe row number. >> fa, sHemaxtabe(A(d-4,))> 3 (©) Let Augup. 27 kogup=[L 2 14 13;0 “4 2 ~b 2:0 0-8 ~ {ULY] be the upper-tiangular matrix in (11). 5 -35;0.0 0 -9 -16) 1.0000 2.0000 1.0000 4.0000 © ~4.0000 2.0000 -5.0000 © 0° =8.0000 =7,5000 Oe oi otooe0 >> Yekugup(1:4,8) 13 2 “35 “18 i [Program 32 (Upper Triangularization Followed by Back Substation), To construct the solution to AX = B. by first reducing the augmented matrix [AIB1 to "upper-iriangular form and then performing back substitution. function X= uptrdk(,B) upul — A is au 8 x N nonsangular matrix * - Bie an Nx 2 matrix Wowrput ~ X is an Wx 1 matrix containing the solution to AX-B Uinitaalize X abd the temporary storage matrix C Te Nias zea) Kezeros(W,1); Crzeros(1, M91); Worm tho augnonted Avge [a 8); ‘Sec.34 GAUSSIAN ELIMINATION AND PIvoriNc for pei:N-1 Wartial pivoting for column p (, jJ=max Cabs (Aug¢p:¥,p))) wintercnange row p and j Crkug(p 3: Aug(p. :)=Aug(j+p-1, 2); nag(sep-1,2)=C;, if Aug(p.p)=0 "A vas singular. Mo unique solution’ bevak ead Welivination process for column p Aug (,p:ll+1) dug (x, p:t+1)-medug(p,p:N+1) 5 ena. and Weack Substitution on (UIY] using Program 3.1 Kebacksub(Aug(1:N, 1:10 ,Aug(4:N,W+1)) 5 ination and Pivoting Exercises for Gaussian In Exercises 1 through 4 show that AX = B is equivalent o the upper-triangular system DY = ¥ ana find the sation 1 2 +4n-6= 4 244 - Oye -4 x1¢5nt35= 10 3n+ 6y= 12 xtintig= 5 3 2 n+ ntone 7 at nt 7 “atta tI 2 de | a 9 Bo2n Hee 10 3 d-Intin= 6 Int B= 13 sttn-dn= 3 & -sntin~9= -1 w40m $35— 5 But 2+6n= 17 5, Find the parabola y = A + Bx + Cx? that passes through (1.4), Tr and @, 14).138 CHAH.3 The SOLUTION OF LINEAR SYSTEMS AX = 2 6, Find the parabola y = A + Bx + Cx? that pases though (1,6), (2, $). and (3,2), 7. Bind the cubic y = A+ Bx + Cx? + Ds that passes through (0,0), (1,1). (2,2). and (3,2). In Exercises 8 through 10, show thar A = Ris equivalent to the upper-tiangularey:tem UX = Y and find the solution, By +8nt4y404= 8 : arto ban sus 4 3x43 —3u= 6 x1 az 1 Tay 42m 10 futdae 12 mi43m+0n—2rs -4 9. Dei + 412 — 435 +04 In t4n- an bOu= 12 3n-3n-3u= 12 4y42te= 0 antag Day Pay Bus 9 $0. xy 122 1083 y= AVRO age 9 2a +3a— H+On wan m4 2ee Ory 4g + 283 — 5 tn tiny SntSm+25—4y= 32 15k |, Fin the solution tothe following linear system. wit d =7 2 +3n— nyt Bay 1 3ay = 10 2ey = dee = 12 12. Find the solution tothe following linear ayeten, nin ee Bi nts 9 Bey arnt 2a = 19 2a + 6x4 13, The Rockmore Cor, is considering the purchase of @ new computer and wil chonse either the DoGood 174 or the MightDo 11, They test both computers” ability to solve the near sytem Mx 59y—21=0 55x 4 89y 4=0. “The DoGood 174 computer gives. 0.11 and y = 0.43, and its check fr accuracy SEC.3.4 GAUSSIAN FLnAINATION AND PIVOTING 130 1s found by substitution: oor 0.00. 34(-0.11) + 550.48) -2 55(—0.11) + 89(0.45)— 3 10.9 and y = 1.01, and its check for accuracy ‘The MighiDo 11 computer gives is found by substitution 34(-0.99) + $5(L.01) ~21 = 0.89 '5(-0.99) + 89(1.01) ~ 34 = 1.44, ‘Which computer gave the hetter answer? Why? 14, Solve the following linear systems using (2) Gaussian climination with partial pivot- ing, and (i) Gaussian elimination wit sealed paral pivoting. (2 Set 101) yt 2-23 + 0.001440 eek Wy; 0001; =0 Oy Sot M— Olu =1 3x1 — 100r2 4+ 0.0123 = 0 Sn + m~100x3— 104 =0 2 100R- y+ -H=O 15, The Hilbert matrix is classical il-conditioned matrix and small changes in ts coet feicats will produce a large change inthe colton to the perturbed system () Find the exact solution of AX = B (leave all numbers as fractions and do exact arithmetic) using the Hilbert matrix of dimension 4x 4: t (0) Now solve AX = B using four-digit rounding antamenc 1.0000 0.5000 0.3333 0.2500 fe 0.5000 0.3333 9.2900 0.2000] yo A=lo3a33 0.2500 0.2000 0.1667 |" ol’ 0.2500 0.2000 0.1667 0.1429 o Note, The coefficient matrix in part (b)is an approximation to the coefficient ‘matrix in part (8).sau Cian.3 Tue SoLUTION oF LineaR Systems AX Algorithms and Programs 1. Many applications involve matrices with many zeros Of practical importance are ‘idioma systems (see Exercises {1 and 12) of the form diy ten aun + da +3 apna tds toate aman a die am 4 beat = Bn n-1aN-1 + dyn = by. Construct a prosram that will scive a tridiagonal system. You may ascume that row interchanges are not needed and that row k can be used to eliminate xy in tow k + 1 2. Use Program 3.210 find the sith-degre polynomial y =a) ape r aus! + aux? + and (6, 1. Use the plot command to plot te polynomial and the given points onthe saune graph Explain any discrepancies in Your grap, 3. Use Program 3.2 solve the linear syst AX = B, where A= laylyew and ay =i", and B = [by lv, where bus = N and by, = 22/61) fort > 2. Use N = 3,7,and 11. Theexactsoluion is X= [1/1 .., 1 1]. Explain any 4 Const progamthatchangesth pivot Seen Poeam 3210s pais ting ‘5. Use your sae paral protng rogram ftom Problem 4 solve the system given in Problem 3 for N = 11. Fxpain any improwements inthe sitions 6, Modify Program 3.2so that it will ficiently solve M linear systems with he sume covfficient mats A but difecat column matices B. The Mf linear syste look like AX, AX = Bp AXw = By. 7. The following discussion is presented for matices of dimension 3 x 3, but the con- cepts apply 10 matrices of dimension Wx N. If A is nonsingular, then A™" exists and AMES T Lat, C5, and C3 be the columns of A~! and Ry, Fs, and B be the columns off. The equation AA™! = I can be represented as A[Cr C2 Cs)=[B E> Bs] This matrix product is equivalent to the three linear systems ACL = Ey. AC2— Bz, and AC = By Sec.3.5 TRIANGULAR FACTORIZATION 11 ‘Thus finding A~" is equivalent to solving the three linear eystems. ‘Using Program 3.20r your program frm Problem 6, ind the inverse ach of the ‘olioing maize, Check your answer by compotng he prodoct AA and also by ting the command invA). Explain any ferences sot 16-120 MO 140 : 120 am 27001680 @) i : H| (©) | 240-2700 6480-4200 a 140 1680 —4200 2800. 35. ‘Triangular Factorization In Section 3.3 we saw how easy it isto solve an uppertriangular system, Now we introduce the concept ot factonzation of given mattis into Ue product of a lower ‘siangular matvix T that has I's along the main diagonal and an upper-riangular ma tix U with nonzero diagonal elements. For ease of notation we illustrate the concepts ‘with matrices of dimension 4 x 4, but they apply to an abitrary system of dimension NeW Definition 3.4. The nonsingular asin A has a triangular factorization if it can be expressed as the product of @ lowerriangular matrix Zand an upper-tiangular matrix U- v. a A In matrix form, this is written as ay a2 ay aie 10 0 OP fay aie ay aie ay an an aul _ [mn 1 0 0} | 0 up ms ws Sis ect tt bata tat ite ait rail ae aie ag | fa ase awd | 0 Od allan “The condition that A is nonsingular implies that uxs # U forall &, The mutation forthe cies in L ie my, and the reason forthe choice of m.; instead of will be pointed out soon Solution of a Linear System Suppose thatthe coefficient matrix A forthe linear system AX = B has «triangular factorization (1); then the solution 0 2 LUX=B142 CHAP.3. THE SoLUTION oF LINEAR SvSTEMS A. e canbe obtained by defining ¥ = UX and then solving two systems ° frstsolve LY = B forY; then solve UX = ¥ for X In equation form, we must fst solve the lower-nangula system ” =H may oo -b muntmmt om = mary + may tmasys+¥4 to obtain y, 9, yay and yg and use them in solving the upper-riangular system 4 wna +x tunes + max = y1 upon + waxy + zara = yo ants + waa 6 a Example 320. Solve 2+ Bert bry bar = 3x 410+ 1 +814 =79 xy + 122 + 103 + 6xy = 82. Lie te wing ectriton method an he ft at baad [room «4 4-1 S$ aln|3 id allo o 2 3 /ae2 sno] biz ieoo (ete arvana sve LY = #: 2mm Sntmt ne Ayr ty +29 +94 = 82. ‘Compute the values y1 = 21, yz = 52— 221) = 10,95 4021) — 1066) = 24, of ¥ © 79 — 301) ~ 10 = 6, ani 106 =24). Next write the 0 bintay+ x= 2 o 4r-254204= 10 Pesta 6x SEC. 3.5. TRIANGULAR FACTORIZATION M3 [Now use back substitution and compute the solution x4 = ~24/(~6) = 4,x3 314))/(-2) = 3, x, = (10 24) + 28))/4=2, and xs = 2-4 4) 20) = 1, [i234] i: ‘Triangular Factorization ‘We now discuss how to obtain the triangular factorization. If row interchanges are not nececeary when using Gaussian elimination, the mulipliers mj, are the cubdiagonal entries in L. [Reample 121. Ulee Ganesion elimination to conetruc the triangular Factorization of the smateix 3-1 43 2 6 ‘The matrix L will he constructed from an identity matrix placed atthe lef. For each row ‘operation used to construct the upper-tiangular matri, the multipliers mi; will be putin to ofa 3-4 a=|o 1 o//-2 -4 5]. norte G Row 1 is used to eliminate the elements of A in column 1 below ayt. The multiples may = 0.5 unl my = 0.25 of sow 1 ave subwacted from rows 2 and 3, respectively. These multipliers ae putin Ube matrix tthe left and the result is 10 0)fs 3 =r -05 1 offo 23 45 2s o 1ffo 125 625 ‘Row 2 is used to eliminate the elements of A in column 2 below azz, The muliple ms2 = 0.5 of the second row i subtracted from row 3, andthe multiplier is entered in te matrix atthe left and we have the desired tiangular factorization of A. 1 00)fs 3 = ~os 1 o|fo -23 45 o2s -05 ij[o 0 85 i ‘Theorem 3.10 (Direct Factortantion A = 1.1!. No Row Interchanges). Suppose ‘hat Gaussian elimination, without row interchanges, can be successfully performed to Solve the general Linear system AX = B. Then the matrix A can be factored as the rreduct ofa lower-triangular matrix L and an wpper-triangular matrix U: ry e AnWw.144 CHARS THE SOLUTION OF LinEaR SySTEMS AX = & Furthermore, L can be constructed to have I's on its diagonal and U will have nonzero diagonal elements. After finding I. and U, the solution X is computed in two steps: 1. Solve LU = B for ¥ using forward substitution. 2. Solve UX = ¥ for X using back substitution. Proof. We will show that, when the Gaussian elimination process is followed and B ic stored in column N {1 of the augmented matrix, the result after the upp ‘eiangularization step is the equivalent upper-siangular system UX = ¥. The matrices £.U, B, aad ¥ will have the form © Toga ommaug com aire g Ba | ote a, my) mya omy Py ea yg ai a “ ato? ay ay olka oO 4a 2 2 ea ay ha 2 2 2 vs[% o a Ow), =| eter at Mea ay faye ‘Remark, To find just L and U, the (N+ 1st colaun is wot uzeded. Step 1. Store the coefficients in the auemented matrix. The superscript on a! ‘tweans tha this isthe frst time that a number is stored in location (7, c) og Qi ant git) an 2 a3 4 | Sine oo WL, > a a of [el al) aS al mt | ayy ah a all oo all Fallon Siep 2 hinnimate x; in rows 2 rough Vand store the muliplier m1, used to eliminate xy in row r, in the matrix at location (r. 1) Su. 3.3 TRIANGULAR FACTORIZATION Ms forr=2:N rma maa fore =2: N+ 22) = of) — mn eal sn 3 The new elements are written a2) to indicate that this is the second time that a ‘number has been stored in the matrix at location (¢,c). The result after step 2 is mar a 8) 0 | ee alee mur a a ah a2... ‘rep 5, Bliminate x2 in sows 3 dough N and store the multiplier mra, used t0 eliminate x2 in row r, in the matrix at location (r, 2) torre 3.4 ee The new elements are writen af?) to indicate that this is the third time that a num: ‘ber has been stored in the matrix at the location (rc). on ata a ma a oP oB | a, eee ee daa) | a | 2), mm mor of oh |aPan Step p +1. This is the general step, Eliminate x» in rows p+ 1 through N and store the mulkipliers atthe location (, p)146 Cuap 3. Tue SouuTION oF LINEAR SYSTEMS AX. forr=ptiiw arp = aD 13D np = 1b fags Op = Pap fore=p+i:N+1 ae" = gle) — end end wal rep +0 “The final result after xy -1 has been eliminated form row Nis oa a “ Pe ale ci of oo oe mom am ol ae mt maa mina off far | “The upper-triangular process is now complete. Notice that one array is used to store the elements of both Zand U. The I's of L are not stored, nor are the 0’s of Land © that ie above and below wie diagonal, respectively. Only the essential coefficients needed to reconstruct L and U are stored! ‘We must now verify that the product LU = A. Suppose that D = LU and consider the ease when r S ¢. Tae dre is 9) dpc = mir! + mya? +--+ Mra? +a, Using the replacement equations in steps | through p-+1 = r, we obtain the following substitutions matt =a? a2, maa = a2) — af ina = ae — ah, uy aoe aes ane Wen the substitutions in (10) ae used in (9), the sesutis ema 0 +0 a4 oa =a a al? The other case, r > ¢, is similar to prove. . See. 4.8 TRiansiitan FacroRizArion w ‘Computational Complexity ‘The process for wiangularizing Is the same for both the Gaussian elimination and tn ‘angular factorization methods. We can count the operations if we look at the first NY columns of the augmented matrix in Theorem 3.10. The outer loop af step p + 1 re- guises WV — p = 0 ~ (p+ 1) + I divisions wo compute the multipliers mrp. inside the loops, but forthe fist N columns only a total of (WV — p)(N ~ p) multiplications and the same number of subtractions are required to compute the new row elements af?*? “Tis process is cared out tor p = 1, 2,...,.W — 1, hus the mangular factorization portion of A = LU requires a : ay Sa-mav-p+n= 22% muipcatons an dvsions, wos uw I=IN (2) Dev = p= p= EA EEN sotractions mi ‘Tocstablish (11), we use the summation formulas MMOD a Sy MoeDaMeD 6 OnkW/the triangular factorization A = LU has been obtained, the solution to the lower-ulangulas systeus LY = B will require 0+ Lp oe NL = (NP = N92 ‘witplications and subtractions: no divisions are required because the diagonal ele- ments of Z are 1's. Then the solution of the upper-tiangular system UX = ¥ requires 1424-004 = (N?-+N)/2 multiplications and divisions and (N? — N)/2 sub- twactions. Therefore, finding the solution to LUX = B requires 1N? multiplications and divisions. and N? — N subtractions.MB Chap. 3 The SoLUTION oF Lincan Systems AX = B ‘We see that the bulk of the calculations lies in the tiangularization portion of the solution. Ifthe linear system is to be solved many times. with the same ecefficien: ‘matrix A but with different column matrices B. ic is not necessary to triangularize the ‘matrix each time ifthe factors are saved. This isthe reason the triangular factorization ‘method is usually chosen over the elimination method. However, if only one linear system is solved, the two methods afe the same, except thatthe triangular factorization ‘method stores the multipliers. Permutation Matrices ‘The A = LU factorization in Theorem 3.10 assumes that there are no row inter: hues. ICis possibie that a nonsingular matnx A cannot be ditecty tactored as Aw LU. 10: Example 3.22. show that me tollowang matrix cannot be directly factored as A fiz 4 [a3 Suppose that A has a direct factorization LU: then 12 6) 4 0 0] fa eu as a3 48 -1/=/m 1 olf 0 uz us 23 5} [mi ma tfLo 0 ass ‘The matrices L and V onthe right-hand side of (13) can be multiplied andcach element of the product compared with the corresponding clement of the matin A. In de fst column, 1 = Jay, then 4 = mayiys = may, and finally ~2 = msquii the second columa, 2 = Taya, then 8 (42) + pe implies that up2 = 0, and finally 3 — msaung + mgauga ~ (=2)Q) 1 my2(0) — A, which ipa contradiction, Therefore, A does not havea LU factorization . A permutation ofthe first N positive integers 1,2,..., V isan arrangement ki, ko, «ky ofthese integers in definite order. For example 1,4, 2, 3,Sisa permutation of the five intogers 1, 2,3, 4,5. The standard base vectors E; —(00--- 01,0 --- Oh are used inthe next definition, Definition 3.8. An Wx permudation matrix P ix a malin with previsely one enuy whose value is 1 in each column and row, and all of whose other entries are 0. The rows of P are a permutation ofthe rows of the identity matrix and can be writen as, (ay P=[E, Bi Bul Spc.3.5 Triancutas Facroarzariow ” ‘he elements of P = [pj] have the form jak, 1 PY 10. otherwise. Forexample, the following 4 x 4 matrix is permutation matrix, 0100 : ooo era as) ra[b 08 Ol Lie ey ay my. . oo10 Theorem 3.1. Suppose that P= (Bi, Ej, ... Ej] is permutation mati Tie product A is a new matrix whose rows consist of the rows of A rearranged in the onder rows, A. to, A, rom, A Beample 825, et A bead mata P eth patna gen them PA icthe mats when ewe rene Pr) ow) A, 104 A, 10W3 A. ‘Computing the product, we have 010 O} fan a2 ax aw] fay ax a2 an 10 0 Ojfoy a a ax|_ as a2 as aw 000 t}Jey a2 a3 au) “fan an a6 ave Oot offen an ax au) Las am ass ase . ‘Theorem 3.12. Uf P isa permutation matrix, then itis nonsingular and P~ ‘Theorem 3.13. If A is ¢ nonsingular mauin, dhen there exists permutation matnx P so thal PA has a tiangular factorization a6), PAWLU, ‘Ihe proots can be found in advanced linear algebra texts Example 3.24. If rows 2 and 3 of the matrix in Example 3.22 are interchanged, then the resulting matrx #A has a triangular Factorization ‘The permutation matrix that switches rows ? and Tie P= [F\ EBL] Comput ing the product PA, we obtain 100 oot 910, P.180 Cuae 3 Tus SoLuTion oF LINEAR SYSTEMS AX = B [Now Gaussian elimination without row interchanges can be used: po [12 6 my=-2}223 $ mie tL 4 8 AL ‘After x has been eliminated from column 2, row 3, we have Tague pvr 10 7 17] =u. ma=0 [0 0 25 i. Extending the Gaussian Elimination Process ‘The following theorem is an extension of Theorem 3.10, which includes the cases ‘wmen row interchanges are required. 1hus tnangular factorization can be used to tin! the solution to any linear system AX’ = B, where A is nonsingular ‘Theorem 3.14 (Indirect Factorization: PA = LU). Let A be a given N x. ‘matrix. Assume that Gaussian elimination can be performed successfully to solve the ‘general linear system AX — B, but that row interchange are required, Then there exists a permutation matrix P so thatthe product PA can be factored as the product of alower-triangular matrix Z and an upper-triangular matrix U: PA=LU. Furthermore, L can be constructed to have 1s om its main diagonal and U will hss nents. The solution X is found in four steps nonzero diagonal 1 Constnict the mattioes 1. U7. and P. 2. Compute the column vector P B. 5. Solve LY = 2° for ¥ using forward substitution 4. Solve HI = ¥ foe ¥ using hack eabituion Remark: Suppose that AX = B is to be solved for a fixed mavix A and several diffe ‘ent column matrices B. Then sep 1 is performed only once and steps 2 through 4 a: used to find the solution X that corresponds to B. Steps 2 through 4 aze a computatio ally efficient way to construct the solution X and require O(N?) operations insteed the O(N5) operations requited by Gaussian elimination. Ste 3.8 TRIANGUTAR Facroar7arIon 1st MATLAB ‘The MATLAB command {L,U,P]=1u(A) creates the lower triangular matrix L. the ‘uppertrangular matrix U (from the triangular factorization of A), and the permutation inatcix P from Theorem 3.14. Example 3.25. Use the MATLAB command (L,U,P]=20(A) on the matrix A in Ex- sample 3.22. Verify that A= PAU (equivalent to showing that Pa = LU). (126 40 -1,-23-51; On) 1.0000 0 9 ~0,5000 1.0000 0 v.z00 0° 1.0000 4.0000 8.0000 -1.0000 (ou ore elasn i o10 oon 100 deny (P)eLed ae 235 . Aspreviously indicated the tiangular factorization meted is often choseu over the ‘iination method. In addition, itis used in the inv(A) and det (A) commands in MATLAB. For example, from the study of linear algebra we know that the determinant of a nonsingular matrix A equals (-1)* det 8, where U isthe uppertriangular matin {from the triangular factorization of A and isthe numberof row interchanges required ‘tw obtain P from the identity matrix £. Since U is an upper-riangular matrix, we ‘know that the determinant of U is just the product of the elements on its main diagonal (Theorem 3.6). ‘The reader should verily in Example 3.25 that; dew(A) = 175 (1PC175) = (1 detw), The Jollowing program implements tne process described in the proof of Theo- rem 3.10. [cis an extension of Program 3.2 and uses pail pivating. The interchang- {og of rows due to partial pivoting is recorded in the matrix R. The matnx R is then ‘uped in the forwasd substitution step ta tine the mann ¥.4152 Ciap.3. THe SOLUTION OF LINEAR SYSTEMS AX = B Program 3.3 (PA = LU: Factorization with Pivoting), “To construct the solu tion tothe linear system AX = B, where A is a nonsingular matrix. function X = lufact (A,B) Aimput - A is an NxM matrix & 8 is an Nx 1 matrix output - X ie an Nx 1 matrix containing the solution to A = B {Initialize X, ¥, the tesporary storage matrix C, and the row 4% permutation information matrix 8 Di wD-aiz0(A); Aezeros (W,1); Yezeros(N, 1); crzeren (1.0); ReiiW; for pet:N-1 APind Ue pivot cow for colum p (maxi, j]=max(abs(A(p:N,p))): interchange rov p and j Ap, = )=AC ApH) AC}p~t 2265 a-RCp) 5 ROp)=RGjtp-1) RCjtp-t) as Af A(p,p)—=0 ’4 ig singular. No unique solution’ break ond Yearevlate multiplier and place in subdiagonal portion of A for kepti:l BULT=ACK,P/ACP PD AGe.p) = mul AC, pet :N) *ACK, pet:N)-multaa(p, pet ND; wud end Wsolve for ¥ ya) # BUR); for kad YQO= B(RGD)-AGK, 1H) AY ed); Yseive for XQD=YQD/ACEND ; ‘Sue. 3.5 TRIANGULAR FACTORIZATION for ket-1:—1:4 XC) =(Y CE) “AC 1) 9X (e611) /A 0 5 and Exercises for Triangular Factorization 1. Solve LY = 8, UX = Y, and verily tha B = AX for(a) B= [-4 10. 5] and (B=[20 49 22] .whew A= LU is is Ola ie OulO]|2ieed Hes 3 |e as aie ol) Obst 6: ese ola ary ogee 2. Solve LY — BUX — ¥, and varity hat B — AN for (a) B~ (7 2 10J/ and @)B=[23 35 7. where A = LU is eee tog) at -t 2 9)=[-1 1 of[o 3 15 i-2oy Li 1 aie oy 3. Find the triangular factorization A — LU forthe matrices, santa 120 es @ | 10 3 m [31 6 atte ager Find the triangular factorization A = LU forthe matrices aaa re @ [2 5 -2 wm |s 2 aes nas 5, Solve LY = B, UX = ¥, and verify that B = AX for(a) B a eg) (l io oLiraga eG 4 -afefp ro elje ss ital) (eal loi0 aia o-2} Lg -p tlooo 1 6, Find the wianglar factorization A = LU forthe matrix 1 etHork aio aa 3026 17. Exablish the formula in (12)