,

,~ .' '~.' ".'.

. .......

1

© A. J. M. Spencer, D. F. Parker, D. S. Berry, A. H. England, T. R. Faulkner, W •. A. Green, J. T. Holden, D. Middleton and T. G. Rogers, 1977

ENGINEERING MATHEMATICS

.ISBN 0 442 30146 4 cloth , .~BN 0 442301472 paper

Library of Congress Catalog Card No. 76-45663

All rights reserved. No part of this work covered by the copyright hereon may be reproduced or used

in any form or by any means - graphic, electronic, or mechanical, including photocopying, recording, taping, or information storage or retrieval systems - without written permission of the publishers

I

I Volume 1 !

I

I A. J. M.'SPENCER

! D. F. PARKER

!D.S.BERRY

A. H. ENGLAND

. T. R. FAULKNER iW.A.GREEN

I J. T. HOLDEN

D. MIDDLETON T.G. ROGERS

Published by Van Nostrand ~einhold Company Ltd., Molly Millan Lane, Woldng1lam. Berkshire. England

Published in 1977 by Van Nostrand Reinhold Company A.Division of Litton Educational Publishing, Inc.,

450 West 33rd Street, New York, N, Y. 10001, U.S.A.

Van Nostrand Reinhold Limited

1410 Birchmount Road, Scarborough, Ontario, MIP 2E7

Van Nostrand Reinhold Australia Pty, Limited

17 Queen Street, Mitcham, Victoria 3132, Australia

Library of Congress cataloguing in Publication Data Main entry under title:

Engineering mathematics.

Includes bibliographies and index.

1. Engineering mathematics. I. Spencer, Anthony James Merrill.

TA330.E53 515'.14 76-45663

ISBN 0442-301464

ISBN 0442-30147-2 pbk.

"

; Department of Theoretical Mechanics

: University of Nottingham

Typeset in Great Britain by Preface Ltd., Salisbury Printed in Great Britain by Biddies Ltd, Guildford. Surrey

I

I~

New York - Cincinnati - Toronto - London - Melbourne

V AN NOSTRAND REINHOLD .COMPANY LIMITED

m KT=---

I r - zok I

where r = xi + yj + zk. In spherical polar coordinates, show that

m I (fo)npn(COS8),

Zo n=O

In this expressiorrtBe image source has strength m 1 = -amv\and is placed at the 1 po. (0,0, Zl) which is the image point of (0, 0, zo) in th. .. iere so that ZOZI = a . Image sources of this type also occur in electrostatics, fluid mechanics and in gravitational-theory ..

inear Algebra - Theory

9. The following expansion may be shown to be convergent in I hi < 1:

1 00

------"7""~ = \' hn P n (cos 8) {l - 2 h cos 8 + h 2 } 112 n ~O

where Pn(cos 8) is the Legendre polynomial of degree n. The potential due to a thermal" point so~ce of strength m at the point (0, 0, zo) is

. SYSTEMS OF LINEAR ALGEBRAIC EQUA nONS. MATRIX NOT A nON

problems in engineering, science, economics and other subjects reduce to the of systems of linear algebraic equations. This chapter will be largely

with the theory of the manipulation and solution of such systems. methods of solution are dealt with in Chapter 10. To exemplify the

in which systems of linear algebraic equations can arise in an engineering. We begin with some illustrations from electricity and mechanics.

Networks

the simple electrical network shown in Fig. 8.1, with steady currents .... ,Is in the various conducting wires with resistances R1, R2, ••• .R« as

Kirchhoff's first law states that the net flow of current into any junction must Applying this- to the junctions A, B, C, D in turn gives

m

r>zo·

r<zo

KT= 2 ·2}112

{Zo - 2zor cos 8 +r

( )n+l

~ Pn(cos(J),

This result links the potential of a point source with the separated solutions (94).

BIBLIOGRAPHY

[ 1]

Ames, W. F., Nonlinear partial Differential Equations in Engineering, n,,,,,u--,,,,, Press, New York (1965). .

Coulson, C. A., Waves, Oliver and Boyd, Edinburgh (1958).

Abramowitz, M. and I. A. Stegun, Handbook of Mathematical Functions, New York (1965).

Stokey, W. F., Chapter 7 of Shock and Vibration Handbook (edited by C.

Harris and C. E. Crede), McGraw-Hill, New York (196.1). .

Churchill, R. V., Fourier Series and Boundary Value Problems, McGra New York (1963).

Morse, P. M. and H. Feshbach, Methods of Theoretical Physics, New York (1953).

Sneddon, I. N., Mixed Boundary Value Problems in Potential Theory, Holland, Amsterdam (1966).

[2] [3]

[4] [5 ] [6]

[7]

=1,

12 -13 -14 = 0,

+13 -Is = 0,

14 + Is = I.

(1)

'A
'yyy I.
R3
8 \A ..
12 ' yy IJ
R2
Rs 0
_A A A A.___"
C yrY
I A Is I
R,
AAAA
yyy I,

Fig. 8.1 Electrical network

306

L._~dT Algebra - Theory

Systems of Linear Algebrau: .i-quations. Matrix Notation

3u7

Two points should be noted. (a) The four equations are not independent; the first is the sum of the other three. Thus only three of the equations give information about the system, and one equation could be discarded. (b) The equations are not sufficient to determine the currents h, 12, ••• ,Is in terms of the given current I. To obtain a complete system of equations we might adopt Kirchhoff's second law, which states that the algebraic sum of the products of the current and resistance in any closed circuit without batteries, is zero. By applying this in turn to the circuits ABCA and BCDB there follows

" members. We SUppose that they obey Hooke's Law so that, in any member LT= EAe where T is the tension, e the extension, L the length, A the cross-sectional area and E is ...•. Young's modulus. For simplicity suppose that E and A are the same for each member

. that the members are fixed in position but free to rotate at A, D and E. Let.the ' ,horizontal and vertical components of displacement of the joint B be U and V and

th dl B B.

'. e correspon mg components of displacement of C be Uc and V . Then the

•• extensions of the members are easily calculated, and Hooke's Law ~pPlied to the five

-s.t, +R212 +R3h =0,

- R313 + R414 - RsIs = 0.

A further equation could be obtained by considering the circuit ABDCA, but this would yield no new information. Equations (I) and (2) together determine

I I> 12, ••• ,Is, and are quite easily solved.

Frameworks

(a) A simple pin-jointed plane framework is illustrated in Fig. 8.2(a). Let TI , T2, T3, T4 denote the unknown tensions (a thrust is regarded as a negative tension) in the members, as illustrated, and let XB, Y B. XC, Y c be given loads applied as shown to joints Band C. Since each joint is in equilibrium, the resultant horizontal and vertical components of force on B andC are zero, which gives

=-XB'

=-YB,

=-Xc,

y!f2T3 + T4 = -Yc·

There is no difficulty in solving these four equations for the four unknowns Ti, T1• T3• T4• The structure is determinate.

(b) Suppose now that the framework is strengthened by the addition of the CE shown in Fig. 8.2(b). The equations for equilibrium of the joints now give

=-XB, =-YB,

- y!f2Ts = -Xc,

These are now four equations for five unknowns. There is not a unique solution, . the structure is indeterminate. To produce a system of equations with a unique

__ 1 .... .: .... _ ': .. : ... _ ............ n ......... " +_ "-flo"~ f11rthor t.:ICClltnntinnc:. ~hnl1t thp. nronP.rtiP.~ nf thA

(a)

(b)

Xs

8 T, L

L

T. o

Ys Yc
Xs
8 T, L T,
T,
(e) L
T,
A ~:l"I' Q., Dl""-"II. .... _:_ : ..... :_ .. _...:11 l' ••• __ , __

308

cmear Algebra -

(9)

of Linear Algebraic ~;,aations. Matrix Notation

members gives LTI

they arise in problems of surveying, in certain statistical problems, in all

of vibration problems, in the economics of supply, demand and costing, and o'..L.. .... here, They also appear very frequently in problems of numerical analysis, 'CRIeClilUY as numerical approximations to differential equations. The problem in some of equations (7) are replaced by inequalities is also of practical :"'",nrt"n(~p., and is considered in Chapter 1 of Volume 2.

In principle, equations such as equations (7) may be solved (provided that they

a solution) by the elementary methods of substitution or elimination. However

hand calculations these methods become prohibitively laborious if m and n exceed five or six. It is easy to envisage electrical networks and framed structures which to much larger numbers (possibly hundreds) of equations and unknowns (consider, example, the wiring of a motor car, or a large bridge) and then methods which can

easily adapted to computer solution are needed. A convenient tool for this is lIrO'vid~:C1 by matrix algebra. Throughout this chapter it is important to keep in mind methods are required which will handle large systems of equations, although in space available it is onlypossible to illustrate with quite small systems.

-EAVB

=0,

+EAUB

-EAUc

= 0,

-..jYzEAUc -..jV£AVc = 0,

-EAVc

= 0,

+ ..jV£AUc - ..jYzEAV c = O.

Equations (4) and (5) are nine equations for nine unknowns Tv, T2, ••• , T«, UB, V Uc, V c. In practice the easiest method of solution is to eliminate T1, T2, ••• , Ts by substituting from equations (5) into equations (4). Again the solution is not difficult (see example 4 of §8.2 and exercise 4 of §8.4).

(c) Finally, suppose that the membersAC and CD are removed, as in Fig. 8.2(c). The equations of equilibrium of the joints are now

T2 = -XB'

m x n matrix is an ordered array of numbers arranged in m rows and n columns. example

T2 - ·JIhTs = -Xc,

..jY3s = -Yeo These are now four equations for three unknowns TI > T2, T'«. Unless Xj, + Xc + Yc , the equations are self-contradictory, and have no solution. This absence of a solution " reflects the physically obvious fact that the framework is a mechanism, which in ' general cannot stand in equilibrium under the given loads and will collapse.

Equations (1). (2)(or (1) and (2) together)(3), (4), (5)( or (4) and (5) together) and (6) are examples of systems of linear algebraic equations. A system of linear algebraic equations is a set of equations of the form

alixi +a12x2 + +alnxn=bl,

a21xI +a22x2 + +a2nxn=b2,

( 1)

2 -5 Ih .

( ) is a 2 x 3 matrix, 3 is a 3 x 1 matrix.

1T 0 0.7

-1

are many reasons why matrices are of interest to mathematicians, physicists, lco!nornis1ts and others. However they are usually first encountered, especially as engineering applications, in connection with the study and solution of linear equations. Therefore in this chapter matrix properties will be developed in

to this application. Accordingly, we first observe that a convenient :"'L"A";'oHr.n of equations (7) is

(8)

A, X and B represent the following arrays of numbers

am IX I +am2x2 + ... +amnxn = bm.

Equations (7) comprise m equations in the n unknowns XI, X2, : .. ,Xn· The coefficients a I I; a I 2, •.• ,amn are given constant numbers, and so are b I, b2, ... , The commonest and most important case is that in which m = n, so that the number. equations is equal to the number of unknowns. However the above examples show . this is not necessarily the case, and m may be greater than, equal to, or less than n.

Systems of linear algebraic equations occur directly in many engineering

~ - - ~ ,,_) r __ ~_.J _"'_ ........ " ... ,..~ ..... ".a.'I"1"\nl,t;.a. ...

(all aI2 ... aln) (XI) (,bl)

a2 I a22'" a2n X2 B _ b2

A= X=, - .

...... ' . .

. .

. .

ami am2'" amn

xn bm

A is an m x n matrix, X an n x 1 matrix and Ban m x 1 matrix. The numbers form each array are called the elements of the matrix. The elemen t in the ith and jth column of A is denoted aij so that the first index (i) denotes the row and

_ ...... I't..,,~ I;) r1~nnt~~ th~ t"'nl11rnn Thl1~ i·t!li1t-~~ thp V!al11p~ 1 tn,." !lin,", i t!li1t-p~ thp v!lil11p~

310

Lr Algebra-

of Linear Algebrair"-'lations. Matrix Notation

3'

1 to nand aij is the element at the intersection of the ith row and the jth column We sometimes use the notation

Anti-symmetric matrices If, in a square matrix A, aij = -ajl for all values of i then A is anti-symmetric (or skew-symmetric). Since in particular all = -a 1 1, = -022, etc., all elements on the leading diagonal of an anti-symmetric matrix are For example

A = (aij)

to denote the matrix A whose ijth element is aij' In this notation, unless it is clear from the context, it is necessary to specify the range of values which i andj

in order to know the size of A.

Symbols which represent matrices will be printed in bold face type, and which represent ordinary numbers (also called scalars) will be printed in italic type,

( 0 2 -1)

-2 0 3 _ is a third-order anti-symme~ric matrix.

1 -3 0

Special Matrices

Matrices of various special forms are often encountered in applications. The most important are the following.

(a) Column vectors A matrix such as X or B in equation (9) which consists single column of numbers is a column vector.

Upper and lower triangular matrices A square matrix A is upper (lower) if all the elements below (above) its leading diagonal are zeros. Thus

(1 -1 2)

o 3 5 is upper triangular;

002

(1 0 0)

4 0 0 is lower triangular.

062

, elements on the leading diagonal mayor may not be zero's.

(b) Row vectors A matrix consisting of a single row such as C=(Cl C2 ... cn)

is called a row vector.

Diagonal matrices A diagonal matrix is a square matrix with zeros everywhere on the leading diagonal. A diagonal matrix is of course symmetric. An example

(~ ! -~ ~).

o 0 0-1

Rowand column vectors, as special matrices; have some properties similar to of the vectors described in Chapter 6, but are not identical to them. The vectors described in Chapter 6 usually represent physical quantities, whereas matrices do necessarily do so. Although the terminology is ambiguous, it will usually be clear the context which type of vector is involved in a given application.

(c) Square matrices If m = n, so that A is a square array of numbers with the ...• number of columns equal to the number of rows, then A is a square 12 x n matrix square matrix of order n. If A is a square matrix, then in equation (7) the equations is equal to the number of unknowns.

The leading diagonal of a square matrix is the diagonal set of elements ex from its top left-hand corner to its bottom right-hand corner. Thus, if A is a matrix the elements al 1> a2 2, a33, ... ,ann lie on its leading diagonal, wh-ich of the elements aij for which i = j.

The unit matrix of order n has 1 at each position on the leading diagonal and elsewhere. It is denoted by In or, if its order is clear from the context, simply by

zero or null matrix, of a given size, has zeros for all its elements. It is by 0 and is not necessarily a square matrix.

h =(~ ~

o 0 diagonal matrix of order n,

(d) Symmetric matrices If, in a square matrix A, the element in the ith row the jth column is equal to the element in the jth row and the ith column, that is aij = aji for all values of i and j, then A is symmetric. The symmetry is about the leading diagonal. An example of a fourth-order symmetric matrix is

I_~ : -~ :).

\ <; 1 1 ')

of the following matrices are (1) symmetric, or (2) anti-symmetric, or triangular, or (4) none of these?

(~ ~ -:),

All

(b)

.( 2

'0

A

312 Linear Algebra -
~). (d) G -3) '.~~.-
(c) 0 -5 3
( -: 0 -1 4 -2 I,
1 0 0 -2
-2 -6 4
(e) G 1 -3) (f) (~ 2 6 D·
o -2 , 1 3
o 0 0 0 8.2 ELEMENTARY OPERATIONS OF MATRIX ALGEBRA

Equality of Matrices

Two matrices A and B are said to be equal if (a) they are of the same size, so that number of rows of A equals the number of rows of B, and similarly for columns; (b) all corresponding elements of A and B are equal. Thus, if A = (aij), B = (bij), aij = bij forall values of i and j, we write A = B.

Transposition Suppose A is an m x n matrix. Then from A a new matrix can be formed whose are the columns of A, taken in order, and whose columns are the rows of A, taken order. This new matrix is called the transpose of A; it is an n x m matrix and is denoted by AT. If aij is the ijth element of A, then aji is the ijth element of AT. example, if

(3 7

o -1 AT=

2 -2

4 6

-:) .

-1

:),

-1

o 2 -1 -2 5 -1

then

The following results are immediate consequences of this definition and the definitions of §8.l.

(a) The transpose of a row vector is a column vector, and vice versa.

(b) The transpose of an nth-order square matrix is also an nth-order square matrix.

(c) The transpose of a symmetric matrix is itself, so that A is a symmetric matrix if and only if AT = A.

(d) The transpose of an anti-symmetric matrix is the original matrix with element multiplied by -1, and is itself anti-symmetric. B is an antisymmetric matrix-if and only if BT = -B.

.313

~-~

The transpose of an upper triangular matrix is a lower triangular matrix, and vice versa.

For any matrix A, (A Tf = A.

far we have only introduced a rather large number of definitions and madesome iilPediate deductions from them. The next objective is to construct an algebra of This matrix algebra will be constructed so as to be as close as possible to our

ordinary algebra, but we shall find that it is necessary to introduce some differences from ordinary algebra. The first step in formulating matrix

is to define the operations of matrix addition, subtraction and multiplication. :defmitions to be adopted are ones which have been found in practice to be useful In particular they facilitate the manipulation of systems such as equations (7).

and Subtraction of Matrices

A + C of two m x n matrices A and C is defined in the natural way to be the matrix whose ijth element is aij + Cij' Similarly A - C is the matrix whose ijth

is aij - Cij' It is essen tial that A and C are the same size; it is impossible to add , ... t.,tr<l,f't matrices of different sizes.

4 6 7 -1 2 -3

(28

A +C= -5 8 .94

5 9)

7 8 ,

-3 2

( 2 0 -7 3)

A - C = -9 -6 9 -2 .

1 030

immediately from the definition that if A, Band C are all m x n matrices

(A + B) + C = A + (B + C), (A + Bf = AT + BT.

(10)

lti'iiclmo'lZ by a Scalar

an m x n matrix whose ijth element is aij' and k is a scalar, then kA is defined to m x n matrix whose ijth element is kaij. As a special case k may be zero, and

(11)

Matrix Multiplication

As well as multiplying a matrix by a scalar, it is possible to multiply one matrix by another matrix. Matrix multiplication arises as a generalization of a simple operation elementary algebra. Suppose first that we have two relatio~s

314

Linear Algebra -

x =ay, y = bz,

connecting the pairs of scalar variables x and y, and y and z respectively. By substituting for y from the second equation into the first, there follows

x = abz .

Now suppose we have two relations between three sets of variables (x I, X2, ... ,x (y I, Y2, ... ,Yn) and (z I, Z2, ... ,Zq)' In matrix notation these are written

X=AY, Y=BZ

where A is a p x n matrix, B an n x q matrix and X, Y, Z are column vectors with p, nand q rows respectively. It is natural to 'substitute for Y' from the second equation into the first, and write

X=ABZ

315

AB is interpreted to be the matrix

.~ .. '

..

= c; +a12b21 allbl2 +a12b22 allbl3 +aI2b23) a2Ibll+a22b21 a2IbI2+a22b22 a2IbI3+a22b23 .

(16)

is an example of the operation which is called 'matrix multiplication'. Notice elimination of the y's from equations (14) can be done only when the number of A is equal to the number of rows of B. Only then is the formation of the AB (in that order) possible. If this condition is satisfied, the two matrices are

, ... to be conformable for the product AB. '

, may now extend equation (16) to define formally the process of matrix luum;.'UU'll. Suppose aij is. the ijth element of a p x n matrix A and bi' is the ijth of an n x q matrix B. Then the ijth element Cij of the product C = AB is

and this can be done provided that the product AB is defined in a suitable way. For illustration, suppose p = 2, n = 2, q = 3, so that, written in full, equations (1 2)

XI =aJlYI +aI2Y2, YI =blIZI +b12Z2 +bI3Z3,

X2 =a2IYI +a22Y2, Y2 =b21Z1 +b22Z2 +b23Z3,

where now

By substituting for Y I, Y2 from the second set of equations (I 4) into the first set' have

bas p rows and q columns. Thus, to obtain the element in the ith row and jth of C = AB select the ith row of A, and the jth column of B, multiply Ire!:OOltaiIl,g elements beginning at the left of the row of B and the top of the

of A, and sum all the resulting products. ,

(3X2+IXI 3x7+lxO ;)= 2x2+3xl 2x7+3xO 5x2+4xI5x7+4xO

3 x 6 + I x 3) 2x6+3x3

5x6+4x3

XI = (alibi I +aI2b21)ZI + (allbl2 +aI2b22)Z2 +(allbI3 +aI2b23)Z3, X2 =(a2Ibll +a22b2.)zl +(a2IbI2 +a22b22)Z2 +(a2IbI3 +a22b23)Z

( 7 21 21)

= ! 14 21.

14 35 42

This is consistent with equation (13) and the notation previously employed nrn'.,;..! ....

that A is 3 x 2, B is 2 x 3, AB is 3 x 3.

316

U-·"r Algebra-

~)=(2X3+7X2+6X5

lx3+0x2+3x5 4

2Xl+7X3+6X4) lxl+0x3+3x4

= (50 47)_

18 13

Note that B is 2 x 3, A is 3 x 2, BA is 2 x 2, and that AB =1= BA.

EXAMPLE 3

Then

However A, X and Yare not conformable for the products XA and AY, so these products do not exist.

EXAMPLE 4

In § 8.1 equations (4) and (5) were derived for the tensions and displacements in the framework illustrated in Fig. 8.2(b). These equations can be expressed as follows

(1

=

-1 0 0 o 0 0 1 VYl 0 o VYl

EA L

o 1 0 O'

-1 0 0

o 0 Yl * 000

o 0 -Yl Yl

Eliminating T 1, T2, ••• , Ts from these gives

-1 0 0 o 0 0 1 V* 0 o V*

~A (-i

317

-y~) -! ~

VYl 0 0 0 o 0 -Yl *

o 0 1 0 Yl Yl

o 1 o o

~ ) (~:I =( =~:),

o Uc -Xc

I+VYl Vc -Yc

-1 o

1 +y'* o

which relates the displacements of the joints Band C to the forces applied to these joints.

The definition of matrix multiplication has many consequences, some of which are . the following.

(a) The product of a p x n matrix and an n x q matrix is a p x q matrix.

(b) If A and X are the matrices defined by equations (9), then AX is the column vector whose elements are the expressions on the left-hand side of equations (7). Hence the notation AX introduced in equation (8) is consistent with the definition of matrix multiplication.

(c) In general AB =1= BA, even when both products exist (see example 2 above).

This is the main difference between matrix multiplication and ordinary algebraic multiplication. In carrying out manipulations involving matrix multiplications it is essential to keep factors in their correct order. It is of course possible for the product AB to exist but not BA (see example 3 above).

(d) Distributive law. It follows immediately from the definitions of matrix addition and multiplication that

(A + B)C = AC + BC, A(C + D) :;: AC + AD,

provided that all the sums and products exist. These results may be extended. For example

.

(A + BXC + D) :;: AC + AD + BC + BD.

Note that the order in which the factors occur in each product must not be changed, but that otherwise the matrices here follow the rules of ordinary algebra.

(e) Continued products. If A, B are conformable for the product AB, and B, C are conformable for the product BC, then (AB)C and A(BC) both exist, and

318

. . ear Algebra -

-~)

-'-I

Operations oi; rix Algebra

But the last expression is, by definition, the ijth element of AB, so (BTAT)T = AB.

are equal. They may be written unambiguously without brackets as ABC. This also may be extended so that, if the matrices are suitably conformable, products ABCD, and so on, can be formed.

(f) Powers of a square matrix. When A is an n x n matrix, any positive integer power AP is defined as

AP = AA .... A (to p factors).

(g) Multiplication by a unit matrix. If A is a p x q matrix, then AIq = IpA = A.

(I8)

Equation (17) follows by transposing both sides of equation (I 8). The result extends to any number of matrices so that, provided the matrices are suitably conformable

(ABC ... FG)T = GTFT ... CTBTAT.

Note the reversal of the order of the factors.

(j) If A is p x n, B is n x q, the calculation of AB requires pqn multiplications and pq(n - I) additions. If p, q and n are large, the calculation is laborious. It is, however, one which is easily and rapidly performed on a digital computer.

The unit matrices playa similar role in matrix algebra to that which the integer one plays in ordinary algebra.

EXAMPLE 5
let A = (~ 3 :).
4
Then
~)G 0 D"
c 3
I
4
0 (1+0+00+3+00+0+5) (135) 2+0+0 0+4+0 0+0+6 = 2 46'

A= (_~ ~), B=(~ _~),

A + B, A - 2B, A + 2AT - 3B +4BT, AB, BA, A2, ABAT.

(0101)(2135)=(1+03+05+0) (1365).

4 6 0+2 0+4 0+6, = 2 4

3 -: =~),

_:6 4 2

B=(221 ~ -~ -~),

-I I .

(h) Multiplication by a zero matrix. If the products exist, then OA = 0 and AO = O. A zero matrix corresponds to the zero in ordinary algebra:

(i)

Transpose of a product. If A and B are conformable for the product AB, BT and AT are conformable for the product BT AT, and

(ABl = BTAT.

To prove the result (17), suppose that A = (ajj) is a p x n matrix and B = is an n x q matrix. Then AT is the n x p matrix whose ijth element is aj;, ,,' BT is the q x n matrix whose ijth element is bjj, so BT AT exists, and its ijth element is

2A + 3B, A - 2B, ABT and ATB.

(I 2

B= 3 2 o 0

I --4)

-2 3,

o -I

0) (1-2

o , 'C= 0 2

-I 0 0

AB, BA, AC, A + AT and AA T. Are any of these either (a) triangular or symmetric? If so, could that property have been deduced before the calculation?

r=l

The ijthelement of (BT AT)T is obtained by interchanging i and j, so it is

n n

I brjajr= I ajrbrj ..

r=l . r=l

320

(22)

r : ~ar Algebra -

3'"

5. Let

for a third-order determinant

(21)

al i al2 al3 =alla22a33 -alla23a32 +aI2a23a31-aI2a2Ia33
a21 a22 a23 +a13a21a32 -aI3a22a31'
a31 a32 a33 By considering the product

(XI X2) (:J,

prove that x~ + x~ = z~ + z~.

is essential at the outset to appreciate the distinction between a matrix and a A matrix is an array of numbers; it cannot be expressed as any single

A determinant is a single number which is obtained by performing a certain of mul tiplications, additions and subtractions on the members of a square

of numbers. With every square (but not with anon-square) matrix A there is

CC __ .:_ •• ." a determinant, denoted det A or 1 A I, but A and its determinant are quite

6. If

A= ( 0

-2

~ ),

-2

-1 3

:Jp4'111S,ion of a Determinant

first introduce two definitions.

(a) Minors If the ith row andjth column of the determinant (19) are deleted, is left a determinant of order n - I. This is called the minor of ai; and is denoted

find A2 and A3, and verify that A3 - 2A2 - 2A - 71 = o.

7. Prove that any square matrix can be expressed as the sum of a symmetric matrix and an anti-symmetric matrix.

Cofactors The cofactor Ai; of ai; is the signed minor given by '+'

Ai;.= (-I)' 'lvIi;'

8.3 DETERMINANTS

It is assumed that the reader has some familiarity with the theory and manipulation determinants. This section summarizes, largely without proof, their main properties.· An nth-order determinant is written

all al2 al3 al4
a21 a22 a23 1124
a31 an a33 a34
a41 a42 a43 a44
al I al2 al4 all al3 al4
M23= a31 a32 a34 , M42= a21 a23 a24
a4 I a42 a44 a31 a33 a34
A23 = (_1)2+3 M23 = -M23, A42 = (-1)4+2IM42 =M42. and is a certain homogeneous polynomial of degree n in the n2 quantities all, a12· .. ann· In fact it is a sum of all the products of the type ±alia2ja3k ... where i, j, k ... q are the integers I, 2, 3 ... n, in some order. We shall not go into rule for determining the sign of a given term. In particular, for a second-order determinant

. . An nth-order determinant of a square matrix A is now defined in terms of . _ I jth-order determinants (its cofactors) by the formula

det A = aliA I I + al2A 12 +al 3A I 3 + ... + alnA In~

a second-order determinant is explicitly defined by equation (20), this formula determinants of orders 3, 4, 5, ... , n successively in-terms of their

322

Linear Algebra _.

EXAMPLE 2

all al2 al3

a21 a22 a23 =alIAII +aI2AI2+aI3AI3 a31 a32 a33

= a I I (a22a3 3 - a2 3a3 2) - a 12 (a2 I a3 3 - a2 3a3 d + al 3 (a2 la3 2 - which agrees with equation (21).

A very important property of a determinant, from which most of its other properties follow, is that det A may be evaluated by carrying out an expansion of type given by equation (22), using any row 01 any column. Thus

detA=allAII +a12A12 +a13A13 + +alnAln

=a21A21 +a22A22 +a23A23 + +a2nA2n

=an1Anl +an2An2 +an3An3 + +annAnn

=alIAII +a21A21 +a31A31 + +anIAnl

= ...

=alnAln +a2nA2n +a3nA3n + ... +annAnn·

That is, the determinant may be evaluated by taking the elements of any of its or any of its columns, multiplying each element by its corresponding.cofactor, and adding. This process is called the Laplacian expansion of the determinant.

It is by no means obvious that the lines of equation (23) each yield the same For second and third-order determinants it is easy to verify directly that they do. the general determinant of order n, the equivalence of the different expressions can be proved by induction, by assuming them true for determinants up to order n - 1, and deducing their truth for determinants of order n.

Rules for Manipulation of Determinants

We can now list a number of results which follow more or less directly from the Laplacian expansion.

(a) If all the elements in any row (column) of a determinant are zero, then value of the determinant is zero. This follows immediately on expanding the row or column of zeros.

(b) If only one element in a row (column) is non-zero, the determinant is the product of that element and its cofactor. The proof is again to expand by this row or column.

,323

If the rows and colum,d of a determinant are transposed, the value of the determinant is unchanged; that is

det A = det AT.

The proof is by induction.

If all the elements of one row (column) of a determinant are multiplied by a constant k, the value of the determinant tsmulttplied by k. To prove, expand. by the row or column whose elements are multiplied by k. Note that this multiplication" rule is quite different from that for scalar multiplication of a matrix.

Interchanging two rows (columns) of a determinant changes the sign of the determinant. Again the proof is by induction.

'. If two rows (columns) of a determinant are identical, the value of the determinant is zero. This follows directly from (e).

If the elements of one row (column) 'are proportional to corresponding elements of another row (column) the value of the determinant is zero. This is an immediate consequence of (d) and (f).

Ifeach element in any row (column) is resolved into the sum a/two quantities, the determinant can be expressed as the sum of two determinants. For example

all +bll al2 aln
a21 + b21 a22 a2n
an 1 + bn I an2 ... ann
all al2 aln bll al2 aln
a21 a22 a2n b21 a22 a2n (24)
= +
an 1 an2 ... ann bn 1 bn2 . .. ann This can be verified by expanding each determinant by its first column, Note that the rule for addition of determinants is not the same as for ,addition of matrices.

,.

The value of a determinant is unchanged when to each element of any row (column) is added the corresponding element of any otherrow (column) multiplied by any constant factor. This property follows from (g) and (h). It provides a very useful method of numerically evaluating determinants,

" ,

If the elements of anyone row (column) are multiplied by the cofactors of the corresponding elements of any other row (column) the sum of the resulting products is zero. For example, if det A is a third-order determinant

324

, T 'near Algebra _

= 0, since the first two rows are identical.

(k) The determinant of an upper (lower) triangular matrix is the product of the elements on its leading diagonal.

(I) The determinant of a diagonal matrix is the product of the elements on the leading diagonal.

A further result, which does not follow readily from equation (23), but is of importance is:

(m) The determinant of the product of two square matrices of the same order is equal to the product of their determinants. Thus if A, B are square matrices of the same order, then

det(AB) = det(BA) = det A x det B.

The proof is quite involved and is omitted. A proof is given in Bell [1] .

Exercises
1. Evaluate the following determinants:
I ~~ : I' I~ -~ I' 2 -2 3 7 -5 -3 2 -1 0
I 0 4 , 6 4 I , 0 3 4
3 -5 8 9 5 2 -4
2 -1
2. Evaluate the following determinants:
a2 a4 a b+c+d bcd
b +c c +a a +b b2 b4 b c +d +a cda
b2 + c2 c2 +a2 a2 + b2 c2 c4 c d+a+b dab
d a +b +c abc 3. If (2 1

A= -2 3

4 5

-: ).' B = (~ -~

-3 0

. '5

/

AB, BA, det A, det B, det AB, det BA, and verify that

(AB) = det (BA) ~ det A x det B. '

2 x2 3-x 2 0

o

-2 4-x

(b)

2-x

4

-2

=0,

4 2 -x -2 = O.

-2

THE INVERSE OF A MATRIX

the development of the algebra of matrices we so far have no operation analogous , that of division in ordinary algebra. Division by a number a is equivalent to IIUlliJIIU''' .... v .. by its reciprocal a-I. In this section we construct a reciprocal or matrix A -1 which in matrix algebra corresponds to the reciprocal a -1 in algebra.

Let there be given an nth-order square matrix A = (aij) with cofactors Aij. The matrix of A is defined to be

is, the adjoint of A is obtained by replacing each element of A by its cofactor, transposing. Now consider the ijth element of the product A x adj A. This is given

11 n

(A x adj A)ij = L air(adj A)rj = Lair Ajr.

(26)

r= 1

r= 1

last expression in equation (26) is the sum of the products of the elements in the row of A with the cofactors of the corresponding elements in the jth row of A. If the last expression in equation (26) is just the expansion of det A by its ith row. all the elements of Ax adj A on the leading diagonal are equal to det A. If i =1= t.

the last expression in equation (26) is zero by (j) of §8.3. Hence

. 1 det A

(A x ad) A)ij = t 0

ifi = i, ifi =1= j.

, it can be shown that

{ det A

(adj A x A )ij = 0

ii i=], ifi =1= j.

A x adj A = adj A x A = Ide.~>~~~:::: ~ )= I det A.

o 0 det A

(27)

326

No ... suppose that det A ::/= O. Since det A is simply a number (scalar), the adj A can be divided by the scalar det A to form a new nth-order matrix. This denoted A -I , so that

, 1

A -I = --adj A. det A

Then it follows from equation (27) that AA -I = A -I A = l.

The matrix A -I defined by equation (28) is called the inverse of A. Its vital property is equation (29); multiplication on the left or right of a matrix by its gives the unit matrix. Hence matrix multiplication by A -I is a process ....... vl',uUlj multiplication by a number a-I (or division by a) in ordinary algebra.

EXAMPLE 1

(1 2

If A = 1 3 1 5

~),

12

find A -I.

We have

A21 = _12 1~ 1 = -9, A22 = 1 ~ 31 = 9,
. 5 12
A31 = I~ ~ 1 = 1, A32=_ll 31 .
5 = -2,
. 1 A23 = -I ~

An = 1 ~ ~ 1 = 1,

det A = 1 x 11 + 2 x (-7) + 3 x 2 = 3,

(11 -7 2)T (11 -9 1)

adj A = -9 9 -3 = -7 9 -2 ,

1 -2 1 2 -3 1

of a Matrix

327

(1 2

AA -I = ~ 1 3 1 5

(11 -9

-IA=~ -7 9 2 -3

.the consequences of the definition (28) are the following.

A -I cannot be defined if det A = O. The inverse matrix A -I exists only if (1) Ais a square matrix, and (2) det A::/= O. If det A = 0, A is said to be singular; if det A::/= 0, A is said to be non-singular.

The inverse of a symmetric matrix is symmetric.

The inverse of a diagonal matrix with diagonal elements all, a22, ... ,ann is a diagonal matrix with diagonal elements all, aiL ... ,a;~.

The inverse of an upper (lower) triangular matrix is an upper (lower) triangular matrix.

The inverse of the transpose of a matrix is equal to the transpose of its inverse, that is

(AT)-I = (A -I )T.

This follows easily from the definition of adj A and equation (28). Some further results follow from the fundamental property (29) of A -I.

Taking the determinant of both sides of the equation (29) gives det (AA -I ) .,;, det A x det (A -I) = 1, so that det (A -1)= (det A)'"I .

.. For any positive power p

AP(A -I)P = AP-l AA -I(A -I)P-l = AP-l(A -I)P-l = ... = I. .

Hence (A -I f = (AP) -1. It follows that the rule AP Aq = AP+q applies to both positive and negative powers p and q.

Inverse of a product Let C = AB, where A and B are both n x n square non-singular matrices (since det AB = det A x det B, it follows that C = AB is non-Singular). Multiplying both sides on the left by A -I gives

A -I C = A -I AB = IB = B.

Now multiply both sides on the left by B-1, to obtain B -I A -I C = B-1 B = l.

328

Linear Algebra-

3T

. __./

Finally, multiplying both sides on the right by C -I gives B-IA-I =IC-I =C-I.

Thus, since C = AB,

(AB)-I = B-1 A-I.

Note that, just as itt the transposition of matrix products, the order of the factors is reversed. In general, if all the matrices are n x n square non-singular matrices, then

XI =(Allbl +A21b2 +A31b3 + +Anlbn)/detA,

X2 = (A 12b2 + A22b2 + A32b3 + +An2bn)/det A,

(32)

Xn = (A Inb I + A2nb2 + + Annbn)/det A.

expression in brackets in the first equation above is just the expansion by its first of the determinant which is obtained from det A by replacing the first column by a column b I, b2 , ••• , bn·

(ABC ... G) -I = G -I ... C -I B -I A -I.

Solution oj Linear Equations

If A is an n x n square matrix, and X and Bare n x I column vectors, the equation AX= B

bl al2 al3 ... aln I all al2 al3 ... aln
b2 a22 a23 ... a2n a22 a23 ... a2n
XI = ......... . ........
. ........ . ........
bn an2 an3 ... ann ani an2 an3 ... ann represents (see §8.1) a system ofn linear algebraic equations in n unknowns. that A is non-singular, each side of equation (30) may be multiplied on the left by giving

there are similar expressions for X2, X3, ••• ,Xn (to obtain.the numerator of the mre:ssi()ll for x., replace the rth column of det A by b I, b-i, ... ,bn). This is Cramer's for the solution of a system of linear algebraic equations. It is not an efficient

of solution for systems of more than three equations, and should not De used . It does, however, highlight the fact that the solution encounters serious .... , ... y&,.ov_when det A = 0 (i.e., A is singular), because each of equations (32) then the impossible operation of division by zero. In practical problems, difficulty

also arise when det A is non-zero but numerically small. This situation is discussed § 10.2.

A -I AX = IX= X = A -I B.

Hence X = A -I B is the solution of equa tion (30), and the unknowns x I , X 2, •.• are found by a matrix multiplication once A -I is known.

EXAMPLE 2

Solve x + 2y + 3z = 4, x + 3y + Sz = 2, x + 5y + 12z = 7.

lfiJ.uenless of the Inverse

be shown that if A is non-singular then A -I, as defined by equation (28), is the matrix with the properties AA -I = 1 and A -I A = I. Suppose A possesses another inverse' R such that AR = I. Then

~ ~) and, from example I, A -I = 5 12

Hence

R - A -I = I(R - A -I ) = A -I A(R _ A -I) = A -I (AR _ AA -I ) = A -I (I - I) = A -I 0 = O.

R = A -I, and similarly the only 'left inverse' of A, satisfying LA = I, is L = A -I. value of this result is that it shows that if by·any method a matrix Lor R can be such that LA = 1 or AR = I, then L or R is the inverse A -I. This is important, although the definition (28) proves the existence of the inverse of a nonmatrix and its properties (a}-(e), it does not provide, for n > 3, an efficient

of evaluating A -I . To illustrate the magnitude of the problem, the direct

!O.l'Ui ... 'VU of adj A involves evaluating n 2 determinants of order n - 1. This means n(n!) multiplications. For n = 20 approximately 5 x 1019 multiplications would

so x = II, y = -8, z = 3. It is easily verified by substitution in the equations that the correct solution.

Written in full, using the definition (28) of A -I, the solution X = A -I B of

330

.~ inear Algebra -

be required. There are about 3 x 107 seconds in one year, and a fast computer might perform 5 x lOs multiplications per second. Working non-stop, the calculation be complete in about three million years. For n = 100, which is not large in contemporary applications, the number of multiplications is about 10160, and the time required is about 6 x 10146 years. For comparison, the energy of the sun is expected to last about 3 x 101 0 years. The direct method of inversion is clearly quite impracticable for large matrices. Fortunately, more efficient methods of matrix inversion are available, and the most useful of these are described in § lOA.

Exercises

I. For each of the following matrices, find (1) all the minors, (2) all the cofactors, (3) the adjoint matrix. (4) the inverse matrix. In each case verify that the matrix pre. or post-multiplied by its inverse gives the unit matrix.

(b) ( 3 -2 ~). (c) C 1 4)
-I 2 -I 3 0 ,
0 0 -2 3 I I
(e) C -I -3) (0 C -2 3)
I 0 -2 , o -4 5 .
3 2 0 006 (d) (1 2 3)

2 I 2 ,

321

2. Use the results of exercise I to solve the following systems of equations.

(b) XI +2x2 +3X3 ==4, 2x1 + X2 +2x3 == I, 3xI + 2x2 + X3 == 7.

3. For the matrix A of §8.2, exercise 6, fmd A -I and verify that 7A -I = A2 - 2A

4. Find the inverse of the matrix

( -i

-I o

I +..jYz o

o I o o

which occurs in example 4 of §8.2. Hence, in the notation of that example, express UB, VB' Ue and Ve in terms of XB, YB,Xe, Ye. (This is a simple illustration of an important method in structural analysis).

ORTHOGONAL MATRICES

.. ' A square matrix M is said to be orthogonal if it has the properties

MMT = I, M1'M = I, MT = M -I .

(33)

three statements are equivalent; anyone of them implies the other two. Also det MMT = (det M)2 = det I = I,

det M = ± I. Note that det A ::: ± I is a necessary but not a sufficient condition for A . be orthogonal. For example, if

A=(: ~),

.. then det A = I, but A is not orthogonal.

Orthogonal matrices arise in a number of engineering applications, but are usually . first encountered in connection with coordinate transformations. In the first instance 'ClIJl~J.U'" a rotation of plane rectangular cartesian coordinates through an angle 0, as IUUi'''uw~ in Fig. 8.3. In the coordinate system Ox IX2 P has coordinates (x 1> X2); in system Ox I X2 it has coordinates (x I, X2)' From elementary plane coordinate

_A~.""D,Tr" these coordinates are related by

X I = X I cos 0 + X 2 sin 0, X2 =-XI sinO + X2 cosO,

XI =XI cosO -X2 sinO, X2 =XI sin 0 + X2 cos O.

matrix notation these may be written

(COS 0 sin 0 ) . where M =

-sin 0 cos 0

MMT = ( cos 0 sin 0) (COS 0 -sin 0 )

\ -sin 0 cos 0 sin 0 cos 0

X=MX,

(34)

= ( cos 2 0 + sin2 0 0 ) = I

o cos2 0 + sin2 0 .

the matrix which describes the rotation from one plane rectangular cartesian [CC)Orl(1lnate system to another is orthogonal.

Analogous results hold in three (or more) dimensions. If a typical point P has coordinates (XI ,X2, X3) in a rectangular cartesian coordinate system Ox IX2X3, and i, (XI,X2,X3) in another rectangular system OXIX2X3, then X = (XI X2 x3l and

X= (XI X2 X3"T sat,isfy the relations .

X = MX, X = MTX, (35)

where M is an orthogonal matrix. Geometrically, the rows of M are the direction cosines of the axes OXI, OX2. OX3 with respect to the coordinates OxIX2X3, and the

332

I t=ear Algebra -

(38)

x,

physical systems (for example many electrical networks and framed- structures)

composed of subsystems which are connected by a relatively small number of

Such systems often give rise to sparse matrices, which may be large, but contain proportion of zero elements. Also, the non-zero elements are often arranged in a pattern within the matrix, and frequently' incomparatively small blocks. This it profitable on occasions to deal with groups of elements within a matrix,

than with the matrix as a whole. For example, the matrix of the coefficients of lquat:ions (3),

If X = (XI X2 ... xnl, X = (XI X2 ... xn)T, and X = MX, where M is orthogonal, that x] +x~ + ... +x~ =xI +x~ + ... +x~.

PARTITIONED MATRICES

XI

Fig. 8.3 Rotation of plane rectangular cartesian coordinates

columns of M are the direction cosines of the axes OXI , OX2, OX3 with respect to the coordinates Ox IX2X3.

The same orthogonal matrix M describes the transformation of the components of physical vector from the coordinate system OX IX2X3 to OXIX2X3. If the vector has components (FI, F2, F3) in the first system and (FI' F2 ,1':'3) in the second system, and we denote F = (FI F2 F3)T, F = (FI F2 F3)T, then

F=MF, F=MTF.

o -1 o

(37)

o 0

o 0

A= -----------
0 I y'Yz, 0
I
I
0 0 I y'Yz
I divided up in the indicated manner.

Exercises 1. By calculating MMT, verify that the following matrices are orthogonal, and show , directly that det M = 1.

2. Two sets of rectangular cartesian coordinates are related by XI = I/lXI + %X2 - %X3,

_ 2y'2 I I

X2 = - -3-xl + 3y'2'X2 - 3y'2 X3,

I I

X3 = y'2X2 + y'2 X3·

Use the result of exercise I (b) to express X I ,X2, X 3 in terms of x I , X2, X 3. With respect to the (x I, X2, X3) coordiru es a physical vector has components (2, -3, 5). What are its components with respect to the (x I, X2, X3) system?

(a) M = ~ (1 2y'2)'

3 -2y'2 1

that from an m x n matrix A we strike out m - p rows and n - q columns rows and columns need not be consecutive). This leaves a p x q array of the which is called a submatrix of A.

matrix is divided into submatrices by vertical and horizontal lines between the and rows it is partitioned. For example

C=

CII : CI2 CI3 : C14)

I I

C2 1 I Cn C23 : C24

---~-------~--- .

I ,

C31 i C32 C33 I C34

3. Prove that if S is any anti-symmetric matrix, then (I - S)(I + S) -) is orthogonal.

denote the first sub matrix in A by AI, the second by A2, and so on, and write the

334

Linear Algebra -

matrices as

diagonal matrices

matrix which may be partitioned in to the form

All i 0 I 0

I I I

___ ~ J ~ _

I I I

I I I

o I AZ2 I i 0

I" r

I I

A = ------------------

A=(~J

Multiplication of Partitioned Matrices

As an example, form the product AB of A and B in equations (38). By direct multiplication, this gives

o

AB=

I

I 0

I

I

I Ann

(41 )

each sub matrix Au is a square matrix, is said to be block diagonal. It is easily that its inverse is

a way that All and All are square, and All and A22 are non-singular. Let A-I be the inverse of A and partition B so that A and B are conformably IIrtltllom~u for the product AB. Then

I=AB= (All AIZ),(BII BI2) = (AIIBII +A12B21 AIIBI2.+~I2B22)

A21 A22 ' B21 BZ2 A21BII +AllB21 AZIB12 +A22B22 .

Note that the multiplication is carried out just as though A and B were column and row vectors respectively. It is of course essential that the partitioning be made in a way that the matrix products AI BI, AI B2 etc. all exist; matrices for which this is . ," the case are said to be conformably partitioned. ,

The general rule for forming the product of two conformably partitioned matrices' is exactly the same as the rule for matrix multiplication, with the sub matrices acting though they were matrix elements.

EXAMPLE 1

A-I =

A-I II

i 0

I

o

I

I A-I I 22

I

o

I

I 0

2

o

of Partitioned Matrices

a square matrix A partitioned as follows

2

o

2

o

2

o

-------_ --------
0 i 3 1 I 0 i 3
I I
I I
0 I I 3 0 I I 3
I I (I 0) (I 2) + (3 I) (I O)! (I 0) (I 0). + (3 I) (I 3)

0121 130110101 1331

AIIBI2 +AIZB22 =0, Az I BIZ + AZ2 Bzz = I.

(43)

5 5 3 4
5 5 4 3
---------
4 3 7 10
3 4 10 7 equations (43) for BII, Bzz, BIZ, BZI gives

. -I' (' (AII-AI2Ai'!AzI)-1

B=A =

-Ai'!A21(AII -A12Ai'!A2d-1

-'AilAlz(All - A2IAilAI2)-1 )

(A22-A2IAilA12)-1 .

{4.d,\

1 0

I

(42)

I

I 0

I

I

I A~h

/

this section the solution of the system of linear equations (7) will be investigated IlermnJL~'Ul) • In the matrix notation equations (7) have the form AX = B, where A is a m x n matrix, B a given m x I column vector and X an unknown n x I column which, if possible, has to be determined. AX = B represents a system of m

in n unknowns; m and n are not necessarily equal, and we may have m > n, " nor m < n. If B = 0, so that the right-hand sides of equations (7) are all zero, the are said to be homogeneous; if B "* 0, so that at least one of the equations

a non-zero right-hand side, the equations are inhomogeneous. In this section considered to be inhomogeneous.

already noted, a system of inhomogeneous equations may have either a unique , no solution, or an infinite number of solutions. These three possibilities are by the following simple numerical examples.

+X2 +X3 =3, Precisely one solution XI = I,X2 = I,X3 = 1.

XI+X2-X3=1, (45)

XI - X2 + X3 = 1.

system of equations is said to be consistent (or compatible) if it has at least one and inconsistent (incompatible) if it has none. An important requirement for , investigation of a system of linear equations is a test which will determine whether

336

"inear Algebra -

This result is easily verified by direct multiplication. To evaluate A -I in this way involves four matrix inversions, but the matrices may be chosen to have only of the order of the original matrix, so this approach is useful when computer storage capacity is limited. It is also suitable for the inversion of sparse matrices, especially the corresponding linear equations and unknowns are ordered so that the matrix is 'almost' block diagonal, with a small number of elements which do not fit the block diagonal form.

EXAMPLE 2

Invert the matrix A given by the equation (37).

In this case AI2 = 0, so equation (44) becomes

A-I =

(All 0 )

-I -I -I'

-A22A2IAII A22

where

All =C -I) ( y'lh ~), A21 = (~ ~).
o ' An = y'lh
Now
-I ( 0 ~), A -I _ (y'2 ~) , (-y'2 0)
All = 22 - A2"!A2IAll= 10'
-I . -I
Hence ( -~ 0 ~ ~.).

A -I = -J: 0 y'2 0

-I 0 -I I

This is easily verified by forming the product AA -I.

Exercises

I. By introducing suitable partitioning, evaluate the following matrix products:

(a)

(b) 0 0 I
I I 0 0 0 I I
0 0 2 0 0 2 0
0 0 2 2 2 0 2
2 0 0 2 0 2 0 partitioning suitably, find the inverse of the following matrices:
2 4 0 0 0 0 (b) (-I -I I -}
I 4 0 0 0 0 4 0
0 0 2 4 0 0 0 4
0 0 I 4 0 0 -I
0 0 0 0 2 4
0 0 0 0 4
INHOMOGENEOUS SYSTEMS OF LINEAR EQUA nONS reading this section the reader is recommended to refer to the first part of § 8.1. described some physical problems which give rise to systems of linear algebraic

XI +X2 +X3 =3, XI+X2-X3=1, XI +X2 = 5.

No solution; adding the first two equations gives

X I + X2 = 2, but this contradicts the third. The equations are contradictory and there are no values of X I, X 2 and X 3 which satisfy them all.

The, first two equations imply the third, so the third gives no additional information and is redundant. There is ail infinite number of solutions, say XI = t,X2 = 2 - t,X3 = I, where t may have any value.

XI +X2 +X3 = 3, XI+X2-X3=1, XI +X2 = 2.

338

(~ : ~~ ~).

o 0 3

IlnearAlgebra -

ImroEjmeOUS Systems of Linr»: Equations '_;;

339

or not the system is consistent; if the equations can be shown to have no solutions then there Is no point in proceeding further. One of the main purposes of this sectioD:

is to devise such a test. .

4 -2 -2

~).

11

XI +4X2 -2x3 =3, + X3 = 1, + 2x3 = 11.

2

The Augmented Matrix

Given a system of equations AX = B, it is convenient to introduce the augmented matrix (A I B) which-is the matrix of coefficients A augmented by B as an addi column. For example the augmented matrix for the system of equations (45) is

X I from the second and third equations, by subtracting twice the first from the second, and three times the first equation from the third.

(1 4 -2 i

o -10' 5!

o -11 8 i

XI + 4X2 - 2x3 = 3, - 10x2 + 5x3 = -5,

- llx2 + 8x3 = 2.

3) ,

-5 .

2

(: : -1 ~).

I -1 1 1

the second equation by -10 and eliminate X2 from the third equation.

(~ : ~~

o 0 2~

The augmented matrix completely describes the system of equations.

XI +4X2 - 2x3 = 3,

~ ).

7~

Elementary Row Operations

The following operations can be performed on a system of equations without their solution:

the third equation by 2~.

(a) interchanging any two equations. (b) multiplying or dividing any equation by a fmite non-zero constant. (c) adding to any equation any other equation multiplied by a finite constant. For each of these operations there is a corresponding operation on the aUI~€mtc:di

matrix of the system, namely,

set of equations can now be solved in, turn for X3, X2 lind X I, giving 3,X2 = 2,XI = 1.

(a') interchanging any two rows of the matrix.

(b') multiplying any row of the matrix by a finite non-zero constant.

(c') adding to any row of the matrix any other row multiplied by a finite These are called the elementary row operations on the augmented matrix.

X3 +3X4 = 0, t: -3 -1 3 I~}
2 4
-2 -2
XI from the second and third equations.
, XI - 3X2 - X3 + 3x" == 0, r -3-1 3 I~}
+7X4 = 11, o -1 0 7
7X2 + X3 -8x4 = 6. 071 -8 Solution by Elimination (Pivotal Condensation)

In elementary algebra the reader will have learned to use the operations (a) (b) (c) . " solve systems of linear equations by eliminating each variable in turn. When . systematically applied, this method is an efficient numerical method of solution of linear equations, which is described in § 10.2. In this section we' are more "~"'''''IIU~U with theoretical considerations.

In the examples which follow, in order to make the process systematic the X I, X2 ,X 3, . .. are eliminated in turn. We also use operation (b) to arrange that, in , the flnal system, each variable has the coefficient 1 in the equation in which it fust··· occurs. The corresponding augmented matrix is shown to the right of each system equations.

the second equation by -1 and eliminate X 2 from the third equation.

(1 -3 -1 3 I 0)

o 1 0 -7 l -11 .

o 0 1 41: 83

0,

7x4=-II, X3 + 41x4 = 83.

ecuanons do not have a unique solution; for example X4 may be given any value, the equations can be solved in turn for X3, X2 and x I. The solution can be

340

r·inear Algebra -

expressed in the form

X4=t, x3=83-4It, x2=-11+7t, xl=50-23t,

where t may be any number.

.11f.m(){!e,P'lelJUS Systems of r, '''lear Equations

~

?

XI + X2 -x3=1, XI +2x2 -X3 = 2, XI + X2 + X3 = 6,

2xl -~X2 +X3 = 1.

(: ~ =: ~)

2 -2 I

Redundant Equations The process of elimination may lead to one or more equations of the form 0 = O. This

indicates that one or more of the original equations can be derived from the .

The system then contain's equations which are redundant in that they contain no information which is not present in the remaining equations.

EXAMPLE 3
XI + X2 - X3 = 0, (i -I ~)
XI + 2X2 - X3 = 2, 2 -I
XI + X2 + X3 = 6,
2x I - 2x2 + X3 = 1. -2 Eliminate X I from the second, third and fourth equations.

XI +X2 - XJ =0, (~ -I n
X2 = 2, I 0
2x3 = 6, 0 2
-4X2 + 3X3 = 1. -4 3
Eliminate X2 from the fourth equation.
XI +X2 - X3 = 0, (~ -I ;).
X2 = 2, I 0
2x3 = 6, 0 2
3X3 = 9. 0 3 Divide the third equation by 2 and eliminate X 3 from the fourth.

XI+X2-X3=0,

X2 = 2,

X3 = 3,

0=0.

Although there are initially more equations than unknowns the system has the solution XI = I, X2 = 2, X3 = 3 because one of the equations is redundant.

Inconsistent Equations The process of elimination may also lead to one or more equations of the form 0 This indicates that the original equations contradict one another and have no the system is inconsistent.

This differs from the system of example 3 only in the right-hand side of the first 1IU .... Uvu. Carrying out the same sequence of operations as in example 3 leads to the

0=1.

system is inconsistent and has no solution.

Row Echelon Reduction of a Matrix

practice the elimination process is most conveniently carried out by operating on

augmented matrix (A I B) rather than on the equations themselves. In each of 1-4 the final augmented matrix displayed has the properties

, The first non-zero element in any row is a 'I'.

•...• (2) This' I ' lies to the right of the first non-zero element in the row above it,

this form was achieved by performing elementary row operations (a') (b') (c') on augmented matrices of the system. A matrix with the above properties (1)

is said to be in row echelon form. Applying the elimination process to a system fequatilons is equivalent to reducing the augmented matrix of the system to row

form by sucessively applying elementary row operations.

reduction of the augmented matrix to row echelon form is not unique; it may in many different ways but each will yield the same solution (or absence of of the associated system of equations. It should be noted that the successive matrices obtained during the red uction are in no sense equal to one

They are merely connected by elementary row operations. It is also important imrlern:ber that the operations may only be carried out on the rows of the I matrtx, on its columns.

the augmented matrix (A I B) of the system of equations AX = B is reduced row echelon form (AR I BR)' Then (AR I BR) is called the reduced augmented If (AR I BR) contains a row of the form

. (46)

thie rnw r.nrrplI:.nnnti~ tn "lin .Oo"'II~+~""" n - 1 .... ~....1 4.L ..I.!_

IIfIIlllllenleOI:lS Systems or Linear tiquauons

The simplest case is m = n ,,- . ./ A is theri a non-singular square matrix (that is,'" A:/; 0, and A -I exists) and the system has the unique solution X = A -I B.

... Ifm > r, then m - r of the equations are redundant.

If n > r, the equations are consistent but do not have a unique solution. Arbitrary can be given to n - r of the unknowns, and r equations solved for runknowns

terms of the remaining n - r unknowns. .

. ..... :1-4 above illustrate some of these possibilities. In each of these examples

. and r(A I B) can be read off from the displayed reduced augmented matrices.

342

Linear Algebra -

Rank of a Matrix

The rank r(A) (or simply r) of a matrix A is the largest integer r for which there a non-singular r x r submatrix of A (non-singular matrices were defined in §8.4 and sub matrices in §8.6). For instance, the matrix

G ~ ~ D

is of rank 2, because, for example

I ~ ~ I :/; 0, or \ ~ ~ I * 0,

but the determinants of all the 3 x 3 sub matrices are zero. If A is a non-singular n x matrix, so' that det A * 0, then r(A) = n. If A is an m x n matrix, no submatrix can have more than m rows or n columns, so r";; m, r";; n.

Row Echelon Reduction and Rank

By using elementary properties of determinants it can be verified that the row operations (a') (b') (c') do not change the rank of a matrix A. Hence if A has

r, so does any matrix derived from A by elementary row operations. in particular,if is reduced by elementary row operations to a matrix AR which has row echelon then AR has rank r also. Moreover the rank r of a matrix which has row echelon is easily seen to be the number of rows which do not consist entirely of zeros. Thus method of finding the rank 'of any matrix.is to reduce it to row echelon form, and count the number of rows which do not consist entirely of zeros.

Rank and Consistency

It has been Shown that the equations AX = B are inconsistent if the reduced augmented matrix (AR I BR) contains a row of the form (46). The corresponding of AR then consists entirely of zeros. Hence r(AR I BR) = r(AR) + 1. But

r(AR I BR) = r(A I B), r(AR) = r(A), and so we have the following alternative form the test for inconsistency: the system AX = B is inconsistent if r(A I B) >r(A).

Conversely, if r(A I B) = r(A), the reduced augmented matrix does not contain a row of the form (46), there is no contradiction, and the equations are consistent.

All possibilities are now covered for, because A is a sub matrix of (A I B), it is not possible to have r(A I B) < r(A).

For each of the following pairs of matrices A, B

reduce the augmented matrix (A I B) to row echelon form hence find the rank of A and the rank of (A I B)

state whether or not the system AX = B is consistent

in the consistent cases, solve the system AX = B

A=(~ ~ :} B=(:} ~) A=G =: =~). B=G}

A=(~ -: -~), B=(:)' (d) A= C : ;), B= (~),

2 -1 -3 2

. ,

A= (~ ~ ~). B=m· (n A=(~ ~ l B=m·

HOMOGENEOUS SYSTEMS OF LINEAR EQUA nONS

system of equations (7) is homogeneous if all the constants bi ; bs ; ... ,bm are In matrix notation the system of homogeneous linear equations has the form

=0 .

. A system of homogeneous equations always has the solution

XI = 0, X2 = 0, r, X3 = 0,

Xn =0,

(47)

Summary of Cases

Now suppose that the equations are consistent, so that r(A I B) = r(A) = r. Recall r";; m, r";; n, where m is the number of equations and n the number of unknowns in the system AX = B.

more concisely, X = O. Since the system always has this solution, it is always

...... '~i.~tjmt. The zero solution (47) is called the trivial solution. Interest usually centres whether or not there exist additional solutions besides the trivial solution; such IOIIlIUUI11~, if they exist, are called non-trivial solutions. It is clear that the most we may to determine in finding a non-trivial solution is the ratios of the variables

344

Linear Algebra -

XI, X2, ... ,Xn, because if X I, X2, X3, ... ,Xn satisfy the equations, so do QXI' QX2, QX3, ••• ,QXn, where 0: is any constant.

Test for Non-trivial Solutions in the Case m =: n

The most important case is that of n equations in n unknowns, so that A is a square matrix. Suppose that det A =f=. 0, so that A -I exists. Then the solution of AX = 0 is X = A -10 = 0 so that the only solution is the trivial solution. Hence a necessary condition for the existence of non-trivial solutions is that det A = 0. It can be shown that this is also a sufficient condition. Thus a necessary and sufficient condition for system AX = 0 of n equations in n unknowns to have non-trivial solutions is that det A = 0.

EXAMPLE 1

Find the non-trivial solutions of the system

It is easily verified that the determinant of the coefficients is zero, so that non- . trivial solutions exist. Consequently one of the original equations is redundant, and elimination we find that the ~iginal system is equivalent to the new system

XI+X2+ X3+ X4=0,

X2 + 5X3 + 3X4 = 0,

X3+ X4=0.

Hence, solving in turn.x , = -X4,X2 = -5X3 - 3x4 = 2X4,

Xl = -X2 - X3 - X4 = -2x4' so that X I :X2 :X3 :X4 = -2: 2:-1: I, or

XT = c(-2 2 -I I), where c is any constant.

The Casem =n

The general case in which m =f=. n can be dealt with by the row echelon reduction an inhomogeneous system. The main difference is that since now B = 0, the i1U~~menbll matrix (A I B) consists of A augmented by a column of zeros, so that any sub matrix (A I B) is either a submatrix of A or contains a column of zeros. Hence r(A) = r(A I this confirms that the system is always compatible, having the trivial solution X=

The conditions for the system to have non-trivial solutions can be obtained by minor modifications of the arguments used in §8.7. The necessary and sufficient conditions for the existence of non-trivial solutions is that r(A) < n. It is of course necessary that r(A) ~ m.

.34C

The solution of simultaneous linear ordinary differential equations with constant

ts by the method of elimination was described in § 1.9. For systems of :'''nmOl!en:eOl. ,H equation~ of this type it is usually preferable to proceed in a similar way

that which was used In the solution of a single homogeneous linear differential

,with ~onstant coefficients; that is, by substituting into the equations a trial tWIUUlJll In which the unknowns are exponential functions of the independent variable. differentia~ equations are thereby reduced to a homogeneous system of linear 'a1l!ebraic equations, The following examples illustrate the method.

AMPLE 2

Solve the simultaneous differential equations

(3~ + 7)y + (3~ + 5)z = 0, (40+11)y- (D+5)z=0, D denotes the operator d/dx.

.. This system was solved by elimination as example 2·of § 1.9. Here we proceed by ,seeking solutions of the form

y = Aemx , z = Cemx.

substituting these into the equations and cancelling the factor emx, it follows that (3m + 7)A + (3m + 5)C= 0,

(4m + ll)A - (m + 5)C= 0. (48)

require non-trivial solutions in which A ande are not both zero. These non-trivial toiU:tiOilS are possible only when m is chosen so that

3m + 51 = 0.

-m-5

I:ltDaI1UUl~ the determinant and Simplifying gives m 2 + 5m + 6 = 0, with solu tions -2 and m = -3. When m = -2, equations (48) become .

A - C = 0, and 3A - 3C = 0,

have the non-trivial solution A = C Hence one solution of the differential is

z =Ae-2x.

, taking m = -3, and replacing A and Cby Band E respectively, equations become

and -B - 2£= 0,

have the non-trivial solution E = -'h1J. Hence a second solution of the

346

dz

- + x + y + 2z = O. dt

'inear Algebra -

347

differential equations is

y =Be-3X, z= -!6Be-3x.

!Detenmlllie whether or not the following systems of homogeneous equations have solutions, and find these solutions when they exist:

The general solution is the sum- of the two solutions obtained above, namely, y =Ae-2x -s«>, z =Ae-2x _ y,...Be-3x,

which agrees with the result ob tained in § 1.9.

(b)

= 0,

X2 + X3 = 0, X I - X 2 - 2x 3 = 0, 2x1 -X2 - 3X3 =0.

(d) XI + 3X2 = 0,

2x1+2x2=0,

,

EXAMPLE 3

Solve the simultaneous differential equations

2x1- X2+3x3=0, 5xI - 4X2 + 2x3 = O.

dx

dr= -y - z ;

dy

-=-X- z

dt '

dz

dt =-X +y.

-XI +4X2 = O.

(f) XI + 2x2 + 3X3 = 0,

-XI - 4X2 + . X3 = 0,

-XI - 6x2 + 5x3 =0,

XI - 2x2 + IIx3 =0.

The trial. solution is X = Aemt,y::: Bernt, z = Ce?", and this satisfies the equations if

mA + B+ C=O, A +mB+ C=O, A - B+mC=O.

7XI+3x2t4x3+ X4=0, XI+5x2- X3+2x4=0, -2x1 -7X2 - 2x3 - 2x4 = 0, 2x1 + 3X2 + X3 + X4 = O.

m

= 0, which gives

.," Solve the following systems of differential equations:

dy dz

- + 3y - z = 0, - + 2z - y = O.

dt dt

(2D2 -6D+4)y+(D2 -D-2)z=0,

(D2 + D - 2)y + (2D2 + 6D + 4)z = 0, where D denotes d/dx.

For non-trivial solutions,

m

-1 m

m = 0, m = -I, m ::: I. WheH m = 0, the solution of equations (49) is easily found to A :B: C = 1: 1: -I. Hence, since eO = 1, a solution of the differential equations is

X ::: A I , Y ::: A I" Z = -A 1

Similarly, when In = -1, the solution of equations (49) is A:B:C::: 1:1 :0, so a second' solution of the differential equations is ".

d .

2 + X + 2y + z = 0, dt

dx

- + 2x + Y - z = 0, dt

equations governing the currents in the primary and secondary circuits of a IasJfonner, when there is no applied e.m.f. and resistances are neglected, are

. d2/1 d2/2 11_0 L d2/2+Md2/1 +12=0

L1""dt2 + M dt2 .+ C1 -, 2, dt2 dt2 C2 •

general solution by adopting -the trial solution

12 = B cos(pt + e). •

Finally, for m = I, the solution of equations (49) is A: B: C::: 1 :0: -I, which gives as

third solution of the differential equations _

The general solution of the differential equations is obtained by adding the three solutions given above.

ALUES AND EIGENVECTORS

In examples 2 and 3 the roots of the equation for m turned out to be real' numbers In other cases, some or all of these roots will be complex. It is then usually to express the solution in terms of sine" cosine and exponential functions in the manner described in § 1.5.

eigenvalue problems involve systems of homogeneous linear equations in

the coefficients depend in a certain way on a parameter. In general such a possesses only the trivial solution, but non-trivial solutions may exist for special values of the parameter. Many different physical and other problems be formulated as algebraic eigenvalue problems; they arise particularly in the

·348

- 'near Algebra - study of vibrations and of stability, and in finding principal axes of inertia, stress and strain.

A Vibration Problem

Consider the system of three springs (with moduli k I, k2 and k3) and three masses ml, m2 and m) shown in Fig. 8.4. The two masses m I and m2 are connected by a rigid rod of negligible mass, and the spring with mod ulus k 3 is connected to the mid. point of this. The system might be interpreted as a very idealized model of a motorcycle suspension, with m J and m2 representing the unsprung masses of the two and m ; the sprung mass of the cycle and rider. The vertical upward displacements of the three masses from their equilibrium positions are Yl, Y2 and j/j , as illustrated. Hence the extensions, from the equilibrium configuration, of the springs of moduli k I ,k2 and k 3 are, respectively, Y I ,Y2 and Y 3 - ~(y I + Y 2)' If there are no damping forces present, the equations of motion of the three masses are

mdil = -klYI + ~k3{Y3 - ~(yl + Y2)}, m2Y2 = -k2Y2 + ~3 {Y3 - ~(yl + Y2 )}, m3Y3 = -k3{y3 - ~(yl + Y2 n.

After rearrangement, these equations may be written

-\I4k3/ml

-(k2 + \l4k3 )/m2

Yzk3/m3

These equations may be solved by the' methods described in § 1.9 or, more e .

by the method given at the end of § 8.8. However for a system such as this, in which

Fig. 8.4 Vibrations of three sprung masses

, 1

is no damping, our experience indicates that the masses, once set in motion, may undamped oscillations of a fixed frequency. This suggests seeking solutions in the masses undergo simple harmonic oscillations with frequency w/2rr and 1IIJ1)l1tUQ(~SXI ,X2 and X3, so that the solution is of the form

Y2 =X2 cos(wt+e), Y3 =X3 cos(wt+e), (54)

e is a constant. Substituting equations (54) into equations (53) and cancelling factor cos( wt + €) gives

(XI) ( -(kl + Y4k3)/ml

-c;i X2 = -\I4k3/m2 -(k2 + \4k3)/m2

. X3 ~3/m3

~3/ml)(XI)

'hk3/m2 . X2 .

....,k3/m3 X3

(55)

be definite, suppose that in suitable units the masses and moduli have the

mi = l,m2; l,m3 =4/3,kl = l,k2 = l,k3 =4. Then equations (55) become

C) C -1 i)e:}
_w2 X2 == -1 -2
X3 % 3f2 -3 X3
cW' =i,) (:}C)
or I 2-w2 (56)
--% -% 3 - w X3 ° equations always have the solution X I = 0, X2 = 0, X3 = 0, but this solution no motion and is not of interest. Other solutions are possible only for values of w, namely those for which the determinant of the matrix of i:Geltuc.lents. of the system of homogeneous equations (5·6) has the value zero, so

2-w2 -2
2-w2 -2 =0.
-% -f2 3 _w2 expansion, this gives

(I - W2XW4 - 6w2 + 3) = 0,

has the solutions w2 == I, w2 = 3 + ../6, and w2 = 3 - ../6. These values of w the natural frequencies of the system, and oscillations of the form (54) are

only when w has one of these values. If for example the value w2 = I is equations (56) become

Xl + X2 - 2X3 = 0, Xl + X2 - 2x3 = 0, _32XI - 32X2 + 2x3 = 0.

350

~ 'ar Algebra -

Since the first two of these equations are identical, the equations are not now independent; in fact w has been determined in sucE._ a way that they shall not be independent. The equations give immediately X3 =0 and X] = -X2, so that

XI :X2 :X3 = I: -I: O. This describes a normal mode of vibration in which amplitude ratios X I :X2 :X3 are I: -I: O. Hence for this normal mode

where al and EI are constants. In this normal mode, the mass 11Z3 is at rest, while ml and m2 oscillate with equal amplitude but opposite phase.

Similarly the values w2 = 3 + y6 and w2 = 3 - y6 give two further non-trivial solutions of equations (56) in which the amplitude ratios x I :Xl :X3 are, respectively, y2: y2: - y3 and -n. y2: Y3. Hence for these second and third normal modes

where W2 = y(3 + y6) ~ 2.334 and W3 = y(3 - y6) ~ 0.742. In these second and third normal modes Y I and Y2 have equal amplitudes and are in phase; in the second mode they are out of phase withv, and in the third mode they are in phase withY3. It may be shown that the general solution of the original problem is the superposition of the solutions corresponding to the three normal modes, namely

(::) =a](~:)cOS(t+EI)+a2( ~~) COS(W2t+E2)

Y3 0,. -y3

where a] .az , a 3, E] , E2, E3 are arbitrary constants.

Eigenvalues and Eigenvectors

The problem discussed above reduced to the solution of a system of equations of the form

AX=XX

where A is a given square matrix, X an 'unknown column vector, and X = w2 is a scalar. The problem is to find values of X for which equation (57) has solutions other than the

. Eigenvalues and Eigenvecf~~

351

trivial solution X = 0, and the .corresponding values of X. Only ratios of the elements of X can be found, not their absolute values.

In the problem discussed A was it 3 x 3 matrix. In general A may be a square matrix of any size; for example the vibrations of a mechanical system with' four degrees of

1

freedom would result in A being a 4 x 4 matrix. In studying vibrations of complicated

systems A may well be a large matrix.

The values of A for which equation (57) has non-trivial solutions are called eigenvalues of A (other terms in common use are characteristic roots and latent roots); the corresponding values of X are the eigenvectors of A. To find the eigenvalues and eigenvectors, first write equation (57) in the form

AX= AlX,

and then as

(A - AI)X = o.

The condition for this homogeneous system of equations to have non-trivial solutions is (see §8.8)

det(A - AI) = O. (58)

Equation (58) gives the eigenvalues A. When A is an eigenvalue, at least one of equations (57) is redundant, and the system has a non-trivial solution. The procedure

is best illustrated by an example.

EXAMPLE 1

Find the eigenvalues and eigenvectors of

( I -3 Y2)

A = -3 I -y2 .

y2 -y2 4

I - A -3 y2

The eigenvalue equation (58) is -3 1 - A -y2 ,,;, O.

y2 -y2 4 - A

Oil expanding the determinant, this becomes . A 3 - 6A 2 - 4A + 24 = 0,

or

(A ~6)(A - 2)(A + 2)= 0,

so that the eigenvalues are A = 6,2, -2. To flnd the eigenvector corresponding to (say) X = 2, substitute the value of A and A = 2 in equations (57), which gives

- Xl - 3X2 +y2x3 = 0,

-3XI - X2 -y2x3 = 0,

y2x1 - y2X2 + 2X3 = O.

352

Urear Algebra -

Since A :::: 2 is an eigenvalue, only two of these equations are independent (in fact the third equation is obtained by subtracting the second from the first and dividing by V2) so one equation, say the third, may now be discarded and the first two solved for the ratios x I :X2 :X3. This gives, for the eigenvector corresponding to the eigenvalue A:::: 2

x =(::) = a(~} where a may be any constant,

Since the eigenvectors are indeterminate to within a constant multiplier, any value of Q except a :::: 0 gives an equally good eigenvector. The eigenvectors corresponding to the eigenvalues A :::: 6 and A:::: -2 can be found in a similar way (see exercise 2).

Evidently, to fmd the eigenvalues of an n x n matrix A directly, it is necessary to (a) expand the determinant (58) as an nth degree polynomial in A, and (b) solve the nth degree algebraic equation obtained by setting this polynomial equal to zero. If n is large, as it may well be in practice, both of these processes are formidable and indirect methods have to be used. Some numerical methods of finding eigenvalues and eigenvectors are described in §10.5.

Further Results

Eigenvalues and eigenvectors are of central importance in advanced linear algebra and its applications. Many of their properties are of practical as well as of theoretical interest. The following are some of the more important results.

(a) If A is an n x 11 matrix with eigenvalues Al , A2, ... , An, then det A:::: Al A2 ... An. To prove this, note first that the coefficient of A" in the expansion of det(A - Al) is (-I t. Hence, since AI, A2, ... , An are the roots of det(A - AI):::: 0, we have:

det(A - AI) :::: (AI - A)(A2 - A) ... (An - A),

which is an identity in A. Therefore, setting A :::: 0, det A :::: Al A2 ... An.

An immediate consequence is that if one of the eigenvalues of A is equal to zero, th~n det A :::: 0, and conversely, if det A :::: 0, then at least one eigenvalue is equal to zero.

(b) In general, there is no necessity for the roots of equation (58) to be real numbers even if the elements of A are real. Hence the eigenvalues of A may be complex numbers. However, the following will be proved: if A is a real symmetric matrix, its eigenvalues are all real numbers.

Let Al be an eigenvalue (possibly complex) of A and XI (possibly complex) the corresponding eigenvector, so that

AXI :::: AIXI·

f;·.I:"'~D .. "",I'llD~ and Eigenvectors

Since A is symmetric, A :::: AT, and transposing equation (59) gives X[ A:::: Al X;.

353

(60)

Now let 'XI denote the complex conjugate of Al and XI the complex conjugate of XI. Since A is real, taking the complex conjugate of equation (60) gives

X[A=XIX;, (61)

Now multiply equation (59) on the left by X[ and equation (61) on the right by XI. This gives

X[ AXI :::: AI X[XI, X[ AXI :::: XI X[XI, from which it follows that

- -T -

(AI - AI )XI XI :::: O.

However X[XI :::: ( I xi I + I xi 1+' ... + I x~ I) and is not zero. Hence AI :::: XI, so the imaginary part of AI is zero, and AI is real. As an illustration of this result, note that the eigenvalues of the symmetric matrix in example I are real.

Many of the matrices which arise in practical problems are symmetric, and so the above result, and those which follow below, are of considerable importance and utility.

The following definition is now required. Suppose A is an n x n matrix, and X is an n x 1 column vector. Then XT AX is a 1 x 1 matrix with a single element. If this olement (which is called a quadratic form) is positive for all choices of the vector X

(except X :::: 0), then A is said to be a positive definite matrix. .

EXAMPLE 2 Suppose

Then XT AX:::: (xi - 2xIX2 +2xi). However xi - 2xIX2 + 2xi :::: (XI - X2)2 +xt which is a sum of squares and therefore positive for all values of XI and X2. Hence A is positive definite.

EXAMPLE 3 Suppose

A::::(_~ -~), X:::: (::).

Then XT AX = (xi - 4XIX2 + 2xn. Now xi - 4XIX2 + 2x~ :::: (XI - 2x2)2 - 2xL and so is positive for some values of XI and X2, and negative for other values of XI and X2. Hence in this case A is not positive definite.

(c) A symmetric matrix A is positive definite if and only if all of its eigenvalues are positive. The proof of this result is not difficult, but is omitted.

354

V "r Algebra -

Many l'llysical systems can be reduced to equations of the form AX = ABX, where A is a symmetric matrix and B is a positive definite symmetric matrix. Then

B;-1 AX = AX, and so the equations have non-trivial solutions when A is an eigenvalue of B-1 A. The following extension of result (b) above is then useful.

(d) If A and B are real symmetric n x n matrices, and B is positive definite. then the ' eigenvalues of B -I A are all real numbers.

The proof follows the lines of the proof of result (b), and details are omitted. To illustrate the result we note that equations (55) are more naturally written in the form

'(kI:~k3 k2~~k3 =::)(::)=W2(:I ~2 ~)(::).

\ ·-Wc3 -~3 k3 X3 0 0 m3 X3

This is of the form AX = w2 BX, where B is positive definite because the masses m I, m2 and m , are positive. Hence the eigenvalues w2 are real.

(e) Two column vectors X, Yare said to be orthogonal if XTy = O. (Here 0 represents the 1 x 1 zero matrix (0). It will be shown that if A is symmetric. the eigenvectors associated with two distinct eigenvalues of A are orthogonal. Let AI, A2 be the eigenvalues, XI, X2 the corresponding,eigenvectors. Then

AXI =AIXI, 'AX2 =A2X2• Transposing the first of these gives, since A is symmetric,

X[ A = AI xi. (63)

Multiply the second of equations (62) on the left by xL and equation (63) on the

right by X2, to obtain .

X[ AX2 = A2 X[X2, X[ AX2 = AI X[X2.

Thus (AI - A2 )X[X2 = 0, and since AI =1= A2 it follows that X[X2 = 0, so that XI and X2 are orthogonal. For an illustration of this result, see exercise 2.

Exercises

1. Find the eigenvalues and corresponding eigenvectors of the following matrices.
(a) ( 5 -2) (b) (_~ :)
-2 3
(c) G 0 D (d) C ~)
5 0
0 0 (e) C -2 ] (f) C1 2 I~)
.!. 2 -2
9
3 8 10 -5
(g) ('0.6 sin 6 0)
-s~n 8 cos 8 a .
a 1 2. Find the remaining two eigenvectors of the matrix of example 1. Verify that the eigenvectors of this matrix are mutually orthogonal.

3. The equations of motion of the system of three springs ~nd two masses shown in Fig. 8.5 are

mlYI = -k1YI - k2(Y1 - Y2),

mzY2 = k2(YI - Y2) - k3Yi·

Find the natural frequencies and normal modes of vibration of the system if, in suitable units, ml = 1, m2 = 4, kl = 1, k2 = 4, k3 = 4.

Fig. 8.S Figure for exercise 3

4. Three identical simple pendula are connected by two iden~ical elastic springs as shown in Fig. 8.6. For small oscillations the equations of motion are

mzii-I = -mg81 - kl(fJ I - 82),

mlii 2 = -mg8'2 + kl( 0 I - 0'2 ) - kl( O2 - 03), ml83 = -mgO 3 + kl( O2 - e 3 ).

356

Lir=ar Algebra -

k

k

Fig. 8.6 Figure for exercise 4

Find the natural frequencies and normal modes of vibration, and describe the nature of the vibrations for each normal mode.

5. If the matrix A has eigenvalues AI , A2, ... , An' prove that (a) kA has eigenvalues kAI , kA2, ... , kAn,

(b) A - pI has eigenvalues AI - p , A2 - p , ... , All - p, (c) A 2 has eigenvalues Ai, A~, ... , A~,

(d) A -I has eigenvalues All, Ail, ... , A;I .

PROBLEMS

1. The matrices A lind Bare both non-singular n x n matrices, such that A is symmetric and B is anti-symmetric, State whether the following matrices are

( 1) symmetric, (2) anti-symmetric, or (3) neither symmetric nor anti-symmetric.

(a) AT (b) BT (c) A -I (d) B-1

(e) A+B (f) A-B (g) A+AT (h) B+BT

(i) AB (j) ABT (k) AAT (1) BTB

(m) ABA (n) BT AB (0) AB + BA (p) AB - BA

(q) (A + B)(AT + BT) (r) (A - B)(AT _ BT).

2. Show that the determinant of a matrix which may be partitioned into a triangular array of submatrices as

o

is the product of the determinants of the diagonal submatrices, which are square.

357

Given that the quantities Xi, Yi, zi (i = 1, 2, 3) are related by the equations

XI = YI +3Y2 - 2Y3, YI = -2zl +Z2 + Z3,

X2 = 4YI + 2Y2 + Y3, Y2 = 2z1 - Z2

X3 = YI - 2Y2 + 3Y3, Y3 = Z2 + 2z3,

use matrix methods to express (a) the Xi in terms of the Zi and (b) the zi in terms of the Xi.

4. A particle is moved from the point with coordinates (XI, X2, X3) to the point with coordinates (YI, Y2, Y3). These coordinates are related to the coordinates of a third point (ZI, Z2, Z3) by the two sets of equations

XI = 3z1 - 2Z2 + 7z3, Yl = Y<&v'3z1 + Y<&Z2 - Yzv'3z3,

X2 = -2zi + 4Z2 + Z3, Y2 = --VZZI + 1hv'3z2

Express each of these sets of equations in matrix form and verify that the matrix relating (Y'I, Y2, Y3) to (ZI, Z2, Z3) is orthogonal. Hence obtain the relations for (XI, X2, X3) in terms of (YI ,Y2, Y3).

S. By partitioning, determine the inverse of

Hence obtain the solution X of the algebraic equations represented by AX = H where HT=I(l -22 -4)'-

6. Determine the rank of the matrix

(: _~ -2 -l~)

3 1 -3 - 8

and find the most general solution of the system of equations

X + Y - 2z = 7,

4x - 2y + Z = -11, 3x + Y - 3z = 8.

7. Determine the rank of the matrix

358

Linear Algebra -

Hence state whether the system of equations 4x] + 2X2 = -2,

3xI - X2 = 3V2,

2x] - 2X2 = 5,

4x] + X2 = 0,

has (a) no solutions, (b) a unique solution, or (c) an infinity of solutions.

8. Prove that the equations
2X1 + X2 - 3X3 + 2X4 = 5,
-X1 - 2X2 + 2X3 - X4 = -2,
4X1 - X2 - 5x3 + 4X4 = 8,
X1 - 4X2 + X4 = 4, are not compatible. The above set contains a set of three equations which are compatible. Find this compatible set and obtain all their solutions.

9. Discuss the general solutions of

X1 + 3X2 - X3 - 4X4 = 2, 3x~ - 3X2 + 3X3 - 2X4 = -4,

-2x] -x3+3x4=~'

for all values of ~.

10. Show that one of the eigenvalues of the matrix

has the value 5. Find the remaining two eigenvalues and the eigenvector corresponding to the smallest eigenvalue.

11. Determine the eigenvalues and eigenvectors of the matrix

A=( l~ -: ~).

-IS 9-7

Hence write down the eigenvalues and eigenvectors of the matrices A 3 and A -1.

12. Prove that the eigenvalues of a real symmetric matrix are real. Find the eigenvalues of the matrix

and an eigenvector corresponding to the smallest eigenvalue.

159

The equations dVI

- =-2V1 -3V2 - V3, dt

dV2

-=V

dt I,

govern the behaviour of an electric circuit.i'l'aking V I = X I eht, V2 = X2 eht,

V 3 = X3 eht , determine the values of A and the corresponding values of x I : X2 : x 3 which satisfy these equations.

The oscillations-of a mechanical system are governed by the equations Mlh =-AIY1 +A2(Y2 -YI),

M2Y2 = - A2(Y2 - yd + A3(Y3 - Y2), M3Y3 = - A3(Y3 - Y2) - ~Y3'

Assuming the displacements Yi may be expressed in the form Yi = Xi cos cot (i = I, 2, 3), show that the equations may be written in the matrix form (A - w2 I)X = O. If M I = I, M2 = 2, M 3 = 3, Al = I, A2 = I, A3 = 2, ~ = ~, determine the natural frequencies and amplitude ratios of the system.

15. Oscillations of a framework are known to be governed by the system of equations

XI = -axl + X2,

X2 = XI -2X2 + X3,

i3= X2-2X3,

in·which the coefficient ex is not given. However, it is observed that the system has a normal mode of the form X = D cos r../2. Deduce the value of ex, find the frequericies of the remaining two normal modes of the system and find the eigenvector· corresponding to the lowest frequency.

By using the method of Lagrange's multipliers, show that the greatest value of AX, subject to XTX =!(R2),lwhere X is a column vector and A is a real svmmetnc matrix, occurs when X is an eigenvector of A.

17. If X and A are as in problem 16, and B is a symmetric positive definite rna trix, show that the stationary values of XT AX, subject to XTBX = (c2), occur when X is a

solution of the eigenvalue problem AX = ABX. '

BIBLIOGRAPHY

Bell, W. W., Matrices for Scientists and Engineers, Van Nostrand Reinhold, Wokingham (1975).

360

1 'near Algebra - Cohn, P. M., Linear Equations, Routledge and Kegan Paui,' London (1958). Edelen, D. G. B. and A. D. Kydoniefs, A n Introduction to Linear Algebra for Science and Engineering, Elsevier, New York (1972).

Mirsky, L., Anl.ntroductioll to Linear Algebra, Clarendon Press, Oxford (1955).'

Noble, B., A pplied Linear Algebra, Prentice-Hall, New Jersey (1969). .

[2] [3]

[4] .15]

ntroduction to Numerical nalysis

NUMERICAL APPROXIMA nON

the preceding chapters, solutions of equations representing a large variety of situations have been obtained in closed form or as converging series of functions. A mathematician would be satisfied with such solutions; an 'rne1l1el~r would not, because ultimately he requires a numerical value or values. For ;ClUUUI"''', the solutions of some beam vibration problems are shown in § 7.2 to be i'~II'U"'''J expressed in terms of the roots of the equation

engineer would need the numbers corresponding to those roots. The deflection of a uniformly loaded beam is in the form of a quartic; to find the positions of maximum deflection requires the solution of the cubic equation corresponding to

ts of zero slope. Again the numerical values of the roots are required. As a final pie, the reader should recall, that in the basic method of solution of ordinary

.' differential equations with constant coefficients (see § 1.5 and § 1.7) it is necessary to for the roots of the auxiliary equation. This equation is a polynomial, and in

,. practice it must be solved numerically.

Another important consideration is that often physical problems cannot be

, represented adequately by equations which can be solved analytically. In these cases engineer must resort to numerical methods of solution.

Numerical analysis is the study of the numerical solution of problems. Apart from developing methods of solution, the theory is also concerned with the inaccuracy which is always introduced in any numerical computation. Without an estimate of this error, a numerical solution is worth little more than no solution at all.

In this chapter, we introduce the elements of numerical analysis, together with its elementary applications in evaluating formulae and finding the roots of single algebraic and transcendental equations. More advanced numerical analysis is treated in the subsequent chapters on the solution of systems of linear equations (Chapter 10), numerical interpolation, integration and differentiation (Chapter II) and the

numerical solution of differential equations (Volume 2, Chapter 7). For

comprehensive treatments of the subject, the reader is referred to Froberg [I] , Noble

[2], [3] and Williams [4]. '

362

Introduct=n to Numerical

Numbers, Decimals and Significant Figures

In mathematics, numbers are assumed to have an exact value, so that a mathematician knows precisely what is signified by the values of n, .n, sin 1 and any other irrational numbers. In practice, these values cannot be written exactly since the representation would require an infinite number of digits. It is customary to represent numbers in decimal form.Ithough digital computers use binary form). Thus the number 1.4142 has four decimals or decimal places and represents the value of v'2 accurately to four decimals, being correct to within five units in the fifth decimal place. It has five significant figures, which are defined as the number of correct digits starting with the first non-zero digit at the left.

Errors and Mistakes

An error € is introduced whenever an exact number N is replaced by an approximation n; it is defined by

€=n-N.

The error introduced by replacing v'2 by 1.4142 is of a particular type called round-off error. It is produced .by rejecting ·all digits after a selected decimal place and arises as an inevitable consequence of the practical necessity of having to work with a finite number of significant figures. The standard procedure for rounding off is to leave the last digit unchanged if the discarded portion is less than half the last unit retained, increase the last digit by one if the discarded portion is more than half the last unit, and round off to the nearest even digit if the discarded portion is exactly half the last unit retained. Thus, on rounding off to four significant figures, we have

1.41421 ~ 1.414, 0.0618237 ~ 0.06182, 0.31625 ~ 0.3162,

41.755 ~ 41.76, 1.75948 ~ 1.759.

It is worth noting that successively reducing the number of significant figures can lead to a wrong answer, as can be seen from the values 1.75948 ~ 1.7595 ~ 1.760.

Occasionally it is useful to retain some of the discarded information by placing the first discarded digit (rounded off) as a suffix after the number. If so, care must be taken to interpret the value correctly, since the notation can be confusing:

1.4142 ~ 1.4142, 3.1416 ~ 3;1426•

Another type of error, called truncation error, is introduced whenever the computational method depends on 'cutting off an infinite series representation. This truncation is implicit in many (if not most) numerical methods of solution (see Volume 2, Chapter 7, or Froberg [1] ).

Either or both of these errors may enter each stage of a computation, and can accumulate as the numerical procedure continues. This accumulated error is often difficult to control. Even when the magnitude of the round-off and truncation errors are controlled at each stage of the computation, the accumulated error can still grow

Numerical Approximatio»

163

and may swamp the true solution. Its presence can often be detected by comparing the two answers yielded when the computation is carried out by using two different proceduresor by repeating the. same procedure but with different step-lengths (see

Volume 2, Chapter 7, or Noble [3]). .

Errors should not be confused with mistakes (or blunders). These arise from human fallibility or from machine defects. The most common examples are the transposition of digits when writing down numbers (for example, writing 54235 in place of 54325), misreading repeated digits (5.42235 instead of 5.42335), faulty programming for automatic computers, and wrong experimental data. Great care must be taken to

guard against numerical mistakes in calculations carried out on a hand-machine, though the most effective numerical methods have checks specifically built in as safeguards. Numerical mistakes occur very rarely on automatic computers.

Finally, it should be noted that there is such a thing as being 'too accurate'. In hand-machine work it is unnecessary and even counter-productive to include more figures than are Significant. This only gives a greater opportunity f~r making mistakes. Similarly, it is useless and misleading to give an answer with more nominal accuracy than is justified by the accuracy of the given initial data.

Absolute and Relative Errors

It is convenient to distinguish between the error € (= n - N), its absolute magnitude I e I and its magnitude compared with the exact number N. The positive value I €I is called the absolute error, and the ratio I €IN I is the relative error. Since only n, and

not N, is known, the relative error is more usefully given by I €In I provided € is known . to be small compared with n.

Let NJ and N2 be two numbers which are taken as positive, without any loss of generality. If nJ and n2 denote the approximations to NJ and N2 in any calculations,

then the absolute errors for addition, subtraction, multiplication and division satisfy

I (nJ + n2) - (NJ + N2) I = I €J + €2 1< eJ + e2

l(nJ -n2)-(NJ -N2)1=I€J -€21<eJ +e2

I nJn2 - NJN2 1:>< I €Jn2 + €2nJ I <eJn2 +e2nJ

I nJ _ NJ 1:>< 1 ~n2 ~ €2nJ 1< (nJ) (~+ e2)

n2 N2 n2 n2 nJ n2

where the last two relations are based on the assumption that the errors are small compared with NJ and N 2. Here eJ and e2 denote upper bounds on I e J I and I €2 I, so that I €J I < eJ , I €2 I < e2; they are sometimes called maximum errors.

The relative errors in

NJ +N2, NJ -N2• NJN2• NJIN2

are deduced in a similar manner; the approximate upper bounds are, respectively,

364

Introduct+n to Numerical

EXAMPLE

Compute the maximum possible relative errors in N 1 + N 2 and N 1 - N 2 if the numbers N 1 and N 2 have been rounded off to n 1 = 49.84 and 112 = 49.82 respectively.

Rounding off implies that both numbers are accurate to within ± 0.005. Thus

I I: 1 I, I 1:2 I ~ 0.005 and the maximum relative errors in N 1 + N 2 and N 1 - N 2 are respectively

0.005 +0.005 = 0.0001

49.84 + 49.82 '

0.005 + 0.005 = 0.5. 49.84 - 49.82

The second part of this example illustrates the fact that the subtraction of almost .: equal numbers is a numerically bad procedure, leading to a loss of significant figures. also shows how division by a small number magnifies any error in a computation. These two features are particularly important when the calculating machine has a display limited to very few significant figures, such as with a pocket calculator.

Order of Error Normally the error in any numerical solution cannot be determined exactly (unless the exact solution can be found by another method, in which case the numerical solution is unnecessary). We can specify exact bounds on round off errors but not for truncation errors. However, a measure of the magnitude of a truncation error is provided by the (neglected) next term of the series. In fact, it is this order of error which is the useful quantity. If the error depends on the value of some small parameter h, then we say that the error. is 'of order h', written O(h), if the ratio elh tends to a finite limit as h approaches zero:

I: = O(h) < > lim(l:/h) = c

h ..... O

where c is some finite constant. If c = 0, we may write I: = o(h), but this notation is not 'important in numerical analysis. More generally,

I: = O(hP) <=> lim (l:/hP) = c.

h ..... O

9.2 EVALUATION OF FORMULAE

All problems involving numerical computation eventually reduce to the evaluation of formulae. At this stage, considerable savings of time and reduction of error can be made by suitably arranging the numerical procedure. The choice of arrangement is largely a matter of experience, but there are a few basic rules which are obvious once they are pointed out.

The first rule is always to arrange formulae so that as few operations as possible are involved. This saves time, reduces accumulation of error and, on a hand machine, minimizes the chance of mistakes. Another rule with a similar advantage for hand

'ronlputamcm is to' choose the procedure which involves as little intermediate writing as r:IIOISSllJlC. Furthermore, it should always be borne in mind that division takes longer multiplication which in turn is far more time-consuming than addition. On a.

III""'-""VV~ computer such as the ICL 1906, the respective ratios are roughly 12:6: L ,As an illustration of the application of these rules, we consider the following example.

P(x) =anxn +an_lXn-1 + ... +alx +ao

as to give the most efficient procedure for its evaluation.

Inspection shows that in its given form the evaluation involves n additions and

+ 1) multiplications. Also, if no storage mechanism is available, the value of each must be written down. By performing the calculation in reverse order and

a record of x, x2 , ••• , the number of multiplications is reduced to 2n - 1. standard rearrangement is even more efficient, involving n additions but only + 1) multiplications and no storage facility:

p(x)=« ... «an x +an_dx +an_2)x + ... +a2)x +adx +ao·"

proced ure is to first calculate the innermost bracket (anx + an - 1 ); the result is multiplied by the value of x and added to an _ 2, this result again being multiplied'

x and then added to an-3; and so on.

Division is often avoided if possible. This is not only for reasons of time-saving, but to avoid errors which may be introduced through round off at a fixed decimal when dividing by very large numbers. The dangers associated with division by numbers have already been noted in the previous section. Therefore one would

(a x b x c x ... )/(u x v x w x ... ) not (a/u) x (b/v) x (c/w) x .... The reader should always treat the above rules as guidelines and not as being le. Sometimes two or more of the rules will be in conflict. This occurs

when the advantages of time-saving are weighed against loss of accuracy; rirc:umlstanc~~s must then determine what should be done. In the following example,

MPLE 2

"Evaluate the roots of x 2 - 48x + 2 = 0 using five significant figures.

The two roots' are given by

x = 24 ± V574 = 24 ± 23.958

SO that one root Xl = 47.958 is accurate to five significant figures whereas the other

.' X2 = 0.042 has only two significant figures. However, if the subtraction giving x , replaced by a division using the property that x.x, = 2, one obtains

= 2/47.958 = 0.041703, and retains five significant figure accuracy.

366

Introductior '., Numerical Analysis

Exercises

1. The radii of two circular cylinders are measured to be 12.783 ± 0.002 and 0.5 31 ± 0.002. Calculate the area of cross-section of each cylinder, first using

7T = 3.14 ± 0.005 and then using 7T = 3.142 ± 0.0005. Give the absolute and relative errors in each case. Is there any gain in taking 7T = 3.1416 ± 0.00005?

2. If A = 7.13291, B = 7.13709, x = 0.142 and y = 0.315, calculate the value of C= (Ay - Bx)/(y - x)

both directly and by using the alternative form

B-A C=A -x --. y-x

Estimate the error in each case.

3. Evaluate the roots of the quadratic x 2 - 15x + 1 = 0 (a) directly, using the standard formula, and (b) using the formula for the larger root and the result that the product of the roots is unity to determine the smaller root. Estimate the error in each case and check your answers by adding the roots.

9.3 FLOW DIAGRAMS OR CHARTS

When an automatic computer is used to determine a numerical solution, it is obvious that the machine must be instructed on each individual operation it must make, and in an unambiguous manner.' This series of instructions is termed a computer program and it must be written in a 'language' which can be interpreted by the machine.

Programming itself is not considered to be a part of the subject of numerical analysis. The logical ordering of the basic steps is a part, as it incorporates error estimates and safeguards against mistakes. In this context, the term 'step' means a set of related instructions, such as those required to evaluate a formula.

This sequential arrangement of steps is usually displayed in the form of a flow diagram or flow chart, and is relevant to both hand and automatic computation. Each step is represented, as in Fig. 9.1, by a box containing one of two types of commanddirect and alternative (or 'if') commands. Each box is linked to another by means of an arrow indicating the direction of the logical order. Direct commands include arithmetical operations, 'write' instructions and input of data; only one arrow leaves each such box, which is usually drawn in a rectangular shape. Alternative or 'if commands are always associated with a choice of direction in which to proceed; thus more than one arrow leaves each such box, which is often drawn as a diamond or some other non-rectangular shape. An alternative command usually occurs as a 'test' statement involving an inequality. There are three such statements in the flow chart shown in Fig. 9.1.

Fig, 9.1 Flow chart illustrating various types of statements

Row Diagrams or Chartr

Print m, n,

'complex for all (e)

greater n'

EXA'MPLE

Write down a flow diagram for finding and printing all the real numbers x given by

x=-m +(m2 _n)1I2 '

for ~ll pairs (m, n) where m changes in steps of ~ from 0 to M and iI is an integer ~rymg from -N to N. M and N represent positive integers which are to be specified as mput data.

A suitable flow diagram is shown in Fig. 9.1' the (a) (b) (c) '" t th

r 11 . , '" ••• reter 0 e

10 ~Wl~g comments. (a) is a read statement, allowing the user a freedom of choice in

. speclfymg~ and N. At (b) a decision is made to select the pairs (m, n) in the sequence (0, -N), (0, -N + 1), ... , (01 N), (~, -N), (~, -,N + 1), ... rather than the order

368

Introductir=to Numerical Analysis

(0, -N), (~, -N), ,(M, -N), _ ... Sometimes there is a gain ~in choosing a .

particular order, but the order is not important in this example. The step shown In

(c) is not strictly necessary in a flow chart, but demonstrates how computer time can be saved. If (c) were excluded and the next step changed to read D = m 2 - n, the machine would have to compute the same m2 for 2N + I times. The first 'if statement (at (d ) ensures that only real numbers are computed. If, at any stage, D is negative, implying complex x, the machine is told to bypass the next three steps by jumping to the step (g); the 'print' statement (e) on.the way is optional. Note that the direction of the flow is unchanged. This is in contrast with the consequences of the other two 'if statements at (f) and (g), which reverse the flow and are, therefore, drawn on the opposite side of the main flow path. These are sometimes termed '[oops' or 'cycles', and occur whenever a series of steps is repeated. It is good practice never to have loops intersecting one another, since normally this indicates a fault in the logic and at best an inefficient program. As an example of this, the reader should consider the effect of interchanging steps (f) and (g).

Most computers (including hand machines) now have a built-in square-root facility, so that computing yD in a procedure is, in effect, an arithmetical operation. If the facility is not available, then the instruction (h) implies that a sub-program (within the main program) must be used in order to evaluate yD. Such sub-programs are called subroutines.

Exercises

I. Write a flow diagram for evaluating the polynomial P(x) = ao + a 1 x + ... + anxn for arbitrary input of values of aD' al , ... ,an and integer n, and for values of x increasing from x = 0 to x = I in steps of 0.1.

2. Construct a flow chart for evaluating n! for n = 0, 1,2, ... , 10.

3. Write down a flow chart for evaluating the continued fraction

N9 + I N10

to a specified accuracy, and where the val ues No, N 1 , ••• , N 1 0 'are to be inserted as input data.

9.4 SOLUTION OF SINGLE ALGEBRAIC AND TRANSCENDENTAL EQUATIONS

There are two types of method for solving problems numerically =direct and iterative. Direct methods are 'one-off methods. rlesicned to uive the answer direct lv Th»

"69

Solution of Single-Algebr:": and Transcendental Equations

./

procedures are, therefore, usually lengthier ana more complicated than iterative methods, which involve repeating the same simple procedure many times until the answer 'settles down'. However, the greater number of calculations required by iterative methods makes these less attractive for use on hand machines.

The simplest example of a direct method of solution is the use of the formula

-b + (b2 - 4ac)1I2

x = (5)

2a

for obtaining one of the roots of the quadratic equation ax2 + bx + c = O. The root is easily computed by. using formula (5), provided square root tables are available to the required accuracy; if not, the direct solution is not straightforward. Checks must be incorporated in any direct solution in order to guard against mistakes; in this example, a check is provided by directly substituting the answer into the quadratic.

Although direct methods are important for solving systems of linear equations (see Chapter 10), they are of very little value in solving single equations. Formulae exist for solving cubic and quartic equations, but these formulae are so complicated that they are rarely used. It is not possible to produce formulae for the solution of higher order polynomial equations or for transcendental equations (which involve trigonometric or

- exponential functions). Hence the general equation

f(x) = 0

(6)

. must be solved numerically for its roots x = X using iterative methods.

A function g(x) is chosen in such a way that the sequence of values x 0, X 1> X 2, •.• generated by using an iteration formula

Xn+l=g(Xn), n=0,1,2, ....

converges to the required root X of f(x) = 0: lim xn ~ X.

n-+oo

(7)

(8)

Hence g(x) must be a function such that

X=g(X).

(9)

Ther.e are an infinite number of possible choices o£g(x) such that both equations (6) and (9) are satisfied when x = X. For example, this occurs for any choice of the coefficien ts ai' a 2, •• -:- , am such that

(10)

Different forms of g(x) lead to different iterative methods. The choice of g(x) is governed by the form of the function f(x) and by whether the number of iterations is important (as it would be if hand computation were necessary). In particular,g(x) must be chosen such that the procedure converges.

In practice, the first approximation Xo is determined graphically or by computing y = f(x) for various x and finding when the value of y changes sign. The numerical procedure described by equations (7) and (8) is stopped when two successive values

,.l!t:t: __ L __ 1 .d~

370

introduction to Numerical AnalysiJ

have taken place. The latter alternative is to safeguard against divergence of the sequence, or against prohibitively slow convergence.

Convergence of Xn + 1 = g(xn)

If Xn is regarded as an approximation to X, so that En =Xn - X

is the error in the approximation, then the basic iterative procedure (7) converges if g(x) is such that

I En+dEn I < I

for sufficiently large n. From equations (7) and (II) it follows that

X + En+ 1 = g(X + En)

= g(X) + Eng' (~n)

where ~n is a value in the range (xn, X). Here we have used the first mean value theorem (see §4.2) for a continuous differentiable function:

g(xn) - g(X) = (xn - X)g' (~n), ~n in (xn, X)

where g' (~n) represents the value of the derivative' of g(x) at x = ~n' Using equations (9) and (l3) then shows that

En+l/En =g'(~n)'

Hence the procedure converges provided g(x) is a continuous differentiable function such that

Ig'(~n)I<1.

This result is illustrated graphically in Fig. 9.2, where y = x and y = g(x) have been.

y

y

X x, x, Xo X

(a)

(b)

(c)

Fig .. 9.2 Geometrical representation of xn+ 1 = g(xn) for variousg(x); (a) convergent procedure, (b) divergence, (c) slow convergence

tion of Single Algebrai« and Transcendental Equations

171

drawn for several different functions g(x). The iterative procedure (7) is given by following the arrowed paths. Starting from an arbitrary value Xo of x on the curve

y = g(x), we move horizontally to the line y = x. This new value of x is equal to g(xo) and is therefore x 1 by the 'defmition (7). Moving vertically to the curve y = g(x) then gives Yl = g(Xl) so that the subsequent horizontal move to the line y == x will givex2. Repetition of this procedure gives x 3, X4, ••• , and shows whether or not the method converges (compare Figs. 9.2(a) and 9.2{b ».

Since X, and hence ~n, are not known a priori, it is convenient to express the condition for convergence in terms of known values such as xn. Expanding g' (~n) in a , Taylor series about Xn gives

g'(~n) = g'(xn) + (~n - xn~"(xn) + ... so that, from equation (14),

En+i/En = g'(xn) + O(En)·

(16)

" Hence an alternative condition for convergence is that g(x) should be such that at any of the computation

{I 7)

This is a sufficient, but not a necessary, condition for convergence, as can be seen from Fig. 9.2(c) where I g'(xd I> 1.

The Newton-Raphson Method

ntis very important procedure for solvingftx) = 0 is described by f(xn)

Xn+l =xn - ['(xn) ,

the special choice of g(x) now being

{I 8)

f(x) g(x) = x - ['(x)"

The choice is suggested by the following analysis, in which all terms involving " f;, ~~, ... are neglected in the Taylor series expansion:

0= f(X) = f(xn - en) "'" f(xn) - E,,(Xn)·

Hence an approximate correction En isf(xn)/['(xn), leading to the formula (18). From equation (19) it is straightforward to show that in this case

(19)

'(X) = f(X)(' (X)

g If'(X)} 2

Therefore, since f(X) = 0,

g'(xn) = g'(X + En) = g'(X) + Eng"(X) + ... = O(En).

'This is a remarkable result, showing from equation (16) that not only is I En+lliess

372

Introductior 4') Numerical

(al

(b)

Fig. 9.3 Newton-Raphson method

y

tc)

than I €n I but is, in fact, of order €~, indicating very rapid convergence. The numerical implication of this is that the number of correct figures is effectively doubled by each iteration.

A graphical illustration of the method is shown in Fig. 9.3(a). The slope of the tangent at Po on the curve y = f(x) isf'(xo). The slope is given also by f(xo)/(xo - XI) so that X 1 is the same value as that given by equation (18) with n = O. The figure shows how the procedure moves the values X 0, X I, ... progressively closer to the exact solution at x = X. It should be noted, however, that convergence to a required root X is not certain even if x 0 appears to be sufficiently close to it. Equation (18) indicates that difficulties must be expected whenever f'(x) is zero near x = X, and particularly when the equation f(x) = 0 has two nearly equal roots in this region. A graphical illustration of each case is given in Figs. 9.3 (b), (c). The first approximation x 0 is much closer to the root XI thgn to X2 in Fig. 9.3(c), but the procedure still converges to X2• In each case the root can be located using the fixed secant method described later in this section.

EXAMPLE 2

Compute the single real root of the cubic equation

3x3 + x-I = 0

giving the answer to three decimal accuracy.

Yes

Fig. 9.4 Flow chart for the Newton-Raphson procedure

With f(x) = 3x3 + x-I, we note that f(x) passes through a zero as x increases from y' 0 to 1; hence the iteration is started from Xo = O. Since (x) = 9x2 + 1, the Newton'RllDhl~on method gives

3x~ +xn - 1 xn+l =x.; - 9x~ + 1

. Starti~g from x 0 = 0 and following the numerical procedure described in. example 1 above, we obtain XI = 1 and the sequence

n

o o

2 0.7

3 0.60

EXAMPLE 1

Construct a flow diagram for the Newton-Raphson procedure.

The diagram is shown in Fig. 9.4, and illustrates how simply a flow diagram can be constructed for an iterative procedure. In the form shown, only the required number or the word 'divergent' is printed. If all the intermediate values of Xn are required, then the print statement should be moved into the loop. Note that by specifying 0 and N the user is postulating his practical definitions of convergence and divergence, and these values must be decided a priori for automatic computation. Note also that the instruction to calculate f(xn) and f'(xn) would normally require a subroutine if the function f(x) were at all complicated.

4 0.54

5 6

0.537 0.5366

showing that the root is at X = 0.537 to 3 decimal places. This may be verified by

substitution into the original equation. .

EXAMPLE 3

Obtain to four decimal accuracy the smallest non-zero root of cos x 'cosh x = 1.

This is an equation governing the solution of many vibration problems (see §.7.2).

374

Introduction to Numerical

y

Fig. 9.5 Locating approximate roots of cos x cosh x = 1

An approximate value for the root is given by considering the intersection of the y 7 cos x and y = sech x. Figure 9.5 suggests that Xo = 4.7 is a suitable choice. Then; withf(x) = cos x cosh x-I, we obtain the iteration formula

cos Xn cosh Xn - 1

xn+ I = xn - ----.:..:...._--.:..:...._----

cos X n sinh xn - sin Xn cosh x n

leading to XI = 4.731, X2 :: 4.7300 = X3, so that the solution is X = 4.7300.

Figure 9.5, in fact, shows that the roots are very nearly the roots of cos x = 0 for I x I > 1(/2, i.e. x ~ ± 31(/2, ± '51(/2, .... If these values are used as first approximations, then equation (20) immediately gives the next approximation to be

x I = ± ( (2r + 1)~ - (-I r sech (2r + 1)~ ] , r = 1, 2, . .. .

of Single Algebraic and Transcendental Equations

formula is valid for positive and negative values of N. Thus an iterative method

determining I/A is given by equation (22) with N = -1: .

Xn+1 = (2 -Axn)xn'

Xn+1 = ;2xn(3 -Ax~).

is a remarkable formula in that it does not use division (apart from *); it can also

used to compute Ai12 as AA-1I2• •

ron1p/l~ Roots

Newton-Raphson formula can be used to obtain the roots of any equation. This is irrel,pe,ctl\'e of whether those roots are real or complex and irrespective of any complex toelIlcll;;lIL~in the equation. The only restriction is thata complex starting value x,

be used if a complex root of an algebraic equation is sought: However, if the r:qU<~LI""" is algebraic and has real coefficients, Bairstow's method should be used. This use of the fact that all complex roots of such equations occur in conjugate pairs

lb. The reader is refe~red to Froberg [I] for details of the method which,like the

:Ne'ivto:n·· 'Kapi method, converges rapidly with En + 1 = O(E~). .

II''',nlll,h the Newton-Raphson method is a very good numerical method, the reader soon find when using a desk calculator that the need to compute the derivative

) and to perform the division by (xn) at each iteration is a definite drawback, in cases being extremely time-consuming. The same criticism also applies whenever automatic computer is used, particularly with complicated functions requiring a .

EXAMPLE 4

By consideringf(x) = x' - A, derive an iterative formula for computing the square root A 1/2 of any positive number A.

A 112 is the root of f(x) == x2 - A = O. Applying the Newton-Raphson formula (18) ,

to this choice of f(x) gives ,

This drawback is overcome in the FIXed tangent method which replaces the variable, !'(xn) of the Newton-Raphson method by the fixed initial value!'(xo). The of f(xn)/!'(xo) is obtained by multiplying f(xn) by the constant I/(xo). The proced ure is then

Xn+1 =xn +: X;_;:A=*(xn +~).

This formula is almost universally used for automatic computation of square roots.

Nth Roots and Reciprocals

Formula (21) is' a special case of the general formula derived from f(x) = xN - A to give A '!".

xn+1 = ~(N - I)xn +AX~-N] .

,Un,"p.Vler application of this method requires much greater care than does the . :J'IC'WLlIU-I''''If'l1,''Vl1' method, particularly when choosing the initial value xo, as is from the following example.

AMPLE 5

example 2 using the fixed tangent method, and choosing (a)xo = I, and =0.

(a)xo = I implies!'(xo) = 10, so thatxn+l =x, - %of(xn):

n

o

2

3

6

4

5

7

8

9

10 0.539.

xn 0.7 0.63 0.59 0.57 0.56 0.55 0.545 0.542 0.540

375

I

I

! !

I

j

I i

I

I

!

376

Introductior ~') Numericai

The values converge to 0.537, albeit progressively more slowly. (b) Xo = 0 implies ['(xo) = 1, so that Xn+1 = Xn - f(xn}

n

o o

3 25

4 -46874;

2 -2

the process is now obviously divergent.

The contrasting behaviours resulting from the different choices of Xo are explained by considering the condition (17) for convergence. In the fixed tangent method, g(x} and g' (x) take the forms

g(x}=x +cf(x}, g'(x} = 1 +c['(x}

where c = -l/['(xo}. Condition (17) then shows that convergence is obtained if - 2 < cf" (xn) < 0

which in 'this case implies the condition

o < ['(xn )!f'(xo} < 2.

In the solution (a) of example 5, ['(xn )/['(xo} = (9x~ +.1 }/10 ';;;'1, satisfying condition (25); on the other hand, for (b) the ratio is 9x~ + 1 with a value of 10 when n = 1 so that condition (25) is violated.

Methods not Requiring a Derivative

The simple analysis leading to condition (24) holds for any choice of c in equation (23) and need not depend on any derivative. Thus an['(xo} leading to an 'awkward' value for c in a hand-computation should be replaced by a more convenient value. Alternatively, by choosing x., and XI on opposite sides of the root X, a more suitable c could be given by c = - (XI - xo)/{[(xl} - f(xo)}, so that

This is the fixed secant method, and is graphically illustrated in Fig. 9.6(a}. The relevant criterion for convergence is still given by condition (24) which in this case implies that

o < X I - X 0 ['( )

f(xd - [(xo) Xn < 2.

A similar method, using the most 'up to date' chord (that joining the points corresponding to Xn _ 1 and xn), is given by

Xn - Xn_1

= xn -1[(Xn} - xn[(xn - I} [(xn) - [(xn_ I)

y

y = fIx}

(a)

(b)

Methods not requiring a derivative; (a) the fixed secant method, (b) the variable secant

This method is called the variable secant method or 'the method of regula falsi, and is shown gra phically in Fig. 9 .6(b }.

EXAMPLE·6

Repeat example 2 using the fixed and variable secant methods.

Since the sign of [(x) = 3x 3 + x-I changes between x = 0 and x = 1, we choose Xo = 0 and XI = 1. Thus j'(x.,') = -1 ,[(XI) = 3 and the fixed secant method gives

xn+1 =xn - ~(3x! +xn - I).
n 0 2 3 4 5 6 7 8
Xn 0 0.25 0.43 0.51 0.53 0.536 0.5365 0.5366.
The variable secant method gives
xn_I(3x! +xn -l}-xn(3x!_1 +xn_1 -I}
x,,+1 - 3x! +xn -3x!_1 -Xn-I
n 0 2 3 4 5 6 7 8
'Xn 0 1 0.25 0.39 0.61 0.52 0.535 0.537 0.5366. The relatively slow rate of convergence of the fixed tangent and secant methods compared with the Newton-Raphson method has been explained by the order of the dependence of €n + 1 on €n. If, in an iterative method,

€n+1 =O(€~},

the method is said to be of pth order and the power p would give the order of convergence. Thus the fixed tangent method is a first-order method and the Newton-Raphson a second-order method.

Introduction to Numerical

The variable secant method can be shown to be in between these methods, being of order ~(1 + ..;'5) ~ ] .62. Its convergence is better than the first-order methods even though no derivative is required. It should be noted, however, that for some functions (polynomials, for example) evaluation of (xll- xll_ I )j{xn )/[t{xll) - j{xll- I )1 takes much longer than the computation of j{xn)/(xn)·

Methods with third-order (or higher order) convergence exist, but are rarely used in practice because usually the additional complexity of computation is not worth the reduction of number of steps.

Acceleration of Convergence

Since the simple first-order methods often result in comparatively slow convergence, the following method of accelerating their convergence is important, particularly for hand-computation. It is known as Aitken's o2-process.

For any first-order procedure, equation (I 4) shows that en+ .ten = g'(~n) =g'(X) + O(en)

and Similarly

en/en-I =g'(X) +O(en-d.

Hence, neglect of terms of O(e) gives

en + I /en = en /en- I'

Substituting Xn - X for en' etc. and solving for X gives

(xn - xn _°1)2

X""'Xn - .

Xn - 2xn_1 +Xn-2

The value given by formula (27) is then used as the starting value for a new sequence of iterations.

Applying the method to the values in example 5(a) above, when convergence becomes obviously very slow at n = 8, would mean that a new sequence at n = 9 should start at

(-0.003)2

0.542 - 0.002 = 0.5375.

The new sequence then gives Xn as 0.5375, 0.5372, 0.5370 (working with four decimal places since accuracy is required to three) and a further use of the approximation (27) gives the required answer as 0.5366.

Exercises

I. Construct a flow diagram for the variable secant method.

2. Use (a) the fixed tangent method, (b) thevariable secant method and (c) the

(27)

,379

Newton-Raphson method, LO find the real root near x = I of x3+x-3=0.

Make the answer accurate to four decimal places.

3. Use the fixed secant method, with accelerated convergence if necessary, to determine the real roots, accurate to three decimal places, of

f(x) = 4.l7x3 - 20.03x2 - X + 5.21.

4. By applying the Newton-Raphson method with the function f(x) = I - Ax- N, deduce an iterative formula for obtaining A I IN. Show that it is not the same as formula (22). Use each formula to determine y3 accurate to five decimal places.

5. Use (a) the Newton-Raphson method, and (b) an iterative method not requiring a derivative, to solve x = exp( -:-x2 ) to four significant figures.

PROBLEMS

l. An infinite continued fraction f is given by lim fn where

n .... oo

al

fn =bo +--bl +a2

b2 +a3

b3 +.

bn-I + an bn

and a I , a2 , ... , bo, b I , . ; . , bn are specified numbers. The value of fn is given by [n = An/Bn where An and Bn are determined from the recurrence relations

An = bnAn_1 + anAn-2, Bn = bnBn-1 +anBn-2

with.A -I = I,Ao = bo, B-1 = 0, Bo = 1.

, Write a flow chart for determining j to a specified accuracy. Hence compute the

, -I

value of tan 0.35 to four decimal places, given that

_I X

tan x =--,-

I +x2

-----".

3 + (2X)2

5 + (3xi

7 + ...

2. Graphically examine the convergence of each of the iterative procedures

2 .

(a) xn + I = ¥.!(xn - 1) and (b) xn + I = 4 + I /xn. Hence compute the two roots of the

380

Introduction to Numerical A

quadra .. -: equation

x2 - 4x - 1 = 0

giving your answer to three decimal accuracy.

3. Use Taylor's theorem to obtain an approximation for [(x) = eX sin x near x = 0 in the form of a cubic in x and use the remainder term to determine the range of values of x over which the approximation is valid if the error is not to exceed ± 0.0005. Hence obtain approximate values of [(0.2) andj'( -0.3), and compare your answers with those obtained using tables.

4. Use two different iterative methods, one involving the use of a derivative and the other not, to determine to five significant figures the smallest non-zero positive root of x - 2 sin 2x = o.

5. Let (X, y) denote the exact solution of two simultaneous equations [(x, y) = 0, g(x, y) = O. By expanding fiX + h , Y + k) and g(X + h, Y + k) about (X, y), and neglecting terms involving h2 , hk ; k2 and higher powers of hand k, deduce Newton's iterative method of solution:

where

a[ ag

-g-+[-

ax ax

Kix , y)

a[ ag a[ ag· ax ay ay ax

Hence solve the simultaneous equations

x- sin(y/x) = 0, y=cosy+sinx, giving your answer to three decimal accuracy.

6. By sketching the two curves 3x2 + y2 = 9, ~ + 2 sin x = 2, show that they have two points of intersection for one of which x is positive and for the other x is negative. Use iteration schemes of the form

Xn+1 =F(xn,Yn), Yn+1 =G(Xn+I,Yn)

to solve the equations to three decimal accuracy.

BIBLIOGRAPHY

[1] Froberg, C.-E., Introduction to Numerical Analysis, Addison-Wesley, Reading,

Mass. (I965).

[2] Noble, B., Numerical Methods: 1, Oliver and Boyd, Edinburgh (1964). [3] Noble, B., Numerical Methods: 2, Oliver and Boyd, Edinburgh (1964). [4] Williams, P. W., Numerical Computation, Nelson, London (I 972).

.. Linear Algebra ethods

Numerical

10.1 INTRODUCTION

· In all branches of engineering there are practical problems which reduce to the solution of sets of linear algebraic equations. It-is not uncommon for there to be hundreds or even thousands of such equations although often the coefficient matrices , are sparse, that is, contain a large number of zeros. From a theoretical point of view

· the solution of a system of n equations in n unknowns may be written down using determinants (Cramer's rule, as described in §8.4). Though the use of determinants may be convenient for very small systems of, say, three or four equations, it was

pointed out in Chapter 8 that this method proves quite impracticable for much higher order systems. This is because the number of arithmetic operations needed to evaluate

· an nth-order determinant directly is of the order of n! and this increases very rapidly

with n. Even evaluating the determinants in the best manner possible requires a number of operations proportional to n", The direct methods for the solution of linear equations described in this chapter reduce the computation to a number of operations proportional to n3• Hence it is far more efficient numerically to use one of the direct

or iterative methods described in § 10.2 and § 10.3. All the methods of this chapter

are suitable for large systems although it is only practical here to illustrate them by small systems of orders 2, 3 and 4.

Direct methods are generally economical on computing time as the solution is obtained in a fixed number of arithmetic operations, but they require more programming time and more computer store than iterative methods. On the other hand, for the. iterative methods which are much easier to program, the computing time is ~ependent upon the accuracy required of the solution and the method may be very time-consuming for a slowly convergent system. Most computer systems which are used for scientific work have readily available standard library subroutines. These libraries always include programs for solving linear algebraic equations which are based upon the direct methods of § 10.2.

This chapter is concerned with the system of n equations in n unknowns

al IXI +a12x2 + ..• +al nXn = bI, a2Ixi +a22x2 + ... +a2nXn = b2,

Linear Alget»:« - Numerical

Methods for the Sotutton oj Ltnear equanons

382

where the coefficients aij and right-hand sides b, are all constants. This system is written in matrix form as

value of:n is found dirb,-<y from the last equation. Then this value is substitute ...

the previous equation to yield the value of x and so on Thi . k

. n -I, . IS IS nown as

..... 1·_.(llh( IH11 tio n.

When the .calculations are performed by hand using a desk calculating machine a checking procedure can be induded. This consists of carrying an extra column numbers, the elements of ~hich are the sums of the coefficients and the right-hand element of each row. This extra sum column is operated on in th

. e same manner as

equatIOns themselves. At each stage the elements of this extra column should b

sum of all the coefficients of each new equation (apart from round' ) The

. '11 d . mg errors . e

...vo,>ollllrt' IS I ustrate In the following example.

AX= B, where X is the column vector of unknowns and B is the column vector formed the right-hand sides of the equations. lt is assumed that B is not the zero vector

that det A#- O. The difficult situation when det A is very small compared with elements of A is considered in § 10.2. The equations are then said to be ill-condittoe

Formally the solution of equations (1) may be written as

X=A-1B, and sometimes it is useful actually to compute the inverse matrix A-I. The inverse

a matrix is defined in terms of the adjoint matrix (see §8A)which is formed f cofactors of det A. As with Cramer's rule this is not a practical method for nt"nn.;' .... computation and the two methods of matrix inversion described in § lOA are extensions of the direct methods of § 10.2 for the solution of linear equations.

Closely associated with the problem of the solution of linear equations is that of ,'. finding the eigenvalues and eigenvectors of a matrix and in § 10.5 a simple iterative' method is described for finding the largest eigenvalue and corresponding to'V"n"p",,-

the system of equa tions O.6x + 0.8y + O.lz ::: 1, l.Ix + OAy + 0.3z = 0.2,

x + y + 2z = 0.5.

The coefficient 1.1 is the largest in the first column and is selected as the first

Thus the seco~d equation is the pivotal equation and is used to eliminate x from

other two equations. A new first equation is obtained by adding to the first --:-0.6/1.1 (= -0.545455) times the second equation. As a check this same IS perf~rmed on the numbers 2.5 and 2.0 which are the sums of the

. '} and nght-hand sides of the first and second equations respectively. This

in ~.5 - 0 .. 545455 x 2.0 = l.409090 which is exactly the sum of the

iDetticlent a~d ng~t-hand side of the new first equation. The new third equation is by adding to It -1/1.1 (= -0.909091) times the second equation. The check is by 4.5 --:- 0.909091. x 2.0 = 2.681818, which only differs from the sum of the iDeUfici€mts an~ nght-ha~d side of the new third equation by 1 in the last figure. The sc~eme IS set out.m table 10.1, where the multipliers are in the column headed the pivots are underlined and the sum column is headed E.

The pivotal equations are

l.lx + OAy + O.3z = 0.2,

0.636364y + l.727273z = 0.318182,

- 1.642857z = 0.6.

substitution gives the following results, to five decimal places,

z = -0.36522, y = l.49130 and x = ..,..0.26087.

". a check o~ the final solution, these values can be substituted into the original set of or into the sum of the equations. For this example the sum is

(0.6 + 1.1 + l)x + (0.8 +004 + 1}y + (0.1 +0.3 + 2)z = 1 +0.2 +0.5,

2.7x + 2.2y + 2Az = 1.7,

2.7 x (-0.26087) + 2.2 x l.49130 + 204 x (-0.36522) = l.69998.

10.2 DIRECT METHODS FOR THE SOLUTION OF LINEAR EQUATIONS

Pivotal Condensation or Gaussian Ettmination

This direct method for the solution of a system of linear equations is based on the straightforward elimination method of elementary algebra applied in a systematic fashion. The method is similar to the procedure of row echelon reduction urscussen detail in § 8.7. The manner of the elimination is particularly important when only . limited number of decimal places can be used in the calculations. It was devised so to minimize the computational error by ensuring that all the multiplying factors

less than unity.

The first step is to examine the system of equations (1) and select the largest

coefficient of x 1 as the first pivot. The equation containing this coefficient is the pivotal equation and multiples of this equation are added to each of the equations to make the coefficients of x 1 zero. If the first pivot is asl then the multiplying factors are -a 1 dasl , -a2 dasl , .... This eliminates the variable Xl

(n _ 1) of the equations. The pivotal equation is then set aside leaving (n - 1) in the variables x 2, X 3, ..• , xn. The largest coefficient of x 2 is chosen as the pivot and the elimination procedure is repeated. In this manner a set of pivotal equations is obtained having an upper triangular coefficient matrix of the form

asixi +as2x2 + ..

CX22X2 + CX2 3X3 + ...

+ asnxn = bs' + CX2 nXn = d 2,

CXn_l,n-\lXn-1 +CXn_I,nXn =dn-I, CXn. nX" = dn·

384 Linear Algebr»= Numerical
TABLE 10.1
x y z b ~
m
-0.545455 0.6 0.8 0.1 2.5
1.1 0.4 0.3 0.2 2.0
-0.909091 1 2 0.5 4.5
-0.914285 0.581818 -0.063637 0.890909 1.409090
0.636364 1.727273 0.318182 2.681818
-1.642857 0.600000 -1.042856 Methods for the Solir=n of Linear Equations

x y z b ~
0.6 0.8 0.1 2.5
1.1 0.4 0.3 0.2 2.0
1.71 1.2 0.41 1.2 4.52
0.3789 -0.0439 0.5789 0.9139
-0.3]20 0.0362 -0.5720 -0.9077
-0.0069 -0.0036 -0.0104 ill-conditioned Equations

It is important to note that numerical methods may fail to yield the required when the solution is very sensitive to the values of the coefficients. Equations of type are said to be ill-conditioned. The phenomenon is associated with the ~"' .. '''.uu . when the elements of A-I are very much larger than the elements of A itself. A guide to this situation is given by the determinant of A which can be found as a by-product of pivotal condensation. Since the determinant of a matrix is unchanged .'. by the elementary row operations of this method, and the determinant of a matrix is the product of its diagonal elements, then

",,"'Ollrjsrm shows that on working with only four decimal places the answer

is accurate only to one decimal place. However, this solution does fit the very well. If the values

y = 1.5883, Z = 0.5217

into the left-hand sides of the original equations they give

example the ill-conditioning is caused by the third equation being very close to sum of the other two.

det A = ±as1a22a33 ... an,,' The sign depends upon the reordering necessary to obtain equations (2). When the modulus of det A is very small, the elements of A-I tend to be very much larger the elements of A and Ill-conditioning occurs.

. To improve the accuracy of the solution of an ill-conditioned system of equations, the computations must be performed carrying more significant figures or .1~\,LlV ... c may be computed as follows. Let X be the exact solution, X' the solution, and C = X - X' the required correction vector. Then, since

EXAMPLE 2

Solve the system of equations

0.6x + 0.8y + O.lz = 1, l.lx T DAy + O.3z = 0.2, l.71x + 1.2y + OAlz = 1.2.

Using the method of pivotal condensation and working with four decimal places results of table 10.2 are obtained giving

C must be the solution of the system AC = B - AX' = R.

z=0.5217,

y = 1.5883, x = -0.5380.

components of R are called the residuals. Further improvements can be made by II!;UI14""5 a new set of residuals and repeating the process.

The accuracy of this solution is suspect because there is a loss of significant in the coefficient of z in the second pivotal equation and also in the coefficients third pivotal equation. In fact the determinant of A is ±0.0045 which is small compared with the individual elements of A. The solution found by using many significant figures is

z= 0.54545, y= 1.59091, x =-0.54545.

MPLE 3

the corrections to the solution computed in example 2.

residuals corresponding to the solution

x = -0.5380, y = 1.5883, z = 0.5217,

386

lAflt;:;UI .tilOt:UIU - l' UIILC./U ... -W

are

( I) (0.6

R = 0.2 - l.l

1.2 1.71

0.8 0.1) (-0,5380) (-0.00001)

OA 0.3 1.5883 = -0.00003 .

1.2 OAI 0.5217 0.00012

Thus the required correction is the solution of the system 0.6cI + 0.8C2 + 0.lc3 = -0.00001,

1.1 CI + OAC2 + 0.3C3 = -0.00003,

1.7lcl + 1.2C2 +OAlc3 = 0.00012,

where CI, C2 and C3 are the three components of the correction vector C. The condensation scheme of example 2 gives the pivotal equations

1.7lcl + 1.2C2 + OAlc3 = 0.00012,

0.3789c2 -0.0439c3 = -0.00005211, -0.0069c3 = -0.0001584.

Back substitution gives

C3 = 0.0230, C2 = 0.0025, CI = -0.0072.

Adding these corrections on to the first solution gives the improved solution

x = -0.5452, Y = 1.5908, Z = 0.5447,

which is now accurate to three places of decimals.

The method of elimination works best when all the elements of A are of the order of magnitude. When some elements are very much larger than others it is either to scale one or more of the unknowns or combine the equations suitably bring all the coefficients to the same order. Finally, it is a simple matter to use direct elimination method for solving a set of equations with a number of right-hand sides (see example 1 of § lOA). The row operations are applied to all right-hand columns at the same time.

equations are equivalent to the twosystems of equations

(4)

(5)

InlYI +ln2Y2 + ... +lnnYn =b i ,

may be solved for Y by forward-substitution, that is, Y I is found from the first this value is used in the second equation to give Y2 and so on until all the

of the elements of Y have been found. The computed value of Y is now used in (4), which has the form

Un-I,n_IXn_1 +un_I,nxn = Yn-I, un,nxn = Yn'

system of equations is solved by back-substitution, that is, Xn is found first from

last equation, then Xn -I, and so on back to x I'

core of the method is thus the decomposition (3) and this is now illustrated for . x 3 matrix A. For convenience the diagonal elements of L are chosen to be unity (3) becomes

0 nn' u" u,,) C ali a,,)
U22 U23 = a21 a22 a23· .
132 0 U33 a31 a32 a33
out the left-hand side gives Triangular Decomposition or Choleski's Method

This method depends upon the result of matrix algebra which states that a square· matrix A can be expressed in the form

where L is a lower triangular matrix and V is an upper triangular matrix. The only: condition on A is that the determinants formed from the first r rows and r ,",UIIUIIHK' should be non-zero for r = 1,1, ... , n - I. As L and V together have n2 + n elements and A has only n2 elements, the diagonal elements of either L or V chosen arbitrarily, and the remaining elements are then determined uniquely.

A= LV,

proceeds by alternately finding a row of V and a column of L. The first of V is found directly by equating the corresponding elements of the first row of

equatio. ,6) giving

UII=all, uI2=aI2, UI3=aI3.

Next the first column of L is completed by equating the second and third elements' the first columns of equation (6) to give

12l = a 2 du I" 13 I = a 3 .;U I I .

The second row elements of (6) are equated to complete the second row of U with

U22 =a22 -/2IuI2~ U23=a23 -/2IuI3'

The second column of L is completed from the third elements of the second colunm of (6), namely

13 2 = (a 3 2 - 13 I U I 2 )/U 2 2 ,

and finally equating the remaining element completes U with

U 33 = a 33- 13 lUI 3 - 132 U 23·

The general formulae for calculating the elements of Land U when A is an n x n matrix are

Ulj=alj,/il=ail/Ul1' i-I

Ujj = aij - I Ijsusj 1 < i ~j,

s=1

and

lij = (aij - jf liSUSj) /Ujj i ~ j > 1.

s=1

These equations are used alternately to complete U and L. 111is method is often in computer programs as it is very economical on storage space. It is unnecessary store the O's in either Lor U and the 1 's on the diagonal of L. Furthermore the elements of A are used only once and the program can be arranged to replace the elements of A in the store by the elements of Land U.

EXAMPLE 4

Solve the system of equations

0.6x + 0.8y + O.lz = 1, l.lx + OAy + O.3z = 0.2,

x + y + 2z = 8.5,

by triangular decomposition.

Replacing the elements ajj by the numbers in this example, equation (6)

UII =0.6,

uI3=0.1, 12Iu'3+u23=0.3,

c 0 ~} C"6 0.8 0.1 )
L = 1.833 1 U= 0 -1.066 0.1166 .
1.666 0.3125 0 0 1.796875
(5) takes the form = 1 ,

= 0.2,

1.666y, + 0.3125Y2 + Y3 = 0.5,

YI = 1, Y2 = 0.2 - 1.833 = -1.633,

Y3 = 0.5 - 1.666 + 0.3125 x 1.633 = -0.656250.

these values of Y the equation (4) becomes

O.6x + 0.8y + O.lz = 1,

-1.066y + 0.1166z = -1.633, 1.796875z = -0.656250,

z= 0.656250/1.796875 = -0.36522

Y = -(-1.633 +0.1166 x 0.36522)/1.066 = 1.49130,

X = (l + 0.1 x 0.36522 - 0.8 x 1.49130)/0.6 = -0.26086.

amount of computation may be red uced for the special case in which A is . In this case it is possible to choose an L such that

that the matrix L does not have unit elements on the leading diagonal. general formulae for calculating the non-zero elements of L for an n x n

(7)

390

Linear Algebrr - Numerical

matrix A are

i-I

1121 = all, IiI = aii - L li~. (i> 1),

5=1

EXAMPLE 5

Solve the system of equations

0.6x + l.ly + O.lz = 1.2, l.lx + OAy + O.3z = 0.2, O.1x + O.3y + O.lz = 0.5,

by triangular decomposition.

With direct reference to the equations (8) III = (0.6)1/2 = 0.774597,

121 = 1.1 /0.774597 = 1.420094, 131 = 0.1/0.774597 = 0.129099,

122 = [004 - (1.420094)2]1/2 = 1.271482 i,

132 = (0.3 -1.420094 x 0.129099)/1.271482 i = --0.091757 i

133= [0.1-(0.129099)2 +(0.091757)2]1/2 = 0.302907.

The equations to determine y I ,Y2 and y 3 are

0.774597y 1 = 1.2,

1.420094y 1 + 1.271482 iY2 = 0.2,

0.129099YI-0.091757 iY2 +0.302907Y3 =0.5,

giving

YI = 1.549193, Y2 = 1.572967 i, Y3 = 0.513920.

The final solution is then given by

0.774597x + 1.420094y +0.129099z = 1.549193, 1.271482y-0.091757z = 1.572967, 0.302907z = 0.513920,

where a factor of i has been removed from the second equation, giving z = 1.69663, Y = 1.35955, x = -0.77528.

Methods for the Solut+« of Linear Equations

391

the solution, correct to two decimal places, of the system of equations

x + 3y + z = 10 x + y + 5z = 18,

(a) pivotal condensation and (b) triangular decomposition.

the method of triangular decomposition to solve each of the second-order

3x + 2y = 7 x + 5y = 11

to three significant figures.

(b) 10x+ y=15 x + 5y = 23,

ITERATIVE METHODS FOR THE SOLUTION OF LINEAR EQUATIONS

the coefficient matrix is sparse, that is, has many zeros, iterative methods for the of linear equations are normally used in preference to the direct methods of section unless the matrix has a very simple structure. In contrast to the direct which require a fixed number of arithmetic operations, the amount of

!IIPI" ..... lVU for iterative methods is dependent upon the accuracy required.

the iterative methods described in this section are based on the rewriting of the so that each unknown, in turn, is expressed in terms of the remainder of the First, the original system of equations is rearranged, if necessary, so that no lie on the leading diagonal of A and this rearrangement is always possible. Each

is then divided by the corresponding diagonal element after which the

matrix is split into three parts so that

(9)

Land U are lower and upper triangular matrices respectively with zeros on the diagonals, I is the unit matrix and D is formed from the new right-hand sides rearrangement and division.

equation (9) is written

(10)

LE 1

of equations considered in example 1 of § 1 0.2 may be written as

x +0.36364y +0.27273z = 0.18182,

y + 0.125z = 1.25,

O.5x + 0.5y :" z = 0.25,

(11 )

392

Linear Algebra - Numerical

and then rearranged into the matrix form (10) as

(}G~;18}U::5 -L DC) +G

-0.036364 o

o

_0.27273)(

-0.125

o z

Notice that the first and second equations have been interchanged in order that all elements of Land U be less than unity as this helps to ensure convergence.

Jacobi Iteration

This is an iterative scheme of the form

X(n+l) = D + LX(n) + UX(II),

where X(n) is the nth approximation to the solution. In this method, a first approximation X(O) is substituted into the right-han.d side of .equation (~2) to calculate x" ). This new approximation is then put Into the right-hand Side to calculate X(2), and so on. This simple or Jacobi iteration is also known as the of simultaneous corrections '.

TABLE 10.3
n x y z
0 0.000000 0.000000 0.000000
1 0.181818 1.250000 0.250000
2 -0.340909 1.082386 -0.465909
3 -0.084710 1.5153920 -0.120739
4 0.353951 1.328625 -0.489605
5 -0.167789 1.576664 -0.237337
10 -0.287957 1.459618 -0.402080
20 --0.263547 1.488197 -0.368864
30 --0.261133 1.490998 --0.365577
40 -0.260895 1.491274 -0.365253
50 -0.260871 1.491301 -0.365221
51 . 0.260867 1.491306 -0.365215
52 -0.260871 1.491302 -0.365220
53 -0.260868 1.491305 -0.365216
54 -0.260870 1.491303 -0.365219
55 --0.260868 1.491305 -0.365216 Methods for the Sotution of Linear Equations

Jacobi iteration scheme for example I is x(n+l) = 0.181818

/n+l) = 1.25 /n+ I) = 0.25

- 0.36364y(n) - 0.27273z(n),

- 0.75x(n) -- 0.125z(n),

- 0.5x(n) - 0.5y(n).

Starting from a first approximation (0,0, 0) the results shown in table 10.3 are Note that it takes about 55 iterations to obtain the solution accurate to places.

enractice Jacobi iteration is not often used, because the rate of convergence may

be improved by using the new values as soon as they become available. That is

, . d . (n + I) ,

first equation etermines x I and this value is used instead of x(n) in the

equation to give a new value x~n +1). Both x~n+l) and x~n +1) ar: used in

third equation, and so on. This is Gauss-Seidel iteration and may be written as X(n+l) = D + LX(ll+l) + UX(>I).

(13)

example becomes

_ 0.75x(n+ I)

- 0.36364y(lI) - 0.27273z(II),

- 0.1 25z(n),

z(n+l) = 0.25

starting with (0,0,0) gives the results shown in table 10.4. Thus for this problem

x y z
0.000000 0.000000 0.000000
0.181818 1.113636 -0.397727
-0.114669 1.385718 -0.385524
-.0.216936 1.460892 -0.371978
-0.247966 1.482472 ~0.367253
-0.257102 1.488733 -0.365816
-0.259771 1.490555 --0.365392
-0.260549 1.491086 -0.365268
-0.260776 1.491240' -0.365232
-0.260842 1.491285 -0.365222
-0.260861 1.491298 -0.365219
-0.260867 1.491302 -0.365218
-0.260868 1.491303 -0.365218 394

Linear Algebm. - Numerical

30'"

Successive Over-relaxation

Gauss-Seidel iteration convergence is 5 times faster than the Jacobi iteration as the method gives the solution accurate to 5 decimal places in only 11 iterations.

Irim,·ric,a! Methods of Matrir tnversion

Obtain a solution, correctto two decimal pIa f h

ces, 0 t e system

4.2x +- 2y + z= 11.2

x + 3y + z == 10

x + y + Sz "" 18

the methods of (a) Jacobi and (b) Gauss-Seidel.

However, iterative methods do not always converge. As a general rule it is to arrange the equations so that the largest coefficients lie on the leading diagonal.·· fact, it can be shown that if A is diagonally dominant, that is, in each row the value of the diagonal element is greater than the sum of the absolute values of tht other elements in that row, then convergence is assured. Furthermore, the more dominant the diagonal elements, the faster the rate of convergence.

the method of successive over-relaxation to solve exercise 2

. ronvergence. using two different choices for w. . Compare the rates

A further improvement in the rate of convergence may often be obtained by a known as successive over-relaxation. If the Gauss-Seidel iteration is writtenin

X(1l + 1 ) = X(Il) + {LX(n + 1) + (U - I)X(n) + D}, then the term {LX(n+ 1) + (U -I)X(ll) + D} is the correction or improvement the 11th iterate to give the (n + 1 )th iterate. A parameter w is introduced to 'overcorrect' at each stage by the formula

X(ll+]) = X(Il) + w{LX(n+]) + (U - I)X(1I) + D}.

This parameter w is called the relaxation factor, The obvious difficulty of this is how to choose the value of w that gives the fastest rate of convergence. This optimum value of w lies between 1 and 2 and is usually found by trial and although a formula is available for calculating it (see Modern ComputingME~tlu>th p. 39). It is particularly worth while trying to find a good value of co when the of equations is to be solved a number of times with different right-hand sides.

NUMERICAL METHODS OF MATRIX INVERSION·

inverse A -] of a non-singular square matrix A is defl e d i §84 d h

n 111 ,an as the

, . (15)

section IS concerned with the extension of th di

of finding A +I . e irect methods of § 10.2 to the

is the rth column of the matrix A -] and e . th h

h . -1 r IS e rt column of the unit mat'

t e matrices A and I can be partitioned (see §8,6) as nx

A-I = (XI X2 ... Xn) and 1== (el e2 ... en),

l_uaL .• VlI (IS) may be written

EXAMPLE 4

For the system (11) considered in this section the formulae are

x(1l + 1) = x(n) + w{ 0.181818 - x(n) - 0.36364y(n) - 0.27273z(n)}, /n+ 1) = y(n) + w{ 1.25 _ 0.7Sx(n+l) _ y(ll) - 0.I25z(n)}, Z(,/+l)= z(n) + w{0.2S _ 0.5x(n+l) - 0.5/n+1) - z(n)}.

When t» = 1.1 an accuracy of 5 decimal places is obtained in 8 iterations, only a little faster than the Gauss-Seidel method. For some problems the is very much greater than this,

Probl:~;:~~~e:~~:~~~~~_~a:n;:~ds~S!~::h~f erations which all have the

of pivotal condensation to find the inverse A -1 co umns of I. ~o use the.

are performed on all the columns of I h ' all t.he oper.ah~ns described

.. form at t e same time. This gives equations

Exercises

1. Use the Gallss-Seidel iteration method to obtain a solution correct to two places of the system

lOx + y=15

x + Sy = 23.

(r= I, 2, ... n),

A' is upper diagonal and fr are the new right-h .

Itk.l;ub:stitutions with each right-hand sid f' and ~ldes. Then a sequence of

matrix. e r 1I1 turn gives all the columns X, of

396

metnoas OJ lVJatruc Inversion

397

EXAMPLE 1

Find the inverse of the matrix

matrix of the same ty l'<! (§ 8.4( d)) the inverse of the matrix

L= (;:: I~' ~),

131 132 133

of the matrix equation

(0.6 0.8 0.1)

1.1 0.4 0.3 .

1 1 2

The method follows the pattern of example 1 of § 10.2 and is shown in

i(i:: ;:: D(::: ::: X~}C r ~}

. are the elements of the inverse matrix L -I. The left-hand side of this is evaluated to give

TABLE 10.5
m xi Yi Zi el e2 e3
- 0.545455 0.6 0.8 0.1 1 0 0
1.1 0.4 0.3 0 1 0
--0.909091 1 1 2 0 0 1
--0.914285 0.581818 -0.063637 1 -0.545455 0
.0.636364 1.727273 0 -0.909091 1
-1.642857 0.285713 -0.914285 Illxll 121xII +/22X21

o

o ) (1 0 0)

o :: 0 1 °

133X33 0 0 1

12 2X2 2 132X22 +/33X32

corresponding elements from both sides the unknown values X·· may be

turn as follows IJ

Then a sequence of back-substitutions using the right-hand sides (0, 1, 1 )T, (1, -0.909091, 0.285713l and (0,1, -0.914285l in turn generates the A -I and gives

1=1//11, X21 =-/2IXII/122' X31 =-(/3IXII +/32X21)/I33,

X32 :: -/32X22/133' X33 = 1//33,

similar fashion the inverse of the upper triangular matrix U is given by

1 ~_.!2_ UI2U23_~

UII UIIU22 UIIU22U33 UIIU33

(-0.43478

A -I = 1.65217

-0.60870

1.30435 -0.17391) -0.95652 0.06087.

-0.17391 0.55652

o

Triangular Decomposition or Choleski's Method This method is convenient for inverting matrices because the inverse of a matrix is also a triangular matrix of the same type, and triangular matrices are invert. We first decompose A into the product

o

o

and U-I the product U-I L -I is formed to give the required formulae for calculating the components x·· of L -I h L . 1

x n matrix are IJ w en IS a ower

X'j"- 0; I,,x,j )fii (j<i)

tSYij of U-I when U is an upper triangular n x n matrix the

A=LU,

where L is a lower triangular matrix and U an upper triangular matrix. The the matrix A is

A-I = V-I L-I,

remembering (§ 8.4) the result that the inverse of a product of matrices is the ... of the inverse matrices but formed in the reverse order. The method of trilm!!:. decomposition is described in § 10.2 and thus the problem here is to invert triangular matrices Land U. Since the inverse of a triangular matrix is itself a

(i <j).

Linear Algeora - ivumencas

398

EXAMPLE 2 Invert the matrix

(~'.~ ~:: ~:~).

I I 2

In example 4. of § 10.2 this matrix was decomposed into the product

LU == (1.:33 ~ ~)(0~6 _1~~86t~ 0~·:66).

1.666 0.3125 1 0 0 1.796875

The inverse L -I of L'is given as the solution of the matrix equation

o

0.3125x22 +X32

so that

( 1

L -I == -1.833 -1.093750

o

-0.3125

Similarly

(1.666

U-I == 0

o

-0.173913) 0.060870 ,

0,556522

.1.25 -0.937500 o

and the inverse of A is the product

(-0.43478 1.30435

U-I L -r l == 1.65217 -0.95652

-0.60870 -0.17391

-0.17391) 0.06087 .

0.55652

It was noted in § 10.2 that for a real symmetric matrix the decomposition the form

A== LLT.

In this case only one triangular matrix need be found and inverted since

A-I ==(LLT)-I ==(LT)-IL-J ==(L-llL-I.

lYle"'UUI> UJ l'lUII LA. l"VC::II>IU"

0 .. 6 1.1 0.1) l.l 0.4 0.3 .

0.1 0.3 O. I

. decomposition of this symmetric matrix was presented in example 5 of § 10.2. Lis

o 1.271482 i --0.091757 i

03~2907) .

elements of the inverse L -I are given by a matrix equation similar to equation

( 1.290994

-\ == 1.441886 i

-0.987000

o )

o ,

3.301343

o -0.786484 i 0.238243

( 0.56180

== (L +I fL -I == 0.89887

-3.25843

0.89887 -0.56180 0.78652

-3.25843) 0.78652 .

10.89887

(a) pivotal condensation and (b) triangular decomposition the inverse of

(b)

G ~) .

the following symmetric matrix

-0.064 0.283 -0.076 0.020

-~:~~: -~:~~~).

0.289 -0.077

-0.077 0.289

400

10.5 EIGENVALUES AND EIGENVECTORS

(18)

4'"

The eigenvalues of a square matrix A are those values of A for which the system

CI is the numerically largest element of the vector Z( I). Similarly 1 n

Z(2)= - I o, "A,. (AXr)

CI r=1

AX = AX,

has non-trivial solutions (see §8.9). The vectors X associated with each A are the eigenvectors. The direct method given in § 8.9 for finding the eigenvalues is for large matrices as it involves finding the roots of a large order algebraic Pfllll<lti ...... so other methods must be used. The corresponding eigenvectors are the solution system of homogeneous 'equations and are determined only to within an arbitrary .•. constant multiplier. In the iterative methods of this section the multiplier is that the largest element of each eigenvector is equal to unity.

n

= IO:rA;Xr,

CI r=1

after k iterations that

Largest Eigenvalue

The following is an iterative method for finding the eigenvalue with the largest mod ulus and its associated eigenvector.

Suppose the matrix A has n real distinct eigenvalues AI , A2, . , . , An where I Al I> I A21 > . , . > I An I and corresponding eigenvectors Xj , X2, ... , Xn· starting with an arbitrary vector y(O) the scheme

Z(i) = Ay(i-I),

r=1

I ArlAI I < 1· for r i= 1, then I 'Ar/AI I k -+ 0 as k -+ 00. Furthermore the divisor Ck to ensure that the largest element of y<k) is always unity, and hence

y(k) -+ XI as k -+ 00.

, 1 (')

y(t) =-z t ,

Cj

where Cj is the numerically largest element of Z(i), gives a sequence of vectors y(i) which in the limit i -+ 00 tends to the eigenvector XI' The eigenvalue AI is given by limit of the sequence of numbers c., as i -+ 00.

The above method follows from the result that an arbitrary vector y(O) may expressed as a linear combination of all the eigenvectors X, as follows

as k -+ 00,

the component of Z(k) that corresponds to the unit component of y(k-I)

tend to the eigenvalue AI . A difficulty with the above method can occur should vector Yo be accidentally chosen such that 0:1 is very small, in which case ...... ~,Up'·OPln"p is very slow. An improvement in the rate of convergence can

be made by a different choice of initial vector y(O).

11

y(O) = I o, x.,

r=1

where the a,. are constants. Then from equation (17) Z(l)=AY(O)

largest eigenvalue and corresponding eigenvector of the matrix

( 1~ : l~).

-5 -5 -3

n

= I a,.(AXr)

r = I

/I

cOI?ponents of Z(i) are z~j), z~i), z~i) and the components of y(i) are ,y~'), then for this matrix

z(i) = 17y(i-l) + 9y(i-l) + 5y(i-l)

'. I I 2 3,

'z(i) = 8,,(i-l) + 1O,,(i-l)

2 v: 2 .r 3 ,

z(i) = -5y(i-l) - 5y(i-l) - 3y(j-l)

3 I 2 3'

r=l

and

402 l..A"r:;Uf r1.'6LU' w.
TABLE 10.6
\i) z~i) Z)i) y\i) Ai) Ai)
z
0 1.0000 1.0000 1.0000
1 31.0000 18.0000 -13.0000 1.0000 0.5806 -0.4194
2 20.1290 0.4516 -6.6452 1.0000 0.0224 -0.3301
3 15.5513 -·3.1218 -4.1218 1.0000 -0.2007 -0.2650
4 13.8681 -4.2564 -·3.2012 1.0000 -0.3069 - .. 0.2308
5 13.0836 --4.7636 -2.7729 1.0000 --0.3641 -0.2119
6 12.6635 - 5.0321 -2.5437 1.0000 -0.3974 -0.2009
7 12.4193 -5.1877 -2.4105 1.0000 -0.4177 -0.1941
8 12.2701 --5.2827 -2.3292 1.0000 -0.4305 -0.1898
9 12.1761 - 5.3425 -2.2779 1.0000 --0.4388 -0.1871
10 12.1157 - 5.3809 -2.2449 1.0000 --0.4441 -0.1853
15 12.0149 -5.4451 -2.1899 1.0000 -0.4532 -0.1823
20 12.0020 --5.4533 -2.1829 1.0000 -0.4544 -0.1819
25 12.0003 -5.4544 - 2.1820 1.0000 - 0.4545 --0.1818 (A - pl)Xj = U\j - p)Xj, for any p, the eigenvectors of the matrix A - pi are the same as those of A, but eigenvalues differ by an amount p. Suppose for example Al = 6, A2 = 5. Then of convergence is determined by the rate at which (5/6)k ~ 0 as k ~ 00. But if of A, A - 31 is used, then the two largest eigenvalues are 3 and 2 and the rate

:ate at which (2/3l ~ O. This is more than twice as

(2) 24

and 3 = 0.00005.

E2

the rate of convergence in example 1 may be increased by using A - 41

of A.

_41=(1~ : 1~)'

-5 -5·-7

the same method as in example 1 and starting with the vector (1, 1, I)T the results of table 10.7 are obtained.

II ("-r) k

L ay Al x, ~ 0 as k ~ 00.

r=2

However the largest of the quotients "-rIAl is A2/AI and so the rate of

, k .

depend upon the rate at which (A2/Ad ~ 0 as k ~ 00. For this reason the rate.

convergence is often slow but it may sometimes be improved by a simple device .••.

Since

z~i) z~i) y~i) Ai) Ai)
1.0000 1.0000 1.0000
27.0000 14.0000 -17.0000 1.0000 0.5185 -0.6296
:14.5185 -4.2222 -3.1852 1.0000 -0.2908 -0.2194
9.2857 -3.3571 -2.0102 1.0000 -0.3615 -0.2165
8.6637 -3.6110 -1.6769 1.0000 -0.4168 -0.1936
8.2811 -3.6027 .: 1.5611 1.0000 -0.435 I -0.1885
8J419 -3.6254 -1.505 I 1.0000 -0.4453 -0.1848
I 8.0682 -3.5297 -1.4796 1.0000 -0.4499 -0.1834
8.0342 -3.6334 -1.4669 1.0000 -·0.4522 -0.1826
8.0169 --3.6348 --1.4607 1.0000 -0.4534 -0.1822
8.0085 -3.6356 -1.4576 1.0000 -0.4540 -0.1820
-3.6363 -1.4546 1.0000 -0.4545 -0.1818 and if c, is the numerically largest of z~j), z~i) and z~i) then y~j) = z~i)lcj, Ai) = z~i)lci' y~j) = z~i)lcj.

Starting with the vector y(O) = (1, 1, I)T the results of this scheme are table 10.6. (i) From the table it may be seen that the component of Z that corresponds

unit component of y(i -I), to three decimal places, i~ 12 and the corresponding eigenvector has components (1, -0.4545, -0.1818) .

Referring back to equation (18) the rate at which y(k) tends to XI is by the rate at which the terms

the table it may be seen that to an accuracy of three decimal places the eigenvalue of A - 41 is 8, so that the largest eigenvalue of A is 12. The corresponding to this eigenvalue is the same as before, namely

, -0.1818). The rate of convergence is appreciably increased as three place accuracy is obtained in 15 iterations, as compared with 25 iterations in 1.

of the inverse matrix A -I are Ail, A;I , etc. Therefore to find the smallest eigenvalue of A the numerically largest eigenvalue of A -I is

404

.'+UJ

sought by replacing A by A -1 in the formulae (17) giving Z(i) = A-I y(i -I ) ,

z~i) z~i) Ai) Ai) y~)
1.0000 1.0000 1.0000
0.4063 -1.2813 1.1250 . -0.3171 1.0000 -0.8780
-0.2612 0.7246 -0.4797 -0.3604 1.0000 -0.6620
. -0.2108 0.5446 --0.3357 -0.3871 1.0000 -0.6164
..
-0.2025 0.5 III -0.3089 -0.3962 1.0000 -0.6044
-0.2006 0.5029 -0.3023 -0.3990 1.0000 -0.6011
.... -0.2002 0.5007 -0.3006 -0.3997 1.0000 -0.6003
-{l.2000 0.5002 -0.3002 -0.3999 1.0000 -0.6001
. -0.2000 0.5000 -0.3000 --0.4000 1.0000 -0.6000
the matrix to a simpler form and are used when all the eigenvalues are and

y(il = _!_ Z(i). c·

I

It is unnecessary to compute the inverse A -I explicitly as (20) may be written

AZ(i) = y(i-I).

Now at each iteration a system of equations with the same left-hand side must solved to calculate the vector Z(i). Either of the direct methods of § 10.2 mn'''' .. ' .... suitable because the main calculation on the coefficient matrix A need only be once and each system (21) can then be solved with a minimum of computation.

This method of finding the smallest eigenvalue may be used to find the nearest to a given value p. The eigenvalues of the matrix A - pI are Al - P, 11.2 - p , ... , An - p and if the numerically smallest eigenvalue of A - pi is As - P then As is the eigenvalue nearest to p.

by iterative methods (a) the largest eigenvalue and corresponding eigenvector the smallest eigenvalue and corresponding eigenvector of the matrix

EXAMPLE 3

Find the smallest eigenvalue of the matrix

(l~ : l~)'

-5 -5 -3

the eigenvalue of largest mod ulus of the matrix

To find the smallest eigenvalue the scheme is to solve the system 17z~i) + 9z~i) + Sz~i) = y~i -I),

8z~i) + I Oz~i) = Ai-I),

-Sz(i) - Sz(i) - 3z(i) = y(i-I)

I 2 3· 3 '

and then calculate

Y(i) = z(i),c' y(i) = z(i)/c. y(i) = z(i),c'

I I t' I' 2 2 I' 3 3 t' I'

where c, is the numerically largest of z\i), z~i) and z~i). Starting with the vector y(O) = (I, I, I)T the first eight iterations for this scheme are shown in table I The largest eigenvalue of A -I is then 0.5 to four decimal places and rner .. t~" smallest eigenvalue of A is 1/0.5 = 2.0. The corresponding eigenvector to an of four decimal places is (-0.41, 1, __ 0.6)T.

the Gauss-Seidel iteration method to obtain the solution of the system of

XI+IOX2+ X3 =10,

3xI + IOx3 + X4 4,

2x 2 + I Ox 5 - X 6 = 0,

3X2 + 2xs 6,

+ 20X4 + 4X6 = 12,

X2 - 3X3 + IOx6 = 0,

to two decimal places.

Finally, these iterative methods, which are simple to use, would not normallvl used to find all the eigenvalues and eigenvectors of a matrix. Other methods, those due to Jacobi,Givens and Householder (see, for example, Modern l.,O'mfJlfIII Methods [1] and Bell [4]), are based upon the use of orthogonal

406

Linear Algebra - Numerical

2. Write down a simple generalization of the Gauss-Seidel metnod for the the non-linear system

x - O.lyZ + 0.5zz = 0.7, y + 0.5xz - O.lxz = 0.5, z + OAyz + O.lx)' = 1.2.

Solve the system with an accuracy of two decimal places.

3. Devise a direct elimination procedure for the solution of the system of line8l equations AX = B' when A has the tri-diagonal form

b , Cl 0 0 0
az bz Cz 0 0
0 a3 b3 ('3 0
A=
0 0 an_l bn-1 C>I_(
0 0 0 an b" [This is one of the special forms of A for which a direct method is preferred to iterative method even though A is sparse.] Use your method to find the solution correct to three decimal places of the system

4x - y

x + 3y - z

)' + 3z-

= 3.2, = 0.8,

w = 1.3,

- z+4w=2.7.

4. The system of equations

2Xl +2xz - X3 -4X4 = 8,

XI - Xz + 3X3 - X4 = -3,

2,

10XI + 3xz + 2.lx3 - 6X4 = 5,

is somewhat ill-conditioned. Use the method of pivotal condensation, working three decimal places, to calculate an approximate solution. Improve this . computing corrections to this solution in the manner described in § 10.2.

S. Find the inverse of each of the matrices

(a)

( I ~ -I \ (b) (~~ I - I)

~ -14 _:)' _: 4_:

By modifying the meth oc.j ; § 10.5 derivean iterative method of finding the eigenvalue and corresponding eigenvector for the more general eigenvalue

A and B are both sq uare matrices of the same size. Use this method to find the' eige11:value and corresponding eigenvector when

and B = C -~).

. Modern Computing Methods, H.M.S.O., London (1961).

Williams, P. W., Numerical Computation, Nelson, London (1972).

Fox, L., A n Introduction to Numerical Linear A lgebra, Clarendon Press, Oxford (1964).

Bell, W. W., Matrices for Scientists and Engineers, Van Nostrand Reinhold, Wokingham (1975).

CHAPTER II

in effect, that it is~;::,lsible to approximate any continuous function, to degree of accuracy in any finite interval, by a polynomial of sufficiently

Finite Differences

this polynomial is chosen to coincide with the data at suitably. chosen which case it is called a collocation polynomial. In practice it is usually to specify these points to lie at equal intervals of x, i.e., at constant

11.1 INTRODUCTION

imeltiffiles it is more useful to choose the approximating polynomial in such a way of the discrepancies between the data and the polynomial is minimized, actually required to be zero. An example of this is the 'method of least

described in Volume 2, Chapter 3. Such an approximation is often termed a approximation. This chapter will not, however, consider formulae

from such procedures, although some can be obtained by using analysis

that used for collocation polynomials.

In order to assess the behaviour and performance of a physical process or engineer often has to analyse data. The data may be accumulated and 1'";;;)I;:IIu:a.,. number of different ways and required for several types of operation; Typical examples are (a) when the data are in the form of discrete valuesfo,ft, ... specified values x 0, XI, •.. of the independent variable x, and the value lis some intermediate value x; (b) when the data are given analytically by a f(x), and its integral is required in cases for which an analytical solution is

(for example, f~' exp( _X3/2 )dx); or (c) the data may be presented as a graph against x, and the derivative or rate of change of f is required at some pa or values of x. Of all these examples of representing data, the graphical method ' appear to be the most attractive, since the value of I could be read off 'by eye'~ integral would be given by measuring the area under the curve, and even the ' could be estimated by measuring the slope of the tangent at the required point. However, these would necessarily be approximate values and there would be no of correctly estimating the error. To obtain more accurate values, numerical are required.

Graphs, being a visual representation of the behaviour of a function, are of for processing on a desk machine or automatic computer and must be replaced equivalent set of numbers. Similarly, many standard functions often exist, to and purposes, only as tabulated values, and discrete data are by definition just

in describing numerical procedures to analyse data, we consider our raw U'Q,.t:lLilI a table of values of the function specified at certain values of the independ This table is obviously fundamentally important to the subsequent numerical and checking its accuracy is usually the first step when setting out to compute values derived from the table.

Apart from accurate tabulation, the three operations most commonly n .. rf'n __ . a set of data are interpolation, integration and differentiation. The latter two operations of calculus involving limits and infinitesimals, so that they cannot be directly numerically. If, however, the data ~re replaced by an approximating such as a polynomial, then the calculus operations are easily carried out, as is interpolation also. Polynomials are not the only type of approximation for alternative representations are Fourier series or eigenfunction expansions of types, etc. - but they are the most convenient, and most numerical methods for dealing with interpolation, integration and differentiation are based on this approximation. The theoretical basis for these methods is Weierstrass's Theorem

find that the numerical formulae are usually expressed not only in terms of values themselves but also in terms of differences between successive values. are called finite differences and are most conveniently displayed in a table, such as that illustrated in table 11.1. The column of first differences of the differences of successive values of the function f itself. The next

of second differences, which are the first differences of the column of first ; higher order differences are defined in a similar manner. Note that the

First Second Third Fourth
differences differences differences differences
-18182
1.81818 3031
-15151 -701
1.66667 2330 203
-12821 -498
(53846 1832 131
-10989 -367
1.42857 1465 93
-9524 -274
1.33333 1191 63
-8333 -211
1.25000 980 48
-7353 -163
1.17647 817
-6536 410

"2n+l,, _"2n,, ,,2nj

u Jr+1I2-u Jr+l-u Jr,

decimal point and unnecessary zeros are omitted, leaving only the significant each difference. This is conventional in difference tables, and eliminates a source of mistakes. Also note that the entries in table 11.1 are the values of I will be used in some examples treated later in the chapter.

order backward difi __ ences are similarly given by fr== V(Vn-1f)r== Vn~Ifr - Vn-Ifr_I'

to see that now we need to go backward for the necessary information and larger n is, the further backward do we need to go:

V3 fo ::: fo - 3f-1 + 3f_2 - f-3.

ILl we see that the value -0.12821 is the first backward difference at

not x = 0.60 as for the forward differences; 0.01191 is the second backward at x = 0.80, as well as being the second forward difference at x == 0.70.

Notation

If x 0 is a given value of x, and h is a constant step-length, we denote x o + rh f(xr) by [; where r is an integer. Thus

XI ==Xo +h, X2 ==Xo +2h, ... ,xr =xo +rh, x-r ==Xo - rh, ...

We use the notation x == xo(h)xn to denote x varying over the range Xo to xn of length h.

again defined in a manner analogous to the forward and backward

but now draw on information both forward and backward from the point: However, although even order central differences can be associated

. particular choice of x, odd order differences must now be associated with two values of x. The first central difference at x; and Xr+ 1 is defined by

Forward Differences

The first forward difference of f~) at x == x; is denoted by !:1fr and defined by .

!:1fr == [; + 1 - fr .

Thus in table 11.1 the value -0.12821 is the first forward difference at x == Similarly, the second forward difference of f(x) at x == xr is denoted by !:12fr defined as the first forward difference of the first forward differ.ence at x = x, .

!:12fr == !:1(!:1f)y = !:1fr+I - !:1fr·

that here the suffix r + ~ is a notational convenience - it does not represent at x = x; + ~. We also note that ofr has no meaning. However, 02 [; does unambiguous meaning denoting the second central difference at x= xr:

Hence in table 11.1 the second forward difference at x = 0.60 is 0.01832 and at x == 0.70 is 0.01191. In general, the nth forward difference at x = x; is

!:1nfr == !:1(!:1n-1f)r == !:1n-1fr+l - !:1n-Ifr'

Each difference, however high its order, can be expressed in terms of the values. For example, the definition of !:12 fr gives

!:12fr=(fr+2 -fr:l)-(fr+l -fr)=fr+2 -2fr+I +f;,.

The higher the order n, the further forward do we need tabular values; !:12fo values offatxo,xl and x-, but !:13fo needs the value at x , also:

!:13fo = !:12fl - !:12fo == f3 - 3f2 + 3fl - [«.

can be expressed in terms of tabular values as

order central differences are again defined similarly:

.. '. 2" "2n-l,, "2n-l,,

. 6 fr::: u J r+ 1/2 - U J r-: 1/2,

can also be expressed in terms of tabular values: ,63 fr+ 1/2::: fr+2 - 3fr+ 1 + 3fr - fr-l, {j4fr:::fr+2-4f~+1 +6fr-4f~-1 +fr-2'

II .1, the second cen tral difference at x == 0.65 is 0.01832, and the third difference at x == 0.65 and 0.70 is-0.00367.

we note that in table 11.1 the entry 0.01191 now describes the second difference at x == 0.75 as well as being the second backward difference at

and the second forward difference at x == 0.70. This illustrates the fact that entry in a difference table corresponds to each of the three types of finite rrerC;U",,'i>. The general result is, for integers rand n,

ll"fr::: Vnfr+n == {jnfr+'hn'

Backward Differences

Backward differences are defined in a manner analogous to forward differences denoted by V instead of !:1. Thus the first backward difference off at x ::: xr is

vt. = fr -i..«.

412

+6€

EXAMPLE 1

Express Vf3, !l? f _I ,03 f3/2 in terms of the function values fO,[1 , ... ,etc.

V/3 is the first backward difference at x 3 and !l? f _I is the second difference at X_I' Hence, from the definitions,

result follows inu. __ )iately. In particular, the nth difference of a

Vh=f3-h, !l.2f_I=!l.f_o-!l.f_l=fl-2fo+f_I' -,

03/31l denotes the third central difference at x I and x 2 ; thus

03h12 = 02 f2 - 02 I, = f3 - 2h + I, - (J2 - it, + fo) = f3 - 3h + 3/.

+ a 1 x + ... + anxn) = 0 + 0 + ... + ann !hn ,

proportional to h'", and higher differences are zero. This shows that the 1'011111:;1<;11""0 of a polynomial become smaller, provided the step- length h is small. Consequently, with Weierstrass's theorem, we can expect the i.diffplrf>nees of any data to decrease as the order of difference increases, for small h.

Finite Differences of a Function

Although the definitions given above are expressed in terms oftabular values, to extend the definitions to include finite differences of a continuous Thus the first differences of f(x) with step-length hare

wise to ensure that a table of values does not contain a mistake caused, by misreading-an experimental value or by transposing two digits when the value. A convenient safeguard is provided by constructing a

table. Whilst the differences of a correct table of values should decrease in as the order of differences increases, the effect of a mistake in one of the values is to increase the differences and occurs in the following convenient

~f(x) = 1(x + h) - f(x)~ Vf(x) = f(x) - fix - II),

of (x) = 1(x + Yzh) - f(x - ~h).

Higher order differences are defined in a similar manner.

The difference operators ~, V and 0 behave very much like algebraic satisfying the associative law of addition, commutativity, etc. Thus, if f and g two continuous functions,

the value off given at x 0 is written as fo + e instead of the correct valuefo. discrepancy e is propagated in the difference table in the manner shown in

.2, in which only the discrepancies are shown in the second and higher

Table 11.2 indicates that not only is the effect of the mistake magnified as of differences increases, but also that the effect is oscillatory for any

order. Hence isolated mistakes can not only be detected but also pinpointed. same time the sum of the effects for any order is zero; for example, in the

of second differences the sum of the faulty entries is

(62f_1 +€)+(02fo -2€)+(02fl +€)=02f_1 +02fo +02fl'

same result as when the correct value fo is used. This result can be used to the pinpointed mistake.

this procedure will usually reveal random mistakes, it may well miss mistakes. These are usually associated with experimental error, and could

~(J + g) = ~f + ~g,

~m(~nf) = ~1I(~mf) == ~m+nf,

~(cf) = cN,

where c is a constant.

Finite Differences of a Polynomial The ith differences of an nth degree polynomial are polynomials of degree n - i o < i';;;; n and are zero if i > n. Bearing in mind the central role of polynomials theory, this is obviously an important result. It can be proved by considering a term akxk, 0 ,;;;; k';;;; n. Successive forward differences of xk at a typical x are

&k = (x +h)k _xk = khxk-1 + We(k _ l)h2xk-2 + + khk-1x

~2xk =k(k_I)h2xk-2 +k(k-I)(k- 2)h3xk-3 + +(2k -2)hk,

04f

---

---

-----

Of-3/2 __ ------- +c

------ +€

Of-1/2 + e

of

(or M, \If)

+€

-4€

-3€

to. i k - k(k I ) (k . l)hi k - i

ux - - ... -1+ x + ... ,

-2€

__ Of1l2 - € +3€

------ +€ -~

Of312 ---- ~€

-------- +€

-_

--

414

(3)

Differences and Dtfference Tables

\

41 "

be caused by faulty measuring devices, for example, or by a persistent fault in the performance of the experiment.

the entry 6248 should, in fact, be about fo + (-180) giving a corrected value of

EXAMPLE 2

Construct the difference table for the following data, and use it to detect and

tnistake is evidently a transposition mistake. To confirm that 'this is so, and to against any other mistakes, one should re-difference using the corrected value. _'''V''b'' in this example e has been estimated by using the second differences, it be noted that the same result could be obtained by using the third or fourth

. However, it is usually better to use second differences once the mistake

pinpointed.

any mistakes in the given data:

o 1736 3420 5000 6248 7660

-, 8660 9397 9848

The difference table is shown in table 11.3 and shows quite clearly the fourth differences) that the incorrect entry is 6248. The 82 f column shows the average of the three relevant values of 8 2 f (which must be the same as the average) is:

82 fa"'" 1/3(-- 332 + 164 - 412) "'" - 197.

11.2 also shows how an error (such as that due to round-off) is propagated. lhe.rmlDre it shows that apparently negligible errors introduced at any early stage eventually dominate a higher order difference column.

Since the tabulated, incorrect value is 82 fa .: Ze , we have -197-2e""'164, or e""'-180.

TABLE 11.3

general type of finite difference is required if the data are tabulated at nonstep-lengths. It is convenient to retain the notation x 0, X I , ••• , etc., with ordered as

[ 8[

., .<X_I <xo <XI <X2 < ...

now X2 - x I need not be equal to x I - X 0, for example. Then the first divided of fo and I, is defined by

0000

1736

1736

-52

// --52 ///

- 1 04 /// - 176

//

1580 /// --228

///

5000 -> --332

//

//1248

6";48

<,

.................. -1412

7660 ........................... :::.412

1000 ........................ 149

...........

-263 ............ -172

............

-23 <, <, ...... ,

1684

3420

(I)

724

, the second divided difference of [«, fl and f2 is defined by

1- flX2,xd - f[xi ,xol

,XI,X2 - ,

X2 -Xo

(2)

496

164

-1072

-576

725

f[xn+1"",xll -flxn,··.,xol

Xn+l - Xo

8660

737

9397

-286

10

451

-13

II xl-~' Ii

Xo,x" ... , n - L ,

i=o (Xi - XO)(xi - XI)'" (xi - Xi-l )(Xi -Xi+l)'" (Xi -xn)

that all divided differences are symme_tric under interchange of any two of

. arguments.

9848

-299

152

10000

416

EXAMPLE 3

Determine the divided differences of (a) f(x) = 2x2, (b) l(x) = x3 at a typical ofx.

6fr+3/2 + of,- 1/2 = 20fr~ 1/2 + 03 fr+ 112'

~2n-If +£2n-1r =202n-1j' +02n+lf

u 3/2 u - 112 1/2 112 •

(a)

/

the data for f are known at specific points xo, x I, .... The problem of !fO(J,latilon is to obtain a useful estimate of the value of f at any intermediate point. this estimate is made on the basis of replacing (n + 1) data points by an nth·

polynomial function P(x). Normally we should require that the formulae

would at least predict the correct values for fat the given data points.

choices of these points lead to different forms for the collocation polynomial ilc(IflSA::QUently to different interpolation formulae. The procedure is represented

in Fig. 11.1, which shows the data as discrete points; X is a typical

iate point in (x 0, x d. P I (x), P 2 (x) and P3 (x) are three examples of possible arU.lUll"'"'''''' polynomials and each gives a different prediction (F I, F2 and F 3) for f The graph of PI (x) is a straight line, that of P2(x) is the parabola passing through o andfl, and that of P3(x) is the parabola passing throughfo,fl andf2'

The third and higher divided differences are obviously zero.

and again the higher divided differences are zero.

By induction and using the symmetry property of divided differences, it shown that, just as for nth differences, the nth divided differences of a degree n are constant and higher differences zero. Taken in conjunction with Weierstrass's Theorem, this result again implies that a table of divided be used to detect mistakes; however, it does not now give an easy method of those mistakes.

I
I
I
f I
II
18
Xo X XI X2 X Exercises

1. Derive difference tables, accurate to four decimal places, for the functions

(a) f(x) = 1/(1 +x),

(b) f(x) = 4x 3 - 3x + 1,

for the two cases x = 0(0.5)3 and x = 0(0.25)3. (Keep these for use later in the chapter.)

Fig. 11.1 Approximating polynomials passing through particular data points

2. Re-difference table 11.3 with the entry 6248 replaced with 6428 and confrrm there ar.e no other mistakes in the data.

3. ~onstruct a table of divided differences, accurate to three decimal places, for cubic

simplest method of interpolating for the value of f(x) at a typical point x in

< x < X I is to approximate the behaviour of f(x) against x by a straight line joining fo)to (XI,fI)so that between x., and x , (=xo +h):

f(x)~ fo + ([I - fo)(x - xo)/h

f(x) = 4x3 - 3x + 1

for the val ues x = 0, 1.0, 1.8, 2.4, 2.8, 3.0, and confirm that the fourth divided differences are zero.

= fo + uofl!2 = fo + uD.fo = fo + UVfl,

(4)

418

Finite

where, for convenience here and in subsequent formulae, we iiuroduce u=(x-xo)/h, O~u~l.

This formula (4) is often adequate if h is sufficiently small, but there is no indication here of the error in the approximation. The approximating polynomial obviously of degree 1; the theory for higher degree polynomials provides the error estimate.

/

Newton-Gregory Backward Difference Formula

This formula is obtained by constructing the interpolating polynomial P(x) to withf at the (n + 1) equally spaced points xo, x -I, X -2' ... ,x-n' Thus we

P(x) = ao + a I (x - xo) + a2 (x - xo)(x - X -I) +a3(x - xo)(x - x_d(x - X-2) + ...

+au(x - xo)(x - x-d (x - X-Il+1),

and choose the coefficien ts ao, a I, , an so tha t

P(xo)=fo, P(X-d=f-I, ... ,P(x-n)=f-n.

The reason for choosing such an apparently cumbersome form for P(x) is that it to a formula involving finite differences. Substituting x = x 0, X -I' ... ,x-n in equation (6) gives the simultaneous equations

fo = ao. f-I = ao - a ih, f-2 = ao - 2a1h + 2a2h2.

... ,[-11 = ao -- na I h + n(n - 1 )a2h2 + •.. + (-1 )nn!

Solving in turn for ao. a I •..• , an (a simple matter of forward substitution) then results in

«s=i«. alh=ao-f-I=fo-f-I= Vfo, ... , (J,nhn=Vnfo/n!.

Hence the interpolating polynomial (6) can be expressed very conveniently in backward differences:

1 2 1 ....

p(x) = P(xo + hu) = fo + uV fo + -u(u + I)V fo + -u(u + I )(u + 2)'\73 1'0·

2 3! JI

1

'" +,u(u + 1) ... (u +n -1)Vnfo. n.

In the same way we can construct the Newton-Gregory forward difference. by using values for fat Xo ,XI, X2, ... , Xn:

11' P(x) = P(xo + hu) = fo + ufj,fo + '2u(u - l)fj, 2 fo + 3! u(u - 1 )(u .,.- 2)fj,3 fo

1

'" +,u(u - 1) ... (u - n + 1)fj,nfo. n.

LE 1

differences in table 11.1 to compute the value off at x = 0.57 accurate to four places.

x = 0.57 we choose Xo = 0.55 so that, with h = 0.05, we have u = 0.4. Then formula (8) with the corresponding forward differences gives

[(0.57) = 1.81818 + (0.4)(-0.15151) + !(0.4)(-0.6XO.02330)

+ 1/6 (0.4)( -0.6)( -·1.6)(-0.00498)

+ f4 (0.4 X -0.6X-l.6X -2.6 XO.00131)

+ rh(0.4)(-0.6X -1.6)(-2.6X -3.6XO.00038)

= 1.81818 - 0.06060 - 0.00280 - 0.00032 - 0.00005 - 0.00001 = 1.7544. _

calculation, accuracy is maintained by retaining an extra decimal place until the is obtained, and by confirming that the fifth and higher differences cannot

the last decimal place.

's Interpolation Formula

. formula is the central difference equivalent to the Newton-Gregory formulae, based on the polynomial approximation:

p(x) = bo +bl(x - xo) +b2(x - xoXx - x.d

+ b3(x - xo)(x - xd(x - X-I) + b4(x - xoXx - xd(x - x_d(x - X2)

+ ...

many terms as required. Then choosing bo, bl, etc. so that P(x) now agrees with atxo,XI,x_l ,X2,X-2,." we have

. 1 2 1 3

P(x)=fo +UOfIl2 + iU(u - 1)0 fo + 3!u(u - l)(u + 1)0 fill + ...

(9)

_latlrlg after a finite number of terms. It should be emphasized here that the (7), (8) and (9) are not infinite series.

formula (9) is in fact not used in practice, but it is the basis of most of the formulae, such as Everett's, Bessel's and Stirling's.formulae.

formula involves only even-order central differences, which conveniently lie on lines. It is obtained by substituting in equation (9) for the odd-order Iferlenc,es (for example, 03 fl/2 = 02 fl - 02 fo) to give

P(x) = Eofo + E202 fo + E404fo + ... + Fofl + F'i02fl + F404fl + . . . (10)

where

82

SI ::: ~2U, S2 ::: ~U2, S3 lu(u2 - 1)/3!, S4::: U2(U2 - I )/4!,

~U(U2 - 1)(U2 - 22 )/5!, ... are the Stirling coefficients.

£0::: 1 _ .. U,

£2::: -u(l - u)(2 - u)/3!,

£4::: -(-I - u)u(1 - u)(2 -u)(3 - u)/5!,

Fo::: U,

F2 ::: U(U2 -. 1)/3!,

F4 :::U(U2 _1)(u2 --4)/5!, ..

of the Interpolation Formulae; Order of Error

, the error in using any of these formulae is of the order of the first term to tOIIIiW~O. Thus the Gauss formula (9) implies that linear interpolation carries an

of the order of ~u(u - 1)02[. In general it is easy to see from a difference table terms must be included in a particular formula to obtain a required

Because of the importance of this formula, the values of these Everett £0, £2, ... , F 0, F2 , ... may be found in convenient standard tables (see [4], for

examplej for Il es u K l . /

EXAMPLE 2

Use Everett's formula to determine from table 11.1 the value of f at x == 0.61, maintaining accuracy to four decimal places.

Taking Xo ::: 0.60 and h ::: 0.05 gives u ::: 0.2, so that

£0::: 0.8, £2::: -0.048, £4::: -0.008, Fo::: 0.2, F2::: -0.032, F4::: 0.006; f(0.61)::: 1.6393.

In this case the fourth differences do not affect ~ccuracy, so no higher di are required.

which involve central differences give the best accuracy fora given

of computation. They also have the philosophical merit of giving equal weight

tabulated data on each side of the relevant data point! Hence the Newtonformulae are used only where central differences cannot be ohtained, i.e., . the ends of the table as in example I.

formula is mainly used for deriving formulae for numerical integration and ; the practical interpolation formulae are those of Everett and Bessel. If

differences are negligible for the accuracy required, then Bessel's formula is

and hence better. The criterion for neglecting the 03 f term has already been If 03 f"> 60 units in the last decimal, then Everett's formula is best.

, we note that both £4 and F4 have maximum values of about 0.012, so that in using Everett's formula only up to second differences is of the order of

64 f.Thus we can neglect any fourth difference less than 20 units in the last

If the fourth difference is greater than 20 but less than 40.0 units (1000 units and rr. are the same sign), then the same second difference formula can be with negligible loss of accuracy provided the second differences are replaced by vmoaittea second differences

Bessel's Formula This is a form which is symmetrical about the midpoint u ::: ~. If in equation (9 substitute the easily derived relations

02fo == ~(02fo + 02fl - 03fI/2), 04fo::: ~(04fo + rt, - 05fIl2), etc.

and collect similar terms, we obtain Bessel's formula:

p(x) == fo + UOfll2 + B2(02fo + 02 fd + B303fl!2 + B4 (04fo + 04fl) + ... where B2 ::: ~u(u - I )/2! , B3 ::: u(u - ~)(u - 1)/3!, ... are the Bessel coefficients._ These coefficients, like Everett coefficients, may be found in standard tables (see -It should be noted that B 3 has a maximum value of about 0.008 for x 0 ".;; x ".;; X I> that the third and higher difference terms in equation (II) can be neglected if 03/ more than 60 units in the last decimal (that is, 60 x 10-111 where m is the number decimal places retained in f).

(13)

kable result is obtained by noting that £4/£2 varies very little over the 0"';; u ".;; I, with 0.15 ".;; -£4/£2 ".;; 0.20. The value 0.184 minimizes the error in the special case of constant fourth differences. When the 04fare not a direct evaluation of the error introduced will show that it is almost negligible.

Stirling's Formula

This formula IS symmetrical about Xo (or equivalently u == 0) and is derived by substituting in equation (9) the relations

Of1l2::: ~(OfIl2 +Of-1/2 +02fo), 03f1l2::: ~(03fI/2 + 03f_1I2 + 04fo), etc.

to four significant figures the value of f at x ::: 0.815 given that the table includes the following:

f 1.2500

of

x

giving

P(x) ::: fo + S I (Of-l/2. + Of1l2) + S202fo + 83(03 f-1/2 + 03f1l2) + S404 fo + ...•

-735

-16

0.85

1.1765

422

Finite

u(2) = 0.68 - ~(0.68)(-0.32) ~= 0 667

-735 . ,

4?'l

Since 03 f < 60, Bessel's formula should be used, with u = 03.-(hus f(0.815) = J .2500 + (0.3)(-0.0735) + ~(0.3X-0.7)( -0.0180)

= 1.2270.

iffer<encles satisfy the re1ation{

f= fo + (x - xo)f[x,x01, flx,x01 = f[xo,xj 1 + (x - xj)f[x,xo,Xj 1, f[x,xo,xt1 = f[xo,xl ,x21 + (x - x2)f[x,xo,xj,xd, .... substitution then immediately gives Newton's formula:

f(x)=fo +(x-xo)f[xo,xd +(x-xo)(x-xl)f[xo,xl>x21 + ....

(14)

EXAMPLE 4

Repeat example 2, using modified second differences.

The modified second differences are

I

polynomial functions, equation (14) gives exact interpolation values (within error, of course). For non-polynomial functions, the order of the error

on the higher divided differences, just as in the case of ordinary differences.

o;"fo : 2330 - 0.184 x 203 = 2293, 0;'/1: 1832-0.184x 131 = 1808.

Everett's modified formula to second difference table en tries then gives

f(0.61) = (0.8)(1.66667) - (0.048 )(0.02293) + (0.2)(1.53846) - (0.032 = 1.63935,

The accuracy would be given to five decimal placesYthe data were given to six instead of just five.

imetimes ~e need to determine the intermediate point x which corresponds to a valuef, say, for f(x). The usual procedure is to use an iterative method based on UlJl,dU'-'" formula. For example, the iterative solution for Bessel's formula (11) ,

differences is given by ,

u(n+I)=I-'to _~u(n)(u(n)_1)(02fo +02fl)

of (15)

,112 ' 8fl/2

u(n) denotes the nth approximation to the exact solution u. It should be borne here that the accuracy of the solution is already restricted by the fact that the

differences have been assumed negligible.

6

values given in example 3 above to determine x accurate to three decimal whenf(x)= 1.2.

the data are suitable for Bessel's interpolation formula, we use equation (I5) (0) = 0 to give

Extrapolation Sometimes it may be necessary to estimate a value for f at a point outside the the given data. This operation is termed extrapolation, and should be treated with' caution since it is easy to extrapolate functions into regions where they do not The basic procedure is to use the Newton-Gregory interpolation formulae, the backward difference formula to move forward beyond the table and the forward difference formula to move backward.

EXAMPLE 5

Use table ILl to extrapolate f to x = 0.45 and x = 0.95, to four decimal places.

For f(0.45) we have u = -1 and require the forward difference formula (8) x = 0.50.

f(0.45) = 2.00000 - (-0.18182) + 0.03031 - (-0.00701) + 0.00203 - (-0.00072) + 0.00034

= 2.22222'

(I) _ 1.2 - 1..25 _

u - -0.0735 - 0.680,

u(3) = 0.680 - ~(0;667)(-0.333) ~= 0 666

-735 . ,

U(4) = 0.680 - ~(0.666)(-0.334)~= 0 666

-735 . .

Here we have needed to compute the fifth and sixth differences in order to accuracy. This is also necessary for f(0.9 5) since again I u I = 1, resulting in nondecreasing coefficients. The result is f(0.9 5) = 1.0526.

x = 0.8 + 0.666 x 0.05 = 0.833.

Newton's Divided Difference Formula

For data given with non-constant step-length, the interpolation formula must in terms of divided differences. From their definitions (1), (2) and (3), divided

the d~ta given in table 11.1, (a) computef(0.88) to four decimal accuracy ) deterrmne the value of x for which f equals 1.28.

424

2. Working with the appropriate difference tables derived in exercise 1 (a) of § 11.2, determinef(I.12) when the step-length is (i) h == 0.5 and (ii) h == 0.25. Give reasons your choice of formula, and compare the results with the exact solution. Also determine the value ofx for which f equals 0.35.

11.4 NUMERICAL INTEGRA nON

In principle the numerical problem of computing a definite integral is similar to of interpolation in that the data are replaced by a suitable approximating function P(x) and the value of the integral determined by directly integrating P(x):

I: f(x) dx ~ f: P(~) dx.

Usually P(x) is chosen to be a collocation polynomial, and again different choices data points give different integration formulae. Normally the data points are equidistant, and this will be assumed in deriving the more commonly used U""l',ldl~ formulae. The forms chosen for P(x) then give error estimates in terms of finite differences. If the data points are not equidistant, the errors can be estimated in of divided differences.

Graphically the various approximations are equivalent to the areas ABCD under various curves, as represented in Fig. 11.1.

Trapezoidal Rule

This is the simplest integration formula, obtained by assuming CD to be a straight then the area ABCD is simply y,.l!(fo + fl)' The error involved in this may be deduced by integrating Bessel's formula (11) to second differences:

JX1 f(x)dx""fx, P(X)dX==hfl P{x(u)}du

x 0 x 0 0

= h J: (fo + u8f1l2 + !u(u - 1 )(82 fo + 02fdJ du ==h[fo + Hfl12 - n(82fo + 82fZ)1

== ~ h(fo + fz) - i4h(02 I« + 02 fz).

Hence the error involved in using the trapezoidal rule over the single interval is approximately i4h(02 [« + 02 I, ). By adding the integrals over successive ranges (x 1, X2), (X2, X3), ... of width h, we obtain

fxn

f(x) dx ~ ~ h(fo + 2fl + 2f2 + ... + 2fn - 1 + fn)

Xo

-i4h(82fo +202fl +28212 + ... +282fn_l +82fn).

- _t: .-.~. " .... r .... nnnns; to the trapezoidal rule over the entire range;

group gives an approx.s.ation to the error involved in using this rule, and is of

02 f represents the average value of th-e relevant second differences.

. . . JO.7 I

the values given In table 11.1, determine - dx using the trapezoidal rule.

0.5 x

With h = 0.05, table 11.4 shows 82.r~ 0.02 so that the trapezoidal rule gives an of order (0 .2)(0.02)/12 = 0.0003. Hence the answer can be accurate only to decimal places; because of the factor Yzh, this implies that only three decimal accuracy is necessary for the data. Using equation (17) we. obtain

JO•7 1 0.05 -

:; dx "" -2-{2.000 + 2(1.818 + 1.667 + 1.538) + 1.429} == 0.3379, 0.5

the suffix 9 implies rounding off of the four decimal result 0.3369.

integrating Stirling's formula (12) to fourth differences between X-I and XI we

J:~I f(x) dx "" h f ~ 1 P{x(u)} du

== h J ~ I [(0 + !U(8f-1/2 + 8f1/2) + !u2 02 fo

+ n u(u2 - 1 )(83 f-1I2 + 83 f1l2) + -n u2 (u2 - 1 )84 fo] du =2h(Jo +!02f-m04fo)

=~h(f_1 +4fo +fd-foh04fo.

, for the typical interval (X2n x2r+2),

fX2r+2

f(x)dx ~ ~h(hr + 4hr+1 + hr+2) -:- ifah84hr+ 1

x2r

that the definite integral over the entire range (xo, X2n) is

fX2n f(x)dx "'" ~h{(fo + hn) + 4(f1 + f3 + ... + hn- d + 2(f2 + ... + hn-2)}

Xo

(18)

first three groups of terms give Simpson 'I; rule, geometrically equivalent to

g f(x) between Xo and X2n by parabolic segments joining points (X2r, hr)

Finite

426

d ( f) for r = 0 1 n - 1 The last group of terms estimates the

an x2r+2' 2r+2 , , , ••• ,' 4 hi 1 .

in Simpson's rule and, being of order m(X2 n - xo)6 f, shows why t IS ru e gives

such an accurate numerical value for an integral.

EXAMPLE 2 "

, JO.7 1

Use Simpson's rule to determine - dx with h = 0.05 and h = O.l .

0.5 x

Withh = 0.05, table 11.1 shows that 04/""" 0.002. Thus Simpson'.s rule \18) error of order rh (0.2)(0.002) """ 0.000002, implying accuracy to five decimal So, compared with the trapezoidal rule used in examp~e 1, we noy ge~ tw~ extra decimal places for the additional work of only tabulating, or even estimating, the and fourth differences. We thus find

fO.7 1

- dx """ hO.05){2.00000 + 1.42857 + 4(1.81818 + 1.53846) + 2(1

0.5 X .

= 0.336491•

With h = 0.1 , the new difference table 11.4 for l/x shows that the error term is order (0.2)(0.02)/180:::: 0.00002 implying accuracy still to four decimal places. Then

fo. 7 1

- dx "'" ~(0.1){2.00000 + 4(1.66667) + 1.42857} 0.5 x

= 0.3365,.

TABLE 11.4

x

/

0/

0.50 2.00000
-33333
0.60 1.66667 9523
-23810 -3570
0.70 1.42857 5953 1585
-17857 -1985
0.80 l.25000 3968
-13889
0.90 1.11111
Other Formulae Based on Finite Differences Integrating Stirling's formula (I 2) to fourth differences over three intervals gives

i'J/ul'11prICal Integration \

.:»

'and shows that the error is of order (3h/80)04/1' This is therefore somewhat less te than Simpson's rule, but useful if we are forced to integrate over an odd of intervals. '

A higher order of accuracy, but more complicated computation, is given by, (D1I,!;I51""U'5 Stirling's formula to sixth differences over four intervals, when

(X, 21z

Jx _/(X)dX "'" 4"5 (7f-2 + 32/-1 + 12/0 + 32/1 + 712) ,

an error of order (8h/945)66/0; or, if we integrate over six intervals:

(20)

J:: f(x)dx ,"'" h(6/0 + 90210 + tt04/0 + No 0 6/0) + 0(08/0) replacing 41/140 by 42/140 = 3/10 gives Weddle's nile

f x, 3h

I(x)dx "'" 10 (/-3 + 5/-2 +1-1 + 6/0 + 11 + 5/2 + 13)

X -3

(21 )

an error of order mh 06fo.

3

JO•8 I

Weddle's rule to determine - dx.

0.5 X

Applying equation (21) gives

fo.8 1 3(.05)

- dx "'" --{2.00000 + 5(1.81818) + 1.66667 + 6(1.53846) + 1.42857

0.5 x 10

+ 5(1.33333) + 1.25000}

= 0.4700036.

truncation error rtoh 0610 is of order 0.00000005, but this is now swamped 'the round-off error implicit in the number of decimal places given in table 11.1.

and Use 01 These Formulae

trapezoidal and Simpson's rules and those given above are but a sample of the formulae available for numerical integration (see Bibliography). Of these, the important is undoubtedly Simpson's rule. Its accuracy and simple form suggest

it should be used in most cases. If we are 'not free to choose the step-length h (such the data are given at specific points) and if fourth-order differences are not ~~';llJ"" then the more accurate formulae (20) and (21) should be used whenever

, For automatic computation, the simple form of the trapezoidal rule is often Dn't:1l1t:HI, particularly when variable step-lengths are required. The step-lengths are

428

Finite

successively reduced until an acceptable accuracy has been achievt:d or until no intermediate data (for example, at mid-points) are available. Successive halving step-length is a particularly effective method of achieving accuracy with any of integration formulae, since the difference between successive results for the ,n •. ~_." can be used to control the error. Further detals of such refinements may be Noble [3] , Chapter 9.

For hand computation it is good practice to construct a difference table if only safeguard against mistakes in the data. In these circumstances it is sensible to treat error terms in the integration formulae as corrections, and add them to the rule in order to improve the accuracy of the numerical result. Thus, in example 1 can obtain a solution very close to four decimal accuracy by incorporating the 0.0003 to give the improved result 0.3366. TIle accuracy could be even further improved by using modified differences (see equation (I 3)).

f(x) == t]x, equation (23y{hows that for h == 0.15,

fO.8 I 0.15 { 1 1 1

-dx"""- 5 +8-+5 }

o.s x 9 0.65 - 0.15(0.7746) 0.65 0.65 + O.l 5(0.7746)

== 0.4700036.

of the integration formulae can be used directly in cases when (a)f(x) is infinite point in the range of integration (although the integral itself need not be

or (b) when the range itself is infinite,

case (a) the difficulty often can be overcome analytically by either integrating by or subtracting out the singularity. Thus, for example

Gaussian Formulae

When an automatic computer is used, many of the advantages of integration based on finite differences disappear. If the data are given in the form of a COIlltiDlIIII function of x, then much more efficient formulae can be obtained by choosing suitable values for x which are not necessarily data points. Such formulae are Gaussian.

Consider a very simple example:

1: eXx-1/2 dx == f: x-1I2 dx + f: (eX _1)X-1I2 dx == 2 + Ll (eX - l)x-1I2 dx the new integral presents no difficulty, as

(eX _ I)X-1I2 -70 as x -7 O.

case (b) the way out is often by means of an appropriate substitution to render fmite, or by cutting off the range at a finite limit and analytically bounding ~resiulting truncation error. Hence, for example, substituting x == tan (J gives

fooe-Xdx J1'/2

--2 == e-tan e d(J,

1 I + x 1'/4

is easily computed. Truncation in another example gives

r exp(-x2)dx == tX exp(-x2)dx +E(X), X>O,

a bound for the error E(X) can be found as

IXI

X f(x)dx """ Af(xo - exit) + Bf(xo) + Af(xo + exit)

. -I

where A, B and a are constants. Simpson's rule (18) is the special case a == 1, A B == 4h/3. Now we choose A, B and a to give the smallest error possible.

By expanding both sides of equation (22) in Taylor series, it can be shown that both sides agree up to terms in h 6 if we set

2A +B==2h, A~2h2 ==h3/3, Aa4h4 ==hsI5.

Thus

JX1 h

f(x)dx == - [5f(xo - exit) + 8fo + 5f(xo + exit)} + O(h 7)

X 9

-I

where a2 == 3/5, and the coefficient of the error term is very small ([(Vi) /15750).

More general Gaussian formulae can be constructed, but these are outside the of this chapter. Normally the coefficients of the function values are irrational, not : rational as in this simple case.

E(X)==J"" exp(-X2)dx<J"" ~exp(_x2)dx=_1 exp(-X2).

X x X 2X

X can be chosen to give an acceptable order of accuracy.

EXAMPLE 4

fO.8 I

Use the Gaussian formula (23) to compute - dx.

o.s x

with the appropriate difference tables derived in exercise I (b) of § 11.2,

430

(28)

Differentiation

431

determine j:[(x)dx using Simpson's rule with (i)h = 0.5 and ~ii) h = 0.25. Check results against the exact analytical solution. Why do all three answers agree this case?

d approximate _- (ex)

dx

/ 2. Prove the formulae (19), (20), (21) and (23).

1.0000 ---------- _

1.6487 ------------- _

::~:~~ ~=======l-----l l

2.7 2.72 2.723 2.8430

3. Derive the integration formula relevant to tabulated divided differences:

JX.

[(x)dx = (XI - xo)fo + !ex 1- XO)2[[XO' xd

Xo

- teXI - xo)3[[XO' Xl> x21 + ....

3.1946

4. Take a set of values of a standard function (for example, sin x) tabulated in a of tables, and use Simpson's rule and the various other formulae to integrate it numerically over some interval. Check your answers against the analytical solution.

2.7183

'2.72io_J=0.001 hJ=O.OI hJ=o.l 'Jl=O.5

2.7456 -------- .

3.0042 ---------- _

4.4817 ---------- _

7.3891 -------------- _

h = 1

11.5 NUMERICAL DIFFERENTIATION

Obtaining a reasonable estimate of the derivative of a function at a given point is numerically a much more 'difficult' problem than that of estimating its integral.' is because the process is inherently prone to inaccuracy, as is easily seen by conSideQi the definition of the derivative of f(x):

d2[ 1 d2P 1 [

/"=. dx2 "" h2 du2 = h2 02[0 + !U(03f-1I2 + 03[112) + n(6u2 -I)04fo]. (26)

, si~ilar f?rmulae may be obtained for higher derivatives by taking Stirling's

to higher differences, These differences may also be incorporated to improve UIi"Ul·a"y. ~ effect, formulae such as equations (25) and (26) are a combination of and interpolates, Normally we require the values of derivatives at the data These are given by putting u = 0 in the formulae to give, for example,

.' 1

',. f~= 2h [«5[-112 +0[112 -!W[-1I2 + 03[1i2) + n, (0 5f_1I2 +05[112) - ... ), (27)

1

f~' = h2 [02[0 - 004[0 + ifo06[0 - •.• 1.

I _ d[ . [(x + h) - [(X - h)

[=-=lim .

dx /)->0 2h

In principle we should get a progressively more accurate approximation for [' by taking successively smaller values of h. In practice this leads to computing the two very small quantities, which is numerically bad; this is demonstrated in where the single numbers on the right (2.7,2.72, ... ) denote various ap!Jroxinlatioli for d(eX)jdx at X = I. The exact solution is, of course, e = 2.71828 ... and the shows that with eX given accurately to four decimal places, the definition (24) improving approximation as h decreases from 1 to 0.01, but then the values moreinaccurate again.

Just as practical integration formulae are obtained by integrating the polynomials, practical formulae for differentiation are obtained by interpolation formulae based on the approximating polynomials P(x). Again, formula is usually the best suited of these formulae. From Stirling's formula (12)' fourth differences we have

the formulae gi:e not only a numerical approximation for the derivatives, but order of maglli~ude. for the t~uncation errors involved. Thus equation (27)

that the approXlITIatlOn for.fo based on equation (24) carries with it an error of Jh -103[. Similarly equation (28) suggests use of the approximation

," ~ [I - 2[0 + [-I

10 - h2

I d[ dP I dP

[ =.-=-=--

dx dx h du

since

x -Xo u=---

h '

= ~[~(0[-1/2 + 0[1/2) +u02[0 + f1(3u2 - 1)(03[_1/2 +03[112) + nu(2u2 - 1)04[0] ,

an error of order n h -204 [0.

formulae. also demonstrate th e close relation between the derivatives and th .

",ppro1xirna1:iorls . elf

li ill ~erms of differences. Thus the nth-order derivative involves

the nth-order difference by the nth power of h, and equivalently we deduce

x 1.010

[(x) -1.08286

x 1.040

[(x) +0.14333

1.020 -0.66998

1.030 -0.26158

.. '

\

.. ' Neglecting fourth difference~, evaluate the following numbers:

that

[(3.25), (b) [(3.02), (c) 1'(3.2), (d)
[(x) is given by the following table:
3.1 3.2 3.3 3.4 3.5
22.198 24.533 27.113 29.964 33.115. J3.4

[(x) dx, 3.0

432

Since h must be small, these relations show that whereas numerical integration gives solutions of greater accuracy than that of the data, numerical differentiation

gives less.

EXAMPLE

Using the values of table 11.1, evaluate to three decimal places the derivative of 1

x = 0.65.

From equation (27) and table 11.1 we see that neglecting the 53 f term would introduce an unacceptable error of order (0.005)/(0.3) affecting the second place. However, the 5 sf term is of. order (0.0005)/(1.5) and would not affect the req uired accuracy. Hence, keeping the fifth decimal place in the data in order to to four decimal places with round-off to three, we calculate

. the acc~racy of each of your results. Determine to three decimal places the

of x for which [(x) = 25.0.

and correct the transposition mistake in the following table of values of

o

6 12 18 24 30 36 42

3.2361 3.4203 3.6820 3.8637 4.1336 4.4454 48

. 097.

f~ "" _!_ [-0.12821 - 0.1 0989 - H -0.00498 - 0.00367)] 0.1

= -2.367,

J36.

~:wnpi)Ull'S rule on the corrected values to determine [(x) dx, and show that

. 12

involved is about 0.0002. Also determine [(25) and find the value of x f

fix) = 4.0. ' or

which agrees with the exact value.

the Ne,,:,ton-Greg~ry forward difference formula to third differences to derive for interpolation to halves

Exercises

1. Working with the appropriate difference tables derived in exercise 1 (a) in § 1 determine df/dx at x = 1.12 when the step-length is (i) h = 0.5 and (ii) h = 0.25_ Compare your results with the exact values for f(x) = 1/(1 + x).

3. Write down the formula for f; equivalent to equation (27). Then use the of exercise 4 in § 11.2 to deduce the formula

~h(f~ + f;) = 5f1l2 + 1253 f1/2 - m5s f1/2 + ....

is. an ex.tract fro~ a table of a function g(ex, I/J) of two variables ex and I/J.
to obtain by interpolation an approximate value of g(25 12iAo) G· . h
. f. ' 2. ive, Wit
an estimate 0 the magnitude of the error in this calculation.
0° ¢
5° 10° 15° 20° 25°
0 873 1745 2618 3491 4363
0 873 1745 2617 3489 4359
0 873 1744 2615 3483 4348
0 872 1743 2611 3473 4330
0 872 1742 2606 3462 4308
0 872 1740 2601 3450 4284 2. Derive the mid-point formula

hf;/2 "" 5f1/2 - 1453 f1l2 + do5s f1/2 - ....

PROBLEMS

1. The following table contains a mistake in one of the values of [(x). Locate mistake and estimate the correct value, Using Bessel's interpolation formula to differences, determine to six decimal places the value of x for which [(x) = o.

1.015 -0.87590

1.025 -0.46511

B. R., Numerical Approximation, Routledge and Kegan Paul, London C.-E., Introduction to Numerical Analysis Addison Wesley Readin

(1965). " g,

B., Numerical Methods: 2, Oliver and Boyd, Edinburgh (1964).

tnrerrurtat and Allied Tables, H.M.S.O., London (1956).

1.045 1.050

+034413 +0.54393.

Introduction to

~~~- ------~-~-

?-30

quadratic equation

x2 - 4x - 1=0

giving your answer to three decimal accuracy.

ear Algebra - Numerical thods

3. Use Taylor's theorem to obtain an approximation for [(x) = eX sin x near x the form of a cubic in x and use the remainder term to determine the range of x over which the approximation is valid if the error is not to exceed ± Hence obtain approximate values of f(0.2) and f( -0.3), and compare your with those obtained using tables.

4. Usc two different iterative methods, one involving the use of a derivative other not, to determine to five significant figures the smallest non-zero ,",v:,u,.-e,rno X - 2 sin 2x = O.

Lbr;anches of engineering there are practical problems which reduce to the

of sets of linear algebraic equations. It is not uncommon for there to be

or even thousands of such equations although often the coefficient matrices that is, contain a large number of zeros. From a theoretical point of view of a system of n equations in n unknowns may be written down using ;;;"";n~nt~ (Cramer's rule, as described in §8.4). Though the use of determinants convenient for very small systems of, say, three or four equations, it was

out in Chapter 8 that this method proves quite impracticable for much higher 'l'SlirStem~. This is because the number of arithmetic operations needed to evaluate determinant directly is of the order of n! and this increases very rapidly

Even evaluating the determinants in the best manner possible requires a number ~atil)DS proportional to n4. The direct methods for the solution of linear described in this chapter reduce the computation to a number of operations to n3• Hence it is far more efficient numerically to use one of the direct

methods described in § 10.2 and § 10.3. All the methods of this chapter for large systems although it is only practical here to illustrate them by of orders 2, 3 and 4.

methods are generally economical on computing time as the solution is in a fixed number of arithmetic operations, but they require more

.. ""mllw time and more computer store than iterative methods. On the other

the iterative methods which are much easier to program, the computing time IIIeIIUvJIlL upon the accuracy required of the solution and the method may be very 1'Q-1Il0."ULLUf> for a slowly convergent system. Most computer systems which are

for scientific work have readily available standard library subroutines. These always include programs for solving linear algebraic equations which are based the direct methods of § 10.2.

chapter is concerned with the system of n equations in n unknowns

5. Let (X, n denote the exact solution of two simultaneous equationsf(x,y) g(x, y) = O. By expanding [(X + h, Y + k) and g(X + h , Y + k) about (X, Y), and neglecting terms involving h2 , hk, e and higher powers of hand k, deduce iterative method of solution:

where

af ag

g--[-

ay ay

H(x, y) = a[ ag a[ ag ,

ax ay ayax

af ag

-g- +t t:

ax ax

K (x , y) = a[ ag a[ ag' ax ay ay ax

Hence solve the simultaneous equations

x- sin(y/x) = 0, y=cosy+sinx, giving your answer to three decimal accuracy.

6. By sketching the two curves 3x2 + y2 = 9, y + 2 sin x = 2, show that they points of intersection for one of which x is positive and for the other x is iteration schemes, of the form

xn+1 =F(x.1,Yn), Yn+1 =G(Xn+1,Yn)

to solve the equations to three decimal accuracy.

BIBLIOGRAPHY

[I J Froberg, C.-E., Introduction to Numerical Analysis, Addison-Wesley,

Mass. (1965).

[2J Noble, B., Numerical Methods: I, Oliver and Boyd, Edinburgh (1964). [3 J Noble, B., Numerical Methods: 2, Oliver and Boyd, Edinburgh (1964). [4 J Williams, P. W., Numerical Computation, Nelson, London (1972).