24 views

Uploaded by glopat

- ST7201-Finite Element Method
- Practical Implementation Of Matlab Based Approach For Face Detection Using Feedforward Network
- Previous Gate by Kanodia Practise Questions All Subjects.pdf
- Design Considerations for Environmental Sustainability in High Density Development a Case Study of Hong Kong
- Green Acre 2007 Mc a Col
- Teknik Menjawab Matematik Spm
- Quick Linear Algebra for Econometrics
- Tutorial 2
- Introduction to matrices-Sec1block3
- SubDyn Manual
- Yearly Lesson Plan Mathematics Form5 2017
- wikimama Class 12 Ch 4 Determinants
- Simulation of Transients in Unerground Cables With Frequency Dependent Modal
- hw6_2011_05_16_01_solutions
- EC 400 Quizzes for Class 2
- NotaFizikMathUptoCh3
- Teoria de Grupos
- Determinan 4x4
- مرجع 9
- Chapter 6 Matrix Calculations

You are on page 1of 16

C

Matrix Algebra:

Determinants, Inverses,

Eigenvalues

C1

Appendix C: MATRIX ALGEBRA: DETERMINANTS, INVERSES, EIGENVALUES C2

TABLE OF CONTENTS

Page

C.1. Determinants C3

C.1.1. Some Properties of Determinants . . . . . . . . . . . C3

C.1.2. Cramers Rule . . . . . . . . . . . . . . . . C5

C.1.3. Homogeneous Systems . . . . . . . . . . . . . . C6

C.2. Singular Matrices, Rank C6

C.2.1. Rank Deciency . . . . . . . . . . . . . . . . C7

C.2.2. Rank of Matrix Sums and Products . . . . . . . . . . C7

C.2.3. Singular Systems: Particular and Homegeneous Solutions . . C8

C.2.4. Rank of Rectangular Matrices . . . . . . . . . . . . C8

C.3. Matrix Inversion C8

C.3.1. Explicit Computation of Inverses . . . . . . . . . . C9

C.3.2. Some Properties of the Inverse . . . . . . . . . . . . C10

C.4. Eigenvalues and Eigenvectors C11

C.4.1. Real Symmetric Matrices . . . . . . . . . . . . . C12

C.4.2. Positivity . . . . . . . . . . . . . . . . . . . C12

C.4.3. Normal and Orthogonal Matrices . . . . . . . . . . C13

C.5. The Inverse of a Sum of Matrices C13

C.6. The Sherman-Morrison and Related Formulas C13

C.6.1. The Sherman-Morrison Formula . . . . . . . . . . . C14

C.6.2. The Woodbury Formula . . . . . . . . . . . . . C14

C.6.3. Formulas for Modied Determinants . . . . . . . . . . C14

C. Notes and Bibliography . . . . . . . . . . . . . . . . . . . . . . C14

C. Exercises . . . . . . . . . . . . . . . . . . . . . . C15

C2

C3 C.1 DETERMINANTS

This Chapter discusses more specialized properties of matrices, such as determinants, eigenvalues

and rank. These apply only to square matrices unless extension to rectangular matrices is explicitly

stated.

C.1. Determinants

The determinant of a square matrix A = [a

i j

] is a number denoted by |A| or det(A), through which

important properties such as singularity can be briey characterized. This number is dened as the

following function of the matrix elements:

|A| = det(A) =

a

1 j

1

a

2 j

2

. . . a

n j

n

, (C.1)

where the column indices j

1

, j

2

, . . . j

n

are taken from the set {1, 2, . . . n}, with no repetitions

allowed. The plus (minus) sign is taken if the permutation ( j

1

j

2

. . . j

n

) is even (odd).

Example C.1. For a 2 2 matrix,

a

11

a

12

a

21

a

22

= a

11

a

22

a

12

a

21

. (C.2)

Example C.2. For a 3 3 matrix,

a

11

a

12

a

13

a

21

a

22

a

23

a

31

a

32

a

33

= a

11

a

22

a

33

+a

12

a

23

a

31

+a

13

a

21

a

32

a

13

a

22

a

31

a

12

a

21

a

33

a

11

a

23

a

32

. (C.3)

Remark C.1. The concept of determinant is not applicable to rectangular matrices or to vectors. Thus the

notation |x| for a vector x can be reserved for its magnitude (as in Appendix A) without risk of confusion.

Remark C.2. Inasmuch as the product (C.1) contains n! terms, the calculation of |A| from the denition is

impractical for general matrices whose order exceeds 3 or 4. For example, if n = 10, the product (C.1) contains

10! = 3, 628, 800 terms each involving 9 multiplications, so over 30 million oating-point operations would be

required to evaluate |A| according to that denition. A more practical method based on matrix decomposition

is described in Remark C.3.

C.1.1. Some Properties of Determinants

Some useful rules associated with the calculus of determinants are listed next.

I. Rows and columns can be interchanged without affecting the value of a determinant. That is

|A| = |A

T

|. (C.4)

II. If two rows, or two columns, are interchanged the sign of the determinant is changed. For

example:

3 4

1 2

1 2

3 4

. (C.5)

C3

Appendix C: MATRIX ALGEBRA: DETERMINANTS, INVERSES, EIGENVALUES C4

III. If a row(or column) is changedbyaddingtoor subtractingfromits elements the corresponding

elements of any other row (or column) the determinant remains unaltered. For example:

3 4

1 2

3 +1 4 2

1 2

4 2

1 2

= 10. (C.6)

IV. If the elements in any row (or column) have a common factor then the determinant equals

the determinant of the corresponding matrix in which = 1, multiplied by . For example:

6 8

1 2

= 2

3 4

1 2

V. When at least one row (or column) of a matrix is a linear combination of the other rows (or

columns) the determinant is zero. Conversely, if the determinant is zero, then at least one

row and one column are linearly dependent on the other rows and columns, respectively. For

example, consider

3 2 1

1 2 1

2 1 3

. (C.8)

This determinant is zero because the rst column is a linear combination of the second and

third columns:

column 1 = column 2 +column 3 (C.9)

Similarly, there is a linear dependence between the rows which is given by the relation

row 1 =

7

8

row 2 +

4

5

row 3 (C.10)

VI. The determinant of an upper triangular or lower triangular matrix is the product of the main

diagonal entries. For example,

3 2 1

0 2 1

0 0 4

= 3 2 4 = 24. (C.11)

This rule is easily veried from the denition (C.1) because all terms vanish except j

1

= 1,

j

2

= 2, . . . j

n

= n, which is the product of the main diagonal entries. Diagonal matrices are

a particular case of this rule.

VII. The determinant of the product of two square matrices is the product of the individual deter-

minants:

|AB| = |A| |B|. (C.12)

This rule can be generalized to any number of factors. One immediate application is to matrix

powers: |A

2

| = |A||A| = |A|

2

, and more generally |A

n

| = |A|

n

for integer n.

VIII. The determinant of the transpose of a matrix is the same as that of the original matrix:

|A

T

| = |A|. (C.13)

This rule can be directly veried from the denition of determinant, and also as direct

consequence of Rule I.

C4

C5 C.1 DETERMINANTS

Remark C.3. Rules VI and VII are the key to the practical evaluation of determinants. Any square nonsingular

matrix A (where the qualier nonsingular is explained in C.3) can be decomposed as the product of two

triangular factors

A = LU, (C.14)

where Lis unit lower triangular and Uis upper triangular. This is called a LUtriangularization, LUfactorization

or LU decomposition. It can be carried out in O(n

3

) oating point operations. According to rule VII:

|A| = |L| |U|. (C.15)

But according to rule VI, |L| = 1 and |U| = u

11

u

22

. . . u

nn

. The last operation requires only O(n) operations.

Thus the evaluation of |A| is dominated by the effort involved in computing the factorization (C.14). For

n = 10, that effort is approximately 10

3

= 1000 oating-point operations, compared to approximately 310

7

from the naive application of (C.1), as noted in Remark C.1. Thus the LU-based method is roughly 30, 000

times faster for that modest matrix order, and the ratio increases exponentially for large n.

C.1.2. Cramers Rule

Cramers rule provides a recipe for solving linear algebraic equations in terms of determinants. Let

the simultaneous equations be as usual denoted as

Ax = y, (C.16)

where A is a given n n matrix, y is a given n 1 vector, and x is the n 1 vector of unknowns.

The explicit form of (C.16) is Equation (A.1) of Appendix A, with n = m.

The explicit solution for the components x

1

, x

2

. . ., x

n

of x in terms of determinants is

x

1

=

y

1

a

12

a

13

. . . a

1n

y

2

a

22

a

23

. . . a

2n

.

.

.

.

.

.

.

.

.

.

.

.

y

n

a

n2

a

n3

. . . a

nn

|A|

, x

2

=

a

11

y

1

a

13

. . . a

1n

a

21

y

2

a

23

. . . a

2n

.

.

.

.

.

.

.

.

.

.

.

.

a

n1

y

n

a

n3

. . . a

nn

|A|

, . . . (C.17)

The rule can be remembered as follows: in the numerator of the quotient for x

j

, replace the j

t h

column of A by the right-hand side y.

This method of solving simultaneous equations is known as Cramers rule. Because the explicit

computation of determinants is impractical for n > 3 as explained in Remark C.3, this rule has

practical value only for n = 2 and n = 3 (it is marginal for n = 4). But such small-order systems

arise often in nite element calculations at the Gauss point level; consequently implementors should

be aware of this rule for such applications.

Example C.3. Solve the 3 3 linear system

_

5 2 1

3 2 0

1 0 2

__

x

1

x

2

x

3

_

=

_

8

5

3

_

, (C.18)

C5

Appendix C: MATRIX ALGEBRA: DETERMINANTS, INVERSES, EIGENVALUES C6

by Cramers rule:

x

1

=

8 2 1

5 2 0

3 0 2

5 2 1

3 2 0

1 0 2

=

6

6

= 1, x

2

=

5 8 1

3 5 0

1 3 2

5 2 1

3 2 0

1 0 2

=

6

6

= 1, x

3

=

5 2 8

3 2 5

1 0 3

5 2 1

3 2 0

1 0 2

=

6

6

= 1. (C.19)

Example C.4. Solve the 2 2 linear algebraic system

_

2 +

1 +

_ _

x

1

x

2

_

=

_

5

0

_

(C.20)

by Cramers rule:

x

1

=

5

0 1 +

2 +

1 +

=

5 +5

2 +3

, x

2

=

2 + 5

0

2 +

1 +

=

5

2 +3

. (C.21)

Remark C.4. Creamers rule importance has grown in symbolic computations carried out by computer algebra

systems. This happens when the entries of Aand y are algebraic expressions. For example the example system

(C.20). In such cases Cramers rule may be competitive with factorization methods for up to moderate matrix

orders, for example n 20. The reason is that determinantal products may be simplied on the y.

C.1.3. Homogeneous Systems

One immediate consequence of Cramers rule is what happens if

y

1

= y

2

= . . . = y

n

= 0. (C.22)

The linear equation systems with a null right hand side

Ax = 0, (C.23)

is called a homogeneous system. From the rule (C.17) we see that if |A| is nonzero, all solution

components are zero, and consequently the only possible solution is the trivial one x = 0. The case

in which |A| vanishes is discussed in the next section.

C6

C7 C.2 SINGULAR MATRICES, RANK

C.2. Singular Matrices, Rank

If the determinant |A| of a nn square matrix A A

n

is zero, then the matrix is said to be singular.

This means that at least one row and one column are linearly dependent on the others. If this row

and column are removed, we are left with another matrix, say A

n1

, to which we can apply the

same criterion. If the determinant |A

n1

| is zero, we can remove another row and column from it

to get A

n2

, and so on. Suppose that we eventually arrive at an r r matrix A

r

whose determinant

is nonzero. Then matrix A is said to have rank r, and we write rank(A) = r.

If the determinant of A is nonzero, then A is said to be nonsingular. The rank of a nonsingular

n n matrix is equal to n.

Obviously the rank of A

T

is the same as that of A since it is only necessary to transpose row and

column in the denition.

The notion of rank can be extended to rectangular matrices as outlined in section C.2.4 below.

That extension, however, is not important for the material covered here.

Example C.5. The 3 3 matrix

A =

_

3 2 2

1 2 1

2 1 3

_

, (C.24)

has rank r = 3 because |A| = 3 = 0.

Example C.6. The matrix

A =

_

3 2 1

1 2 1

2 1 3

_

, (C.25)

already used as an example in C.1.1 is singular because its rst row and column may be expressed as linear

combinations of the others through the relations (C.9) and (C.10). Removing the rst row and column we are

left with a 2 2 matrix whose determinant is 2 3 (1) (1) = 5 = 0. Consequently (C.25) has rank

r = 2.

C.2.1. Rank Deciency

If the square matrix A is supposed to be of rank r but in fact has a smaller rank r < r, the matrix

is said to be rank decient. The number r r > 0 is called the rank deciency.

Example C.7. Suppose that the unconstrained master stiffness matrix K of a nite element has order n, and

that the element possesses b independent rigid body modes. Then the expected rank of K is r = n b. If the

actual rank is less than r, the nite element model is said to be rank-decient. This is usually undesirable.

Example C.8. An an illustration of the foregoing rule, consider the two-node, 4-DOF, Bernoulli-Euler plane

beam element stiffness derived in Chapter 12:

K =

EI

L

3

_

_

_

12 6L 12 6L

4L

2

6L 2L

2

12 6L

symm 4L

2

_

_

, (C.26)

in which EI and L are nonzero scalars. It may be veried that this 4 4 matrix has rank 2. The number of

rigid body modes is 2, and the expected rank is r = 4 2 = 2. Consequently this model is rank sufcient.

C7

Appendix C: MATRIX ALGEBRA: DETERMINANTS, INVERSES, EIGENVALUES C8

C.2.2. Rank of Matrix Sums and Products

In nite element analysis matrices are often built through sum and product combinations of simpler

matrices. Two important rules apply to rank propagation through those combinations.

The rank of the product of two square matrices A and B cannot exceed the smallest rank of the

multiplicand matrices. That is, if the rank of A is r

a

and the rank of B is r

b

,

rank(AB) min(r

a

, r

b

). (C.27)

Regarding sums: the rank of a matrix sumcannot exceed the sumof ranks of the summand matrices.

That is, if the rank of A is r

a

and the rank of B is r

b

,

rank(A +B) r

a

+r

b

. (C.28)

C.2.3. Singular Systems: Particular and Homegeneous Solutions

Having introduced the notion of rank we can now discuss what happens to the linear system (C.16)

when the determinant of A vanishes, meaning that its rank is less than n. If so, (C.16) has either no

solution or an innite number of solutions. Cramers rule is of limited or no help in this situation.

To discuss this case further we note that if |A| = 0 and the rank of A is r = n d, where d 1 is

the rank deciency, then there exist d nonzero independent vectors z

i

, i = 1, . . . d such that

Az

i

= 0. (C.29)

These d vectors, suitably orthonormalized, are called null eigenvectors of A, and form a basis for

its null space.

Let Z denote the n d matrix obtained by collecting the z

i

as columns. If y in (C.16) is in the

range of A, that is, there exists an nonzero x

p

such that y = Ax

p

, its general solution is

x = x

p

+x

h

= x

p

+Zw, (C.30)

where wis an arbitrary d 1 weighting vector. This statement can be easily veried by substituting

this solution into Ax = y and noting that AZ vanishes.

The components x

p

and x

h

are called the particular and homogeneous portions respectively, of the

total solution x. (The terminology: homogeneous solution and particular solution, are often used.)

If y = 0 only the homogeneous portion remains.

If y is not in the range of A, system (C.16) does not generally have a solution in the conventional

sense, although least-square solutions can usually be constructed. The reader is referred to the

many textbooks in linear algebra for further details.

C.2.4. Rank of Rectangular Matrices

The notion of rank can be extended to rectangular matrices, real or complex, as follows. Let A be m n.

Its column range space R(A) is the subspace spanned by Ax where x is the set of all complex n-vectors.

Mathematically: R(A) = {Ax : x C

n

}. The rank r of A is the dimension of R(A).

The null space N(A) of A is the set of n-vectors z such that Az = 0. The dimension of N(A) is n r.

Using these denitions, the product and sum rules (C.27) and (C.28) generalize to the case of rectangular (but

conforming) A and B. So does the treatment of linear equation systems Ax = y in which A is rectangular.

Such systems often arise in the tting of observation and measurement data.

In nite element methods, rectangular matrices appear in change of basis through congruential transformations,

and in the treatment of multifreedom constraints.

C8

C9 C.3 MATRIX INVERSION

C.3. Matrix Inversion

The inverse of a square nonsingular matrix A is represented by the symbol A

1

and is dened by

the relation

AA

1

= I. (C.31)

The most important application of the concept of inverse is the solution of linear systems. Suppose

that, in the usual notation, we have

Ax = y (C.32)

Premultiplying both sides by A

1

we get the inverse relationship

x = A

1

y. (C.33)

More generally, consider the matrix equation for multiple (m) right-hand sides:

A

nn

X

nm

= Y

nm

, (C.34)

which reduces to (C.32) for m = 1. The inverse relation that gives X as function of Y is

X = A

1

Y. (C.35)

In particular, the solution of

AX = I, (C.36)

is X = A

1

. Practical methods for computing inverses are based on directly solving this equation;

see Remark C.4.

C.3.1. Explicit Computation of Inverses

The explicit calculation of matrix inverses is seldom needed in large matrix computations. But

ocassionally the need arises for the explicit inverse of small matrices that appear in element com-

putations. For example, the inversion of Jacobian matrices at Gauss points, or of constitutive

matrices.

A general formula for elements of the inverse can be obtained by specializing Cramers rule to

(C.36). Let B = [b

i j

] = A

1

. Then

b

i j

=

A

j i

|A|

, (C.37)

in which A

j i

denotes the so-called adjoint of entry a

i j

of A. The adjoint A

j i

is dened as the

determinant of the submatrix of order (n 1) (n 1) obtained by deleting the j

t h

row and i

t h

column of A, multiplied by (1)

i +j

.

This direct inversion procedure is useful only for small matrix orders: 2 or 3. In the examples

below the inversion formulas for second and third order matrices are listed.

Example C.9. For order n = 2:

A =

_

a

11

a

12

a

21

a

22

_

, A

1

=

1

|A|

_

a

22

a

12

a

21

a

22

_

, (C.38)

in which |A| is given by (C.2).

C9

Appendix C: MATRIX ALGEBRA: DETERMINANTS, INVERSES, EIGENVALUES C10

Example C.10. For order n = 3:

A =

_

a

11

a

12

a

13

a

21

a

22

a

23

a

31

a

32

a

33

_

, A

1

=

1

|A|

_

b

11

b

12

b

13

b

21

b

22

b

23

b

31

b

32

b

33

_

, (C.39)

where

b

11

=

a

22

a

23

a

32

a

33

, b

21

=

a

12

a

13

a

32

a

33

, b

31

=

a

12

a

13

a

22

a

23

,

b

12

=

a

21

a

23

a

31

a

33

, b

22

=

a

11

a

13

a

31

a

33

, b

32

=

a

11

a

13

a

21

a

23

,

b

13

=

a

21

a

22

a

31

a

32

, b

23

=

a

11

a

12

a

31

a

32

, b

33

=

a

11

a

12

a

21

a

22

,

(C.40)

in which |A| is given by (C.3).

Example C.11.

A =

_

2 4 2

3 1 1

1 0 1

_

, A

1

=

1

8

_

1 4 2

2 0 4

1 4 10

_

. (C.41)

If the order exceeds 3, the general inversion formula based on Cramers rule becomes rapidly useless

as it displays combinatorial complexity. For numerical work it is preferable to solve the system

(C.38) after Ais factored. Those techniques are described in detail in linear algebra books; see also

Remark C.4.

C.3.2. Some Properties of the Inverse

I. The inverse of the transpose is equal to the transpose of the inverse:

(A

T

)

1

= (A

1

)

T

, (C.42)

because

(AA

1

) = (AA

1

)

T

= (A

1

)

T

A

T

= I. (C.43)

II. The inverse of a symmetric matrix is also symmetric. Because of the previous rule, (A

T

)

1

=

A

1

= (A

1

)

T

, hence A

1

is also symmetric.

III. The inverse of a matrix product is the reverse product of the inverses of the factors:

(AB)

1

= B

1

A

1

. (C.44)

This is easily veried by substituting both sides of (C.39) into (C.31). This property generalizes

to an arbitrary number of factors.

IV. For a diagonal matrix D in which all diagonal entries are nonzero, D

1

is again a diagonal

matrix with entries 1/d

i i

. The verication is straightforward.

C10

C11 C.4 EIGENVALUES AND EIGENVECTORS

V. If S is a block diagonal matrix:

S =

_

_

_

_

_

_

S

11

0 0 . . . 0

0 S

22

0 . . . 0

0 0 S

33

. . . 0

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

0 0 0 . . . S

nn

_

_

= diag [ S

i i

] , (C.45)

then the inverse matrix is also block diagonal and is given by

S

1

=

_

_

_

_

_

_

S

1

11

0 0 . . . 0

0 S

1

22

0 . . . 0

0 0 S

1

33

. . . 0

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

0 0 0 . . . S

1

nn

_

_

= diag [ S

1

i i

] . (C.46)

VI. The inverse of an upper triangular matrix is also an upper triangular matrix. The inverse of a

lower triangular matrix is also a lower triangular matrix. Both inverses can be computed in

O(n

2

) oating-point operations.

Remark C.5. The practical numerical calculation of inverses is based on triangular factorization. Given a

nonsingular nn matrix A, calculate its LUfactorization A = LU, which can be obtained in O(n

3

) operations.

Then solve the linear triangular systems:

UY = I, LX = Y, (C.47)

and the computed inverse A

1

appears in X. One can overwrite I with Y and Y with X. The whole process

can be completed in O(n

3

) oating-point operations. For symmetric matrices the alternative decomposition

A = LDL

T

, where L is unit lower triangular and D is diagonal, is generally preferred to save computing time

and storage.

C.4. Eigenvalues and Eigenvectors

Consider the special form of the linear system (C.16) in which the right-hand side vector y is a

multiple of the solution vector x:

Ax = x, (C.48)

or, written in full,

a

11

x

1

+ a

12

x

2

+ + a

1n

x

n

= x

1

a

21

x

2

+ a

22

x

2

+ + a

2n

x

n

= x

2

a

n1

x

1

+ a

n2

x

2

+ + a

nn

x

n

= x

n

(C.49)

This is called the standard (or classical) algebraic eigenproblem. System (C.48) can be rearranged

into the homogeneous form

(A I) x = 0. (C.50)

C11

Appendix C: MATRIX ALGEBRA: DETERMINANTS, INVERSES, EIGENVALUES C12

A nontrivial solution of this equation is possible if and only if the coefcient matrix A I is

singular. Such a condition can be expressed as the vanishing of the determinant

|A I| =

a

11

a

12

. . . a

1n

a

21

a

22

. . . a

2n

.

.

.

.

.

.

.

.

.

.

.

.

a

n1

a

n2

. . . a

nn

= 0. (C.51)

When this determinant is expanded, we obtain an algebraic polynomial equation in of degree n:

P() =

n

+

1

n1

+ +

n

= 0. (C.52)

This is known as the characteristic equation of the matrix A. The left-hand side is called the

characteristic polynomial. We known that a polynomial of degree n has n (generally complex)

roots

1

,

2

, . . .,

n

. These n numbers are called the eigenvalues, eigenroots or characteristic

values of matrix A.

With each eigenvalue

i

there is an associated vector x

i

that satises

Ax

i

=

i

x

i

. (C.53)

This x

i

is called an eigenvector or characteristic vector. An eigenvector is unique only up to a scale

factor since if x

i

is an eigenvector, so is x

i

where is an arbitrary nonzero number. Eigenvectors

are often normalized so that their Euclidean length is 1, or their largest component is unity.

C.4.1. Real Symmetric Matrices

Real symmetric matrices are of special importance in the nite element method. In linear algebra

books dealing with the algebraic eigenproblem it is shown that:

(a) The n eigenvalues of a real symmetric matrix of order n are real.

(b) The eigenvectors corresponding to distinct eigenvalues are orthogonal. The eigenvectors

corresponding to multiple roots may be orthogonalized with respect to each other.

(c) The n eigenvectors form a complete orthonormal basis for the Euclidean space E

n

.

C.4.2. Positivity

Let A be an n n square symmetric matrix. A is said to be positive denite (p.d.) if

x

T

Ax > 0, x = 0 (C.54)

Apositive denite matrix has rank n. This property can be checked by computing the n eigenvalues

i

of Az = z. If all

i

> 0, A is p.d.

A is said to be nonnegative

1

if zero equality is allowed in (C.54):

x

T

Ax 0, x = 0 (C.55)

1

A property called positive semidenite by some authors.

C12

C13 C.6 THE SHERMAN-MORRISON AND RELATED FORMULAS

A p.d. matrix is also nonnegative but the converse is not necessarily true. This property can be

checked by computing the n eigenvalues

i

of Az = z. If r eigenvalues

i

> 0 and n r

eigenvalues are zero, A is nonnegative with rank r.

A symmetric square matrix A that has at least one negative eigenvalue is called indenite.

For the concept of inertia of a Hermitian matrix (which includes symmetric matrices as special

case), see Appendix P.

C.4.3. Normal and Orthogonal Matrices

Let A be an n n real square matrix. This matrix is called normal if

A

T

A = AA

T

(C.56)

A normal matrix is called orthogonal if

A

T

A = AA

T

= I or A

T

= A

1

(C.57)

All eigenvalues of an orthogonal matrix have modulus one, and the matrix has rank n.

The generalization of the orthogonality property to complex matrices, for which transposition is

replaced by conjugation, leads to unitary matrices. These are not required, however, for the material

covered in the text.

C.5. The Inverse of a Sum of Matrices

The formula for the inverse of a matrix product: (AB)

1

= B

1

A

1

is not too different from its scalar

counterpart: (ab)

1

= (1/a)(1/b) = (1/b)(1/a), except that factor order matters. On the other hand,

formulas for matrix sum inverses in terms of the summands are considerably more involved, and there are

many variants. We consider here the expression of (A + B)

1

where both A and B are square and A is

nonsingular. We begin from the identity introduced by Henderson and Searle in their review article [166]:

(I +P)

1

= (I +P)

1

(I +P P) = I (I +P)

1

P, (C.58)

in which P is a square matrix. Using this we develop (A +B)

1

as follows:

(A +B)

1

=

_

A(I +A

1

B)

1

_

1

= (I +A

1

B)

1

A

1

B

=

_

I (I +A

1

B)

1

A

1

_

A

1

= A

1

(I +A

1

B)

1

A

1

BA

1

.

(C.59)

Here B may be singular. If B = 0, it reduces to A

1

= A

1

. The check B = A also works. The last

expression in (C.59) may be further transformed by matrix manipulations as

(A +B)

1

= A

1

(I +A

1

B)

1

A

1

BA

1

= A

1

A

1

(I +BA

1

)

1

BA

1

= A

1

A

1

B(I +A

1

B)

1

A

1

= A

1

A

1

BA

1

(I +BA

1

)

1

.

(C.60)

In all of these forms B may be singular (or null). If B is also invertible, the third expresion on (C.60) may be

transformed to the commonly used variant

(A +B)

1

= A

1

A

1

(A

1

+B

1

)

1

A

1

. (C.61)

The case of singular A may be handled using the notion of generalized inverses. This is a topic beyond the

scope of this course, which may be studied, e.g., in the textbooks [35,49,267]. The special case of B being of

low rank merges with the Sherman-Morrison and Woodbury formulas, covered below.

C13

Appendix C: MATRIX ALGEBRA: DETERMINANTS, INVERSES, EIGENVALUES C14

C.6. The Sherman-Morrison and Related Formulas

The Sherman-Morrison formula gives the inverse of a matrix modied by a rank-one matrix. The Woodbury

formula extends the Sherman-Morrison formula to a modication of arbitrary rank. In structural analysis these

formulas are of interest for problems of structural modications, in which a nite-element (or, in general, a

discrete model) is changed by an amount expressable as a low-rank correction to the original model.

C.6.1. The Sherman-Morrison Formula

Let Abe a square n n invertible matrix, whereas u and v are two n-vectors and an arbitrary scalar. Assume

that = 1 + v

T

A

1

u = 0. Then

_

A +uv

T

_

1

= A

1

A

1

uv

T

A

1

. (C.62)

This is called the Sherman-Morrison formula after [283]. (For a history of this remarkable formula and its

extensions, which are quite important in many applications such as statistics, see the reviewpaper by Henderson

and Searle cited previously.) when = 1. Since any rank-one correction to A can be written as uv

T

, (C.62)

gives the rank-one change to its inverse. The proof is by direct multiplication as in Exercise C.5.

For practical computation of the change one solves the linear systems Aa = u and Ab = v for a and b, using

the known A

1

. Compute = 1 +v

T

a. If = 0, the change to A

1

is the dyadic (/)ab

T

.

C.6.2. The Woodbury Formula

Let again A be a square n n invertible matrix, whereas U and V are two n k matrices with k n and an

arbitrary scalar. Assume that the k k matrix = I

k

+ V

T

A

1

U, in which I

k

denotes the k k identity

matrix, is invertible. Then

_

A +UV

T

_

1

= A

1

A

1

U

1

V

T

A

1

. (C.63)

This is called the Woodbury formula, after [349]. It reduces to (C.62) if k = 1, in which case is a

scalar. The proof is by direct multiplication.

C.6.3. Formulas for Modied Determinants

Let

A denote the adjoint of A. Taking the determinants from both sides of A +uv

T

one obtains

| A +uv

T

| = | A| + v

T

Au. (C.64)

If A is invertible, replacing

A = |A| A

1

this becomes

|A + uv

T

| = |A| (1 + v

T

A

1

u). (C.65)

Similarly, one can show that if A is invertible, and U and V are n k matrices,

| A + UV

T

| = |A| | I

k

+ V

T

A

1

U|. (C.66)

Notes and Bibliography

Much of the material summarized here is available in expanded formin linear algebra textbooks. For example,

Bellman [33] and Strang [296]. The most thorough coverage of the general matrix eigenproblem is still

Wilkinsons book [339]. For the symmetric eigenproblem, the most thorough coverage is that of Parlett [247].

For inverses of matrix sums, there are two SIAM Review articles: [151,166]. For an historical account of the

topic and its close relation to the Schur complement, see the bibliography in Appendix P.

C14

C15 Exercises

Exercises for Appendix C: Determinants, Inverses, Eigenvalues

EXERCISE C.1 If A is a square matrix of order n and c a scalar, show that det(cA) = c

n

det A.

EXERCISE C.2 Let u and v denote real n-vectors normalized to unit length, so that u

T

= u = 1 and v

T

v = 1,

and let I denote the n n identity matrix. Show that

det(I uv

T

) = 1 v

T

u (EC.1)

EXERCISE C.3 Let u denote a real n-vector normalized to unit length, so that u

T

= u = 1 and I denote the

n n identity matrix. Show that

H = I 2uu

T

(EC.2)

is orthogonal: H

T

H = I, and idempotent: H

2

= H. This matrix is called a elementary Hermitian, a

Householder matrix, or a reector. It is a fundamental ingredient of many linear algebra algorithms; for

example the QR algorithm for nding eigenvalues.

EXERCISE C.4 The trace of a n n square matrix A, denoted trace(A) is the sum

n

i =1

a

i i

of its diagonal

coefcients. Show that if the entries of A are real,

trace(A

T

A) =

n

i =1

n

j =1

a

2

i j

(EC.3)

EXERCISE C.5 Prove the Sherman-Morrison formula (C.63) by direct matrix multiplication.

EXERCISE C.6 Prove the Sherman-Morrison formula (C.63) for = 1 by considering the following block

bordered system

_

A U

V

T

I

k

_ _

B

C

_

=

_

I

n

0

_

(EC.4)

inwhichI

k

andI

n

denote the identymatrices of orders k andn, respectively. Solve (C.66) twoways: eliminating

rst B and then C, and eliminating rst C and then B. Equate the results for B.

EXERCISE C.7 Showthat the eigenvalues of a real symmetric square matrix are real, and that the eigenvectors

are real vectors.

EXERCISE C.8 Let the n real eigenvalues

i

of a real n n symmetric matrix Abe classied into two subsets:

r eigenvalues are nonzero whereas n r are zero. Show that A has rank r.

EXERCISE C.9 Show that if A is p.d., Ax = 0 implies that x = 0.

EXERCISE C.10 Show that for any real m n matrix A, A

T

A exists and is nonnegative.

EXERCISE C.11 Show that a triangular matrix is normal if and only if it is diagonal.

EXERCISE C.12 Let A be a real orthogonal matrix. Show that all of its eigenvalues

i

, which are generally

complex, have unit modulus.

EXERCISE C.13 Let A and T be real n n matrices, with T nonsingular. Show that T

1

AT and A have the

same eigenvalues. (This is called a similarity transformation in linear algebra).

EXERCISE C.14 (Tough) Let A be m n and B be n m. Show that the nonzero eigenvalues of AB are the

same as those of BA (Kahan).

EXERCISE C.15 Let Abe real skew-symmetric, that is, A = A

T

. Show that all eigenvalues of Aare purely

imaginary or zero.

C15

Appendix C: MATRIX ALGEBRA: DETERMINANTS, INVERSES, EIGENVALUES C16

EXERCISE C.16 Let A be real skew-symmetric, that is, A = A

T

. Show that U = (I +A)

1

(I A), called

a Cayley transformation, is orthogonal.

EXERCISE C.17 Let P be a real square matrix that satises

P

2

= P. (EC.5)

Such matrices are called idempotent, and also orthogonal projectors. Show that all eigenvalues of P are either

zero or one.

EXERCISE C.18 The necessary and sufcient condition for two square matrices to commute is that they have

the same eigenvectors.

EXERCISE C.19 A matrix whose elements are equal on any line parallel to the main diagonal is called a

Toeplitz matrix. (They arise in nite difference or nite element discretizations of regular one-dimensional

grids.) Show that if T

1

and T

2

are any two Toeplitz matrices, they commute: T

1

T

2

= T

2

T

1

. Hint: do a

Fourier transform to show that the eigenvectors of any Toeplitz matrix are of the form {e

i nh

}; then apply the

previous Exercise.

C16

- ST7201-Finite Element MethodUploaded byVishal Ranganathan
- Practical Implementation Of Matlab Based Approach For Face Detection Using Feedforward NetworkUploaded byijcsis
- Previous Gate by Kanodia Practise Questions All Subjects.pdfUploaded bySarferaz Ahamed
- Design Considerations for Environmental Sustainability in High Density Development a Case Study of Hong KongUploaded byWilbur Fernandes
- Green Acre 2007 Mc a ColUploaded byhubik38
- Teknik Menjawab Matematik SpmUploaded byrosli87
- Quick Linear Algebra for EconometricsUploaded bySeymur
- Tutorial 2Uploaded byROBERT
- Introduction to matrices-Sec1block3Uploaded byakhassadnan
- SubDyn ManualUploaded byOsmanKatli
- Yearly Lesson Plan Mathematics Form5 2017Uploaded byzamzandra
- wikimama Class 12 Ch 4 DeterminantsUploaded byWikimama
- Simulation of Transients in Unerground Cables With Frequency Dependent ModalUploaded byMadhusudhan Srinivasan
- hw6_2011_05_16_01_solutionsUploaded byMorokot Angela
- EC 400 Quizzes for Class 2Uploaded byAvijit Puri
- NotaFizikMathUptoCh3Uploaded byImran Firdaus
- Teoria de GruposUploaded byLuis Vilchez
- Determinan 4x4Uploaded byMas Gund
- مرجع 9Uploaded byBayan Zatari
- Chapter 6 Matrix CalculationsUploaded byaihr78
- Bao Lin Wang2005Uploaded byrezza ruzuqi
- The Psychology of Money-9dbc86Uploaded bychrishallberg
- Farkas a.-the Use of the AHP in Civil EngineeringUploaded byYisvi Higuera
- A Least-squares Algorith for the Equiform Transformation From Spatial Marker Co-OrdinatesUploaded bystillife
- Abdi MCA2007 PrettyUploaded byHuma_Bukhari_8546
- Appendix AUploaded bycemnuy
- alg827.psUploaded bybinukumar100
- 6502 Hand Out 15Uploaded byRoy Vesey
- Academics Tr Tamu Cs Tr 2005-11-4Uploaded byiamnootan
- diffeq1Uploaded byJosé Urrieta

- Bac 2016 Pondichery Lv1 TechnologiqueUploaded byResultatsExams
- Concrete Cap1Uploaded byfamilyrusmi
- mathtools ManualUploaded byJisoo Eric Lee
- Mon Taj Grav Mag InterpretationUploaded byrenzoquim
- Logiq e Service ManualUploaded byChristopher Poole
- 10 signs you’re in a rebound relationshipUploaded byConrad Manacap
- 10.1.1.79Uploaded bykrinunn
- B335Uploaded bydasarisurya
- Xtra Introduction-Mass CommunicationUploaded byfairaadhi
- Thailand and ST Related Venture Capital Rev 0 8Uploaded byHimanshu Sachdeva
- The Branches of ScienceUploaded byHydee Samonte Trinidad
- sayadiUploaded byCatherine G. Idul
- Custom PC - August 2014 UKUploaded byqinlao
- KaizenUploaded byamittal111
- Awakening - Greg MaroneyUploaded byJeroen The Boss
- ARRI Digital - Signal Processing - 2009-09-03Uploaded byTruong Minh
- Simons Foundation Annual Report 2012Uploaded byThe Simons Foundation
- bill becomes law - rubric 3Uploaded byapi-354421161
- O level Oral Conversation TopicsUploaded byongk0043
- Filters for Power Factor Correction in the Presence of Non-Linear LoadsUploaded byOsama Nour
- 1697Uploaded byAntony Boston Michael
- CBSE project on logic gatesUploaded byAbhishek Singh
- Expeditionary Advance Base Operations - EABO Concept Toolkit (Public) - Marinecorpsconceptsandprograms.com (2017)Uploaded bysvbaoudval
- KTA 19 (Parts Catalogue)Uploaded byAhmed Hussain
- Atv312 Installation Manual en Bbv46391 01Uploaded byNikola Lordan
- Effectiveness Analysis of Additive Manufactured Heat Exchanger through Selective Laser Sintering Process for HRVUploaded bySaad Saleem
- caret(1)Uploaded byertertre
- StatisticsUploaded bymunish_tiwari2007
- Sol6Uploaded byakjsdnf
- 15 ppm BILGE SEPARATOR OPERATION MANUAL.pdfUploaded byPrince Dhaka