Appendix
Vector and
matrix algebra
Concepts
Sealars
Vectors, rows and columns, matrices
Adding and subtracting vectors and matrices
‘Multiplying them by scalars
Products of vectors and matrices, scalar and dot products
‘Systems of linear equations, linear substitution
‘Transposition
Unit vectors and identity matrices
Gauss and Gauss—Jordan elimination
Invertible and singular matrices, inverses
Determinants
‘This appendix summarizes the clementary linear algebra used in this book.
‘Much of itis simple vector and matrix algebra that you can learn from the
summary itself, particularly ifyou devise and work through enough two- and
‘threo-dimensional examples as you read it. Some of the techniques summarized
here require you to solve systems of linear equations by methods covered in
school mathematics and commonly subsumed under the title Gauss elimination.
‘There are no examples here to lead you through a full review of elimination,
so if you need that, you should consult a standard linear algebra text.' Almost
all the linear algebra used in the book is two- or three- dimensional, so there's
For example, Larson and Edwards 1991, chapter 1
s29
8:24 6 Feb 2430 APPENDIX €
little need for the Full multidimensional apparatus, particularly for determi
nants. However, many simple techniques, even in three dimensions, are best
explained by the general theory summarized here.
In the context of vector and matrix algebra, numbers are often called scalars.
For the material in this appendix, the scalars could beany complex numbers,
or you could restrict them to real numbers. Applications in this book only
need real scalars.
Vectors
An n-Luple (pair triple, quadruple, ...) of scalars can be written asa horizontal
row or vertical column. A column is called a vector. In this book, a vector
is denoted by an uppercase letter; in this appendix it's in the range O to
Z. Its entries are identified by the corresponding lowercase letter, with
subscripts. The row with the same entries is indicated by a superscript t.
For example, consider
a
x=|) x
[x1 es%el
You can also use a superscript f to convert a row back to the corresponding
column, s0 that X"'= X for any vector X. Occasionally it’s useful to con-
sider a Scalar as a column or row with a single entry.
In analytic geometry it’s convenient to use columns of coordinates for
points. Coefficients of linear equations are usually arranged in rows. For
points, that convention tends to waste page space. This book uses the com-
pact notation to stand for the column [2,%,,%|'.
You can add two vectors with the same number of entries:
%) [nu] [atn
xty=|,f+li f=] 5
%, Yn Hy tIn
Vectors satisfy commutative and associative laws for addition:
X+Y=Y+X 9 X4(V+Z)=(X+¥)+Z.
‘Therefore, as in scalar algebra, you can rearrange repeated sums at will and
omit many parentheses.
‘The zero vector and the negative of a vector are defined by the equations
0 a] [ma
O=l; yeyefe
0 *,
8:24 6 Feb 2VECTOR AND MATRIX ALGEBRA san
Clearly,
x+0
X4+(X)
‘You can regard vector subtraction as composition of negation and addition.
For example, X - Y= X + (-¥), and you can rewrite the last equation
displayed above as X - X = 0. You should state and verify appropriate
‘manipulation rules.
‘You can multiply a vector by a scalar:
‘This product is also written sX.* You should verify these manipulation
rules:
xXi= X X0=0 X(-8) = (Xs) = (-X)s
XV) = -X ot=0
(Xr)s = X(rs) (associative law)
X(r-+8)=Xr+Xs (distributive laws)
(X+Y)s= Xs + Yo
Similarly, you can add and subtract rows X‘ and Y' with the same num-
bor of entries, and define the zero row and the negative of arow. The product
ofa scalar and a row is
5X’
Lys enya] = [9% 00058]
‘These rules are useful:
XE YS (KEY) AVS CX 8X) = (XY!
Finally, you can multiply arow by a vector with the same number of entries
to get their scalar product:
an
XY Slade [Sat tae
Mn
A notational variant used in analytic geometry is the dot product: X+Y =
X'Y. (Don't omit the dot. Vector entries are often point coordinates, and
® Xe is more closely compatible with matrix multiplication notation, discussed later. Hach
orm has advantages, so this book uses both
8:24 6 Feb 2482 APPENDIX €
the juxtaposition XY usually signifies thedistance between X and Y.) With
alittle algebra you can verify the following manipulation rules
O%X=0=X'0 GXOY = (XY)
X'Y=¥'X CXOY = (XY) = xO)
(X4YOZ= XZ 4D (distributive laws)
XY +Z) = XV + XZ,
Matrices
‘An mXn matrix is a rectangular array of mn sealars in m rows and n
columns. In this book, a matrix is denoted by an upperease letter; in this
appendix it'sin therange A to O. Itsentries are identified by the correspond-
ing lowercase letter, with double subscripts:
Oy ~ Oy
qi Sp
columns
A iscalled square when m =n. The a,, with i=/ are called diagonal entries.
mx 1 and 1xn matrices are columns and rows with m and n entries, and
1X1 matrices are handled like scalars.
You can add or subtract mxn matrices by adding or subtracting cor-
responding entries, just as you add or subtract columns and rows. A matrix
whose entries are all zeros is called a zero matrix, and denoted by O. You
‘ean also define the negative of a matrix, and the product sA of a scalar s
and a matrix A. Manipulation rules analogous to those mentioned earlier
for vectors and rows hold for matrices as well; check them yourself.
You can multiply an mxn matrix A by avector X with n entries; their
product AX is the vector with m entries, the products of the rows of A
by X:
ey ~ Oy
AX ‘ : ‘
Oq1 Son
You can verify the following manipulation rules:
OX=0=A0
8:24 6 Feb 2VECTOR AND MATRIX ALGEBRA 433
(A+ B)X=AX4 BX (distributive laws)
AUK + ¥) = AX +AY.
‘The definition of the product of a matrix by a column was motivated by
the notation for a system of m linear equations inn unknowns x, to
x4; you can write AX = R as an abbreviation for the system
@yz, + to,z, = 7
Apt beat,
Similarly, you can multiply arow X' with m entries by an m Xn matrix
A; their product X'A is the row with » entries, the products of X* by the
columns of A:
Oy Oy
KAR Ere] | i H
F Laay, Fo FH egyy very Mag FH Lag]
Similar manipulation rules hold. Further, you can check the associative law
XY) = (AVY.
You can multiply an [xm matrix A byan mxn matrix B. Their product
AB isan [xn matrix that you can describe two ways. Its columns are the
products of A bythe columns of B, andits rows arethe products of the rows
of A by B
Sy ~ Sim |f bu ~ Py
aB=|;
yy ~ Gin || Om
By by Ft FOO yd to +O Pan
Aydt + inborn + + Gib
‘The i,kth entry of AB is thus a,.b,, + +jqbq4. You can check these
‘manipulation rules:
AO=0=0B — (sA)B
(aye
(A+B)C =AC+BC (distributive laws)
A(C +D) = AC + AD
8:24 6 Feb 2aa APPENDIX €
‘The definition of the product of two matrices was motivated by the formulas
for linear substitution; from
2p Ist + Onn M1 but to + Oe,
2 = OI + +n Ym = Omit +o + oy
you can derive
2, = by +~ + Oigdy I, +> + (bay F~ + Oinb re),
z . Coban 4 + Sgn 8, 4+ (big #~ Oye)»
‘That is, from Z = AY and Y= BX you can derive Z = (AB)X. In short,
A(BX) = (AB)X. From this rule, you can deduce the general associa-
tive law:
A(BC) = (AB)C.
Proof: jth column of A(BC) = A(jth column of BC)
= A(B(jth column of C))
(AB)( jth column of C)
= jth column of (AB)C. #
‘The commutative law AB = BA doesn’t generally hold. For example,
oo}foo]_foo
o1
oo}foo]_foo
ro] lio rojlo1] loo
‘This example also shows that the product of nonzero matrices can be O.
Every mxn matrix A has a transpose A‘, the nXm_ matrix whose
Jvith entry is the i, jth entry of A
a, ~ a, }
Aly i yee '
Om. On Oy, = Ong
‘The following manipulation rules hold:
AoA o'=0
(A+B) =A'4B! (3A)!
(AD.
‘The transpose ofa vector isa row, and vice-versa, so this notation is consistent
with the earlier use of the superscript ¢. If A isan [xm matrix and B
isan mxn matrix, then
(AB)! = BA’
8:24 6 Feb 2VECTOR AND MATRIX ALGEBRA 495
Proof: j,ith entry of (AB)' = i,jth entry of AB
= ( ith row of A)(jth column of B)
= ( jth column of B)‘(ith row of A)
= ( jth row of B)(ith column of A)
= jrith entry of B'A'.¢
Consider vectors with n entries. Those of the jth unit vector U? are
all 0 except the jth, which is 1. For any row X' with n entries, X'U? ththe j
entry of X‘. For any mxn matrix A, AU? isthe jth column of A. For
example,
1 1
ur=|O) xu stx.ml] Od =
° °
My ty ~ Oy )1L) [oy
AUt =] % %2 Gn |] 0] — | oar
Sut One 0 co
‘The nxn matrix I whose jth column is the jth unit vector is called an
identity matrix. Its only nonzero entries are the diagonal entries 1
Clearly, I’ = 1. For any mxn matrix A, Al =A. Proof:
{th column of AI = A(jth column of 1)
= AU! = jth column of A.#
In particular, for any row X' with n entries, XI =X‘.
Similarly, you may consider rows with m entries. ‘The wnit rows
(U5! are the rows of the mXm identity matrix J. You can verify that for
any column X with m entries, (U‘)'X is the ith entry of X. For any
mXn matrix A, (U')'A is the ith row of A. This yields [A =A for any
mXn matrix A. In particular, IX =X for any column X of length m.
Gauss elimination
‘The most common algorithm for solving a linear system AX = R is called
Gauss elimination. Its basic strategy is to replace the original system step
bystep with equivalent simpler ones until you can analyze the resultingsystem
easily. Two systems are called equivalent if they have thesame sets of solution
vectors X. You need only two types of operations to produce the simpler
systems:
L jerchange two equations,
TI. subtract from one equation a scalar multiple of adifferent equation,
8:24 6 Feb 2436 APPENDIX €
Obviously type (I) operations don’t change the set of solution vectors; they
produce equivalent systoms. Ifyou perform atype (II) operation, subtracting,
s times the ith equation from the jth, then any solution of the original system
clearly satisfies the modified one. On the other hand, you can reconstruct
the original from the modified system by subtracting (-s) timesits ith row
fromits th, so any solution ofthe modified system satisfies the original—the
systems are equivalent.
‘Thesimpler systems ultimately produced by Gauss climination have matrices
A ofspecial forms. Alincar system anditsmatrix A are called upper triangular
if a,, = 0 whenever i >j, and diagonal if a,, =0 whenever i
‘The first steps of Gauss elimination, called its downward pass, are type
(1) and (II) operations that convert the original mxn system
yt, Hoyt +> 40,2, = 7,
Ggyty + Ogata +~ +0y,%, = Ty
Oy FOgaty + FOZ My
into an equivalent upper triangular system
02, F Oia tm HO Ho TG, = Ty
Gata + + Gagtn t+ Oq%, =F if msn, or
Opn + Spy hy Toy
ent, Hayty + Hoyt, =
yt, + +02, = Ty
5 1
ifm>n
Gan
OMe
1
0 =r,
‘The algorithm considers in turn the diagonal coefficients aj; to @n-1m-+
called pivots. Ifapivot iszero, search downward for anonzero coefficient; if you
find one, interchange rows—a type (I) operation—to make it the new pivot.
Hfnot, then proceed to the next equation. Use anonzero pivot a,, with type
(ID) operations to eliminate the x, terms from all equations after the kth.
‘This process clearly produces an equivalent upper triangular system.
You can apply the downward pass to any linear system. In this book, it's
used mostly with square systems, where m = n. Until the last heading of
this appendix—Gauss-Jordan elimination—assume that’s the ease.
‘Ifthe downward pass yields a square upper triangular matrix with no zero
pivot, the original system and its matrix are called nonsingular. This property
8:24 6 Feb 2VECTOR AND MATRIX ALGEBRA 47
is independent of the right-hand sides of the equations; it depends only on
the original matrix A. In the nonsingular case, you can perform more type
(ID operations— constituting the upward pass—to convert any system AX =
R to an equivalent diagonal system:
aye, =“
Gat “7
uty = Tu
‘This system clearly has the unique solution
Ka forany values of s. In general, proceed back
up the diagonal, solving thesystem as though you were performing the upward
pass. When you encounter a zero pivot, give the corresponding X entry an
arbitrary value—the parameter s in this example. Use adistinet parameter
for each zero pivot.
‘The previous two paragraphs are crucial for the theory of matrix inverses,
hence they’re worth recapitulation. Ifan n Xn matrix A isnonsingular—the
downward pass yields an upper triangular matrix with no zero pivot—then
for every nxp matrix C, the equation AB = C has a unique solution
B. But if A is singular, then at least one such equation—in particular,
AX = 0 —has multiple solutions,
Matrix inverses
A matrix A is called invertible if there's a matrix B such that AB ==
BA. Clearly, invertible matrices must be square. A zero matrix O isn’t
8:24 6 Feb 2438 APPENDIX €
invertible, because OB = 0 #1 forany B. Also, some nonzero square matri-
ces aren't invertible. For example, for every 2x2 matrix B,
oo 0 Olfo, ba]_[o 0
o1 0 1] [ba ba] [bar bee
hence the leftmost matrix in this display isn’t invertible. When there exists
B such that AB = [= BA, it’s unique; if also AC =1 = CA, then B=
BI=B(AC) =(BA)C=IC=C. Thusan invertiblematrix A hasa unique
inverse A“ such that
AAT =I= AA,
B
41,
Clearly, I is invertible and I”! = 1
‘The inverse and transpose of an invertible matrix are invertible, and any
product of invertible matrices is invertible:
(AytsA (ADS (A (ABBY? = BAM
Proof: The first result follows from the equations AA! == AA; thesecond,
from A(A")'=(A"A)'=I' =I and (A7YA' = (AA)! 5 I' = 1. The
third follows from (AB)(B"1A) = ((AB)B")A” = (A(BB™))A7 =
(ADA" = AA" =I and equation (B“'A)(AB) = I, which you can
chock. #
‘The main result about inverses is that a square matrix A is invertible if
andonly ifit’snonsingular. Proof: If A isnonsingular, use Gauss elimination
to solve equation AB = I. To show that also BA = I, the first stop is to
verify that A’ is nonsingular. Were that not so, you could find X# 0 such
that A'X = 0, as mentioned under the previous heading. But then X = K
=(AB)'X = B'A'X = B'0 = 0 —contradiction! ‘Thus A’ must be nonsingular,
and you can solve equation A’C = I. That entails
BA =I'BA" = (A'C)'BA" = C'A"BA" = C'ABA" = C'L
= CAN = (A'C) aT aT.
‘Thus B is the inverse of A. Conversely, if A has an inverse B, then A
must be nonsingular, for otherwise you could find X #0 with AX =O,
which would imply O = BAX = IX = X —contradiction!
Determinants
‘The determinant of an nXn matrix A is
At A = Ly ging Gan SIEM P,
where the sum ranges over all n! permutations g of {1,....n), and
sign g = #1 depending on whether @ is even or odd. Tn each term of the
8:24 6 Feb 2VECTOR AND MATRIX ALGEBRA a9
sum there's one factor from each row and one from each column. For example,
the permutation 1,2,3~>3,2,1 is odd because it just transposes 1 and 3,
so it corresponds to the term @4@,,¢,,(-1) in the determinant sum for a
3x3 matrix A. For the theory of permutations, consult a standard algebra
text.
‘Usually you don’t need the full apparatus of the theory of permutations.
‘Most of the determinants you’ll meet in this book are 2x2 or 3X3, and
for them it’s enough to write out the sums in full. For the 2x2 case there
are two permutations of {1,2}, and
eye) |
On Ox °
Clearly, the determinant ofa 2x2 matrix is zero if and only ifone row or
column is a scalar multiple of the other.
For the 3x3 case, there are six permutations of (1,2,3) and
det A = det}
My Oy Os
Ot A = det | ag, ayy 9g |= Airdrnt + Oi tiax ty + dye,
Oy On Oey :
Figure .1 showsa handy scheme for remembering this equation. Theindicated
diagonals in the diagram, with their signs, contain the factors of the terms
in the determinant sum,
‘Themost important properties of determinants are closely tied to thelinear
system techniques summarized under the previous two headings.
Sy Hyg
ay My , on ne
det |x. a @2/= nn Gee
Oa, Om 9
Gy Mp agg
Figure C.1 Evaluating a 3x8 determinant
First, the determinant of an upper triangular matrix is the product of its
diagonal entries; in particular, identity matrices have determinant 1, Proof
® For example, Mostow, Sampson, and Meyer 1963, section 10:3.
8:24 6 Feb 240 APPENDIX €
Each term in the determinant sum excopt 2,,4,.""-4,, contains at least one
factor a,, with i >, and that factor must be zero.
‘Next, if B results from a square matrix A by interchanging two rows,
then det B= -det A. Proof: Each term in the sum for det B corresponds
to aterm with the opposite sign in the sum for det A. #
A square matrix A with two equal rows has determinant zero. Proof:
Interchanging them reverses the sign of the determinant but doesn’t change
the matrix. #
Tall rows of square matrices A, B, and C are alike except that the ith
row of A is the sum of the corresponding rows of B and C, then
det A = det B + det. Proof: Bach term in the sum for detA is the sum
of the corresponding terms of det B and det C. #
Asquare matrix A with a row of zeros has determinant zero. Proof: By
the previous paragraph, detA = detA + deta. #
If B results from a square matrix A by multiplying the ith row of A
by ascalar s, then detB = sdetA. Proof: Each term in the sum for
det B is § times the corresponding term of det A. #
If B results from a square matrix A by subtracting s timesits ith row
from its jth, then det B = dot A. Proof: Construct C from A by replacing
its jthrowby (-s) timesits ithrow,so that det B= det A + dot C. Construct,
D from A by replacing its jth row by its ith, so that det C = ~s det D. Then
D has two equal rows, so detD = 0, hence detC = 0, hence det B =
det A. +
‘A-square matrix A has determinant zero if and only if it’s singular
Proof: By the previous discussion, det A is (-1)* times the product of the
diagonal entries of the matrix that results from A through the downward
pass of Gauss elimination; & is the number of row interchanges required
in that process. #
An nxn matrix A has the same determinant as its transpose
Proof: det ' isthe sum of the terms @:;°°"@ny9 Sign for all the permu-
tations g of (1,....n}. You can rearrange each term's factors and write
itim the form a, y9""°@y yn SIEM = 04,457" yn SBD, Where 7= 9"!
Since the permutations’ correspond one-to-one with their inverses 7,
et A! = La yi" pw Sig Z= det A. +
By the previous paragraph, you can formulate in terms of columns some
of the earlier results that relate determinants to properties of their rows.
‘The next sequence of results leads slowly to the important equation
det AB = det A det B. Its proof uses some special matrices.
8:24 6 Feb 2VECTOR AND MATRIX ALGEBRA aa
Adype (I) elementary matrix E' results from the nxn identity matrix
by interchanging its ith and jth rows, where i 4. Clearly, det BY = ~
You can check that interchanging the ith and jth rows of any nxn matrix
A yields the matrix B¥A. Thus detHYA = det E'VdetA and E¥EY =I,
so E is its own inverse
Alype (II elementary matrix E* results from the n xn identity matrix
by subtracting ¢ times its ith row from its jth, where i 4 j. Clearly,
dot EY" = 1. You can chock that subtracting c times the ith row from the
jth of any nxn matrix A yields the matrix BA. Thus dot E'A =
dot EY dotA and BYE =I, so (EY)! =", another type (II)
elementary matrix.
If D and A are nxn matrices and D is diagonal, then detDA =
dot D detA. Proof: Each row of DA isthe product ofthe corresponding row
of A and diagonal entry of D.#
It A and B are nxn matrices, then detAB = det AdetB. Proof: If
AB hasan inverse X, then A(BX) = (AB)X =I, so A hasan inverse.
‘Thusif A is singular, sois AB, and det AB = 0 = det A det B. Nowsuppose
A is invertible. Execute the downward pass of Gauss elimination on A,
performing type (1) and type II) operations until you get an upper triangular
matrix U. Bach operation corresponds to left multiplication by an elementary
matrix, so E,B,_"°E,E,A =U for some elementary matrices E, to Ey.
‘The diagonal of U contains no zero, so you can perform more type (II)
operations until you get a diagonal matrix D. Thus B\E,_y"°Ey..B,,U =
D for some more elementary matrices E,,, to B;. This yields
EyB,y Ey" ByE,A=D A= Ej1B;'-Ey! D.
‘These inverses are all elementary matrices and
det AB = det(Ey'By'-By'-B,E"DB)
dot B" det Bj" det H;?--- dot K, ! det By" det D det B
= det (BE! -By'--B,E;'D) dot B
= det A det B. +
Determinants play important roles in analytic geometry. In three
dimensions, these involve the cross product of two vectors. For a detailed
description, see section 5.6.
Gauss-Jordan elimination
‘To solvea linear system AX = B whose mXn matrix A isn’t square, first
perform the downward pass of Gauss elimination, converting it to an upper
triangular system as described earlier. Instead of the upward pass, however,
‘complete the process described next, called Gauss—Jordan elimination.
8:24 6 Feb 2402 APPENDIX €
If m>n, then the last m -n equations have the form 0 =r, with m<
kn. Ifany of these r, #0, the system has no solution. Otherwise, you
‘ean ignore those last equations, and the system is equivalent to an mXm.
upper triangular system. Proceed as described earlier.
If m