(JAN2016) Lecture Linear Algebra PDF

MAT1024 LINEAR ALGEBRA & APPLICATIONS
CHAPTER 1
1.1)
SYSTEMS OF LINEAR EQUATIONS AND

MATRICES
Introduction to Systems of Linear Equations
A system of m linear equations in n unknowns can be written as
a11x1 a12 x2 a1n xn b1

a21x1 a22 x2 a2n xn b2
am1 x1 am2 x2 amn xn bm

If b1 b2 bm 0 , then the system is called homogeneous
The augmented matrix for the system is
a11
a 21

a
m1
a12
a 22
am2
a1n b1
a 2 n b2

a mn bm
Example 1: A system of 3 linear equations in 5 unknowns
3x1 7 x2 2 x3 x4 3x5 2
4 x3 2 x 4
6
2x1
4 x1 5x2 4 x3
x5 0
Example 2: A homogeneous system of 3 linear equations in 4 unknowns
2 x1 x2 3x3 x4 0
2 x3 3 x 4 0
x1
3x1 x2 4 x3
0
Example 3: Find the augmented matrix for the system of linear equations
2 x3 3
x1
2 x1 x2 3x3 4
3x1 4 x2 2 x3 6
2
3
1 0
Solution : 2 1 3 4
3 4 2 6
Example 4 : Find a system of linear equations corresponding to the augmented matrix
3 0 2 5
4 3
7 1
0 2 1
7
Solution : 3x1
2 x3 5
7 x1 x2 4 x3 3
2 x 2 x3 7
A system of linear equations has either
(I) No solution
-the system is said to be inconsistent
(II) Exactly one solution
-the system is said to be consistent
(III) Infinitely many solutions -the system is said to be consistent
Illustration for the system of 2 equations in 2 unknowns
Case (I)
(Two lines are parallel)
(No solution)
Case (II)
(Two lines intersect)
(Exactly one solution)
Case (III)
(Two lines coincide)
(Infinite number of solutions)
Example 5:
Case (I)
l1 : x 2 y 3 l 2 : 2 x 4 y 1
Case (II)
l1 : x 2 y 3 l 2 : 2 x 3 y 5
Case (III)
l1 : x 2 y 3 l 2 : 2 x 4 y 6
Any homogeneous systems of linear equations has the trivial solution x1 0 , x2 0 ,,

xn 0 . Therefore, any homogeneous systems of linear equations is consistent, that is it
has either
(a) Exactly one solution (in which case the solution is trivial) or
(b) Infinitely many solutions
Illustration for the system of 2 equations in 2 unknowns
Case (a)
Case (b)
Example 6:
Case (a)
l1 : x 2 y 0 l 2 : 2 x 3 y 0
Case (b)
l1 : x 2 y 0 l 2 : 2 x 4 y 0
Theorem 1.1 Any homogeneous system of linear equations with more unknowns than
equations has infinitely many solutions.
Example 7: Consider the homogeneous system of 2 equations in 3 unknowns
2 x1 3x2 4 x3 0
x1 2 x2 5x3 0
This system has infinitely many solutions.
Note that the equations represent two planes passing through the origin and they intersect
at a line.
Example 8: Consider the system of equations

ax by k
cx dy l
ex fy m
What can be said about the relative positions of the lines l1 : ax by k , l 2 : cx dy l ,
and l 3 : ex fy m when
(a) the system has no solution;
(b) the system has exactly one solution;
(c) the system has infinitely many solutions?
Solution:
(a)
(b)
(c)
1.2)
Gaussian Elimination
Three types of Elementary Row Operation (ERO)

1) Multiply one row by a nonzero constant
2) Interchange two rows
3) Add a multiple of one row to another
Example 1
Multiply row 2 by a 3 :
1 2
1 2
R2 3R2
3 4
9 12
Interchange row 1 and row 2 :
1 2
3 4
R1 R2
3 4
1 2
1 2
1 2
R2 R2 (3) R1
Add -3 times of row 1 to row 2 :

3 4
0 2
Example 2 Perform the indicated elementary operation on the matrix
1 1 2 9
(a) 2 4 3 1 R2 3R2
3 6 5 0
1 1 2 9
(b) 2 4 3 1 R1 R2
3 6 5 0
1 1 2 9
(c) 2 4 3 1 R2 R2 (2) R1
3 6 5 0
Example 3
Perform the indicated elementary operation on the matrix
1 1 2 9
2 4 3 1 R2 R2 (2) R1
3 6 5 0
R3 R3 (3) R1
(Row-echelon form) (REF)

1: All rows consisting entirely of zeros occur at the bottom of the matrix.
2: For each row that does not consist entirely of zeros, the first nonzero entry is 1 (called
a leading 1)
3: For two successive (nonzero) rows, the leading 1 in the higher row is farther to the left
than the leading 1 in the lower row
(Reduced row-echelon form) (RREF)
1: All rows consisting entirely of zeros occur at the bottom of the matrix.
2: For each row that does not consist entirely of zeros, the first nonzero entry is 1 (called
a leading 1)
3: For two successive (nonzero) rows, the leading 1 in the higher row is farther to the left
than the leading 1 in the lower row
4: Each column that contains a leading 1 has zeros in every position above and below its
leading 1.
Example 4
1 2 3 4
0 2 1 1
0 0 1 3
1 2 1 2
0
0 0 0
0 1 2 4
Neither REF nor RREF
1 2 1 4
3
0 1 0
0 0 1 2
1 5
0 0
0 0
0 0
REF
0 1 0 5
0 0 1 3
0 0 0 0
0
0
0
1
0
0
2 1 3
1 3 2
0 1
4
0 0
1
0 1
0 2
1 3
0 0
RREF
Example 5
Determine whether the following matrix is in row-echelon form, reduced row-echelon
form or neither
1 0 0 0
1 1 0
1 1 0
0 1 1 0
(a) 0 2 0
(b) 0 1 0
(c)
0 0 0 1
0 0 0
0 3 0
0 0 0 0
0
(d)
0
1 3
0 0
0 0
0 0
0
1
0
(e)
0
0
1
0
0
3
0
0
0
0
1
0
(f)
0
2
1
0
0
0
1
0
0
0
1
In solving the systems of linear equations,

(a) Gaussian Elimination (GE) reduces the augmented matrix to REF
(b) Gauss-Jordan Elimination (GJE) reduces the augmented matrix to RREF
Example 6 For the following 3 cases, obtain an augmented matrix and reduce them to
REF and/or RREF
Case (I)
l1 : x 2 y 3 l 2 : 2 x 4 y 1 (No solution)
1 2 3
1 2 3
1 2 3
R2 R2 (2) R1
(REF)
R2 15 R2
2 4 1
0 0 1
0 0 5
1 2 0
(RREF)
R1 R1 (3) R2
0 0 1
Case (II) l1 : x 2 y 3 l 2 : 2 x 3 y 5 (Exactly one solution)
1 2 3
1 2 3
1 2 3
R2 R2 (2) R1
R2 (1) R2
(REF)
2 3 5
0 1 1
0 1 1
1 0 1
(RREF)
Note: The solution is x 1, y 1
R1 R1 (2) R2
0 1 1
Case (III) l1 : x 2 y 3 l 2 : 2 x 4 y 6 (Infinitely many solutions)
1 2 3
1 2 3
R2 R2 (2) R1
(RREF)
2 4 6
0 0 0
What observation can be you make regarding the RREF for these cases?
Example 7 Solve the system
x1 x2 2 x3 9
2 x1 4 x2 3x3 1
3x1 6 x2 5x3 0
Solution:
1 1 2 9
R2 R2 (2) R1
2 4 3 1
3 6 5 0 R3 R3 (3) R1
9
1 1 2
1
0 2 7 17 R2 2 R2
0 3 11 27
9
1 1 2
1 1 2
7
17
7
0 1 2 2 R3 R3 (3) R2 0 1 2
0 0 1
0 3 11 27
1 1 2
7
0 1 2
0 0 1
172
3
172 R3 (2) R3
32
row-echelon form
x1 x2 2 x3 9
x2 72 x3 172
x1 1
x2 2
x3 3
Example 8 Solve the system by Gaussian elimination or Gauss-Jordan elimination
x1 2 x2 3x3 2
3x1 2 x2 x3 2
4 x1 5x2 3x3 6
Solution:
Example 9 (System has infinitely many solutions)

Solve the system by Gaussian elimination
x1 8x3 5x4 6
x2 4 x3 9 x4 3
x3 x 4 2
Solution:
1 0 8 5 6
The augmented matrix for the system is 0 1 4 9 3 . Note that this matrix is
0 0 1 1 2
already in row-echelon form. To solve the system, we may let x4 t .

Then x3 2 t , x2 5 13t , x1 10 13t . The solution is,
x1 10 13t 10 13

x 2 5 13t 5 13
x
t
~
x3
2 t 2 1

0 1
x
t

4
Example 10 (System has no solution)
Solve the system by Gauss-Jordan elimination
2 x1 3x2 2
2 x1 x2 1
3x1 2 x2 1
Solution:
2 3 2
1 . The reduced row-echelon form

The augmented matrix for the system is 2 1
3 2
1
1 0 0
is 0 1 0 and the system has no solution. Notice that the three lines above do not
0 0 1
intersect at a common point.
10
Example 11
Determine the values of k for which the following system have
(a) No solution (b) Exactly one solution (c) Infinitely many solutions
x1 2 x2 3x3 4
3x1 x2 5x3 2
4 x1 x2 (k 2 14) x3 k 2
Solution:
3
4
1 2
5
2
The augmented matrix for the system is 3 1
4 1 k 2 14 k 2
3
4
1 2
R2 R2 (3) R1
14
10
0 7
R3 R3 (4) R1
2
0 7 k 2 k 14
R3 R3 (1) R2
3
4
1 2
14
10
0 7
0 0 k 2 16 k 4
Pay attention to the last row of the above augmented matrix, we notice that,
If k 4 , then the system has no solution
If k 4 , then the system has infinitely many solutions
If k 4,4 , then the system has exactly one solution
11
Example 12
Determine the value(s) of k (where k 1 ) for which the following system have no
solution
x1 x2 kx3 3
x1 kx2 x3 2
kx1 x2 x3 1
Solution:
1 1 k 3
The augmented matrix for the system is 1 k 1 2

k 1 1 1
1
k
3
1
R2 R2 (1) R1
1
0 k 1 1 k
R3 R3 (k ) R1
2
0 1 k 1 k 1 3k
1
k
1
1 k
R3 R3 R2 0 k 1
0
0
2k k2
1
3k
Pay attention to the last row of the above augmented matrix, we notice that,
If we let 2 k k 2 0 , after factoring, we get (2 k )(1 k ) 0 . Therefore k 2 .
12
1.3)
Matrices and Matrix Operations
a11
a
An m n matrix A is given by A 21
a
m1
a12
a 22
am2
a1n
a2n
.

a mn
The number a ij is the (i, j ) th entry of A.

The i th row and j th column of A are ai1
ai 2 ain
a1 j
a2 j
and
respectively.

a
mj
A 1 n matrix is called a row matrix and an m 1 matrix is called a column matrix

An n n matrix is called a square matrix.
A matrix whose entries are all zero is called a zero matrix.
Diagonal matrix
a
A (aij ) where aij ii
0
i j
i j
Identity matrix, I
a 0 0
Example: 0 b 0
0 0 c
1 0 0
Example: I 3 0 1 0
0 0 1
Upper triangular matrix

A (aij ) where aij 0 for i j
1 2 3
Example: U 0 5 6
0 0 9
Lower triangular matrix
A (aij ) where aij 0 for i j
1 0 0
Example: L 4 5 0
7 8 9
Transpose of a matrix
If A (aij ) , then the transpose of A, denoted by AT is given by AT (a ji )
13
Symmetric matrix
1 2 3
A matrix A is symmetric if A A . The matrix A 2 5 6 is symmetric because

3 6 9
T
A A
T
Trace of a matrix
n
Let A (aij ) be an n n square matrix. Then the trace of A, tr(A) aii

i 1
Operations on Matrices
Addition, subtraction, scalar multiplication, matrix multiplication
1 2 3
1 2
1 2 3
C
Example: Let A 4 5 6 , B
3 4
4 5 6
7 8 9
1 2 3
1 2 3
1 2
1 2 3
is undefined, AC 4 5 6
is undefined
AB 4 5 6
3
4
4
5
6
7 8 9
7 8 9
1 2 3
1 2 3
4 5 6
CA
4 5 6 7 8 9
(1)(1) (2)(4) (3)(7) (1)(2) (2)(5) (3)(8) (1)(3) (2)(6) (3)(9)

(4)(1) (5)(4) (6)(7) (4)(2) (5)(5) (6)(8) (4)(3) (5)(6) (6)(9)
Example 1
Classify each of the following matrices
1 0 2
(a) 0 0 0
2 0 3
1 0 0
(b) 0 3 1
0 0 2
1 0 0
(c) 1 3 0
0 2 1
2 0 0
(d) 0 3 0
0 0 4
14
1.4)
Inverses; Rules of Matrix
Theorem 1.4.1 If R is the reduced row-echelon form of an n n matrix A, then either R

has a row of zeros or R is the identity matrix I n
Example 1:
Find the reduced row-echelon form of the matrices
1 4
1 3
(b)
(a)
2 3
2 6
Solution:
1 3
R2 R2 (2) R1
(a) A
2 6
1 3
contains a row of zeros

0 0
1 4
R2 R2 (2) R1
(b) A
2 3
1 4
R2 ( 15 ) R2
0 5
1 4
0 1
1 0
I 2 the identity matrix
R1 R1 (4) R2
0 1
Theorem 1.4.2 (on transpose) Let A be a square matrix. Then

(a) ( AT ) T A (b) ( A B)T AT B T (c) (kA) T kAT (d) ( AB) T B T AT
Example 2:
1 4
2 1
, B
, k 3
Verify the above theorem for A
2 3
3 4
Solution:
1 4
1 2
1 4
, AT
, ( AT ) T
. Therefore ( AT ) T A
(a) A
2 3
4 3
2 3
1 2 2 3 3 5
3 3
3 5

and AT B T
(b) ( A B)
4 3 1 4 3 7
5 7
3 7
T
15
1 2 2 3 1 1
1 5
1 1

and AT B T
( A B)
4
3
1
4
1
5
1
5
3 12
3 6 1 2

3
3 AT
(c) (3 A)
6
9
12
9
4
3
1 4 2 1
14 15
14 13
(d) ( AB )
13 10
15 10
2 3 3 4
T
2 3 1 2 14 13

.
B T AT
1 4 4 3 15 10
Therefore ( AB) T B T AT
If A is a square matrix, and if B a matrix such that AB BA I , then A is said to be

invertible and B is called an inverse of A. If no such matrix B exists, then A is said to be
singular.
a b
where ad bc 0 . Then
Theorem 1.4.3 (on inverse) Let A
c d
A 1
1 d b
ad bc c a
1 4
Example 3: Find A 1 if A
2 3
Solution:
A 1
3 4
1
1 3 4
(1)(3) (4)(2) 2 1
5 2 1
16
Theorem 1.4.4 (on inverse) Suppose A and B are invertible. Then

(a) AB is invertible and ( AB) 1 B 1 A1
(b) A n is invertible and ( An ) 1 ( A1 ) n
(c) for any nonzero constant k, kA is invertible and (kA) 1
1 1
A
k
(d) AT is invertible and ( AT ) 1 ( A1 )T
Example 4: Verify the above theorem for

2 1
1 1
, B
, k 3 , n 2
A
9 4
5 4
Solution:
4 1
4 1
, B 1
(a) A 1
5 1
9 2
2 1 1 1 3 2

AB
9 4 5 4 11 7
7 2
4 1 4 1 7 2
, B 1 A1

(AB ) 1
11 3
5 1 9 2 11 3
Therefore, ( AB) 1 B 1 A1
(b)
17
(c)
(d)
18
1.5)
Elementary Matrices and A Method for Finding Inverse
An n n matrix E is called an elementary matrix if it can be obtained from I n by a single

elementary row operation.
Example 1: Which of the following matrices are elementary?
1 0 0
0 3 0
0 0 1
1 0 0
0
1
0
1 0 0
0 1 0
0 0 0
1 0 0
0 0 1
0 1 0
1 0
2
1
1 0 0
0 2 0
0 0 1
Theorem 1.5.1 Let E be the elementary matrix obtained by performing an elementary

row operation on I m . If the same elementary row operation is performed on an m n
matrix A, then the resulting matrix B is given by the product B=EA.
1 0 2 3
Example 2: Let A 2 1 3 6 .
1 4 4 0
1 0 2 3
(a) Perform ERO R2 3R2 on A, the resulting matrix is B 6 3 9 18 .

1 4 4 0
1 0 0
Perform this same ERO on I 3 , we obtained E 0 3 0 . It can be checked that

0 0 1
1 0 2 3
B 6 3 9 18
1 4 4 0
1 0 0 1 0 2 3
0 3 0 2 1 3 6 EA
0 0 1 1 4 4 0
1 0 2 3
(b) Perform ERO R2 R3 on A, the resulting matrix is B 1 4 4 0 .

2 1 3 6
1 0 0

0 1 0
1 0 2 3
B 1 4 4 0
2 1 3 6
1 0 0 1 0 2 3
0 0 1 2 1 3 6 EA
0 1 0 1 4 4 0
19
1 0 2 3
(c) Perform ERO R3 R3 (1) R1 on A, the resulting matrix is B 2 1 3 6 .

0 4 2 3
1 0 0

1 0 1
1 0 2 3 1 0 0 1 0 2 3
B 2 1 3 6 0 1 0 2 1 3 6 EA
0 4 2 3 1 0 1 1 4 4 0
Example 3: Find a sequence of elementary matrices that can be used to write the matrix A
0 1 3 5
in row-echelon form, where A 1 3 0 2 .

2 6 2 0
Solution:
Elementary row operation
R1 R2
R3 R3 (2) R1
R3
1
R3
2
Matrix
1 3 0 2
B 0 1 3 5
2 6 2 0
1 3 0 2
C 0 1 3 5
0 0 2 4
1 3 0 2
D 0 1 3 5
0 0 1 2
Elementary matrix
0 1 0
E1 1 0 0
0 0 1
1 0 0
E2 0 1 0
2 0 1
1 0 0
E3 0 1 0
0 0 1
2
Note
B E1 A
C E2 B
D E3 C
Now the three elementary matrices E1 , E2 , E3 are related to A and D by
1 0 0 1 0 0 0 1 0 0 1 3 5
D E3 E2 E1 A 0 1 0 0 1 0 1 0 0 1 3 0 2
0 0 1 2 0 1 0 0 1 2 6 2 0
20
Example 4: Find a sequence of elementary matrices that can be used to write the matrix A
1 3 1
1
3 0 3 .
in row-echelon form, where A 2
3 2 1 0
21
Theorem 1.5.2 If E is an elementary matrix, then E 1 exists and is an elementary matrix

Example 5:
1 0
1 0
exists and is an elementary matrix
E 1
E
1
0 2
0 2
0 1
0 1
E 1
E
1 0
1 0
1 0
1 0
E 1
E
2 1
2 1
Example 6: (Inverse ERO)

(a) The inverse ERO for Ri kRi is Ri 1k Ri .
1 0 0
For example, E 0 1 0 is obtained from I 3 using the ERO R3 2R3 , so

0 0 2
1 0 0
1
E 0 1 0 is obtained from I 3 using the ERO R3 12 R3
0 0 1
2
(b) The inverse ERO for Ri R j is Ri R j .
1 0 0
For example, E 0 0 1 is obtained from I 3 using the ERO R2 R3 , so

0 1 0
1 0 0
1
E 0 0 1 is obtained from I 3 using the ERO R2 R3
0 1 0
(c) The inverse ERO for Ri Ri kR j is Ri Ri (k ) R j .
1 0 0
For example, E 0 1 0 is obtained from I 3 using the ERO R3 R3 2R1 , so

2 0 1
1 0 0
1
E 0 1 0 is obtained from I 3 using the ERO R3 R3 (2) R1
2 0 0
22
Example 7:
Write down the inverse of the following 3 3 elementary matrices
1 0 0
(a) 0 3 0
0 0 1
1 0 0
(b) 0 1 0
4 0 1
1 0 0
(c) 0 0 1
0 1 0
Example 8:
Write down the inverse of the following 4 4 elementary matrices
0
(a)
0
0
1
0
0
0
0
1
0
0
0
0
(b)
0
0
0
0
1
0
0
1
0
1
0
0
(c)
0
0
1
0
0
0
0
a
0
0
0
Example 9:
Write down the inverse of the following elementary matrices
1 0 0
0 1 0
0 0 2
0
0
0
1
0
0
0
0
0
0
0 0 0 0
0 0 1 0
0 1 0 0
1 0 0 0
0 0 0 1
3
0
1
0
0
0
1 0 3
0 1 0
0 0 1
0
0
0
0
0
0
0
0
1
0
0
1
0
0
0 0 1
0 1 0
1 0 0
0
0
0 0 0 0
1 0 0 0
0 1 0 0
0 0 3 0
0 0 0 1
0
0
0
0
0
0
0
1
0
0
0 0
1 0
0 1
0 0
0 0
0
0
0
0
4
0
3 0
0 0
1 0
0 1
0
23
Example 10:
2
3
3
3
1
1
Consider the matrices A 1 4 1 , B 1 4 1

0
1
1
0
2
3
Find elementary matrices E1 , E 2 , E 3 , E 4 such that

(a) E2 E1 A B
(b) E4 E3 B A
Solution:
(a) E2 E1 A B
2
3
1
0
A 1 4 1 R1 R3 1
0
1
1
0
1 0 0
0 0
I 3 0 1 0 R1 R3 0 1
0 0 1
1 0
1
0
4 1 R1 R1 R3
2
3
0 E1
3
3
1
1 4 1 B
1
2
3
1 0 0
I 3 0 1 0 R1 R1 R3
0 0 1
1 0 1
0 1 0 E2
0 0 1
(b) E4 E3 B A B ( E4 E3 ) 1 A E31 E41 A E2 E1 A from part (a)
1
3
1 0 1
0 0 1
1
1
E2 E3 E 0 1 0 and E4 E1 E4 E1 0 1 0
0 0 1
1 0 0
1
2
24
Example 11:
3 2
0
1 2 0
Consider the matrices A 1 2 4 , B 0 0 4

1 2 0
0 3 2
Find elementary matrices E1 , E 2 , E 3 , E 4 such that

(a) E2 E1 A B
(b) E4 E3 B A
25
Theorem 1.5.3 (Equivalent Statements)

If A is an n n matrix, then the following statements are equivalent
(a) A is invertible
(b) A x 0 has only the trivial solution
~
(c) the reduced row-echelon form of A is I n

(d) A is expressible as a product of elementary matrices
Example 8:
1 2
Verify the Equivalent Statements (a), (b), (c), and (d) above for the matrix A
3 1
Solution:
1 2
A
3 1
(a) Since ad bc (1)(1) (2)(3) 5 0 , therefore A is invertible.
1 2 x1 0
and
(b) Consider A x 0 . Then
~
~
3 1 x 2 0
x1 2 x2 0
3x1 x2 0
By inspection x1 0, x2 0 . Therefore A x 0 has only the trivial solution
~
1 2
1 2
R2 R2 (3) R1
R2 ( 15 ) R2
(c) A
3 1
0 5
1 0
I 2
R1 R1 (2) R2
0 1
Therefore the reduced row echelon form of A is I 2
1 2
0 1
(d) From part (c), we have E3 E2 E1 A I 2 where
1 0
1 0
1 2
, E3
, E 2
and A ( E3 E2 E1 ) 1 .
E1
1
3 1
0 1
0 5
1
1
1
Thus A E1 E 2 E3
1 0 1 0 1 2 1 2

3
1
0
1
3
1
0
Therefore A is expressible as a product of elementary matrices
26
1.6)
Further Results on Systems of Equations and Invertibility

(a) A is invertible
~

(e) A x b is consistent for every n 1 column matrix b
~
(f) A x b has a unique solution x A b for every n 1 column matrix b

~
Example 1:
1
Verify the Equivalent Statements (e) and (f) above for the matrix A
3
Solution:
1
Consider the system A x b . The augmented matrix for the system is
~
~
3
Apply elementary row operations
1 2 b1
R2 R2 (3) R1
3 1 b2
b1
1 2
R2 ( 15 ) R2
0 5 b2 3b1
2 b1
.
1 b2
b1
1 2
1
0 1 5 (3b1 b2 )
1 0 15 (b1 2b2 )
R1 R1 (2) R2
1
0 1 5 (3b1 b2 )
Therefore x1 15 (b1 2b2 ) and x2 15 (3b1 b2 ) . This implies that the system
A x b is consistent for every n 1 column matrix b .
~
The solution x1 15 (b1 2b2 ) and x2 15 (3b1 b2 ) can be rewritten as
x
1 1 2 b1
A 1 b
x 1
~
~
5 3 1 b2
x2
Therefore A x b has a unique solution x A 1 b for every n 1 column matrix b
~
27
Example 2:
Find the condition that b's must satisfy for the system to be consistent
x1 2 x2 5x3 b1
4 x1 5x2 8x3 b2
3x1 3x2 3x3 b3
Solution:
1 2 5 b1
The augmented matrix for the system is 4 5 8 b2 .

3 3 3 b
3
1 2 5 b1
R2 R2 (4) R1
Apply ERO 4 5 8 b2
3 3 3 b R3 R3 3R1
3
1 2 5
R2 R2 0 1 4
0 3 12
1
3
1
3
b1
1 2 5
0 3 12 b2 4b1
0 3 12 b 3b
3
1
b1
(b2 4b1 ) R3 R3 3R2

b3 3b1
b1
1 2 5
1
0 1 4 3 (b2 4b1 )
0 0
0 b1 b2 b3
b1 b1

Therefore the system is consistent if b1 b2 b3 0 , ie b b2 b2

~
b b b
3 1 2
Note:
The system
A x b is only consistent if the b's satisfy the condition

~
b1 b2 b3 0 .
The system A x b is NOT consistent for every n 1 column matrix b
1 2 5
By the Equivalent Statements, the matrix A 4 5 8 is NOT invertible.

3 3 3
28
Example 3:
x1 2 x2 3x3 b1
2 x1 5x2 3x3 b2
8x3 b3
x1
Solution:
1 2 3 b1

1 0 8 b
3
1 2 3 b1
R2 R2 (2) R1
Apply ERO 2 5 3 b2
1 0 8 b R3 R3 (1) R1
3
b1
1 2 3
b2 2b1 R3 (1) R3
0 1 3
0 0 1 b 2b 5b
3
2
1
3
b1
1 2
0 1 3 b2 2b1 R3 R3 2R2
0 2 5
b3 b1
b1
1 2 3
b2 2b1
0 1 3
0 0 1 b 2b 5b
3
2
1
Therefore x3 b3 2b2 5b1 , x2 3b3 5b2 13b1 and x1 9b3 16b2 40b1 .
Note:
The system A x b is consistent for every n 1 column matrix b
1 2 3
By the Equivalent Statements, the matrix A 2 5 3 is invertible.

1 0 8
29
Example 4:
x1 2 x2 2 x3 b1
2 x1 3x2 5x3 b2
3x1 4 x2 8x3 b3
Solution
1 2 2 b1

3 4 8 b
3
Apply ERO
1 2 2 b1
R2 R2 (2) R1
2 3 5 b2
3 4 8 b R3 R3 3R1
3
b1
1 2 2
1 b2 2b1 R3 R3 2R2
0 1
0 2 2 b 3b
3
1
b1
1 2 2
b2 2b1
0 1 1
0 0 0 b 2b b
3
2
1
b1 b1

Therefore the system is consistent if b3 2b2 b1 0 , ie b b2 b2

~
b b 2b
2
3 1
Note:
The system
A x b is only consistent if the b's satisfy the condition

~
b3 2b2 b1 0 .
The system A x b is NOT consistent for every n 1 column matrix b
1 2 2
By the Equivalent Statements, the matrix A 2 3 5 is NOT invertible.

3 4 8
30
Example 5
Determine whether the coefficient matrix of the following system of linear equation is
invertible by checking whether the system is consistent for all bs
x1 x2 x3 b1
x3 b2
x1
2 x1 x2 3x3 b3
Solution
31
1.7)
Properties of Diagonal, Triangular and Symmetric Matrices
Diagonal Matrices
A diagonal matrix D is invertible if and only if its diagonal entries are nonzero
Example 1:
13 0

1
0
2
3 0
(a) D
0 2
0 0
(b) D
0 2
D 1 does not exist
Triangular Matrices
A triangular matrix is invertible if and only if its diagonal entries are nonzero
Example 2:
3 0
(a) L
1 2
L1 exists
0 3
(b) U
0 2
U 1 does not exist
Symmetric Matrices
Theorem 1.7.1 Let A and B be symmetric matrices and k any scalar. Then
(a) AT is symmetric
(b) A + B and A B are symmetric
(c) kA is symmetric
Example 3:
1 3
1 2
and B
.
Verify the above theorem for the matrices A
3 2
2 4
32
Solution:
T
1 3
1 3

AT AT is symmetric
(a) ( A )
3 2
3 2
T
1 3 1 2
2 5
2 5

(b) ( A B)
5 6
5 6
3 2 2 4
T
1 3

3 2
1 2
A B
2 4
A B is symmetric
T
1 3 1 2
0 1
0 1

( A B)
1 2
1 2
3 2 2 4
T
1 3 1 2
A B

3 2 2 4
A B is symmetric
(c) (kA)T kAT kA kA is symmetric

Theorem 1.7.2 If A is an invertible symmetric matrix, then A 1 is also symmetric
Proof:
( A1 )T ( AT ) 1 A1 A 1 is symmetric
Theorem 1.7.3 The matrices AAT and AT A are symmetric
Proof:
( AAT )T ( AT )T AT AAT AAT is symmetric

( AT A)T AT ( AT )T AT A AT A is symmetric
Theorem 1.7.4 Let A and B be two symmetric matrices. Then AB is symmetric if and only
if AB=BA
Proof:
() : Suppose AB is symmetric. Then AB ( AB)T B T AT BA
() : Suppose AB BA . Then ( AB)T B T AT BA AB implies that AB is symmetric.
33
Example 4:
1 2
1 3
and B
Verify the above theorem for A

2 3
3 4
Solution:
7 11
is symmetric,
Note that A and B are two symmetric matrices. Since AB
11 18
therefore we can conclude that AB BA .
Example 5:
1 3
4 1
and B
commute.
Determine whether the matrix A
3 2
1 2
Solution:
5
1
is not symmetric,
Note that A and B are two symmetric matrices. Since AB
10 1
therefore AB BA and the matrix A and B do not commute.
Example 6:
3 1
1 2
and B
commute.
Determine whether the matrix A
1 2
2 3
Solution:
34
CHAPTER 2
2.1)
DETERMINANTS
Determinants by Cofactor Expansion
Minors, Cofactors and Adjoint of a Matrix

If A is a square matrix, then the minor M ij of the element a ij is the determinant of the
matrix obtained by deleting the i th row and j th column of A. The cofactor C ij is given
by Cij (1) i j M ij . The matrix of cofactors has the form
C11 C12
C 21 C 22

C
n1 C n 2
C1n
C2n

C nn
The transpose of this matrix is called the adjoint of A, denoted as adj(A), that is
C11 C12
C 22
C
adj(A) 21
C
n1 C n 2
C1n
C2n

C nn
Theorem Let A be an n n square matrix. Then the determinant of A is given by

n
det( A) aij Cij
(i th row expansion) or
j 1
n
det( A) aij Cij
(j th column expansion)
i 1
Example 1:
a11 a12
Suppose A a 21 a 22
a
31 a32
a13
a 23 . Write out the formula for the determinant of A using

a33
various rows or columns expansion.
35
Solution:
1st row expansion:
det( A) a11C11 a12C12 a13C13
2nd row expansion:
det( A) a21C21 a22C22 a23C23
3rd row expansion:
det( A) a31C31 a32C32 a33C33
1st column expansion:
det( A) a11C11 a21C21 a31C31
2nd column expansion:
det( A) a12C12 a22C22 a32C32
3rd column expansion: det( A) a13C13 a23C23 a33C33

Example 2
1 2 2
Let A 4 3 2 . Evaluate det(A) by various cofactor expansions

5 0 3
Solution:
1st row expansion: det( A) a11C11 a12C12 a13C13
(1)C11 (2)C12 (2)C13

(1)(1)11
3 2
0
(2)(1)1 2
(2)(1)13
5 0
17
2nd row expansion: det( A) a21C21 a22C22 a23C23
(4)C21 (3)C22 (2)C23

(4)(1) 21
2 2
0 3
(3)(1) 2 2
1 2
5 3
(2)(1) 23
1 2
5 0
17
36
3rd row expansion: det( A) a31C31 a32C32 a33C33

(5)C31 (0)C32 (3)C33
(5)(1) 31
(0)C32 (3)(1) 33
3 2
(5)(10) (0)C32 (3)(11)
17
1 2
4
1st column expansion: det( A) a11C11 a21C21 a31C31
(1)C11 (4)C21 (5)C31

(1)(1)11
3 2
0
(4)(1) 21
2 2
0 3
(5)(1) 31
3 2
17
2nd column expansion:
det( A) a12C12 a22C22 a32C32
3rd column expansion: det( A) a13C13 a23C23 a33C33
37
Example 3:
Evaluate det(A) by an appropriate cofactor expansion
1 2
0
2
(b)
3 6
3 4
1 2 3
(a) A 2
0
1
3 4 5
0 3
1 3
0 1
3
0
(c)
2 2
4
1
5 6
0 5
2 2
0 0
3 2
0 3
Solution:
(a) det( A) a21C21 a22C22 a23C23 (using 2nd row expansion)
(2)C21 (0)C22 (1)C23
(2)(1) 21
(0)C 22 (1)(1) 23
4 5
(2)(2) (0)C22 (1)(2)
6
(b) We use 3rd column expansion (Why?)
det( A) a13C13 a23C23 a33C33 a43C43

(0)C13 (0)C23 (2)C33 (0)C43
1
(2)(1) 33 2
3
(2)(6)
12
2
0
3
1
(from part (a))
(c) (Answer 51)
38
Theorem 2.1.1 If A is an n n invertible matrix, then A 1
1
adj(A)
det( A)
a b
. Then
Example 4: Let A
c d
M 11 d ; C11 d
M 12 c ; C12 c
M 21 b ; C21 b
M 22 a ; C22 a
det( A) a11C11 a12C12 (a)(d ) (b)(c) ad bc

1
1
adj(A)
A
det( A)
det( A)
1
C11 C12
1

det( A)
C 21 C 22
C11 C 21
1

C12 C 22 ad bc
d b
c a
1 2 2
Let A 4 3 2 . Find A 1
5 0 3
Example 5:
Solution:
M 11
3 2
9 ; C11 9 M 12
M 21
2 2
6 ; C21 6 M 22
M 31
M 33
0 3
3 2
1 2
4 3
4 2
5
1 2
5 3
10 ; C31 10 M 32
2 ; C12 2 M 13
7 ; C22 7
4 2
M 23
4 3
5 0
1 2
5 0
15 ; C13 15
10 ; C23 10
6 ; C32 6
11 C33 11
C11 C12
adj(A) C 21 C 22
C
31 C32
6 10
C13
2 15
9
9
6
C 23 6 7 10 2 7
15 10 11
10 6 11
C33
det(A) a11C11 a12C12 a13C13 17 (see Example 2)
6 10
9
1
1
6
adj( A) 2 7
A
17
det( A)
15 10 11
1
39
Theorem 2.1.2 (Cramers Rule) Let A x b be a system of n linear equations in n

~
unknowns such that det( A) 0 . Then the solution to the system is given by x j
det( A j )
,
det( A)
where A j is the matrix obtained by replacing the entries in the jth column of A by the
entries in b
~
Example 6: Use Cramers rule to solve the following system of linear equations.
x1 2 x2 3x3 0
x3 3
2x1
3x1 4 x2 5 x3 0
Solution:
x1
0 4
a 21C 21
(3)(1) 21
1 0 3
x2
x3
4 0
a 22C 22
a 23C 23
(3)(1) 2 2
1 3
3
(3)(1) 23
6
40
Example 7
Use Cramers rule to solve the following system of linear equations.
(a)
2 x1 2 x2 x3 2
x1 x2 3x3 4
3x1 2 x2 x3 7
(b) x1 2 x2 x3 2
x1 x2 3x3 1
3x1 2 x2 x3 6
(Answer: (a) x1 1, x2 1, x3 2 (b) x1 1, x2 1, x3 1 )

Solution:
41
2.2)
Evaluating Determinants by Row Reduction
Theorem 2.2.1 Let A be a square matrix. Then

(a) If A has a row or a column of zeros, then det(A) = 0
(b) det(A) det( AT )
(c) If A has a row (or column) which is a multiple of another row (or column), then
det(A) = 0
Example 1:
1 4
1 0
det(A) = 0, A
det(A) = 0
(a) A
0
0
2
0
0 1 5
(b) Let A 3 6 9 . Verify that det(A) det( AT )

2 6 1
det(A) a11C11 a12C12 a13C13

(0)C11 (1)C12 (5)C13
(0)C11 (1)(1)1 2
3 9
(5)(1)13
2 1
(0)C11 (1)(15) (5)(30)
165
3 6
2
0 3 2
A 1 6 6
5 9 1
det( AT ) a11C11 a12C12 a13C13

(0)C11 (3)C12 (2)C13
(0)C11 (3)(1)1 2
1 6
(2)(1)13
5 1
(0)C11 (3)(29) (2)(39)
165
1 6
5
det(A) det( AT )
42
1
5
0
(c) Let A 3 6 9 . Show that det(A) = 0

6 12 18
det(A) a11C11 a12C12 a13C13

(0)C11 (1)C12 (5)C13
(0)C11 (1)(1)1 2
6 18
(0)C11 (1)(0) (5)(0)
0
(5)(1)13
6 12
Theorem 2.2.2 Let A (aij ) be an n n triangular matrix. Then det( A) a11a22 ...ann
Example 2:
2 0 0
4 0
(a) A 0
3 1 5
2 5 3
4 1
(b) A 0
0
0 0
5 0 0
(c) A 0 3 0
0 0 4
det(A) = (2)(4)(5) 40
det(A) = (2)(4)(0) 0
det(A) = (5)(3)(4) 60
Theorem 2.2.3 Let A and B be square matrices

(a) If B is obtained from A by interchanging two rows(columns) of A, then
det(B) det( A)
(b) If B is obtained from A by multiplying a row(column) of A by a nonzero constant k,
then
det(B) k det( A)
(c) If B is obtained from A by adding a multiple of a row(column) of A to another
row(column) of A, then
det(B) det( A)
43
Example 3:
a b
(a) Let A d e
g h
c
d e
f and B a b
g h
i
c .
i
Here B is obtained from A by interchanging two rows of A
d e
Hence det(B) det( A) , that is a b
g h
a b
(b) Let A d e
g h
f
a b
c d e
c
f
c
ka kb kc
f and B d e c .
g h i
i
Here B is obtained from A by multiplying a row of A by a nonzero constant k
ka kb kc
a b
Hence det(B) k det( A) , that is d e f k d e
g h i
g h
a b
(c) Let A d e
g h
c
f
i
c
a kd b ke c kf
f and B d
e
f
g
i
h
i
Here, B is obtained from A by performing the ERO R1 R1 kR2
a kd b ke c kf
a b
d
e
f d e
Hence det(B) det( A) , that is
g
h
i
g h
c
f
i
44
Example 4:
3 6 9
1 2 3
(b) 4 5 6 3 4 5 6
7 8 9
7 8 9
1 2 3
4 5 6
(a) 4 5 6 1 2 3
7 8 9
7 8 9
9 12 15 1 2 3
(c) 4 5 6 4 5 6
7 8 9
7 8 9
(Apply R1 R1 (2) R2 on the left matrix)
a b
Example 5: Given that d e
g
d e
(a) g h
a b
2a
(b) d
3g
f
a b
i g h
e
c
f 3.
c
a b
i d e
f
g h
2b
e
2c
a b
f (2)(3) d e
3h 3i
a g bh ci a b
e
f d e
(c) d
g
h
i
g h
c
f 3
(Interchange rows two times)
c
f (2)(3)(3) 18 (Factoring two times)
i
c
f 3
( R1 R1 (1) R3 )
Using Cofactor Expansion and Theorem 2.2.3 we can evaluate the determinant of
matrices in a less tedious way.
Example 6:
Show that
bc ca ba
b
c 0
(a) a
1
1
1
1
(b) a
a3
1
b
1
c (b c)(c a)(a b)(a b c)
b3
c3
45
Solution:
bc ca ba abc bca cba

b
c
a
b
c
0
(a) a
1
1
1
1
1
1
(using Theorem 2.2.3)
(Note: ( R1 R1 R2 ) )
1
(b) a
1
b
1
1
c a
c
0
ba
b a
3
0
ca
3
c a
3
(using Theorem 2.2.3)

3
(1)[(b a)(c 3 a 3 ) (c a)(b 3 a 3 )]
(using cofactor expansion)
(b a)(c a)[(c 2 ca a 2 ) (b 2 ba a 2 )]
(b a)(c a)(c 2 b 2 ca ba)
(b a)(c a)(c b)(c b a)
(b c)(c a)(a b)(a b c)
Example 7:
a b c
Show that b c a 3abc a 3 b 3 c 3
c a b
Solution: Use cofactor expansion along the 1st row
a b c
b c a a11C11 a12C12 a13C13
c a b
(a)C11 (b)C12 (c)C13
(a)(1)11
c a
a b
(b)(1)1 2
3abc a b c
3
b a
c b
(c)(1)13
b c
c a
46
Example 8
Evaluate the following determinant
1 0 3
(a) 5 1 1
0 1 2
1 1
(b) 1 0
0
1
1 2 3
(c) 2 3 1
3 1 2
5 2 2
(d) 1 1 2
3 0 0
4 1 3
(e) 2 2 4
1 1 0
(Answer: (a) 16 (b) 0 (c) 18 (d) 6 (e) 12 )

Solution
47
We now look at the problem of finding the determinant of 4 4 matrices

Example 9: Evaluate the determinant
0
0
(a)
0
2
2
1
(b)
1
1
0
0
4
1
2
1
1
5
0
2 0
3
1 0
3
1 4
6
5 1
4 2 2
2
3 1
2
1
3 1
3
1
2 1
2
1
0
0
0
2
0 2 0
3 1 3
3 1 3
6 5 6
0
1
1
4
0
2
2
0
0
3
4
1
0
0
0
2
0
2 0
0
1 3
4
1 3
1
5 6
0
0
4
1
0
0
(2)(3)(4)(2) 48
0
2
C 2 C 2 (2)C1
C3 C3 C1
C C C
4
1
4
2
1
1
1
0 0
0
1 0
0
1 0
1
4 8 11
C3 C3 (2)C 2
C 4 C 4 (3)C 2
2
1
1
1
0 0
0
1 0
0
1 1
0
4 11 8
(C3 C4 )
(2)(1)(1)(8) 16
0
0
(c)
0
g
0 0
0 b
d e
h i
a
g
c
0
f
0
j
0
h i
0 b
d e
0 0
j
g
c
0
f
0
a
0
h i
d e
0 b
0 0
j
f
gdba
c
a
48
Example 10
Evaluate the following determinant
0 2 4 5
3 0 3 6
(a)
2 4
5 7
5 1 3 1
2
1
(b)
0
2
6
4
1
2
3 1
2 2
1 4
1 3
(Answer: (a) 585 (b) 8 )

Solution:
49
2.3) Properties of the Determinant Function

Theorem 2.3.1 Let A and B be n n square matrices. Then
(a) det( AB) det( A) det(B)
(b) det(kA) k n det( A)
1
(c) If A is invertible, then det( A 1 )
det( A)
Example 1:
1 4
1 2
, B
and k 3
Verify the two theorems above for A
2 3
3 4
Solution:
1 4 1 2
13 18
det
(13)(16) (18)(11) 10
det( AB ) det
2 3 3 4
11 16
1 4
(1)(3) (4)(2) 5
det( A) det
2 3
1 2
(1)(4) (2)(3) 2
det(B) det
3 4
3 12
27 72 45 and 32 det( A) 32 (3 8) 45
det(3 A) det
6 9
3 4
1
1
1
( 15 ) 2 (3 8) and
det( A 1 ) det 15
5
det( A) 5
2 1
Example 2:
a b
Let A d e
g h
f where det( A) 6
i
(a) det(3 A) 33 det( A) 33 (6) 162
(b) det( A 1 )
1
1
det( A) 6
1 4
(c) det(2 A 1 ) 2 3 det( A 1 ) 2 3.
6 3
50
(d) det((2 A) 1 )
(e) det b
c
g
h
i
1
1
1
1
3
3
det(2 A) 2 det( A) 2 .6 48
d
a
e det b
c
f
d
e
f
h det( AT ) det( A) 6
i
Example 3:
Let A be a 4 4 matrix where det( A) 3
(a) det(4 A)
(b) det( A 1 )
(c) det(3 A1 )
(d) det((3 A) 1 )
Example 4
Suppose A and B are two n n matrices such that det( A) 3 and det(B) 2 . Then
(a) det( AB) det( A) det(B) (3)(2) 6
(b) det( A2 ) det( AA) det( A) det( A) (3)(3) 9
(c) det(B 1 A) det(B 1 ) det( A)
1
3
det( A)
det(B)
2
(d) det(3BT ) 3n det(B T ) 3n det(B) 3n (2) (2)3n

(e) det( AAT ) det( A) det( AT ) det( A) det( A) (3)(3) 9
51

(a) A is invertible
~

(e) A x b is consistent for every n 1 column matrix b
~
(f) A x b has a unique solution x A b for every n 1 column matrix b

~
(g) det( A) 0
Example 4:
Verify the Equivalent Statements (a) and (g) for the matrices
1 2
A
3 1
1 3
and B
2 6
Solution:
det(A) (1)(1) (2)(3) 0 A is invertible
det(B) 0 B is not invertible
Example 5:
Determine the value of k such that the matrix A k 2

0
Solution:
k 0
2 k is invertible
k k
det( A) k (2k k 2 ) k (k 2 k k.0) k 2 (2 k k 2 ) k 2 (2 k )(1 k )

Let det( A) 0 . Then k 2 (2 k )(1 k ) 0 . By the Equivalent Statements, A is invertible
if and only if det( A) 0 . Hence k 2,0,1
We can check that
2 2 0
0 0 0
1 1 0
det( 4
2 2 ) 0 , det( 0 2 0 ) 0 , and det( 1 2 1 ) 0
0 2 2
0 0 0
0 1 1
52
Example 6:
Determine whether the following is TRUE or FALSE
b c c a b a
(a) The matrix a

b
c is not invertible
1
1
1
(Answer: TRUE)
(b) The matrix a

a3
(Answer: TRUE)
1
(c) The matrix
1
4 2 2
3 1
2
is invertible
3 1
3
2 1
2
1
b
3
c is not invertible if a b c 0
c 3
(Answer: TRUE)
53
2.4)
A Combinatorial Approach to Determinants
Let A be a square matrix. The determinant of A is denoted as A or det(A) .
a12
a
, then det(A) = a11a22 a12a21
If A 11
a 21 a 22
a11 a12
If A a 21 a 22
a
31 a32
a13
a 23 , then det(A) can be found by first writing out the matrix in the
a33
a11 a12
following manner: a 21 a 22
a
31 a32
a13 a11 a12
a 23 a 21 a 22
a33 a31 a32
Then det(A) =
0 1 5
Example 1: A 3 6 9
2 6 1
0 1 5 0 1
First write out the matrix in the following manner 3 6 9 3 6

2 6 1 2 6
Then det(A) =
Example 2:
Evaluate the following determinant using the combinatorial approach discussed in this
section
1 0 3
(a) 5 1 1
0 1 2
1 1 0
(b) 1 0 1
0 1 1
1 2 3
(c) 2 3 1
3 1 2
5 2 2
(d) 1 1 2
3 0 0
4 1 3
(e) 2 2 4
1 1 0
(Answer: (a) 16 (b) 0 (c) 18 (d) 6 (e) 12 )

54
55
Example 3: Connection between the above formula and cofactor expansion
a b
In evaluating the determinant of the matrix A d e

g h
a b
the form d e
g h
f , we first arrange the matrix in

i
ca b
f d e
i g h
Then det( A) (aei bfg cdh) (bdi afh ceg )

a(ei fh) b( fg di) c(dh eg )
b(
)c
(a)M 11 (b)(M12 ) (c)M13

(a)C11 (b)C12 (c)C13
a11C11 a12C12 a13C13
(This is the cofactor expansion along the 1st row, discussed in Section 2.1)
Problem: Starting as above, but with different pairing, show that the other 5 cofactor
expansion formulas could be obtained
56
CHAPTER 3
EUCLIDEAN VECTOR SPACES
3.0) Vectors in 2-space and 3-space

The set of all ordered n-tuples a1 , a2 ,..., an is called n-space and is denoted by R n
a1 , a2 is an ordered pair
a1 , a2 , a3 is an ordered triple
The length of a vector u in 2-space is called the norm of u and is denoted as u . If
~
u u1 ,u 2 , then u u12 u 22 .
~
In 3-space, if u u1 , u 2 , u3 , then u u12 u 22 u32

~
If u and v are two vectors in 2-space or 3-space, and is the angle between the two
~
vectors, then the dot product or Euclidean inner product is defined by u .v u v cos .
~ ~
The dot product can also be computed using the formula u .v u1v1 u2 v2 for 2-space
~ ~
and u .v u1v1 u2 v2 u3v3 for 3-space.

~ ~
Orthogonal projection
Suppose two vectors u and a are positioned so that their initial points coincide at a
~
point. We can decompose the vector u as shown below

~
In the above, the vector v is called the orthogonal projection of u on a or the vector
~
component of u along a . It is denoted as proja u .

~
The vector w is called the vector component of u orthogonal to a . This vector can
~
written as w u proja u .
~
57
Theorem 3.0.1 proja u

~
u .a
~
2 ~
a
~
Example 1:
Let u 2,1 and a 3,2 .
~
Then,
u .a u1a1 u2 a2 (2)(3) (1)(2) 4 , and
~
a a12 a22 32 2 2 13
~
The vector component of u along a , proja u

~
u .a
~
a 134 3,2
~
2 ~
The vector component of u orthogonal to a is

~
u proja u u
~
u.a
~
a 2,1 134 3,2 137 2,3
~
2 ~
Example 2:
Let u 2,1,3 and a 4,1,2 .
~
Then,
u .a u1a1 u2 a2 u3 a3 (2)(4) (1)(1) (3)(2) 15 , and
~
a a12 a22 a32 4 2 (1) 2 2 2 21

~
The vector component of u along a , proja u

~
u .a
a 15
4,1,2 75 4,1,2
21
~ ~
2 ~
a
~
The vector component of u orthogonal to a is

~
u proja u u
~
u .a
a 2,1,3 75 4,1,2 17 6,2,11
~ ~
2 ~
a
~
58
Cross Product
If u u1 , u 2 , u3 and v v1 , v2 , v3 , then the cross product or vector product u v is
~
the vector which can be represented by

i
u v u1
u2
u3
v1
v2
v3
Example 3:
Let u 1,1,3 , v 2,0,3 , and w 2,3,5 . Compute
~
(b) u (v w)
(a) v w
~
(c) (u v) w
(d) (u v) (v w)
(e) u .(v w)
Solution:
i
k
~
(a) v w 2 0 3 i
~
2 3
5
i
(b) u (v w) 1
~
j
~
0 3
3
j
~
2 3
2
k
~
2 0
2 3
9 i 16 j 6 k
~
k
~
9 16 6
59
Example 4:
Suppose u and v are two vectors in 3-space Show that u .(u v) 0 and v .(u v) 0
~
hence deduce that u v is orthogonal to both u and v .

~
Solution:
Let u u1 , u2 , u3 and v v1 , v2 , v3 . Then
~
k
~
u v u1
u2
u3 i
v1
v2
v3
u2
u3
v2
v3
j
~
u1 u3
v1
v3
u1 u 2
~ v
v2
1
(u2 v3 u3v2 ) i (u1v3 u3v1 ) j (u1v2 u2 v1 ) k

~
Therefore,
u .(u v) (u1 )(u2 v3 u3v2 ) (u2 )((u1v3 u3v1 )) (u3 )(u1v2 u2 v1 ) 0
~
v .(u v)
~
Scalar Triple Product

The scalar triple product of u u1 , u 2 , u3 , v v1 , v2 , v3 and w w1 , w2 , w3
~
can be calculated from the formula
u1
u .(v w) v1
~ ~
~
w1
u2
v2
u3
v3
w2
w3
Example 5:
Use the above formula and the properties of determinant to deduce that u .(u v) 0 and
~
v .(u v) 0
~
Solution:
60
3.1) Euclidean n-space

Let u u1 , u 2 ,..., u n and v v1 , v2 ,..., vn where u , v R n
~
Then
u v u1 v1 , u 2 v2 ,..., u n vn
~
k u ku1 , ku2 ,..., kun

~
u u12 u 22 ... u n2
Euclidean norm
u . v u1v1 u 2 v2 ... u n vn
Euclidean inner product
~ ~
Note that u . u 0 with equality if and only if u 0

~
d (u, v) u v (u1 v1 ) 2 (u 2 v2 ) 2 ... (u n vn ) 2
Euclidean distance
~ ~
Example 1:
Let u 3,1,2 and v 1,4,2 . Find
~
(a) the Euclidean norm u and v

~
(b) the Euclidean inner product u . v

~
(c) the Euclidean distance between u and v

~
Solution:
(a) u 32 (1) 2 (2) 2 14
~
v 12 (4) 2 2 2 21
~
(b) u . v (3)(1) (1)(4) (2)(2) 3

~ ~
(c) d (u, v) u v (3 1) 2 (1 (4)) 2 (2 2) 2 29

~ ~
61
Theorem 3.1.1 (Cauchy-Schwarz Inequality)

u .v u v
~ ~
Example 2:
Let u 3,1,2 and v 1,4,2 . Verify that the Cauchy-Schwarz Inequality holds.
~
Solution:
u .v 3 3
~ ~
u 14
v 21
Since 3 14 21 , therefore u . v u v and the Cauchy-Schwarz Inequality holds.

~ ~
Theorem 3.1.2 (Triangle Inequality) Let u , v , w R n . Then

~
u v u v
~
and d (u, v) d (u, w) d ( w, v)

~ ~
Example 3:
Let u 3,1,2,2 , v 4,2,1,3 and w 0,3,8,2
~
(a) Verify the triangle inequality u v u v

~
(b) Show that d (u, v) d (u, w) d ( w, v)

~ ~
Solution:
62
Theorem 3.1.3 Let u , v R n . Then

~
u .v
~ ~
2
1
1
u v u v
4 ~ ~
4 ~ ~
Example 4:
Let u and v be such that u v 2 and u v 4 . Find u . v
~
Solution:
Orthogonality
Two vectors u , v R n are orthogonal if u . v 0
~
~ ~
Let u , v R n be two orthogonal vectors. Then u v

~
u v
~
Example 5:
Let u 0,3,3,2 , v 4,3,1,3 .
~
(a) Show that u and v are orthogonal.

~
(b) Verify that u v

~
u v
~
Solution:
63
Linear Transformation from R n to R m
3.2)
x x1 , x2 ,..., xn R n
~
w w1 , w2 ,..., wm R m
~
T : R n R m is a transformation
The transformation can also be written as
T x1 , x2 ,..., xn w1 , w2 ,..., wm
f1 ( x1 , x2 ,..., xn ), , f 2 ( x1 , x2 ,..., xn ) ,, f m ( x1 , x2 ,..., xn )
Now if f1 ( x1 , x2 ,..., xn ) , f 2 ( x1 , x2 ,..., xn ) ,, f m ( x1 , x2 ,..., xn ) are all linear functions of
x1 , x2 ,..., xn , then
T : R n R m is a linear transformation
T : R n R n is called a linear operator

Example 1:
1) The equations
w1 x1 x2 f1 ( x1 , x2 )
w2 3x1 x2 f 2 ( x1 , x2 )
w3 x12 x22 f 3 ( x1 , x2 )
define a transformation T : R 2 R 3 where T x1 , x2 x1 x2 ,3x1 x2 , x12 x22
This transformation is not linear
64
2) The equations
w1 2 x1 3x2 x3 5x4 f1 ( x1 , x2 , x3 , x4 )
w2 4 x1 x2 2 x3 x4 f 2 ( x1 , x2 , x3 , x4 )
w3 5x1 x2 4 x3 f 3 ( x1 , x2 , x3 , x4 )
define a linear transformation T : R 4 R 3 where
T x1 , x2 , x3 , x4
2 3 1 5
The matrix A 4 1 2 1 is called the standard matrix for T

5 1 4
0
Some simple linear transformation

Example 2: (Reflection about a line)
T : R2 R2
(a) About the y-axis
w1 x1
w2 x 2
w 1 0 x1
1 0
Standard matrix A
1
0 1
w2 0 1 x2
1 0
(b) About the x-axis Standard matrix A

0 1
0 1
(c) About the line y x Standard matrix A

1 0
0 1
(d) About the line y x Standard matrix A

1 0
65
Example 3: (Reflection about a plane)
T : R3 R3
(a) About the xy-plane
w1 x1
w2 x 2
w3 x3
w1 1 0 0 x1
1 0 0

w2 0 1 0 x 2 Standard matrix A 0 1 0
0 0 1
w 0 0 1 x
3
1 0 0
(b) About the xz-plane Standard matrix A 0 1 0

0 0 1
1 0 0
(c) About the yz-plane Standard matrix A 0 1 0

0 0 1
66
Example 4: (Orthogonal projection)

(a) T : R 2 R 2
(i) on the x-axis
1 0
Standard matrix A
0 0
(ii) on the y-axis
0 0
Standard matrix A
0 1
(b) T : R 3 R 3
(i) on the xy-plane
1 0 0
Standard matrix A 0 1 0
0 0 0
(ii) on the xz-plane
1 0 0
0 0 1
(iii) on the yz-plane
0 0 0
0 0 1
67
Example 5: (Rotation)
(a) T : R 2 R 2
cos
Standard matrix A
sin
sin
cos
(b) T : R 3 R 3
(i) counter-clockwise about the positive x-axis through an angle
0
1
Standard matrix A 0 cos

0 sin
sin
cos
(ii) counter-clockwise about the positive y-axis through an angle
cos
Standard matrix A 0
sin
0 sin
1
0
0 cos
(iii) counter-clockwise about the positive z-axis through an angle
cos
Standard matrix A sin

0
sin
cos
0
0
1
Example 6: (Dilation & Contraction)

(a) T : R 2 R 2
(b) T :
R3
R3
k 0
Standard matrix A
0 k
k 0 0
Standard matrix A 0 k 0
0 0 k
68
Composition of Linear Transformation

Let T1 : R n R k and T2 : R k R m be two linear transformation with associated
standard matrix A1 and A2 respectively. Then the standard matrix for the composition of
linear transformation T2T1 : R n R m is given by A2 A1
Example 7: Find the standard matrix for the composition of linear transformation in R 2
a) A rotation of 90 0 , followed by a reflection about the line y x
b) An orthogonal projection on the y-axis, followed by a contraction with factor k
c) A reflection about the x-axis, followed by a dilation with factor k 3
1
2
Solution:
cos 90 0
A1
0
sin 90
(a) Here, T1 is a rotation of 90 0
sin 90 0
cos 90 0
0 1
T2 is a reflection about the line y x A2

1 0
The standard matrix for the composition is

0 1 cos 90 0

A A2 A1
0
1 0 sin 90
sin 90 0 1 0

cos 90 0 0 1
(b)
(c)
69
3.3)
Properties of Linear Transformation from R n to R m
A linear transformation T : R n R m is one-to-one if T maps distinct vectors in R n

into distinct vectors in R m . Let u , v R n . Then T is one-to-one if
~
T (u ) T (v) u , v R n
~

Let T : R n R n be a linear transformation and let A be the standard matrix for T.
Then the following statements are equivalent
(a) A is invertible
(b) the range of T is R n
(c) T is one-to-one
Example 1:
Verify the above theorem for the following linear operator
(i) orthogonal projection on the x-axis in R 2
(ii) reflection about the line y x in R 2
(iii) rotation about the z-axis in R 3
Solution:
1 0
. Note that A is not invertible.
(i) A
0 0
Since every point in R 2 is projected on the x-axis, therefore the range of T is the x-
axis and not R 2 .

Since T a, b a,0 T a, c for any b and c, therefore T is not one-to-one
0 1
. Note that A is invertible.
(ii) A
1 0
Since points in R 2 are reflected to points in R 2 , therefore the range of T is R 2 .
Since every point is reflected to a distinct point, therefore T is one-to-one.
70
cos
(iii) A sin
0
sin
cos
0 . Since det( A) 1 0 , A is invertible.

0
1
Since points in R 3 are rotated into points in R 3 , therefore the range of T is R 3 .
Since every point is rotated into a distinct point, therefore T is one-to-one.
Inverse Operator
Let T : R n R n be a linear operator and let A be the standard matrix for T. Then
T 1 : R n R n is also a linear operator and the standard matrix for T 1 is A 1

Example 2:
Determine whether the linear operator T : R 3 R 3 defined by the equations
w1 x1 2 x2 2 x3
w2 2 x1 x2 x3
w3 x1 x2
is one-to-one. If so, find the standard matrix for the inverse operator T 1
Solution:
1 2 2
The standard matrix is A 2 1 1 . Since det( A) 1 0 , A is invertible.

1 1 0
By the Equivalent Statements, T is one-to-one.
The standard matrix for the inverse operator T
is A
71
Theorem 3.3.2 Let u , v R n . Then T : R n R m is a linear transformation if and

~
only if (a) T (u v) T (u) T (v) (b) T (c u ) cT (u )

~
Example 3:
Determine whether T : R 3 R 2 is a linear transformation where
T x, y, z x, x y z
Solution:
Let u u1 , u 2 , u3 and v v1 , v2 , v3
To show whether T is a linear transformation, we need to check the two conditions
(a) T(u v) T (u) T (v) and (b) T (c u) cT ( u)
Condition(a)
T(u v) T u1 v1 , u 2 v2 , u3 v3 u1 v1 , u1 v1 u 2 v2 u3 v3
T (u) T (v) T u1 , u 2 , u3 T v1 , v2 , v3
u1 , u1 u 2 u3 v1 , v1 v2 v3
u1 v1 , u1 v1 u 2 v2 u3 v3
T(u v) T (u) T (v)

Condition (b)
T (c u) T cu1 , cu 2 , cu 3
cu1 , cu1 cu 2 cu3
c u1 , u1 u2 u3
cT ( u)
T (c u) cT ( u)
T is a linear transformation.
72
Theorem 3.3.3 Let T : R n R m be a linear transformation.
1
0
0

1
0
0
Let e1 , e2 ,, en . Then the standard matrix for T is A =
~
~
~

0
1
0

T (e1 ) T (e2 ) T (en )

~
~
~
( e1 , e2 ,..., en are called standard basis vectors)
~
For the transformation T : R 2 R 2

1 a
0 c
Suppose T (e ) T and T (e ) T , then
~1
~2
0 b
1 d
a c
the standard matrix for T is A T (e1 ), T (e2 )

~
~
b d
For the transformation T : R 3 R 3
1 a
0 d
0 g

Suppose T (e ) T 0 b , T (e ) T 1 e and T (e ) T 0 h , then
~2
~3
~1
0 f
1 i
0 c

a
b
the standard matrix for T is A T (e1 ), T (e2 ), T (e3 )
~
~
~
c
d
e
f
h
i
73
Example 4: (Reflection about the y-axis)

1 1
0 0
Since T (e1 ) T and T (e2 ) T ,
~
~
0 0
1 1
1 0

~
~
0 1
Example 5: (Reflection about the xy-plane)
1 1
0 0
0 0

Since T (e1 ) T 0 0 , T (e2 ) T 1 1 and T (e3 ) T 0 0 ,
~
~
~
0 0
0 0
1 1

1 0 0
the standard matrix for T is A T (e1 ), T (e2 ), T (e3 ) 0 1 0

~
~
~
0 0 1
Example 6: (Orthogonal projection on the x-axis)
Example 7: (Orthogonal projection on the xy-plane)
Example 8: (Rotation)
1 cos
0 sin
and T (e2 ) T
,
Since T (e1 ) T
~
~
0 sin
1 cos
cos
~
~
sin
sin
cos
74
CHAPTER 4
4.1)
GENERAL VECTOR SPACES
Real Vector Spaces
Let V be a non-empty set of objects. If the following 10 axioms are satisfied, then V is
called a vector space.
Let u , v, w V
~
Axiom 1: If u, v V then u v V
~ ~
Axiom 2: u v v u
~
Axiom 3: u (v w) (u v) w
~
Axiom 4: 0 V such that 0 u u 0 u u V

~
Axiom 5: u V such that ( u ) u u ( u ) 0 u V

~
Axiom 6: If u V and k scalar, then k u V

~
Axiom 7: k (u v) k u k v
~
Axiom 8: (k l ) u k u l u
~
Axiom 9: k (l u ) (kl) u
~
Axiom 10: 1 u u
~
Example 1:
Let V denote the set of all triples of real numbers x, y, z with the operations
x, y, z x, y , z x x, y y , z z and k x, y, z 0,0,0
Determine whether V is a vector space. If V is not a vector space, list all axioms that fail
to hold.
Solution:
Let u u1 , u 2 , u3 , v v1 , v2 , v3 , w w1 , w2 , w3 V
~
~ ~
Axiom 2: u v v u
~
~
Axiom 4: 0 0,0,0 V such that 0 u u 0 u u V

~
Axiom 5: u u1 ,u 2 ,u3 V such that ( u ) u u ( u ) 0 u V

~
75
Axiom 6: If u V and k scalar, then k u ku1 , ku2 , ku3 V

~
Axiom 7: k (u v) k u1 v1 , u 2 v2 , u3 v3 0,0,0
~
k u k v 0,0,0 0,0,0 0,0,0

~
Axiom 8: (k l ) u 0,0,0
k u l u 0,0,0 0,0,0 0,0,0
Axiom 9: k (l u) k 0,0,0 0,0,0
(kl) u 0,0,0
Axiom 10: 1 u 0,0,0 u Axiom FAILS

~
V is NOT a vector space.

Example 2:
Let V denote the set of all pairs of real numbers x, y with the operations
x, y x, y x x 1, y y 1 and k x, y kx, ky
to hold.
Solution:
Let u u1 ,u 2 , v v1 , v2 , w w1 , w2 V
~
~ ~
Axiom 2: u v v u
~
~
Axiom 4: 0 1,1 V such that 0 u u 0 u u V

~
Axiom 5:
u u1 2,u 2 2 V such that ( u ) u u ( u ) 0 u V
~
Axiom 6: If u V and k scalar, then k u ku1 , ku2 , ku3 V

~
Axiom 7: k (u v) k u1 v1 1, u 2 v2 1 k (u1 v1 1), k (u 2 v2 1)

~
k u k v ku1 , ku2 kv1 , kv2 ku1 kv1 1, ku2 kv2 1

~
Axiom FAILS
Axiom 8: (k l ) u (k l ) u1 , u 2 (k l )u1 , (k l )u 2
~
k u l u ku1 , ku2 lu1 , lu 2 ku1 lu1 1, ku2 lu 2 1

~
Axiom FAILS
Axiom 9: k (l u) k lu1 , lu 2 klu1 , klu2
~
(kl) u klu1 , klu2

~
76
Axiom 10: 1 u u1 , u 2 u
~
V is NOT a vector space.

Example 3:
Let V denote the set of all positive real numbers with the operations
x y xy and k x x k
to hold.
Solution:
Let u u , v v , w w V
~
Axiom 1: If u, v V then u v u v uv V
~ ~
Axiom 2: u v v u
~
~
Axiom 4: 0 1 V such that 0 u u 0 u u V

~
Axiom 5:
u
~
1
V such that ( u ) u u ( u ) 0 u V
~
~
~
~
~
~
u
Axiom 6: If u V and k scalar, then k u ku u k V

~
Axiom 7: k (u v) k (u v) k (uv) (uv) k

~
k u k v k u k v uk vk uk vk
~
Axiom 8: (k l ) u (k l ) u u k l
~
k u l u k u l u u k u l u k u l u k l
~
Axiom 9: k (l u) k (l u) k u l (u l ) k u kl
~
(kl) u (kl) u u kl
~
Axiom 10: 1 u 1 u u u u
1
V is a vector space.
77
4.2)
Subspaces
A subset W is a subspace of V if W is itself a vector space.

Example 1: The subspaces of R 2 are
(1) the origin
(2) lines through the origin
(3) R 2
Example 2: The subspaces of R 3 are
(1) the origin
(2) lines through the origin
(3) planes through the origin
(4) R 3
Theorem 4.2.1 Let W be a subset of a vector space V. Then W is a subspace of V if and
only if (a) u , v W u v W (b) u W k u W, k scalar
~
Example 3: (Subspaces of R 3 )
Let W be the set of all vectors of the form 0, a,0 and let V R 3 . Show that W is a
subspace of V.
Solution:
Let u 0, a1 ,0 , v 0, a2 ,0 W
~
(a) u v 0, a1 ,0 0, a2 ,0 0, a1 a2 ,0 W
~
(b) ku k 0, a1 ,0 0, ka1 ,0 W
~
Therefore W is a subspace of V.
78
Example 4: (Subspaces of M 22 )
a b
such that a b c d 0 and let V M 22 .
Let W be the set of all matrices
c d
Show that W is a subspace of V.
Solution:
a
Let u 1
~
c1
b1
a
, v 2
d1 ~ c 2
b2
W .
d 2
Therefore a1 b1 c1 d1 0 and a2 b2 c2 d 2 0
a
(a) u v 1
~
~
c1
b1
d1
a2
c2
a
(b) ku k 1
~
c1
b1 ka1

d1 kc1
b2 a1 a 2
d 2 c1 c2
b1 b2
W
d1 d 2
kb1
W
kd1
Example 5: (Subspaces of M nn )
Let W be the set of all n n symmetric matrices and let V M nn . Show that W is a
subspace of V.
Solution:
Let u A , v B W .
~
(a) u v A B W since A B is symmetric

~
(b) ku kA W since kA is symmetric

~
For questions on subspaces of Pn refer to Tutorial 5.
79
Theorem 4.2.2 Let W be the set of solution vectors of the homogeneous linear system
A x 0 of m equations in n unknowns. Then W is a subspace of V = R n
~
Example 6: The case where n 3 (Refer to Example 2)
1 2 3 x 0 1 2 3 0
1 2 3 0
1 0 0 1
1 0
(a) 3 7 8 y 0 0 1
4
0 0 19 0
1
2 z 0 0 9 10 0
The solution is x 0, y 0, z 0
(The origin, which is a subspace of R 3 )
1 2 3 x 0 1 2 3 0
(b) 3 7 8 y 0 0 1 1 0
2 4 6 z 0 0 0 0 0
x
y
z
5 1 1
(Line through the origin, which is a subspace of R 3 )
The solutions are x 5t , y t , z t or
1 2 3 x 0 1 2 3 0 1 2 3 0
(c) 2 4 6 y 0 2 4 6 0 0 0 0 0
3 6 9 z 0 3 6 9 0 0 0 0 0
The solutions are x 2 y 3z 0

(Plane through the origin, which is a subspace of R 3 )
0 0 0 x 0

(d) 0 0 0 y 0
0 0 0 z 0

The solutions are x r, y s, z t
(R 3 ,which is a subspace of R 3 )
80
Linear Combination
A vector w is a linear combination of the vectors v1 , v2 ,..., vr if
~
w k1 v1 k 2 v2 ... k r vr
~
where k1 , k 2 ,..., k r are scalars.

We consider two problems
Problem 1: Determine whether w is a linear combination of the vectors v1 , v2 ,..., vr
~
Problem 2: Express w as a linear combination of the vectors v1 , v2 ,..., vr

~
Example 7: (Problem 1)
Determine whether w 2,2,2 is a linear combination of v1 0,2,2 and
~
v2 1,3,1
~
Solution:
Let w k1 v1 k 2 v2 . Therefore 2,2,2 k1 0,2,2 k 2 1,3,1
~
By comparing the components, we have

k2 2
2k1 3k 2 2
k1 2, k 2 2
2 k1 k 2 2
The values of k1 2, k 2 2 also satisfy the third equation

Therefore w is a linear combination of the vectors v1 and v 2
~
Example 8: (Problem 2)
Let w 2,3,4 , v1 1,0,3 , and v2 0,3,10 .
~
Express w as a linear combination of the vectors v1 and v 2

~
Solution:
81
Theorem 4.2.3 Let v1 , v2 ,..., vr V and W be the set of all linear combinations of
~
v1 , v2 ,..., vr . Then
~
(a) W is a subspace of V
(b) W is the smallest subspace of V, ie every other subspace of V that contains
v1 , v2 ,..., vr must contain W
~
(Illustration)
Spanning
Let S {v1 , v2 ,..., vr } and let W be the set of all linear combinations of v1 , v2 ,..., vr . Then
~
W is a subspace spanned by v1 , v2 ,..., vr and S spans W

~
Note that S spans W if every vector in W can be expressed as a linear combination of the
vectors in S
Example 9: Determine whether v1 1,0,0 , v2 0,1,0 , and v3 0,0,1 span R 3
~
Solution:
Let S {v1 , v2 , v3 } and W R 3 . Let a, b, c be any vector in W. Then
~
a, b, c a 1,0,0 b 0,1,0 c 0,0,1 av1 bv 2 c v3 . Therefore S spans W.

~
82
Example 10: Determine whether v1 1,1,2 , v2 1,0,1 , and v3 2,1,3 span R 3

~
Solution:
Let S {v1 , v2 , v3 } and W R 3 . We want to know whether S spans W, ie whether
~
every vector b b1 , b2 , b3 W
~
can be expressed as a linear combination of
v1 , v2 , and v3 .
~
Now, S spans W
every vector b b1 , b2 , b3 W can be expressed as a linear combination of

~
v1 , v2 , and v3 .
~
b1 , b2 , b3 k1 v1 k 2 v2 k 3 v3
~
b1 , b2 , b3
b1 , b2 , b3 k1 1,1,2 k 2 1,0,1 k3 2,1,3
b1 , b2 , b3
k1 k 2 2k 3 b1
k1
k 3 b2
b1 , b2 , b3
2k1 k 2 3k 3 b3
the system is consistent b1 , b2 , b3

1 1 2
det( A) 0 , where A 1 0 1
2 1 3
But then det( A) 0 , therefore S do not span W

Example 11: Determine whether v1 1,1,3 , v2 0,1,1 ,and v3 3,1,2 span R 3
~
83
4.3)
Linear Independence
The set S {v1 , v2 ,..., vr } is called a linearly independent (LI) set if

~
k1 v1 k 2 v2 ... k r vr 0
~
has only the trivial solution k1 k 2 ... k r 0 . Otherwise S is called a linearly

dependent (LD) set.
Methods of determining LI/LD
(Method I) Use Equivalent Statements (or solve directly if det(A) is not defined)
Example 1: Determine whether the set S {v1 , v2 , v3 } is LI or LD, where
~
v1 1,2,3 , v2 5,6,1 , v3 3,2,1

~
Solution: Let k1 v1 k 2 v2 k 3 v3 0
~
k1 5k 2 3k 3 0
2k1 6k 2 2k 3 0
3k1 k 2 k 3 0
Now, det(A) = 0 A is not invertible

A x 0 has nontrivial solution ie not all k1 , k 2 , k3 are zero
~
S is LD
Example 2: Determine whether the set S {v1 , v2 , v3 } is LI or LD, where

~
v1 0,0,2,2 , v2 3,3,0,0 , v3 1,1,0,1

~
Solution: Let k1 v1 k 2 v2 k 3 v3 0
~
3k 2 k 3 0
3k 2 k 3 0
0
2k1
k3 0
2k1
Solving the system, we obtained k1 k 2 k 3 0 S is LI
84
(Method II) Use linear combination (Can only check LD, cannot check LI)
Theorem 4.3.1 The set S {v1 , v2 ,..., vr } , r 2 is linearly dependent if and only if at
~
least one of the vectors in S is a linear combination of the remaining vectors in S

Example 3: Determine whether S {v1 , v2 , v3 } is LI or LD, where
~
v1 2,1,0,3 , v2 1,2,5,1 , v3 7,1,5,8

~
Solution: By inspection v2 3 v1 v3 , the vector v 2 is a linear combination of the

~
remaining vectors v 2 , v 3 in S. Therefore, S is LD.

~
(Method III) Use comparison (Can only check LD, cannot check LI)
Theorem 4.3.3 Let S {v1 , v2 ,..., vr } be a set of vectors in R n . If r n , then S is
~
linearly dependent
Example 4: Determine whether the set S {v1 , v2 , v3 , v4 } is LI or LD, where
~
v1 2,0,1 , v2 3,2,5 , v3 6,1,1 , v4 7,0,2

~
Solution: Since r 4 3 n , by Theorem 4.3.3, S is LD.
Example 5:
Determine whether the set S {v1 , v2 , v3 } is LI or LD, where
~
(a) v1 2,1,1 , v2 4,5,2 , v3 1,3,2

~
(b) v1 0,0,1,1 , v2 2,2,0,0 , v3 1,1,0,3

~
(c) v1 2,1,0,3 , v2 1,5,6,3 , v3 1,1,2,3

~
(Hint for part (c): Express v 2 as a linear combination of v1 and v3 )

~
85
4.4)
Basis and Dimension
Definition(Basis)
Let S {v1 , v2 ,..., vr } be a set of vectors in a vector space V. Then S is a basis for V if
~
(a) S spans V
(b) S is linearly independent
Example 1: Determine whether S {v1 , v2 , v3 } is a basis for R 3 ,
~
where v1 1,0,0 , v2 0,1,0 , v3 0,0,1 .

~
Solution: The set S is linearly independent and S spans R 3 . Therefore S is a basis for R 3 .
~
where v1 1,2,1 , v2 2,9,0 , v3 3,3,4 .

~
Solution:
1 2 3
Let A 2 9 3 . Since det( A) 0 , the set S is LD.

1 0 4
Therefore S is NOT a basis for R 3 . Note that S do not span R 3 too.

~
where v1 2,0,1 , v2 2,1,0 , v3 1,2,3 .

~
Solution:
2 2 1
Let A 0 1 2 . Since det( A) 11 0 , the set S is LI and S spans R 3 .

1 0 3
Therefore S is a basis for R 3 .
86
Example 4: Determine whether S {v1 , v2 , v3 } is a basis for P2 ,

~
where v1 2 x , v2 2 x , v3 1 2 x 3x 2 .
2
Solution:
2 2 1
Let A 0 1 2 . Since det( A) 11 0 , the set S is LI and S spans P2 .

1 0 3
Therefore S is a basis for P2 .

Example 5: Determine whether S {v1 , v2 , v3 , v4 } is a basis for R 4 ,
~
where v1 2,1,0,2 , v2 6,4,1,2 , v3 3,2,1,1 , v4 1,2,4,3 .

~
Solution:
1
Let A
0
6
4
1
2
3 1
2 2
. Since det( A) 8 0 (See Example 10 of Chapter 2),
1 4
1 3
the set S is LI and S spans R 4 . Therefore S is a basis for R 4 .

Example 6: Determine whether S {v1 , v2 , v3 , v4 } is a basis for M 22 ,
~
2 1
4 3
2 1
2 2
, v2
, v4
.
, v3
where v1
~
1 1 ~ 3 2 ~ 1 1 ~ 3 2
Solution:
1
Let A
1
4 2 2
3 1
2
. Since det( A) 16 0 (See Example 9 (b) of Chapter 2),
3 1
3
2 1
2
the set S is LI and S spans M 22 . Therefore S is a basis for M 22 .
87
Definition (Dimension)
Let S {v1 , v2 ,..., vn } be a basis for a vector space V. The dimension of V, dim(V) = n.
~
The zero vector space has dimension zero.

Example 3:
(a) dim(R n ) = n a basis for R n has n linearly independent vectors
(b) dim ( Pn ) = n +1 a basis for Pn has n + 1 linearly independent vectors
(c) dim (M mn ) mn a basis for M mn has mn linearly independent vectors
Example 4:
(a) a basis for R 4 has 4 linearly independent vectors
(b) a basis for P3 has 4 linearly independent vectors
(c) a basis for M 23 has 6 linearly independent vectors
(d) dim(R 5 ) = 5
(e) dim ( P4 ) = 5
(f) dim (M 43 ) 12
88
4.5)
Row Space, Column Space & Nullspace of a Matrix
a11
a
Let A 21
a
m1
a12
a 22
am2
a1n r~1 a11 , a12 ,..., a1n
a2n

a mn
c1 a11 , a21 ,..., am1

~
The vectors r1 , r2 ,..., rm R n are the row vectors of A

~
The vectors c1 , c2 ,..., cn R m are the column vectors of A

~
Definition
Row space of A : subspace of R n spanned by the row vectors of A
Column space of A : subspace of R m spanned by the column vectors of A
Nullspace of A : the solution space of the homogeneous system A x 0
~
89
Finding a basis for the row space, column space & nullspace of A
Basis for the row space and the column space of A
Theorem 4.5.1 If a matrix B (in row-echelon form) can be obtained from A by
elementary row operations, then the nonzero row vectors of B form a basis for the row
space of A
Example 1: (Basis for the row space and the column space, at the same time)
Find a basis for the row space and the column space of
1
0
A 3
3
2
3
1
0
4
0
1
0
6 1
2 1
4 2
1
Solution: Using elementary row operations, rewrite A in row echelon form

1
0
B 0
0
0
3 1 3
1 1 0
0 0 1
0 0 0
0 0 0
The nonzero row vectors of B, r1 1,3,1,3 , r2 0,1,1,0 , and r3 0,0,0,1 form a

~
basis for the row space of A.

1
3
3

0
1
0
The column vectors c1 3 , c 2 0 , and c 4 1 form a basis for the column
~

~
~
3
4
1
2
0
2

space of A.
90
Example 2: (Basis for the row space and the column space, at the same time)
Find a basis for the row space and the column space of
3
1 1
1
6
1
0
2
A 1 3 1
2
7 2 1
2
3
0 2 1
Solution:
91
Example 3: (Basis for the column space, directly)

Find a basis for the column space of
1
0
A 3
3
2
3
1
0
4
0
1
0
6 1
2 1
4 2
1
Solution: Take the transpose of A,
3
T
A
1
0 3 3
2
1 0
4
0
1 6 2 4
0 1 1 2
Using elementary row operations, rewrite AT in row-echelon form
0
*
B
0
0 3 3
2
1 9 5 6
0 1 1 1
0 0
0
0
The nonzero row vectors of
B* ,
r1 1,0,3,3,2 , r2 0,1,9,5,6 , and

*
r3 0,0,1,1,1 form a basis for the row space of A .

*
1
0
0

0
1
0
The column vectors v1 3 , v 2 9 , and v3 1 form a basis for the column

~

~
~
3
5
1
2
6
1

space of A.
92
Example 4: (Basis for the column space, directly)

Find a basis for the column space of
3
1 1
1
6
1
0
2
A 1 3 1
2
7 2 1
2
3
0 2 1
Solution:
93
Basis for the nullspace of A

Recall that the nullspace of A is the solution space of the homogeneous system A x 0
~
Example 5: (Basis for the nullspace)

1
0
Find a basis for the nullspace of A 3
3
2
3
1
0
4
0
1
0
6 1
2 1
4 2
1
Solution: Consider the homogeneous system A x 0

~
0
From Example 1, the row-echelon form is 0
0
0
Now,
x1 3x2 x3 3x4 0
x2 x3
3 1 3 0
1 1 0 0
0 0 1 0
0 0 0 0
0 0 0 0
0
x4 0
Let x2 s . Then, x3 s , and x1 2s
x1 2s 2

x2 s 1
s
The solution space, x
~
x3
s 1

x 0 0

4
2

1
Since spans the solution space and
1
0
2

1
is linearly independent,
1
0
2

1
therefore form a basis for the nullspace of A.
1
0
94

3
1 1
1
6
1
0
2
Find a basis for the nullspace of A 1 3 1
2
7 2 1
2
3
0 2 1
Solution:
95

Find a basis for the nullspace of
1 2 2 1
A 3 6 5 4
1 2 0 3
Solution: Consider the homogeneous system A x 0

~
1 2 0 3 0
The row-echelon form is 0 0 1 1 0

0 0 0 0 0
Now,
x1 2 x2
3x4 0
x3 x4 0
Let x2 s and x4 t . Then, x3 t , and x1 2s 3t
x1 2s 3t 2 3

s
x2
1 0
s
t
The solution space, x
~
x3
t 0 1

0 1
x
t

4
2 3

1 0
Since , spans the solution space and
0 1
0 1
2 3

1 0
, is linearly independent,
0 1
0 1
2 3

1 0
therefore , form a basis for the nullspace of A.
0 1
0 1
96
4.6)
Rank and Nullity
Definition
rank(A) = dim(row space of A)
= number of vectors in a basis for the row space of A.
rank(A) = dim(column space of A)
= number of vectors in a basis for the column space of A.
nullity(A) = dim(nullspace of A)
= number of vectors in a basis for the nullspace of A.
Theorem 4.6.1
(a) rank(A) = rank( AT )
(b) rank(A) + nullity(A) = n, where A contains n columns
Example 1:
1
0
Find rank(A) and nullity(A), where A 3
3
2
3
1
0
4
0
1
0
6 1
2 1
4 2
1
Solution:
From Example 1 of Section 4.5, we know that,
The vectors r1 1,3,1,3 , r2 0,1,1,0 , and r3 0,0,0,1 form a basis for the row
~
space of A. Therefore rank(A) = 3.

From Theorem 4.6.1 (b), we have nullity(A) = n rank(A) = 4 3 =1
Alternatively, we have from Example 5 of Section 4.5, since
2

1
form a basis for the nullspace of A, therefore nullity(A) = 1.
1
0
97
Example 2:
3
1 1
1
6
1
0
2
Find rank(A) and nullity(A), where A 1 3 1
2
7 2 1
2
3
0 2 1
Solution:
Example 3:
Given the size of a matrix A is 5 9 and rank(A) = 2,
(a)
(b)
(c)
(d)
dim(row space of A) = rank(A) = 2

dim(column space of A) = rank(A) = 2
dim(nullspace of A) = nullity(A) = n rank(A) = 9 2 = 7
dim(nullspace of AT ) = nullity( AT ) = m rank( AT ) = m rank(A) = 5 2 = 3
Example 4:
Given the size of a matrix A is 8 6 and rank(A) = 3,
(a)
(b)
(c)
(d)
dim(row space of A) =
dim(column space of A) =
dim(nullspace of A) =
dim(nullspace of AT ) =
98
CHAPTER 5 INNER PRODUCT SPACES

5.1)
Inner Products
Let V be a real vector space and u, v V . An inner product on V is a function that

~ ~
associates a real number u, v satisfying the following axioms

~ ~
Axiom 1: u, v v, u
~ ~
~ ~
Axiom 2: u v, w u, w v, w
~
Axiom 3: ku, v k u, v
~
~ ~
Axiom 4: u,u 0 with equality if and only if u 0

~ ~
Definition Let V be an inner product space and u, v V .

~ ~
(i) norm of u , u u, u
~
1
2
~ ~
(ii) distance between u and v , d (u, v) u v

~
~ ~
Example 1: (Euclidean Inner Product)

Let u u1 , u 2 , u3 , v v1 , v2 , v3 .
~
The expression u, v u . v u1v1 u 2 v2 u3 v3 defines an inner product in R 3 .

~ ~
~ ~
(i) Find the norm of u , u

~
(ii) Find the distance between u and v , d (u, v) .

~
~ ~
99
Example 2: (Weighted Euclidean Inner Product)

Let u u1 , u 2 , u3 , v v1 , v2 , v3 .
~
The expression u, v 3u1v1 2u2 v2 u3v3 defines an inner product in R 2 .

~ ~

~

~
~ ~
Example 3: (Inner Product on Matrices)
a1 a 2
b b2
, v B 1
.
Let u A
~
a3 a 4 ~
b3 b4
The expression u, v A, B tr( AT B ) a1b1 a2 b2 a3b3 a4 b4 defines an inner
~ ~
product in M 22 .
~

~
~ ~
100
Example 4: (Inner Product on Polynomials)

Let u p( x) a0 a1 x a2 x 2 , v q( x) b0 b1 x b2 x 2 .
~
The expression u, v p( x), q( x) a0 b0 a1b1 a2 b2 defines an inner product in

~ ~
P2 .
~

~
~ ~
Example 5: (Various Inner Product)

Find the norm u and distance d (u, v) for each of the following
~ ~
(a) u 1,2,3 , v 0,1,3 where u, v u . v u1v1 u 2 v2 u3 v3

~
~ ~
~ ~
(b) u 1,2,3 , v 0,1,3 where u, v 2u1v1 u2 v2 3u3v3

~
~ ~
1 1
0 2
, v B
(c) u A
~
1 2 ~
3 1
(d) u p( x) 2 x x 2 , v q( x) 3x 2 x 2
~
101
5.2)
Angle and Orthogonality in Inner Product Space
Theorem 5.2.1 (Cauchy-Schwarz Inequality)

Let V be an inner product space. For any u, v V , u, v u v
~ ~
~ ~
Example 1: Verify that the Cauchy-Schwarz Inequality holds for the vectors
u 3,1,0,2 and v 2,1,3,0 with respect to the Euclidean inner product.
~
Solution:
u, v
~ ~
u v
~
Therefore u, v u v and the Cauchy-Schwarz Inequality holds.

~ ~
Example 2: Verify that the Cauchy-Schwarz Inequality holds for the vectors
u 3,1,0,2 and v 2,1,3,0 with respect to the weighted Euclidean inner
~
product defined by u, v 3u1v1 2u2 v2 u3v3

~ ~
Solution:
u, v
~ ~
u v
~
Therefore u, v u v and the Cauchy-Schwarz Inequality holds.

~ ~
Theorem 5.2.2 & 5.2.3

(a) u v u v
~
(b) d (u, v) d (u, w) d (w, v)

~ ~
(c) cos
u, v
~ ~
u v
~
, is the angle between u and v

~
102
Orthogonality
Two vectors u and v are orthogonal if their inner product u, v 0
~
~ ~
Example 3:
1 1
0 2
and v B
are orthogonal
(a) The two vectors u A
~
~
2
1
1
0
(b) The two vectors u p( x) 1 2 x x 2 and v q( x) x 2 x 2 are orthogonal

~
Orthogonal Complement
Definition Let W be a subspace of an inner product space V. The set of all vectors in V
that are orthogonal to W is called the orthogonal complement of W. The orthogonal
complement of W is denoted as W . We say that W and W are orthogonal complements.
Theorem 5.2.4 Let A be an m n matrix. Then

(a) The nullspace of A and the row space of A are orthogonal complements in R n with
respect to the Euclidean inner product.
(b) The nullspace of AT and the column space of A are orthogonal complements in R m
with respect to the Euclidean inner product.
103
Determine a basis for an orthogonal complement

Example 4: Let W be a subspace of R 5 spanned by the vectors w1 2,2,1,0,1 ,
~
w2 1,1,2,3,1 , w3 1,1,2,0,1 , w4 0,0,1,1,1 . Find a basis for W .

~
Solution:
1
2 2 1 0
1 1 2 3 1
Let A
. Now, W is the row space of A. By Theorem 5.2.4(a),
1 1 2 0 1
0 0
1
1
1
W is the nullspace of A . We therefore find a basis for the nullspace of A.

Consider the homogeneous system A x 0 .
~
0
The row echelon form of the augmented matrix is
0
Now,
x1 x2 2 x3
x5 0
x3
x5 0
x4
0
1 2
0 1
0 0
0 0
0 1
0 1
1 0
0 0
0
0
Let x2 s and x3 t . Then, x5 t , and x1 s t

x1 s t 1 1

x2 s 1 0
The solution space, x x3 t s 0 t 1

~

x4 0 0 0
x

5 t 0 1
1
1
1
1

1
0
1
0
Therefore 0 and 1 form a basis for the nullspace of A 0 and 1 form a

0
0
0
0
0
1
0
1

basis for W .
104
5.3)
Orthonormal Bases; Gram-Schmidt Process
The set {v1 , v2 , v3 } is orthogonal if v1 , v2 v2 , v3 v3 , v1 0

~
The set {q1 , q 2 , q3 } is orthonormal if

~
(i) q1 , q2 q2 , q3 q3 , q1 0 , and
~
(ii) q1 q 2 q3 1
~
In an inner product space, a basis consisting of orthogonal (orthonormal) vectors is called

an orthogonal basis. A basis consisting of orthonormal vectors is called an orthonormal
basis.
Theorem 5.3.1 Let S {v1 , v2 , v3 } be an orthonormal basis for an inner product space V.
~
Let u V . Then u u, v1 v1 u, v2 v2 u, v3 v3
~
Proof:
Since S {v1 , v2 , v3 } is a basis, a vector u V can be expressed in the form
~
u k1 v1 k 2 v2 k3 v3
~
Now,
u,v1 k1 v1 k 2 v2 k3 v3 ,v1
~
k1 v1 ,v1 k 2 v2 ,v1 k3 v3 ,v1

~
k1 v1 ,v1 k 2 v2 ,v1 k3 v3 ,v1

~
(Axiom 3)
k1
Similarly, we have u, v2 k 2 and u, v3 k3 . Therefore,
~
u u, v1 v1 u, v2 v2 u, v3 v3
~
Theorem 5.3.1 is a special case of a more general theorem shown below

Theorem 5.3.2 Let S {v1 , v2 ,..., vn } be an orthonormal basis for an inner product space
~
V. Let u V . Then u u, v1 v1 u, v2 v2 ... u, vn vn

~
105
The scalars u,v1 , u,v2 , u, v3 in the Theorem 5.3.1 are the coordinates of the
~
vector u relative to the orthonormal basis S {v1 , v2 , v3 } and

~
(u ) S u, v1 , u, v2 , u, v3
~
is the coordinate vector of u relative to this basis.

~
Example 1: The set S {v1 , v2 , v3 } is an orthonormal basis for R 3 , where v1 0,1,0 ,

~
3 4
4 3
v2 ,0, , v3 ,0, . Express the vector u 1,1,1 as a linear combination
~
5 5
5 5
~
~
of the vectors in S, and find the coordinate vector (u) S
~
Solution:
7
1
u, v1 1 , u, v 2 , u, v3
~ ~
~ ~
~ ~
5
5
7
1
By Theorem 5.3.1, we have u v1 v 2 v3
~
5 ~ 5 ~
~
1 7
The coordinate vector of u relative to S is (u ) S 1, ,
~
~
5 5
Example 2: The set S {v1 , v2 , v3 } is an orthonormal basis for R 3 , where v1 0,0,1 ,

~
12 5
5 12
v2 , ,0 , v3 , ,0 . Express the vector u 1,1,1 as a linear
~
13 13
13 13
~
~
combination of the vectors in S, and find the coordinate vector (u) S
~
106
Gram-Schmidt Process
Given any basis {u1 , u 2 , u3 } , we can transform the basis into an orthogonal basis
~
{v1 , v2 , v3 } and then normalize the orthogonal basis vectors to obtain an orthonormal
~
basis {q1 , q 2 , q3 } . This process is called the Gram-Schmidt Process. The construction of
~
these orthogonal and orthonormal bases are outline in the following steps. Recall from
u .a
Theorem 3.0.1, proja u ~ ~2 a .
~
~ ~
a
~
Step 1
v1 u1
~
Step 2
v2 u 2 projv1 u 2 u 2
~
u 2 , v1
~
~
2
v1
v1
~
Step 3
v3 u3 projv1 u3 projv2 u3 u3
~
u3 , v1
~
2
v1
v1
~
u3 , v2
~
Orthogonal basis : {v1 , v2 , v3 }

~
Step 5
v2
v2
Step 4
~
2
Orthonormal basis : {q1 , q 2 , q3 } where q1

~
v1
~
, q2
v1
v2
~
, q3
v2
v3
~
v3
~
107
Example 3: Use the Gram-Schmidt process to transform the basis {u1 , u 2 , u3 } into an
~
orthonormal basis, where u1 1,1,1 , u 2 0,1,1 , u3 0,0,1

~
Solution:
Basis S 1,1,1 , 0,1,1 , 0,0,1
Let u 1,1,1 , u 0,1,1 , u 0,0,1

~2
~1
~3
v1 u1 1,1,1
~
v2 u 2
~
u 2 , v1
~
~
2
v1
~
v1
~
2
1
1,1,1 2,1,1
3
3
u
,
u 3 , v1
3 v2
~
~
~
~
v3 u 3
v1
v2
2
2
0,1,1
v1
[Choose v2 2,1,1 ]
~
v2
~
1
1
1
0,0,1 1,1,1 2,1,1 0,1,1
3
6
2
[Choose v3 0,1,1 ]
~
Orthogonal basis: {v1 , v2 , v3 }

~
Orthonormal basis: {q1 , q 2 , q3 } where

~
q1
~
v1
~
v1
~
1
3
1,1,1
q2
~
v2
~
v2
~
1
6
2,1,1
q3
~
v3
~
v3
1
2
0,1,1
108
5.4)
QR-decomposition
Theorem 5.4.1
If A is an m n matrix with linearly independent column vectors, then A can be factored
as
A QR
where Q is an m n matrix with orthonormal column vectors q1 , q2 ,..., qn (resulting from
~
applying the Gram-Schmidt process), and R is an n n invertible upper triangular matrix

given by
u1 , q1 u 2 , q1
~ ~
~
~
0
u 2 , q2
~
~
R
0
0
u n , q1
~
~
u n , q2
~
~
u n , qn
~
~
We can show that the column vectors of the matrix A are linearly independent if and only
if A is invertible. Hence we can conclude that every invertible matrix A has a QRdecomposition.
Example 1:
1 0 0
Find the QR-decomposition of the matrix A 1 1 0 .

1 1 1
Solution:
The column vectors of A, u1 1,1,1 , u 2 0,1,1 , u3 0,0,1 are linearly
~
independent since det( A) 1 0 . Applying the Gram-Schmidt process and subsequent

normalization to these column vectors, we obtained the the orthonormal vectors
q1
~
1
3
1,1,1 , q2
~
1
6
2,1,1 , q3
~
1
2
0,1,1
109
The matrix R is given by
u1 , q1 u 2 , q1 u3 , q1
~
~
~
~
~ ~
R
0 u 2 , q2 u3 , q2
~
~
~
~
0
0
u
,
q
3
3
~
~
3
3
0

0
2
3
2
6
1
3
1
6
1
2
Thus the QR-decomposition of the matrix A is
1 0 0
1 1 0
1 1 1
1
3
1
3
1
3
2
6
1
6
1
6
0 33
1
0
2
1
0
2
2
3
2
6
1
3
1
6
1
2
Example 2:
0
1 1
0 .
Find the QR-decomposition of the matrix A 1 1
0 1 3
Solution:
The column vectors of A, u1 1,1,0 , u 2 1,1,1 , u3 0,0,3 are linearly
~
independent since det( A) 0 . Applying Gram-Schmidt process and subsequent

normalization to these column vectors, we obtained the the orthonormal vectors
q1
~
, q2
, q3
The matrix R is given by
u1 , q1 u 2 , q1 u3 , q1
~
~
~
~
~ ~
R
0 u 2 , q2 u3 , q2
~
~
~
~
0
0
u
,
q
3
3
~
~

0

0 0
0
1 1

0
The QR-decomposition of the matrix A is 1 1
0 1 3
0
0 0
110
CHAPTER 6 EIGENVALUES, EIGENVECTORS

6.1) Eigenvalues and Eigenvectors
Definition Let A be an n n matrix. A nonzero vector x is an eigenvector of A if for
~
some scalar , A x x . The scalar is called an eigenvalue of A.

~
Definition det (I A) 0 characteristic equation of A

Definition Nullspace of (I A) solution space of (I A) x 0
~
eigenspace of A corresponding to the eigenvalue

Computation of Eigenvalues, Eigenvectors and Finding a Basis for the Eigenspace of A
Step 1 Find eigenvalues of A by solving det (I A) 0
Step 2 For each eigenvalue, find the corresponding eigenvector
Step 3 Obtain a basis for the eigenspace of A
Example 1:
Find the characteristics equation, eigenvalues, the corresponding eigenvectors, and bases
0 0 2
for the eigenspace for the matrix A 1 2 1 .

1 0 3
Solution:
Let det (I A) 0
det( 0
0
det 1
1
( 2)
0 0 0

1 0 1
0 1 1
0
2
2 1
0 2
2 1 ) 0
0 3
0
3
0
2
1 3
(2nd column expansion)
111
( 2)( ( 3) 2) 0
( 2)(2 3 2) 0
( 1)( 2) 2 0 1, 2, 2
0
2 x1 0

To find the eigenvectors, consider (I A) x 0 , ie 1 2 1 x2 0
~
~
1
0
3 x3 0
For the eigenvalue 1 , we have
0
2 x1 0 1 0
2 0 1 0 2 0 1 0 2 0
1
1 1 1 x 2 0 1 1 1 0 0 1 1 0 0 1 1 0
1 0 2 x 0 1 0 2 0 0 0 0 0 0 0 0 0
Now,
x1
2 x3 0
x2 x3 0
Let x3 s . Then, x2 s and x1 2s
x1 2s 2

The solution space x x2 s s 1
~
x s 1
3

x1 2

the solution space of (I A) x 0 is of the form x x2 s 1
~
~
~
x 1
3
2

1 is a basis for the nullspace of (I A) corresponding to 1
1

2

1 is a basis for the eigenspace of A corresponding to 1
1

112
2 0 2 x1 0 2 0 2 0 1 0 1 0 1 0 1 0

1 0 1 x2 0 1 0 1 0 1 0 1 0 0 0 0 0
1 0 1 x 0 1 0 1 0 1 0 1 0 0 0 0 0

Now,
x1
x3 0
Let x3 s and x2 t . Then, x1 s
x1 s 1 0

The solution space x x2 t s 0 t 1
~
x s 1 0
3
x1 1 0

the solution space of (I A) x 0 is of the form x x2 s 0 t 1
~
~
~
x 1 0
3
1 0

0 , 1 is a basis for the nullspace of (I A) corresponding to 2
1 0

1 0

0 , 1 is a basis for the eigenspace of A corresponding to 2
1 0

113
Example 2:
3 0 1
for the eigenspace for the matrix A 0 2 4

0 0 1
Solution:
Let det (I A) 0
1 0 0 3 0 1
det( 0 1 0 0 2 4 ) 0
0 0 1 0 0 1
0
1
3
2 4 0
det 0
0
0
1
( 3)( 2)( 1) 0
(upper triangular matrix)
1, 2, 3
0
1 x1 0
3

2 4 x2 0
To find the eigenvectors, consider (I A) x 0 , ie 0
~
~
0
0
1 x3 0
2 0 1 x1 0 2 0 1 0
0 1 4 x2 0 0 1 4 0
0
0
0 x3 0 0
0
0 0
Now,
2 x1
x3 0
x2 4 x3 0
Let x3 s . Then, x2 4s and x1 12 s
114
x1 12 s
1

1
The solution space x x2 4s 2 s 8
~
x s
2
3

x1
1

1
the solution space of (I A) x 0 is of the form x x2 2 s 8
~
~
~
x
2
3

1

2

1

2

1 0 1 x1 0 1 0 1 0

0 0 4 x2 0 0 0 4 0
0 0 1 x 0 0 0 1 0
3
Now,
x1
x3 0
4 x3 0
x3 0
Therefore x1 0 , x3 0 and we let x2 s
x1 0 0

The solution space x x2 s s 1
~
x 0 0
3
x1 0 0

the solution space of (I A) x 0 is of the form x x2 s s 1
~
~
~
x 0 0
3
0

0

115
0

0

0 0 1 x1 0
0 0 1 0
0 1 4 x2 0 0 1 4 0
0 0 2 x 0
0 0 2 0
Now,
x3 0
x2 4 x3 0
2 x3 0
Therefore x2 0 , x3 0 and we let x1 s
x1 s 1

The solution space x x2 0 s 0
~
x 0 0
3
x1 s 1

~
~
~
x 0 0
3
1

0

1

0

116
Example 3:
1 2 2
for the eigenspace for the matrix A 2 5 2 .

6 6 3
Solution:
117
Theorem 6.1.1 If k is a positive integer, is an eigenvalue of a matrix A, and x is a

~
corresponding eigenvector, then is an eigenvalue of A

k
and x is a corresponding
~
eigenvector.
Example 4:
Find
the
eigenvalues
0 0 2
7
A where A 1 2 1
1 0 3
Solution:
and
corresponding
eigenvectors
of
the
matrix
The eigenvalues of A are 2 and 1 the eigenvalues of A 7 are 2 7 128

and 17 1 .
2

An eigenvector of A correspond to 1 is 1 an eigenvector of A 7 correspond to
1

2
1

7
1 1 is also 1 . An eigenvector of A correspond to 2 is 1 an
1
1

1

eigenvector of A correspond to 2 128 is also 1
1

7
Example 5:
Find
the
eigenvalues
and
1 2 2
9
matrix A where A 2 5 2 .
6 6 3
Solution:
the
corresponding
eigenvectors
of
the
118
6.2)
Diagonalization
Definition A square matrix A is diagonalizable if an invertible matrix P such that

P 1 AP is a diagonal matrix. The matrix P is said to diagonalize A.
Theorem 6.2.1 Let A be an n n matrix. Then A is diagonalizable if and only if A has n
linearly independent eigenvectors.
0 0 2
Example 1: Find a matrix P that diagonalizes A 1 2 1

1 0 3
(Refer to Example 1 Section 6.1)

Solution: The characteristic equation of A is found to be ( 1)( 2) 2 0
2

We also found that 1 is a basis for the eigenspace of A corresponding to 1 , and
1

1 0

0 , 1 form a basis for the eigenspace of A corresponding to 2 .
1 0

Since A has 3 linearly independent eigenvectors, A is diagonalizable
2
1
0

We let p1 1 , p 2 0 , and p3 1 .
~
~
1 ~ 1
0

2 1 0
0 1
The matrix P 1
1
1 0
1 0 0
1
P AP 0 2 0 .
0 0 2
diagonalizes
A.
It
can
be
checked
that
119
1 2 2
Example 2: Find a matrix P that diagonalizes A 2 5 2 .

6 6 3
(Refer to Example 3 Section 6.1)
Solution:
120
Theorem 6.2.2 If an n n matrix A has n distinct eigenvalues, then A is diagonalizable.

Example 3:
1 1 1
3 1 and decide whether A is diagonalizable.

Find the eigenvalues of A 1
3 1 1
Solution:
Let det (I A) 0
( 2)( 2)( 3) 0 2 , 2 , 3
Since A has 3 distinct eigenvalues, therefore A is diagonalizable.
Example 4:
4 3 2
Find the eigenvalues of A 0 1 1 and decide whether A is diagonalizable.

0 0 2
Solution:
121
Example 5:
3 0 0
Determine whether the matrix A 0 2 0 is diagonalizable. If so, find a matrix P

0 1 2
1
that diagonalizes A and find P AP
Solution:
Let det (I A) 0
3, 2, 2
Since A has 2 distinct eigenvalues, we cannot use Theorem 6.2.2 to decide whether A is
diagonalizable or not. We therefore find how many linearly independent eigenvectors A
has.
0
0 x1 0
3

2
0 x2 0
To find the eigenvectors, consider (I A) x 0 , ie 0
~
~
0
1 2 x3 0
0 0 0 x1 0 0 0 0 0
0 1 0 x2 0 0 1 0 0
0 1 1 x 0 0 0 1 0
Now,
x2 0
x3 0
Let x1 s .
x1 s 1

~
x 0 0
3
x1 s 1

~
~
~
x 0 0
3
122
1

0

1

0

1 0 0 x1 0 1 0 0 0
0 0 0 x2 0 0 0 0 0
0 1 0 x 0 0 1 0 0
Now,
x1 0
x2 0
Let x3 s .
x1 0 0

~
x s 1
3
x1 0 0

~
~
~
x s 1
3
0

1

0

1

Since A has only 2 linearly independent eigenvectors, therefore by Theorem 6.2.1, A is
NOT diagonalizable.
123
Example 6:
2 0 0
Determine whether the matrix A 0 1 1 is diagonalizable. If so, find a matrix P

0 0 1
that diagonalizes A and find P 1 AP

Solution:
124
Theorem 6.2.3 Let D P 1 AP be a diagonal matrix for some invertible matrix P. Then
Ak PD k P 1
0 0 2
Example 7: Compute A where A 1 2 1

1 0 3
(Refer to Example 1)
13
2 1 0
0 1 diagonalizes A and
Solution: It was shown that the matrix P 1
1
1 0
1 0 0
1
D P AP 0 2 0 . Thus A13 PD13 P 1
0 0 2
13
2 1 0 1
1
0 1 0
1
1 0 0
0
213
0
0
213
0
16382
1 0 1 8190
8191
1 0 2 8191 8192
1 1 1 8191
0
16383
1 2 2
Example 8: Compute A where A 2 5 2 .

6 6 3
(Refer to Example 2)
10
Solution:
125
126
CHAPTER 7
APPLICATIONS OF LINEAR ALGEBRA
7.1) Geometric Linear Programming

Consider the following general linear programming problem in two variables
PROBLEM: Find values of x1 and x 2 that optimizes the objective function
Z c1 x1 c2 x2
subject to the constraints
a11x1 a12 x2 ()()() b1

a21x1 a22 x2 ()()() b2
am1 x1 am 2 x2 ()()() bm
and non-negativity constraints
x1 0 , x2 0
A pair of values ( x1 , x2 ) satisfying all of the constraints is called a feasible solution. The
set of all feasible solutions determines a subset of the x1 x2 - plane called the feasible
region. A solution that optimizes the objective function is called an optimal solution.
Note that each of the constraints defines a line or a region in the x1 x2 - plane. The feasible
region is therefore an intersection of finitely many lines or regions. The boundary points
of a feasible region that are intersections of two of the lines are called corner points.
Theorem 7.1.1
(a) If a feasible region of a linear programming problem is nonempty and bounded,
then the objective function attains the optimum values and these values occur at
the corner points of the feasible region
(b) If a feasible region of a linear programming problem is unbounded, then the
objective function may or may not attain the optimum values, however, if it
attains an optimum value, these values occur at the corner points of the feasible
region
127
Example 1:
Find values of x1 and x 2 that maximizes the objective function
Z x1 3x2
2 x1 3x2 24
x1 x2 7
x2 6
x1 0 , x2 0
Solution:
The value of the objective function at the corresponding corner points are shown below
( x1 , x2 )
(0,6)
(3,6)
(9,2)
(7,0)
(0,0)
Z x1 3x2
18
21
15
7
0
The maximum value of Z is 21, which is attained at ( x1 , x2 ) (3,6)

Example 2:
Find values of x1 and x 2 that minimizes the objective function
Z 2 x1 x2
2 x1 3x2 12
2 x1 3x2 0
x1 0 , x2 0
Solution:
The value of the objective function at the corresponding corner points are shown below
( x1 , x2 )
(3,2)
(6,0)
Z 2 x1 x2
4
12
The minimum value of Z is 4, which is attained at ( x1 , x2 ) (3,2)
128
7.2) Markov Chain

Suppose a mathematical system undergoes a process of change such that at any moment
it can occupy one of the finite numbers of states. For example, the share price of a stock
could be in one of three possible states: up, down, or unchanged.
Suppose that such a mathematical system changes with time from one state to another
and at scheduled times the state of the system is observed. If the state of the system at any
observation cannot be predicted with certainty, but the probability that a given state
occurs can be predicted by just knowing the state of the system at the preceding
observation, then the process of change is called a Markov chain or Markov process.
Consider a Markov chain with k possible states. The probability that the system is in state
i after it was in state j is denoted as pij , and is called the transition probability from state
j to state i. The matrix P [ pij ] is called the transition matrix of the Markov chain.
p11
For a 3-state Markov chain, the transition matrix has the form P p 21
p
31
p12
p 22
p32
p13
p 23 . The
p33
entry p32 is the probability that the sytem will change from state 3 to state 2. The sum of
each column p1 j p2 j p3 j 1. This is because if the system is in state j at one
observation, it is certain to be in one of the three possible states in the next observation.
Example 1
A car rental agency has three rental locations, denoted by 1,2, and 3. A customer may
rent a car from any of the three locations and return the car to any of the three locations.
The manager finds that customers return the cars to the various locations according to the
following probabilities
This matrix is a transition matrix of a Markov chain. The probability is p23 0.6 that a
car will be returned to Location 2 after being rented from Location 3. The probability is
p11 0.8 that a car will be returned to Location 1 after being rented from Location 1.
129
Example 2:
The alumni office of a university finds that 80% of its alumni who contribute to the
annual fund one year will also contribute the next year and 30% of those who do not
contribute one year will contribute the next. This can be viewed as a Markov chain wih
two states. State 1 may correspond to an alumnus contribute in any one year, State 2
correponds to the alumnus not contributing in that year. The transition matrix is given by
0.8 0.3
P
0.2 0.7
Definition
The state vector for an observation of a Markov chain with k states is a column vector
whose ith component is the probability that the system is in the ith state at that time.
In a Markov chain with k states, we might describe the possible state of the system at
x1

x
some observation time by a column vector x 2 , where x i is the probability that the
~

x
k
system is in the ith state at that time. Observe that the entries in any state vector for a
Markov chain are nonnegative and have a sum of 1, so such state vector is also called a
probability vector.
Theorem 7.2.1
Let P be the transition matrix of a Markov chain and let x
~
observation. Then x
~
( k 1)
Px
(k )
be the state vector of the kth
From Theorem 7.2.1, it follows that x

transition matrix P determine x
(k )
(n)
(n)
Pn x
~
( 0)
. The initial state vector x

~
(0)
and the
for n 1,2,...
130
Example 3:
Consider the transition matrix in Example 2. We are interested to know the probable
future contribution record of a new graduate who did not contribute in the initial year
after graduation. In this case, the system is initially in State 2 with certainty, so the initial
0
(0)
state vector is x . From Theorem 7.2.1, we have
~
1
x
(1)
( 2)
( 3)
Px
( 0)
Px
(1)
Px
( 2)
Compute x
~
0.8 0.3 0 0.3

0.2 0.7 1 0.7
0.8 0.3 0.3 0.45

0.2 0.7 0.7 0.55
0.8 0.3 0.45 0.525

0.2 0.7 0.55 0.475

( 4)
, x
~
(8)
, and x
(10)
0.6
. (Intepret this result)
~
0.4
The state vectors converges to a fixed vector as the number of observations increases.
For larger values of n, we have x
(n)
Example 4:
Consider the transition matrix in Example 1. Suppose a car is rented initially from
0

(0)
Location 2, so that the initial state vector is x 1 . From Theorem 7.2.1, we have
~
0

(1)
( 2)
Px
(0)
Px
~
(1)
0.8 0.3 0.2 0 0.3

0.1 0.2 0.6 1 0.2
0.1 0.5 0.2 0 0.5

0.8 0.3 0.2 0.3 0.40
0.1 0.2 0.6 0.2 0.37

0.1 0.5 0.2 0.5 0.23
Do the state vectors converge to a fixed vector as the number of observations increases?
131
Definition
A transition matrix P is regular if for some positive integer m, all the entries of the matrix
P m are positive. A Markov chain with a regular transition matrix is called a regular
Markov chain.
Theorem 7.2.2
Let P be a regular transition matrix, and x is any state vector. Suppose q1 , q2 ,...qk are
~
positive numbers such that q1 q2 ... qk 1 . Then
q1
q
(a) P n 2
q
k
q1 q1
q2 q2
as n , and
q k q k
q1

q2
(b) P n x
~

q
k
q1

q
The vector q 2 in Theorem 7.2.2 above is called the steady-state vector of the
~

q
k
regular Markov chain. To compute the steady-state vector, we make use of the following
theorem.
Theorem 7.2.3
The steady-state vector q of a regular transition matrix P is the unique vector that
~
satisfies the equation P q q

~
132
Example 5:
0.8 0.3
, we begin
To find the steady-state vector for the regular transition matrix, P
0.2 0.7
by solving the equation P q q or ( I P) q 0 . The linear system simplifies to
~
0.2 0.3 q1 0

0.2 0.3 q 2 0
1.5 0.6
1
The solution is q s where s
0.4
1.5 1
~
1 0.4
This means that over the long run, 60% of the alumni will contribute to the annual fund in
any one year, and 40% will not.
Example 6:
Refer to Example 1. The steady-state vector for the regular transition matrix,
34
0.8 0.3 0.2
0.5573
61
14

P 0.1 0.2 0.6 , is q 61 0.2295 . The entries give the long-run probabilities
~
13 0.2132
0.1 0.5 0.2
61
that any one car will be returned to Locations 1, 2, or 3, respectively. If the car agency
has a fleet of 10,000 cars, it should design its facilities so that there are at least 5,573
spaces at Location 1, at least 2,295 spaces at Location 2, and at least 2,132 spaces in
Location 3.
133
7.3) Leontief Economics Model

Matrix theory has been very successful in describing the interrelations among prices,
outputs, and demands in economic systems. We will discuss two models, the Leontief
Closed (Input-Output) Model and the Leontief Open (Production) Model. We evaluate
certain parameters, such as prices or output levels, in order to satisfy a desired economic
objective.
Leontief Closed Model

Consider an economic system consisting of a finite number of industries which we
number as industries 1,2,k. Over some fixed period of time, each industry produces an
output of some goods or services that is completely utilized in a predetermined manner
by the k industries. An important problem is to find a price structure to represent an
equilibrium position for the economy. Specifically, we wish to find suitable prices to be
charged for these k outputs so that for each industry, total expenditures equal total
income.
Let p i be the price charged by the ith industry for its output, eij be the fraction of the total
output of the jth industry purchased by the ith industry, for i, j 1,2,..., k . With such
definitions, we have
(i) pi 0 (ii) eij 0
(iii) e1 j e2 j ... ekj 1
p1
e11 e12 e1k

p2
e21 e22 e2 k
Form the price vector p , and the input-output matrix E
p
e
e
kk
k1 k 2
k
Now, in order that the total expenditures of each industry to be equal to its total income,
the following matrix equation must be satisfied
Ep p
~
or
(I E) p 0
~
It can be proved that the system E p p always has a nontrivial solution p whose
~
entries are nonnegative.
134
Example 1:
Three homeowners a carpenter, an electrician, and a plumber agree to make repairs in
three homes. They agree to work a total of 10 days each according to the following
schedule
Days of Work in Home of
Carpenter
Electrician
Plumber
Work performed by
Carpenter Electrician Plumber
2
1
6
4
5
1
4
4
3
The three of them must pay each other a reasonable daily wages, even for the work each
does on his or her own home. Their normal daily wages are about RM100, but they agree
to adjust their respective daily wages so that each homeowner will come out even, that is
the total paid out by each is the same as the total amount each receives. To satisfy the
equilibrium condition, we require that
Total expenditure = Total income
Let p i be the price charged by (daily wages of) the ith industry for its output (work done).
Let p1 , p2 , p3 be the daily wages of the carpenter, electrician, and plumber, respectively.
[Note: As an example, the carpenter pays a total of 2 p1 1 p2 6 p3 for the repair of his
own home and receives 10 p1 . For equilibrium, 2 p1 1 p2 6 p3 10 p1 or
0.2 p1 0.1 p2 0.6 p3 p1 ]
0.2 0.1 0.6
Based on the 10-day period, the input-output matrix is given by E 0.4 0.5 0.1
0.4 0.4 0.3
Since E p p , we have
~
0.2 0.1 0.6 p1 p1

0.4 0.5 0.1 p 2 p 2
0.4 0.4 0.3 p p
3 3
p1 31

Solving we obtained p p2 s 32 . The constant s is a scale factor which the
~
p 36
3
homeowners may choose for their convenience. Since their normal daily wages are
RM100, they may choose s 3 so that the corresponding daily wages are RM93, RM96,
and RM108, for the carpenter, electrician, and plumber respectively.
135
Leontief Open Model

While in the Leontief Closed Model, the outputs of k industries are distributed only
among themselves, the Leontief Open Model attempts to satisfy an outside demand for
the outputs. Portion of these outputs may still be distributed among the industries
themselves to keep them operating, but there is to be some excess, some net production,
with which to satisfy the outside demand.
In the Leontief Closed Model, the outputs of the industries are fixed, and the objective is
to determine the prices for these outputs so that the equilibrium condition, that
expenditures equal incomes, is satisfied.
In the Leontief Open Model it is the prices that are fixed, and our objective is to
determine levels of outputs of the industries needed to satisfy the outside demand.
Let x i be the monetary value of the total output of the ith industry, d i be the monetary
value of the output of the ith industry needed to satisfy the outside demand, and cij be the
monetary value of the output of the ith industry needed by the jth industry to produce one
unit of monetary value of its own input.
x1
d1

x2
d
Form the production vector x , the demand vector d 2 and the consumption
~
~

d
x
k
k
c11 c12 c1k
c 21 c 22 c 2 k
matrix C
c
k2
kk
k1
The theory further suggests that
x1

x2

x
k
c11 c12 c1k x1 d1

c21 c22 c2 k x 2 d 2

c
d
c
c
kk x k
k1 k 2
k
This leads to
x C x d
~
or
(I C) x d
~
for the demand to be exactly met, without any surpluses or shortages.
136
Example 2:
A town has three main industries, a coal mining operation (Industry 1), an electric powergenerating plant (Industry 2), and a local railroad (Industry 3).
To mine RM1 of coal, the mining operation must purchase RM0.25 of electricity to run
its equipment and RM0.25 of transportation for its shipping needs.
To produce RM1 of electricity, the generating plant requires RM0.65 of coal for fuel,
RM0.05 of its own electricity to run auxiliary equipments and RM0.05 for transportation.
To provide RM1 of transportation, the railroad requires RM0.55 of coal for fuel and
RM0.10 of electricity for its auxiliary equipment.
In a certain week the coal-mining operation receives orders for RM50,000 of coal from
outside the town, and the generating plant receives orders for RM25,000 of electricity
from outside. There is no outside demand for the local railroad.
How much must each of the three industries produce in that week to exactly satisfy their
own demand and the outside demand?
Solution:
For the one-week period, let x i denote monetary value of the total output of the ith
industry. The consumption matrix of the system is
0.65 0.55
0
C 0.25 0.05 0.10

0.25 0.05
0
The linear system ( I C ) x d is then

~
1.00 0.65 0.55 x1 d1 50,000
0.25 0.95 0.10 x 2 d 2 25,000

0.25 0.05 1.00 x d 0
3 3
756 542 470 50,000 102,087
1
220 690 190 25,000 56,163
The solution is given by x ( I C ) d
~
~
503
200 170 630 0 28,330

1
The total output of the coal-mining operation should be RM102,087, the total output of
the power-generating plant should be RM56,163, and the total output of the railroad
should be RM28,330.
137
7.4) Cryptography
Cryptography is the study of encoding and decoding secret messages.
Plaintext Uncoded message
Code Cipher
Ciphertext Coded message
Enciphering process of converting Plaintext to Ciphertext
Deciphering process of converting Ciphertext to Plaintext
Enciphering
Substitution Ciphers
The simplest ciphers, called substitution cipher, are those whereby each letter is replaced
by a different letter. A disadvantage of substitution cipher is that they preserve the
frequescies of individual letters, making it relatively easy to break the code by statistical
methods.
Example 1:
Convert the following plaintext to ciphertext by replacing a letter with another which is at
the next four position, ie A is replaced with D, B is replaced with E, , X is replaced
with A, Y is replaced with B, Z is replaced with C.
ACTUARIAL STUDIES
Solution:
Hill n-cipher
A polygraphic system is a cryptography system in which the plaintext is divided into sets
of n letters, each of which is replaced by a set of n cipher letters. A special class of
polygraphic systems called the Hill n-cipher, are based on matrix transformations. In a
Hill n-cipher, plaintext is grouped into sets of n letters and enciphered by an n n matrix
with integer entries.
Example 2:
1 2
to obtain the Hill 2-cipher for the plaintext I AM HIDING
Use the matrix
0 3
138
Solution:
Group the plaintext into pairs of letters IA, MH, ID, IN, GG. The dummy letter G is
introduced to fill out the last pair.
To encipher the pair IA
1 2 9 11

0 3 1 3
To encipher the pair MH
1 2 13 29
CX
0 3 8 24
To encipher the pair ID
1 2 9

0 3 4
17

12
QL
To encipher the pair IN
1 2 9 37

0 3 14 42
KP
To encipher the pair GG
1 2 7

0 3 7
21

21
KC
UU
The ciphertext is KCCXQLKPUU

Note that in Example 2 above, integers greater than 25 were replaced by their remainder
after division by 26. This technique is referred to as modular arithmetic, and has its
importance in cryptography.
Modular Arithmetic
Let m be a positive integer. An integer a is said to be equivalent to another integer b
modulo m, written a b (mod m) if a b is a multiple of m. Note that b is the remainder
when a is divided by m.
Example 3:
8 3 (mod 5) , 17 1 (mod 2) , 2 24 (mod 26) , 12 0 (mod 3)
The residue modulo m of an integer is the remainder when an integer is divided by m.

The set of residues modulo m, is denoted as Z m {0,1,2,..., m 1}
139
Example 4:
Find the residue modulo 26 of 87, 38 , and 26
Solution:
87 9 (mod 26) , 38 14 (mod 26) , 26 0 (mod 26)
Let a Z m {0,1,2,..., m 1} . A number a 1 Z m {0,1,2,..., m 1} is called a reciprocal

of a modulo m if aa 1 a 1a 1 (mod m)
Example 5:
(a) Consider 3 Z 26 and 9 Z 26 . Since (3)(9) 1 (mod 26) ,
therefore we have 31 9 (mod 26) and 9 1 3 (mod 26) .
(b) Consider 5 Z 26 and 21 Z 26 . Since (5)(21) 1 (mod 26) ,
therefore we have 5 1 21 (mod 26) and 211 5 (mod 26) .
(c) 7 1 15 (mod 26) and 15 1 7 (mod 26) .
(d) 111 19 (mod 26) and 19 1 11 (mod 26) .
(e) 17 1 23 (mod 26) and 231 17 (mod 26)
(f) 25 1 25 (mod 26)
(g) 11 1 (mod 26)
Deciphering
Every useful cipher must have a procedure for decipherment. In the case of Hill n-cipher,
decipherment uses the inverse (mod 26) of the enciphering matrix.
Let A (aij ) where aij Z m {0,1,2,..., m 1} . If there exist a matrix B (bij ) where
bij Z m {0,1,2,..., m 1} such that AB BA I (mod m) , then A is said to be invertible
modulo m.
Theorem 7.4.1 A square matrix A (aij ) where aij Z 26 {0,1,2,...,25} is invertible

modulo 26 if and only if det(A) is not divisible by 2 or 13.
140
Example 6:
9 6
is invertible modulo 26 because det( A) 3 is not divisible by
(a) The matrix A
4 3
2 or 13.
6 3
is NOT invertible modulo 26 because det( A) 12 is divisible
(b) The matrix A
2 3
by 2.
8 17
is NOT invertible modulo 26 because det( A) 39 is
(c) The matrix A
1 7
divisible by 13.
a b
where a, b, c, d Z 26 . Then
Theorem 7.4.2 Let A
c d
d b
(mod 26)
A 1 (ad bc) 1
c a
Example 7:
5 6
d b
. Then A 1 (ad bc) 1
(mod m)
Let A
2 3
c a
3 6
(mod 26)
[(5)(3) (6)(2)]1
2 5
3 6
(mod 26)
31
2 5
3 6
(mod 26)
9
2 5
27 54
(mod 26)

18 45
1 24
(mod 26)

8
19
141
Example 8:
5 6
.
Decipher the following Hill 2-cipher, which was enciphered by the matrix A
2 3
GTNKGKDUSK
Solution:
The numerical equivalent of this ciphertext is 7 20 14 11 7 11 4 21 19 11

1 24
(mod 26)
Next, we multiply each pair of ciphertext by A 1
8 19
The 1st pair
1 24 7 487 19

(mod 26)
8 19 20 436 20
The 2nd pair
1 24 14 278 18

(mod 26)
8 19 11 321 9
The 3rd pair
1 24 7 271 11

(mod 26)
8 19 11 265 5
The 4th pair
1 24 4 508 14

(mod 26)
8 19 21 431 15
The 5th pair
1 24 19 283 23

(mod 26)
8
19
11
361

23
These yield the message STRIKE NOW
Example 9:
Decipher the IMPORTANT message from Dr Ho, a message which was enciphered by
5 6
the matrix A
2 3
YLXFRENSGTYHGQERFMXELMVYWE
142
7.5) Age-Specific Population Growth

One of most common models of population growth used by demographers is the Leslie
model developed in the 1940s. This model describes the growth of the female portion of a
human or animal population.
Suppose the maximum age attained by any female in the population is L years and we
divide the population into n age classes so that each class is Ln in duration.
Let x1( 0) , x2( 0) ,..., xn(0) denote the initial number of females in the corresponding n classes.
We can form the initial age distribution vector, x

~
(0)
x1( 0 )
(0)
x
2 .

x (0)
n
As time progresses, the number of females within each of the n classes changes due to
three biological processes namely birth, death, and aging. We can observe the population
at discrete times. The Leslie model requires that the duration between any successive
observation times be the same as the duration of the age interval. Therefore we set
kL
for k 0,1,2,... . Next we define two demographic parameters.
tk
n
Let a i denote the average number of daughters born to each female during the same time
she is in the ith age class. Let bi denote the fraction (probability) of females in the ith age
class that can be expected to survive and pass into the (i+1)th age class. With these
definitions, we have ai 0 and 0 bi 1 .
x1( k )
(k )
x
kL
(k )
Define the age distribution vector x 2 at time t k
, where xi(k ) is the
~
n

x (k )
n
kL
number of females in the ith age class at time t k
. Therefore, we have
n
x1( k ) a1 x1( k 1) a2 x2( k 1) ... an xn( k 1)
and
x2( k ) b1 x1( k 1) , x3( k ) b2 x2( k 1) ,, xn( k ) bn1 xn( k11)
143
Using matrix notation, we can write the above two equations as

a1
x1( k )
( k ) b1
x2
0
x (k )
n 0
Or more compactly, x
(k )
Lx
~
( k 1)
(k )
Lk x
a3 an 1
b2
bn 1
an ( k 1)
x
0 1( k 1)
x
0 2
( k 1)
x
0 n
where L is the Leslie matrix
a1
b1
L 0
It follows that x
a2
a2
a3 an 1
b2
bn 1
an
0
0
( 0)
Example 1
Draw a diagram depicting the numbers of female population in each age class and at each
time for the case where the maximum attained age is 80 years and the population is
divided into 4 age classes so that each class is 20 years in duration.
Solution:
0Class 120Class 240Class 360Class 480
At time t 0
x1( 0)
x2( 0)
x3( 0 )
x4( 0)
At time t 20
x1(1)
x2(1)
x3(1)
x4(1)
At time t 40
x1( 2)
x2( 2)
x3( 2 )
x4( 2)
At time t 60
x1(3)
x2(3)
x3( 3)
x4(3)
At time t 80
At time t t k
x1( 4)
x1( k )
x2( 4)
x2( k )
x3( 4 )
x4( 4)
x4( k )
x3( k )
Notice that,
144
x1( k ) a1 x1( k 1) a2 x2( k 1) a3 x3( k 1) a4 x4( k 1)
x2( k ) b1 x1( k 1)
x3( k )
b2 x2( k 1)
b3 x3( k 1)
x4( k )
Therefore,
x1( k ) a1
(k )
x2 b1
x (k ) 0
3
x (k ) 0
4
a1
b1
The Leslie matrix is given by L
0
a2
0
b2
0
a2
0
b2
0
a3
0
0
b3
a3
0
0
b3
a4
0
0
x1( k 1)
( k 1)
x2
x ( k 1)
3
x ( k 1)
4
a4
0
0
Example 2:
Suppose that a certain animal population is divided into two age classes and has a Leslie
1 2
100
. Beginning with the initial age distribution vector x ( 0)
,
matrix L 2
~
0
5 0
(1)
calculate x , x
~
( 2)
( 3)
, x , x
~
( 4)
, and x
(5)
Solution:
145
Example 3:
Suppose that the maximum age attained by the females in a certain animal population is
15 years and we divide the population into three age classes with equal durations of five
0 4 3
years. Let the Leslie matrix for this population be L 12 0 0 . There are initially
0 1 0
4
(0)
1000 x1

(0)
1,000 females in each of the three age classes, so that x 1000 x2( 0 ) .
~
1000 x ( 0 )
(1)
( 2)
x
~
( 3)
0 4 3 1000 7000 x1(1)

( 0)
L x 12 0 0 1000 500 x2(1)
~
0 1 0 1000 250 x (1)

3
4
0 4 3 7000 2750 x1( 2 )

(1)
L x 12 0 0 500 3500 x2( 2 )
~
0 1 0 250 125 x ( 2 )

3
4
Lx
~
( 2)
0 4 3 2750 14375 x1( 3)

12 0 0 3500 1375 x2( 3)
0 1 0 125 875 x ( 3)

3
4
Therefore, after 15 years, there are 14,375 females between 0 and 5 years of age, 1,375
females between 5 and 10 years of age, and 875 females between 10 to 15 years of age.
146
Next, we investigate the eigenvalues and eigenvectors of the Leslie matrix.
(The 2 2 case)
a
Let L 1
b1
a2
. The characteristic equation of L is given by
0
p( ) det(I L) 0
a1 a2
0
b1
( a1 ) a2b1 0
2 a1 a2b1 0
Let q( )
a1
a2b1
for 0 . Therefore p( ) 0 implies q( ) 1
Since a1 , a2 , b1 are non-negative, we see that q( ) is monotonically decreasing for 0 ,

has a vertical asymptote at 0 , and q( ) 0 as . Consequently, there exists a
unique , say 1 such that q(1 ) 1 , that is the matrix L has a unique positive
x 1
eigenvalue. An eigenvector corresponding to 1 is x 1 b1 (Show this!)
~
x2 1
147
(The 3 3 case)
a1
Let L b1
0
a3
0 . The characteristic equation of L is given by

0
a2
0
b2
p( ) det(I L) 0
a1 a2
b1
0
a3
0 0
b2
( a1 )2 a2b1 a3b1b2 0
3 a12 a2b1 a3b1b2 0
Let q( )
a1
a2b1
a3b1b2
for 0 . Therefore p( ) 0 implies q( ) 1
Since a1 , a2 , a3 , b1 , b2 are non-negative, we see that q( ) is monotonically decreasing for

0 , has a vertical asymptote at 0 , and q( ) 0 as . Consequently, there
exists a unique , say 1 such that q(1 ) 1 , that is the matrix L has a unique
x1 1

positive eigenvalue. An eigenvector corresponding to 1 is x x2 b11 (Show this!)
~
x b1b2
3 12
148
Starting with an initial x

~
(0)
, it can be proved that x

~
(k )
1 x
~
( k 1)
for large value of k. This
implies that for large values of time, each age distribution vector is a scalar multiple of
the preceding age distribution vector, the scalar being the positive eigenvalue 1 .
Consequently, the proportion of females in each of the age classes becomes constant.
These limiting proportions can be determined from the eigenvector x .
~
Different values of 1 explains the different situation concerning the population. If

1 1 , the population is eventually increasing; if 1 1 , population is eventually
decreasing; while the population eventually stabilizes if 1 1 .
The case 1 1 corresponds to zero population growth. It can be shown that 1 1 is an
eigenvalue of a 2 2 Leslie matrix if and only if a1 a2b1 1 and is an eigenvalue of a
3 3 Leslie matrix if and only if a1 a2b1 a3b1b2 1 .
Accordingly, the expression a1 a2b1 and a1 a2b1 a3b1b2 are the net reproduction rate
of the respective population.
Example 4
Suppose that the maximum age attained by the females in a certain animal population is
15 years and we divide the population into three age classes with equal durations of five
0 4 3
years. Let the Leslie matrix for this population be L 12 0 0 .

0 1 0
4
The characteristic equation of L is given by
3 a12 a2b1 a3b1b2 0

3
3 2 0
8
3
8 16 3 0
(2 3)(42 6 1) 0
149
The positive eigenvalue is 1 32 .

x1 1 1

An eigenvector corresponding to 1 is x x2 b11 13
~
x b1b2 1
3 12 18
For large values of k, we have x

~
(k )
32 x
~
( k 1)
. Hence, every five years the number of
females in each of the three classes will increase by about 50%, as will the total number
of females in the population. Consequently, eventually the females will be distributed
among the three age classes in the ratio 1 : 13 : 181 or 18 : 6 : 1 . This corresponds to a
distribution of 72% of the females in the first class, 24% of the females in the second
class, and 4% of the females in the third age class.
150

(JAN2016) Lecture Linear Algebra PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

(JAN2016) Lecture Linear Algebra PDF

Uploaded by

Copyright:

Available Formats

MAT1024 LINEAR ALGEBRA & APPLICATIONS

SYSTEMS OF LINEAR EQUATIONS AND

Introduction to Systems of Linear Equations

A system of m linear equations in n unknowns can be written as

a11x1 a12 x2 a1n xn b1

am1 x1 am2 x2 amn xn bm

Example 1: A system of 3 linear equations in 5 unknowns

Example 2: A homogeneous system of 3 linear equations in 4 unknowns

Example 4 : Find a system of linear equations corresponding to the augmented matrix

-the system is said to be inconsistent

(II) Exactly one solution

-the system is said to be consistent

(III) Infinitely many solutions -the system is said to be consistent

Illustration for the system of 2 equations in 2 unknowns

Any homogeneous systems of linear equations has the trivial solution x1 0 , x2 0 ,,

Example 8: Consider the system of equations

Three types of Elementary Row Operation (ERO)

Interchange row 1 and row 2 :

Add -3 times of row 1 to row 2 :

Example 2 Perform the indicated elementary operation on the matrix

(Row-echelon form) (REF)

Neither REF nor RREF

In solving the systems of linear equations,

Example 7 Solve the system

Example 9 (System has infinitely many solutions)

already in row-echelon form. To solve the system, we may let x4 t .

1 . The reduced row-echelon form

intersect at a common point.

The augmented matrix for the system is 1 k 1 2

Matrices and Matrix Operations

The number a ij is the (i, j ) th entry of A.

A 1 n matrix is called a row matrix and an m 1 matrix is called a column matrix

Upper triangular matrix

Lower triangular matrix

A (aij ) where aij 0 for i j

A matrix A is symmetric if A A . The matrix A 2 5 6 is symmetric because

Let A (aij ) be an n n square matrix. Then the trace of A, tr(A) aii

(1)(1) (2)(4) (3)(7) (1)(2) (2)(5) (3)(8) (1)(3) (2)(6) (3)(9)

Inverses; Rules of Matrix

Theorem 1.4.1 If R is the reduced row-echelon form of an n n matrix A, then either R

contains a row of zeros

Theorem 1.4.2 (on transpose) Let A be a square matrix. Then

If A is a square matrix, and if B a matrix such that AB BA I , then A is said to be

Theorem 1.4.4 (on inverse) Suppose A and B are invertible. Then

(d) AT is invertible and ( AT ) 1 ( A1 )T

Example 4: Verify the above theorem for

Elementary Matrices and A Method for Finding Inverse

An n n matrix E is called an elementary matrix if it can be obtained from I n by a single

Theorem 1.5.1 Let E be the elementary matrix obtained by performing an elementary

(a) Perform ERO R2 3R2 on A, the resulting matrix is B 6 3 9 18 .

Perform this same ERO on I 3 , we obtained E 0 3 0 . It can be checked that

(b) Perform ERO R2 R3 on A, the resulting matrix is B 1 4 4 0 .

Perform this same ERO on I 3 , we obtained E 0 0 1 . It can be checked that

(c) Perform ERO R3 R3 (1) R1 on A, the resulting matrix is B 2 1 3 6 .

Perform this same ERO on I 3 , we obtained E 0 1 0 . It can be checked that

in row-echelon form, where A 1 3 0 2 .

Now the three elementary matrices E1 , E2 , E3 are related to A and D by

Theorem 1.5.2 If E is an elementary matrix, then E 1 exists and is an elementary matrix

Example 6: (Inverse ERO)

For example, E 0 1 0 is obtained from I 3 using the ERO R3 2R3 , so

(b) The inverse ERO for Ri R j is Ri R j .

For example, E 0 0 1 is obtained from I 3 using the ERO R2 R3 , so

(c) The inverse ERO for Ri Ri kR j is Ri Ri (k ) R j .