Matrices, transposes, and inverses

Math 40, Introduction to Linear Algebra

Wednesday, February 1, 2012

Matrix-vector multiplication: two views

• 1st perspective: A�
x is linear combination of columns of A
 
�� �� 44 �� �� �� �� �� �� � �
11 −
−22 33   11 −22
− 33 4
33 == 44 + 33
+ + 22
+ =
22 11 55 22 11 55 21
22
A
�x
• 2nd perspective: A�
x is computed as dot product of rows of A with vector �x
   
� � 4 4 � �
1 −2 3    �� � � 4
3 = 2 4  =
2 1 5 dot product of 1 and 3 21
2 5 2
A
�x
Notice that # of columns of A = # of rows of � x.
This is a requirement in order for matrix multiplication to be defined.
Matrix multiplication

What sizes of matrices can be multiplied together?

the matrix product AB
For m x n matrix A and n x p matrix B,
is an m x p matrix.
mxn nxp
“inner”
parameters
must match
“outer” parameters become
parameters of matrix AB

If A is a square matrix and k is a positive integer, we define

Ak = �A · A��· · · A�
k factors

Properties of matrix multiplication

Most of the properties that we expect to hold for matrix multiplication do....

A(B + C) = AB + AC
(AB)C = A(BC)
k(AB) = (kA)B = A(kB) for scalar k

.... except commutativity!!

In general, AB �= BA.
Matrix multiplication not commutative
Problems with hoping AB and BA are equal:
In general,
• BA may not be well-defined. AB �= BA.
(e.g., A is 2 x 3 matrix, B is 3 x 5 matrix)

• Even if AB and BA are both defined, AB and BA may not be

the same size.
(e.g., A is 2 x 3 matrix, B is 3 x 2 matrix)

• Even if AB and BA are both defined and of the same size, they
still may not be equal.
� �� � � � � � � �� �
1 1 1 2 2 4 3 3 1 2 1 1
= � = =
1 1 1 2 2 4 3 3 1 2 1 1

Truth or fiction?

Question 1 For n x n matrices A and B, is

A2 − B 2 = (A − B)(A + B) ?

No!! (A − B)(A + B) = A2 + AB − BA − B 2
� �� �
�=0

Question 2 For n x n matrices A and B, is (AB)2 = A2 B 2 ?

No!! (AB)2 = ABAB �= AABB = A2 B 2

Matrix transpose
Definition The transpose of an m x n matrix A is the n x m matrix
AT obtained by interchanging rows and columns of A,
i.e., (AT )ij = Aji ∀ i, j.

Example
 
� � 1 5
1 3 5 −2  3 3
A= AT =  
5 3 2 1  5 2
Transpose operation can be viewed as −2 1
flipping entries about the diagonal.

Definition A square matrix A is symmetric if AT = A.

Properties of transpose
apply twice -- get back
(1) (AT )T = A to where you started

(2) (A + B)T = AT + B T

(3) For a scalar c, (cA)T = cAT

(4) (AB)T = B T AT
To prove this, we show that
[(AB)T ]ij =
Exercise ..
.
Prove that for any matrix A, ATA is symmetric.
= [(B T AT )]ij
Special matrices
 
Definition A matrix with all zero entries is called 0 0 0 0
A = 0 0 0 0
a zero matrix and is denoted 0.
0 0 0 0

Definition A square matrix is upper-triangular

 
2 1
4 5
if all entries below main diagonal are zero. A = 0 6 0
0 0 −3
analogous definition for a lower-triangular matrix
 3 
−8 0 0 0
Definition A square matrix whose off-diagonal  0 −2 0 0
A=
 0

entries are all zero is called a diagonal matrix. 0 −4 0
0 0 0 1

 
Definition The identity matrix, denoted In, is the
1 0 0
I3 = 0 1 0
n x n diagonal matrix with all ones on the diagonal.
0 0 1

Identity matrix
 
Definition The identity matrix, denoted In, is the
1 0 0

I3 = 0 1 0
n x n diagonal matrix with all ones on the diagonal.
0 0 1

Important property
of identity matrix
If A is an m x n matrix, then
ImA = A and AIn = A.
If A is a square matrix, then
IA = A = AI.
TheThe notion
inverse
The of aofof
inverse inverse
matrix
a matrix
Exploration Let’s think
Let’s about inverses first in theincontext of real num-num-
Exploration
Exploration Consider theabout
think set ofinverses
real numbers,
first andcontext
the say that we have
of real
bers. Say we have equation
the
bers.equation
Say we have equation
3x = 3x23x =
=2
2
and weand
wantwetowant to solve
solve for
x. for x. so, multiply both sides by What1
todo we do?
and we want to for
solve To x.
do To do so, multiply both sides3 by 1obtain
3 to obtain
We multiply both 1 sides 1 of1 the 1equation by 12 to obtain
(3x) =(3x)(2)= (2) =⇒ =⇒ x = 3x. = 2 .
3 13 3 31 3 3 2
For R, For
1 (3x) = (2) of =⇒
1 =
x1 (3)
1. = .1.
3 isR,
the13 multiplicative 3 inverseinverse
is the multiplicative 3of 3 since 3 (3) =
3 since 3
multiplicative inverse 3
1
of 3 since (3) = 1
Now consider 3 following
the
Now consider systemsystem
the following of of NoticeNotice that that
we can we rewrite
can rewrite thesethese
The Now,
equationsinverse
equations
The consider
inverse ofthea linear
matrix
of a system
matrix Notice
equations as weascan rewrite equations as
equations
that
� � � � ��� � � �
Exploration 3x − 3x
5x
Let’s 2 Let’s think about inverses first−5
−= 5x
6
think =
about6 inverses first 3
in the 3 −5 x xof1 real 6 num-6
1
Exploration 1 2
incontext
the 1context
= =of real num-
bers. Say we
−2x have+−2x3xequation
+= 3x
−1 = −1 −2 −2
3 3x2 x2 −1 −1
bers. Say 1 we have 1
2 2
equation � ��� �� � ���� � ���� � �� �� �� �
3x = 3x 2 =2 A A �x �x �b �b
which we which
wantwetowant solve tofor
solvex for
and x x and
. x .
Howfordox.we 1 isolate 2 the vector � x� bysides
itselfby on1 LHS?
1 2
and weand want wetowant solveto solve for To x. doTo so,do multiply � � both
�so, multiply both sides 3 byto 1obtain
x 1
x1 3 to obtain
How
How do we isolate do we isolate
1 the 1vector the vector
1 x1= x by x = by
itself
2 on itselfthe LHS?LHS?
on the
(3x) =(3x)(2)= (2) =⇒ 2 =⇒ x 2
x = x. = . 2
3 3 3 3 � −3 3 �3
� �
1 −2 −5
For R, Multiply
Multiply 1
For
3 is the
R,
both both
1sides
is theofsides
multiplicative
matrixof matrix
multiplicativeinverse
equation equation
of
inverse 3
by since
of by
−3
3
−2 −3
−5
3 (3)
since = (3): = 1.
1 1.
: −3
�
3 � � �� � �� � � �� � � �� � � � � � �� �
3

The notion −3 −5 of inverse

−3 −5 3 −5 x1 3 −5 x 1 −3=
−3 −56 6
−5
Now consider the following
Now consider −3 system
the following
−2 of of
system =
NoticeNotice that−2 we −3
that can
we rewrite
can rewrite thesethese
−2 −3 −2 3−2 x32 x2 −2 −3 −1 −1
equationsequations
Now, consider the linear system � equations � �as we
� � equations �ascan rewrite equations as
� � �1 0� Notice x�� that �−13
1 0 x1 1
=� 3��
−13 ��� � � �
3x1 − 3x 5x12 − = 5x
62=6 0 1 =x2 3 −5 −9 −5 x1 x1 6 6
0 1 � �� x2 � −9 = =
−2x1 + −2x3x12 + −12 = −1
= 3x � �� � I −2 −2 3 3x2 x2 −1 −1
I � �� ���� �� � ����� � ���� � �� �� �� �
� � x1� A�−13A �x �x �b �b
which wetowant to solve 1 for x1xand xx1 2. =
−13
which we want solve
How for do x we and
isolate 2 . the
� vector
=x� 2� � x� by−9 itself on LHS?
� x −9
x
� ��the vector�xx�1= �� by� itself on the
2 1 �� �
How doHow we do we isolate
isolate the vector 3 x −5 = x by itself on the LHS?LHS? 6
? x2 1 x 2 = ?
Thus, the solution to (�) is −2x1 = 3−13 x and2 x 2 = −9.
�−9. �
−1
Thus, the solution to (�) is x 1 = −13
�both sides of��matrix equation and � x
� 2 = �
−3 −5
Multiply Multiply
bothwant sides ofequal
matrix equation by I −3 by−5 −2 −3 :
this to identity matrix, −2 −3 :
Lecture �
� � �� � �� Math �
� � ��
40,�Spring
� � � � �� �
7
−3 −5 3 −5 ���’12, x 1
� �Kindred
Prof.
−3 −56 6 Page 1
Lecture 7 −3 −5 Math3 40, −5 Spring ’12,x1 −3 −5
Prof. Kindred−3= −5 Page 1
−2 −3 =
−2 −3 −2 3−2 x32 −2x2 −3 −2 −3 −2 −3 −1
−1
� �� � � �
� �� � � �−13
� � � 1 �0� 1x�01 �x1 −13 = �� � � �
x1 1 0 x01 1 =x−3 −5 −9 6 −13
= 0 1 x2 = � −2−9 −3 −1 = −9
2
x2 0 � 1�� ��x2�� I
I � � � �
� � x1� �−13
x1 =
−13
=x2 −9
x2 −9
Matrix inverses

Definition A square matrix A is invertible (or nonsingular) if ∃ matrix

B such that AB = I and BA = I. (We say B is an inverse of A.)

Example
� � � �
2 7 4 −7
A= is invertible because for B = ,
1 4 −1 2
� �� � � �
2 7 4 −7 1 0
we have AB = = =I
1 4 −1 2 0 1
� �� � � �
4 −7 2 7 1 0
and likewise BA = = = I.
−1 2 1 4 0 1

The notion of an inverse matrix only applies to square matrices.

- For rectangular matrices of full rank, there are one-sided inverses.

- For matrices in general, there are pseudoinverses, which are a generalization to matrix inverses.

� �
1 1
Example Find the inverse of A = . We have
1 1
� �� � � � � � � �
1 1 a b 1 0 a+c b+d 1 0
= =⇒ =
1 1 c d 0 1 a+c b+d 0 1
=⇒ a + c = 1 and a + c = 0 Impossible!
� �
1 1
Therefore, A = is not invertible (or singular).
1 1

Take-home message: Not all square matrices are invertible.

Lecture 7 Math 40, Spring ’12, Prof. Kindred Page 1

Important questions:
• When does a square matrix have an inverse?
• If it does have an inverse, how do we compute it?
• Can a matrix have more than one inverse?

Theorem. If A is invertible, then its inverse is unique.

Proof. Assume A is invertible. Suppose, by way of contradiction, that the
inverse of A is not unique, i.e., let B and C be two distinct inverses of A.
Then, by def’n of inverse, we have
BA = I = AB (1)
and CA = I = AC. (2)
It follows that
B = BI by def’n of identity matrix
= B(AC) by (2) above
= (BA)C by associativity of matrix mult.
= IC by (1) above
= C. by def’n of identity matrix
Thus, B = C, which contradicts the previous assumption that B �= C.
⇒⇐ So it must be that case that the inverse of A is unique. �

The inverse of a matrix A is unique,

Take-home message:
and we denote it A−1.

Theorem (Properties of matrix inverse).

(a) If A is invertible, then A−1 is itself invertible and (A−1)−1 = A.

Lecture 7 Math 40, Spring ’12, Prof. Kindred Page 2

(b) If A is invertible and c �= 0 is a scalar, then cA is invertible and
(cA)−1 = 1c A−1.

(c) If A and B are both n × n invertible matrices, then AB is invertible

and (AB)−1 = B −1A−1.
“socks and shoes rule” – similar to transpose of AB
generalization to product of n matrices
(d) If A is invertible, then AT is invertible and (AT )−1 = (A−1)T .

To prove (d), we need to show that there is some matrix such that
AT = I and AT = I.

Proof of (d). Assume A is invertible. Then A−1 exists and we have

(A−1)T AT = (AA−1)T = I T = I
and
AT (A−1)T = (A−1A)T = I T = I.
So AT is invertible and (AT )−1 = (A−1)T . �

Question: If A and B are invertible n × n matrices,

what can we say about A + B?

There is no guarantee A + B is invertible even if A and B themselves are

invertible! In other words, we cannot say that (A + B)−1 = A−1 + B −1.

How do we compute the inverse of a matrix, if it exists?

Lecture 7 Math 40, Spring ’12, Prof. Kindred Page 3

Inverse of a 2 × 2 matrix: Consider the special case where A is a 2 × 2
matrix with A = [ ac db ]. If ad − bc �= 0, then A is invertible and its inverse is
� �
1 d −b
A−1 = .
ad − bc −c a

� Exercise: Check that AA−1 = [ 10 01 ] = A−1A.

� �
−2 1
Example For A = , we have
3 −3
� � � 1
�
1 −3 −1 −1 −
A−1 = = 3 .
3 −3 −2 −1 − 23
We can easily check that
� �� 1
� � �
−2 1 −1 − 3 1 0
AA−1 = 2 =
3 −3 −1 − 3 0 1
and � �� � � �
1
−1 − −2 1 1 0
A−1A = 3 = .
−1 − 23 3 −3 0 1

How do we find inverses of matrices that are larger than 2 × 2 matrices?

Theorem. If some EROs reduce a square matrix A to the identity matrix

I, then the same EROs transform I to A−1.
   
  EROs  -1 
A I I A

If we can transform A into I, then we will obtain A−1. If we cannot do so,

then A is not invertible.

Lecture 7 Math 40, Spring ’12, Prof. Kindred Page 4

 � −1 −3 1 �
Example: Find the inverse of the matrix A = 3 6 0 .
1 0 1

   
−1 −3 1 1 0 0 −1 −3 1 1 0 0
R2 +3R1
 3 6 0 0 1 0  −−−−→  0 −3 3 3 1 0 
R3 +R1
1 0 1 0 0 1 0 −3 2 1 0 1
 
1 3 −1 −1 0 0
−R1
−−−− →  0 −3 3 3 1 0 
R3 −R2
0 0 −1 −2 −1 1
 
1 0 2 2 1 0
R +R2
−−1−−→  0 −3 3 3 1 0 
−R3
0 0 1 2 1 −1
 
1 0 2 2 1 0
− 13 R2
−−−→  0 1 −1 −1 − 13 0 
0 0 1 2 1 −1
 
1 0 0 −2 −1 2
R1 −2R3
−− −−→  0 1 0 1 23 −1 
R2 +R3
0 0 1 2 1 −1
Thus, A is invertible and its inverse is
 
−2 −1 2
A−1 =  1 23 −1 .
2 1 −1

Why does this work? =⇒ discussion next class

Lecture 7 Math 40, Spring ’12, Prof. Kindred Page 5