You are on page 1of 37

Homework 1

Find the determinant of the following matrix:

1
2

3
1
2

3
1
2

3
1
2

3

1
2

3
1
2

3
1
2

3
1
2

3
=
1
2

3
0 −
2

2

3

3
0 −
2

2

3

3
0 −
2

2

3

3


Theorem 8
If B = [b
ij
] is a square matrix of order n that is derived from
another square matrix A = [a
ij
] of order n, by adding correspondingly the
elements of a row (or column) to a multiple of the elements of another
row (or column), then det(B) = det(A).
Solution
Solution
1
2

3
0 −
2

2

3

3
0 −
2

2

3

3
0 −
2

2

3

3
=

2

2

3

3

2

2

3

3

2

2

3

3

Method of Cofactors
Expanding about the first column, the
determinant of the original matrix is equal to the
determinant of the submatrix A
11
.
Solution

2

2

3

3

2

2

3

3

2

2

3

3


=
− − + −
2
+ +
2
− − + −
2
+ +
2
− − + −
2
+ +
2


= − − −
1 +
2
+ +
2
1 +
2
+ +
2
1 +
2
+ +
2


Theorem 4
If A has a row (or column) that has a
common factor, then this k may be factored
out of the determinant of A, where a simplified
matrix B is formed.
Solution
1 +
2
+ +
2
1 +
2
+ +
2
1 +
2
+ +
2
=
1 +
2
+ +
2
0 −
2

2
+ −
0 −
2

2
+ −


Theorem 8
If B = [b
ij
] is a square matrix of order n that is derived from
another square matrix A = [a
ij
] of order n, by adding correspondingly the
elements of a row (or column) to a multiple of the elements of another
row (or column), then det(B) = det(A).
Solution
Method of Cofactors
Expanding about the first column, the
determinant of the original matrix is equal to the
determinant of the submatrix A
11
.
1 +
2
+ +
2
0 −
2

2
+ −
0 −
2

2
+ −
=

2

2
+ −

2

2
+ −

Solution

2

2
+ −

2

2
+ −


=
− − + +
− − + +


= − −
1 + +
1 + +



Theorem 4
If A has a row (or column)
that has a common factor, then
this k may be factored out of the
determinant of A, where a
simplified matrix B is formed.
1 + +
1 + +
= + + − + + = −
- Diagonal Method
Final Answer
1
2

3
1
2

3
1
2

3
1
2

3
= − − − − − −
=
3

2

3

2

3

2
+
3

2
+
3

2

3

2


2

3
+
2

3
+
2

3

2

3

2

3
+
2

3

+
3

2

3

2

2

3
+
2

3
+
3

2

2

3


3

2
+
3

2
+
2

3

2

3

3

2
+
2

3

=
1 1 ⋯ 1

0

1

0
2

1
2

2
⋮ ⋮ ⋱ ⋮

0

1

Vandermonde Matrices – matrices of the
following form: (our homework!)
det = det (

) =

>

Alexandre-Théophile Vandermonde showed that
the determinant of such matrix is:
Just trivia…
Outline
1. Matrix Adjoint
2. Matrix Inverse
3. Moore-Penrose Pseudoinverse
4. Solution to Systems of Linear Equations
a. Inverse Method
b.Cramer’s Rule
c. LU Decomposition

GOAL!!
Solve this first!
− + = 1
+ 2 + 3 = 6
4 − 2 + 3 = 5
=?
=?
=?
What method did you use?
- Elimination?
- Substitution?
- Guessing?
- Graphical? (Goodluck!)
- It’s in the handout, duh!
Answer:
= 1
= 1
= 1
Systems of Linear Equations
Methods for Solving Systems of Linear Equations:
- Elimination
- Substitution
- Guessing
- Graphical
- Inverse Method
- Cramer’s Rule
- LU Decomposition

Relaxation Methods
1. Jacobi Method
2. Gauss-Seidel Method
3. SOR

Krylov Subspace Methods
1. Conjugate Gradient
2. Biconjugate Gradient
3. Minimal Residual (GMRES)
Direct Iterative
Review
Recall: Submatrix, M
ij
=



=


=


=



=



=



=




Review
Recall: Minor or Complementary Minor, |M
ij
|
=



=


=


=


11
=


11
= −ℎ

23
=


23
= ℎ −

31
=


31
= −
Review
Recall: Cofactor or Algebraic Component, (-1)
i+j
|M
ij
|
=



11
= −ℎ
23
= ℎ −
31
= −
= 1
= 1
−1
+
= 1
= 2
= 3
−1
+
= −1
= 3
= 1
−1
+
= 1

11
= −ℎ
31
= −
23
= − ℎ
Matrix Adjoint
Adjoint – the transpose of the matrix of cofactors.
=




=

11

12

13

21

22

23

31

32

33

=

=

11

21

31

12

22

32

13

23

33

11
= −1
1+1
− ℎ = − ℎ

12
= −1
1+2
− = −

13
= −1
1+3
ℎ − = ℎ −

33
= −1
3+3
− = −
Adjugate Matrix
Adjoint Matrix
Matrix Inverse
Inverse – the adjoint of a matrix divided by its determinant.
=




−1
=
()
||

On your seats, get the inverse of matrix X:
First three to get the right answer will each get (+3) in 1
st
LE.
=
1 1 1
1 2 2
1 2 3

−1
=?
MATRIX
SCALAR
Matrix Inverse
Answer:

−1
=
2 −1 0
−1 2 −1
0 −1 1

Quick Formula for 2 X 2 matrices:
=


−1
=
1




Matrix Inverse
Do all square matrices have an inverse?
NO! Try getting the inverse of: =
1 1
2 2

Definition
A square matrix is called singular if and only if its
determinant is 0.
“A square matrix that is not invertible
is singular or degenerate. But if a matrix
is invertible, its inverse is unique.”
Matrix Inverse
Properties of Matrix Inverse:

1)
−1
=
−1
=

2)
−1 −1
=
3)
−1
=
−1

−1
, for a nonzero scalar, k
4)
−1
=
−1

5)
−1
=
−1

−1

6) det
−1
= det
−1

Matrix Inverse
Moore-Penrose Pseudoinverse, A
*
In MATLAB,
inv(A)
Computes the inverse of A.
pinv(A)
Computes the Moore-Penrose pseudoinverse of A.
Properties of Moore-Penrose Pseudoinverse:

1)



2)

=
3)


=


4)

=


5)

=



We won’t compute
this anymore…
Outline
1. Matrix Adjoint
2. Matrix Inverse
3. Moore-Penrose Pseudoinverse
4. Solution to Systems of Linear Equations
a. Inverse Method
b.Cramer’s Rule
c. LU Decomposition

GOAL!!
Systems of Linear Equations
Let’s go back to our example:
1 − 1 + 1 = 1
1 + 2 + 3 = 6
4 − 2 + 3 = 5
=?
=?
=?
This is the same as:
1 −1 1
1 2 3
4 −2 3

=
1
6
5

Systems of Linear Equations
Let’s go back to our example:
1 − 1 + 1 = 1
1 + 2 + 3 = 6
4 − 2 + 3 = 5
=?
=?
=?
This is the same as:
1 −1 1
1 2 3
4 −2 3

=
1
6
5

= Find X!!
Systems of Linear Equations
=

−1
=
−1


=
−1


=
−1

Given

Left-multiply the inverse of A in both sides

Property of the inverse matrix

Property of an identity matrix

=
1 −1 1
1 2 3
4 −2 3
−1
1
6
5

Inverse Method!
Inverse Method

=
1 −1 1
1 2 3
4 −2 3
−1
1
6
5
=
1
−7
12 1 −5
9 −1 −2
−10 −2 3
1
6
5

= −
1
7
12 1 +1 6 − 5(5) = 1
= −
1
7
9 1 −1 6 −2(5) = 1
= −
1
7
−10 1 −2 6 + 3 5 = 1
Determinant Adjoint
Inverse
Answer:
= 1
= 1
= 1
Cramer’s Rule
1 −1 1
1 2 3
4 −2 3

=
1
6
5

=
=
−1 1
2 3
−2 3
1 −1 1
1 2 3
4 −2 3
=
1 1
1 3
4 3
1 −1 1
1 2 3
4 −2 3
=
1 −1
1 2
4 −2
1 −1 1
1 2 3
4 −2 3

Solve for these determinants, in place of the inverse!
We can do away with the
inverse, but calculate
determinants only!
Forward & Backward Substitution
Let’s explore simpler cases…
Given: AX = B
What if A is a lower triangular matrix? For example,
What if A is an upper triangular matrix? For example,
1 0 0
2 −5 0
1 3 1

=
4
3
8

1 2 −7
0 1 −2
0 0 1

=
1
3
2

= 4
2 − 5 = 3
+3 + = 8
+2 −7 = 1
− 2 = 3
= 2

=
4
1
1

=
1
7
2

Forward & Backward Substitution
Given: AX = B
Backward Substitution:
1 0 0
2 −5 0
1 3 1

=
4
3
8

1 2 −7
0 1 −2
0 0 1

=
1
3
2

= 4
2 − 5 = 3
+3 + = 8
+2 −7 = 1
− 2 = 3
= 2

=
4
1
1

=
1
7
2

Forward Substitution:
LU Decomposition
- An upper or lower triangular matrix of coefficients is readily
solvable by back/forward substitution, respectively.
- Happily, all square matrices can be decomposed into a
product of a lower and upper triangular matrix! For example,
1 0 0
2 −5 0
1 3 1

=
4
3
8

1 2 −7
0 1 −2
0 0 1

=
1
3
2

Forward substitution Backward Substitution
LU
Decomposition!
1 2 3
1 −3 −1
1 −2 3
=
1 0 0
1 1 0
1 4/5 1
1 2 3
0 −5 −4
0 0 16/5

LU Decomposition
How to decompose a matrix into L and U??
The decomposition is not unique:

11

12

13

21

22

23

31

32

33
=
1 0 0

21
1 0

31

32
1

11

12

13
0
22

23
0 0
33

11

12

13

21

22

23

31

32

33
=

11
0 0

21

22
0

31

32

33
1
12

13
0 1
23
0 0 1

11

12

13

21

22

23

31

32

33
=

11
0 0

21

22
0

31

32

33

11

21

31
0
22

32
0 0
33

Doolittle’s Method
Crout’s Method
Cholesky’s Method
LU Decomposition
How to decompose a matrix into L and U??
The decomposition is not unique:

11

12

13

21

22

23

31

32

33
=
0 0

21
0

31

32

11

12

13
0
22

23
0 0
33

11

12

13

21

22

23

31

32

33
=

11
0 0

21

22
0

31

32

33

12

13
0
23
0 0

11

12

13

21

22

23

31

32

33
=

=

Doolittle’s Method
Crout’s Method
Cholesky’s Method
Unitriangular
LU Decomposition
Let’s do Doolittle’s
Method:
1 2 3
1 −3 −1
1 −2 3
=
1 0 0

21
1 0

31

32
1

11

12

13
0
22

23
0 0
33

11
= 1 =
11
1 + 0 0 + 0 0

21
= 1 =
21

11
+ 1 0 + 0(0)

31
= 1 =
31

11
+
32
0 +1(0)

12
= 2 =
12
1 +
22
0 + 0 0

22
= −3 =
21

12
+
22
1 + 0 0

32
= −2 =
31

12
+
32

22
+ 1(0)

13
= 3 =
13
1 +
23
0 +
33
0

23
= −1 =
21

13
+
23
1 +
33
0

33
= 3 =
31

13
+
32

23
+
33
(1)

11
= 1

21
= 1

31
= 1

12
= 2

22
= −5

32
= 4/5

13
= 3

23
= −4

33
= 16/5
This is
applicable to
any LU
decomposition
method!
LU Decomposition
How can we use LU Decomposition to solve systems of
linear equations?
=

=

= =

=

=
Given

Decompose A into an L or U triangular matrix

Let Y be the product of matrices U and X.

Solve for Y here, by forward substitution

Solve for X here, by backward substitution
LU Decomposition
Example: Solve for x
1
, x
2
, and x
3
using LU Decomposition.

1
+2
2
+ 3
3
= 5

1
− 3
2

3
= 2

1
−2
2
+ 3
3
= 9
In matrix form:
1 2 3
1 −3 −1
1 −2 3

1

2

3
=
5
2
9

From earlier:
1 2 3
1 −3 −1
1 −2 3
=
1 0 0
1 1 0
1 4/5 1
1 2 3
0 −5 −4
0 0 16/5

A = L U
LU Decomposition
1 0 0
1 1 0
1 4/5 1
1 2 3
0 −5 −4
0 0 16/5

1

2

3
=
5
2
9

=
=
1 0 0
1 1 0
1 4/5 1

1

2

3
=
5
2
9

=
=
5
−3
32/5

1 2 3
0 −5 −4
0 0 16/5

1

2

3
=
5
−3
32/5

=
1
−1
2

This is the Final Slide
Questions?
Homework 2. Solve the following system using
the (a) Inverse Method, (b) Cramer’s Rule, and
(c) LU Decomposition by Doolittle’s Method.
19 4 3
15 3 2
4 3 3
   
  
   
z x y
z y x
z y x