GE 122 Lec3 Direct Methods Handout

02/10/2021
NOT FOR SHARING OR DISTRIBUTION

Only for the educational purpose of GE 122
GE 122 L3: LINEAR SYSTEMS

(DIRECT METHODS)
Ransie Apura
Assistant Professor
UPD DGE
UP Diliman Department of Geodetic Engineering

RJAA | GE 122: Mathematical Methods in Geodetic Engineering

▪ Linear Systems and its

Solutions
o Definition and Types
o Linear System in Matrix
Form
▪ Direct Methods
OUTLINE o Inverse
o Cramer’s Rule
o Gauss Elimination and
Gauss-Jordan Elimination
❖ Reduced Row Echelon
o LU Decomposition
❖ Doolittle’s
❖ Cholesky’s
RJAA | GE 122: Mathematical Methods in Geodetic Engineering 2
1
02/10/2021

SOLVING LINEAR EQTNS W/ SEVERAL VARIABLES
REVIEW: How do you solve linear algebraic equations with several

unknown variables?
Example: Solve for the values of p, q, and r satisfying the following

equations:
𝑝+𝑞+𝑟 =3
𝑟 − 𝑞 = −2
𝑟+𝑞−𝑝 =1


Example: Solve for the values of p, q, and r satisfying the following

equations: Step 1: Consider any 2 equations and eliminate one of the
𝑝 + 𝑞 + 𝑟 = 3 1 variables. (e.g., adding eqtn 1 and 3 to eliminate p)
𝑟 − 𝑞 = −2 2 𝑝+𝑞+𝑟 =3
𝑟+𝑞−𝑝 =1
𝑟+𝑞−𝑝 = 1 3
2𝑞 + 2𝑟 = 4 → 𝑞 + 𝑟 = 2 4
Step 2: Consider equation 4 and 2 to solve for either r or q.
(e.g., eqtn 4 and 2 to solve for r)
𝑞+𝑟 =2
𝑟 − 𝑞 = −2
2𝑟 = 0 → 𝑟 = 0
Step 3: Do variable substitutions to solve for p and q.
𝑞=2 𝑝=1
2
02/10/2021

• Variable substitution is an ok method for solving linear equations

with few unknown variables, but it becomes taxing if we are
talking about 50 or 100 equations with corresponding 50 or 100
or more unknown variables.
• Imagine having to do 100 variable substitutions in order to solve

such problem!
• The first part of GE 122 is devoted to introducing higher-level

mathematics for solving such problem. We’ll be acquainted
with analytical and numerical methods for solving systems of
linear equations. Let’s talk about analytical methods, first!

MATRICES (IN GENERAL)

• Utilized for solving systems of linear equations (different
operations for easier solving).
• Use elementary row operations (primarily for determining

certain information about matrices).

3
02/10/2021

SYSTEM OF LINEAR EQUATIONS

• When we say systems of linear equations, it is a set of equations
in the form of:
a11x1 + a12x2 +… + a1nxn = b1
a21x1 + a22x2 + … + a2nxn = b2
…………………………
am1x1+ am2x2 + …+ amnxn = bm
• Here we have m number of equations with n number of

unknown variables.
RECAP: Last week, you were introduced to matrices. Its definition, types, and operations.
This week, we’ll be using those concepts in defining various analytical methods for solving
systems of linear algebraic equations.


• Instead of variable substitution as solution to these set of
equations, we’ll be utilizing matrices for ease of solution.
Step 1: Transform these systems of linear equations into its

corresponding matrix form/equation: 𝐴𝑋 = 𝐵
𝑥1 𝑏1
𝑎11 𝑎12 ⋯ 𝑎1n
𝑥2 𝑏2
𝑎21 𝑎22 ⋯ 𝑎2n
𝐴= ⋯ ⋮ B= ⋮
⋯ ⋯ ⋯ X=
⋮ ⋮
𝑎𝑚1 𝑎𝑚2 ⋯ 𝑎𝑚𝑛
𝑥𝑛 𝑏𝑚
4
02/10/2021


𝑥1 𝑏1
• Matrix Form: 𝐴𝑋 = 𝐵 𝑎11 𝑎12 ⋯ 𝑎1n
𝑥2 𝑏2
𝑎21 𝑎22 ⋯ 𝑎2n
𝐴= ⋯ ⋮ B= ⋮
⋯ ⋯ ⋯ X=
⋮ ⋮
𝑎𝑚1 𝑎𝑚2 ⋯ 𝑎𝑚𝑛
𝑥𝑛 𝑏𝑚
A: Matrix Coefficient of Unknowns NOTE: We have several analytical

methods for solving systems of
linear equations employing this
X: Matrix of Unknowns matrix form.
B: Column Vector of Constants

Matrix multiplication of A and X which results to matrix B


• Solution → is any set of values for the unknowns that satisfy all
equations.
o Consistent System → system of linear equations with either
single (unique) or infinite number of
solutions.
o Inconsistent System → system of linear equations with no
solution.
m → number of equations
n → number of unknown variables
o Overdetermined System: 𝑚>𝑛
o Determined System: 𝑚=𝑛
o Undetermined System: 𝑚<𝑛
10
5
02/10/2021

SOLUTIONS TO LINEAR SYSTEMS

• Direct Methods (Analytical)
o Inverse
o Cramer’s Rule
o Gauss Elimination and Gauss-Jordan Elimination
o LU Decomposition – Doolittle’s and Cholesky’s
• Indirect Methods (Numerical)

o Gauss-Seidel NOTE: Indirect Methods will be
covered in the 3rd major
o Jacobi division of the course

11

(1) Inverse Method

• We are solving for matrix X or matrix of unknowns
𝐴𝑋 = 𝐵
• Multiplying both sides with 𝐴−1 : 𝐴−1 𝐴𝑋 = 𝐴−1 𝐵
• Knowing that 𝐴−1 𝐴 = 𝐼: 𝑋 = 𝐴−1 𝐵
• Matrix equation will have no solution or an infinite number of

solutions if 𝐴−1 does not exist.
When will the inverse of a matrix exist or not exist?

12
6
02/10/2021

(1) Inverse Method

Example: Solve the following system of 3 equations with variables
p, q, and r using the inverse method (if applicable).
𝑝 + 𝑞 + 𝑟 = 3 Step 1: Re-arrange the unknowns per equation. All unknowns
𝑟 − 𝑞 = −2 must be all on the same side and constants on the other side
𝑟 + 𝑞 − 𝑝 = 1 of equation.
𝑝+𝑞+𝑟 =3
−𝑞 + 𝑟 = −2
−p + q + r = 1
Step 2: Transform into matrix form/equation 𝐴𝑋 = 𝐵
Step 2a: For matrix A: write down the coefficients of each
unknowns per line and enclosed these in a bracket.
1 1 1 1 1 1
𝐴 = 0 −1 1 = 0 −1 1
RJAA | GE 122: Mathematical Methods in Geodetic Engineering −1 1 1 −1 1 1 13
13

(1) Inverse Method

𝑝 + 𝑞 + 𝑟 = 3 Step 2b: For matrix X: this is the column vector of unknowns or
𝑟 − 𝑞 = −2 matrix of unknowns. It is assumed that A and X are
𝑟 + 𝑞 − 𝑝 = 1 conformable, sthence, number of columns of Ast is the number
of rows of X. 1 column of A corresponds to 1 row of X and so
on.
𝑝 𝑝
𝑋=𝑞= 𝑞
𝑟 𝑟

14
7
02/10/2021

(1) Inverse Method

𝑝 + 𝑞 + 𝑟 = 3 Step 2c: For matrix B: this is the column vector of constants. If
𝑟 − 𝑞 = −2 A and X are conformable, resulting size of matrix B is the
𝑟 + 𝑞 − 𝑝 = 1 number of rows of A and number of columns of X. Since,
equations are already arranged, list down constant values
(right side of equation) and enclosed these in a bracket.
3 3
𝐵 = −2 = −2
1 1

15

(1) Inverse Method

𝑝 + 𝑞 + 𝑟 = 3 Step 3: matrix equation is
𝑟 − 𝑞 = −2 1 1 1 𝑝 3
0 −1 1 𝑞 = −2
𝑟+𝑞−𝑝 = 1 𝑟 −1 1 1 1
Step 4: Evaluate 𝐴 (use any method if nothing is specified)

𝑨 = −𝟒 → ≠ 0 (A is nonsingular/invertible)
Step 5: Use 𝑋 = 𝐴−1 𝐵 to solve for X (i.e., solve for the inverse of
matrix A and multiply it to matrix B).

16
8
02/10/2021

(1) Inverse Method

𝑝 + 𝑞 + 𝑟 = 3 Step 5: Using Adjoint (Determinant) for determining 𝐴−1
𝑟 − 𝑞 = −2 1 −1
0
𝑟+𝑞−𝑝 = 1 2 2
−2 −1 −1 𝑇 1−2 0 2 1 1 −1 1
𝐴−1 = 0 2 −2 = −1 2 −1 =
−4 −4 4 2 4
2 −1 −1 −1 −2 −1
1 1 1
4 2 4

17

(1) Inverse Method

𝑝 + 𝑞 + 𝑟 = 3 Step 5: We now have 𝐴−1 and 𝐵. Solve for matrix X.
𝑟 − 𝑞 = −2 1 −1 1 1
0 3 + 0 −2 − 1
𝑟+𝑞−𝑝 = 1 2 2 2 2 𝑝
3 1 −1 1
1 1 −1 1
𝑋= −2 = 3 + −2 + 1 = 2 = 𝑞
4 2 4 1 4 2 4 0 𝑟
1 1 1 1 1 1
3 + −2 + 1
4 2 4 4 2 4

18
9
02/10/2021

(2) Cramer’s Rule

Solving for 𝑨𝑿 = 𝑩, m equations with n unknown variables, which
has a unique solution, the value of the ith unknown variable is:
𝑨𝒊 NOTE: Only applicable when
𝑿𝒊 = 𝐴 ≠ 0 (unique solution).
𝑨
where 𝐴 : determinant of the coefficient matrix A
𝐴𝑖 : determinant of the matrix resulting from A after
replacing its ith column with the constants in
column vector B.
*Partial solution: preferred method if you are only interested in

values of the few unknown variables.
19

(2) Cramer’s Rule

Example: Solve for q in the following system of 3 equations with
variables p, q, and r using Cramer’s Rule (if applicable).
𝑝 + 𝑞 + 𝑟 = 3 Step 1 to Step 4: Similar with Inverse Method
𝑟 − 𝑞 = −2 Step 5: Compared with Inverse Method that solves all
𝑟+𝑞−𝑝 = 1 unknown variables of X matrix, Cramer’s Rule will allow
solving of a particular unknown variable of interest.
Using equation
𝑨𝒊
𝑿𝒊 =
𝑨
identify unknown variable of interest: q (for this example)

20
10
02/10/2021

(2) Cramer’s Rule

𝑝 + 𝑞 + 𝑟 = 3 Step 6: Identify which column in the coefficient matrix or
𝑟 − 𝑞 = −2 matrix A are the coefficients of unknown q
𝑟+𝑞−𝑝 = 1 variable: column 2
Step 7: Form 𝐴2 by replacing column 2 of matrix A with the

constants in vector B.
1 3 1
𝐴2 = 0 −2 1
−1 1 1

21

(2) Cramer’s Rule

𝑝 + 𝑞 + 𝑟 = 3 Step 8: Evaluate 𝑿𝒊 = 𝑨𝒊 (i.e., 𝑋2 for this example since we are
𝑨
𝑟 − 𝑞 = −2 solving for q, the 2nd unknown variable in matrix X).
𝑟+𝑞−𝑝 = 1
1 3 1
0 −2 1
𝑋2 = −1 1 1 = −8 = 2
𝐴 −4
Use any method for determining determinant (if nothing is

specified). Try solving p and r using Cramer’s Rule.
22
11
02/10/2021

(3) Gauss Elimination

Gauss Elimination uses elementary row operations until row
echelon form (REF) is achieved for the matrix of interest.
A matrix is in REF if the following conditions are met:

• In every non-zero row, its leading entry appears to the left of
leading entries below it (i.e., leading entry is the 1st non-zero
element in a row).
• Elements below leading entry (column) are all zeros.
• Zero rows (row of zeros) appear at the bottom (if there are any).

23


Solving for 𝑨𝑿 = 𝑩,
• Form augmented matrix 𝐴|𝐵 .
• Elimination Phase → use elementary row operations until REF.
• Analyze REF if consistent or inconsistent. If consistent, analyze if

solution is unique or infinite.
• Back substitute (if consistent) to determine solution.

24
12
02/10/2021


p, q, and r using Gauss Elimination.
𝑟 − 𝑞 = −2 Step 5: Form augmented matrix 𝐴|𝐵 .
𝑟+𝑞−𝑝 = 1
1 1 1 3
𝐴|𝐵 = 0 −1 1 −2
−1 1 1 1
Step 6: Elimination phase until augmented matrix is in REF.

25


𝑝 + 𝑞 + 𝑟 = 3 Step 6: Elimination phase
𝑟 − 𝑞 = −2 1 1 1 3
𝐴|𝐵 = 0 −1 1 −2
𝑟+𝑞−𝑝 = 1
−1 1 1 1
1 1 1 3
= 0 −1 1 −2 R3=R1+R3
0 2 2 4
1 1 1 3
REF = 0 −1 1 −2 R3=2R2+R3
0 0 4 0
26
13
02/10/2021


𝑝 + 𝑞 + 𝑟 = 3 Step 7: Analyze REF (return augmented matrix to AX=B form).
𝑟 − 𝑞 = −2 ANALYZING REF: Check
1 1 1 𝑝 3
𝑟+𝑞−𝑝 = 1 bottom row to see if system
𝑞
has solution or no solution. ☺ 0 −1 1 = −2
0 0 4 𝑟 0
𝑝 1 System is consistent with unique solution!
𝑋= 𝑞 = 2
𝑟 0 Step 8: row 3 gives 0p + 0q + 4𝑟 = 0 → 𝑟 = 0
back substitute 𝑟 = 0 to row 2: −1 𝑞 + 1 𝑟 = −2
−𝑞 + 0 = −2 → 𝑞 = 2
back sub r and q to row: 𝑝 + 𝑞 + 𝑟 = 3 → 𝑝 + 2 = 3
𝑝=1 27
27

(4) Gauss-Jordan Elimination

Similar to Gauss Elimination that uses elementary row operations
but until reduced row echelon form (RREF) is achieve for the matrix
of interest.
A matrix is in RREF if the following conditions are met:

• Similar with REF but leading entry of each non-zero row is 1.
• Elements below leading entry (column) are all zeros.
• Zero rows (row of zeros) appear at the bottom (if there are any).
1 0 1 0 1 0 1 2
0 1 1 0 , 0 1 1 1
0 0 0 1 0 0 0 0
28
14
02/10/2021


Solving for 𝑨𝑿 = 𝑩,
• Form augmented matrix 𝐴|𝐵 .
• Elimination Phase → use elementary row operations until RREF.
• Analyze RREF if consistent or inconsistent. If consistent, analyze if

solution is unique or infinite.
• For unique solution, value of each unknown variable is simply

equated to the constant in last column of same row.
29


p, q, and r using Gauss-Jordan Elimination.
𝑟+𝑞−𝑝 = 1
1 1 1 3
𝐴|𝐵 = 0 −1 1 −2
−1 1 1 1
Step 6: Elimination phase until augmented matrix is in RREF.

30
15
02/10/2021


𝑝 + 𝑞 + 𝑟 = 3 Step 6: Elimination phase (We’ll use REF result from Gauss
𝑟 − 𝑞 = −2 Elimination example) 1 1 1 3
𝐴|𝐵 = 0 −1 1 −2
𝑟+𝑞−𝑝 = 1 0 0 4 0
1 1 1 3 1
= 0 −1 1 −2 R3= R3
4
0 0 1 0
1 1 0 3
𝑅1 = −1 𝑅3 + 𝑅1
= 0 −1 0 −2
𝑅2 = −1 𝑅3 + 𝑅2
0 0 1 0
31


𝑟 − 𝑞 = −2 1 0 0 1
= 0 1 0 2 R1=R2+R1; R2= −1 R2
𝑟+𝑞−𝑝 = 1
0 0 1 0
Step 7: Analyze RREF (return augmented matrix to AX=B form).
𝑝 1 1 0 0 𝑝 1
𝑋= 𝑞 = 2
𝑟 0
0 1 0 𝑞 = 2
0 0 1 𝑟 0
System is consistent with unique solution!

32
16
02/10/2021

*Reversed Gauss??
• Gauss and Gauss-Jordan Elimination both uses elementary row
operations to transform matrix A (side of augmented matrix)
into an upper triangular matrix. Back substitution of unknown
variables happen from bottom row to top.
• Now what if instead of upper triangular matrix, we transform it

to a lower triangular matrix? We’ll be doing forward substitution
instead of back substitution (top row to bottom).
𝐴𝑋 = 𝐵 L𝑋 = 𝐶
L is a lower triangular matrix and C is a resulting matrix of
constants after Reversed Gauss.
33

*Reversed Gauss??
Solving 𝐴𝑋 = 𝐵 L𝑋 = 𝐶
• Elementary row operations for LX=C.
• Forward substitution (1st equation), solve for the only unknown.
• Solve 2nd equation using solution from previous equation.
• For the next, do similarly, etc.

34
17
02/10/2021

*Reversed Gauss??
p, q, and r using Reversed Gauss.
𝑟+𝑞−𝑝 = 1
1 1 1 3
𝐴|𝐵 = 0 −1 1 −2
−1 1 1 1
Step 6: Elimination phase until augmented matrix is in LX=C

form.

35

*Reversed Gauss??
𝑟 − 𝑞 = −2 1 2 0 5
= 0 −1 1 −2 R1=R1-R2
𝑟+𝑞−𝑝 = 1
−1 1 1 0
1 2 0 3
𝑝 1 = 1 −2 0 −3 R2=R2-R3
𝑋= 𝑞 = 2
𝑟
−1 1 1 1
0
Lower Triangular
2 0 0 2
Matrix = 1 −2 0 −3 R1=R1+R2
−1 1 1 1
36
18
02/10/2021

*Reversed Gauss??
𝑝 + 𝑞 + 𝑟 = 3 Step 7: Solve for the unknown variables
𝑟 − 𝑞 = −2 2 0 0 2
𝑟+𝑞−𝑝 = 1 = 1 −2 0 −3 R1=R1+R2
−1 1 1 1
𝑝 1
𝑋= 𝑞 = 2 Row 1 gives 2𝑝 = 2 → 𝑝 = 1
𝑟 0 Forward sub p to equation 2 𝑝 − 2𝑞 + 0𝑟 = −3 → 𝑞 = 2
Forward sub to equation 3 −𝑝 + 𝑞 + 𝑟 = 1 → 𝑟 = 0

37

Consistent or Inconsistent System?

• Example 1 used is a consistent system of linear equations with
unique solution. Now, how do we know if system is consistent or
not? And if it is consistent, does it have a unique or infinite
number of solutions?
REVIEW: Differentiate consistent from inconsistent system. You

might want to reread a previous slide about it. ☺

38
19
02/10/2021


Example 2: Assuming the augmented matrix of a linear system has
been brought to REF shown below. Determine its solution. Assume
that unknown variables are 𝑥1 , 𝑥2 , 𝑎𝑛𝑑𝑥3 .
1 0 1 2 system is consistent with infinite
𝑥3 can be zero and can also be
other values (i.e., zero times any
0 1 1 1 solutions.
value is zero). 0 0 0 0
Analyzing the augmented matrix, 3rd equation suggests that 𝑥3

can be any value (i.e., any element of real numbers). Back sub to
equation 2 then, back sub to equation 1.

39


1 0 1 2
0 1 1 1
3rd Equation: 𝑥3 = 𝑡 0 0 0 0
2nd Equation: 𝑥2 + 𝑥3 = 1 → 𝑥2 = 1 − 𝑡
1st Equation: 𝑥1 + 𝑥3 = 2 → 𝑥1 = 2 − 𝑡 𝑥1 2−𝑡
𝑥
𝑥 = 2 = 1 − 𝑡 ,𝑡 ∈ ℝ
*𝑡 is any element of real numbers
𝑥3 𝑡
This is an example of a consistent system with infinite number of solutions
40
20
02/10/2021


1 0 1 2
𝑥3 cannot be zero and cannot
be any real number because it
0 1 1 1
will not satisfy the equations. 0 0 0 1
Analyzing the augmented matrix, 3rd equation suggests that 𝑥3

cannot be solved, thus, system is inconsistent.

41

LU Decompostion (Factorization)
Solving 𝐴𝑋 = 𝐵 LU𝑋 = 𝐵
• Decompose matrix A into LU (lower and upper triangular

matrices).
• Solve LY=B where Y=UX (solve for Y by forward substitution).
• Solve UX=Y (solve for X by backward substitution).
*Decomposition plus forward and backward substitutions.

Next methods for solving systems of linear equations utilize LU Decomposition.
42
21
02/10/2021

LU Decompostion (Factorization)
LU factorization (for a given matrix) is not unique. There are many
LU methods that arise from applying constraints on other factors.
Methods Constraints
Doolittle's Lii =1, i = 1.2,...,n

Decomposition
Crout's Decomposition Uii =1, i = 1.2,...,n
Cholesky's L = UT
Decomposition
For GE 122, we’ll only be discussing Doolittle’s and Cholesky’s
43

(5) Doolittle’s Decomposition

• Solving 𝐴𝑋 = 𝐵 LU𝑋 = 𝐵
• Decompose matrix A into L and U matrices. 𝐴 = 𝐿𝑈

o U is an upper triangular and L is a lower triangular matrix.
o Off diagonal elements of L are pivot row multipliers used
during elimination.
𝑎11 𝑎12 𝑎13 𝑎14 1 0 0 0 𝑢11 𝑢12 𝑢13 𝑢14

𝑎21 𝑎22 𝑎23 𝑎24 𝑙21 1 0 0 0 𝑢22 𝑢23 𝑢24
𝐴= 𝑎 𝑎32 𝑎33 𝑎34 = 𝐿𝑈 = 𝑙31 𝑢33 𝑢34
31 𝑙32 1 0 0 0
𝑎41 𝑎42 𝑎43 𝑎44 𝑙41 𝑙42 𝑙43 1 0 0 0 𝑢44

44
22
02/10/2021


• Set-up LY=B
𝑦1 = 𝑏1
𝐿21 𝑦1 + 𝑦2 = 𝑏2
⋮ ⋮ ⋮
𝐿𝑘1 𝑦1 + 𝐿𝑘2 𝑦2 + ⋯ + 𝐿𝑘,𝑘−1 𝑦𝑘−1 + 𝑦𝑘 = 𝑏𝑘
⋮ ⋮ ⋮
𝐿𝑛1 𝑦1 + 𝐿𝑛2 𝑦2 + ⋯ + 𝐿𝑘,𝑘−1 𝑦𝑘−1 + ⋯ + 𝐿𝑛,𝑛−1 𝑦𝑛−1 + 𝑦𝑛 = 𝑏𝑛
• Solve for Y (Forward Sub)

𝑦1 = 𝑏1
𝑘−1
𝑦𝑘 = 𝑏𝑘 − ෍ 𝐿𝑘𝑗 𝑦𝑗 , 𝑘 = 2,3, . . . , 𝑛
𝑗=1
45


• Set-up UX=Y
𝑈11 𝑥1 + 𝑈12 𝑥2 + 𝑈13 𝑥3 + 𝑈14 𝑥4 + ⋯ + 𝑈1𝑛 𝑥𝑛 = 𝑦1
⋮ ⋮ ⋮
𝑈𝑘𝑘 𝑥𝑘 + 𝑈𝑘,𝑘+1 𝑥𝑘+1 + ⋯ + 𝑈𝑘𝑛 𝑥𝑛 = 𝑦𝑘
⋮ ⋮ ⋮
𝑈𝑛−1,𝑛−1 𝑥𝑛−1 + 𝑈𝑛−1,𝑛 𝑥𝑛 = 𝑦𝑛−1
𝑈𝑛𝑛 𝑥𝑛 = 𝑦𝑛
• Solve for X (Back Sub)

𝑥𝑛 = 𝑦𝑛 Τ𝑈𝑛𝑛
𝑛
1
𝑥𝑘 = 𝑦 − ෍ 𝑈𝑘𝑗 𝑥𝑗 , 𝑘 = 𝑛 − 1, 𝑛 − 2, 𝑛 − 3, . . . , 1
𝑈𝑛𝑛 𝑘
𝑗=𝐾+1

46
23
02/10/2021


a, b, c, and d using LU Decomposition (Doolittle’s).
4𝑎 + 2𝑏 + 4𝑐 = 20 Step 1: All unknowns are on one side and constants are
2𝑎 + 2𝑏 + 3𝑐 + 2𝑑 = 36 on the other side of equations.
4𝑎 + 3𝑏 + 6𝑐 + 3𝑑 = 60 Step 2: Transform into matrix form/equation AX=B
2𝑏 + 3𝑐 + 9𝑑 = 122 4 2 4 0 𝑎 20
𝐴= 2 2 3 2 𝑋= 𝑐 𝑏 𝐵 = 36
4 3 6 3 60
0 2 3 9 𝑑 122
4 2 4 0 𝑎 20
2 2 3 2 𝑏 = 36
4 3 6 3 𝑐 60
0 2 3 9 𝑑 122
47


4𝑎 + 2𝑏 + 4𝑐 = 20 Step 3: Set-up LY=B
2𝑎 + 2𝑏 + 3𝑐 + 2𝑑 = 36 1 0 0 0 𝑌𝑎 20
4𝑎 + 3𝑏 + 6𝑐 + 3𝑑 = 60 𝑙21 1 0 0 𝑌𝑐
= 36
2𝑏 + 3𝑐 + 9𝑑 = 122 𝑙31 𝑙32 1 0 𝑌𝑏 60
𝑙41 𝑙42 𝑙43 1 𝑌𝑑 122
Solve for the elements of L and U matrices. As you solve

for L elements, U is consequently formed from matrix A
by row elementary operations.

48
24
02/10/2021


4𝑎 + 2𝑏 + 4𝑐 = 20 Step 3: L and U elements
2𝑎 + 2𝑏 + 3𝑐 + 2𝑑 = 36 4 2 4 0 1 0 0 0
𝑙21 1 0 0
4𝑎 + 3𝑏 + 6𝑐 + 3𝑑 = 60 𝐴= 2 2 3 2 𝐿=
2𝑏 + 3𝑐 + 9𝑑 = 122 4 3 6 3 𝑙31 𝑙32 1 0
0 2 3 9 𝑙41 𝑙42 𝑙43 1
Step 3a: Solve for the 1st column elements of L

𝑎21 2 1 𝑎31 4 𝑎41 0
𝑙21 = = = 𝑙31 = = =1 𝑙41 = = =0
𝑎11 4 2 𝑎11 4 𝑎11 4
49


4𝑎 + 2𝑏 + 4𝑐 = 20 Step 3a: Use 1st column L elements as pivot for row
2𝑎 + 2𝑏 + 3𝑐 + 2𝑑 = 36 elementary operations to matrix A (U transformation).
4𝑎 + 3𝑏 + 6𝑐 + 3𝑑 = 60 4 2 4 0
𝑅2 = 𝑅2 − 𝑙21 𝑅1
Transform A to ′ via the
𝐴′ = 0 1 1 2
2𝑏 + 3𝑐 + 9𝑑 = 122 𝐴
following operations 𝑅3 = 𝑅3 − 𝑙31 𝑅1
0 1 2 3
𝑅4 = 𝑅4 − 𝑙41 𝑅1
0 2 3 9
Step 3b: From matrix 𝐴′ , solve for the 2nd column

elements of L.
′ ′
𝑎32 1 𝑎42 2
𝑙32 = ′ = =1 𝑙42 = ′ = =2
𝑎22 1 𝑎22 1
50
25
02/10/2021


4𝑎 + 2𝑏 + 4𝑐 = 20 Step 3b: Use 2nd column L elements as pivot for row
2𝑎 + 2𝑏 + 3𝑐 + 2𝑑 = 36 elementary operations to matrix 𝐴′ (U transformation).
4𝑎 + 3𝑏 + 6𝑐 + 3𝑑 = 60 4 2 4 0
Transform ′ to 𝐴′′ via
𝐴′′ = 0 1 1 2
2𝑏 + 3𝑐 + 9𝑑 = 122 𝐴 𝑅3 = 𝑅3 − 𝑙 32 𝑅2
the following operations 𝑅4 = 𝑅4 − 𝑙 𝑅2 0 0 1 1
42
0 0 1 5
Step 3c: From matrix 𝐴′′ , solve for the 3rd column
elements of L.
′′
𝑎43 1
𝑙43 = ′ = =1
𝑎33 1
51


4𝑎 + 2𝑏 + 4𝑐 = 20 Step 3c: Use 3rd column L element(s) as pivot for row
2𝑎 + 2𝑏 + 3𝑐 + 2𝑑 = 36 elementary operations to matrix 𝐴′′ (U transformation).
4𝑎 + 3𝑏 + 6𝑐 + 3𝑑 = 60 4 2 4 0
Transform ′′ to 𝐴′′′ via
𝐴′′′ = 0 1 1 2
2𝑏 + 3𝑐 + 9𝑑 = 122 𝐴 𝑅4 = 𝑅4 − 𝑙43 𝑅3
the following operations 0 0 1 1
0 0 0 4
Notice that 𝐴′′′ is already in REF and all L elements have

already been solved. This means that matrix 𝐴′′′ is the U
matrix (transformed from A via row elementary
operations).
52
26
02/10/2021


4𝑎 + 2𝑏 + 4𝑐 = 20 Step 4: Set-up L and U matrices with solved elements.
2𝑎 + 2𝑏 + 3𝑐 + 2𝑑 = 36 1 0 0 0 4 2 4 0
1
4𝑎 + 3𝑏 + 6𝑐 + 3𝑑 = 60 1 0 0
L= 2 U= 0 1 1 2
2𝑏 + 3𝑐 + 9𝑑 = 122 1 1 1 0 0 0 1 1
0 2 1 1 0 0 0 4
Step 5: Set-up LY=B and solve for Y (forward sub)

1 0 0 0 𝑌
1 𝑎 20
1 0 0 𝑌𝑏 = 36
2 𝑌 60
1 1 1 0 𝑐
𝑌 122
0 2 1 1 𝑑 53
53


4𝑎 + 2𝑏 + 4𝑐 = 20 Step 5: Set-up LY=B and solve for Y (forward sub)
2𝑎 + 2𝑏 + 3𝑐 + 2𝑑 = 36 𝑌𝑎 = 20
1
4𝑎 + 3𝑏 + 6𝑐 + 3𝑑 = 60 𝑌 + 𝑌𝑏 = 36 → 𝑌𝑏 = 26
2 𝑎
2𝑏 + 3𝑐 + 9𝑑 = 122 𝑌 + 𝑌 + 𝑌 = 60 → 𝑌 = 14𝑎 𝑏 𝑐 𝑐
2𝑌𝑏 + 𝑌𝑐 + 𝑌𝑑 = 122 → 𝑌𝑑 = 56
20
𝑌 = 26
14
56
54
27
02/10/2021


4𝑎 + 2𝑏 + 4𝑐 = 20 Step 6: Set-up UX=Y and solve for X (backward sub)
2𝑎 + 2𝑏 + 3𝑐 + 2𝑑 = 36 4 2 4 0 𝑎 20
4𝑎 + 3𝑏 + 6𝑐 + 3𝑑 = 60 0 1 1 2 𝑏 = 26
2𝑏 + 3𝑐 + 9𝑑 = 122 0 0 1 1 𝑐 14
0 0 0 4 𝑑 56
𝑎 6 4𝑑 = 56 → 𝑑 = 14
𝑋 = 𝑏𝑐 = −2 c + d = 14 → 𝑐 = 0
0
𝑑 b + c + 2d = 26 → 𝑏 = −2
14
4a + 2b + 4c = 20 → 𝑎 = 6

55

(6) Cholesky’s Decomposition

• Solving 𝐴𝑋 = 𝐵 L𝐿𝑇 𝑋 = 𝐵
• Decompose matrix A into L and LT matrices. 𝐴 = 𝐿𝐿𝑇

o L is a lower triangular and LT is the transpose of L (not the
same L matrix in Doolittle’s).
• Requires getting square roots of elements of A (must be positive
definite).
• A must also be symmetric (matrix A must be the same as its
transpose matrix).
𝐴 = 𝐴𝑇
56
28
02/10/2021


A = LLT
𝑎11 𝑎12 𝑎13 𝑙11 0 0 𝑙11 𝑙21 𝑙31
𝑎21 𝑎22 𝑎23 = 𝑙21 𝑙22 0 0 𝑙22 𝑙32
𝑎31 𝑎32 𝑎33 𝑙31 𝑙32 𝑙33 0 0 𝑙33
◦ Multiplying L and LT
2
𝑎11 𝑎12 𝑎13 𝑙11 𝑙11 𝑙21 𝑙11 𝑙31
𝑎21 𝑎22 𝑎23 = 𝑙11 𝑙21 2
𝑙21 + 𝑙222
𝑙21 𝑙31 + 𝑙22 𝑙32
𝑎31 𝑎32 𝑎33 𝑙11 𝑙31 𝑙21 𝑙31 + 𝑙22 𝑙32 2
𝑙31 2
+ 𝑙32 2
+ 𝑙33

57


• Equating elements of upper (or lower) triangular and A.
Compute for L column-wise (left to right):
𝑓𝑖𝑟𝑠𝑡𝑐𝑜𝑙𝑢𝑚𝑛:
2
𝑎11 = 𝑙11
⇒ 𝑙11 = 𝑎11
𝑎21 = 𝑙11 𝑙21 ⇒ 𝑙21 = 𝑎21 Τ𝑙11
𝑠𝑒𝑐𝑜𝑛𝑑𝑐𝑜𝑙𝑢𝑚𝑛:
𝑎31 = 𝑙11 𝑙31 ⇒ 𝑙31 = 𝑎31 Τ𝑙11
2 2 2
𝑎22 = 𝑙21 + 𝑙22 ⇒ 𝑙22 = 𝑎22 − 𝑙21
𝑎32 = 𝑙21 𝑙31 + 𝑙22 𝑙32 ⇒ 𝑙32 = 𝑎32 − 𝑙21 𝑙31 Τ𝑙22
2
𝑡ℎ𝑖𝑟𝑑𝑐𝑜𝑙𝑢𝑚𝑛: 𝑎33 = 𝑙31 2 2 2 2
+ 𝑙32 + 𝑙33 ⇒ 𝑙33 = 𝑎33 − 𝑙31 − 𝑙32
58
29
02/10/2021


Determining elements of L: First Column
• Based on equality, multiplication and symmetricity, the lower
triangular elements of A and LLT are related as follows:
𝑗
𝑎𝑖𝑗 = ෍ 𝑙𝑖𝑘 𝑙𝑗𝑘 , 𝑖 = 𝑗, 𝑗 + 1, … , 𝑛, 𝑗 = 1,2, … , 𝑛

𝑘=1
• The elements in the first column, where j=1, are:
𝑙11 = 𝑎11 𝑙𝑖1 = 𝑎𝑖1 Τ𝑙11, 𝑖 = 2,3, … , 𝑛

59


Determining elements of L: Succeeding Columns
• Proceeding to other columns, use the following expression
derived from the general formula expressing elements of A in
terms of elements of L by isolating the term lijljj, the last term of
the summation: 𝑗−1
𝑎𝑖𝑗 = ෍ 𝑙𝑖𝑘 𝑙𝑗𝑘 + 𝑙𝑖𝑗 𝑙𝑗𝑗

𝑘=1
• For main diagonal term where i=j, the formula is
𝑗−1
2
𝑙𝑗𝑗 = 𝑎𝑗𝑗 − ෍ 𝑙𝑗𝑘 , 𝑗 = 2,3, … , 𝑛
𝑘=1
60
30
02/10/2021


Determining elements of L: Succeeding Columns
• For non-diagonal term in the same column as ljj:
𝑗−1
𝑙𝑖𝑗 = 𝑎𝑖𝑗 − ෍ 𝑙𝑖𝑘 𝑙𝑗𝑘 ൘𝑙𝑗𝑗 , 𝑗 = 2,3, … , 𝑛 − 1, 𝑖 = 𝑗 + 1, 𝑗 + 2, … , 𝑛

𝑘=1
• From the two formulas for diagonal and off-diagonal terms, one
needs the following, to compute lij:
• aij, i.e., the element of A in the same position
● elements of L in the first j-1 columns in rows i and j.
• ljj computed first before the off-diagonal elements in the same

column.
61


a, b, c, and d using LU Decomposition (Cholesky’s).
4𝑎 + 2𝑏 + 4𝑐 = 20 Step 1 to 2: Same with Doolittle’s.
2𝑎 + 2𝑏 + 3𝑐 + 2𝑑 = 36 4 2 4 0 𝑎 20
4𝑎 + 3𝑏 + 6𝑐 + 3𝑑 = 60 2 2 3 2 𝑏 = 36
2𝑏 + 3𝑐 + 9𝑑 = 122 4 3 6 3 𝑐 60
0 2 3 9 𝑑 122
Step 3: Check if A is symmetric: 𝐴𝑇 = 𝐴

4 2 4 0
𝐴𝑇 = 2 2 3 2
4 3 6 3
0 2 3 9
62
31
02/10/2021


4𝑎 + 2𝑏 + 4𝑐 = 20 Step 4: Set-up LY=B
2𝑎 + 2𝑏 + 3𝑐 + 2𝑑 = 36 𝑙11 0 0 0 𝑌𝑎 20
4𝑎 + 3𝑏 + 6𝑐 + 3𝑑 = 60 𝑙21 𝑙22 0 0 𝑌𝑐
= 36
2𝑏 + 3𝑐 + 9𝑑 = 122 𝑙31 𝑙32 𝑙33 0 𝑌𝑏 60
𝑙41 𝑙42 𝑙43 𝑙44 𝑌𝑑 122
Solve for the elements of L matrix. Transpose L matrix

afterwards to produce 𝐿𝑇 .

63


4𝑎 + 2𝑏 + 4𝑐 = 20 Step 4: L elements
2𝑎 + 2𝑏 + 3𝑐 + 2𝑑 = 36 4 2 4 0 1 0 0 0
𝑙21 1 0 0
4𝑎 + 3𝑏 + 6𝑐 + 3𝑑 = 60 𝐴= 2 2 3 2 𝐿=
2𝑏 + 3𝑐 + 9𝑑 = 122 4 3 6 3 𝑙31 𝑙32 1 0
0 2 3 9 𝑙41 𝑙42 𝑙43 1
Step 4a: Solve for the 1st column elements of L

𝑙11 = 𝑎11 = 4 = 2 𝑎31 4
𝑙31 = = =2
𝑎11 2
𝑎21 2
𝑙21 = = = 1 𝑙 = 41 = 0 = 0
𝑎
𝑎11 2 41
𝑎11 2
64
32
02/10/2021


4𝑎 + 2𝑏 + 4𝑐 = 20 Step 4b: Solve for 2nd column elements of L.
2𝑎 + 2𝑏 + 3𝑐 + 2𝑑 = 36
4𝑎 + 3𝑏 + 6𝑐 + 3𝑑 = 60 𝑙22 = 𝑎22−𝑙212 = 2− 1 2 =1
2𝑏 + 3𝑐 + 9𝑑 = 122
𝑎32 − 𝑙21 𝑙31 3 − 2(1)
𝑙32 = = =1
𝑙22 1
𝑎42 − 𝑙41 𝑙21 2 − 0(1)

𝑙42 = = =2
𝑙22 1

65


4𝑎 + 2𝑏 + 4𝑐 = 20 Step 4c: Solve for 3rd column elements of L.
2𝑎 + 2𝑏 + 3𝑐 + 2𝑑 = 36
4𝑎 + 3𝑏 + 6𝑐 + 3𝑑 = 60 𝑙33 = 𝑎33− 𝑙31 2 +𝑙 2
32
= 6− 2 2+ 1 2 =1
2𝑏 + 3𝑐 + 9𝑑 = 122
𝑎43 − 𝑙41 𝑙31 + 𝑙42 𝑙32 3−( 0 2 + 2 1 )
𝑙43 = = =1
𝑙33 1
Step 4d: Solve for 4th column element(s) of L.
𝑙44 = 𝑎44− 2 +𝑙 2 +𝑙 2 = 9− 0 2 + 2 2 + 1 2 =2
𝑙41 42 43

66
33
02/10/2021


4𝑎 + 2𝑏 + 4𝑐 = 20 Step 5: Set-up L and LT matrices with solved elements.
2𝑎 + 2𝑏 + 3𝑐 + 2𝑑 = 36 2 0 0 0 2 1 2 0
4𝑎 + 3𝑏 + 6𝑐 + 3𝑑 = 60 L= 1 1 0 0 𝐿𝑇 = 0 1 1 2
2𝑏 + 3𝑐 + 9𝑑 = 122 2 1 1 0 0 0 1 1
0 2 1 2 0 0 0 2
Step 6: Set-up LY=B and solve for Y (forward sub)

2 0 0 0 𝑌𝑎 20
𝑌
1 1 0 0 𝑏 = 36
2 1 1 0 𝑌𝑐 60
0 2 1 2 𝑌𝑑 122
67


4𝑎 + 2𝑏 + 4𝑐 = 20 Step 6: Set-up LY=B and solve for Y (forward sub)
2𝑎 + 2𝑏 + 3𝑐 + 2𝑑 = 36 2𝑌𝑎 = 20 → 𝑌𝑎 = 10
4𝑎 + 3𝑏 + 6𝑐 + 3𝑑 = 60 𝑌𝑎 + 𝑌𝑏 = 36 → 𝑌𝑏 = 26
2𝑏 + 3𝑐 + 9𝑑 = 122 2𝑌𝑎 + 𝑌𝑏 + 𝑌𝑐 = 60 → 𝑌𝑐 = 14
2𝑌𝑏 + 𝑌𝑐 + 2𝑌𝑑 = 122 → 𝑌𝑑 = 28
10
𝑌 = 26
14
28

68
34
02/10/2021


4𝑎 + 2𝑏 + 4𝑐 = 20 Step 7: Set-up LTX=Y and solve for X (backward sub)
2𝑎 + 2𝑏 + 3𝑐 + 2𝑑 = 36 2 1 2 0 𝑎 10
4𝑎 + 3𝑏 + 6𝑐 + 3𝑑 = 60 0 1 1 2 𝑏 = 26
2𝑏 + 3𝑐 + 9𝑑 = 122 0 0 1 1 𝑐 14
0 0 0 2 𝑑 28
𝑎 6 2𝑑 = 28 → 𝑑 = 14
𝑋 = 𝑏𝑐 = −2 c + d = 14 → 𝑐 = 0
0
𝑑 b + c + 2d = 26 → 𝑏 = −2
14
2a + b + 2c = 10 → 𝑎 = 6

69

Upon completion of GE 122 Lecture 3, please proceed to

Seatwork 3 Guide.

70
35
02/10/2021

REFERENCES
• Kreyszig, Erwin (2011). “Advanced Engineering Mathematics,
10th Edition.” John Wiley and Sons.
• Strang, Gilbert (1988). “Linear Algebra and its Applications, 3rd
Edition.”
• Ymas, Sergio Jr. (2004). “Numerical Methods.” Ymas Publishing
House.

71

Questions? ☺
raapura@up.edu.ph

72
36

GE 122 Lec3 Direct Methods Handout

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

GE 122 Lec3 Direct Methods Handout

Uploaded by

Copyright:

Available Formats

02/10/2021

NOT FOR SHARING OR DISTRIBUTION

GE 122 L3: LINEAR SYSTEMS

UP Diliman Department of Geodetic Engineering

NOT FOR SHARING OR DISTRIBUTION

▪ Linear Systems and its

NOT FOR SHARING OR DISTRIBUTION

SOLVING LINEAR EQTNS W/ SEVERAL VARIABLES

REVIEW: How do you solve linear algebraic equations with several

Example: Solve for the values of p, q, and r satisfying the following

UP Diliman Department of Geodetic Engineering

NOT FOR SHARING OR DISTRIBUTION

SOLVING LINEAR EQTNS W/ SEVERAL VARIABLES

Example: Solve for the values of p, q, and r satisfying the following

NOT FOR SHARING OR DISTRIBUTION

SOLVING LINEAR EQTNS W/ SEVERAL VARIABLES

• Variable substitution is an ok method for solving linear equations

• Imagine having to do 100 variable substitutions in order to solve

• The first part of GE 122 is devoted to introducing higher-level

NOT FOR SHARING OR DISTRIBUTION

MATRICES (IN GENERAL)

• Use elementary row operations (primarily for determining

UP Diliman Department of Geodetic Engineering

NOT FOR SHARING OR DISTRIBUTION

SYSTEM OF LINEAR EQUATIONS

• Here we have m number of equations with n number of

NOT FOR SHARING OR DISTRIBUTION

SYSTEM OF LINEAR EQUATIONS

Step 1: Transform these systems of linear equations into its

NOT FOR SHARING OR DISTRIBUTION

SYSTEM OF LINEAR EQUATIONS

A: Matrix Coefficient of Unknowns NOTE: We have several analytical

B: Column Vector of Constants

NOT FOR SHARING OR DISTRIBUTION

SYSTEM OF LINEAR EQUATIONS

NOT FOR SHARING OR DISTRIBUTION

SOLUTIONS TO LINEAR SYSTEMS

• Indirect Methods (Numerical)

UP Diliman Department of Geodetic Engineering

NOT FOR SHARING OR DISTRIBUTION

(1) Inverse Method

• Matrix equation will have no solution or an infinite number of

UP Diliman Department of Geodetic Engineering

NOT FOR SHARING OR DISTRIBUTION

(1) Inverse Method

NOT FOR SHARING OR DISTRIBUTION

(1) Inverse Method

UP Diliman Department of Geodetic Engineering

NOT FOR SHARING OR DISTRIBUTION

(1) Inverse Method

UP Diliman Department of Geodetic Engineering

NOT FOR SHARING OR DISTRIBUTION

(1) Inverse Method

Step 4: Evaluate 𝐴 (use any method if nothing is specified)

UP Diliman Department of Geodetic Engineering

NOT FOR SHARING OR DISTRIBUTION

(1) Inverse Method

UP Diliman Department of Geodetic Engineering

NOT FOR SHARING OR DISTRIBUTION

(1) Inverse Method

UP Diliman Department of Geodetic Engineering

NOT FOR SHARING OR DISTRIBUTION

(2) Cramer’s Rule