SSC2113: Linear Algebra

Chapter 8. Applications of Linear Algebra

Application 1. Balance chemical reactions

Glucose (𝐶6 𝐻12 𝑂6 ) with oxygen produces carbon dioxide and water.
The chemical equation is
𝑂2 + 𝐶6 𝐻12 𝑂6 → 𝐶𝑂2 + 𝐻2 𝑂

Balance this equation.

Application 1. Balance chemical reactions
◼ Insert unknowns
𝑥𝑂2 + 𝑦𝐶6 𝐻12 𝑂6 → 𝑧𝐶𝑂2 + 𝑤𝐻2 𝑂

◼ Compare the number of each atom on the left side with the number on the right
𝐎: 2𝑥 + 6𝑦 = 2𝑧 + 𝑤
𝐂: 6𝑦 = 𝑧
H: 12𝑦 = 2𝑤

◼ Rewriting these equations in standard form

2𝑥 + 6𝑦 − 2𝑧 − 𝑤 = 0
6𝑦 − 𝑧 = 0
12𝑦 − 2𝑤 = 0
Application 1. Balance chemical reactions
2 6 −2 −1 0
Augmented matrix 0 6 −1 0 0
0 12 0 −2 0
Reduced row echelon form
1 0 0 −1 0
1
0 1 0 − 0
6
0 0 1 −1 0
𝑤
Solution: 𝑥 = 𝑤, 𝑦 = and z = 𝑤.
6
It’s a convention to choose values so that all the unknowns are positive integers having no
common factors

6𝑂2 + 𝐶6 𝐻12 𝑂6 → 6𝐶𝑂2 + 6𝐻2 𝑂

Application 2. The Leontief Input-Output Model

Wassily Leontief
 A simple economy of three sectors

Three interdependent sectors: coal, steel, and

electricity. Coal mining
industry
▪ The coal mining industry uses coal, electricity and
steel (input) in the production of coal (output).
▪ The iron-steel industry uses coal, electricity and
steel (input) in the production of steel (output).
▪ The power industry uses coal, steel, and electricity
(input) in the production of electricity (output).
Power Iron-Steel
industry industry
 A simple economy of three sectors
◼ To model such a system, we start by expressing all
outputs and demands in money terms.
◼ Suppose that Coal mining
industry
▪ the production of each unit’s coal requires 0.2 unit’s coal,
0.4 unit’s steel, and 0.1 unit’s electricity.
▪ the production of each unit’s steel requires 0.3 unit’s coal,
0.1 unit’s steel, and 0.3 unit’s electricity.
▪ the production of each unit’s electricity requires 0.2 unit’s
of coal, 0.2 unit’s steel, and 0.2 unit’s electricity.
Power Iron-Steel
Coal Steel Electricity industry industry
Coal 0.2 0.3 0.2
Steel 0.4 0.1 0.2
Electricity 0.1 0.3 0.2
Each column shows what is needed by each industry.
A simple economy of three sectors Final demand

Coal Steel Electricity

Coal 0.2 0.3 0.2 Coal mining
Steel 0.4 0.1 0.2 industry
Electricity 0.1 0.3 0.2

The final demand for coal, steel, and electricity are 10, 5,
6, respectively.
Power Iron-Steel
Question: How much coal, steel, and electricity should be industry industry
produced to meet the final demand?
 A simple economy of three sectors
Coal Steel Electric
Coal 0.2 0.3 0.2
Question: How much coal, steel, and electricity should be
Steel 0.4 0.1 0.2
produced to meet the final demand?
Electric 0.1 0.3 0.2

The final demand for coal, steel, and

▪ Let x, y, and z be the total output of C, S, and E, respectively. electricity are 10, 5, 6, respectively.
▪ An observation: x, y, and z should be greater than the final
demand of C, S, and E respectively, because the input of C,
S, and E which we call the internal demand, is needed in
order to produce the final demand.
Total output = Internal demand + Final demand

▪ How can we find the internal demand of C, S, and E?

 A simple economy of three sectors
Coal Steel Electric
Coal 0.2 0.3 0.2
▪ Let x, y, and z be the total output of C, S, and E, respectively. Steel 0.4 0.1 0.2
▪ How can we find the internal demand of C, S, and E? Electric 0.1 0.3 0.2
the internal demand of
▪ C: 0.2𝑥 + 0.3𝑦 + 0.2𝑧
▪ S: 0.4𝑥 + 0.1𝑦 + 0.2𝑧
▪ E: 0.1𝑥 + 0.3𝑦 + 0.2𝑧
▪ Can be represented in matrix form: 𝐴𝐱
0.2 0.3 0.2 𝑥
where A = 0.4 0.1 0.2 and 𝐱 = 𝑦
0.1 0.3 0.2 𝑧
A: input-output matrix (technology matrix)
x: total production vector
 A simple economy of three sectors
Coal Steel Electric
Coal 0.2 0.3 0.2
Question: How much coal, steel, and electricity should be
Steel 0.4 0.1 0.2
produced to meet the final demand?
Electric 0.1 0.3 0.2

The final demand for coal, steel, and

Total output = Internal demand + Final demand electricity are 10, 5, 6, respectively.

0.2 0.3 0.2 𝑥

▪ Internal demand: Ax, where A = 0.4 0.1 0.2 and 𝐱 = 𝑦
0.1 0.3 0.2 𝑧
10
▪ Final demand: 𝐝 = 5
6
▪ Hence, the system becomes A𝐱 + 𝐝 = 𝐱
 A simple economy of three sectors
0.2 0.3 0.2 𝑥 10
Solve for the linear system A𝐱 + 𝐝 = 𝐱 A = 0.4 0.1 0.2 , 𝐱 = 𝑦 , 𝐝 = 5
0.1 0.3 0.2 𝑧 6
I−A 𝐱=𝐝

795
32
1985 24.84
𝐱= ≈ 20.68
96 18.36
1175
64
Application 3. Linear regression

◼ Dependent variable is a function of the independent variables 𝑥1 , 𝑥2 , … , 𝑥𝑘

𝑦 = 𝐹(𝑥1 , 𝑥2 , … , 𝑥𝑘 )

◼ Linear regression:
𝑦𝑖 = 𝛽0 + 𝛽1 𝑥1,𝑖 + 𝛽2 𝑥2,𝑖 + · · · +𝛽𝐾 𝑥𝑘,𝑖 + 𝜀𝑖 , where 𝑖 = 1,2, … , 𝑛

𝒀 = 𝑋𝜷 + 𝛆

Our goal is to obtain estimates of the

population parameters in the β vector
 Ordinary least squares (OLS) estimator
𝒀 = 𝑋𝜷 + 𝛆

• Our estimates of the population parameters are referred to as 𝜷. ෡

෡ that
• The criteria we use for obtaining our estimates is to find the estimator 𝜷
minimizes the sum of squared residuals ( σ 𝑒𝑖 2 ).
• The vector of residuals 𝐞 = 𝒀 − 𝑋𝜷 ෡
• The sum of squared residuals is 𝒆𝑇 𝒆
 Ordinary least squares (OLS) estimator 𝒀 = 𝑋𝜷 + 𝛆
• The criteria we use for obtaining our estimates is to find the estimator 𝜷 ෡ that minimizes
the sum of squared residuals ( σ 𝑒𝑖 2 ).
• The vector of residuals 𝐞 = 𝒀 − 𝑋𝜷 ෡
𝑻
𝑇 ෡
• The sum of squared residuals is 𝒆 𝒆 = 𝒀 − 𝑋𝜷 𝒀 − 𝑋𝜷 ෡
= 𝒀𝑻 𝒀 − 𝒀𝑻 𝑋 𝜷෡−𝜷 ෡ 𝑻𝑋𝑇 𝒀 + 𝜷 ෡ 𝑻 𝑋 𝑇 𝑋𝜷
෡
= 𝒀𝑻 𝒀 − 𝟐𝜷 ෡ 𝑻𝑋𝑇 𝒀 + 𝜷 ෡ 𝑻 𝑋 𝑇 𝑋𝜷
෡ (1)

• To find the 𝜷෡ that minimizes the sum of squared residuals, we need to take the derivative
of Eq. 1 with respect to 𝜷 ෡ . This gives us the following equation:
𝝏𝒆𝑇 𝒆
= − 2𝑋 𝑇 𝒀 + 2𝑋 𝑇 𝑋𝜷 ෡=𝟎
𝝏𝜷෡
• To check this is a minimum, we would take the derivative of this with respect to
𝛽መ again – this gives us 𝑋 𝑇 𝑋, which is positive definite if X is of full rank.
Ordinary least squares (OLS) estimator 𝒀 = 𝑋𝜷 + 𝛆

෡ = 𝑋𝑇 𝒀
𝑋 𝑇 𝑋𝜷

• Two things to note about the 𝑋 𝑇 𝑋 matrix

1. it is always square since it is k × k.
2. it is always symmetric.

• 𝑋 𝑇 𝑋 and 𝑋 𝑇 𝒀 are known from our data but 𝜷 ෡ is unknown.

If the inverse of (𝑋 𝑇 𝑋) exists, then pre-multiplying both sides by this inverse gives us
the following equation:

෡ = (𝑋 𝑇 𝑋)−1 𝑋 𝑇 Y
𝜷