Gaussian Elimination

Malay
c K. Das, mkdas@iitk.ac.in

1 Linear Equation
An equation that contains unknown x in polynomial form only, as shown below, is
known as algebraic equation.

an xn + an−1 xn−1 + . . . + a1 x + a0 = 0 (1)

An algebraic equation is known as linear algebraic equation when

an = an−1 = . . . = a2 = 0 and a1 6 = 0 (2)

A system of linear algebraic equations contain many equations and many unknowns.
The primary goal of linear algebra is to solve such system of linear equation. We have
many techniques to solve system of linear equations. We will discuss some of them.
Most common of them is the Gaussian elimination, which would be our staring point.1
we will see the technique first in its most primitive form. Afterward we will spend
some time going deeper into the technique.

2 Gaussian Elimination
Gaussian elimination intends to solve system of linear Equations. Let’s first see the
idea behind Gaussian elimination by taking a system with 2 equations and 2 un-
knowns

3x + 2y = 5 (3)
4x + 5y = 9 (4)
1 As the name suggests the technique is named after the Famous German mathematician and physicist

Carl Freidrich Gauss(1777-1855). Gauss is one of the most prolific mathematician and physicist ever
known. He started his life from a very humble background showed his acumen in mathematics at a
very early age. He contributed in various branches of mathematics and physics including algebra, vector
calculus, probability and electromagnetism. While the Guassian elimination technique was probably
developed in ancient China, the technique bears the name of Gauss to recognize the contributions of
Gauss in various fields of Mathematics.

1
 The goal now is to eliminate the coefficients below the diagonal elements, a step
known as forward elimination. Therefore, in this case the first step is

no change in first row 3x + 2y = 5 (5)

4 7 7
row2 (new)=row2 (old)-row1× y= (6)
3 3 3
Now the second step: from the equation 6,

y=1 (7)

and now the third and final step, known as back substitution: using equations 5 and 6

5 − 2y
x= =1 (8)
3
Let’s extend our understanding to a 3 × 3 system

x1 + x2 − x3 = 1 (9)
3x1 + x2 + x3 = 9
x1 − x2 + 4x3 = 8

Now after the first step of forward elimination, we have

x1 + x2 − x3 = 1 (10)
−2x2 + 4x3 = 6
−2x2 + 5x3 = 7

Elimination one more time and we have

x1 + x2 − x3 = 1 (11)
−2x2 + 4x3 = 6
x3 = 1

Now the back substitution provides

x1 = 3 (12)
x2 = −1
x3 = 1

We argue that the systems 9, 10, 11, 12 are equivalent to each other. Two linear
system are equivalent to each other if one can be deduced from the other through
elementary operations. Elementary operations involve: multiplication of an equation by
a constant, exchange of two equations of the same system, replacing an equation with
a linear combination of the equation with other equations of the system.

2
 We will revisit the simple linear systems once again. But for now let’s see how the
Gaussian elimination works for the most general system of equations

a11 x1 + a12 x2 + · · · + a1n xn = b1 (13)

a21 x1 + a22 x2 + · · · + a2n xn = b2
..
.
an1 x1 + an2 x2 + · · · + ann xn = bn

The above Equations may be written in a simpler way, known as index notation or
Einstein notation:
X
n
aij xj = bi where aij xj = (aij xj ) for i = 1, 2, . . . , n (14)
j=1

2.1 Forward Elimination

In forward elimination step our goal is do something to make ak+1,k = 0 where k =
1, 2, . . . , n. To do so, the first step is to set all elements of column 1, except the a11 , to
zero, as follows
a21
row2(new) = row2(old) − row1 × (15)
a11
a31
row3(new) = row3(old) − row1 ×
a11
..
.
an1
rown(new) = rown(old) − row1 ×
a11
Repeating the same procedure, we can make all the elements below diagonal to be
zero. The above step will change the A matrix and b vector, as follows

a11 x1 + a12 x2 + · · · + a1n xn = b1 (16)

0 0
a22 x2 + · · · + a2n xn = b20
..
.
0
ann xn = bn0

2.2 Back Substitution

Back substitution starts from the last row:
bn0
xn = 0
(17)
ann

3
The (n − 1)-th row now provides the value of xn−1 :
0
bn−1 0
− an−1,n xn
xn−1 = 0 (18)
an−1,n−1

In general, xi for i = n − 1, n − 2, . . . , 1 may be obtained from

0
Pn 
0

bi − j=i+1 aij xj
xi = 0 (19)
aii

We have now introduced Gaussian elimination without any use of matrices. We

are now ready to introduce the matrix notation and we will see how the change in
notation will help us to understand linear system in a more profound way.

3 Exercise
1. Select a system of 4 × 4 linear equations. Solve using Gaussian elimination.

2. Solve the above system of equations using Gaussian elimination in a computer

software (such as Matlab )
R

3. List the rules that creates equation 14 from 13