Roundoff in Gaussian Elimination

•• What
  is the exact solution to 3 digits?

• What is the computed result?

Roundoff in Gaussian Elimination
•  Permute the linear system (swap rows)
  𝑥1 + 𝑥 2=2
 𝜖 𝑥 + 𝑥 =1
1 2

• Gaussian Elimination works fine:

Roundoff in Gaussian Elimination
•• Given
  the previous linear system

• Swap rows and solve

Errors
• Look
  at the triply nested for loop:
for
for
for

• Roundoff error in all quantities:

• has undergone previous operations
• If is large, roundoff is amplified
• Small pivot elements lead to large multipliers and worse
roundoff errors
Partial Pivoting
•• When selecting a pivot equation, select the equation with the largest magnitude leading
 coefficient.
1) Select the pivot row:

2) Swap rows:

3) Eliminate (factors and ):

Partial Pivoting (cont.)
•• Select
  the next pivot

• Swap rows:

• Eliminate:
Partial Pivoting (cont.)
•• Row-echelon
  system:

• Perform back-substitution

 standard Gaussian elimination:

Partial Pivoting (example)
• Consider
  the following linear system and solution

• Reduce using Gaussian elimination and calculate

• Reduce using Gaussian elimination with partial pivoting and

calculate
Complete pivoting
• Select the pivot equation with the largest maximum value in
the row
• Improves round-off error when the linear system is not well
balanced
• If all coefficients in a row are of the same order of magnitude, the
system can generally be thought of as well balanced
• Requires swapping columns
• implementation is much harder
• calculation is slower
• Generally not used – I’m unaware of any commercial
applications that use complete pivoting
Scaling
•  Consider the linear system:

• Using partial or complete pivoting, equation 1 will be

selected as the pivot
• This will result in similar round-off errors:
Scaling
•  Note that this linear system

can be represented as a scaled version of each row:

by multiplying each row by , , and respectively

• Therefore, the order of pivot equations used depends on
some external factor when using pivoting
Scaling
•  Take a linear system with a large variation in values:

• This system can be scaled by dividing each row by the

absolute value of its highest magnitude component:

• Identify the pivot point.

• This method is never used. Why?
Scaled partial pivoting
• Scaling
  causes unnecessary roundoff error
• Scaling is only necessary for locating the pivot
• Actual division occurs during elimination – only do it once

• Maintain a scaling vector :

• is the largest element in row

• Maintain a row index vector :
Scaled partial pivoting
(algorithm)
•    3129   1

for to
𝑠= 22
9724 [ ] [] 𝑙= 2
3
1. set to be the index associated with the largest ratio in the set:

2. swap with
3. perform elimination for rows to

• Demonstrate on the matrix:

  2 3 1

[ 0
0
5
2
0
5
2
2
]
Scaled partial pivoting (example
1)
•  Solve the following system using scaled partial pivoting:
  2 1

4 [] []
𝑠= 2 𝑙 = 2
3

• Step 1:

• Step 1:

  1

𝑥=
[]
3
3
1
3
Scaled partial pivoting (example
2)
• Solve
  the following system using scaled partial pivoting:

  2.01 1

• 1:
[ ] []
𝑠= 1.99 𝑙 = 2
11.2 3

• 2:

− 2.98
 
[ ]
𝑥= 1.91
0.967
Scaled partial pivoting
•  Advantages
• Invariant to scaling. Equivalent linear systems are operated on in
the same way, despite the equations being scaled.
• Fast. Maintaining the scale and index vectors reduce computational
speed (not complexity)
• Easy to implement. Relatively. You have to juggle some pointers.
• Robust. One of the best methods for avoiding loss of precision.
• Extendable. This algorithm is useful for: determinants,
decomposition, and finding matrix inverses

• What are the potential problems?

• We aren’t re-scaling values in at every iteration – this is considered
“ok”
Gaussian Elimination is used to rank
computer systems – the Top500 list

Performance doubling
period on average:
No 1 – 13.64 months
No 500 – 12.90 months

www.top500.org
c/o Lennart Johnsson