The Condition of a Matrix

Jayadeep U. B., MED, NITC

July 27, 2006

1 What is meant by the Condition of a Matrix?

It is obvious that if the determinant of the coecient matrix of an n x n algebraic system is zero, i.e., the
system is singular, we can not solve the system, except for some special cases of right-side vector. In this
article, our interest does not lie in this extreme situation, but in cases where the matrices are approaching
singularity. It could be anticipated that we might have diculty in solving such a matrix. This property,
which we will be exploring more in the paragraphs below, is referred to as the condition of the matrix.
Well-conditioned matrices are the systems in which we do not face any diculty in solving. However, in
case of ill-conditioned systems, small errors, introduced due to numerical computation (like round-o errors)
or approximation, appear much magnied in the computed solution. Also, we might face diculty in getting
convergence, if we use an iterative solution pocedure like Gauss-Seidel Iteration.

2 Example of an Ill-Conditioned System

Consider
· the following
¸½ ¾2 × 2½system:
¾
1 1 x 0
=
1 0.999 y 1
The solution is obtained as: x = 103 , y = −103
Due
· to errors
¸½in computation,
¾ ½ ¾ if we obtained the system as:
1 1 x 0
=
1 1.001 y 1
the solution would be: x = −103 , y = 103

In this illustrative example, a small error in calculation of an element of the matrix (the error is only
about 0.2% ) caused an error of 200% in the solution!!!
The determinant of this matrix is -0.001, which can be considered as very small.

3 Geometrical Interpretation
The equations above can be considered as equations of two lines, x + y = 0 and x + 0.999y = 1. The slopes
of these lines are −1 and −1/0.999 = −1.001 respectively. The solution is given by the intersection of these
two lines. Since these lines are having nearly same slopes, there is a large range of solutions, for which the
equations are approximately satised. In other words, a small change in values of the coecients or constants
in the equation, signicantly alters the results obtained. Thus, the geometrical interpretation also points to
the diculty in obtaining a solution and the devastating eects of small numerical errors on the solution of
an ill-conditioned system.
If we consider a well-conditioned system of size 2 × 2, the geometrical representation will be two lines,
with signicantly dierent slopes, resulting in a well dened solution. Hence, small changes in the coecients
or constants, will not make any serious impact.

1
4 How to Quantify the Condition of a Matrix?
The value of determinant could be small, if the elements of the matrix themselves are small. In other words,
a matrix which is equally ill-conditioned as the matrix above, could have a much higher determinant, only
because its elements are bigger. Therefore, the determinant can not be used as a good measure of the
condition of a matrix.
The condition of a matrix is generally expressed in terms of Condition Number. Among the various
denitions used for expressing the condition number, one commonlyqPusedP measure is:
n n
cond[A] = norm[A] × norm([A]−1 ), where norm[A] ≡ kAk = i=1
2
j=1 Aij
For a well-conditioned matrix, the cond[A] is close to unity (Note: cond[A] dened in this manner is
always greater than one). In case of an ill-conditioned matrix, cond[A] is much larger than one. In the above
system, cond[A] = 3.998 × 103 . (Note: In MATLAB, use the command cond(A) to calculate the condition
number of a matrix A).
As a thumb-rule, it can be said that a computer can loose log(cond[A]) decimal places to the round o
errors, while doing Gauss Elimination.
However, a major deciency in computing the condition number in this manner is the need to nd [A]−1 ,
which is one of the computationally costliest tasks in matrix algebra. Further, [A]−1 is normally computed
using a solution routine, which itself will be aected by the condition number of the matrix!!!
Many other measures have been proposed for evaluating the condition of a matrix, however, all of them
are computationally expensive.

5 Living with Ill-Conditioned Matrices

• The best way to handle an ill-conditioned system is to prevent the occurrence of it!!! This could
be possible in cases, where ill-conditioning is created by the computational methods, like the penalty
parameter method s used in various numerical procedures.
• Since, the errors associated with computations become signicantly magnied due to ill-conditioning,
the damages could be reduced by using higher precision arithmetic (thus reducing the round-o errors).

• Complete pivoting during Gauss elimination can help to improve the solution accuracy.

• There are various algorithms, which perform better in case of ill-conditioned systems, as compared to
the traditional solution routines.

• In extreme cases of ill-conditioning, researchers have even tried Genetic Algorithm s to solve matrices,
with encouraging results.