Condition number

1 Matrices

In the eld of numerical analysis, the condition number

of a function with respect to an argument measures how
much the output value of the function can change for a
small change in the input argument. This is used to measure how sensitive a function is to changes or errors in the
input, and how much error in the output results from an
error in the input. Very frequently, one is solving the inverse problem given f (x) = y, one is solving for x, and
thus the condition number of the (local) inverse must be
used. In linear regression the condition number can be
used as a diagnostic for multicollinearity.

For example, the condition number associated with the

linear equation Ax = b gives a bound on how inaccurate the solution x will be after approximation. Note
that this is before the eects of round-o error are taken
into account; conditioning is a property of the matrix, not
the algorithm or oating point accuracy of the computer
used to solve the corresponding system. In particular,
one should think of the condition number as being (very
roughly) the rate at which the solution, x, will change with
respect to a change in b. Thus, if the condition number is
large, even a small error in b may cause a large error in x.
On the other hand, if the condition number is small then
the error in x will not be much bigger than the error in b.

The condition number is an application of the derivative,

and is formally dened as the value of the asymptotic
worst-case relative change in output for a relative change
in input. The function is the solution of a problem and
the arguments are the data in the problem. The condition number is frequently applied to questions in linear
algebra, in which case the derivative is straightforward but
the error could be in many dierent directions, and is thus
computed from the geometry of the matrix. More generally, condition numbers can be dened for non-linear
functions in several variables.

The condition number is dened more precisely to be the

maximum ratio of the relative error in x divided by the
relative error in b.
Let e be the error in b. Assuming that A is a nonsingular
matrix, the error in the solution A1 b is A1 e. The ratio
of the relative error in the solution to the relative error in
b is

A problem with a low condition number is said to be wellconditioned, while a problem with a high condition num

ber is said to be ill-conditioned. The condition number
A1 e / A1 b
.
is a property of the problem. Paired with the problem
e / b
are any number of algorithms that can be used to solve
the problem, that is, to calculate the solution. Some al- This is easily transformed to
gorithms have a property called backward stability. In
general, a backward stable algorithm can be expected to


)
) (
accurately solve well-conditioned problems. Numerical (
A1 e / e b / A1 b .
analysis textbooks give formulas for the condition numbers of problems and identify the backward stable algoThe maximum value (for nonzero b and e) is easily seen
rithms.
to be the product of the two operator norms:
As a rule of thumb, if the condition number (A) = 10k
, then you may lose up to k digits of accuracy on top of


what would be lost to the numerical method due to loss of (A) = A1 A .
[1]
precision from arithmetic methods. However, the condition number does not give the exact value of the maxi- The same denition is used for any consistent norm, i.e.
mum inaccuracy that may occur in the algorithm. It gen- one that satises
erally just bounds it with an estimate (whose computed
value depends on the choice of the norm to measure the
inaccuracy).
(A) 1.

When the condition number is exactly one (which can

only happen if A is a linear isometry), then a solution algorithm can nd (in principle, meaning if the algorithm
introduces no errors of its own) an approximation of the
solution whose precision is no worse than that of the data.
1

2 NON-LINEAR

However, it does not mean that the algorithm will converge rapidly to this solution, just that it won't diverge arbitrarily because of inaccuracy on the source data (backward error), provided that the forward error introduced
by the algorithm does not diverge as well because of accumulating intermediate rounding errors.

If the condition number is not too much larger than one

(but it can still be a multiple of one), the matrix is well
conditioned which means its inverse can be computed
with good accuracy. If the condition number is very large,
then the matrix is said to be ill-conditioned. Practically,
such a matrix is almost singular, and the computation of
The condition number may also be innite, but this im- its inverse, or solution of a linear system of equations is
plies that the problem is ill-posed (does not possess a prone to large numerical errors. A matrix that is not invertible has condition number equal to innity.
unique, well-dened solution for each choice of data -that is, the matrix is not invertible), and no algorithm can
be expected to reliably nd a solution.

2 Non-linear

Of course, the denition of the condition number depends on the choice of norm, as can be illustrated by two
Condition numbers can also be dened for nonlinear
examples.
functions, and can be computed using calculus. The conIf is the norm (usually noted as 2 ) dened in the dition number varies with the point; in some cases one
square-summable sequence space 2 (which matches the can use the maximum (or supremum) condition number
usual distance in a standard Euclidean space), then
over the domain of the function or domain of the question
as an overall condition number, while in other cases the
condition number at a particular point is of more interest.
max (A)
(A) =
,
min (A)
where max (A) and min (A) are maximal and minimal
singular values of A respectively. Hence
If A is normal then


 max (A) 

,
(A) = 
min (A) 
where max (A) and min (A) are maximal and minimal
(by moduli) eigenvalues of A respectively.
If A is unitary then
(A) = 1.
The condition number with respect to L2 arises so often
in numerical linear algebra that it is given a name, the
condition number of a matrix.
If is the norm (usually denoted by ) dened in
the sequence space of all bounded sequences (which
matches the maximum of distances measured on projections into the base subspaces), and A is lower triangular
non-singular (i.e., i, aii = 0 ) then
maxi (|aii |)
.
mini (|aii |)

2.1 One variable

See also:
functions

Signicance arithmetic Transcendental

The condition number of a dierentiable function f in one

variable as a function is xf /f. Evaluated at a point x this
is:
xf (x)
.
f (x)
Most elegantly, this can be understood as (the absolute
value of) the ratio of the logarithmic derivative of f,
which is (log f ) = f /f and the logarithmic derivative
of x, which is (log x) = x /x = 1/x, yielding a ratio
of xf /f. This is because the logarithmic derivative is
the innitesimal rate of relative change in a function: it
is the derivative f scaled by the value of f. Note that if
a function has a zero at a point, its condition number at
the point is innite, as innitesimal changes in the input
can change the output from zero to positive or negative,
yielding a ratio with zero in the denominator, hence innite relative change.

More directly, given a small change x in x, the relative

change in x is [(x + x) x]/x = (x)/x, while the
relative change in f (x) is [f (x + x) f (x)]/f (x).
The condition number computed with this norm is gen- Taking the ratio yields:
erally larger than the condition number computed with
square-summable sequences, but it can be evaluated more
easily (and this is often the only practicably computable [f (x + x) f (x)]/f (x)
x f (x + x) f (x)
=
.
condition number, when the problem to solve involves a
(x)/x
f (x) (x + x) x
non-linear algebra, for example when approximating irrational and transcendental functions or numbers with nu- The last term is the dierence quotient (the slope of the
secant line), and taking the limit yields the derivative.
merical methods.)
(A)

3
Condition numbers of common elementary functions are
particularly important in computing signicant gures,
and can be computed immediately from the derivative;
see signicance arithmetic of transcendental functions. A
few important ones are given below:
Exponential function e : x
x

Natural logarithm function ln(x) :

1
ln(x)

Sine function sin(x) : x cot(x)

Cosine function cos(x) : x tan(x)
Tangent function tan(x) : x(tan(x) + cot(x))
Inverse sine function arcsin(x) :

1x2 arcsin(x)

Inverse cosine function arccos(x) :

Inverse tangent function arctan(x) :

2.2

x
1x2 arccos(x)

x
(1+x2 ) arctan(x)

Several variables

Condition numbers can be dened for any function

mapping its data from some domain (e.g. an m-tuple of
real numbers x) into some codomain [e.g. an n-tuple of
real numbers (x)], where both the domain and codomain
are Banach spaces. They express how sensitive that function is to small changes (or small errors) in its arguments.
This is crucial in assessing the sensitivity and potential
accuracy diculties of numerous computational problems, for example polynomial root nding or computing
eigenvalues.
The condition number of at a point x (specically, its
relative condition number[2] ) is then dened to be the
maximum ratio of the fractional change in (x) to any
fractional change in x, in the limit where the change x in
x becomes innitesimally small:[2]
[
lim

sup

0+ x

]
f (x + x) f (x) x
/
,
f (x)
x

where is a norm on the domain/codomain of (x).

If is dierentiable, this is equivalent to:[2]
J(x)
,
f (x)/x
where J(x) denotes the Jacobian matrix of partial derivatives of at x and J(x) is the induced norm on the
matrix.

See also
Singular value
Ill-posed

4 References
[1] Cheney; Kincaid (2007-08-03). Numerical Mathematics
and Computing. ISBN 978-0-495-11475-8.
[2] Trefethen, L. N.; Bau, D. (1997). Numerical Linear Algebra. SIAM. ISBN 978-0-89871-361-9.

5 External links
Condition Number of a Matrix at Holistic Numerical
Methods Institute
Matrix condition number at PlanetMath.org.
MATLAB library function to determine condition
number
Condition number Encyclopedia of Mathematics

6 TEXT AND IMAGE SOURCES, CONTRIBUTORS, AND LICENSES

Text and image sources, contributors, and licenses

6.1

Text

Condition number Source: https://en.wikipedia.org/wiki/Condition_number?oldid=688887699 Contributors: Ap, Michael Hardy, Delirium, Stevan White, Stevenj, Charles Matthews, Jitse Niesen, Jaredwf, Giftlite, BenFrantzDale, CyborgTosser, Eliosh, Crynyd, Guanabot,
Wonks, Oleg Alexandrov, Simetrical, LOL, Roboto de Ajvol, Tong~enwiki, PAStheLoD, Lunch, Feuer, JJL, Jfgrcar, Kurykh, Nbarth,
Berland, Derek farn, Khazar, Momet, Verdy p, Thijs!bot, Ksmcdani, TomyDuby, Perdelsky, KoenDelaere, Anchor Link Bot, He7d3r,
Antahin, Fgnievinski, SpBot, Legobot, Luckas-bot, Yobot, AnomieBOT, Xqbot, Aykayel, Bunyk, I dream of horses, Kreykesje, EmausBot,
AManWithNoPlan, Mikesmith00, Mariraja2007, Mygskr, Oleskiw, PeterLFlomPhD, Bluemix, Cchohra and Anonymous: 47

6.2

Images

6.3

Content license

Creative Commons Attribution-Share Alike 3.0