EE103 (Fall 2009-10)

7. The condition number

Ax = b when A is singular condition of a set of linear equations matrix norm condition number

7-1

Linear equations with singular coecient matrix

if A is nonsingular, then Ax = b has a unique solution for every b if A is singular, then Ax = b has zero or innitely many solutions: for some righthand sides b, the equation Ax = b has no solution for example, we can take a nonzero vector b in the nullspace of AT Ax = b hence Ax = b is unsolvable if Ax = b is solvable, then it has innitely many solutions: A(x + tv) = b for all scalar t, and all nonzero vectors v in the nullspace of A
The condition number 7-2

0 = bT Ax = bT b > 0

Example

A= is singular: Ax = 0 for x = (1, 1)

1 1 2 2

the equation Ax = b has no solution for b = (1, 0) the equation Ax = b has innitely many solutions for b = (2, 4): x1 x2 is a solution, for any t = 2+t t

The condition number

7-3

Condition of a set of linear equations

assume A is nonsingular and Ax = b if we change b to b + b, the new solution is x + x with A(x + x) = b + b the change in x is x = A1 b condition of the equations: a technical term used to describe how sensitive the solution is to changes in the righthand side the equations are well-conditioned if small b results in small x the equations are ill-conditioned if small b can result in large x

The condition number

7-4

Example of ill-conditioned equations

1 A= 2

1 1 1 + 1010 1 1010

1 1010 1010 1 + 1010 1010

solution for b = (1, 1) is x = (1, 1) change in x if we change b to b + b: x = A

1

b =

b1 1010(b1 b2) b1 + 1010(b1 b2)

small b can lead to extremely large x

The condition number

7-5

Norm of a matrix
m n-matrix A denes a linear function f (x) = Ax

y = Ax

Ax / x gives the amplication factor or gain in the direction x we dene the norm of A as the maximum gain (over all directions): Ax A = max = max Ax x=0 x x =1 (also called spectral norm or 2-norm)

The condition number

7-6

Computing the norm of a matrix

Simple examples where it is easy to maximize Ax / x : zero matrix: 0 = 0 identity matrix: I = 1 diagonal matrix: if 1 0 0 0 2 0 A= . . ... . . . . 0 0 n then A = maxi=1,...,n |i| orthogonal matrix: A = 1 for general A, we have to use a numerical algorithm (in Matlab: norm(A))
The condition number 7-7

Other matrix norms

many other denitions of matrix norms exist, e.g., the Frobenius norm
m n

=
i=1 j=1

a2 ij

has a simple explicit expression in Matlab: use norm(A,fro) in this course: A stands for the operator norm (page 7-6) no simple explicit expression (except for special A) readily computed numerically (in Matlab: use norm(A))

The condition number

7-8

Properties of the matrix norm

A = || A A 0 for all A; A = 0 i A = 0 A+B A + B Ax A x if the product Ax exists AB A B if the product AB exists if A is nonsingular: A A1 1 if A is nonsingular: 1/ A1 = minx=0 ( Ax / x ) (proofs follow directly from the denition)
The condition number 7-9

Bound on absolute error

suppose A is nonsingular and x = A1b Upper bound on x x A1
(follows from property 4 on page 7-9)

small A1 means that x is small when b is small large A1 means that x can be large, even when b is small for any A, there exists b such that x = A1 b (no proof)

The condition number

7-10

Bound on relative error

suppose A is nonsingular, Ax = b with b = 0, and x = A1 b Upper bound on x / x : x A A1 x
(follows from x A1

b b
x )

b and b A

(A) = A A1 is called the condition number of A small (A) means x / x is small when b / b is small large (A) means x / x can be large, even when b / b is small for any A, there exist b, b such that x / x = (A) b / b (no proof)
The condition number 7-11

Condition number

(A) = A A1 dened for nonsingular A (A) 1 for all A (last property on page page 7-9) A is a well-conditioned matrix if (A) is small (close to 1): the relative error in x is not much larger than the relative error in b A is badly conditioned or ill-conditioned if (A) is large: the relative error in x can be much larger than the relative error in b

The condition number

7-12