MATHS 770: Advanced Numerical Analysis

Broyden’s Method
Lecture notes
Exercise
Show a rank one matrix has the form
uv T

where u, v ∈ Rn .

Introduction to Broyden’s Method

Suppose you are solving
f (x) = 0

where f : Rn → Rn (n > 1) using an iterative scheme.

Most schemes for this problem approximate f (x) about xk (the approximation to the solution on
the k-th iteration) by
Mk (x) = fk + Ak (x − xk )

where fk = f (xk ).
Question: what do we use for Ak ?
The obvious answer is J(xk ) (the Jacobian of f (x) evaluated at x = xk ).
If the Jacobian isn’t available, we could at least require M (xk−1 ) = fk−1 i.e. fk + Ak (xk−1 − xk ) =
fk−1 . Re-arranging gives
Ak (xk − xk−1 ) = fk − fk−1

This is called the Secant equation and is analogous to the equation for the Secant method in one
dimension.
Write as
Ak sk−1 = yk−1

This is a system of n equations for the elements of Ak . Since Ak has n2 elements in general, this
system of equations does not uniquely specify Ak .
One way of imposing uniqueness is to choose Ak by minimising as much as possible the change
in M (x) from one iteration to the next, subject to satisfying the Secant equation.
We have

Mk (x) − Mk−1 (x) = fk + Ak (x − xk ) − (fk−1 + Ak−1 (x − xk−1 ))

1
 = fk − fk−1 − Ak (xk − xk−1 ) + (Ak − Ak−1 )(x − xk−1 )
= (Ak − Ak−1 )(x − xk−1 )

For any x ∈ Rn , we have

x − xk−1 = αsk−1 + t

where tT sk−1 = 0. Then (Ak − Ak−1 )(x − xk−1 ) becomes

α(Ak − Ak−1 )sk−1 + (Ak − Ak−1 )t

We have no control over the first term since (Ak − Ak−1 )sk−1 = yk−1 − Ak−1 sk−1 . We can make the
second term zero for all x ∈ Rn by choosing Ak such that (Ak − Ak−1 )t = 0 for all t orthogonal to
sk−1 .
This requires
Ak − Ak−1 = usTk−1

where u ∈ Rn .
Choose u so the Secant equation is satisfied. Get

(yk−1 − Ak−1 sk−1 )sTk−1

Ak = Ak−1 +
sTk−1 sk−1

Exercise
Explain in words what the above update formula means.