4.6 Inverse Power Iteration with Shifts.

In the previous section we saw how the basic inverse power iteration algortihm could be used to approximate
the eigenvalue that is smallest in absolute value. However, it has the same problem as regular power iteration
in that it may converge slowly. In this section we modify the inverse power iteration algorithm by using (A -
I)-1 instead of A-1 where  is a suitably chosen value or sequence of values that approximates an eigenvalue .
The resulting algorithm often gives a sequence of approximations to  that converges faster than either the
basic power iteration of inverse power iteration algorithm.

4.6.1 The Inverse Power Iteration with Shifts Algorithm

Let {k} be a sequence of numbers and x a vector. Let the sequence xk be defined by x0 = x and

(1) xk = =

be the vector that one gets when one applies (A - k-1I)-1 … (A - 0I)-1 to x and normalizes the result. We don't
actually have to compute (A - k-1I)-1 to compute w = (A - k-1I)-1xk-1. Instead, we can find it by solving the
equations (A - k-1I)w = xk-1.

Let's first look at the case where k =  doesn't change with k. Then

(2) xk =

and the shifted inverse power iteration algorithm is just the inverse power iteration algorithm applied to A - I.
The proposition below shows that if there is a single eigenvalue  which is closest to  then as k   the
vector xk is close to an eigenvector for this eigenvalue and (Axk) . xk  . We assume the following.

(3) Suppose A is an n  n matrix and v1, v2,…, vn are n linearly independent eigenvectors of A of unit
length corresponding to the eigenvalues 1, …, n respectively. Suppose  is a number and 1 is
the eigenvalue that is closest to  and that it is strictly closer to  than the other eigenvalues, i.e.
| 1 -  | < | j -  | for j = 2, …, n. Let V be the matrix whose columns are v1, …, vn. Given a
vector x, let c = be the solution to Vc = x. Let sk =

Proposition 1. Assume (2) and (3) and x is such that c1  0. Then skxk  v1 and (Axk) . xk  1 as k  .

Proof. The key is that if v is an eigenvector of A corresponding to eigenvalue  then v is also an eigenvector
of A - I but now corresponding to eigenvalue  - . To see this note that Av = v. If we subtract v this
becomes (A - I) - v = ( - )v. Because of this connection between the eigenvectors and eigenvalues of A
and A -  the assumption (2) implies v1, v2,…, vn are eigenvectors of A-1 corresponding to the eigenvalues (1 -
), …, (n - ) respectively. Furthermore 1 -  is the eigenvalue that is smallest in magnitude and that it is
strictly smaller in magnitude than the other eigenvalues. It follows from Proposition 2 of section 4.4.2 that
skxk  v1 as k  . This in turn implies (Axk) . xk   as k  . //

The key to having this shifted inverse power iteration converge faster than regular power iteration or inverse
power iteration is to come up with an initial estimate of an eigenvalue to use for . There are various ways to
do this, some of which we will look at in later sections. However, one thing one can do is to use a few

4.6.1 - 1
iterations of regular power iteration or inverse power iteration to get a preliminary estimate (Axk) . xk of the
largest or smallest eigenvalue and then use  = (Axk) . xk.

Example 1. Let A = and x = . (This is the matrix from Example 2 of section 4.4.1. Recall that after thirty
iterations of regular power iteration the estimate, 9.02, of the largest eigenvalue, 9, was still off by about 0.02.)
Do five iterations of regular power iteration and then use shifted power iteration. Find xk and (Axk) . xk as k
grows and check for convergence.

To compute the xk we use mathematical software; see the next section. As one can see in the next section, after
ten iterations of the shifted inverse power iteration the estimate (Axk) . xk is within 0.00001 of the largest
eigenvalue 9.

Even for larger matrices the convergence is quite fast provided  is close to 1. If 2 is the second closest
eigenvalue to , then the estimate from section 4.4.2 implies

|| skxk – v1 ||  4C k

If 1 is significantly closer to  than 2 then will be small and k converges to zero rapidly. The methods
conidered in the next section will speed up the convergence.

Usually we can get even faster convergence if we change the k with k by using the Rayleigh quotient from the
previous xk, i.e. k = (Axk) . xk.

As k gets closer to an eigenvalue (A - kI)-1 becomes almost non-invertible. Because of this we stop changing
the k when two successive k agree within a certain amount, e.g. when they agree to 0.01%. Here is an
example.

Example 1 (continued). Let A = and x = . Do five iterations of regular power iteration and then use shifted
power iteration updating the shifts by k = (Axk) . xk. Find xk and (Axk) . xk as k grows and check for
convergence.

To compute the xk we use mathematical software; see the next section. As one can see in the next section, after
four iterations of this version of the shifted inverse power iteration the estimate (Axk) . xk is within 0.00001 of
the largest eigenvalue 9.

4.6.1 - 2