Professional Documents
Culture Documents
Deflating 1 PDF
Deflating 1 PDF
Stephen Roberts
Michaelmas Term
Motivation
Many problems can be cast as eigenvalue - eigenvector problems. e.g.
1
Notes before we proceed.
2
Power Method for eigenvalues and eigenvectors
Assume that for a matrix A there is a unique (ie only one) largest eigenvector 1,
say, where 1 = max j , j = 1,K N .
j
3
After n iterations x n = Ax n 1 = a11n u1 + a 2 n2 u 2 + L + a N nN u N
Since we have defined 1 as the largest eigenvalue, eventually the term a11n u1 will
dominate, provided a1 0 , and 1 > 1. In this case the vector x n = a11n u1 will be
parallel to the eigenvector u1 corresponding to the largest eigenvalue.
To ensure that the limit remains finite and nonzero we modify the above slightly to
take the ratio
x n +1 Ax n a11n +1u1 a n +1 u1
= = 1 1n = 1 as n
x n
x n n
a11 u1 a11 u1
Example
7 4 1
Lets try this on the matrix 4 4 4 which has eigenvalues 12, 6 and 0 and
1 4 7
1 1 1
1 1 1
normalised eigenvectors 1 , 0 and 2 .
3 2 6
1 1 1
4
Write a small MATLAB function:
function [x,lambda]=powermat1(A,x0,nit)
% calculates the largest eigenvalue and corresponding eigenvector of
% matrix A by the power method using x0 as the starting vector and
% carrying out nit interactions.
%
x = x0;
for n = 1:nit
xnew = A*x;
lambda = norm(xnew,inf)/norm(x,inf);
fprintf('n = %4d lambda = %g x = %g %g %g \n', n, lambda, x');
x=xnew;
end
x = x/norm(x); %normalise x
fprintf('n = %4d normalised x = %g %g %g\n', n, x');
%end
powermat1(A,[1 2 3]',10)
n = 1 lambda = 10 x = 1 2 3
n = 2 lambda = 10.8 x = 18 24 30
n = 3 lambda = 11.3333 x = 252 288 324
n = 4 lambda = 11.6471 x = 3240 3456 3672
n = 5 lambda = 11.8182 x = 40176 41472 42768
n = 6 lambda = 11.9077 x = 489888 497664 505440
n = 7 lambda = 11.9535 x = 5.92531e+006 5.97197e+006 6.01862e+006
n = 8 lambda = 11.9767 x = 7.13837e+007 7.16636e+007 7.19436e+007
n = 9 lambda = 11.9883 x = 8.58284e+008 8.59963e+008 8.61643e+008
n = 10 lambda = 11.9941 x = 1.03095e+010 1.03196e+010 1.03296e+010
n = 10 normalised x = 0.577068 0.57735 0.57763
You can see that x slowly turns towards the eigenvector and gets very close to
1 = 12 after 10 iterations.
5
Now try with starting vector x0 = [0 1 -1]T :
powermat1(A,[0 1 -1]',10)
n = 1 lambda = 3 x = 0 1 -1
n = 2 lambda = 6 x = 3 0 -3
n = 3 lambda = 6 x = 18 0 -18
n = 4 lambda = 6 x = 108 0 -108
n = 5 lambda = 6 x = 648 0 -648
n = 6 lambda = 6 x = 3888 0 -3888
n = 7 lambda = 6 x = 23328 0 -23328
n = 8 lambda = 6 x = 139968 0 -139968
n = 9 lambda = 6 x = 839808 0 -839808
n = 10 lambda = 6 x = 5.03885e+006 0 -5.03885e+006
n = 10 normalised x = 0.707107 0 -0.707107
Note that it does not work in this case. We have found the second largest
eigenvalue instead. The problem is that x 0 is perpendicular to u1 and so a1 = 0
0
in x = a1u1 + a2u 2 + L + a N u N . So we must always have a component
0
of x parallel to u1. A solution is to always try a couple of non parallel starting
vectors.
Convergence Properties
6
1
Power method error
10
0
10
-1
10
-2
10
Error
-3
10
-4
10
-5
10
-6
10
0 2 4 6 8 10 12 14 16 18 20
Iteration number
Summary:
1. The Power method can be used to find the dominant eigenvalue of a symmetric
matrix.
3. The method requires an initial guess and it is not obvious how this can be chosen
in practice.
7
Rayleigh Quotient method
(x n )T Ax n
( )
r xn =
(x n )T x n
rayleigh = (x'*xnew)/(x'*x);
Running this with rayleigh1 gives a far more rapid rate of convergence,
and this shows in the error plot from rayleigh2
8
rayleigh1(A,[1 2 3]',10)
n = 1 lambda = 10 rayleigh = 11.1429
n = 2 lambda = 10.8 rayleigh = 11.76
n = 3 lambda = 11.3333 rayleigh = 11.9381
n = 4 lambda = 11.6471 rayleigh = 11.9844
n = 5 lambda = 11.8182 rayleigh = 11.9961
n = 6 lambda = 11.9077 rayleigh = 11.999
n = 7 lambda = 11.9535 rayleigh = 11.9998
n = 8 lambda = 11.9767 rayleigh = 11.9999
n = 9 lambda = 11.9883 rayleigh = 12
n = 10 lambda = 11.9941 rayleigh = 12
n = 10 normalised x = 0.577068 0.57735 0.577632
Comparison
2
of Rayleigh coefficient & Power method error
10
0
10
-2
10
-4
10
Error
-6
Power Method
10
-8
10
-10
10
9
Convergence Properties
Following the previous error analysis, considering only the two largest
eigenvalues,
( )
r xn =
( x n ) T Ax n
(a u + a u ) (a u
n
1 1 1 2
n
2 2
T n +1
1 1 1 + a 2 n2 +1u 2 )
(x n ) T x n (a u + a u ) (a u
n
1 1 1 2
n
2 2
T n
1 1 1
n
+ a2 u 2
2 )
2 n +1
1 + a 2 2
2
a12 12 n +1 + a 22 22 n +1 2
a1 1
= = 1
a12 12 n + a 22 22 n a2 2
2 2n
1 + 2
a1 1
In this case the proportional error thus decays with successive iterations as
2n
2
.
1
2
Since the reduction per iteration is 2 , this is ; quadratic or second order
1
convergence and is much faster.
Engineering Computation ECL4-19
Deflation techniques
10
Deflation techniques
So far we have considered how to find the largest eigenvalue. Suppose
we have done this, then how do we find the rest?
As a first attempt, note that if we can calculate the inverse matrix A-1, we
can calculate the smallest eigenvalue, because if we pre-multiply the
eigenvalue-eigenvector equation Aui = iui by A-1 and reverse the
equation, we get A 1ui = i 1u i , and so the eigenvalues of the inverse of a
matrix are the inverses of the eigenvalues. Unfortunately, finding A-1 for a
large matrix can pose problems.
A more general way forward, is, after finding the largest eigenvalue 1, to
make it into the smallest by deflation and then go on to find the new
largest one, say 2 .
Hotellings deflation:
Consider (A u u )u
1 1
T
1 j ( )
= Au j 1u 1u 1T u j = j u j 1u 1 u 1T u j .
If j = 1, then (A u u )u
1 1
T
1 1 ( )
= 1u 1 1u 1 u 1T u 1 = 0u 1 .
If j 1, then (A u u )u
1 1
T
1 j = j u j 1u 1 (0 ) = j u j .
( )
thus, A 1u 1u 1T has the same eigenvectors as A, and the same eigenvalues
as A except that the largest one has been replaced by 0. thus we can use the
power method with Rayleighs coefficient to find the next biggest and so on.
11
Example
7 4 1
For our matrix 4 4 4 which has a largest eigenvalue 1 = 12 , corresponding
1 4 7
1 1 1 1
1 1
to the eigenvector u1 1 , u u T
= 1 1 1 .
3 3
1 1
1 1 1 1
7 4 1 1 1 1 3 0 3
Replace A by B = 4 4 4 1 1 1 = 0 0 0 and apply Rayleigh:
12
3
1 4 7 1 1 1 3 0 3
rayleigh1(B,[1 2 3]',4)
n = 1 lambda = 2 raleigh = 0.857143
n = 2 lambda = 6 raleigh = 6
n = 3 lambda = 6 raleigh = 6
n = 4 normalised x = -0.707107 0 0.707107
we get the second eigenvalue immediately. But we might have trouble with the
3rd (why?).
Warning. In practice, Hotellings deflation suffers from rounding errors for large
matrices and has been replaced by superior, but similar, methods such as
Wielandts deflation (look it up in Kreyzig 7th ed., p. 1016) (Burden and
Faires).
12
Try MATLAB function eig to get eigenvalues and eigenvectors:
[eigenvectors,eigenvalues] = eig(A)
eigenvectors =
4.0825e-001 -7.0711e-001 5.7735e-001
-8.1650e-001 1.8041e-016 5.7735e-001
4.0825e-001 7.0711e-001 5.7735e-001
eigenvalues =
-3.8257e-015 0 0
0 6.0000e+000 0
0 0 1.2000e+001
Note the rounding errors The 0 values are ~ 10-15, not exactly 0.
Illustrative Example
13
x1
1 -1x1
x1
2 x1
2
2 2
x1
2
2
Kx = 2 x = x ,
2 1 1 0 1 1 0 0 x1
1 2 0 0 1 1 0 1 y
1
1 0 2 1 0 0 1 1 x2
0 0 1 2 0 1 1 1 and co-ordinate vector x = y 2
where K =
1 1 0 0 2 1 1 0 x3
1 1 0 1 1 2 0 0 y3
0 0 1 1 1 0 2 1 x
4
0 1 1 1 0 0 1 2 y 4
14
Run this through MATLAB:
[eigenvectors,eigenvalues] = eig(K)
eigenvectors =
eigenvalues =
2.0000 0 0 0 0 0 0 0
0 2.0000 0 0 0 0 0 0
0 0 -0.0000 0 0 0 0 0
0 0 0 -0.0000 0 0 0 0
0 0 0 0 2.0000 0 0 0
0 0 0 0 0 0.0000 0 0
0 0 0 0 0 0 4.0000 0
0 0 0 0 0 0 0 6.0000
15
Advanced Methods
This has only been an introductory to some of the methods of finding
eignvalues and eigenvectors. We have focused on methods for finding the
dominant eigenvalue/eigenvector and briefly mentioned deflation methods
which can be used to find the other eigenvalues .
In practice you may come across other methods. In particular,
These methods do not form part of the official course syllabus but you should
be aware of their existence and usefulness in practice.
16