You are on page 1of 16

ENGINEERING COMPUTATION

Lecture 4

Stephen Roberts
Michaelmas Term

Computation of matrix eigenvalues and eigenvectors
Topics covered in this lecture:
1. Iterative Power method for approximating the dominant eigenvalue
2. The Rayleigh Quotient method
3. Deflation techniques
4. Awareness of other methods for approximating eigenvalues

Engineering Computation

ECL4-1

Motivation
Many problems can be cast as eigenvalue - eigenvector problems. e.g.
• Vibrating mechanical systems and resonances. You know that the
eigenvalues give ω2 - (resonant frequencies) and the eigenvectors
give the natural modes of vibration.
• Electrical coupled - resonant circuits.
• Stress and strain calculations.
• Geometry transforms using matrices.
• Stability of non-linear systems (eg control engineering)
How can we use computers to find eigenvalues and eigenvectors
efficiently?

Engineering Computation

ECL4-2

1

i. They are optimised and will avoid “nasties”. .. Recall that.g. 1. λ2. i.Notes before we proceed……. Engineering Computation ECL4-4 2 . Rewriting Ax = λ x as determinant equation (A − λ I )x = 0 . function eig in MATLAB). 3. if none of the λi are repeated.. The numerical methods discussed in this lecture come into their own when dealing with large matrices. even though the eigenvalues and eigenvectors could be found directly! 2. If A is symmetric then the eigenvalues are real. Any vector x can be expressed in terms of that orthonormal set: x = a1u1 + a2u 2 + L + a N u N . We will use them on small matrices (~ 3×3) for demonstration purposes.. normalised ( u 2 = 1 ) eigenvectors form an orthonormal set. We’ll keep to symmetric matrices . λ3.e.e. but u iT u j = 0 for all i ≠ j . In practice use the routines found in packages (e. u iT u i = 1 . (A − λ I ) = 0 .λN . Au i = λi u i for each i.general nonsymmetric matrices are much harder! Engineering Computation ECL4-3 Revision on eigenvalues and eigenvectors The eigenvalues or characteristic roots of an N×N matrix A are the N real or complex number λi such that the equation Ax = λ x has non-trivial solutions λ1. the λ’s are the roots of the polynomial The eigenvectors or characteristic vectors of A are the set of N-vectors x = u i (some books use qi ) which are the non-trivial solutions of Ax = λi x .

Then Do it again expressed in terms x1 = Ax 0 = a1Au1 + a2 Au 2 + L + aN Au N = a1λ1u1 + a2λ2u 2 + L + a N λN u N x 2 = Ax1 = a1λ12u1 + a2 λ22u 2 + L + aN λ2N u N After n iterations x n = Ax n−1 = a1λ1nu1 + a2λn2u 2 + L + aN λnN u N Engineering Computation ECL4-6 3 . say. Let’s start with an initial guess x 0 = a1u1 + a2u 2 + L + aN u N of the eigenvectors of A. j = 1.Power Method for eigenvalues and eigenvectors Engineering Computation ECL4-5 Power Method for eigenvalues and eigenvectors Assume that for a matrix A there is a unique (ie only one) largest eigenvector λ1. j Then we can find λ1 by the Power method as described below: Consider the n+1th iteration x n+1 = Ax n .K N . where λ1 = max λ j .

normalised eigenvectors 3  2   6  1 − 1 1  Engineering Computation ECL4-8 4 . 0 and 2 . In this case the vector x n = a1λ1n u1 will be parallel to the eigenvector u1 corresponding to the largest eigenvalue. provided a1 ≠ 0 . eventually the term a1λ1n u1 will dominate. and negative if they alternate.After n iterations x n = Ax n −1 = a1λ1n u1 + a 2 λn2 u 2 + L + a N λnN u N Since we have defined λ1 as the largest eigenvalue. 6 and 0 and 1 4 7  1 1 1  1  1   1   1 . and λ1 > 1. x n +1 will be parallel to the largest eigenvector so we have found both and u1 Engineering Computation λ1 ECL4-7 Example 7 4 1  Let’s try this on the matrix 4 4 4 which has eigenvalues 12. To ensure that the limit remains finite and nonzero we modify the above slightly to take the ratio x n +1 Ax n a1λ1n +1u1 a λn +1 u1 = → = 1 1n = λ1 as n → ∞ n n n a1λ1 u1 x x a1λ1 u1 The sign of λ1 is positive if the sign of the corresponding terms in x n +1 are the same as in x n . As before.

92531e+006 5.nit) % calculates the largest eigenvalue and corresponding eigenvector of % matrix A by the power method using x0 as the starting vector and % carrying out nit interactions.10) n = 1 lambda = 10 x = 1 2 3 n = 2 lambda = 10.Write a small MATLAB function: function [x.9535 x = 5. x').9077 x = 489888 497664 505440 n = 7 lambda = 11. fprintf('n = %4d lambda = %g x = %g %g %g \n'. x').01862e+006 n = 8 lambda = 11.13837e+007 7.lambda]=powermat1(A.8 x = 18 24 30 n = 3 lambda = 11.9941 x = 1. x=xnew.16636e+007 7. lambda. % x = x0.8182 x = 40176 41472 42768 n = 6 lambda = 11.9767 x = 7.59963e+008 8. %normalise x fprintf('n = %4d normalised x = %g %g %g\n'.[1 2 3]'.inf).57763 You can see that x slowly “turns” towards the eigenvector and gets very close to λ1 = 12 after 10 iterations.9883 x = 8. n.x0.inf)/norm(x. for n = 1:nit xnew = A*x. lambda = norm(xnew.03095e+010 1.19436e+007 n = 9 lambda = 11.97197e+006 6. Engineering Computation ECL4-10 5 .3333 x = 252 288 324 n = 4 lambda = 11.03196e+010 1.03296e+010 n = 10 normalised x = 0. end x = x/norm(x). n.58284e+008 8.61643e+008 n = 10 lambda = 11. %end Engineering Computation ECL4-9 Run it with initial guess x0 = [1 2 3]T : powermat1(A.6471 x = 3240 3456 3672 n = 5 lambda = 11.57735 0.577068 0.

03885e+006 = 10 normalised x = 0. linear convergence.03885e+006 0 -5. So we must always have a component 0 of x parallel to u1.707107 0 -0. Engineering Computation ECL4-11 Convergence Properties If we consider only the two largest eigenvalues. this is .m Engineering Computation ECL4-12 6 . We have found the second largest eigenvalue instead. A solution is to always try a couple of non parallel starting vectors. λ   1   λ2    λ   1  n . The problem is that x 0 is perpendicular to u1 and so a1 = 0 0 in x = a1u1 + a2u 2 + L + a N u N .10) = 1 lambda = 3 x = 0 1 -1 = 2 lambda = 6 x = 3 0 -3 = 3 lambda = 6 x = 18 0 -18 = 4 lambda = 6 x = 108 0 -108 = 5 lambda = 6 x = 648 0 -648 = 6 lambda = 6 x = 3888 0 -3888 = 7 lambda = 6 x = 23328 0 -23328 = 8 lambda = 6 x = 139968 0 -139968 = 9 lambda = 6 x = 839808 0 -839808 = 10 lambda = 6 x = 5.[0 1 -1]'.707107 Note that it does not work in this case.Now try with starting vector x0 = [0 1 -1]T : » n n n n n n n n n n n powermat1(A. then x n +1 xn ≈ a1λ1n +1u1 + a2λn2 +1u 2 a1λ1nu1 + a2λn2u 2 = λ1 λ  a1u1 + a2  2   λ1  n +1 u2 n λ  a1u1 + a2  2  u 2  λ1  The proportional error thus decays with successive iterations as Since the reduction per iteration is  λ2   . Confirm this by plotting the error in our example: using powermat2.

Engineering Computation ECL4-14 7 . 3. The method has linear convergence. The method does work if the dominant eigenvalue has multiplicity r. The estimated eigenvector will then be a linear combination of the r eigenvectors. 4.Power method error 1 10 0 10 -1 10 -2 Error 10 -3 10 -4 10 -5 10 -6 10 0 2 4 6 8 10 12 14 16 18 20 Iteration number Engineering Computation ECL4-13 Summary: 1. The method requires an initial guess and it is not obvious how this can be chosen in practice. The Power method can be used to find the dominant eigenvalue of a symmetric matrix. 2.

Running this with rayleigh1 gives a far more rapid rate of convergence. and this shows in the error plot from rayleigh2 Engineering Computation ECL4-16 8 . Modify the power method by calculating the Rayleigh Quotient at each iteration: ( ) r xn = (x n )T Ax n (x n )T x n This can be done with an extra line of code: rayleigh = (x'*xnew)/(x'*x).Rayleigh Quotient method Engineering Computation ECL4-15 The Rayleigh quotient method .

1429 = 2 lambda = 10.577068 0.[1 2 3]'.577632 Engineering Computation ECL4-17 Comparison of Rayleigh coefficient & Power method error 2 10 0 10 -2 10 -4 Error 10 Power Method -6 10 -8 10 -10 10 Rayleigh Method -12 10 0 2 4 6 8 10 12 14 16 18 20 Iteration number Engineering Computation ECL4-18 9 .9998 = 8 lambda = 11.9844 = 5 lambda = 11.9941 rayleigh = 12 = 10 normalised x = 0.9535 rayleigh = 11.9999 = 9 lambda = 11.9381 = 4 lambda = 11.9767 rayleigh = 11.9961 = 6 lambda = 11.76 = 3 lambda = 11.6471 rayleigh = 11.3333 rayleigh = 11.9883 rayleigh = 12 = 10 lambda = 11.8 rayleigh = 11.10) = 1 lambda = 10 rayleigh = 11.» n n n n n n n n n n n rayleigh1(A.999 = 7 lambda = 11.9077 rayleigh = 11.57735 0.8182 rayleigh = 11.

( ) r xn = = ( x n ) T Ax n (x n ) T x n a12 λ12 n +1 + a 22 λ22 n +1 a12 λ12 n + a 22 λ22 n ≈ (a λ u + a λ u ) (a λ u (a λ u + a λ u ) (a λ u n 1 1 n 1 1 1 1 T n 2 2 2 n +1 1 1 2 n 2 T 2 n 1 1 1 1 + a 2 λn2 +1u 2 n 2 + a2 λ u 2 ) ) 2 n +1 2     1 + a 2  λ2  2     a1  λ1  = λ1   2n 2  a2  λ2    1 + 2    a1  λ1    In this case the proportional error thus decays with successive iterations as 2n  λ2    λ  . Engineering Computation ECL4-19 Deflation techniques Engineering Computation ECL4-20 10 . quadratic or second order  λ1  convergence and is much faster. considering only the two largest eigenvalues.  1  2 λ  Since the reduction per iteration is  2  .Convergence Properties Following the previous error analysis. this is .

and so the eigenvalues of the inverse of a matrix are the inverses of the eigenvalues. then If j ≠ 1. then how do we find the rest? As a first attempt. = λ j u j − λ1u 1 (0 ) = λ j u j . Engineering Computation ECL4-21 Hotelling’s deflation: (A − λ u u )u Consider 1 If j = 1. A more general way forward. finding A-1 for a large matrix can pose problems. we get A −1ui = λ−i 1u i . then ( T 1 1 j (A − λ u u )u (A − λ u u )u 1 1 1 1 T 1 T 1 ( ) = Au j − λ1u 1u 1T u j = λ j u j − λ1u 1 u 1T u j . Engineering Computation ECL4-22 11 . is. say λ2 . because if we pre-multiply the eigenvalue-eigenvector equation Aui = λiui by A-1 and reverse the equation. we can calculate the smallest eigenvalue. 1 j ( ) = λ1u 1 − λ1u 1 u 1T u 1 = 0u 1 . note that if we can calculate the inverse matrix A-1.Deflation techniques So far we have considered how to find the largest eigenvalue. thus we can use the power method with Rayleigh’s coefficient to find the next biggest and so on. Suppose we have done this. and the same eigenvalues as A except that the largest one has been replaced by 0. A − λ1u 1u 1T has the same eigenvectors as A. after finding the largest eigenvalue λ1. to make it into the smallest by deflation and then go on to find the new largest one. Unfortunately. ) thus.

707107 0 0. methods such as Wielandt’s deflation (look it up in Kreyzig 7th ed.Example 7 4 1  For our matrix 4 4 4 which has a largest eigenvalue λ1 = 12 . Engineering Computation ECL4-24 12 . Hotelling’s deflation suffers from rounding errors for large matrices and has been replaced by superior. In practice. 1016) (Burden and Faires).4) = 1 lambda = 2 raleigh = 0. Warning.707107 we get the second eigenvalue immediately. corresponding 1 4 7  1 1 1 1 1  1 T to the eigenvector u1 1 = 1 1 1 .857143 = 2 lambda = 6 raleigh = 6 = 3 lambda = 6 raleigh = 6 = 4 normalised x = -0.[1 2 3]'. But we might have trouble with the 3rd (why?). but similar. . p.. Packages such as MATLAB use more sophisticated and robust methods. u u 1 1 3 3  1 1 1 1 7 4 1  1 1 1  3 0 − 3 12 Replace A by B = 4 4 4 − 1 1 1 =  0 0 0  and apply Rayleigh: 3 1 4 7 1 1 1 − 3 0 3  Engineering Computation » n n n n ECL4-23 rayleigh1(B.

8257e-015 0 0 0 6.0711e-001 -8.eigenvalues] = eig(A) eigenvectors = 4. Engineering Computation ECL4-25 Illustrative Example Four unit masses are arranged in a square (see diagram) and are interconnected by springs.2000e+001 Note the rounding errors The 0 values are ~ 10-15.8041e-016 4.0825e-001 -7.7735e-001 5. not exactly 0. and the diagonal springs have two times unit stiffness.Try MATLAB function eig to get eigenvalues and eigenvectors: » [eigenvectors.7735e-001 5.0711e-001 5.7735e-001 eigenvalues = -3. Engineering Computation ECL4-26 13 .0825e-001 7. the horizontal and vertical springs have unit stiffness.0000e+000 0 0 0 1.1650e-001 1.

and a force 1x1 on mass 2. 0 0  x1   2 −1 −1 0 −1 1 − 1 2 0 0 1 − 1 0 − 1 y   1    x2  − 1 0 2 1 0 0 − 1 − 1     0 0 1 2 0 − 1 − 1 − 1  and co-ordinate vector x =  y 2  where K =   x3  − 1 1 0 0 2 −1 −1 0      1 − 1 0 − 1 − 1 2 0 0  y3    x   0 0 −1 −1 −1 0 2 1  4   2   0 − 1 − 1 − 1 0 0 1  y 4  Engineering Computation ECL4-28 14 . 2 resolving horizontally gives a force − 1x1 on mass 1 in the x1 direction.x1 -1x1 1 x1 2 2 x1 − 2 −2 2 x1 2 Consider the effect of moving mass 1 a distance x1 to the right. Consideration of the horizontal forces on mass 1 due to movements of all the masses gives (assuming sinusoidal motion): m&x&1 = −ω 2 x1 = −2 x1 + 1y1 + 1x 2 + 0 y 2 + 1x3 − 1y 3 + 0 x 4 + 0 y 4 Engineering Computation ECL4-27 Writing out the similar equations of motion of all four masses. The spring 12 will exert a force − kx1 = −1x1 on mass 1. x The movement of spring 13 is only 1 and. and reversing the signs gives a matrix eigenvalue/eigenvector equation Kx = ω 2 x = λx .

3410 -0.2493 0.0000 0 0 0 0 0 0 0 0 2.2553 -0.2493 0.0683 -0.2271 -0.3342 -0.5649 -0.3536 -0.3410 0.5836 0.1111 -0.3626 -0.3536 0.0000 0 0 0 0 0 0 0 0 0.3536 -0.1111 -0.0000 0 0 0 0 0 0 Engineering Computation ECL4-29 Why are the eigenvalues 2.2776 0.3536 -0.3536 0.3536 -0.1487 0.4080 -0.0000 eigenvalues = 2.3536 0.3536 -0.3722 -0.2341 -0.Run this through MATLAB: [eigenvectors.3536 0 0 -0.eigenvalues] = eig(K) eigenvectors = -0.3654 -0.4300 -0.1487 0.4516 -0.2553 0.0457 0.3536 -0.4451 0.2271 0.3536 0.3536 0.1806 -0.0000 0 0 0 0 0 0 0 0 -0.0000 0 0 0 0 0 0 0 0 2.5836 -0.4234 -0.6068 0.0000 eigenvalues represent? What is the maximum frequency? Can you identify some of the modes of vibration? Engineering Computation ECL4-30 15 .4516 0.1527 -0.2776 0.6068 0.0000 0 0 0 0 0 0 0 0 6.3654 0.0457 0.3536 0.0683 -0.0000 repeated? what do 0.0000 0 0 0 0 0 0 0 0 4.3536 0.3536 0.3536 -0.2341 0.5649 -0.3722 0.4080 -0.4451 0.3626 0.4234 0.1527 -0.1806 -0.4300 -0.3342 -0.

In practice you may come across other methods. 1. These methods do not form part of the official course syllabus but you should be aware of their existence and usefulness in practice. 2. In this case using the Power or Rayleigh method is more efficient. such as for the analysis of very large systems (eg complex finite element models) which may have millions of variables and you may only be interested in finding a subset of the eigenvalues (100’s). The Inverse Power Method – a variant on the power method that is faster than the latter and finds the closest eigenvalue to the initial guess. We have focused on methods for finding the dominant eigenvalue/eigenvector and briefly mentioned deflation methods which can be used to find the other eigenvalues .Advanced Methods This has only been an introductory to some of the methods of finding eignvalues and eigenvectors. So why don’t we use matrix reduction methods all the time? Matrix methods are not efficient in some cases. Engineering Computation ECL4-31 16 . Matrix reduction methods such as the QR algorithm (Faires and Burden p 573) for finding all the eigenvalues which in practice are better than deflation-based methods which suffer from round-off errors. In particular.