9/12/22

CE-603 Eigenvalue Analysis

Numerical Methods (Contd.)
P ro f. R av i S in h a
( rs in h a @ c iv il.iit b .a c .in )

Lecture 13 – Monday, September 12, 2022

CE-603 Numerical Methods | Autumn Semester 2022-23 2

1 2

Eigen-properties of Symmetric Matrices Jacobi Method

Most problems in science and engineering involve symmetric matrices. Determination of Basic Philosophy
eigen-properties of symmetric matrices is therefore of most interest.
Carry out a sequence of orthogonal similarity transformations,
Jacobi Method:
A (k ) = Q T A (k -1) Q k>0
Proposed by Prof. Charles Jacobi in 1846 as an iterative method to such that after each successive transformation, A(k) is “more” diagonal that A(k-1).
determine eigenvalues.
One of the first scientific papers in numerical methods.
Method is very effective for symmetric matrices (& still in use).
Convergence rate is up to quadratic for later iterations.

Jacobi, C. G. J.: Über ein leichtes Verfahren, die in der Theorie der Säkularstörungen vorkommenden
Gleichungen numerisch aufzulösen. Crelle’s Journal 30, 51–94 (1846)

CE-603 Numerical Methods | Autumn Semester 2022-23 3 CE-603 Numerical Methods | Autumn Semester 2022-23 4

3 4

1
 9/12/22

Jacobi Method Jacobi Method

Basic Philosophy Basic Philosophy
Carry out a sequence of orthogonal similarity transformations, Carry out a sequence of orthogonal similarity transformations,
( k -1)
A =Q A
(k ) T
Q k>0 A (k ) = Q T A (k -1) Q k>0
(k) (k-1)
such that after each successive transformation, A is “more” diagonal that A . such that after each successive transformation, A(k) is “more” diagonal that A(k-1).
No accumulation of No accumulation of Preserves
round-off errors round-off errors eigenvalues

Also preserves Also preserves

Frobenius norm of Frobenius norm of
A A

CE-603 Numerical Methods | Autumn Semester 2022-23 5 CE-603 Numerical Methods | Autumn Semester 2022-23 6

5 6

Jacobi Method Jacobi Method

Basic Philosophy Rotations are carried out using orthogonal rotation matrices:
Carry out a sequence of orthogonal similarity transformations, The rotation matrix, J(p,q,q), is a nearly
é1 ! 0 ! 0 ! 0 ù diagonal Identity matrix:
A (k ) = Q T A (k -1) Q k>0 ê" # " " "ú 1. Almost an Identity matrix, where all
ê ú diagonal entries except two are equal to 1.
such that after each successive transformation, A(k) is “more” diagonal that A(k-1). ê0 ! c ! s ! 0 ú Row p 2. Almost a diagonal matrix, where all non-
Eventually, all remaining off-diagonal terms will be small enough to be considered as zero. ê ú
J(p, q ,q ) = ê " " # " "ú diagonal entries except two are equal to 0.
\The Jacobi method systematically reduces the quantity: ê0 ! - s ! c ! 0 ú Row q 3. The non-unity diagonal entries in pth and
ê ú qth row have values equal to c.
ê" " " # "ú
n n
4. The non-zero off-diagonal entry in
off(A) = åå a
i =1 j =1
2
ij
êë0 ! 0 ! 0 ! 1 úû (p, q) position has value equal to s.
j ¹i 5. The non-zero off-diagonal entry in
through successive rotations of matrix A. Col. p Col. q (q, p) position has value equal to -s.
CE-603 Numerical Methods | Autumn Semester 2022-23 7 CE-603 Numerical Methods | Autumn Semester 2022-23 8

7 8

2
 9/12/22

Jacobi Method Jacobi Method

The entries c and s are determined such that the corresponding (p, q) off-diagonal term in Now, if apq = 0 (i.e., the largest remaining off-diagonal term is zero), just set (c, s) = (1, 0)
A(k) becomes zero. Take the sub-matrix consisting of only row & column p & q terms. and there is no need for further calculations (The transformed matrix is already diagonal as
the largest off-diagonal element is zero). Otherwise,
We chose 1 £ p < q £ n starting with the largest off-diagonal element, and compute cosine-
sine pairs (rotation) such that the resulting sub-matrix given below is diagonal. aqq - app
T Let t = and t = s c (Note that t º tanq )
ébpp bpq ù ébpp 0 ù é c s ù éapp apq ù é c s ù 2apq
êb = =
ë qp bqq úû êë 0 bqq úû êë - s c úû êëaqp aqq úû êë - s c úû The above equation reduces to:
Matrix B is the same as matrix A except for modification in entries in rows and columns p t 2 + 2t t - 1 = 0
and q. sgn(t )
or, t = t ± 1 + t 2 \Now, take t =
For the rotation to diagonalise matrix B, the following must be satisfied: t + 1 +t 2
-1
= This selects the smaller of the two roots
bpq = 0 = apq (c 2 - s2 ) + (app - aqq )cs
t ! 1 +t 2 and ensures that q £ (p/4)
CE-603 Numerical Methods | Autumn Semester 2022-23 9 CE-603 Numerical Methods | Autumn Semester 2022-23 10

9 10

Jacobi Method Jacobi Method

Note that the square of Frobenius norm of the selected (p, q) sub-matrix is given by, The coefficients c and s are found from t as,
2
app + aqq
2
+ 2apq
2
= bpp
2
+ bqq
2
+ 2bpq
2
= bpp
2
+ bqq
2 c= 1 ; s = tc
1 + t2
This follows that The successive similarity transformations can be represented by matrix multiplications as,
n n
A ( p) = diag {l} = ( JTp JTp-1 ! JT1 ) A (0) ( J1 ! Jp-1 Jp ) , p = No. of non-zero off-diagonal terms
off(B)2 = B F - å bii2 = A F - å aii2 + ( app )
2 2 2
+ aqq
2
- bpp
2
- bqq
2

i =1 i =1 or, A ( p) = ˆJT A (0) ˆJ

= off(A)2 - 2apq
2
£ off(A)2
where ˆJ = J1 J2 ! Jp
We therefore get that, after rotation, B is “more” diagonal than A. Clearly, Ĵ is the matrix of eigenvectors of A(0).
Jacobi method directly gives all the eigenvalues and eigenvectors. However, no order is
maintained in the determination of eigenvalues.

CE-603 Numerical Methods | Autumn Semester 2022-23 11 CE-603 Numerical Methods | Autumn Semester 2022-23 12

11 12

3
 9/12/22

Jacobi Method Example - Jacobi Method

1. Jacobi method can be easily parallelised. Example:
2. When Frobenius norm of remaining off-diagonal elements becomes smaller than a Determine the eigenvalues of the following matrix:
specified threshold, the algorithm can be terminated as there may not be much further é0.6532 0.2165 0.0031ù
improvement in accuracy of eigenvalues.
A (0) = ê 0.4105 0.0052ú
3. Jacobi method is one of the most accurate method to determine eigenvalues of real ê ú
symmetric matrices due to control of accumulation of floating point errors. ëê sym. 0.2132 ûú

4. Jacobi method (and its variants) is even now among the best method to determine Clearly, (1,2) element is the largest off-diagonal term. Rotate in (1,2) plane. The calculations
eigenvalues of a dense symmetric matrix. are (to 4 significant digits):
5. The method is also applicable for sparse matrix as only the non-zero off-diagonal aqq - app 0.4105 - 0.6532 sgn(t )
t= = = -0.5605 , t = = - 0.5858
coefficients would be reduced to zero. 2apq 2 ´ 0.2165 t + 1 +t 2

CE-603 Numerical Methods | Autumn Semester 2022-23 13 CE-603 Numerical Methods | Autumn Semester 2022-23 14

13 14

Example - Jacobi Method Example - Jacobi Method

From the calculation of t, we get,
é0.7800 0 0.0053 ù
c= 1 = 0.8628, s = tc = -0.5055 Note: Location of largest off-diagonal
1+t 2
A (1) = ê 0 0.2836 0.0029 ú
ê ú element has now changed from (1,2) to (1,3).
The Jacobi matrix will be, êë 0.0053 0.0029 0.2132 úû
Now, (1,3) element is the largest off-diagonal element. Rotate in (1,3) plane,
é c s 0 ù é0.8628 -0.5055 0 ù
J1 = ê - s c 0 ú = ê 0.5055 0.8628 0 ú t=
aqq - app
=
0.2132 - 0.7800
= -53.47 , t =
sgn(t )
= - 0.0094
ê ú ê ú
êë 0 0 1 úû êë 0 0 1 úû 2apq 2 ´ 0.0053 t + 1 +t 2
The rotated matrix will be, From the calculations of t, we get,

A (1) = JT1 A (0) J1 c= 1 = 1.000, s = tc = -0.0094

1 + t2

CE-603 Numerical Methods | Autumn Semester 2022-23 15 CE-603 Numerical Methods | Autumn Semester 2022-23 16

15 16

4
 9/12/22

Example - Jacobi Method Example - Jacobi Method

The Jacobi matrix will be, Next, rotate in (2,3) plane,
é1.0000 0 -0.0094 ù Note: The off-diagonal term is now aqq - app 0.2132 - 0.2837 sgn(t )
t= = = -12.15 , t = = - 0.0411
J2 = ê 0 1 0 ú much smaller than its original value. In 2apq 2 ´ 0.0029 t + 1 +t 2
ê ú many cases, the value may be close to
ëê0.0094 0 1.0000 ûú zero and the matrix not require any From the calculations of t, we get,
The rotated matrix will be, further rotation. c= 1 = 0.9992, s = tc = -0.0410
1 + t2
The rotated matrix will be,
é0.7801 0 0 ù é0.7801 0 0 ù
A (2) = JT2 A (1) J2 = ( JT2 J1T ) A (0) ( J1 J2 ) = ê 0 0.2837 0.0029 ú A (3) = JT3 A (2) J3 = ( JT3 JT2 J1T ) A (0) ( J1 J2 J3 ) = ê 0 0.2838 0 ú
ê ú ê ú
êë 0 0.0029 0.2132 úû êë 0 0 0.2131úû
We therefore get, l1 = 0.2131, l2 = 0.2838, l3 = 0.7801 (sorted in ascending order).

CE-603 Numerical Methods | Autumn Semester 2022-23 17 CE-603 Numerical Methods | Autumn Semester 2022-23 18

17 18

Example - Jacobi Method Jacobi Method Algorithm

The eigenvector estimates are obtained from the product of Jacobi matrices: Algorithm for Jacobi Method has been uploaded on Moodle (totally 14 steps).
x = J1 J2 J3
Algorithm Jacobi (A, V, N, k, eps)
By multiplying, we get,
Algorithm to find eigenvalues of a real symmetric matrix A using Jacobi method. If
é 0.8628 -0.5054 0.0127 ù é 1.000 -0.5864 0.0127 ù tolerance eps > 0, this algorithm overwrites A = V T AV where V is orthogonal and
x = ê 0.5055 0.8619 -0.0401ú = ê0.5859 1.000 -0.0401ú º [ x 1 x2 x3 ] off (VT AV ) £ tol ´ A F .
ê ú ê ú
êë0.0094 0.0410 0.9996 úû êë0.0109 0.0476 1.000 úû N N
The stopping criteria is based on Frobenius norm A F
= å å aij2
i =1 j =1
These estimated eigenvectors have much less accuracy than the eigenvalues.
NOTE: The eigenvectors are associated with eigenvalues in the order they were found in
the diagonal matrix, and not the order after sorting in ascending order.

CE-603 Numerical Methods | Autumn Semester 2022-23 19 CE-603 Numerical Methods | Autumn Semester 2022-23 20

19 20

5
 9/12/22

Homework 8 Eigenvectors – Inverse Iterations

Homework has been uploaded on Moodle. This is one of the most powerful method to determine eigenvectors after the eigenvalues
have been evaluated.
Determine the eigenvalues of the system matrix for 8-story industrial building given in
earlier Homework. The eigenvalue problem is:
(Due on September 25, 2022 after Mid-Semester Exam) (A - li I) x i = 0
For each eigenvalue, carry out LU decomposition of (A-l I) (required only once),
(A - l I) = L U
L y (m+1) = z(m)
Solve iterates of eigenvector estimates as:
U w (m+1) = y (m+1)
w (m+1)
z(m+1) = m³0
w (m+1)
¥

CE-603 Numerical Methods | Autumn Semester 2022-23 21 CE-603 Numerical Methods | Autumn Semester 2022-23 22

21 22

Eigenvectors – Inverse Iterations Summary

Since (A-l I) is nearly singular for each eigenvalue, the last diagonal term in U will be oJacobi method for determination of eigenvalues and eigenvectors
nearly zero (due to floating point error, it is unlikely to be exactly zero). If it is found to be
oEigenvectors using Inverse Iteration method
exactly equal to zero, change it to some small number e.
For initial guess z(0), take
z(0) = L e, where e = [1, ! , 1]T
The eigenvectors have one or two less significant digits of accuracy than the eigenvalue.

CE-603 Numerical Methods | Autumn Semester 2022-23 23 CE-603 Numerical Methods | Autumn Semester 2022-23 24

23 24

6
 9/12/22

Next Lecture (L-14)

The next lecture (L-14) will be held after the Mid-Term exam on Thursday, September 22,
2022.

End of Lecture 13

CE-603 Numerical Methods | Autumn Semester 2022-23 25 CE-603 Numerical Methods | Autumn Semester 2022-23 26

25 26