Numerical Linear Algebra

MATH10098

Monday 20th December 2021

1300-1500 † *

†
All students: You have an additional 1 hour to assemble and submit your PDF.

Final submission deadline: 16:00

*Students with a Schedule of Adjustment for additional time in examinations:

You are entitled to a further fixed additional 1 hour for this remote examination.

Final submission deadline: 17:00

Attempt all questions

Important instructions

1. Start each question on a new sheet of paper.

2. Number your sheets of paper to help you scan them in order.
3. Only write on one side of each piece of paper.
4. If you have rough work to do, simply include it within your overall answer – put
brackets at the start and end of it if you want to highlight that it is rough work.
MATH10098 Numerical Linear Algebra 1

(1) (Total: 55 marks) This question is about solving a system of linear equations Ax =
b, with A ∈ Rn×n an invertible matrix and b ∈ Rn . In the following, assign unit
cost to the elementary arithmetic operations addition, subtraction, multiplication
and division. Justify all of your answers.

(a) (i) Suppose Ax = b and Ab

x = b + ∆b. Show that

kx − x̂kp k∆bkp
≤ κp (A) ,
kxkp kbkp

where κp (A) is the condition number of A in the p-norm. [9 marks]

 
1 −1
(ii) Consider the particular example A = , for some 0 < α ≤ 1.
−1 1 + α2
Suppose the algorithm that we use to compute the solution to Ax = b
yields a computed solution xb with Abx = b + ∆b and k∆bk kbk∞
∞
≤ 0.01. We
1
have computed the solution for α = 1 and α = 2 , and observed the following
errors:

a) α =? b) α =?
kx−x̂k∞
kxk∞
0.05 0.15

Based on the result in part (i), decide which column corresponds to which
value of α. Carefully justify your answer. [10 marks]
You may use, without proof, the  following
 formulae for the determinant
b11 b12
and inverse of a matrix B = : det(B) = b11 b22 − b12 b21 and
  b 21 b22

−1 1 b22 −b12
B = det(B) .
−b21 b11
(b) Suppose A is symmetric. Then A diagonalises as A = QDQT , where the
diagonal matrix D ∈ Rn×n has the eigenvalues λi of A as diagonal entries, and
Q ∈ Rn×n are the eigenvectors of A,
the columns qi of the orthogonal matrix P
in the same order. This means that A = ni=1 λi qi qT
i .

(i) Show that A−1 = ni=1 λ−1 T

P
i qi q i . [3 marks]
Using part (i), the solution to Ax = b is given by
n
X
x=A b= −1
λ−1 T
i qi qi b. (1)
i=1

(ii) Suppose we want to use (1) to compute the solution x. What is the
computational cost of this approach? Explicitly state the cost of all steps
and the total cost, and do not use O(·) notation. Implement all steps
efficiently. You may assume that the eigenvalues and eigenvectors of A are
computed using Algorithm QR-Eig, and that Algorithm QR-Eig converges
after kmax iterations and uses Algorithm MGS (Modified Gram-Schmidt) to
compute QR factorisations. [12 marks]
(iii) Is the approach in part (ii) an efficient method for computing x? Compare
its computational cost to that of Algorithm GE (Gaussian elimination).
[4 marks]

[Please turn over]

MATH10098 Numerical Linear Algebra 2

(iv) Based on the expression in (1), briefly explain why computing x in this
way is very susceptible to rounding errors in floating point arithmetic when
κ2 (A), the condition number of A in the 2-norm, is large. [5 marks]
(c) Suppose you have already solved the system Ax = b using Algorithm GE
(Gaussian elimination). Show that the solution z to AT z = b can be computed
in O(n2 ) cost. (This does not include the cost of solving the original system
Ax = b.) [7 marks]
(d) The minimiser of kAx − bk2 , for b ∈ Rm and A ∈ Rm×n with m ≥ n and
rank(A) = n, solves the normal equations AT Ax = AT b. Briefly discuss why
computing the minimiser x via solving the normal equations with Algorithm
GEPP (Gaussian elimination with partial pivoting) typically leads to a large
error in the computed solution x
b. [5 marks]

[Please turn over]

MATH10098 Numerical Linear Algebra 3

(2) (Total: 30 marks) This question is about iterative methods for solving a linear
system Ax = b, with A ∈ Rn×n invertible and b ∈ Rn . In the following, assign unit
cost to the elementary arithmetic operations addition, subtraction, multiplication
and division. Justify all of your answers.

Below is the pseudo-code from lectures for a general iterative method:

Algorithm GI: General Iterative Method
Input: A = M + N ∈ Rn×n , b ∈ Rn , x0 ∈ Rn , εr > 0
Output: xk ∈ Rn , with Axk ≈ b
1: k = 0
2: while kAxk − bkp > εr do
3: k =k+1
4: Compute yk = b − Nxk−1
5: Solve Mxk = yk
6: end while

Consider the splitting A = L + D + U, where D is diagonal, and L and U are strictly

lower and upper triangular, respectively.

(a) The Successive Over-relaxation (SOR) method corresponds to the particular

choice M = L + ω1 D and N = (1 − ω1 )D + U.
(i) How would you efficiently solve the linear system in line 5 for the SOR
method? [2 marks]
(ii) What is the computational cost of one iteration of the SOR method?
Assume an efficient implementation. [10 marks]
(b) The Jacobi method corresponds to the particular choice M = D and N = L+U.
(i) Suppose A ∈ Rn×n is upper triangular. Show that the Jacobi method
converges in n iterations, i.e. show that ken kp = 0. [8 marks]
(ii) Suppose you have applied the Jacobi method to tridiagonal matrices A1
and A2 , with entries
 

 8 if i = j 6 if i = j

(A1 )ij = 2 if i = j + 1 or i = j − 1 , (A2 )ij = 2 if i = j + 1 or i = j − 1 .
 
0 otherwise 0 otherwise
 

You observe the following errors ek = x − xk :

k kek k∞ for A1 kek k∞ for A2

1 0.1 0.1
2 ? ?

Fill in the second row of the table, i.e. predict what errors you expect to
observe at iteration k = 2. Carefully justify your answer. [10 marks]

[Please turn over]

MATH10098 Numerical Linear Algebra 4

(3) (Total: 15 marks) This question is about computing eigenvalues and eigenvectors
of a symmetric matrix A ∈ Rn×n . We order the eigenvalues of A in decreasing order
of absolute value,
|λ1 | ≥ |λ2 | ≥ · · · ≥ |λn |,
and choose the corresponding eigenvectors x1 , . . . , xn to be an orthonormal basis of
Rn . Justify all of your answers.

(a) Recall Theorem 6.5 on the convergence of the approximate eigenvalue λ(k) and
approximate eigenvector z(k) computed by Algorithm PI (Power Iteration):
Theorem 6.5 Suppose |λ1 | > |λ2 | and xT1 z(0) 6= 0. Then there exists a sequence
{σk }k∈N with σk = ±1 such that as k → ∞,

kσk z(k) − x1 k2 → 0, and |λ(k) − λ1 | → 0.

(i) What does the approximate eigenvalue λ(k) computed by Algorithm PI

converge to when xT1 z(0) = 0? Assume that xT2 z(0) 6= 0, and that the
algorithm is implemented in exact arithmetic.
You do not need to provide a full proof as in Theorem 6.5, although you
may do so if you wish. You should provide a discussion of the behaviour of
Ak z (0) as k → ∞. [7 marks]
(ii) How might your answer to part (i) change if Algorithm PI is implemented
in floating point arithmetic? [3 marks]
(b) Suppose the symmetric matrix A ∈ R6×6 has eigenvalues −7, −3, 1, 2, 5 and 6.
Which eigenvalue will the following algorithms converge to? You may assume
that the starting guess z(0) satisfies xTj z(0) 6= 0 for all eigenvectors xj of A.
(i) Algorithm PI (Power Iteration),
(ii) Algorithm SII (Shifted Inverse Iteration) with shift s = 0,
(iii) Algorithm SII (Shifted Inverse Iteration) with shift s = 4.
[5 marks]

[End of Paper]