{

"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Simple iteration for systems of linear equations"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"First, generate a random diagonally dominant matrix, for testing."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import numpy as np\n",
"rndm = np.random.RandomState(1234)\n",
"\n",
"n = 10\n",
"A = rndm.uniform(size=(n, n)) + np.diagflat([15]*n)\n",
"b = rndm.uniform(size=n)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# I. Jacobi iteration\n",
"\n",
"Given\n",
"\n",
"$$\n",
"A x = b\n",
"$$\n",
"\n",
"separate the diagonal part $D$,\n",
"\n",
"$$ A = D + (A - D) $$\n",
"\n",
"and write\n",
"\n",
"$$\n",
"x = D^{-1} (D - A) x + D^{-1} b\\;.\n",
"$$\n",
"\n",
"Then iterate\n",
"\n",
"$$\n",
"x_{n + 1} = B x_{n} + c\\;,\n",
"$$\n",
 "\n",
"where \n",
"\n",
"$$\n",
"B = D^{-1} (A - D) \\qquad \\text{and} \\qquad c = D^{-1} b\n",
"$$\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's construct the matrix and the r.h.s. for the Jacobi iteration"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"diag_1d = np.diag(A)\n",
"\n",
"B = -A.copy()\n",
"np.fill_diagonal(B, 0)\n",
"\n",
"D = np.diag(diag_1d)\n",
"invD = np.diag(1./diag_1d)\n",
"BB = invD @ B \n",
"c = invD @ b"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"# sanity checks\n",
"from numpy.testing import assert_allclose\n",
"\n",
"assert_allclose(-B + D, A)\n",
"\n",
"\n",
"# xx is a \"ground truth\" solution, compute it using a direct method\n",
"xx = np.linalg.solve(A, b)\n",
"\n",
"np.testing.assert_allclose(A@xx, b)\n",
"np.testing.assert_allclose(D@xx, B@xx + b)\n",
"np.testing.assert_allclose(xx, BB@xx + c)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Check that $\\| B\\| \\leqslant 1$:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.36436161983015336"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"np.linalg.norm(BB)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Do the Jacobi iteration"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"n_iter = 50\n",
"\n",
"x0 = np.ones(n)\n",
"x = x0\n",
"for _ in range(n_iter):\n",
" x = BB @ x + c"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([ 1.11022302e-16, 1.11022302e-16, -1.11022302e-16,\n",
" 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,\n",
" -1.38777878e-17, 0.00000000e+00, 2.77555756e-17,\n",
" 1.11022302e-16])"
]
},
"execution_count": 6,
"metadata": {},
 "output_type": "execute_result"
}
],
"source": [
"# Check the result:\n",
"\n",
"A @ x - b"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Task I.1\n",
"\n",
"Collect the proof-of-concept above into a single function implementing the
Jacobi iteration. This function should receive the r.h.s. matrix $A$, the l.h.s.
vector `b`, and the number of iterations to perform.\n",
"\n",
"\n",
"The matrix $A$ in the illustration above is strongly diagonally dominant, by
construction. \n",
"What happens if the diagonal matrix elements of $A$ are made smaller? Check
the convergence of the Jacobi iteration, and check the value of the norm of
$B$.\n",
"\n",
"(20% of the total grade)\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# ... ENTER YOUR CODE HERE ..."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# II. Seidel's iteration."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### Task II.1\n",
"\n",
"Implement the Seidel's iteration. \n",
"\n",
"Test it on a random matrix. Study the convergence of iterations, relate to the
norm of the iteration matrix.\n",
"\n",
"(30% of the total grade)"
 ]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# ... ENTER YOUR CODE HERE ..."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# III. Minimum residual scheme"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Task III.1\n",
"\n",
"Implement the $\\textit{minimum residual}$ scheme: an explicit non-stationary
method, where at each step you select the iteration parameter $\\tau_n$ to minimize
the residual $\\mathbf{r}_{n+1}$ given $\\mathbf{r}_n$. Test it on a random matrix,
study the convergence to the solution, in terms of the norm of the residual and the
deviation from the ground truth solution (which you can obtain using a direct
method). Study how the iteration parameter $\\tau_n$ changes as iterations
progress.\n",
"\n",
"(50% of the grade)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# ... ENTER YOUR CODE HERE ..."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
 },
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.2"
}
},
"nbformat": 4,
"nbformat_minor": 2
}