Assignment 4: Linear Algebra & System Analysis
Teaching Material / Class Submission
Difficulty Level: Intermediate / Applied Mathematics
Topics Covered: Eigenvalues, Eigenvectors, Diagonalization, Matrix Powers, Systems of
Linear ODEs
1. Introduction
Linear Algebra is a cornerstone of modern science, providing the tools to analyze and solve
complex systems. Eigenvalue and eigenvector analysis is a key part of this, offering a way to
"simplify" a complex linear transformation by finding the directions in which the
transformation acts as a simple scaling. This concept is critical for understanding system
stability, vibration, and data analysis.
2. Core Concepts
• Eigenvalue and Eigenvector: For a square matrix $A$, a non-zero vector
$\mathbf{v}$ is an eigenvector of $A$ if $A\mathbf{v} = \lambda\mathbf{v}$ for
some scalar $\lambda$. The scalar $\lambda$ is called the eigenvalue
corresponding to $\mathbf{v}$.
• Characteristic Equation: Eigenvalues are found by solving the characteristic
equation: $\det(A - \lambda I) = 0$, where $I$ is the identity matrix.
• Diagonalization: A matrix $A$ is diagonalizable if it can be written as $A = PDP^{-
1}$, where $D$ is a diagonal matrix of eigenvalues and $P$ is an invertible matrix
whose columns are the corresponding linearly independent eigenvectors.
• Systems of ODEs: A system of linear differential equations, $\mathbf{x}' =
A\mathbf{x}$, can be solved using eigenvalues. The solution describes how a system
(e.g., connected springs, predator-prey populations) evolves over time.
3. Applications
• Physics: Finding the principal axes of rotation of a rigid body and the normal modes
of vibration of an oscillating system.
• Computer Science: Google's PageRank algorithm (finding the principal eigenvector
of the web's link matrix) and Principal Component Analysis (PCA) for data
compression and machine learning.
• Engineering: Analyzing the stability of structures and control systems. The signs of
the eigenvalues determine if a system is stable, unstable, or oscillatory.
4. Example Insights
• Diagonalization makes computing high powers of a matrix trivial: $A^k = (PDP^{-
1})^k = PD^kP^{-1}$. This is used to predict the long-term state of a system, such as
in Markov chains.
• The eigenvectors of a system's matrix ($A$) represent the "natural states" or
"modes" of the system. Any state of the system can be described as a combination of
these basic modes.
• In a system of ODEs $\mathbf{x}' = A\mathbf{x}$, if all eigenvalues of $A$ have
negative real parts, the system is stable and returns to equilibrium ($x=0$).
5. Summary
Eigenvalue analysis transforms a complex, coupled system (represented by $A$) into a
simple, uncoupled set of scalar problems (represented by $D$). This decomposition is one
of the most powerful techniques in applied mathematics.
Problems & Solutions
Problem 1
Find the eigenvalues and corresponding eigenvectors for the matrix $A = \begin{pmatrix} 7
& -2 \\ 4 & 1 \end{pmatrix}$.
Solution:
1. Find Eigenvalues: Solve the characteristic equation $\det(A - \lambda I) = 0$.
$$\det \begin{pmatrix} 7-\lambda & -2 \\ 4 & 1-\lambda \end{pmatrix} = 0$$
$$(7-\lambda)(1-\lambda) - (-2)(4) = 0$$
$$7 - 8\lambda + \lambda^2 + 8 = 0$$
$$\lambda^2 - 8\lambda + 15 = 0$$
$$(\lambda - 3)(\lambda - 5) = 0$$
The eigenvalues are $\lambda_1 = 3$ and $\lambda_2 = 5$.
2. Find Eigenvectors: For each $\lambda$, solve $(A - \lambda I)\mathbf{v} = 0$.
o For $\lambda_1 = 3$:
$$\begin{pmatrix} 7-3 & -2 \\ 4 & 1-3 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix}
= \begin{pmatrix} 0 \\ 0 \end{pmatrix} \implies \begin{pmatrix} 4 & -2 \\ 4 & -2
\end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 0 \\ 0
\end{pmatrix}$$
This gives the equation $4x - 2y = 0$, or $y = 2x$. The eigenvector is $\mathbf{v}_1 =
\begin{pmatrix} 1 \\ 2 \end{pmatrix}$.
o For $\lambda_2 = 5$:
$$\begin{pmatrix} 7-5 & -2 \\ 4 & 1-5 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix}
= \begin{pmatrix} 0 \\ 0 \end{pmatrix} \implies \begin{pmatrix} 2 & -2 \\ 4 & -4
\end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 0 \\ 0
\end{pmatrix}$$
This gives the equation $2x - 2y = 0$, or $y = x$. The eigenvector is $\mathbf{v}_2 =
\begin{pmatrix} 1 \\ 1 \end{pmatrix}$.
Answer: $\lambda_1 = 3$ with $\mathbf{v}_1 = \begin{pmatrix} 1 \\ 2 \end{pmatrix}$;
$\lambda_2 = 5$ with $\mathbf{v}_2 = \begin{pmatrix} 1 \\ 1 \end{pmatrix}$.
Problem 2
A system is described by the matrix $A$ from Problem 1. If the initial state is $\mathbf{x}_0
= \begin{pmatrix} 3 \\ 4 \end{pmatrix}$, find the state of the system after $k$ steps,
$\mathbf{x}_k = A^k \mathbf{x}_0$. (You may leave the answer in terms of $k$).
Solution:
1. Express $\mathbf{x}_0$ in the eigenvector basis: We need to find $c_1, c_2$ such
that $\mathbf{x}_0 = c_1\mathbf{v}_1 + c_2\mathbf{v}_2$.
$$\begin{pmatrix} 3 \\ 4 \end{pmatrix} = c_1 \begin{pmatrix} 1 \\ 2 \end{pmatrix} + c_2
\begin{pmatrix} 1 \\ 1 \end{pmatrix} \implies \begin{pmatrix} 1 & 1 \\ 2 & 1
\end{pmatrix} \begin{pmatrix} c_1 \\ c_2 \end{pmatrix} = \begin{pmatrix} 3 \\ 4
\end{pmatrix}$$
Solving this (e.g., subtract row 1 from row 2) gives $c_1 = 1$ and $c_2 = 2$.
So, $\mathbf{x}_0 = \mathbf{v}_1 + 2\mathbf{v}_2$.
2. Apply $A^k$: Use the property $A^k(c\mathbf{v}) = c(A^k\mathbf{v}) =
c(\lambda^k \mathbf{v})$.
$$\mathbf{x}_k = A^k \mathbf{x}_0 = A^k (\mathbf{v}_1 + 2\mathbf{v}_2) =
A^k\mathbf{v}_1 + 2(A^k\mathbf{v}_2)$$
$$\mathbf{x}_k = \lambda_1^k \mathbf{v}_1 + 2(\lambda_2^k \mathbf{v}_2)$$
$$\mathbf{x}_k = 3^k \begin{pmatrix} 1 \\ 2 \end{pmatrix} + 2 \cdot 5^k \begin{pmatrix}
1 \\ 1 \end{pmatrix}$$
Answer: $\mathbf{x}_k = \begin{pmatrix} 3^k + 2 \cdot 5^k \\ 2 \cdot 3^k + 2 \cdot 5^k
\end{pmatrix}$.
• Explanation: This method is far simpler than diagonalizing $A$ or computing
$A^k$ directly. It shows how the initial state $\mathbf{x}_0$ is a mix of the two
"modes" ($\mathbf{v}_1$ and $\mathbf{v}_2$), and each mode grows
independently at a rate set by its eigenvalue ($\lambda_1=3$ and $\lambda_2=5$).
