Professional Documents
Culture Documents
AM
PH
Duong T. PHAM
T.
g.
APPLIED LINEAR ALGEBRA
on
Du
1 Vector spaces
AM
2 Linear independence
PH
3 Bases and Dimension
T.
4 Linear Subspaces g.
on
5 Rank of matrices
Du
AM
1 Associativity of addition: u + (v + w ) = (u + v ) + w
2 Commutativity of addition: u + v = v + u
PH
3 Identity element of addition: ∃0 ∈ V s.t. u + 0 = u ∀u ∈ V (Here,
0 is called zero vector of V )
T.
4 Inverse elements of addition: ∀u ∈ V , ∃(−u) ∈ V s.t. u + (−u) = 0
g.
on
5 Compatibility: α(βu) = (αβ)u ∀u ∈ V , ∀α, β ∈ R
Du
AM
Addition: x + y = (x1 + y1 , x2 + y2 , . . . , xn + yn )
PH
Scalar multiplication: α · x = (αx1 , αx2 , . . . , αxn ) α ∈ R
T.
(Rn , +, ·) is a vector space over R.
g.
on
Ex. V = Mm,n (R) with the addition of matrices (+) and the
multiplication between a number and a matrix (·);
Du
AM
Pn [x] : set of polynomials with real coefficients and orders ≤ n ;
PH
Addition (+): addition of polynomials
Scalar multiplication (·): multiplication between a real number with a
T.
polynomial;
Then g.
on
(P[x], +, ·) is a vector space
Du
AM
Ex. C [a, b] : set of continuous functions on [a, b]
PH
Addition (+): addition of two functions
T.
Scalar multiplication (·): multiplication between a real number and a
function g.
on
Then (C [a, b], +, ·) is a vector space.
Du
AM
u + 01 = 01 + u = u ∀u ∈ V (1)
u + 02 = 02 + u = u ∀u ∈ V (2)
PH
(1) (2)
We have 02 = 01 + 02 = 01
T.
Inverse vector of a vector u is unique
g.
Suppose that there are two inverse vectors u 1 and u 2 of u. Then
on
Du
u 1 + u = u + u 1 = 0 (3)
u 2 + u = u + u 2 = 0 (4)
(3) (4)
We have u 2 = u 2 + 0 = u 2 + u + u 1 = 0 + u 1 = u 1
AM
∀u, v , w ∈ V : u + v = u + w =⇒ v = w
u + v = u + w =⇒ (−u) + (u + v ) = (−u) + (u + w )
PH
=⇒ (−u + u) + v = (−u + u) + w =⇒ 0 + v = 0 + w =⇒ v = w
T.
g.
α · 0 = 0 ∀α ∈ R and 0 · u = 0 ∀u ∈ V
on
0 + α · 0 = α · 0 = α · (0 + 0) = α · 0 + α · 0 =⇒ 0 = α · 0
Du
0 · u + 0 = 0 · u = (0 + 0) · u = 0 · u + 0 · u =⇒ 0 = 0 · u
AM
If α = 0 or u = 0, then it is obvious that α · u = 0.
PH
Assume that α · u = 0 and α 6= 0. Then
1 1 1
T.
· (α · u) = · 0 = 0 ⇐⇒ α ·u =0
α α α
g. ⇐⇒ 1 · u = 0⇐⇒ u = 0
on
(−1) · u = −u
Du
AM
u = α1 u 1 + α2 u 2 + . . . + αn u n
PH
Definition.
T.
. . u n ∈ V are said to be linearly dependent if ∃ α1 , α2 , . . . , αn ,
u 1 , u 2 , .P
where αi2 6= 0, satisfying
g.
on
α1 u 1 + α2 u 2 + . . . + αn u n = 0
Du
Definition.
u 1 , u 2 , . . . u n ∈ V are said to be linearly independent if they are not
linearly dependent.
⇐⇒ If α1 u 1 + α2 u 2 + . . . + αn u n = 0 then α1 = α2 = . . . = αn = 0.
Duong T. PHAM October 16, 2019 10 / 40
Linear Independence
Ex. Let u 1 = (1, 2, −1, 0), u 2 = (1, 0, 1, 3), u 3 = (−1, 1, 0, 2).
Decide if u 1 , u 2 , u 3 are linearly independent.
Suppose that α1 u 1 + α2 u 2 + α3 u 3 = 0. Then
AM
1 1 −1 0
2 0 1 0
PH
−1 + α2 1 + α3 0 = 0
α1
0 3 2 0
T.
1 1 −1 0 1 1 −1 0 1 1 −1 0
2 0 1 0 g.
→
0 −2 3 0
→
0 −2 3 0
on
−1 1 0 0 0 2 −1 0 0 0 2 0
0 13
Du
0 3 2 0 0 3 2 0 0 2 0
1 1 −1 0
0 −2
→
3 0 ⇒ α1 = α2 = α3 = 0. Hence, u 1 , u 2 , u 3
0 0 2 0 are linearly independent.
0 0 0 0
Duong T. PHAM October 16, 2019 11 / 40
Basic Properties
Basic properties
(
linearly independent ⇔ u 1 6= 0
{u 1 } is
AM
linearly dependent ⇔ u 1 = 0
PH
(
linearly independent ⇔ u 1 , u 2 are Not proportional
{u 1 , u 2 } is
linearly dependent ⇔ u 1 , u 2 are proportional
T.
A set of vectors containing 0 is linearly dependent
g.
on
Let S1 ⊂ S2 be two set of vectors. Then
Du
Basic Properties
AM
S is linearly dependent ⇒ a vector in S is a linear combination of
some other vectors in S.
PH
Let {u 1 , . . . , u n } is linearly independent;
T.
{u 1 , . . . , u n , u} is linearly independent if u is NOT a linear
g.
combination of u 1 , . . . , u n .
on
{u 1 , . . . , u n , u} is linearly dependent if u is a linear combination of
Du
u 1, . . . , u n.
AM
B is linearly independent : ∀v 1 , . . . , v n ∈ B; ∀α1 , . . . , αn ∈ R
α1 v 1 + . . . αn v n = 0 =⇒ α1 = . . . = αn = 0
PH
B is a spanning set of V :
∀v ∈ V ; ∃v 1 , . . . , v n ∈ B and ∃α1 , . . . , αn ∈ R s.t.
T.
v = α1 v 1 + . . . + αn v n
g.
on
{(1, 0), (0, 1)} is a basis of R2 since:
Du
It is linearly independent;
Every (x, y ) ∈ R2 is a linear
combination of (1, 0) and (0, 1), i.e.
(x, y ) = x(1, 0) + y (0, 1)
Duong T. PHAM October 16, 2019 14 / 40
Dimension
AM
Definition.
PH
Let V be a vector space; B be a basis of V .
If B has infinitely many elements, then V is a infinitely dimensional
T.
space
g.
If B has finite number of elements, then dimV := size(B)
on
Du
AM
Suppose that x1 , . . . , xn ∈ R such that x1 e 1 + . . . + xn e n = 0
1 0 ... 0 0
PH
x1 = 0
0 1 ... 0 0 ..
=⇒ ... ... ... ... ... =⇒ .
T.
x = 0
0 0 ... 1 0 n
g.
on
{e 1 , . . . , e n } spans Rn ? Because every (x1 , . . . , xn ) ∈ Rn ;
(x1 , . . . , xn ) = x1 e 1 + . . . + xn e n
Du
=⇒ {e 1 , . . . , e n } is a basis of Rn
dimRn = n
AM
Ek` =0
m X
X n
αij E ij = 0m,n =⇒ αij = 0 for all i = 1, m and j = 1, n
PH
i=1 j=1
=⇒ {E ij : i = 1, m, j = 1, n} is linearly independent.
T.
m X
n
g.
For any A = [aij ] ∈ Mm,n (R); A =
X
aij E ij
on
i=1 j=1
{E ij
Du
AM
Assume that a0 1 + a1 x + a2 x 2 + . . . + an x n = 0 ∀x ∈ R
=⇒ a0 = a1 = . . . = an = 0.
PH
Hence, {1, x, . . . , x n } is linearly independent
T.
f (x) = a0 + a1 x + . . . + an x n
g.
on
for some a0 , a1 , . . . , an ∈ R.
Du
AM
Ex. P[x] is the vector space of all polynomials with real coefficients.
PH
P[x] is a infinite dimensional vector space.
T.
{1, x, x 2 , . . . , x n , . . . , } is a basis of P[x]
dimP[x] = ∞ g.
on
Du
AM
Theorem.
Let V be a n-dimensional vector space. There hold
PH
A set which contains more than n vectors is linearly dependent
A set S which contains n vectors is a basis if and only if S satisfies at
T.
least one of the two following conditions:
- S is linearly independent;
g.
- S spans V ;
on
A set S of k linearly independent vectors ( k ≤ n ) can be added
Du
Definition.
Let V be a n-dimensional vector space and {u 1 , . . . , u n } be a basis. Every
u ∈ V can uniquely be written as
AM
u = x1 u 1 + . . . + xn u n .
PH
Then (x1 , . . . , xn ) is called coordinate of u with respect to {u 1 , . . . , u n }
T.
Notation: We write u/{u 1 ,...,u n } = (x1 , . . . , xn )
g.
on
Ex. In R2 , the set {(1, 1), (1, 2)} is a basis.
Du
AM
x1
.
⇐⇒ u 1 . . . u n .. = x
PH
xn
T.
Let U = {u 1 , . . . , u n } and V = {v 1 , . . . , v n } be two bases of V ;
g.
v 1 = a11 u 1 + a12 u 2 + . . . + a1n u n
on
v 2 = a21 u 1 + a22 u 2 + . . . + a2n u n
Du
..................................................
v n = an1 u 1 + an2 u 2 + . . . + ann u n
AM
a1n a2n . . . ann
PH
Proposition.
TUV is the basis transformation matrix from U to V ⇐⇒
T.
u 1 u 2 . . . u n TUV = v 1 v 2 . . . v n
g.
on
Remark:
−1
Du
TUV = u 1 u 2 . . . u n v1 v2 . . . vn
| {z }| {z }
written in columns written in columns
−1
TV U = TUV
Theorem.
Two bases of V : U = {u 1 , . . . , u n } and V = {v 1 , . . . , v n }.
Let u ∈ V and u/U = (x1 , . . . , xn ); u/V = (y1 , . . . , yn );
AM
We have
x1 y1
PH
.. ..
. = TUV . or u/U = TUV · u/V
T.
xn yn
Proof. g.
x1 y1 y1
on
. . ..
u 1 ... u n .. = u = v 1 ... v n .. = u 1 ... u n TUV
Du
.
xn yn yn
x1 y1
.. ..
=⇒ . = TUV .
xn yn
Duong T. PHAM October 16, 2019 24 / 40
Linear Subspace
Definition.
Let (V , +, ·) be a vector space and U ⊂ V . If U satisfies:
U is closed under addition : ∀u, v ∈ U, u + v ∈ U,
AM
U is closed under scalar multiplication : ∀u ∈ U, ∀α ∈ R, αu ∈ U,
PH
then (U, +, ·) is a vector space and called subspace of V .
T.
Ex. Let U1 = {(x1 , x2 , x3 ) ∈ R3 | x1 + x2 + x3 = 0} and
g.
Note that (R3 , +, ·) is a vector space
on
U1 is a subspace of R3 since:
Du
AM
Ex. U2 = {(x1 , x2 , x3 ) ∈ R3 | x1 + x2 + x3 ≥ 0} is NOT a subspace of R3
PH
(1, 1, 1) ∈ U2 since 1 + 1 + 1 ≥ 0
T.
But −1(1, 1, 1) ∈
/ U2 since −1 − 1 − 1 = −3 < 0 =⇒ U2 is NOT
closed under scalar multiplication
g.
on
U2 is a not vector space and it is not a subspace of R3 .
Du
Theorem.
Let U be a subspace of vector space V and dim V = n. Then
AM
dim U ≤ n
PH
dim U = n ⇐⇒ U = V
T.
Definition.
g.
Let V be a vector space and S = {u 1 , u 2 , . . . , u m }⊂ V . The span of S
on
is the set of all linear combinations of S, i.e.
Du
span(S) = {α1 u 1 + α2 u 2 + . . . + αm u m : αi ∈ R, i = 1, . . . , m}
Proposition.
Let V be a vector space and S = {u 1 , u 2 , ..., u m }⊂ V . Then span(S) is a
AM
linear subspace of V
PH
Proof. Let a, b ∈ span(S) and α ∈ R. Then
a = a1 u 1 + a2 u 2 + . . . am u m
T.
b = b1 u 1 + b2 u 2 + . . . bm u m
g.
for some ai , bi ∈ R, i = 1, 2, ..., m. We have
on
Du
Proposition.
AM
Let V be a vector space and S = {u 1 , u 2 , ..., u m }⊂ V . Then
span(S) is the smallest subspace of V which contain S.
PH
S is the spanning set of span(S)
T.
Proof.
Assume that U is a subspace of V and S ⊂ U. For any
g.
α1 , α2 , ..., αm ∈ R , since u 1 , ..., u m ∈ U and U is a vector space,
on
Du
α1 u 1 + α2 u 2 + ... + αm u m ∈ U.
Hence, span(S) ⊂ U.
Proposition.
AM
Given U, W : subspaces of a vector space V , then U ∩ W is also a
PH
subspace of V .
T.
Proof: Let v , w ∈ U ∩ W and let α ∈ R.
g.
v , w ∈ U and U: vector space =⇒ v + w ∈ U and αv ∈ U
on
v , w ∈ W and W : vector space =⇒ v + w ∈ W and αv ∈ W
Du
Proposition.
Let U1 , U2 be linear subspaces of V . We define
AM
U1 + U2 = {a 1 + a 2 : a 1 ∈ U1 , a 2 ∈ U2 }.
PH
Then, U1 + U2 is a linear subspace of V .
T.
a = a1 + a2g. for some a 1 ∈ U1 , a 2 ∈ U2
b = b1 + b2 for some b 1 ∈ U1 , b 2 ∈ U2 .
on
Du
AM
Corollary.
PH
Let U1 , ..., Uk be subspaces of a vector space V . Then
T.
U1 + ... + Uk = {u 1 + ... + u k : u 1 ∈ U1 , ..., u k ∈ Uk }
is a linear subspace of V .
g.
on
Du
Proposition.
If U = hu 1 , . . . , u n i and V = hv 1 , . . . , v m i. Then
AM
U + V = hu 1 , . . . , u n , v 1 , . . . , v m i
PH
Proof.
Let a ∈ U + V . Then a = u + v for some u ∈ U and v ∈ V .
T.
U, V ⊂ hu 1 , . . . , u n , v 1 , . . . , v m i =⇒ u, v ∈ hu 1 , . . . , u n , v 1 , . . . , v m i
g.
=⇒ a = u + v ∈ hu 1 , . . . , u n , v 1 , . . . , v m i
on
=⇒ U + V ⊂ hu 1 , . . . , u n , v 1 , . . . , v m i
Du
AM
PH
Theorem.
Let U1 and U2 be two subspaces of a vector space V . Then
T.
dim(U1 + U2 ) = dimU1 + dimU2 − dim(U1 ∩ U2 )
g.
on
Du
Definition.
Given A ∈ Mm,n (R); the rank of A is an natural number r satisfying:
AM
∃ a submatrix B ∈ Mr (R) of A s.t. det B 6= 0
PH
All submatrices of A whose sizes are greater than r have zero
determinants.
T.
Notation: rank A
Remark. g.
on
(
rank A = n ⇐⇒ det A 6= 0
If A ∈ Mn (R), then
Du
Using determinants:
Given A ∈ Mm,n (R);
AM
PH
Step 1: Find a square submatrix with biggest possible size (saying
= k) whose determinant is non-zero
T.
Step 2: Compute determinants of all submatrices of size k + 1
g.
which contain the submatrix in Step 1:
on
(i) If determinants of all these matrices of size k + 1 equal zero, then
Du
rank A = r
(ii) If ∃B ∈ Mk+1 (R) and det B 6= 0, then Repeat Step 2
AM
Definition ( Row elementary operations ).
The following operations are called row elementary operations:
PH
(i) Adding into one row with a linear combination of other rows
(ii) Exchanging two rows
T.
(iii) Multiplicating or dividing a row with a non-zero number.
g.
on
Theorem.
Du
AM
1 −3 4 5 2 1 −3 4 5 2
0 1 3 4 6 0 1 3 4 6
−3 5 −2 −3 −4 → 0 −4 10 12
A= →
PH
2
−2 3 5 6 4 0 −3 13 16 8
T.
1 −3 4 5 2 1 −3 4 5 2
g.
0 1 3 4 6 0 1 3 4 6
0 0 22 28 26 → 0 0 22 28 26
on
Du
0 0 22 28 26 0 0 0 0 0
rank A = 3
AM
Proposition.
PH
The set of all solutions of (?) is a linear subspace of Rn .
T.
Proof. Assume that x=(x1 , ..., xn ) and y =(y1 , ..., yn ) are solutions of (?)
and α ∈ R. Then
g.
on
A(x + αy ) = Ax + αAy = 0m,1 + 0m,1 = 0m,1 .
Du
Theorem.
If rank A = r , then the dimension of solution space of (?) is n − r .
AM
Ex. Find basis and dimension of the solution space of the system
PH
T.
1 0 −1 0 0 1 0 −1 0 0 1 0 −1 0 0
0 1 2 −1 0 → 0 1 2 −1 0 → 0 1 2 −1 0
1 1 1 −1 0
g.
0 1 2 −1 0 0 0 0 0 0
on
Du
S = {(a, −2a + b, a, b) : a, b ∈ R}