Linear Independent

Tuan Nguyen
Ngày 1 tháng 8 năm 2022

AI For Everyone AI4E Ngày 1 tháng 8 năm 2022 1 / 17

Overview

Group

Vector space

Linear independent

Basis

Rank

AI For Everyone AI4E Ngày 1 tháng 8 năm 2022 2 / 17

Group

Group. Consider a set G and an operation ⊗ : GxG → G defined on G.

Then G := (G, ⊗) is called a group if the following hold:
I ∀x, y ∈ G, x ⊗ y ∈ G
I ∀x, y , z ∈ G, (x ⊗ y ) ⊗ z = x ⊗ (y ⊗ z)
I ∃e ∈ G, ∀x ∈ G : x ⊗ e = e ⊗ x = x
I ∀x ∈ G, ∃y ∈ G : x ⊗ y = y ⊗ x = e
If additionally ∀x, y ∈ G : x ⊗ y = y ⊗ x then G = (G, ⊗) is an _Abelian
group.

AI For Everyone AI4E Ngày 1 tháng 8 năm 2022 3 / 17

Group (cont.)

Look at some examples:

I (R, +)
I (Z, ·)
I (Rn , +)
I (Rn×n , ·)

AI For Everyone AI4E Ngày 1 tháng 8 năm 2022 4 / 17

Vector space

A real-valued vector space V = (V; +; ·) is a set V with two

operations:

+:V +V →V
·:R·V →V

where:
I (V; +) is an Abelian group
I ∀λ, γ ∈ R, x, y ∈ V:
• λ(x + y ) = λx + λy
• (λ + γ)x = λx + γx

I ∀λ, γ ∈ R, x ∈ V : λ(γx) = (λγ)x

I 1·x =x
The elements x ∈ V are called vectors, examples: V = Rn .

AI For Everyone AI4E Ngày 1 tháng 8 năm 2022 5 / 17

Vector space (cont.)

We will denote a vector space (V, +, ·) by V when + and · are the

standard vector addition and scalar multiplication.

Moreover, we will use the notation x ∈ V for vectors in V to simplify

notation.

Default, the vectors are column vectors:

 
x1
 x2 
⇒ x T = x1 x2 ... xn ∈ R1×n
n×1
 
x = ...  ∈ R


xn

AI For Everyone AI4E Ngày 1 tháng 8 năm 2022 6 / 17

Vector subspaces

Let V = (V, +, ·) be a vector space and U ⊆ V, U 6= 0. Then U =

(U, +, ·) is called vector subspace of V (or linear subspace) if U is a
vector space with the vector space operations + and · restricted to U ×U
and R×U. We write U ⊆ V. To determine whether (U, +, ·) is a subspace
of V we still do need to show:
I 0∈U
I • ∀λ ∈ R, x ∈ U : λx ∈ U
• ∀x, y ∈ U : x + y ∈ U

Hình 1: subsets of R2

AI For Everyone AI4E Ngày 1 tháng 8 năm 2022 7 / 17

Linear Combination

Consider a vector space V and a finite number of vectors

x1 , x2 , ..., xk ∈ V . Then, every v ∈ V of the form:

v = λ1 v1 + λ2 v2 + ... + λk vk ∈ V

with λ1 , λ2 , ..., λk ∈ R is a linear combination of vectors

x1 , x2 , ..., xk .

AI For Everyone AI4E Ngày 1 tháng 8 năm 2022 8 / 17

Linear independent

Consider a vector space V and a finite number of vectors

x1 , x2 , ..., xk ∈ V . If ∃λ1 , λ2 , ..., λk ∈ R and ∃λi 6= 0
with i ∈ [1, k]:
k
X
λi vi = 0
i=1

Then, x1 , x2 , ..., xk are linearly dependent. Otherwise, x1 , x2 , ..., xk are

linearly independent.
Intuitively, a set of linearly independent vectors consists of vectors that
have no redundancy, i.e., if we remove any of those vectors from the set,
we will lose something.
Linear independence is one of the most important concepts in
linear algebra.

AI For Everyone AI4E Ngày 1 tháng 8 năm 2022 9 / 17

Linear independent (cont.)

(a) (b)

(c) (d)

AI For Everyone AI4E Ngày 1 tháng 8 năm 2022 10 / 17

Linear independent (cont.)

A practical way of checking whether vectors x1 , x2 , ..., xk ∈ V are linearly

independent is to use Gaussian elimination.
Write all vectors as columns of a matrix A and perform Gaussian
elimination until the matrix is in row echelon form.
I The pivot columns indicate the vectors, which are linearly
independent of the vectors on the left. Note that there is an ordering
of vectors when the matrix is built.
I The non-pivot columns can be expressed as linear combinations of
the pivot columns on their left
All column vectors are linearly independent if and only if all columns are
pivot columns. If there is at least one non-pivot column, the columns
(and, therefore, the corresponding vectors) are linearly dependent.

AI For Everyone AI4E Ngày 1 tháng 8 năm 2022 11 / 17

Linear independent (cont.)

For example: check whether three vectors x1 , x2 , x3 linearly independent

     
−1 2 2
x1 = , x2 = , x3 =
1 1 3
   
−1 2 2 -1 2 2
A= →A=
1 1 3 0 3 5
Hence, x1 , x2 are linearly independent, x3 can be displayed by the
combination of x1 , x2

Why pivot columns indicate independent vectors?

AI For Everyone AI4E Ngày 1 tháng 8 năm 2022 12 / 17

Generating set and Span

Consider a vector space V = (V; +; ·) and set of vectors

A = {x1 , x2 , ..., xk } ∈ V .
I If every vector v ∈ V can be expressed as a linear combination of
x1 , x2 , ..., xk , A is called a generating set generating set of V.
I The set of all linear combinations of vectors in A is called the span
of A. If A spans the vector space V, we write
V = span[A] or V = span[x1 , x2 , ..., xk ]

AI For Everyone AI4E Ngày 1 tháng 8 năm 2022 13 / 17

Basis

Consider a vector space V = (V; +; ·) and set of vectors A ∈ V. A

generating set A of V is called minimal if there exists no smaller set
à ⊆ A ⊆ V that spans V . Every linearly independent generating set of V
is minimal and is called a basis of V.
Let V = (V; +; ·) be a vector space and B ⊆ V, B 6= ∅. Then, following
statements are equal:
I B is a basis of V
I B is a minimal generating set of V
I B is a maximal linearly independent set of vectors in V , i.e., adding
any other vector to this set will make it linearly dependent.
I Every vector x ∈ V is a linear combination of vectors from B, and
every linear combination is unique.

AI For Everyone AI4E Ngày 1 tháng 8 năm 2022 14 / 17

Basis (cont.)
There is no unique basis. However, all bases possess the same number of
elements, the basis vectors.
A basis of a subspace U = x1 , ..., xn can be found by executing the
following steps:
I Write the spanning vectors as columns of a matrix A.
I Determine the row-echelon form of A.
I The spanning vectors associated with the pivot columns are a basis
of U.
5
For example,
  for a vector  U ⊆ R spanned
 subspace   by the vectors:
1 2 3 −1
2 −1 −4 8
       
−1 , x2 =  1  , x3 =  3  , x4 = −5
x1 =        
−1 2 5 −6
−1 −2 −3 1
which vectors x1 , x2 , x3 , x4 are a basis for U

AI For Everyone AI4E Ngày 1 tháng 8 năm 2022 15 / 17

Rank

The number of linearly independent columns of a matrix A ∈ Rm×n

equals the number of linearly independent rows and is called the rank
rank of A and is denoted by rk(A).

Properties:
I rk(A) = rk(AT ), i.e., the column rank equals the row rank, hint:
rk(AT A) = rk(A)
I The columns of A ∈ Rm×n span a subspace U ⊆ Rm . A basis of U
can be found by applying Gaussian elimination to A to identify the
pivot columns.
I The rows of A ∈ Rm×n span a subspace U ⊆ Rn with dim(W) =
rk(A). A basis of W can be found by applying Gaussian elimination
to A.
I For all A ∈ Rn×n it holds that A is regular (invertible) if and only if
rk(A) = n.

AI For Everyone AI4E Ngày 1 tháng 8 năm 2022 16 / 17

Rank (cont.)

I For all A ∈ Rm×n and all b ∈ Rm it holds that the linear equation
system Ax = b can be solved if and only if rk(A) = rk(A|b), where
A|b denotes the augmented system.
I A matrix A ∈ Rm×n has full rank if its rank equals the largest
possible full rank rank for a matrix of the same dimensions. This
means that the rank of a full-rank matrix is the lesser of the number
of rows and columns, i.e., rk(A) = min(m; n). A matrix is said to be
rank deficient if it does not rank deficient have full rank.
   
1 0 0 1 −2 3
Find the rank of following matrix: 0 1 0 , 2 −3 5
1 1 0 3 5 0

AI For Everyone AI4E Ngày 1 tháng 8 năm 2022 17 / 17