A linear transformation = linear mapping

f : V  W transforms a linear combination of vectors

in V into a linear combination of vectors in W.

Note that V ,W may not be of same dimension.

Linear transformation means if v1 , v2  V then

a. f (v1  v2 )  f (v1 )  f (v2 ) and
b. f  v1  f (v1 ) where  is a scalar
with f (v)  W .

Therefore, a generalization from these two rules is

the following: if v1 , v2  V then
f (v1  v2 )  f (v1 )  f (v2 )  w1  w2

where w1 , w2  W

example:

V is the space of 2  2 matrices and W is the space

a b  a
of 2  1 matrices. f :  c d    b  is a linear mapping or
a linear transformation.

Verification:
 a b   e f  ae b f   ae
f :     f   
 c d   g h  c  g d  h  b  f 
a b  e f  a  e   a  e 
Also, f  f 
c d  g
       
h   b  f  b  f 
Similarly, you can show that
  a b  a b 
f      f  
  c d  c d 

The following expansion comes naturally:

a. the zero vector in V maps to the zero vector

in W
b. f ( x)   f ( x)
c. For any set of scalars  i  and vectors  vi  V  ,
f (1v1   2 v2  ...   n vn )  1 f (v1 )  ..   n f (vn )

d. Let Y  V . Then f :Y  Z  W . A subspace maps to a

subspace.

example. Suppose f : 3  3 defined by

f ( a , b, c )  ( a  b, b  c , c  a )

This you can show to be a linear mapping. One

subspace of 3 is the space Y spanned by all
vectors of the form (0, b, c) . Now, for any
(0, a, b)  Y f (0, a, b)  (a, a  b, b)
 All such vectors (a, a  b, b) form a subset. Note that
the transformation is linear:

( a , a  b , b )  ( c, c  d , d )  ( a  c, a  b  c  d , b  d ) and
 ( a , a  b, b )  (  a ,  a   b,  b )

Also, ( f  g )(v1  v2 ) = ( g  f )(v1  v2 )

( f  g )(v1  v2 ) = f  ( g )(v1  v2 )

 f (g (v1 )   g (v2 ))
 fg (v1 )  fg (v2 )

Next consider the following situation of mapping

from one dimension to another.

Consider a linear mapping from 3 to a 2 2

matrix.
a a  b
f (a, b, c )   
b a  c

Then a set of basis vectors gets transformed into

the following matrices:
1 1 0 1
f (1,0,0)  
0

1
f (0,1,0)  
1

0 and
0 0
f (0,0,1)   
0 1
From these we can get the general rule for any
vector (a, b, c) :

f (a, b, c )  af (1,0,0)  bf (0,1,0)  cf (0,0,1)

 1 1 0 1 0 0
 a   b   c 
 0 1 1 0 0 1

a a  b
 
b a  c

That is, even the vectors in this space are subject

to the same linear transformation. This shows that
a linear mapping f :    is possible. n m

Suppose, the vector space V is spanned by an

orthonormal basis set {vi }and W is spanned by an
orthornormal basis set {wi } .

Any vector v  V and w  W can be expanded in their

respective base as:
n m
v   a i vi
i 1
and w   b jv j
j 1

Now if f is a linear transformation f :V  W then

n n m
f (v)  f  ai vi   ai f (vi )   b j w j
i 1 i 1 j 1
 m
This is possible only when each f (vi )   c ji w j
j 1
.
n n m
Check.  ai f (vi )   ai  c ji w j
i 1 i 1 j 1

n m
But  ai c ji
i 1
is some bj . Therefore, f (v )   b j w j
j 1

The crucial issue is that in a linear transformation

the basis is transformed under the transformation
m
f (vi )   c ji w j .
j 1

Obviously, we can generate the converse situation

too.

Proposition: If  : n  m is a linear mapping, then

there exists a unique m n matrix A such that
 ( x )  Ax , x   n
From the matrix framework

Example. Let V be a polynomial vector space.

The basis vectors in this space are:

For V: p1 ( x)  1 p2 ( x)  x, p3 ( x)  x 2 , p4 ( x)  x 3
d
The function f  dx transforms vectors as
d d
f :V  V . p1 ( x)  0, p2 ( x )  1  p1 ( x )
dx dx
d d
p3 ( x)  2 x  2 p2 ( x), p 4 ( x )  3 x 2  3 p3 ( x )
dx dx

So, in this basis the matrix of f is

0 1 0 0
0 0 2 0
 
0 0 0 3
0 0 0 0


Question: What is the mapping transformation f

that maps from 3 to  4 as

f : (a, b, c)  (a, a  b, b  c, a  b  c)

Consider the basis vectors in 3 . How are they

transformed?

f (1,0,0)  (1,1,0,1)  1(1,0,0,0)  1(0,1,0,0)  1(0,0,0,1)

f (0,1,0)  (0,1,1,1)  1(0,1,0,0)  1(0,0,1,0)  1(0,0,0,1) and
f (0,0,1)  (0,0,1,1)  1(0,0,1,0)  1(0,0,0,1)

so the transformation matrix f is the transpose of

the above coefficient matrix.
1 1 0 1
f  0 1 1 1
 
0 0 1 1

 a 
a   ab 
b 
Thus the vector   maps into 
 bc 

as
c 
  a  b  c 
 

 a  1 0 0
 ab  1 a 
1 0  

 bc 

= 
0 1
 b
1  
a  b  c  1  c 
   1 1 

Example. A mapping f : ( a, b)  (a  3b,5b) is thus the

matrix transformation
 a  3b   1  3  a 
   
 5b   0 5  b 

Show that this is linear i.e. show that

f ( a , b )  f (c  d )  f ( a  c, b  d ) and
f ( a ,  b )   f ( a , b )

http://turnbull.mcs.st-and.ac.uk/~sophieh/LinearAlg/SHlintran.pdf
More on mapping.

A mapping f : U  V is one-to-one (1-1 or injective) if

different elements of elements of U gets mapped to
different elements of V .

U
V

A mapping f : U  V is onto (or surjective) if every

element v  V is a mapping of one or more element
in U .

U
V

A mapping f : U  V is bijective if it is both onto and

one-to-one at the same time. This amounts to an
invertible function.
Example. Suppose a neural circuit outputs an
index for a pattern Pi  P . The neural network is
trained to output something. A NN machine is,
therefore, an one-to-one pattern recognizer.

Output Layer
Pattern NN
System

A hashing function where many addresses are

hashed onto the same hashed address is an
example of onto mapping. A cluster is another
example.

A function is an invertible mapping. Every person

has a unique finger-print and behind every finger-
print there’s a unique person.
 We do your
Finger-printing
here.

Identity mapping.

Example. f :    . f ( x)  2e is an one-to-one
x

mapping. x   , y  f (x) . But this is not onto. We

don’t have the situation that y  , x .

where g ( x)  2 x  3 is a bijective mapping. For

g :  

every x   , there is a unique y  

and for every y   there is a unique x   .
 where
h:   h( x )  x  rand () is neither an one-to-one
nor an onto.

Vector space isomorphism. Two vector spaces U

and V are isomorphic if (a) they are defined over a
common field K, and (b) if there is a bijection that
maps U into V (and naturally, from V to U ). An
isomorphism is a bijective linear mapping i.e. it is
both one-to-one and onto. Every isomorphism
f :    is a mapping that admits its inverse
m m

f :   .
1 m m

Kernel and Image of a linear mapping.

Given f :U  V , Ker ( f )  U such that Ker ( f )  {u  U , f (u )  0}

Kernel

Zero

Image or range of the mapping f is the set of

points in V to which points in U are mapped on.
Im( f )  V .
Proposition. If {ui , i  1,..., n} span a vector space U as
its basis and there is a linear mapping f : U  V , then
{vi  f (ui )} span Im f .

Kernel and the Image of a matrix mapping.

Suppose a linear mapping f :    . Suppose the

3 2

 2  3 4
mapping is equivalent to a matrix A  1 2 8 and a
 x1 
vector x   x2  maps to the vector y where
 
 x3 

 x1 
2  3 4   2 x1  3 x2  4 x3 
y  x2   
1 2 8     x1  2 x2  8 x3 
 x3 

Fair enough! Therefore, a typical orthogonal basis

set in 3 {e1 , e2 , e3 } are changed to the following
vectors in  2 :
1 
2 3 4    2
Ae1   0 
1 2 8    1 
0
0 
2  3 4    3
Ae2   1 
1 2 8     2 
0
 4
and Ae3   
8 
Indeed, these are all columns of the
transformation matrix A .

Thus, if Amn is a transformation matrix that yields

A : K n  K m (where K is either real or a complex field)

then the image of A, Im(A), is the set of columns of

A.

Rank and Nullity of a linear mapping.

Rank of a matrix is a measure of its degree of

independence.

Suppose Amn is an m n matrix {a }. The rowspace of ij

A is the subspace of  n spanned by the rows

( a11 , a12 , a13 ,..., a1n ), ( a21 , a22 , a23 ,..., a2n ) , …
( a m1 , a m 2 , a m3 ,..., a mn ) . We call this space rowspace(A) and

its dimension as row-rank(A).

2 3 4
Suppose A   1 2 3 . The rowspace of A is
 
 0 7 2

the subspace of 3 spanned by the vectors

u1  (2 3 4), u 2  (1 - 2 3) and u3  (0 - 7 2)

but u3  u1  2u2 . Therefore,

rowspace( A)  {u , u 2 ,u1  2u 2 }  {u1 , u 2 }

Since the vectors u1 and u 2 are linearly independent
(check this!), they form a basis of the space
rowspace(A) so the row rank of A is the dimension
of this basis dim(Rowspace(A))=2.

Similarly, for the column rank of a matrix. The

column space of A is the set of columns of A
denoted by colspace(A). The dimension of column
space colspace() is the column rank of the matrix.
We would see that

row rank(A) = column rank(A)

Proposition: Consider an Amn matrix. Suppose,

matrices Pmm and Qnn are invertible matrix. Then
Row rank ( A)  Row rank (PA )
= Row rank ( AQ)

and
Colrank ( A)  Colrank (PA )  Colrank ( AQ)

For 1  i  m let ui  (ai1 , ai 2 ,..., aim ) , the ith row of A. Thus,

rank ( A)  dim(U ) where U  u1 , u 2 ,..., u m 

Now, suppose PA  B  (b ) and let

ij vi be the ith row of
B i.e vi  (bi1 , bi 2 ,..., bin ) .
rank ( PA )  dim(V ) , where V  v1 , v2 ,..., vn 

Now bij  k pik akj . Therefore,

vi  ( pik ak1 ,  pik ak 2 ,...,  pik akn )
k k k
  pik (ak1 , ak 2 ,..., akn )   pik u k
k k

Thus, each vector vi is a linear combination of the

vectors u1 , u 2 ,..., un . Therefore, V  U . Therefore,

Rowsapce (PA )  Rowspace ( A)

This holds for any invertible matrix P and any

matrix A . Therefore,

Rowsapce ( P 1 ( PA ))  Rowspace (PA )  Rowspace ( A)

That is, Rowsapce (PA )  Rowsapce ( A) . Therefore,

their dimensions must be the same. But,
rank ( A)  dim( Rowspace( A)) . Hence,

rank ( A)  rank ( PA )

Similarly, for the other part of the observation.

Example. Find the rank of A where

 2 1 1 2
A   1 2 0  1
 
 6 6 6 6 

v1  ( 2,1,1,2) and v2  (1,2,0,1) . Now,

(v1  v2 )  (1,1,1,1) . Therefore, v3  6(v1  v2 ) . Therefore,
v3  v1 , v2  . Therefore, the matrix has a rank of

two.

Revisit to Ax  b .

LU decomposition. Our problem is to solve the

matrix equation Ax  b where A is a matrix and

x and are column vectors. We propose that the

b
matrix A is decomposable to product of two
triangular matrices L and U .

A  LU where

U
The decomposition is not unique. Suppose there is
a matrix E such that

L'  LE and U '  E 1U . Then

L'U '  LU  A

Therefore we may even choose the diagonal

elements of L to be 1, i.e. lii  1, i . Given this, the
problem Ax  b becomes LUx  b . This is equivalent
to two equations:

Solve Ly  b first and then solve Ux  y .

Since Ly  b

L11 y1  b1  y1  b1 / L11
L21 y1  L22 y 2  b2  y 2  (b2  L21 y1 ) / L22

….

This way we get the y vector. After that we solve

the equation

Ux  y

We may need to do pivoting if the raw matrix A

has weaker diagonal elements compared to the
rest of the elements in their rows. In that case, we
factorize not A but PA . That is,

PA  LU where P is a permutation matrix.

Cholesky decomposition

If a matrix A is symmetric and positive definite

then we can even do better than an LU
factorization.

A matrix A is positive definite if for any non-zero

x , we get

x * Ax  0 where x* = xt a complex-conjugate vector.

Cholesky decomposition suggests that in that case

A  LLt (this will make A symmetric).

This means Ax  b is LLt x  b . This suggests the

approach

First, let Ly  b and then Lt x  y yields x .

In this case,
  a11 a12 .. a1n 
a a 22 .. a2 n 
A   12 
=
 .. .. .. .. 
a a2n .. a nn 
 1n

 l11 0 0 0  l11 l 21 .. l n1 
l l 22 0 0  0 l 22 .. l n 2 
 21  
 .. .. .. ..   0 .. .. .. 
l ln 2 .. l nn   0 0 0 l nn 
 n1
This gives us the matrix coefficients.
i 1  i 1 
lii  ( aii   lik2 )
k 1
and l ji   a ji   l jk lik  / lii
 k 1 