Foundations of Data Science

7COM1073

Introduction to
Linear Algebra (I)
Learning Outcomes
By the end of this unit, you should know
• about vectors, how to
❖add and subtract vectors?
❖compute the scalar product?
❖compute the inner product?
❖compute the norm of a vector?
❖find a unit vector with the same direction as a given vector?
• Why do we need to use matrices?
• How to do the following basic matrix operations?
❖Matrix addition
❖Matrix multiplication
❖Diagonal and trace
Vectors – Lists of Numbers
• Lists of numbers and subscripts
➢For example: 5 students Math grades are listed as follows:
82, 90, 65, 78, 46
➢We can denote all the values in the list using only one symbol, for example, 𝑥
with different subscripts; that is
𝑥1 , 𝑥2 , 𝑥3 , 𝑥4 , 𝑥5
➢Each subscript denotes the position of the value in the list.
Such a list of values, 𝐱 = 𝑥1 , 𝑥2 , 𝑥3 , 𝑥4 , 𝑥5 is called a linear array or vector.
• Define a vector in Python:
Vectors in Physics
𝑦
Starting point is the origin
Ending point is the vector
3 (4, 3)
For examples: wind with known speed and
𝑤 direction
𝜃
(0, 0) 4 𝑥

Quantities, which can be represented by arrows having

appropriate lengths and directions and emanating from some
given reference point, for example (0,0), are called vectors.
Vectors in 𝐑 𝐝

𝐑 denotes the field of real numbers; 𝑑 denotes the number of elements in

each vector.

Consider two vectors 𝐱 = 𝑥1 , 𝑥2 , ⋯ , 𝑥𝑑 and 𝐰 = 𝑤1 , 𝑤2 , ⋯ , 𝑤𝑑 .

• Their sum is given by 𝐰 + 𝐱 = (𝑤1 +𝑥1 , 𝑤2 + 𝑥2 , ⋯ , 𝑤𝑑 + 𝑥𝑑 )

• Scalar product 𝑘𝐰 = (𝑘𝑤1 , 𝑘𝑤2 , ⋯ , 𝑘𝑤𝑑 ), where k is a real number.

• Sometimes a vector in 𝐑𝐝 is written vertically.

2
➢ For example, 1 in 𝐑𝟑
5
Example
𝑎 = 2, 4
𝑏 = 5, 1
(2,4)
𝑎 + 𝑏 =?
a a+b
𝑎 − 𝑏 =?
a+(-b)
(5,1)
(0,0) b
-b
Exercise
2 3
• 1 + 6 =?
5 1

2
• 2 × 1 =?
5
Inner Product – Algebra Definition
Consider arbitrary vectors w and x in R𝑑 , say

𝐰 = (𝑤1 , 𝑤2 , ⋯ , 𝑤𝑑 ) and 𝐱 = 𝑥1 , 𝑥2 , ⋯ , 𝑥𝑑 .

The dot product or inner product of w and x is denoted and defined by:

𝐰 ∙ 𝐱 = 𝐰, 𝐱 = 𝑤1 𝑥1 + 𝑤2 𝑥2 + ⋯ + 𝑤𝑑 𝑥𝑑
Norm
• The norm or length of a vector 𝐰 in 𝐑𝐝 , denoted by 𝐰 , is defined
to be the nonnegative square root of 𝐰 ∙ 𝐰. If 𝐰 = (𝑤1 , 𝑤2 , ⋯ , 𝑤𝑑 ),
then

𝐰 = 𝐰∙𝐰= 𝑤12 + 𝑤22 + ⋯ + 𝑤𝑑2

Example – the Distance between Two Points

𝑄𝑃 = (2 − 5)2 +(4 − 1)2 = 18 = 3 2

M(7,5)
P 𝑂𝑀 = (7 − 0)2 +(5 − 0)2 = 74

Q In general, 𝐰 = (𝑤1 , 𝑤2 , ⋯ , 𝑤𝑑 ) and 𝐱 = 𝑥1 , 𝑥2 , ⋯ , 𝑥𝑑 ,

O
𝑑 𝐰, 𝐱 = (𝑤1 − 𝑥1 )2 +(𝑤2 − 𝑥2 )2 + ⋯ (𝑤𝑑 − 𝑥𝑑 )2
Exercise
• Compute the distance between two points: 𝒖 = (1, 2, −3) and 𝒗 =
(5, 4, −1)
 𝟐
Vector Magnitude and Direction in 𝐑 ,
Suppose we have vector 𝒖 = (𝑎, 𝑏)
• Magnitude: 𝒖 = 𝒖 ∙ 𝒖 = 𝑎2 + 𝑏 2
−1 𝑏
• Direction: θ = tan
𝑎
• Components from magnitude and direction:
(𝑎 = 𝒖 cos(θ), 𝑏 = 𝒖 sin(θ) )

Credit:
https://www.khanacademy.org/math/precalculus/x9e81a4f98389efdf:v
ectors/x9e81a4f98389efdf:component-form/a/vector-magnitude-and-
direction-review
Inner Product – Geometric Definition
The geometric definition:
𝐰 ∙ 𝐱 = w x cos 𝜽 ,
where 𝜃 is the angle between 𝐰 and 𝐱.

𝐰∙𝐱= w x cos 𝜽 = 𝑤1 𝑥1 + 𝑤2 𝑥2 + ⋯ + 𝑤𝑑 𝑥𝑑

If 𝐰 ∙ 𝐱 = 0, the vectors 𝐰 and 𝐱 are said to be orthogonal (or

perpendicular).
Unit Vector
• A vector of length 1.
• For any nonzero vector 𝒖 in 𝐑𝐝 , the vector 𝑢ො
𝒖
𝑢ො =
𝒖
is the unit vector in the same direction as 𝒖. The process of finding 𝑢ො
from 𝒖 is called normalising 𝒖.
Hyperplanes
• A Hyperplane H in R is a dot, a Hyperplane H in R2 is a line and a
Hyperplane H in R3 is a plane. A point 𝐱 = (𝑥1 , 𝑥2 , ⋯ , 𝑥𝑑 ), on a
Hyperplane H in R𝑑 satisfies a linear equation

𝑤1 𝑥1 + 𝑤2 𝑥2 + ⋯ + 𝑤𝑑 𝑥𝑑 + 𝑏 = 0
where the vector of coefficients 𝐰 = (𝑤1 , 𝑤2 , ⋯ , 𝑤𝑑 ) is not zero.

• The vector 𝐰 determines the direction of the hyperplane; the value of

𝑏 determines the position of the hyperplane (for example, this value
determines the intersection of the line with the vertical axis in R2 ).
Matrix
• A rectangular arrangement of numbers
➢ made up of rows and columns
➢ a matrix with m rows and n columns is called an m x n matrix
➢ a matrix with the same number of rows and columns is called square
➢ a matrix is usually labelled with a (bold) capital letter
➢ the whole thing is written inside square brackets
➢ the plural of matrix is matrices

• Each element in a matrix (𝑴) is identified by two indices.

𝑀1,1 𝑀1,2
𝑴=
𝑀2,1 𝑀2,2
Matrix
• For example:
➢5 students Maths grades are listed as follows:
82, 90, 65, 78, 46
➢ Their corresponding English grades are listed as follows:
76, 78, 60, 50, 60
➢Again we can denote all the values using only one symbol, for example, 𝑥 with
two subscripts: one for Maths and English; one for different students, that is
𝑥11 , 𝑥12 , 𝑥21 , 𝑥22 , 𝑥31 , 𝑥32 , 𝑥41 , 𝑥42 , 𝑥51 , 𝑥52
Matrix
• The matrix looks like:
82 76
90 78
65 60
78 50
46 60

• Define a matrix in Python

Matrix Addition
• The sum of two matrices 𝑀 and 𝑁, where 𝑀 and 𝑁 must be with the
same size, is the matrix obtained by adding corresponding elements
from 𝑀 and 𝑁.
Matrix Addition -Example

2 4 1 4 2+1 4+4
+ =
0 1 1 0 0+1 1+0
2 4 1 4 3 8
+ =
0 1 1 0 1 1
Scalar Multiplication
• The product of the matrix 𝑀 by a scalar 𝑘, written 𝑘 ∙ 𝑀 or simply
𝑘𝑀, is the matrix obtained by multiplying each element of 𝑀 by 𝑘.
Scalar Multiplication - Example
1 2 5×1 5×2 5 10
• 5× = =
3 5 5×3 5×5 15 25
Exercise
−3 10 −1 2
Let 𝑢 = 9 0.6 , 𝑣= 1 0
1 −5 0 1

Find:
• 2𝑢 − 4𝑣
• −3𝑢 + 2𝑣
Matrix Multiplication
• The product of two matrices 𝐴 and 𝐵 written as 𝐴𝐵, is somewhat
complicated.
The product of a row matrix [𝑎𝑖 ] and
a column matrix [𝑏𝑗 ] with the same
number of elements is defined to be
the scalar obtained by multiplying
corresponding entries and adding.
𝑏11 𝑏12 𝑏13
𝑎11 𝑎12 𝑎13 𝑎14 𝑏21 𝑏22 𝑏23 𝑐11 𝑐12 𝑐13
𝑐12 𝑎23 𝑎24 = 𝑐 𝑐22 𝑐23
𝑎21 𝑎22 𝑏31 𝑏32 𝑏33 21
𝑏41 𝑏42 𝑏43

𝑐12 = 𝑎11 × 𝑏12 + 𝑎12 × 𝑏22 + 𝑎13 × 𝑏32 + 𝑎14 × 𝑏42

𝐶
Credit: egret.psychol.cam.ac.uk
Matrix Multiplication

1 2
2 3 0 𝟐 × 𝟏 + 𝟑 × 𝟕 + 𝟎 × 𝟎 = 𝟐𝟑
7 3 =
1 4 5
0 5
Matrix Multiplication

1 2
2 3 0 𝟐𝟑 𝟐 × 𝟐 + 𝟑 × 𝟑 + 𝟎 × 𝟓 = 𝟏𝟑
7 3 =
1 4 5
0 5
Matrix Multiplication

1 2
2 3 0 𝟐𝟑 𝟏𝟑
7 3 =
1 4 5 𝟏 × 𝟏 + 𝟒 × 𝟕 + 𝟓 × 𝟎 = 𝟐𝟗
0 5
Matrix Multiplication

1 2
2 3 0 𝟐𝟑 𝟏𝟑
7 3 =
1 4 5 𝟐𝟗 𝟏 × 𝟐 + 𝟒 × 𝟑 + 𝟓 × 𝟓 = 𝟑𝟗
0 5
Matrix Multiplication
1 2
2 3 0 𝟐𝟑 𝟏𝟑
7 3 =
1 4 5 𝟐𝟗 𝟑𝟗
0 5
Note: To calculate the multiplication of two matrices, the number
of columns of the first matrix must be equal to the number of
rows of the second matrix.

1 0 2 6 8
3 7 4 5 0
4 2 3 6 4
Exercises

10 3 2 10 7 0
• + =?
1 2 5 16 6 9

3 2
• 3 0 1 =?
5 −1

2 2
10 3 2
• 1 8 =?
1 2 5
2 5
Matrices as Linear Transformations

𝑃 = 1,3 , 𝑄 = 2,3 , 𝑅 = (1,1)

𝑃′ = −3,1 , 𝑄′ = −3,2 , 𝑅′ = −1,1

Matrices as Linear Transformations
• Example – Rotation around the
origin
Rotate vector 𝑢 = [10,0]
cos(𝜃) −sin(𝜃) 45 degree in the anticlockwise direction around the
𝑅= origin.
sin(𝜃) cos(𝜃)
cos(45°) −sin(45°) 10 5.3
𝑅𝑜𝑡𝑎𝑡𝑒𝑑𝑢 = 𝑅𝑢′ = =
sin(45°) cos(45°) 0 8.5
Affine Transformation
• An affine transformation preserves lines or distance ratios but
change the orientation, size or position of the object.

credit:
https://pythonhosted.org/planar/transfor
ms.html

If you want to learn more on how to use Python to do this, you may find the following page is of help:
https://stackabuse.com/affine-image-transformations-in-python-with-numpy-pillow-and-opencv/
Example (credit to Nichola Schulze for slides 34-40)

Why do we multiply matrices in this way?

Let's look at a simple example.

Example
1
50%

A 20% Let's say we have two gas

producers (A and B) that send
30% 2 different proportions of the
gas they produce to three
40%
suppliers (1, 2 and 3)
B
60%
3

1 2 3
A 0.5 0.2 0.3 We can model this as a
B 0.0 0.4 0.6 matrix.
Example
40%
X
1
50% 60%

A 20% Y
70% Those gas suppliers then
30% 2
30% forward gas in different
proportions to four gas
40%
B Z stations (X,Y,Z and W)
60% 50%
3
50%
W

1 2 3 X Y Z W
A 0.5 0.2 0.3 1 0.4 0.6 0.0 0.0 We can model this as a
B 0.0 0.4 0.6 2 0.0 0.7 0.3 0.0 matrix as well
3 0.0 0.5 0.0 0.5
Example
40%
X
1
50% 60%

A 20% Y
70%
30% 2
30%
40%
B Z
60% 50%
3
50%
W

Notice that the number of

1 2 3 X Y Z W columns in the first matrix is
A 0.5 0.2 0.3 1 0.4 0.6 0.0 0.0
equal to the number of rows in
B 0.0 0.4 0.6 2 0.0 0.7 0.3 0.0
the second – so the product
3 0.0 0.5 0.0 0.5
exists
Example
40%
X
1
50% 60%

A 20% Y
70%
30% 2
30%
40%
B Z
60% 50%
3
50%
Here is the product, but
W
what does it represent?

1 2 3 X Y Z W X Y Z W
A 0.5 0.2 0.3 1 0.4 0.6 0.0 0.0 A 0.20 0.59 0.06 0.15
B 0.0 0.4 0.6 2 0.0 0.7 0.3 0.0
B 0.00 0.58 0.12 0.30
3 0.0 0.5 0.0 0.5
Example
X
20%

59%
A 6%
Y
58%
15% In this case it represents
the simplified situation
where the two producers
12%
B Z send gas directly to the
stations
30%

1 2 3 X Y Z W X Y Z W
A 0.5 0.2 0.3 1 0.4 0.6 0.0 0.0 A 0.20 0.59 0.06 0.15
B 0.0 0.4 0.6 2 0.0 0.7 0.3 0.0
B 0.00 0.58 0.12 0.30
3 0.0 0.5 0.0 0.5
Multiplying A Matrix by Itself
• Only square matrices can be multiplied by themselves
• The number of columns must equal the number of rows
• A2 = AA
• A3 = AAA
• etc
• As you'd expect – matrix multiplication applies
Square Matrices
• A square matrix is a matrix with the same number of rows as
columns.
• For examples:
1 5 0 10 1 2
2 4 10 8 3 5
−1 2 3 6 0 1
Diagonal and Trace
𝑎11 ⋯ 𝑎1𝑛
•𝐴= ⋮ ⋱ ⋮
𝑎𝑛1 ⋯ 𝑎𝑛𝑛

• The diagonal of a matrix consists of the elements with the same

subscripts, that is, 𝑎11 , 𝑎22 , 𝑎33 , 𝑎44 , ⋯ , 𝑎𝑛𝑛

• The trace of a square matrix is the sum of the diagonal elements:

𝑡𝑟 𝐴 = 𝑎11 + 𝑎22 + 𝑎33 + 𝑎44 + ⋯ + 𝑎𝑛𝑛
Exercise
1 5 0 10 1 2
Given A = 2 4 10 , 𝐵= 8 3 5
−1 2 3 6 0 1

Compute:
• 𝑡𝑟 𝐴 =? ,
• 𝑡𝑟 𝐵 =?
Exercise
1 5 0
Given A = 2 4 10
−1 2 3

Python code:
Diagonal Matrices
A diagonal matrix is a matrix in which the entries
outside the main diagonal are all zeros.

• Rectangular diagonal matrix

• Symmetric diagonal matrices

For example,
1 0 0 0 1 0 0
0 3 0 0 , 0 3 0
0 0 9 0 0 0 9