LİNEAR ALGEBRA

References: Elementary Linear Algebra with Applications, Bernard Kolman, David Hill, Ninth
Edition, Pearson New International Edition

Course Content:
Matrix Algebra
Determinants
System of Linear Equations
Vector Spaces
Inner Product Spaces
Ortogonal Basis
Linear Transformations
Eigenvalues and eigenvectors
Quadratic Forms

INTRODUCTION

Linear algebra is the branch of mathematics concerning linear equations such as:
c1 x1  c2 x2  ...  ck xk  b

and linear maps such as:

 x1 , x2 ,..., xn   c1 x1  c2 x2  ...  ck xk
and their representations in vector spaces and through matrices. Linear algebra is central to
almost all areas of mathematics. For instance, linear algebra is fundamental in modern
presentations of geometry, including for defining basic objects such
as lines, planes and rotations.
Linear algebra is also used in most sciences and fields of engineering, because it
allows modelling many natural phenomena, and computing efficiently with such models.
For nonlinear systems, which cannot be modelled with linear algebra, it is often used for dealing
with first-order approximations, using the fact that the differential of a multivariate function at
a point is the linear map that best approximates the function near that point.

1
Some Applications of Linear Algebra

There are many applications of Linear Algebra. The following list highlights a few of them.
Constructing Curves and surfaces passing through Specified points
Example: Find the equation of the circle passing through the three points A = (1,4) , B=(-2,3) ,
C= (7,1).

Cryptography
Cryptography is the study of encoding and decoding secret messages. One of the simpler
encoding methods uses matrix transformations and modular arithmetic to encode/decode
messages. These codes are called Linear Codes: they rely on a generator matrix and a parity
check matrix.
Game Theory in Economics
Game Theory is used to model 2-player games requiring the same decision to be made at each
step. Furthermore, these decisions may result in payoffs or penalties for each player at every
step. In such situations, it is often possible for each player to formulate a strategy which will
maximize their return. Linear Algebra can be used to describe this situation.
Equilibrium Temperature distributions used in Physics and Chemistry
The problem of how the heat will eventually be distributed across a thin metal plate, given the
temperature at the edge of the plate can be reduced to solving a system of linear equations.
Electrical Networks (or more generally in Graph Theory)
Electrical networks are used by electrical engineers to model circuits.

2
Computed tomography
Constructing a cross-sectional view of a human body by analyzing X-ray scans leads to an
inconsistent linear system of equations
Computer Graphics
For example, linear transformations and coordinate matrices are used to draw objects from
different views.
Fractals
Fractals, those cool sets of points that look the same when you zoom in, no matter how much,
are generated and described by certain classes of transformations, some of which are linear
transformations. Besides looking cool on posters, fractals are used in computer graphics
(games!), since they replicate the look of ferns, trees or coastal lines.
Leontief Economic models
Matrix theory has been very successful in describing interrelations among prices, outputs, and
demands in economic systems.
Genetics
Genetics is the study of inheritance. Linear Algebra can be used to look at the ideas behind
recessive and dominant traits and determine how a trait will be distributed in future generations.
Anything involving differential equations or data fitting.

3
1. MATRICES
1.1. Basic Definitions and Operations

1.1. Definition A matrix is a rectangular arrangement of numbers into rows and columns.
For example, let A be a matrix having m rows and n columns. Then, A is said to have order
m  n (m by n) or is called a matrix of size m  n and can be represented in this form.

 a11 a12  a1n 

a a22  a2 n 
A   aij    21
    
 
 am1 am 2  amn 

Example: What is the dimension of matrix E?

2 6 7 0 
E 
 2 1 4 7 
1.2. Definition A matrix element is simply a matrix entry. Each element in a matrix is identified
by naming the row and column in which it appears

For example

 2 4 10 
G  5 2 2 
 
1 8 4 

The element g 21  5

1.3. Definition A matrix in which each entry is zero is called a zero matrix, denoted 0. For
example

0 0 0 0 0
022    023   
0 0 0 0 0
1.4. Definition A matrix that has the same number of rows as the number of columns, is called
a square matrix. A squared matrix is said to have order n if it is an n  n matrix.
1.5. Definition Let A   aij  be a square matrix of order n. Then

4
 a) The entries a11 , a22 ,..., ann are called the diagonal entries the principal diagonal of A
b) A is said to be a diagonal matrix if aij  0 for i  j denoted by diag  a11 , a22 ,..., ann  .
4 0
Example 0n  diag  0, 0,..., 0  ,    diag  4,1
0 1 
c) If A  diag  a11 , a22 ,..., ann  and aij  d for all 𝑖 = 1,2, … , 𝑛 then the diagonal matrix is
called a scalar matrix.
d) A  diag 1,1,..,1 is called the identity matrix and is denoted by I n . Example,
1 0 
I2   
0 1 
0 4 1
e) A is said to be an upper triangular matrix if aij  0 for i  j . Example 0 3 2

0 0 1 
0 0 0
f) A is said to be a lower triangular matrix if aij  0 for i  j . Example  2 3 0

 0 2 1 
g) An n  1 matrix is also called an n-vector and is denoted by lowercase boldface letters.
When n is understood we refer to n-vectors merely as vectors.
1
2  3
Example u    is a 4-vector, v  0  is a 3-vector.
 1
  1 
0
1.6. Definition Equality of two matrices. Two matrices A   aij  and B  bij  having the
same order 𝑚 × 𝑛 are equal if and only if aij  bij for each 𝑖 = 1,2. . , 𝑚 and 𝑗 = 1,2, … , 𝑛.

Example. The matrices

x7
1 2 7  1 2 x
y6
A   4 6 2  and B   w y
  2  are equal if and only if
z 1
 5 4 1   5 4 z 
w4

1.2. Vectors in the Plane and in 3-Space

In many applications we deal with measurable quantities such as pressure, mass and speed,
which can be completely described by giving their magnitude. They are called scalars and will
be denoted by lowercase italic letters such as c,d,r,s and t. there are many other measurable
quantities such as velocity, force and acceleration, which require for their description not only
magnitude, but also a sense of direction. These are called vectors, and they will be denoted by
lowercase boldface letters such as u, v, w, x, y, z

5
Vectors in Plane

With each point on the plane we associate an ordered pair (𝑥, 𝑦) of real numbers, its coordinates.
Conversely, we can associate a point in the plane with each ordered pair of real numbers. Point
P with coordinates (𝑥, 𝑦) is denoted by P(𝑥, 𝑦) or simply by (𝑥, 𝑦). The set of all points in the
plane is denoted by ℝ , it is called 2-space.
𝑥
Consider the 2 × 1 matrix 𝐮 = 𝑦 where x and y are real numbers. With 𝐮 we associate the
directed line segment with the initial point the origin and terminal point 𝑃(𝑥, 𝑦).

The directed line segment from O to P is denoted by 𝑂𝑃⃗; O is called its tail and P its head. A
directed line segment has a direction indicated by the arrow at its head.
The magnitude of a directed line segment is its length.

Conversely, with the directed line segment 𝑂𝑃⃗ with tail O(0,0) and head 𝑃(𝑥, 𝑦) we can
𝑥
associate the matrix 𝐮 = 𝑦

𝑥
1.7. Definition A vector in the plane is a 2 × 1 matrix u = 𝑦 where x and y are real numbers,
called the components (or entries) of u. We refer to a vector in the plane merely as a vector
or as a 2-vector.
Since a vector is a matrix, the vectors
𝑥 𝑥′
u= 𝑦 and u′ =
𝑦′
are said to be equal if and only if 𝑥 = 𝑥 and 𝑦 = 𝑦 .

Frequently, in physical applications it is necessary to deal with a directed line segment 𝑃𝑄⃗ from
the point 𝑃(𝑥, 𝑦) (not the origin) to the point Q(𝑥′, 𝑦′). Such a directed line segment will also
be called a vector in the plane, or simply a vector with tail 𝑃(𝑥, 𝑦) and head 𝑄(𝑥′, 𝑦′). The
components of such a vector are 𝑥 − 𝑥 and 𝑦 − 𝑦. Thus the vector 𝑃𝑄⃗ can also be represented
by the vector
𝑥 − 𝑥′
𝑦 − 𝑦′

Consider the vectors 𝑃𝑄⃗ , 𝑃′𝑄′⃗, 𝑃′′𝑄′′⃗ joining the points 𝑃(3,2) and 𝑄(5,5), 𝑃’(0,0) and
𝑄’(2,3), 𝑃’’(−3,1) and 𝑄’’’(−1,4), respectively. Since they all have the same components, they
are equal.

6
Vectors in Space

 x
A vector in space or 3-vector, is a 3 × 1 matrix u   y  where x, y, z are real numbers, called
 z 
the components of vector u

A vector in n-dimensional space is an 𝑛 × 1, matrix or n-vector

𝑥
𝑥
𝐮= ⋮
𝑥

where 𝑥 , 𝑥 ,…, 𝑥 are real numbers called the components of vector 𝐮.

With each n-vector we can associate a point (𝑥 , 𝑥 ,…, 𝑥 ) in n-dimensional space. The set of all these
points is denoted by ℝ .

1.3. Matrix Operations

We next define a number of operations that will produce new matrices out of the given matrix.

Matrix Addition

1.8. Definition If A   aij  and B  bij  are both 𝑚 × 𝑛 matrices then the sum 𝐴 + 𝐵 is an
𝑚 × 𝑛 matrix C  cij  defined by cij  aij  bij i  1, 2,..., m , j  1, 2,..., n . Thus to obtain
the sum of A and B, we merely add corresponding entries.
Example

1 2 3 0 2 1  1 0 4 
Let A    and B    . Then A  B   
2 5 1  5 3 2   7 8 3

7
Vector addition is a special case of matrix addition. We can interpret vector addition
geometrically as follows

u  v  u  u 
Let u   1  and v   1  be two vectors. Then u + v   1 2 
u 2  v2   v1  v2 

Scalar Multiplication

1.9. Definition If A   aij  is an 𝑚 × 𝑛 matrix and r is a real number, then the scalar multiple
of 𝐴 by r, 𝑟𝐴, is the 𝑚 × 𝑛 matrix C  cij  , where cij  raij , i  1, 2,..., m , j  1, 2,..., n .
That is the matrix C is obtained by multiplying each entry of 𝐴 by 𝑟
Example

3 4 3   6 8 6
2 
  
1 5 1  2 10 2 

u 
If u   1  is a vector and c is a scalar, then the scalar multiple cu of u by c is the vector of a
u 2 
 cu 
vector  1  . Thus the scalar multiple cu is obtained by multiplying each component of u by
cu2 
c. If 𝑐 > 0 then cu is in the same direction as u, whereas if 𝑑 < 0, then du is in the opposite
direction .

8
If A and B are two matrices, we write 𝐴 + (−𝐵) as 𝐴 − 𝐵 and call this the difference between
𝐴 and 𝐵.

If A1 , A2 ,..., Ak are 𝑚 × 𝑛 matrices and c1 , c2 ,..., ck are real numbers, then an expression of the
form
c1 A1  c2 A2  ...  ck Ak

Is called a linear combination of A1 , A2 ,..., Ak and c1 , c2 ,..., ck are called coefficients. We may
use the summation notation as
k

c A
i 1
k k  c1 A1  c2 A2  ...  ck Ak

T
1.10. Definition If A   aij  is an 𝑚 × 𝑛 matrix, then the transpose of A, AT   aij  is the
𝑛 × 𝑚 matrix defined by aijT  a ji . Thus the transpose of A is obtained from A by
interchanging the rows and columns of A.
Example

 1 2
1 2 3
A  A   2 5 
T
 
2 5 1   3 1 

9
Matrix Multiplication

1.11. Definition The dot product, or linear product of the n-vectors in R n

 a1   b1 
a  b 
a  2
and b   2 
 
   
 an  bn 
n
is defined as a  b  a1b1  a2b2  ...  anbn   ai bi
i 1

1 1
3 3
Example: The dot product of u    and v    is
 4  2
   
5  5 
u  v  11  3  3  (4)  2  5  (5)  1  9  8  25  27

1.12. Definition If A   aij  is an 𝑚 × 𝑝 and B  bij  is an 𝑝 × 𝑛 matrix, then the dot

product of A and B, denoted AB, is the 𝑚 × 𝑛 matrix C  cij  , defined by
p
cij  ai1b1 j  ai 2b2 j  ...  aip bpj   aik bkj
k 1

Equation (1) says that the (i, j)-th element int the product matrix is the dot product of the
transpose of the I-th row, rowi  A  - that is, rowi  A  - and the j-th column, col j  B  , of B:
T

this is shown in the figure below

Observe that the product of A and B is defined only when the number of rows of B is exactly
the same as the number of columns of A.

10
Example: Let

 2 5 
1 2 1
A  and B   4 3
3 1 4   2 1 

 4 2 
Then A  B   
6 16 
Example: Let

 1 2 3  1 4 
A  4 2 1  B   3 1 
   
 0 1 2   2 2 

Compute the (3,2) entry of 𝐴 ∙ 𝐵

Solution:

 row  A    col2  B  
T
3

I f A is an 𝑚 × 𝑝 matrix and B is a 𝑝 × 𝑛 matrix, then AB is an 𝑚 × 𝑛 matrix. What

about BA? Four different situations may occur:
1. BA may not be defined: this will take place when n  m
2. If BA is defined, which means that m=n, then BA is p  p while AB is m  m ; thus
if m  p , AB and BA are of different sizes.
3. If AB and BA are both of the same size, they may be equal
4. If AB and BA are both of the same size, they may be unequal

Example 1) If A is a 2  3 and B is a 3  4 matrix, then AB is a 2  4 matrix and

BA is undefined
 1 2  2 1
Example 2) A    and B   0 1 , then AB=? and BA=?
 1 3   
Remark. If u and v are n-vectors (𝑛 × 1 matrices) then it is easy to show by matrix
multiplication that

u  v  uT v

11
 1.4. The Matrix-Vector Product Written in Terms of Columns

 a11 a12  a1n   c1 

a a22  a2 n  c 
Let A   21 be an 𝑚 × 𝑛 matrix and C   2  be an n-vector, that is, an
     
   
 am1 am 2  amn  cn 
𝑛 × 1 matrix. Since A is 𝑚 × 𝑛 and C is 𝑛 × 1 , the matrix product Ac is the 𝑚 × 1 matrix

a12  a1n   c1    row1 A   c   a11c1  a12 c2  ...  a1n cn 

T
 a11
a  
a22  a2 n  c2    row2 A T  c   a21c1  a22 c2  ...  a2 n cn 
Ac   21  = =
          
    
 am1 am 2  amn  cn   row A T  c   am1c1  am 2 c2  ...  amn cn 
 m 
 a11   a12   a1n 
a  a  a 
c1  21 
 c2  22 
 ...  cn  21  = c1  col1 A   c2  col2 A   ...  cn  coln A  …… (3)
        
     
 am1   am 2   am1 

Thus the product Ac of an 𝑚 × 𝑛 matrix A and an 𝑛 × 1 matrix c can be written as a linear

combination of the columns of A, where the coefficients are the entries in the matrix c.

2
 2  1  3
Example: Let A    and c   3 . then the product Ac written as a linear
 4 2 2   4 
combination of the columns of A, is:

12
Linear Systems

Consider the linear system of m equations and n unknowns

a11 x1  a12 x2  ...  a1n xn  b1
a21 x1  a22 x2  ...  a2 n xn  b2

an1 x1  an 2 x2  ...  amn xn  bm

 a11 a12  a1n   x1   b1 

a a22 
 a2 n   
x2  b 
If A   21 , x  and b   2  . Then
      
     
 am1 am 2  amn   xn  bm 

 a11 a12  a1n   x1   a11 x1  a12 x2  ...  a1n xn  b 

a a22  a2 n   x2   a21 x1  a22 x2  ...  a2 n xn  b2 
Ax   21  (5)
         
    
 am1 am 2  amn   xn   an1 x1  an 2 x2  ...  amn xn  bm 

The entries in the product Ax at the end of (5) are merely the left sides of the equation in (4).
Hence the linear system (4) can be written in matrix form as
Ax  b
The matrix A is called the coefficient matrix of the linear system (4) and the matrix

 a11 a12  a1n b1 

 
a a22  a2 n b2 
 A b    21
    
 
 am1 am 2  amn bm 

Obtained by adjoining column b to A is called the augmented matrix of the linear system (4).
The augmented matrix of (4) is written as  A b  .

Conversely, any matrix with more than one column can be thought as the augmented matrix of
a linear system. The coefficient and augmented matrices play roles in our method for solving
linear systems.

If b1  b2  ...  bm  0 in (4) the linear system is called a homogeneous system. A homogeneous

system can be written as
Ax  0
Where 𝐴 is the coefficient matrix.

13
 1.5. Algebraic Properties of Matrix Operations

Properties of Matrix Addition

Theorem: Let A, B and C be matrices:

a) A  B  B  A
b) A   B  C    A  B   C
c) There is a unique 𝑚 × 𝑛 matrix 0 such that
A0  0 A  A
For any 𝑚 × 𝑛 matrix A. the matrix 0 is called the𝑚 × 𝑛 zero matrix.
d) For each 𝑚 × 𝑛 matrix A there is a unique 𝑚 × 𝑛 matrix D such that A  D  0
We shall write D by (−𝐴)
A  ( A)  0

Properties of Matrix Multiplication

a) If A, B and C are matrices of the appropriate sizes, then A  BC    AB  C

b) If A, B and C are matrices of the appropriate sizes then
 A  B  C  AC  BC
C  A  B   CA  CB

Properties of Transpose

If r is a scalar and A and B are matrşces of the appropriate sizes then

a) A 
T T
A

 A  B
T
b)  AT  BT

 AB 
T
 BT AT

 rA
T
c)  rAT

It is known that if a and b are real number then ab  0  a  0 or b  0 . But this not true for
matrices. Ex.

1 2  4 6  0 0
A  0 B   0 but AB   
2 4  2 3  0 0

14