Advanced Mathematics for Economics ©2022 Scott Hacker

Basic Matrix Operations and Geometric Interpretations

Why deal with matrix algebra? (a) It allows simpler, more succinct writing of a complicated system of
equations or expressions. (b) It provides a means of solving set of linear equations (and helps you
predetermine if they’re solvable).
A. Fundamental Matrix Algebra
1. Basic Definitions
Matrix: an ordered rectangular array of numbers, parameters, or variables. Each of the numbers,
parameters, or variables is called an element.
10 30 5   5 8 2 1 4   h11 h12 h13  2 5 3 
1 1 / 2   
examples: D = −2 20 8 
  E= 20 3 7 ,
  F= 2 4 
,G = 3 7 ,
  H= h21 h22 h23  ,K= 5 4 0
 
 4 3 10   2 1 0   2 5 h31 h32 h33  3 0 1 

Vector: a matrix with 1 column (a column vector) or 1 row (a row vector)

 v1 
v 
10   2
ex: r = [ 2 20 8] , w= 2,
  u = [ u1 u2 u3 .. un ] , v=  v3 
 
 4  :
v 
 n
Note: D,E,H,K each contain 3 column vectors, 3 row vectors.

Transpose of a vector
The transpose of a row vector turns it into a column vector with the same elements in the same order.
The transpose of a column vector turns it into a row vector with the same elements in the same order.
The transpose of a vector is represented by the superscript of ' or T. For any vector v, (v')'= v.
2
examples: r' = 20 , w' = [10 2 4]
 8 

Dimensions: number of rows × number of columns

examples: D, E, H, and K are 3×3 matrices, F is 2×2, G is 3×2, r is 1×3,w is 3×1.

Square Matrix: An n×n matrix (number of rows=number of columns).

Principal diagonal: the diagonal string of numbers from the northwest corner to the southeast in a square
matrix.

Trace of a Matrix: the sum of the elements along the principal diagonal, e.g. the trace for D is
10 + 10 + 20 = 40.

3. The Additive Properties of Matrices

Matrix addition and subtraction. Add and subtract each corresponding element. This is possible only if
the matrices have the same dimensions.
 10 + 5 30 + 8 5 + 2  15 38 7   10 − 5 30 − 8 5 − 2   5 22 3 
examples: D + E = −2 + 20 20 + 3 8 + 7 
  = 18 23 15 ,
  D–E= −2 − 20 20 − 3 8 − 7 
  = −22 17 1 
 
 4 + 2 3 + 1 10 + 0   6 4 10   4 − 2 3 − 1 10 − 0   2 2 10 

4. The Multiplicative Properties of Matrices

Scalar Multiplication: When a matrix is multiplied by a scalar (another word for a “usual non-matrix real
number"), multiply each element of the matrix by that scalar.

1 4  2 8 
example: 2G = 2 3 7 = 6 14 
  .
2 5 4 10 

1
The inner product of two vectors;
The inner product (or dot product or scalar product) between two vectors, u and v (as defined on the
previous page) is

u•v = u1v1 + u2 v2 + u3v3 +...+un vn .

Note that (a) the result is a scalar (thus the term scalar product), (b) u and v must have the same number
of elements, (c) it does not matter whether u and v are row or column vectors (that aspect of each is
ignored, so for example, u•v = u•v'), and (d) the order in which the inner product is performed does not
matter (it is commutative): u•v = v•u.

example: given the r and w defined above, r•w = w•r = 2(10) + 20(2) + 8(4) = 92.

Vector multiplication resulting in a single element in the matrix: The order in which you multiply two
vectors together is CRITICAL. If a and b are two vectors, ab will very likely not be the same as ba. To
end up with a single element in the matrix from the vector multiplication, the first vector must be a row
vector, the second must be a column vector, and they must have the same number of elements.

If u is an 1×n vector and v as an n×1 vector, we get uv = [u•v] = [ u1v1 + u2 v2 + u3v3 +...+un vn ], which is a 1×1
matrix.

example: given the r and w defined on the previous page,

rw = [2(10) + 20(2) + 8(4)] = [92],

while wr ≠ [92] (we need the information in Matrix multiplication to figure out what wr is instead).

If the multiplication of two vectors results in a 1×1 matrix, it can be treated as a scalar, e.g. in the
example we could also write rw = 92. [Note that doing the reverse, i.e. replacing a scalar with a 1×1
matrix, is not generally appropriate when further calculations follow.]

Matrix multiplication: Again, the order in which you multiply two matrices together is CRITICAL. AB
will very likely not be the same as BA. To multiply at all, you must have the number of columns in the
first matrix equal to the number of rows in the second. Thus if A is an rA×cA matrix and B is an rB × cB
matrix, then for AB to be a possible multiplication, you MUST have cA = rB (if this is true, the matrices
are said to be ”conformable” for such multiplication). If multiplication is possible, then you’ll end up
with a new matrix with the dimensions rA×cB. The ith column vector in the new matrix is found by
performing vector multiplication on each row of A (in order) with the ith column of B:
 b11 b12 b13 .. b1c B 
 a11 a12 a13 .. a1c A 
b b22 b23 .. b2 c B
a
 21 a22 a23 .. a2c A   21 
AB =  a31 a32 a33 .. a3c A 

 b31 b32 b33 .. b3c B 
 
 : : : : : 
a  : : : : : 
a arA 3 .. arA c A 
 rA 1 rA 2  br 1 br 2
 B B
brB 3 .. brB c B 
 a11b11 + a12 b21 +...+ a1c A brB 1 a11b12 + a12 b22 +...+ a1c A brB 2 .. a11b1c B + a12 b2c B +...+ a1c A brB c B 
 a b + a b +...+ a b ... .. 
21 11 22 21 2 c r 1 a21 b12 + a 22 b 22 + + a2 c b r 2 a21b1c B + a22 b2 c B +...+ a2 c A brB c B 
=  :
A B
:
A B
: :  .
 
arA 1b11 + arA 2 b21 +...+ arA c A brB 1 arA 1b12 + arA 2 b22 +...+ arA c A brB 2 .. arA 1b1c B + arA 2 b2c B +...+ arA c A brBc B 

1 4 
 2 3 1   = 2(1) + 3(2) + 1(5) 2(4) + 3(3) + 1(1)  = 13 18
example:    2 3     
1 4 5 5 1  1(1) + 4(2) + 5(5) 1(4) + 4(3) + 5(1) 34 21
 

2
B. Geometric Interpretations of Matrix Algebra

1. Basics of geometric interpretation of vectors

A vector may be interpreted geometrically as an arrow from the origin to the point represented by the
 4
values in the vector. For example, the vector u =   may be interpreted as an arrow from the point
 3
(0,0) to the point (4,3), as in figure (a) below. In this interpretation there is a direction represented by
the way the arrow is pointing. Multiplication of a vector by a positive scalar will stretch the vector
(when the scalar is greater than 1) or contract it (when the scalar is less than 1) but it will point in the
same direction. [Note: The term scalar refers to what multiplying a vector by a scalar does—the scalar
“rescales” the vector length by stretching or contracting the vector.] See the vector 2u in figure (a)
below.

figure (a) figure (b)

To add two vectors together geometrically, shift one of the vectors keeping the direction and length
unchanged such that its starting point is where the other vector ends. Where this shifted vector ends
provides the end point for the new summed vector. The starting point for the new summed vector is the
 4 1  5
again the origin. An example of this is shown in figure (b), with u =   , v =   , and u + v =   .
 3  2 5
In figure (a) the vector 2u can also be interepreted of course as u + u.

If you take the negative of a vector it reverses its direction, as can be seen in figure (c) for –u.
Subracting a vector is the same as adding the negative of it. This is demonstrated in figure (d) with
2v − u .

figure (c) figure (d)

3
2. Vector length and distance between vectors

 u1   v1 
u  v 
Let u =  2  and v =  2  .
: :
   
un  vn 

The norm of a vector u, written as ||u||, is its length, i.e. its distance of its end-point from the origin, and
it is determined by the following generalization of the Pythagorean theorem.

|| u ||= u12 + u 22 + ... + u n2 = u ⋅ u ,

This is demonstrated for the 2-element vector case below, for which || u ||= u12 + u 22 . In the example
4
shown in the figure below, u =   , so || u ||= 42 + 32 = 16 + 9 = 25 = 5 .
 3

The distance between two vectors u and v is the distance between the end-points of those vectors and we
can denote that distance by d(u,v).

In Euclidian space we have the following rule, also associated with the Pythagorean theorem:

d (u, v ) = (u1 − v1 ) 2 + (u 2 − v 2 ) 2 + ... + (u n − v n ) 2 = (u − v ) ⋅ (u − v ) .

4  12 
The figure below displays the distance between u and v when u =   and v =   , which is
 3 − 3
d (u, v ) = (4 − 12) 2 + (3 − (−3)) 2 = (−8) 2 + 6 2 = 64 + 36 = 100 = 10.

More examples:
0   3
The distance between u =   and v =   is (0 − 3) 2 + (4 − 0) 2 = 25 = 5 .
 4 0 
 4 4
The distance between u =   and v =   is (4 − 4) 2 + (3 − (−3)) 2 = 36 = 6 .
 3 − 3
4
Note the following:

(a) since ||u|| and d(u,v) are dealing with distances, it is always the positive square root that we are
dealing with in their calculations (the negative solution to the square root is ignored)

(b) the formula for the norm is simply a special case of the formula for the distance between two
vectors—if you calculate the distance between a vector and a vector of all zeros (with the same
dimensions as the first vector), then that is the same as calculating the length of the first vector.

(c) if u = v d(u, v) = 0
(d) if u ≠ v d(u, v) ≠ 0
(e) if w ≠ u and w ≠ v d(u, v) ≤ d(u, w) + d(w, v) (this is the “triangle inequality”)

3. Linear combinations of vectors, linear dependence, and vector spaces

A linear combination of vectors v1, v2, v3, . . . , vn occurs when each of the vectors is multiplied by a
constant and the resulting vectors are summed up: 1
n

∑k v
i =1
i i = k1v1 + k2 v 2 + ... + kn v n

A set of vectors is linearly dependent if and only if one of the vectors may be written as a linear
combination of the others. If this is not the case, the set of vectors is linearly independent.

For example the set of vectors

5  4 7 
 2 , 1 , and 4 is linearly dependent since
     

 5   4  7 
3  − 2  =   ,
 2  1   4 

5  4
but   and   are linearly independent from each other since you cannot find a constant and multiply
 2 1 
by one of the vectors and get the other vector.

1
Note that each of v1, v2, v3, . . . , vn represents an entire vector, i.e. they are not elements of a single vector.
5
A vector space is a set of vectors for which vectors in that space can be summed and they can be
multiplied by scalars, following some standard axioms on these operations. A vector subspace is a non-
empty subset of a vector space for which if you add any two vectors in that space, the resulting vector
will be in that space, and if you multiply a vector in that space by a scalar, the resulting vector will be in
that space. Given the axioms in the definition of vector space, a vector space is also vector subspace.
The smallest subspace is the zero-dimensional space containing only the origin (all elements zero). Keep
in mind also that a vector subspace is itself a vector space.
x 
Suppose you are only dealing with vectors that have only two elements:  1  for any x1 and x2. This
 x2 
two- dimensional space is referred to as R2. If you add any vectors in that space together, or multiply
one vector in that space with a constant you’ll get another two-dimensional vector. Now suppose we
consider the set of all two-dimensional vectors with any value for the first element and 0 for the second,
 x1  x 
that is, with the form c   for any x1. Clearly this simply a special case of  1  with x2 equal to a
0  x2 
 x1   y1 
constant (zero). If you add two such vectors in this space,   and   for any x1 and y1, then you get
0 0
 x1   y1   x1 + y1 
 0  +  0  =  0  , and if you multiply a vector in this space by a constant, c, then you get
     
 x  cx 
c  1  =  1  . Thus in the set of two-dimensional vectors with any value for the first element and 0 for
0  0 
the second, addition of vectors in this set or multiplication of a vector in this set by a scalar results in
another vector in this set. Therefore this set of vectors is a vector subspace. Likewise the set of vectors
0 
of the form   , for which this vector is the only element in the set, counts as a vector subspace since
0 
0  0  0  0  0 
+ =
0  0  0  and c 0  = 0  .
         

A set of vectors is said to span a vector space if some linear combination of them can produce any
vector in the space. Let’s return to the full two-dimensional vector space. If you take any two vectors in
that vector space that are linearly independent from each other, then any other two-element vector must
be a linear combination of them. In this sense any two linearly independent vectors can cover (“span”)
the whole of a two-dimensional vector space through linear combinations of them. For example, since
5  4 5   4
2 and 1 are linearly independent, then k1 2 + k 2 1  can be used to produce any two-dimensional
       
5  4 5   4 7 
vector if k1 and k2 are chosen correctly, so   and   span R2. Now it is true that   ,   , and  
 2 1   2  1   4
also can be said to span the two-dimensional vector space, but one of them is redundant, since the first
two were sufficient for this spanning. If a set of vectors spans a vector space and that set is linearly
5  4
independent, then those vectors are said to provide a basis for that vector subspace. Thus   and  
 2 1 
5  4 7 
provide a basis for R2, but not   , 1 , and  4 .
 2    

6
If all the two-element vectors that you have are linearly dependent for each pair of vectors, then only a
one-dimensional vector space is spanned by those vectors. For example,

5 10 20

 2 , 4, 8
     

are pairwise linearly dependent—the second vector is two times the first and the third is two times the
second and four times the first. Therefore they span only one dimension, mapped out by varying c on c
5
2 , and any one of them would form a basis for that one dimension.
 

Suppose instead you are only dealing with vectors that have only three elements. If you take any three of
them that are linearly independent from each other, then any other three-element vector must be a linear
combination of them. In this sense any three linearly independent vectors can span (and provide a basis
for) the whole of three-dimensional vector space, R3, through linear combinations of them. On the other
hand if the number of linearly independent three-element vectors available is two, then those vectors can
span only a two-dimensional vector space. Likewise if the number of linearly independent three-element
vectors available is one, then those vectors can span only a one-dimensional vector space. Figure 4.3 in
the textbook will be discussed in this context in class.

Generalizing, suppose you are only dealing with vectors that have only n elements. If you take any n of
them that are linearly independent from each other, then any other n-element vector must be a linear
combination of them. In this sense any n linearly independent vectors can span (and provide a basis for)
the whole of n-dimensional vector space through linear combinations of them. The n-space we are
discussing is more specifically referred to as “Euclidan n-space”.

4. Orthogonality
Two non-zero vectors u, and v are orthogonal (at a right angle) to each other if and only if

u ⋅ v = 0.

A vector is orthogonal to a vector space if and only if it is orthogonal to every vector in that space.

7
C. Interpretation of a set of equations as a vector equation
Suppose we are given the two equations

2x – y = -1
x+y=4

To solve these equations graphically in the standard way (taught first to students in primary or secondary
education), we focus on the rows—in the x-y plane, each equation represents a line, and the intersection
of those two line provides a solution for the two equations simultaneously. This way of looking at things
may be considered the “row picture”.

Alternatively, one could focus on the columns (giving the “column picture”): the set of equations may
be rewritten as

 2 − 1 − 1
x  + y  =   .
1  1 4
 2 − 1 − 1
Thus, some linear combination of the column vectors   and   results in   , with that linear
1  1 4
combination given by the weights x and y. By expanding or contracting the length of these vectors
through multiplication (by x and y for the respective first two vectors) and adding them together we get
the right-side vector. This is shown in class graphically, with x = 1, y =3 (the solution to the equations).

D. Interpretation of a set of equations with the unknowns combined into a vector and solving for
the unknowns using matrix algebra

The vector equation above may be rewritten as

2 − 1  x  − 1
1 1   y  =  4 
    
2 − 1  x − 1
or Ax = b, with A =   , x =  y , and b =  4 .
1 1     

More generally, if you know that

a11x1 + a12x2 + a13x3 + ...+ a1nxn = b1
a21x1 + a22x2 + a23x3 + ...+ a2nxn = b2
a31x1 + a32x2 + a33x3 + ...+ a3nxn = b3
:
am1x1 + am2x2 + am3x3 + ...+ amnxn = bm,

You can turn this system into the matrix algebra form, Ax = b, where
 b1 
 a11 a12 a13 .. a1n   x1  b 
a
 21 a22 a23 .. a2 n


x 
 2  2
A = a31 a32 a33 .. a3n , x =  x3  , b =  b3  . A is called the “coefficient matrix.”
 
   
 : : : : :   :   
a
 m1 am 2 am3 .. amn 
  
 xn  :
bm 
Ax = b is only solvable if b can be written as a linear combination of the column vectors in A. In other
words, for it to be solvable, b must be within the space spanned by the columns of A (referred to as the
column space). To solve this, we need to be able to perform operations on both sides of the equation so
that A vanishes from the left side. For that we need the inverse of A, A-1, which is introduced in the next
lecture.
8