1: Vectors

Vectors

Examples

Addition and scalar multiplication

Inner product

Complexity

Source: Introduction to Applied Linear Algebra, Boyd & Vandenberghe

D. Matsypura, A. Wong, QBUS1040 1/29

 Vectors 1

• a vector is an ordered list of numbers

• we’ll use two styles of notation:
! $
−1.2
" 0.0%
" %
# 3.6& or (−1.2, 0.0, 3.6, −7.2)
−7.2
• numbers in a vector are called entries, coefficients, or elements
• length of vector is its size, length, or dimension
• vector above has dimension 4; its third entry is 3.6
• vector of length n is called an n-vector
• numbers are called scalars

1: Vectors 2/29
 Vectors via symbols 1

• we’ll use symbols to denote vectors, e.g., a, X, p, β, E aut

• other conventions (that you should not use in this course): g, "a

• ith element of n-vector a is denoted ai

• if a = (5, 7, 1, 4, 6) then, a3 = 1

• in ai , i is the index

• for an n-vector, indices run from i = 1 to i = n

• ar:s = (ar , . . . , as ) is a subvector of a, and a2:4 = (7, 1, 4)

• warning: sometimes ai refers to the ith vector in a list of vectors

• two vectors a and b of the same size are equal if ai = bi for all i

• we overload “=” and write this as a = b

1: Vectors 3/29
 Block vectors 2

• suppose b, c, and d are vectors with sizes m, n, p

• the stacked vector or concatenation (of b, c, and d) is

! $
b
a = # c & = (b, c, d)
d

• also called a block vector, with (block) entries b, c, d

• a has size m + n + p

a = (b1 , b2 , . . . , bm , c1 , c2 , . . . , cn , d1 , d2 , . . . , dp )

1: Vectors 4/29
 Zero, ones, and unit vectors 3,4

• n-vector with all entries 0 is denoted 0n or just 0

• n-vector with all entries 1 is denoted 1n or just 1

• a unit vector has one entry 1 and all others 0

• denoted ei where i is entry that is 1

• unit vectors of length 3:

! $ ! $ ! $
1 0 0
e1 = # 0 & , e2 = # 1 & , e3 = # 0 &
0 0 1

1: Vectors 5/29
 Sparsity 5

• a vector is sparse if many of its entries are 0

• can be stored and manipulated efficiently on a computer

• nnz(x) is number of entries that are nonzero

• examples: zero vectors, unit vectors

1: Vectors 6/29
 Outline

Vectors

Examples

Addition and scalar multiplication

Inner product

Complexity

1: Vectors 7/29
 Location or displacement in 2-D or 3-D 6

• 2-vector (x1 , x2 ) can represent a location or a displacement in 2-D

1: Vectors 8/29
 Images, video

Monochrome (black and white) image

grayscale values of M × N pixels stored as M N -vector (e.g., row-wise)
5
5
0
0
0 .6
0 .0
0 .2
0 .2 ! $ ! $
x1 0.65
" x2 % "0.05%
" % " %
" x3 % "0.20%
" % " %
" . % " . %
x = " .. % = " .. %
" % " %
"x62 % "0.28%
" % " %
#x63 & #0.00&
x64 0.90
0
8
0
0
1 .0
0 .2
0 .0
0 .9

Color image: 3M N -vectors with R, G, B values of the M N pixels

Video: KM N -vector represents K monochrome images of M × N pixels

1: Vectors 9/29
 Portfolio vector

Portfolio
• n-vector represents stock portfolio or investment in n assets

• ith element is amount invested in asset i

• elements can be number of shares, dollar values, or fractions of total dollar

amount

p = (5000, 2000, 8000, 3000)

1: Vectors 10/29
 Resource vector

Resource vector
• elements of n-vector represent quantities of n resources or commodities

• sign indicates whether quantity is held or owed, produced or consumed, ...

• example: bill of materials gives quantities needed to create a product

r = ( −2, 1, 0, −1)

1: Vectors 11/29
 Feature vectors

contain values of variables or attributes that describe members of a set.

Examples
• age, weight, blood pressure, gender, . . . of patients

• size, number of bedrooms, list price, . . . of houses in an inventory

Area (m2 ) Bedrooms Bathrooms Cars Pets Weekly rental price ($)
252 3 2 2 1 1400

Note
• vector elements can represent very different quantities, in different units

• can contain categorical features (e.g., 0/1 for male/female)

• ordering has no particular meaning

1: Vectors 12/29
 Word count vectors 7

• a short document:

Word count vectors are used in computer based document analysis.

Each entry of the word count vector is the number of times the
associated dictionary word appears in the document.

• a small dictionary (left) and word count vector (right)

! $
word 3
in "2%
" %
number "1%
" %
horse "0%
" %
the #4&
document 2

• dictionaries used in practice are much larger

1: Vectors 13/29
 Polynomials 8

a polynomial of degree n − 1

f (t) = c1 + c2 t + c3 t2 + · · · + cn tn−1

can be represented by an n-vector (c1 , c2 , . . . , cn )

Example
f (t) = 3 + t − 5t2 + 2t3
would be represented by the n-vector (3, 1, −5, 2)

1: Vectors 14/29
 Outline

Vectors

Examples

Addition and scalar multiplication

Inner product

Complexity

1: Vectors 15/29
 Vector addition 9

• n-vectors a and b can be added, with sum denoted a + b

• to get the sum, add corresponding entries:

! $ ! $ ! $
0 1 1
# 7 &+# 2 &=# 9 &
3 0 3

• subtraction is similar

Properties of vector addition:

• commutative: a + b = b + a

• associative: (a + b) + c = a + (b + c)
(so we can write both as a + b + c)

• a+0=0+a=a

• a−a=0
1: Vectors 16/29
 Scalar-vector multiplication 10

• scalar β and n-vector a can be multiplied

βa = (βa1 , . . . , βan )
• also denoted aβ
• example:
! $ ! $
1 −2
(−2) # 9 & = # −18 &
6 −12

Properties of scalar-vector multiplication:

• associative: (βγ)a = β(γa)

• left distributive: (β + γ)a = βa + γa

• right distributive: β(a + b) = βa + βb

1: Vectors 17/29
 Linear combinations 11

• for vectors a1 , . . . , am and scalars β1 , . . . , βm ,

β1 a 1 + · · · + βm a m

is a linear combination of the vectors

• β1 , . . . , βm are the coefficients

• a very important concept

• a simple identity: for any n-vector b,

b = b1 e 1 + · · · + bn e n

1: Vectors 18/29
 Outline

Vectors

Examples

Addition and scalar multiplication

Inner product

Complexity

1: Vectors 19/29
 Inner product 12

• inner product (or dot product) of n-vectors a and b is

aT b = a1 b1 + a2 b2 + · · · + an bn

• other notation used (that you should not use in this course):
〈a, b〉, 〈a|b〉, (a, b), a · b

• example:
! $T ! $
−1 1
# 2 & # 0 & = (−1)(1) + (2)(0) + (2)(−3) = −7
2 −3

1: Vectors 20/29
 Properties of inner product 12

• aT a = a21 + a22 + · · · + a2n ≥ 0

• aT a = 0 if and only if a = 0

• commutative: aT b = bT a

• associative with scalar multiplication: (γa)T b = γ(aT b)

• distributive with vector addition: (a + b)T c = aT c + bT c

can combine these to get, for example,

(a + b)T (c + d) = aT c + aT d + bT c + bT d

1: Vectors 21/29
 Simple examples

• eTi a = ai (inner product with ith unit vector picks out ith entry)

• (ei − ej )T a = ai − aj (differencing)

• 1 T a = a1 + · · · + an (sum of entries)

• (1/n)T a = (a1 + · · · + an )/n (average of entries)

1: Vectors 22/29
 Example: Weighted sum

• w is vector of weights

• f is vector of features

• inner product wT f = w1 f1 + w2 f2 + · · · + wn fn is total score

Example

w = ( 0.25, 0.25, 0.5) (assessment weights)

f =( 60, 80, 70) (assessment marks)

wT f = 0.25 × 60 + 0.25 × 80 + 0.5 × 70

= 15 + 20 + 35
= 70

1: Vectors 23/29
 Example: Cost

• p is vector of prices of n goods

• q is vector of quantities purchased

• inner product pT q = p1 q1 + p2 q2 + · · · + pn qn is total cost

Example

p = ( 1.00, 2.50, 5.00) (fruit prices)

q=( 2, 4, 1) (fruit quantity)

1: Vectors 24/29
 Example: Portfolio return

• h is portfolio vector, with hi the dollar value of asset i held

• r is vector of fractional returns over the investment period:

pfinal
i − pinit
i
ri = , i = 1, . . . , n
pinit
i

where pinit
i and pfinal
i are the prices of asset i at the beginning and end of
the period

• rT h = r1 h1 + · · · + rn hn is the total return, in dollars, over the period

1: Vectors 25/29
 Example: Portfolio return
Example: We have two companies we have invested in. We invested $200 into
company 1 and $100 into company 2. This means that:

h = (200, 100)

pfinal
1 − pinit
1 27.10 − 54.20
r1 = init
= = −0.5
p1 54.20
pfinal
2 − pinit
2 50.00 − 40.00
r2 = = = 0.25
pinit
2 40.00
Our fractional returns are then:
r = (−0.5, 0.25)
Our total returns are then:
rT h = −0.5 × 200 + 0.25 × 100 = −100 + 25 = −75
1: Vectors 26/29
 Outline

Vectors

Examples

Addition and scalar multiplication

Inner product

Complexity

1: Vectors 27/29
 Flop counts

• computers store (real) numbers in floating point format

• basic arithmetic operations (+, −, ∗, /, . . .) are called floating point

operations or flops

• complexity of an algorithm or operation: total number of flops needed, as

a function of the input dimension(s)

• crude approximation of time to execute:

number of operations (flops)

run time ≈
computer speed (flops per second)

• current computers are around 1Gflop/sec (109 flops/sec)

1: Vectors 28/29
 Complexity

Operation count (flop count)

• total number of operations in an algorithm
• in linear algebra, typically, a polynomial of the dimensions of the problem

Dominant term: the highest-order term in the flop count

1 3 1
n + 100n2 + 10n + 5 ≈ n3
3 3
Order: the power in the dominant term
1 3
n + 10n2 + 500 = order n3
3

Examples of vector operations in this lecture (for vectors of size n):

• addition, substraction: n flops
• scalar multiplication: n flops
• inner product: 2n − 1 ≈ 2n flops
these operations are all of order n
1: Vectors 29/29