RECAP

Sorting fish
Sensors

Y
As
S
P to
Y Salmon
lo.bo.no

Et
Classification vs Regression
Predicting marks stockprices
Sortingfish into various classes
will thestock
go up or down

Nvidia in the nextweek

will the
pricego
up down classification
what is the
price
after I week regression
dog us cat no inherent ordering

price marks inherent ordering

 breadth a Dataset collection
of
observations

i Tuna
ji Observation t data
point
Labeled Dataset observation
length
Correct ans

Training
data

mi class
input Model

IIIs

Dataset lows
f obseration
dti dimensional vector

features dimensions

weight length breadth class

1 3 2 1.5 Tuna
I
obseration
2
3
Salmon

Ah Nx dti
t d feats I day
 Mr mom
breadth n
if length t

print Tuna

else
Salmon print Salmon

Tuna

5,0 length

Ce axis line

Simplest ML model a bunch of axis parallel lines to separate

the classes
Random forests

i im
i
Salmon

Mc model

what is the Mc model

find what this model is

him does the

On which side f my
point lie on I É
how in
point front I
far is line
 2 dimensions separating entity 2D Line

Equation line
f a

Slope Intercept mxtc

y S
slope
Y intrap
P by

by b ox
I

2 3
y
X I 21 3
y 5
sin Perpe hypo ios base hype
some have
people curly brown hair m
tightly pulled back slope tan a
tan per base
tan O Pb

Standard
form
ax 7
by c o X t
Ey t Ea o

W X twixt we
I y
x
E
a
Equation line in 2D je Ex f
f
a

y y
 Generalize to higher Dimension

2D separating line I 55
Wix wax t w x two o
3D ax
by cz d o
separating plane
UD
s separating hyperplane wit twitz t wa xd two to

Wi Xi two 0

Vector Notation vector column vector d x1

w X

111

ETA
I x two 0

dot product always b w a low vector I a sodvector

 w it wa
WT scalar Scalar

W X t we
Xz t wax d
scalar

ML Model separating hyperplane in d dimension

w Ix t Wo o
f Le option featurevector
weight
 in d dimension
collection
of values
a
Victor
anag
list
tape

ax x2 x xd d dimensional vector

X 2 3
2,3 Xi x2
F
H s 11 11 fi
f It 412 norm euclideanDistanceor length
6,07
Length of vector
at

Fixx
Éj
x

p
a

Fei EÉx
Vector addition subtraction

if
x
fi xx
II

X
movement rector

2 DIE
it as first mom along
x 11,4 x then
dongXz

Hyperplanes

general equation wt x two o

Oh
Line passes through 112
point 1,2 lies on the line

point 1,2 satisfies th equatinforth

g an

Origin satifies tin eq for our

hyperplane
 WTx t Wo 0

1.23 9.5
Epd
Origin x
1 Cia Rb 2a k

a I
o wi AER 2
two o 9

way 9 7

a ER 0,0 o1 op
T d
2,3
4,0 4 in

Wo o Kot Ld o two to
a eat
o
113,7
Wo

Anga b w two vectors

X X
prong
ill Hall case
dot
I

can Itta
 x2
Exit 5 3
s
x E

i s x2

l I

I s s is

Gigg
30
cos

Hail fest Fy
11 211 Fish Fu

cos
II ios

28.07

1 Cas x

Vtr
g
 o
Eq for Hyperplane Gtx two

geometric Interpretation of lg f a
plane

2 0
a 31
hyperplane passing through origin y x to

live 2x 3 0
2D 2

at
x
y
w
s
2 39 0

3D plane

W is a veto that passes through the origin is 1 to

the Wtx two

plane to
Wtx o hyperplane passing through origin

70

w Tx o
all satisfy
a
56 eg fate
origin plan WT

WTxp 0
IT WTX o

I w lolyall cos o o

70 70 T
f
O

Cosa o D 900

cos a base
hypot
a
go 1 11
My 11 11 Cosa a

É
E

Origin d

dis origin
Ext
from Hull.LY two o

lull a two o
 wo a llull

It
a

Instead of arbitrary w

HI we use to unit vector

Wo a

T Gtx two to

1
unitrecta
distance
for origin

Interpretation
for eg f
a
plan
GTX two o X two o

w vector I to all vectors on on

the planepassing through
origin
w distance
w OT
distended fall 1
 70

It

On which does lie on

side of the
plan
a
point
space
ve half
t
halfspot yo
ve

ÉÉ Wix two o

É
ite
Origin
I wTx 00 0

assuming O f 900 Cos O is the

1
11011.11 1 cos a two 0

eve tre tve o

11 4174411
Gtx co
11 11 Hwy
given a x to find which side it lies on
point
o one half spae
w TX two 0 on th
plan
70 re half space

x o
Tp wo

wt x two o

distance
f point from hyperplane

de brose
I
yE
70

o to a

Egg
20 acoso I wTx two o

ay
 b 11 11 a

11 11
Fa
Ixtle
if

FEE i
i
cost
d beso
Fifty

and
if
we

distance atx two

 Equation
for a
separating hyperplane in d dimensions

WT x two O

W E Rd x ed Wo E IR

which side the plane does the point x lie on

of

It WTX two D

just plug in x
Lo x lies in ve
half space
wt X two
y 0 X lies on IT

so x lies in tre half space

find the best

hyperplane
0.00
I
Brute Rd
farra w w wa Wo findthese
min a
values
ft t in range k
for a in
range4
for wa in range Id
measure how
good th hyperplane is
Stone loss
ifloss min
min loss
best w w watyo
dataset
Roth Ra
n'd

Quantifying how a dataset is

good
dimensions

go fun Salmon

given D xinyi Xie Rd gie 1,1

I class label
d dimensional

observations
featurevectors

find best
Ta

Count the
f mi classifications
connect o
incorrect o

for Xinyi in dataset

pred cot x two

if find so and
y.tt Don pnedsoandy jj
I correct t 1

else
incorrect
I
comet An dataset
accuracy
enon incomet an dataset min
 Salmon

Tuna
I
i

i
I

Is

Maximize the confidence Minimize to error

d wi two
dist Xi
y gw
It a

if the comet label is

maximize wtxi two

y
I F at data points

if the comet lable is the

maximize o Tx two
y
1
Ye
find the Ta given D Gigi 3
such that
Ttt.ge is maximized

wow

E Fitt
on
minimize yi

why an we
adding why not multiply

datatype will not have enough precision to hold th ans

S a
Sz s Sn
dI bz on 0.7 I

1.2 2
4g
3.1 0.7 Y
inf
ifany value is o it will make everything 0

multiplication is slower than addition

2 3ns I 0.2ns

lox slower

Ig sa sa's s lg s
Igs t
lgso
 I

144 12 13
I 30

F
6 7 12 10 3
13 in 1
7

12

IT
Tz

6 1,03 t
w w

7,4 92797
a ayes o
1 3 1 a
y E E
or

7 n
T2

Wix tux two o

w
Cruz co
IT
 dirt x
41 4,5

IT passes than IT 4 to

gg II X xD

6
y II x o

a
g
f x

6 o

g f x 6 Ex t 1 y
Gr Wo
w

W X t Oz Xe two o
6
go 6

0
I
8 6
w

No
I
6
1
Zz
 Distances

fuclidean Distance 22 norm shortest lingala b w two

points

P
P
Pr
x y z
norm Xxpztg.gg
Pr xz y z

dist ay Fftitutthis fax

tate

Ix

Manhattan Distance naam

abs x x tabs Y y tabs z za

m m ab hint absly
 Cosine
Similarity anga bl two vector

msimilakxsxd.ca
Fox 11 111 Ilxell cos 0

Cos D Xi x
t.mn

cos 0 vectors am I to each other

iff
0

Il to each other
cos 0 1
iff vector an

M Mr M
d
N Students
Matrix Intut 50fouldot
d movie
g y 0.3 0.7 null
9 3

i
Sn C 3

o d 0.95

1,0 2
Projection
f vector PCA

xd is the projection
f x ontox2

it t.gg ex
Magnitude

I x ll Hallcase

É Direction
unit rector along X2

unit
If
vector along to

it It
IT cos 0

Ila
Ga 1050
I
so
i
ii
Wit Fo
IEEE
 5k
Unit vector along to

tf
if
xi Ax ll I

11 11

W Wo w we w

W X t wz two 0
y
y
Ex E

X t y y

Xt y y o
Recap

Vector

Physics Something that has magnitude direction that lives in

some space

velocity fora field

go
mm
Computer
I
anay collection
fratus
1,2 37

Math anything for which the following operation on diff

in
Addition b w two vectors
your onto

I I I

Multiply with a scalar E IR

velocity I I I
Jot Is

at 1 3,5 at C 13,5
92 2 6,8 Hal 31 5 5
3,3
9 92 f

Cowl
Op 3 O'a
cows day
Polynomials

play ax't bid text't y Xt Z

E ai x

Pid x aged a xd ta at a
a x

P x badtabard tab at ab x tab

Ptp x
at ba x'd t t

itbixtaotb c.P
d
x c aux t c a x t c a

ur

sun
 Fshould alsoSam
be in th
space
a set which is
vector space
f rectos

dojfgfatios jp Closed under

vetoaddition
scalar
multiplication

S ol
Sis closed under rector addition to V ES Ctv Es

Scalar iff to.es ACER v.ES

iff
multiplication c

Axioms for a vector space

dosed under S Atv ES

vector addition UE f ve

closed under scalar multiplication VES CE IR c UES

Vector addition must be commutative u ve f un ut

vector addition is associative at
to v7 w

Should have an additive identity

define tense
u n
that it is in our
identityfor
space
n
a I n I is id
n o is
 g g
min o u n o is id
min
Max f n n n o is id
may

Set all polynomials degree d constitute a vector

f f
span
g g xd too xd t 0 x to

Should have an additive inverse ES u as

X O id
u f u F
inverse
ya
X t O
additive inverse

X o
1
Lemultiplicative invest

Scalar multiplication should be distributive

c Itv
c T t cat

Scalar multiplication should be associative e d T

ad I
I u U

vector I polynomial a ventor

an fait
vector
I hier pay
vector
i
space
o

f
la
auto
space
degree d polynomials
Set f rectors
Satisfies the axioms
analogy
S 2dplane justtakethe 8 8
is Spa vector space I yes satisfies all axioms

5 1 13 X doesn't all axioms

satify
is S a recta space

v i oil's v to
4s

a bid EIR
I set of all 2 2 matrices
vector

iss a reata space Yes

i all
iii id
2D 3D a E Ya
always vector spaces
 t

v 130 7 j

if
C
in i 13.1 7 13 1
its
If
a

It It j J
a it b I
1 1
9 It b 2
j 1.1 2
j a 15 6 a tab l
 b i o 7
a 13

Is

ax't by c
quadratic
ax C
by linen
d
tey y
Basis a set f rectors
set
f starting rectos fundamental vectors

take linear then set rectos to get reach new nectar

Combinatons
f f
Set f all this basis
Madaba vector is called the span f

5 a b EIR A
i f i f
we span

e
i i i f
s

i f it
i I I s

j 3 i
i f j linearly dependent

s
H I b 3a
3x
a b EIR

ft
 set D i i
Basis
just
a
f rectors

span f Basis B D S L cfjtc.it

a b c ER

linear Combination

ng
I

I i
I i
I
x a
j
b tc I

fiji
r
ng span

spanli if p

11,11 x

I
Linearly dependent

a u an linearly dependent
self vector ur un

t c o t c un t fn.vn o has only I solution which

is C c 0
z

a i un

U I a U b Vr

v in 2 v
u
i

tenant
u
1 u
i V V2 y

u n f v
I linearly independent

qq.yyg.gg 6 2
2 a o t l b
G a 6 3 b E Ga et a 0

7 a z y p 7 2 Ot d 2
D E
 Recap

scalar multiplication ie defined

iff
Vector vector addition

d dimensional
examples point in a span

Hittitet otitis
polynomials f degree d

p'd A t a x
and Egix
x
act t

Vector a set
of
vectors that satisfy thefollowing 10 axioms
space

vector addition
should be
Space
doh under rector addition
f a tu Es
if u Iu E

Addition is associative U tht w Gta w

is commata u to V tu

Additive inverse should exist wtf u I

Fdditiveinverse
Additive
identity
M to u
Xo x
 Scalar multiplication

dosed under scalar UES c ul

multiplication
CE IR

Associative dot
c f d I

distributive c Ito cot too

frontboth sides ret c cat c t

identity for scalar multiplication I I I

Basis a Vector
f space

Basis
f S isthe setof vectors 13 3 such
a vector space u u

that they are

linearly independent
the entire 5
they should spy space

a Basis
Span of
basis 13 u u
3 is the setof all rectos whim
f an
a
span
echos

licker
Span B
x1 x c u cute wt
 Basis with d will span a d dimensional
independent vector
hyperplane in some space

space IR

I B IB
span is E cer
y Idimensiond

1
Ia I ly text wax too
Linear
Transformations

T is a transformation a vector space V to a vector

from
Transformation space W

T V W f D c

Tie a lull function that assigns a value I EW

to every value I EV

T is linear
iff it
pesthlmacobinaions
T a t TU TCU e

c T u T Cau

t It
d
 I Ji t
j
T T
Tf 35 taj 3 Tfi 3 T
j

T i i Il
T o o

Cj
1 31 D
T
t

1 I 3 18 1
11 1 C

when does i land

s
if if
land
am does
j
 Find the transformation that rotates spare courtsclockwise by
900

i I

cnn.in

I find a transformation that scales horizontally by 2x

oI o
to 20

ta I
tf TG I
 3
T
1
É

O O
BI
 F
HI 4

franfornet points

3 13 1
Point
I
3 11 1 fi
2 2 X 2 3
L
2 3

i a
1 i f
 1.1 3

ggpypjpy.gg
T as a sin
sing fine cos0

wi
 h shear
f notation
g scaling

flglacxi

lotation
JIM
Siding
 Recap
vectorspace
a input
Linear Transformation
T V W output vector span
transformation

Transformation is linear presented as a matrix

iff afterwards

Limsstay lines To di G
un Tutt
where does each basis vectorend

Origin doesn't move upafter the transformation

T Con C T u

given a vector I when done it end the

up after transformation

T V W A
matrix describing this transformation

Tf A T

Linear
Transformation an like functions can be composed

T Tale A AG CA Az T
S
T oT

Gift
Determinant Matrix only for square matrices
f a

M
f Im f 1.4 2.3

M
f 4 5 G
1m Moo cofactor

Mo
o.o

fofacto moi
M cofactor mm

Cofactor ai Matrix after removing it now dj col

o 2

fatale
tiff
me 4 5 6
c

so

o i 2 o 2

me fast o
if
c
4 5 6

L D
M
f 4 5 G
1m Moo cofactor o.o

Mo ffact moi
M cofactor mm

Mt so a I s I

9.10 517
I 15.191 6 181 2 14 91 6 711 t 3

1 45 48 2136 72 t 3 32 35

3412 9 0

mt 4 5 G

int 2 s
tile of
2 36 92 5 9 21 t d 6 if
12 60 48
 48
33

tho
O 1 6 d
M find M 320
as
2 7 I 3 too
1 6 2 11 79
226
181
322
3322
241
100
 Subspaces f a H is a
given a
space subspace

subsetof S such that H itself is also a

space

why can't fun 8

H must satify the following
for H be different

H
front for s
D of S should also be in then Swould
U r E n ut v E H have 28
VEM CE IR c v e th Thm then
for any operation

can be atmost 1 identity

S entire 2Dplane
Contradiction
S I proofby
Assume
liftsof c thatthere are
2 distinct identities for
g id O x X Ax
id X X FX
S is a subspace
fs
line thatpassesthose origin is
id
if if
any
a subspace id id
o is a subspace if
gas LHS is Sam Ras alsohasto
be tou Sam

him Image 03
S o
any plan passingthroge origin any
 7 g

I1e
ftp.IIF E EIR

I ves
Q R
E abide er

I
is IRS a subspace
f
s No different dimensions
but u as
is
a b cd e Yes a E R
Ig a sabspna.gs
u
I 4s

gf
is a bc d E
f
a s Yes
subspace

gf
is a bc d
EM a
subspace f s No no 8

Yes
at btc d te
of
is a subspanfs
Ig 0
G W X to exit west w o
 a
191
I I a

49 o
9Éttd v r o
Ayaz tb tb

is at btc d te 3 a subspanfs No no 5
Ig

all 5
I sappon S digne polynomials

is all polynomials No
f degm 33 a subspace
f s

we can't saythat x x 3 5

Effigy's
X
polynomial
 Span Null space
Matrix Row Column
Subspaces
f
a
Space

a
of

0 la o

Ax b

does not have a Sol

when A 0

if
0 0 then we have no solution
 Column
space I a matt is the setof all possible

linear
f th colume
combination

actually tells what the output space

you
looks like

I 11 al s
la I
1 3
go
a
I 2 6
s
a 9 b
I c
I p

teaming
11 o

f
dimension Col space column spare is all possible
whatspace doth outputrectors live in
rakoff

I
rank 2
a
U cols have
f
we
linearly indigent
 dim colspacas n

mI ijn.n
each value in
outfit
span live in a n dimension plane

Row linear combination of all now

Space

in dim Rowen
JI
m

Et
 M
2 3 Hill
3 din

dinkol m 2 din no m 3

ii

lank f linearly independent

cols

all possible solution to egadia

Mall space set
f
Ax
p
what input vectors th origin
get squished on to

iff A I to Ax
is solution
full hank
rank o
to the trivial solution 5
ft output
rank 0

it is not fun rank infinite solution to Ax o

iff
 I
Linea Equation
System
f
A X t b t E Z d
g
faux t buy t C Z dz

as x big t c a d
I

AT 3
a b c

1 11
 Linear equations
System
f
A x t b C Z d
y System Linear
of

as X
Oz X t bz y Cz z
t b
y C Z dz
da
3 Equations

Picture when do theseplans intersect

exactly I point exactly 1 solution

points line plane a solution

no intersection no solutions
IT

gift outputs

I
as
by
Ifs
I
f
É
Picture what is the rector I that lands on 5 off
applying the transformation A

if Colla is the same as original

input space
Unique solution

is not the same either no satin

gyp
or a solutions
gauss Jordan Elimination Algorithm

Solution to Ax b
determinant A
f
A
rank
f A

Ix 72 5
T
Ly
a

a
z a
l f x
f
b

Create an
augmented matrix

Git coefficients output

now to convert
Apply operation to

leduced low Echelon

form
no sousa
 Ix 72 5
Ly t
n t z d
n t z d
Tx 72 5
2g
O
A t
3g swfq N t
3g
o

o c d

E i IF f
1 0 1 3 2 7
45

two
Equivalent Systems
if
linear equation
systems
f
A b Ailbe
have fin same solution set then
they an equivalent

Row operations certain operation that maintain equivalence

Swap any two rows

multiply any now with

a non zero constant

add scalar multiple to anotm

a
of a now
now
 5
9 65 Liz
Ix 72 3
Ly eq
n t z d
n t z d
U O
U 0 3g
3g

I eq z

0 O

n t z d

U o
3g

3 72 5
Ix Ly 72 5 20
O 3 Z d
n t z d

4 g
g x
t3y 0

3g
 Ix 72 5
Ly
n t z d

n t o
3g
pivot
g

L t
I
ont
3 0 O

R Ra l R
Ree Re azo R

O
43
1 3
y ay
g
7 35 3
i
RTR as R
0
1
43 7 3
21
3
43
o
54
293
o

f
3
ret 2 R

Rat Rz

ii
O 11 3 73 8 3 miss O o

Rst
5 103 2

15 R
R
X 5 3 Ilc Rs
Ey
t
71,2

Eee
t 22 29
J 2
2 29 2
z 103 10
Row Echelon Aom
Row Echelon fam

All now at the bottom

having only zero entries
an

Leading entry of every non zero now is to the right f

Pirot
the
above it
of every now
leading entry

141 2

Row Echelon Fom

29 2 It
f y
fr A F R

E Io it t iiiit
Eg 23

671 06
If
 X 35 19
I Solve
7
g 21719
2x 3g
2
3x 5g o
43021 7 y
3 5
53,1 105 05
0

1 jet It
R R2 3 R

Ip c 0 19 2
t
Jk Ri Ri

I list
x 35 19

g 21119
 X I
2x 3g t Z 13
TEE
Ly z 9
X t 32 16

I o 3 16 I o 3 16 p
Ye Rs R

3 2 12

ft 0

o
I
32 5
12

2
9
191217
2

o 32 5
2 1912 p
RFR Er I E E 1 7

s
I 12
912
o 0 0
s 1 5

RE R YR
 R R
I
ER 3h la 4
il I
R R
ER

fi g
a 2
y
O o n z E

Consisting

if 1 or a solutions consistent

if O solutions inconsistent
 2x By Z 4
x 6
Yy t 32

X 22
Fy t 10

2 3 e s

f
ftp.iigy

I 7 2 10 I 7 2 10

R Rat R
I R R R

4 h m

F
G 11 2 5
2
8
O

6
11 2

11 2 5
5 2

2
8
8

R R I R2

IF
1
o I 51
11
1111

O
0

O
I 5 71

O
11
16 11

O
 4 E
4

j Il
O 1 5 11 5
16D11 y 117 16 1

0 O

Z is
fi
o solutions what an tin solutions

fit
s zen
153 21.8

Z E R X 2 1 415 1 Z

Y 1611 5
1 z
 E u
y 12
n t
y 13

I C p
H
X 12
y
o I

after reducing we
get
f I É if
It
z is
fru w 3 2g z
g is free a 2

for what im following

I h
values k is
syst
consistent

U y h
Gx By K
 I life fill
Ree Ret GR

f
n
ka ah
O K 3h
I
med
for th
given system to have a sobeutra are

R 3h 0

5 18 5 30
Determinant scalar value that encodes how match does
the area volume of any shape change by oft
applying to
transformation

M out m

det m 4

detcon o

outer 1 3 t.at
adf.CI
nxn af
6
al

10 I
Leibniz formula

o of
 Using Ganas
flimination
of

Properties
f Determinant

def non not

square manias defined

Fgm a solution
O solution

At
only definedfor square matias

dit A dit AT

Swap two rows

sign f dit A flips

It
 if two rows an same data o

if
if R R d d
a d o

dit A B ditch dit is

dit Atr det A det B

Ax
scales a direction b o
F
det A 3

Is rotate by 300 clockwise

dit B 1
7

AB
by It axis
on x
first rotate by 30 than scale

III
 Multiplying a now with a scalar increases

the diff by that scalar

A b c

a mac a
93 C 93 by C
S

te

a a c

I fair Ita
Ilya Y Y Y a by
cans
Cs

Adding a linear combination to now

fanotm now a

does not change the detonint

R2 Rate R
A G C

f y I
z b Cz anza baab catz
as b c
as b c

dit A when a is a triangular matrix diagonal

 9 b C
a b c 9
Lana
as b
yay
c
i
f a b
ftp41 I
9 by Cs

Defiant
when air is th entries
dit A
If 4th
J the
reduced echelonfam
matrix

S IT of all scalars
that an multiplied with

k
E
f feet y
1 0 6 I
1
0 6
I 3 5
I 3 5

re Re R
R
Rat R
I 12 32

i ii t
o 1 9 0 92
912
o 72 13 2
41
 I Rs Rs E Re

It g p
0 1 g

2 I 3

l d
e
1
1

Ip'd
dit m
fan

6
45 1
4
2 5 7 70