uppant Vectorachine

CsYM)
* can solve both classiicationand Regsession
poblenm.

classikicaiion SVc (Suppo&+Vecto

classiieo)

2 Regsession SVR (
Suppoct Nectos
Regsesso6 )
Some basicsS
line
Equation of

mo+C OR

Bo+Bi OR

aNtby +C 0
a coekkicient
-C
a +bo2+C 0 2 Xb/
Iniercept

'+b-O
line passes h&ouqh
Oiqin wTo =6
 matria multiplicahorp
U02

Cw b0 (uo w T6ansposco)

Equaion oftne passing 4h6ough 0igin is

P.c
ahgle g0° wehave +oLnd the
distance distance O} point
TCplane
eom he plane

distance
ine
2) and in20

angle 90°
P plane in 3D)

Distance ofo point to heplane,

distance (d) = w'P

,
11
co11

11w11.11P11 cose

where w'P w TEanspose wNecto,

loll magniude okw
 CAnit vec}o A vectos which has a magnitude o
is basicall called unit Vectos.

d 32+42
25
3
d
Cwheevectos, d= d
(11d maanitude
315,4ls) C3r5(415 s1os

unit vecto is a
way to get ocased on disection
bot on maqni fude
podvectoe
d wTP
d luwll:11P.l). cose

point above he plane

as 9<3o° 6<go0
Cose uwil always be +VC. e>g0 PaCota,to

point belou he plans, ase90

Cose willalways be NG
any point latting abovehe plane,then
must be iess han90° (+value)

anq pointaling below the plane, then

must be qfeadect. than 90 ve
vatuc)
6
ownusardvecoE

(-ve)

(+ve)
u E
E
point below heplane Pat2)E
6300
cose wil always bé t+ve'

polnt above he plane, e>6

cose will always be -ve'
 Gteomelic Intuition Bebind Suppos Vectog
Machine

Suppest Vectos_ classiRet (SVC)

maxaina plane
bestHt line
Suppo&+ Veco& manginas
points which axe plane
C&ossed by
maginal pane)
Equidistant
d should be maoxi mum
you can havemose than One Suppo@tvectoe

There willbe many_possible hypexplanes +hat sepaxate

dikeent classes.

we haVe leant in LR 4hat the p&obability ot a point

class given at vey élose
belonging to any
to the hypeeplan wi be close to o.s.

So, we want a bypeeplane hat sepexates (tve) ps

and (-ve)pts as kad auay as possible.

keykey idea o SVm Such hyperplane iscalled

magin- mamimiging plane
 Hand maxgina Sotmaxqina
Maxgina plane Line passes h6ough neaxest
points.

Had maxqinal: In hard maxgin, we will not ind

Ony. eooSS and w
any will be able to
clely sepatate all he points by
using he maqina plane.

BatinSealwdald thexe will be many overlapp

w ith many etrosS, soMarginal planetinës (witl
be called so t maxgin

margina plane should be equidistance

rom best it line
SVM Mathematica lnuiion

Ave

-Ve

sTotb ao'+b 0

w+bz+1
w'o+bE +1

fori
-s (-X2)
2fb:-1 2

w+b-0 unit vecto8 o

d=distance should
1-2
w
be maximum 2
llwl

distance (d)=_w (1-) 2

 Cos Funclions

we bave to mamimi2e the valuc Ok2

the valucs oh w,b
by changing
2 distance bedoeen maHqinad plane.

consttaint such that i

eAsdtteps
fosall classiked cossect points,

constaints> qi T+b)1

macimize 2 minimie 1lwll lossfunchon

w,b w,b

minimia

lossunchion o cus On minimization

COst uncion

minimiae by cha
2
min 1ull
2 C Hinge loss
3o makqin.
o&

ohere, Gi Hoomany points uwe can igno6e

to mi-classikcalion.
Hypeepakamelee
CiCeta): Summalion o he distance
o incos 6ect data points k6om
Ahe maeginal plane.

Suppoet Vecfo& Kegsesso&:

Psoblem Stadement 3asedonthe siae o the house,

we have topeedict psiGe o
Psice
he house
w+b+€
cw b
wx+b-6
maxqinal
:epsilo
eto
Sig
 CostPunctio:

minimige oll
2
Ci
sb Hinqe loss

mAE
Etaoi
consitaint:yiwixis +Ei

TEuth point peilon

pedicted
Point
E: magin euof
of
Eieutds above he
(todecide osiginas plane)
E
maxgin.

Hyperparameter

keep adijusing E
+o qetbest mostgin
wecant say
imcosseat point in
Segsesso&.
 Peice w+bt6.w +'b

w+b -E

1 pcdicted.

sige

In Regsessof, no compleie incossect Value,

In because it willbe coninuouS Valuue.

s Sym impacked by he outiexs5

Yes, 3VM isimpacked by the ouHiers.

Does Standadizaton isneed in svm?

Yes, eneed 4o perhosm No&mal izaton
and standakdi2aion.
SVm kexnel

2D
kennels
 VM Kenels nean 5VC
SVmkernel

o4a
mea
Sepetable dat
inear Scperable data.

when we cceale this (O)ype o

bestkt
line

and maginas plane we ae

acualy
Solving heinea Sepeable daka.

called as Linea svc,(Fig@)

Idata is not a linea Seperable daa, qouaill

not bcabté to ceeate best kH ine and not able
to CSeate'a magin al plane even though
we cseate it heaccuracy win be verylocu
(fiq2)
Fo his typc opfoblcms, we have some
moe SVMketnels.
 obat does sVm ketnels do some tramslosmahan
The main aim is toaPply
dechnique (Some mahemaltcal \osmeda)
on the dataset

the dimension
Thisanslo&mation incteases
he daa. (mahemaical kasmala

T6anskofmaton Increasing the

SVMKexnels dimension olthe data,

:iesto
line
lincqe sepekable
>willgive eixo& :

we will ranshocm thedata trom 10 20:!

ao2
Ae
Now, we can use lineam

Seperable line.
 So
2-3
= =7 49
9 and soon:

wbat isthe advantage o doing 4his ranslosmalion3

ae ansosmation)uwe can apply tinead

3vm O6 SVC.
*when wedide Conveet D 20hen wc
can divide all 4he points using single tine
which is called Linea svc

ypes o SVM kernel

1 Polynomia kerned
2. RBF kexne
Siamoid
3. kexnel

keene
2
olunomial

A not seperable wit

best kit line.

S0,we ne edto
ConveNt 20 3D.

Fig 3
 Top vic ow
Owe ain aim waS to
înceasC dimcnsion
20 3B0
<
30, hypexplane is magina
CEeated Jplame

best
itlinc

Fomula
1a Jos Polynomial kcrnel Fia

C,)=( 2+1
d dimension
I
we axeconvekting 2030, hevalue of d3.

2
2|
2, 2
:.
2

:
3 unique valucs 2,

ow, iniiially
at the time o Fig 3 we have:3 caara

dee (i
 cSe
2e
No, akHe, the Yanskosmaton
polymomia kenel

Heuu
we have 6

2
lcatweS cam be
1o6mewa
Jeatuwres

plotted
ob

as he30
a4: which means that

and once we have

wiwill be
2

all
be
uoill
wilt be 22

hese poinis, uwe

hc points.
will be
ablc to clealy sepeeale

use polynomial keenel, t qet bettes accumacy

(2 Radiad
Radia Basis Function Kexne (RBF Keenel)

-11-f
K(,)= e 22
hype paUramctet
Sigmold kexne
Hcan be used as he psooyo6 necueo
netO&ks.
k(o,) =tanb(6xxit