chap13

© All Rights Reserved

0 views

chap13

© All Rights Reserved

- KPI Dashboard - Excel Model
- AI
- 051021
- FDheat
- Numerical Analysis
- gfjfjf
- Sampling
- assgn
- lec10svm
- XORを学習させてみる。
- univ of raj
- Maxflow FF
- 4th Quadratic Factorisation & Solving Worksheet
- Cg 33504508
- Program c#
- Applied Numerical Methods
- Chapter 6
- Informe_Estadistico_final2[1]
- Result of Solution Using Gauss-Jordan Elimination
- Array Lists

You are on page 1of 27

INTRODUCTION

TO

MACHNE

LEARNNG

3RD EDTON

ETHEM ALPAYDIN

The MIT Press, 2014

alpaydin@boun.edu.tr

http://www.cmpe.boun.edu.tr/~ethem/i2ml3e

CHAPTER 13:

KERNEL MACHNES

Kernel Machines

3

first

Define the discriminant in terms of support vectors

The use of kernel functions, application-specific

measures of similarity

No need to represent instances as vectors

Convex optimization problems with a unique solution

Optimal Separating Hyperplane

4

if C1

X x , r t where r

t

t t t 1 x

1 if x t

C 2

find w and w0 such that

w T xt w0 1 for r t 1

w T xt w0 1 for r t 1

which can be rewritten as

r t w T xt w0 1

Margin

5

on either side

Distance of x to the hyperplane is w T xt w0

w

r t w T xt w0

We require , t

w

min w subject to r t w T xt w0 1, t

1 2

2

Margin

6

min w subject to r t w T xt w 0 1, t

1 2

2

Lp w t r t w T xt w 0 1

N

1 2

2 t 1

w r w x w 0 t

N N

1 2 t t T t

2 t 1 t 1

Lp N

0 w t r t xt

w t 1

Lp N

0 t r t 0

w 0 t 1

7

Ld w w w T t r t xt w0 t r t t

1 T

2 t t t

w w t

1 T

2 t

r r x x t

1 t s t s t T s

2 t s t

subject to t r t 0 and t 0, t

t

Most t are 0 and only a small number have t >0; they are

the support vectors

8

Soft Margin Hyperplane

9

r t w T x t w0 1 t

Soft error

t

t

New primal is

1

2

2

Lp w C t t t t r t w T x t w0 1 t t t t

10

Hinge Loss

11

0 if y t r t 1

Lhinge (y , r )

t t

1 y t t

r otherwise

n-SVM

12

1 1

min w - n t

2

2 N t

subject to

r t w T x t w 0 t , t 0, 0

Ld r r x x

1 N t s t s t T s

2 t 1 s

subject to

1

t t t

r 0 ,0 t

N t

, t

n

Kernel Trick

13

z = (x) g(z)=wTz

g(x)=wT (x)

The SVM solution

w t r t z t t r t xt

t t

gx w x r x

T t t

x

t T

gx t r t K xt , x

t

Vectorial Kernels

14

Polynomials of degree q:

K x , x x x 1

t T t q

K x, y xT y 1

2

x1y1 x 2 y 2 12

1 2 x1y1 2 x 2 y 2 2 x1 x 2 y1y 2 x12 y12 x 22 y 22

x 1, 2 x1 , 2 x 2 , 2 x1 x 2 , x , x 2

1

2 T

2

Vectorial Kernels

15

Radial-basis functions:

xt x 2

K xt , x exp

2s 2

Defining kernels

16

Kernel engineering

Defining good measures of similarity

String kernels, graph kernels, image kernels, ...

Empirical kernel map: Define a set of templates mi

and score function s(x,mi)

(xt)=[s(xt,m1), s(xt,m2),..., s(xt,mM)]

and

K(x,xt)= (x)T (xt)

Multiple Kernel Learning

17

K x, y K1 x, y K 2 x, y

K x, y K x, y

1 2

m

K x , y i K i x, y

i 1

t s r t r s i K i xt , x s

1

Ld t

t 2 t s i

g(x) t r t i K i xt , x

t i

t i

Multiclass Kernel Machines

18

1-vs-all

Pairwise separation

Error-Correcting Output Codes (section 17.5)

Single multiclass optimization

1 K

min w i C it

2

2 i 1 i t

subject to

w zt T xt w zt 0 w i T xt wi 0 2 it , i z t , it 0

SVM for Regression

19

f(x)=wTx+w0

Use the -sensitive error function

if r t f xt

e r , f x t

t t 0

r f x t

otherwise

min w C t t

1 2

2

t

r t w T x w0 t

w x w r

T

0

t

t

t , t 0

20

Kernel Regression

21

Kernel Machines for Ranking

22

but at least +1 unit margin.

Linear case:

1

min w i C it

2

2 t

subject to

w T xu w T xv 1 t , t : r u r v , it 0

One-Class Kernel Machines

23

min R 2 C t

t

subject to

x t a R 2 t , t 0

Ld x x r r x x

N

t t T s t s t s t T s

t t 1 s

subject to

0 t C , t 1

t

24

Large Margin Nearest Neighbor

25

D(xi, xj)=(xi-xj)TM(xi-xj)

For three instances i, j, and l, where i and j are of

the same class and l different, we require

D(xi, xl) > D(xi, xj)+1

and if this is not satisfied, we have a slack for the

difference and we learn M to minimize the sum of

such slacks over all i,j,l triples (j and l being one of k

neighbors of i, over all i)

Learning a Distance Measure

26

similar approach where M=LTL and learns L

Kernel Dimensionality Reduction

27

PCA on the

kernel matrix

(equal to

canonical PCA

with a linear

kernel)

Kernel LDA, CCA

- KPI Dashboard - Excel ModelUploaded bypeterd87
- AIUploaded byChristin Swanson
- 051021Uploaded bysriashokcute
- FDheatUploaded byਹਰਸਿਮਰਨ ਸਿੰਘ
- Numerical AnalysisUploaded byzidaaan
- gfjfjfUploaded byRizky Putra Affandi
- SamplingUploaded byRemya Sree
- assgnUploaded byniket_shah15
- lec10svmUploaded byWanChien Tan
- XORを学習させてみる。Uploaded byAkira Kobashi
- univ of rajUploaded byChitrangi Sharma
- Maxflow FFUploaded byNurkholismath
- 4th Quadratic Factorisation & Solving WorksheetUploaded byHema Bhaskar
- Cg 33504508Uploaded byAnonymous 7VPPkWS8O
- Program c#Uploaded byShanmuga Sundaram Chellam
- Applied Numerical MethodsUploaded byJustin White
- Chapter 6Uploaded bysmartlife0888
- Informe_Estadistico_final2[1]Uploaded byAlexander Cieza
- Result of Solution Using Gauss-Jordan EliminationUploaded byTundeOyedotun
- Array ListsUploaded byEd Z
- Genetic AlgorithmUploaded bySayali
- 11Uploaded byGaganVishwakarma
- divetimperaUploaded byhoolap
- Amit Konar, Diptendu Bhattacharya-Time-Series Prediction and Applications. a Machine Intelligence Approach-Springer (2017)Uploaded byAnca Vochescu
- Bank Queuing Problem_Group 10Uploaded bypragatigupta14
- P3 Pengaruh Volatilitas Laba SNA LampungUploaded byanon_780898691
- Problems 7Uploaded bysoumyadeepta
- ANOVA Management Efficiency AnalysisUploaded byAmitava Dey
- Assignment 2(17MAT41)Uploaded byLucky Lakshmi
- Frame EUploaded byEve

- i2ml3e-chap1.pdfUploaded byvarun3dec1
- 1st May Puleet BrochureUploaded byvarun3dec1
- Admissiongitiw ITI QuotaUploaded byvarun3dec1
- Chapter 6- Chd Admin Institutions-rev-CCETUploaded byvarun3dec1
- Stochastic Gradient Descent - Mini-batch and More - Adventures in Machine LearningUploaded byvarun3dec1
- Rights of Persons With Disabilities 5 PerUploaded byvarun3dec1
- D(Res-II)-DESWUploaded byvarun3dec1
- Anti Ragging _ Ragging in College _ Anti Ragging AffidavitUploaded byvarun3dec1
- CFP12321Uploaded byvarun3dec1
- Conv Neural NetsUploaded byArannya Monzur
- Kashmiri Migrant Press Note ChdUploaded byvarun3dec1
- Wear TableUploaded byvarun3dec1
- com_instUploaded byvarun3dec1
- Rotation in Govt College of ArtsUploaded byvarun3dec1
- CN 123121.pdfUploaded byvarun3dec1
- 9780262028189_TOC11.pdfUploaded byvarun3dec1
- Google NetUploaded byNitin Panj
- ISTC Admission-2017 Imp DatesUploaded byvarun3dec1
- Beamer LogoUploaded byrghome
- Word2Vec Tutorial - The Skip-Gram Model · Chris McCormickUploaded byvarun3dec1
- 1506.00019.pdfUploaded bypreethamat208815
- lrec_skipgramsUploaded byvarun3dec1
- Python TensorFlow Tutorial - Build a Neural Network - Adventures in Machine LearningUploaded byvarun3dec1
- LSTM PaperUploaded byvarun3dec1
- i2ml3e-chap6Uploaded byvarun3dec1
- i2ml3e-chap9.pptxUploaded byvarun3dec1
- i2ml3e-chap12Uploaded byvarun3dec1
- i2ml3e-chap13Uploaded byvarun3dec1

- Antithetic Markov Chain Monte Carlo Algorithm and Computing Dominant Eigenpair: Variance Reduction ApproachUploaded byTI Journals Publishing
- MTH3011 Exercise Sheet 03 2015Uploaded byjeff
- viewcontent.cgi.pdfUploaded bymakanbhupindersingh
- Simplex Method - Maximisation CaseUploaded byJoseph George Konnully
- lebesgueUploaded bySainath Bharadwaj
- Basic Hypergeometric SeriesUploaded byrgkelly62
- Meshfree Chapter 15Uploaded byZenPhi
- Analysis of VarianceUploaded by03435013877
- 2.0 PEMBELAJARAN GEOMETRI_Nota Ringkas Kuliah 2Uploaded byQuek Quekquek
- Exam GuideUploaded byRajesh Khanna
- Image Registration Using Log Polar Transform and Fft Based Scale InvariantUploaded byAnonymous 7VPPkWS8O
- Statistical Process Control & Software Reliability Trend an Analysis Based on Inter Failure Time DataUploaded byNavneet
- LfnewtonUploaded byGopi
- MT261tutorial3Uploaded byGilbert Furia
- statistics group projectUploaded byapi-384638689
- LaboratoryValidationDefinitionsAndTerminologyUploaded byasad bashir
- business research methodUploaded byabhijeet108
- Algebraic Inequalities in Math OlympiadsUploaded byryszard_lubicz
- Laplace Table ProofsUploaded byJom Agullana
- Elements of the Differential and Integral Calculus - W. GranvilleUploaded byAnthony Chiew Han Yang
- ND Mathematical Methods Lecture notesUploaded byucaptd3
- Taylor Series and Numerical MethodsUploaded byGeorge Ezar N. Quiriado
- Library GenesisUploaded byribporto1
- pr_l6(1)Uploaded bycrennydane
- MITRES_6_007S11_lec08Uploaded byAnkit Anand
- 1. Statistical AnalysisUploaded byVivian Lam
- f12booklistUploaded byWill Black
- Methods in Case Study Analysis by Linda t KohnUploaded byallenchiew
- Mathematics Paper 2 HLUploaded byVíctor Calderón Callao
- GATE ME SP 2014 by NodiaUploaded byRahul Kumar

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.