0 Up votes0 Down votes

4 views36 pagesPresentation about SVM

Apr 01, 2016

Support vector machin, an excellent tool

© © All Rights Reserved

PPT, PDF, TXT or read online from Scribd

Presentation about SVM

© All Rights Reserved

4 views

Support vector machin, an excellent tool

Presentation about SVM

© All Rights Reserved

- SUPERVISED FEATURE SELECTION FOR DIAGNOSIS OF CORONARY ARTERY DISEASE BASED ON GENETIC ALGORITHM
- Machine Learning Workflow eBook
- Predicting Movie Success Based on IMDB Data
- Chang Liang et al- Scaling up Kernel Grower Clustering Method for Large Data Sets via Core-sets
- IJETTCS-2013-10-23-061
- IJAIEM-2014-03-29-121
- Hc 3612711275
- p09_1028_9768
- A SURVEY OF SENTIMENT CLASSSIFICTION TECHNIQUES
- Chapter 6
- Privacy Preservation Techniques in Data Mining
- 2013 Ieee Paper
- A6DE496Ed01
- finalpaper1.pdf
- A support vector machine classifier with rough set-based feature selection for breast cancer diagnosis
- Isha Poojavarsha
- Visual Tracking Using HOG and SVM
- 11
- FuChenWei-HongKongStockIndexForecasting
- Text Content Dependent Writer Identification

You are on page 1of 36

Vector Machines

Martin Law

Outline

History of support vector machines (SVM)

Two classes, linearly separable

How to make SVM non-linear: kernel trick

Demo of SVM

Epsilon support vector regression (-SVR)

Conclusion

08/11/05

Law

History of SVM

SVM is a classifier derived from statistical

learning theory by Vapnik and Chervonenkis

SVM was first introduced in COLT-92

SVM becomes famous when, using pixel maps

as input, it gives accuracy comparable to

sophisticated neural networks with elaborated

features in a handwriting recognition task

Currently, SVM is closely related to:

reproducing kernel Hilbert space, Gaussian process

08/11/05

Law

Separable Case

Many decision

boundaries can

separate these

two classes

Which one should

we choose?

Class 2

Class 1

08/11/05

Law

Boundaries

Class 2

Class 1

08/11/05

Class 2

Class 1

Law

Should Be Large

away from the data of both classes as

possible

Class 2

Class 1

08/11/05

m

CSE 802. Prepared by Martin

Law

{1,-1} be the class label of xi

points correctly

A constrained optimization problem

08/11/05

Law

problem

w can be recovered by

08/11/05

Law

w is a linear combination of a small number of data

Sparse representation

The decision boundary is determined only by the SV

Let t (j=1, ..., s) be the indices of the s support

j

vectors. We can write

Compute

and

classify z as class 1 if the sum is positive, and class 2

otherwise

08/11/05

Law

A Geometrical Interpretation

Class 2

8=0.

6

10=0

5=0

4=0

9=0

Class 1

08/11/05

7=0

2=0

1=0.8

6=1.4

3=0

CSE 802. Prepared by Martin

Law

10

Some Notes

error on unseen data for SVM

The larger the margin, the smaller the bound

The smaller the number of SV, the smaller the

bound

data are referenced only as inner product,

xTy

08/11/05

Law

11

Separable

Class 2

Class 1

08/11/05

Law

12

We want to minimize

theory

margin

08/11/05

Law

13

w is also recovered as

The only difference with the linear separable

case is that there is an upper bound C on i

i

08/11/05

Law

14

Boundary

dimensional space to make life easier

Feature space: the space of (xi) after

transformation

Why transform?

Linear operation in the feature space is

equivalent to non-linear operation in input

space

The classification task can be easier with a

proper transformation. Example: XOR

08/11/05

Law

15

Boundary

good estimate

simultaneously

Kernel tricks for efficient computation

Minimize ||w||2 can lead to a good

( ) classifier

( )

(.)

08/11/05

( )

( ) ( ) ( )

( )

( )

( )

( ) ( )

( ) ( )

( )

( ) ( )

( )

( )

Input space

Law

16

Example Transformation

without going through the map (.)

08/11/05

Law

17

Kernel Trick

K and the mapping (.) is

(.) indirectly, instead of choosing (.)

Intuitively, K (x,y) represents our desired

notion of similarity between data x and y and

this is from our prior knowledge

K (x,y) needs to satisfy a technical condition

(Mercer condition) in order for (.) to exist

08/11/05

Law

18

networks

different applications is very active

08/11/05

Law

19

Handwriting Recognition

08/11/05

Law

20

Function

Change all inner products to kernel

functions

For training,

Original

With

kernel

function

08/11/05

Law

21

Function

class 1 if f 0, and as class 2 if f <0

Original

With

kernel

function

08/11/05

Law

22

Example

class 1 and 4, 5 as class 2 y1=1, y2=1, y3=-1,

y4=-1, y5=1

K(x,y) = (xy+1)2

C is set to 100

08/11/05

Law

23

Example

The support vectors are {x =2, x =5, x =6}

2

4

5

and all give b=9

08/11/05

Law

24

Example

Value of discriminant function

class 1

08/11/05

class 1

class 2

2

Law

25

Multi-class Classification

SVM is basically a two-class classifier

One can change the QP formulation to allow

multi-class classification

More commonly, the data set is divided into

two parts intelligently in different ways and

a separate SVM is trained for each way of

division

Multi-class classification is done by combining

the output of all the SVM classifiers

Majority rule

Error correcting code

Directed acyclic graph

CSE 802. Prepared by Martin

08/11/05

Law

26

Software

A list of SVM implementation can be found

at http://www.kernelmachines.org/software.html

Some implementation (such as LIBSVM)

can handle multi-class classification

SVMLight is among one of the earliest

implementation of SVM

Several Matlab toolboxes for SVM are also

available

08/11/05

Law

27

Prepare the pattern matrix

Select the kernel function to use

Select the parameter of the kernel function

and the value of C

software, or you can set apart a validation set to

determine the values of the parameter

the i

Unseen data can be classified using the i

and the support vectors

08/11/05

Law

28

Demonstration

08/11/05

Law

29

SVM

Strengths

Tradeoff between classifier complexity and error

can be controlled explicitly

Non-traditional data like strings and trees can be

used as input to SVM, instead of feature vectors

Weaknesses

08/11/05

Law

30

(-SVR)

Unlike in least square regression, the error

function is -insensitive loss function

Intuitively, mistake less than is ignored

This leads to sparsity similar to SVM

Penalty

08/11/05

Penalty

Value off

target

CSE

802. Prepared by Martin

Law

Value off

target

31

(-SVR)

values {u1, ..., un}, we want to do -SVR

quadratic programming

problem

CSE 802. Prepared by Martin

08/11/05

Law

32

(-SVR)

influence of the error

The ||w||2 term serves as controlling the

complexity of the regression function

values of i and i*, which are both zero if

xi does not contribute to the error function

08/11/05

Law

33

A lesson learnt in SVM: a linear algorithm

in the feature space is equivalent to a

non-linear algorithm in the input space

Classic linear algorithms can be

generalized to its non-linear version by

going to the feature space

independent component analysis, kernel

canonical correlation analysis, kernel k-means,

1-class SVM are some examples

08/11/05

Law

34

Conclusion

SVM is a useful alternative to neural

networks

Two key concepts of SVM: maximize the

margin and the kernel trick

Many active research is taking place on

areas related to SVM

Many SVM implementations are available

on the web for you to try on your data set!

08/11/05

Law

35

Resources

http://www.kernel-machines.org/

http://www.support-vector.net/

http://www.support-vector.net/icml-tutorial

.

pdf

http://www.kernel-machines.org/papers/tuto

rial-nips.ps.

gz

http://www.clopinet.com/isabelle/Projects/

SVM/applist.html

08/11/05

Law

36

- SUPERVISED FEATURE SELECTION FOR DIAGNOSIS OF CORONARY ARTERY DISEASE BASED ON GENETIC ALGORITHMUploaded byCS & IT
- Machine Learning Workflow eBookUploaded byNguyen Thi Hoang Giang
- Predicting Movie Success Based on IMDB DataUploaded byIntegrated Intelligent Research
- Chang Liang et al- Scaling up Kernel Grower Clustering Method for Large Data Sets via Core-setsUploaded byTuhma
- IJETTCS-2013-10-23-061Uploaded byAnonymous vQrJlEN
- IJAIEM-2014-03-29-121Uploaded byAnonymous vQrJlEN
- Hc 3612711275Uploaded byAnonymous 7VPPkWS8O
- p09_1028_9768Uploaded byDuyệt Trần
- A SURVEY OF SENTIMENT CLASSSIFICTION TECHNIQUESUploaded byJournal 4 Research
- Chapter 6Uploaded bysmartlife0888
- Privacy Preservation Techniques in Data MiningUploaded byInternational Journal of Research in Engineering and Technology
- 2013 Ieee PaperUploaded byVinod Thete
- A6DE496Ed01Uploaded byItalo Chiarella
- finalpaper1.pdfUploaded byrupaj_n954
- A support vector machine classifier with rough set-based feature selection for breast cancer diagnosisUploaded byAbdul Rahman
- Isha PoojavarshaUploaded byPooja Angurala
- Visual Tracking Using HOG and SVMUploaded byijaert
- 11Uploaded byGaganVishwakarma
- FuChenWei-HongKongStockIndexForecastingUploaded byManikantan Thanayath
- Text Content Dependent Writer IdentificationUploaded byesatjournals
- PMBedUploaded bydebrarinn12
- Support Vector Machine Big NotesUploaded by1balamanian
- EVALUATION OF TEXTURE AS AN INPUT OF SPATIAL CONTEXT FOR MACHINE LEARNING MAPPING OF WILDLAND FIRE EFFECTSUploaded bysipij
- Fraud AnalyticsUploaded byPrashanth Mohan
- Emoticon Smoothed LanguageUploaded byparis odell
- Cancer Prediction and Prognosis Using Machine Learning TechniquesUploaded byInternational Journal of Innovative Science and Research Technology
- Format Laporan Akhir Insinas 2017 Tahun TerakhirUploaded bySandy
- Instance basedUploaded byHoàng Nam
- ArticleUploaded byVaqar Sayyed
- Mathematical Programming in Machine Learning and DUploaded byTran Trung

- Op Research for Mgmt DecitionUploaded byjrcg0914
- CI ManagerUploaded byjrcg0914
- 315-Article Text-319-1-10-20190505Uploaded byjrcg0914
- Guide-5SP_12.28.17Uploaded byAndré Luiz Lima
- costUploaded bymy_khan20027195
- El aprendizaje significativo de AusubelUploaded byjrcg0914
- introductory-probability-theory.pdfUploaded byjrcg0914
- Sci.pdfUploaded byjrcg0914
- Importance of Data Collection and Validation for Systematic Software Development ProcessUploaded byAnonymous Gl4IRRjzN
- Operational Performance ManagementUploaded byjrcg0914
- 2013 Beven. Simulation MeaningUploaded byjrcg0914
- Quality Diagn ProblUploaded byjrcg0914
- Case Study_reduced-defects-sales-growth-3m.pdfUploaded byavinash_k007
- Root Cause CaseUploaded byjrcg0914
- Spc for EverydayUploaded byjrcg0914
- Carleton. Improving the software processwith SPC.pdfUploaded byjrcg0914
- P diagram - art.pdfUploaded byjrcg0914
- milstd_1629A -04- fmeca.pdfUploaded byjrcg0914
- Ford Machinery f Me AUploaded bygdj1973
- Beer Brewing PfmeaUploaded byjrcg0914
- Aztec Autumn by Gary Jennings- several reviews - Copy-part II.docxUploaded byjrcg0914
- Fu IeldsUploaded byjrcg0914
- Valentine´s day.docxUploaded byjrcg0914
- Fisica - Serway - Cap 1Uploaded bySebastián Caro Soto
- McKinsey - Have You Tested Your Strategy LatelyUploaded byNadia Rosas

- A Comparative study on Classification and Clustering Techniques Using Assorted Data Mining ToolsUploaded byIJAFRC
- image denoisingUploaded byEmon Khan
- 07 Chapter 3Uploaded byAnuranjani Dhivya
- A Mixture-Of-modelers Approach to Forecasting NCAA Tournament OutcomesUploaded byBartoszSowul
- w13374.pdfUploaded bycastjam
- CBOs&NGOsUploaded bymkshri_in
- Problem Set 8Uploaded byJonahJunior
- An Overview of Trolling StrategiesUploaded bymaleficetorrent
- The Comparison of GEV Log-Pearson Type 3 and Gumbel DistributionUploaded byrajesh005
- LedolterHoggChapter5Uploaded byprogineral
- Student Resources.pdfUploaded byMuffet Lkr
- Stochastic Simulation IntroductionUploaded byTuany R. Cassiano
- Deep Learning for Time Series ModelingUploaded byFelipe Angel
- DIS_ch_7.pptxUploaded byVictoria Liendo
- Normal Distribution bUploaded byKarim Qasim
- Calculation of Optimum Fishing TimeUploaded byÇağlar Öztürk
- Unit II Evolution of TqmUploaded bysbmallur
- ARDL paperUploaded byMd Fouad Bin Amin
- microinsurance in china.pdfUploaded bygaconconcon
- sample surveyUploaded byApam Benjamin
- Tri Thai_GSA-2FAdditional - Use of Data Analytics for Effective Program Oversight_CLPUploaded bydummy yummy
- Tripos GuideUploaded byShivam Kumar
- Crash Course in Analytics for Non Analytics managersUploaded byUzair Faruqi
- A Comparative Study of Data ClusteringUploaded byKosa
- hw6Uploaded bymelankolia370
- Ps and Solution CS229Uploaded byAnonymous COa5DYzJw
- Problem_Set1.pdfUploaded byAnubhavAgarwal
- BTech II Sem Syllabus of Civil Engg Mechanical Engg Etc R16Uploaded bylosafer
- Erbacher Vita 2010Uploaded byHuntM
- SSRN_ID2494065_code1899177.pdfUploaded byn2oh1

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.