Support Vector Machine

2553

: .
:

505261

Support Vector Machines

(
)
SVMs

SVM

SVMs

Tadee

Name

:Ms.Chalotorn

Title

: Support Vector Machines

Advisor
Keywords

: Dr.Ut Goenchanart
: statistical learning

ID

505261

Abstract
Support Vector Machines are supervised learning machines based on statistical learning
theory thatcan be used for pattern recognition and regression. Statistical learning theory can identify
ratherprecisely the factors that need to be taken into account to learn successfully certain simple
typesof algorithms , however , real-world applications usually need more complex models and
algorithms(such as neural networks), that makes them much harder to analyse theoretically.
SVMs can beseen as lying at the intersection of learning theory and practice. They construct
models that arecomplex enough (containing a large class of neural networks for instance) and yet that
are simpleenough to be analysed mathematically. This is because an SVM can be seen as a linear
algorithm in a high-dimensional space .
In this document , we will primarily concentrate on Support Vector Machines as used in
pattern recognition. In the first section we will introduce pattern recognition and hyperplane
classifiers,simple linear machines on which SVMs are based. We will then proceed to see how SVMs
are ableto go beyond the limitations of linear learning machines by introducing the kernel function,
whichpaves the way to find a nonlinear decision function. Finally, we sum it all up and mention
someareas in which Support Vector Machines have been applied and given excellent results.


CPE 489 (Seminar in Computer Engineering)
Support Vector Machines

.....
........
...
..

.
1.1 Introduction .....
1.2 ....
1.3 .....

1
1
1
1

Support Vector Machines ....

2.1 (SVMs) ...................
2.2 Formalization ...............
2.3 Primal
2.4 Dual ...
2.5 hyperplanes .......
2.6 Transductive ..

2
2
3
6
8
9
9

2.7 ........
2.8 .
2.9 ..
2.10
2.11 Multiclass SVM

10
10
11
14
18

()

.......

20

3.1 ..
3.1.1 .........
3.1.2 ...
3.1.3 ...
3.1.4 Neural Network ..

20
21
21
21
22

.. ..
4.1 ......

23
23

24

2.1
2.2.1
2.2.2
2.10.1

..............................................................................
hyperplane margin ...........................................................................................
Hyperplane ..............................................................................................................
Linear PCA .................................................................................................................

3
4
5
13

1.1. Introduction


, ,

1.2.

1.3.
Machine

,
,

2
Support Vector Machines
2.1. (SVMs)

SVM
hyperplane
hyperplanes
hyperplane datapoints



SVM

K (x, y)
hyperplanes
hyperplanes
hyperplane x
hyperplane

nstant
con

K (x, y) Y x,
X corresppoinding x

x hyperrplane
c
convoluted

p - 1 hyperplaane hyperplannes

hyperplanee

hyperrplane

hyperpllane
hypperplane

 2.1. 1

2.2. Forrmalizationn

D, n
,

1,1

C i 1
-1
p - h
hyperplane

I = 1 I = -- 1 hyyperplane

2.2.1
2 hyperpllane margin
m 2
SVM
M

1
2

http://en.wikipedia.org/wiki/Support__vector_machine
http://en.wikipedia.org/wiki/Support__vector_machine

 hyperplanne
hyperplane
h

| ||
||
b

hypperplanes

hyperplanees

hypperplanes

hyperplanes
h

||

||

||W|||

2.22.23 hyperplane

http://wwww.cs.brownn.edu/coursess/archive/20006-2007/.../Buurges98.pdf

WX

for

:
1, , )
(

|| ||

2.3. Primal

||W|| W
||W|| || || ( w
b) (QP)

1
|| ||
2
(

1, , )
1


min

, ,

1
| |
2

hyperplanes
1 0

min

max

| |
1

cx

,
:
1

2.4. Dual
unconstrained hyperplane
,

|| ||

SVM
(

1, , )

0

0

2.5. hyperplanes
hyperplane hyperplanes
hyperplanes
hyperplane B = 0

2.6. Traansductive
Transducctive
SVMs

D
|

transduuctive

:
( W , b ,

1
| |
2
(

1
1

1,1
Transductivee Vladim
mir Vapnik 1998

2.7.

SVM
Ms .

regularrization Tikhhonov

SVVM , Leisch Hornik

2.8.

1995 Corinnna Cortes Vladimir Vapnik

V

[2]
[

mislabeled
hyperplanee """"
M
Margin
hyperplane

m
misclassificat
tion

min
n ,

1
| |
2

)
1

 (2)

min , , max ,

1
|| ||
2

C

Cortes Vapnik 2008 Kanellakis ACM

2.9.
hyperplane Vladimir Vapnik 1963
1992, Boser , Guyon Vapnik
( Aizerman et al) hyperplanes

margin - hyperplane
.
hyperplane
Gaussian , Hilbert
regularized

() :
(inhomogeneous) :

Radial Basis Function :

exp

1
|

> 0

Gaussian :

exp

tanh KX X

() > 0 c <0

Dot w

 2.10.1 Linear PCAA

4
micrroarray
2
3 PCA
2D 2

(ELM

Map2D)

22D PCA

P (PCA2DD) ""

ELMap2DD
PCA2D

PCCA K -

http://en.wikipedia.org/wiki/Principaal_component_analysis

2.10.

SVM
S

,
C, ) (

(
;
)

2.10.1

SVM Vapnikk 's

2.125 Vapnik
V

y = 1
y = -- 1

http://www..springerlink..com/index/kkm7krm468022r2114.pdf

(x) ()
2.10.2
hyperplane

2.12.1 SVM6

, Lagrangian

Lagrangian .

w :

.
QP
constrains (8) hyperplane
2.10.3 SVM
SVM reformulating

http://www.springerlink.com/index/km7krm46802r2114.pdf

SVM (LS - SVM)

y i -- (w T (x i) + B)
LS - SVM

2.12.2 LS - SVM7

http://www.springerlink.com/index/km7krm46802r2114.pdf

 hypeerparamters regulaarization

LS - SVM

regressor LS - SVVM Lagrangiaan :

optimality

w e

K j = (x i T) (x j) = (x i, x j)

2.11.

Multiclass SVM

Multiclass SVM

multiclass


(i) (ONE - - all) (ii) (ONE - one) -
- (
) - one
- Max

3.1.
(Artificial neural network)
(neural network neural net)
(connectionist)
(Pattern
Recognition) ( Knowledge deduction)
(bioelectric network)
(synapses)
"" (Dendrite) input
"" (Axon) output

3.1.1.

3.1.2.
Neurons input output
input weight input neuron threshold
input output neurons
neuron

3.1.3.
Neural networks input network input weight
input neuron threshold
threshold neuron output output
input neuron network threshold output
weight threshold

pattern "back propagation"
feed-forward neural networks back-propagation
(network weight)
(output)

 input 9
3x3 3x3
1,2,3,4,6,7,8,9

neural network

3.1.4. Neural Network

(Neural Network)

( inputs outputs
inputs outputs )
()



(Model Predictive Control)



HVAC

4.1.
Support Vector Machines
Support Vector Machines 100%
Support Vector Machines

Support Vector Machines

Support Vector Machines

Support Vector Machines

http://www.support-vector.net/icml-tutorial.pdf
http://www.cs.columbia.edu/~kathy/cs4701/.../jason_svm_tutorial.pdf
http://www.svms.org/tutorials
http://www.autonlab.org/tutorials/svm.html
http://www.dtreg.com/svm.htm
http://www.cs.brown.edu/courses/archive/2006-2007/.../Burges98.pdf
http://www.support-vector-machines.org/SVM_review.html
http://www.springerlink.com/index/km7krm46802r2114.pdf