3.

Examples of SVM:

Example 1:

Consider the following posititvely labelled data points in R2 :

3 , 3 , 6 , 6
{( ) ( ) ( ) ( )}
1 −1 1 −1

And following are negatively labelled data points in R2 :

1 , 0 , 0 , −1
{( ) ( ) ( ) ( )}
0 1 −1 0

Figure 13: Sample data points in R2. Blue diamonds are positive and red squares are negative examples

From the figure 13 it has been observed that there are three support vectors

x1 = 1 , x2 = 3 , x 3= 3
{ ( ) ( ) ( )}
0 1 −1

Vectors are augmented with a 1 as a bias input, so if x 1=( 10 ) ,it now becomes x 1=( 101 )and
similarly x 2=( 311 ) and x 3=( 3−11 ) .

Using the objective function

l l l
1
W(α)=∑ α i - ∑ ∑ αi α j y i y j K ( ⃗x i , ⃗x j ) (a)
i=1 2 i=1 j=1

Here, the kernel function is identity. Therefore, K ( ⃗x i , ⃗x j ) =⃗xi ∙ ⃗x j

 3 3 3
1
W(α)=∑ α i – ∑ ∑ αi α j y i y j ⃗x i ∙ ⃗x j (b)
i=1 2 i=1 j=1

3
= ∑ αi− 12 ¿+α 1 α 2 y 1 y 2 ( ⃗x 1 ∙ ⃗x 2 ) +…+ α3 α3 y 3 y 3 (⃗x 3 ∙ ⃗x 3 )¿
i=1

(c )

1 3
x
⃗ ∙ x
⃗
( 1 1) = [ 1 01 ]
[]
0 =2
1
x
⃗ ∙ x
⃗
( 1 2) = [ 1 01 ] 1 =4
1 []
Similarly,

( ⃗x 1 ∙ ⃗x 3 ) =4 ( ⃗x 2 ∙ ⃗x 2 ) =1 ( ⃗x 2 ∙ ⃗x 3 ) = 9 ( ⃗x 3 ∙ ⃗x 3 ) =11

Substituting these values in equation (c ), we will get

3
1 2 2 2
W(α)=∑ α i− [2 α 1−4 α 1 α 2−4 α 1 α 3 −4 α 1 α 2 +11α 2+ 9 α 2 α 3 −4 α 1 α 3 +9 α 2 α 3 +11α 3 ]
i=1 2
(d)

1
= α 1 +α 2+α 3 −¿ [2 α 21 −8 α 1 α 2−8 α 1 α 3+11 α 22 +18 α 2 α 3+11 α 23 ]
2
(e)

Maximize objective function of equation (e ) with respect to α

−1
w.r.to α 1: 1 ¿ -8 α 2−¿8α 3 ¿=0
2

2 α 1 -4 α 2 −¿4α 3=1 (f)

w.r.to α 2:4 α 1 -11α 2−9 α 3=−1 (g)

w.r.to α 3:4 α 1 -9 α 2−¿11α 3=−1 (h)

By adding equation (g) and (h) we get,

α 2=α 3 (i)
Put value of α 2 from equation (i) into equation (f), we get

1
α 1 -4 α 2 = (j)
2

Multiply equation (g) with 9 and (h) with 11, and subtracting them, we get

4 α 1 −¿20α 3=−1 (k)

Since α 2=α 3 , equation (k) becomes

4 α 1 −¿20α 2=−1 (l)

Multiply equation (j) with 4 and add with (l), we get

α 2=0.75=α 3 and α 1 = -3.5 (m)

Now that we have the α i values, we can find the hyperplane that discriminates the positive
from the negative examples.

l
w = ∑ αi y i ⃗
⃗ xi
i=1

1 3 3
[] [] [ ]
= -3.5 0 + 0.75 1 + 0.75 −1
1 1 1

1
= 0
−2[] (n)

w is the value of bias b.

Since, the vectors are augmented with bias, the last entry in vector ⃗
w = [10]
i.e. b=-2 and ⃗

Now, the hyperplane equation is

w . ⃗x + b = 0
⃗ (o)

Since, the dataset is two-dimensional, the above equation becomes as,

 w 1 x 1 +w2 x 2+ b=0 . (p)

w and b
Substituting values of ⃗

1 ∙ x 1 +0 ∙ x 2−2=0

x 1=2 (q)

This is the equation for hyperplane. Plotting of same gives the expected decision surface.

Figure 14: shows the hyperplane for x=2

Use of Machine:

For example, we want to classify the point x=(4 1), then

f (x)=sgn(⃗x . ⃗
w + b) (r )

f ( 4 )=sgn( 1 . [ 4 1 ] −2)
[]1 0[] ¿ sgn(4 +0−2) ¿ sgn(+2)

f 4 =Positive Class
([ ])
1
(s)