Support Vector Machine (SVM)

Support Vector Machine (SVM) is a supervised machine learning algorithm

used for both classification and regression. However, it is mostly used in
classification problems.
In the SVM algorithm, we plot each data item as a point in n-dimensional
space Then, we perform classification by finding the hyper-plane that
differentiates the two classes very well.
Support Vectors
These data points are closest to the hyperplane. These are the critical
elements. Since removing them may alter the position of the dividing
hyperplane.
Hyperplane
The hyperplane is a line which linearly divides and classifies the data.
When we add the new testing data, whatever side of the hyperplane it
goes will eventually decide the class that we assign to it.
Margin is the distance between the hyperplane and the closest point from
either set.

The objective of SVM algorithm is to find a hyperplane in an N-

dimensional space that distinctly classifies the data points. The dimension
of the hyperplane depends upon the number of features.
 • If the number of input features is two, then the hyperplane is
just a line.
• If the number of input features is three, then the hyperplane
becomes a 2-D plane.
• It becomes difficult to imagine when the number of features
exceeds three.

Selecting the best hyper-plane:

In case of linear SVM:

One reasonable choice as the best hyperplane (L2) is the one that
represents the largest separation or margin between the two classes. So,
we choose the hyperplane whose distance from it to the nearest data
point on each side is maximized. If such a hyperplane exists it is known
as the maximum-margin hyperplane/hard margin.
The SVM algorithm has the characteristics to ignore the outlier and finds
the best hyperplane that maximizes the margin. SVM is robust to outliers.
In these cases, SVM finds maximum margin as done with previous data
sets along with that it adds a penalty each time a point crosses the
margin. So, the margins in these types of cases are called soft margin.

In case of non-linear SVM:

When we cannot separate data with a straight line we use Non – Linear
SVM. In this, we have Kernel functions. They transform non-linear
spaces into linear spaces. It transforms data into another dimension so
that the data can be classified.
The SVM kernel is a function that takes low dimensional input space and
transforms it into higher-dimensional space, i.e it converts not separable
problem to separable problem. It is mostly useful in non-linear separation
problems.

It transforms two variables x and y into three variables along with z. the
new variable y is created as a function of distance from the origin.
Therefore, the data have plotted from 2-D space to 3-D space. Now we
can easily classify the data by drawing the best hyperplane between
them.

Advantages of SVM:
1. Effective in high dimensional cases
2. Its memory efficient as it uses a subset of training points in the
decision function called support vectors
3. It works really well with a clear margin of separation
4. It is effective in high dimensional spaces.
5. It is effective in cases where the number of dimensions is greater
than the number of samples.
Cons:
1. It doesn’t perform well when we have large data set because the
required training time is higher
2. It also doesn’t perform very well, when the data set has more noise
i.e., target classes are overlapping

Linear vs non Linear SVM

Applications of SVM
1. Sentiment analysis.
2. Spam Detection.
3. Handwritten digit recognition.
4. Image recognition challenges