SVM - ANN

Introduction To SVM:
They are powerful machine learning algorithms which are used for
classification and regression.
They are mostly used for classification problems.
They have their way of implementation when compared to other
algorithms.
They are popular because they are able to handle continuous and
categorical variables.
Working Of SVM:
It is a representation of different classes in a hyperplane in multi-dimensional
space.
The hyper-plane will be generated in an iterative manner by SVM so that error
can be minimized.
The main aim of SVM is to divide the datasets into classes to find the
maximum marginal hyperplane(MMH).
The following are important concepts in SVM:
Support Vectors - Datapoints that are closest to the hyperplane are called
support vectors.
Separating line will be defined with the help of these data points.
SVM Concepts:
Hyperplane: It is decision plane which is divided between a set of objects
having different classes.
Margin : It is nothing but the gap between two lines on the closest data
points of different classes.
It can be defined as the perpendicular distance from the line to the
support vectors.
Large margin is considered as good margin and small margin is
considered as bad margin.
SVM Concepts:
The main goal of SVM is to divide the datasets into classes to find a
maximum marginal hyperplane(MMH).
It can be done in the following two steps:
SVM will generate hyperplanes iteratively that separates the classes in
best way.
Then , it will choose the hyperplane that separates the classes.
Implementing SVM in Python:
SVM Kernel:
SVM Algorithm is implemented with a kernel that transforms a input
data space into the required form.
It uses a trick called kernel trick which transforms a low dimensional
space and transforms into high dimensional space.
Kernel separates non-separable problems into separable problems by
adding more dimensions to it.
It makes SVM more flexible and accurate.
Type Of Kernels Used By SVM:
Linear Kernel: It can be used as a dot product between any two observations.
K(x,xi) = sum(x*xi)
Product between any two vectors x and xi is the sum of the multiplication of
each pair of input values.
Polynomial Kernel:
It is a general form of linear kernel and differentiate curved or nonlinear input
space.
K(X,Xi) = 1 + sum(X*Xi)^d
d is the degree of the polynomial which we need to specify manually in the
learning algorithm.
Redial Basis Function(RBF) Kernel:
RBF kernel used in SVM classification, maps input space in
Indefinite dimensional space.
K(x,xi)=exp(−gamma∗sum(x−xi^2))
Gamma range from 0 to 1.
We need to specify manually in the learning algorithm.
Default value of gamma is 0.1
SVM is used for linearly separable data and python can be used for data
that is not linearly separable.
Example:
Following is an example of SVM :
We will be using iris dataset from skikit-learn:
Start by importing following packages:
import pandas as pd
import numpy as np from sklearn
import svm, datasets
import matplotlib.pyplot as plt
Example:
Load the input data:
iris = datasets.load_iris()
From the dataset , we are taking first two features as follows:
X = iris.data[:, :2]
y = iris.target
Example:
Plot the SVM boundaries with original data as follows:
x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
h = (x_max / x_min)/100
xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min,
y_max, h))
X_plot = np.c_[xx.ravel(), yy.ravel()]
Example:
Provide the value of regularization parameter:
C = 1.0;
Next, SVM classifier object can be created as follows −
Svc_classifier = svm.SVC(kernel='linear', C=C).fit(X, y)
Z = svc_classifier.predict(X_plot)
Z = Z.reshape(xx.shape)
plt.figure(figsize=(15, 5))
plt.subplot(121)
plt.contourf(xx, yy, Z, cmap=plt.cm.tab10, alpha=0.3) plt.scatter(X[:, 0], X[:, 1],
c=y, cmap=plt.cm.Set1)
plt.xlabel('Sepal length')
plt.ylabel('Sepal width')
plt.xlim(xx.min(), xx.max())
plt.title('Support Vector Classifier with linear kernel')
DOUBTS:
Example:
For creating SVM classifier with rbf kernel, we can change the kernel to rbf as follows
Svc_classifier = svm.SVC(kernel = 'rbf', gamma =‘auto’,C = C).fit(X, y)
Z = svc_classifier.predict(X_plot)
Z = Z.reshape(xx.shape)
plt.figure(figsize=(15, 5))
plt.subplot(121)
plt.contourf(xx, yy, Z,
cmap = plt.cm.tab10, alpha = 0.3)
plt.scatter(X[:, 0], X[:, 1],
c = y, cmap = plt.cm.Set1)
plt.xlabel('Sepal length')
plt.ylabel('Sepal width')
plt.xlim(xx.min(), xx.max())
plt.title('Support Vector Classifier with rbf kernel')
Output
Text(0.5, 1.0, 'Support Vector Classifier with rbf kernel')
Artificial Neural Networks:
The popular approach of using deep learning is using “Artificial Neural
Network”.
They behave similar to human brain which is the most complex organ of
our body.
Human Brain is made up of several billion tiny cells called neurons.
Neurons are connected to one another through nerve fibre called “axons”
and “dendrites”.
Axon is used to pass the information from one neuron to another to
which it is connected.
Artificial Neural Networks:
Dendrites are used to receive the information being transmitted by the
axon of another neuron to which it is connected.
Each neuron process the small information, passes the result to another
neuron and then this process continues.
This method is used by the human brain to process huge amount of
information like text,visualetc and extract essential information from it.
Artificial Neural Networks:
Based on the human brain model , first artificial neural network (ANN)
was invented by psychologist,Frank Rosenblatt in the year 1958.
Artificial neural networks are made up of multiple nodes which are
similar to neurons.
Nodes are tightly packed and grouped into different hidden layers.
Finally , the output layer predict something useful about the input data.
For eg , the input is an image and the output is something identified in an
image like “lion”.
Artificial Neural Networks:
A single neuron is called as a perceptron in artificial neural
networks(ANN).
It contains
Multiple inputs along with weights which represent dendrites.
Sum of inputs along with activation function represents neurons.
Sum is nothing but the computed value of all inputs.
Activation function represent a function which modify the sum value to 0,1
or 0 to 1.
Actual output represent axon and output will be received by the neuron in
the next layer.
Different Types Of ANN:
Multi-Layer Perceptron:
It is a basic form of ANN(Artificial Neural Network).
It consists of a single input layer , one or more hidden layer and an output layer.
Each layer consists of a collection of perceptron.
Input layer is nothing but one or more features of input data.
Every hidden layer consists of one or more neurons .
They process certain aspect of the feature and send the processed information
into next hidden layer.
The output layer process receives the data from the last hidden layer and finally
output the result.
Convolutional Neural Network(CNN)
It is one of the most popular Artificial Neural Network(ANN).
It is used in the fields of image and video recognition.
It is based on a maths concept called convolution.
It is same like multi-layer perceptron .
The only difference is that it contains a series of convolution layer and
pooling layer before a fully connected hidden neuron layer.
Layers Of Convolutional Neural Network:
It contains three important layers.
Convolution Layer : It is the basic building block and perform
computational tasks based on convolution function.
Pooling layer: It is stacked next to the convolution layer.
It is used to reduce the size of inputs by removing unwanted information
so that computation can be performed faster.
Fully Connected Layer: It comes next to the series of convolution and
pooling layer.
It classifies inputs into various categories.
CNN Representation:
2 series of convolution and pooling layer is used and it receives and
process the input(eg:image).
A single fully connected layer is used and it is used to output the
data(eg:classification of image).
DOUBTS:
Recurrent Neural Network(RNN)
They are used to address the flaw in many artificial neural
networks(ANN).
ANN don’t remember the steps from previous situations.
They make decisions based on context in training.
RNN stores the past information and they make decisions based on what
they have learnt from the last.
This kind of approach is used in image classification.
Bidirectional RNN is used to learn from the past and predict the future.
We can have handwritten samples in multiple inputs.
Recurrent Neural Networks(RNN)
Suppose , if one input doesn’t provide the desired result , then we have to
check for other inputs.
We need to find the correct context which takes decision from the past.
ANN – Working Process
Following are the various steps involved in deep learning.
Keras can be helpful in the process of deep learning.

Gather the Required Data:

Deep learning requires a lot of data to learn and predict the results.
Collect as much data as possible.