PART 6

Neural Network

1
 Introduction
Humans have an ability to identify patterns with a high
degree of accuracy.

When you see a car or a bicycle you can immediately

recognize what they are. This is because we have learned
over a period of time how a car and bicycle looks like and
what their distinguishing features are.

Artificial neural networks are computation systems that

intend to imitate human learning capabilities via a complex
architecture that resembles the human nervous system.

2
 The Human Nervous System
Human nervous system consists of billions of neurons.
They process input received from sensory organs, process
the information, and decides what to do in reaction to the
input.

A typical neuron in the human nervous system has three

main parts (see next slide):

Dendrites: information passed to a neuron is received by

dendrites.
Nucleus: processes this information.
Axon: passes output to other neurons
3
4
5
Perceptron

Artificial neural networks are inspired by the

human neural network architecture.

The simplest neural network consists of only

one neuron and is called a perceptron (see fig.
on next slide).

6
7
• A perceptron has one input layer and
one neuron.
• Input layer acts as the dendrites and is
responsible for receiving the inputs.
• The number of nodes in the input layer
is equal to the number of features in
the input dataset.
• Each input is multiplied with a weight
(initialized with some random value)
and the results are added together.
8
• The sum is then passed through an
activation function.
• The activation function of a perceptron
resembles the nucleus of human neuron.
• It processes the information and yields an
output.
• In the case of a perceptron, this output is
the final outcome.
• However, in the case of multilayer
perceptron, the output from the neurons in
the previous layer serves as the input to the
neurons of the proceeding layer. 9
Multilayer Perceptron
Multilayer perceptron is commonly known as
artificial neural network.

A single layer perceptron can solve problem

where data is linearly separable.

In case of non-linearly separable data,

multilayer perceptron work efficiently.

10
Linearly Separable Dataset 11
Linearly Non-separable Datasets
12
Multilayer perceptron is a
combination of multiple neurons
connected in the form a network.

An artificial neural network has an

input layer, one or more hidden
layers, and an output layer. This is
shown in the image on next slide.

13
14
15
16
A neural network executes in two phases: Feed-
Forward and Back Propagation.
Feed-Forward
• the values received in the input layer are
multiplied with the weights.
• A bias is added to the summation of the inputs
so that neuron fires when a threshold is
reached.
• Neurons have an activation function (such as
step, sigmoid, relu, or tanh) that operates upon
the value received from the input layer.

17
Back propagation
• The weights of different neurons are updated in a way
that the difference between the desired and predicted
output is as small as possible.
• The error is calculated by quantifying the difference
between the predicted output and the desired output.
• This difference is called "loss" and the function used to
calculate the difference is called the "loss function".
• Loss functions can be of different types e.g. mean
squared error or cross entropy functions.
• Once the error is calculated, the next step is to minimize
that error. The error is minimized through gradient
decent method that we have already covered.
• This one cycle of feed-forward and back propagation is
called one "epoch". This process continues until a
reasonable accuracy is achieved. 18
Example:
We will build a classification model to detect diabetes.
Dataset: The diabetes dataset contains 768 observations
and following nine variables
1. pregnancies - Number of times pregnant
2. glucose - Plasma glucose concentration
3. diastolic - Diastolic blood pressure (mm Hg).
4. triceps - Skinfold thickness (mm).
5. insulin - Hour serum insulin (mu U/ml).
6. bmi – Body mass index
7. dpf - Diabetes pedigree function.
8. age - Age in years.
9. diabetes - “1” represents the presence of diabetes while
“0” represents the absence of it. This is the target variable.
Produce confusion matrix and classification report. 19
42

Confusion Matrix

139 18

32 42
20
 Evaluation Metrics

Accuracy = (TP+TN) / (TP+TN+FP+FN)

Precision = TP / (TP+FP)

Recall = TP/ (TP+FN)

F1 Score = 2*(Recall * Precision) / (Recall +

Precision)

21
# Step 1 - Load the Required Libraries and Modules

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import sklearn
from sklearn.neural_network import MLPClassifier
from sklearn.neural_network import MLPRegressor
from sklearn.model_selection import
train_test_split
from sklearn.metrics import mean_squared_error
from math import sqrt
from sklearn.metrics import r2_score
22
# Step 2 - Reading the Data and Performing Basic
Data Checks

data = pd.read_csv(“file location.csv")

data.shape

data.head()

data.describe()

23
# Step 3 - Creating Arrays for the Features and the Response
Variable

# (a) Feature arrays

X = data.drop('Outcome', 1)

print (X.head())

# (b) Target array

y = data['Outcome']

print (y.head())

24
# Step 4 - Creating the Training and Test Datasets

from sklearn.model_selection import

train_test_split
X_train, X_test, y_train, y_test = train_test_split(X,
y, test_size=0.3, random_state=0)
print(X_train.shape); print(X_test.shape)

25
# Step 5 Normalize features
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)

print (X_train)

NOTE: train data is fit and transformed. Test data is

only transformed.

26
Normalization process
Normalization is used to center (features scale) data.

Generally, a Z-score method is used for normalization.

Here,

z = z-score

x = individual observations

µ = mean
27
Normalization process
Normalization is used to center (features scale) data.

Generally, a Z-score method is used for normalization.

The following functions are used for normalization:

1.Fit(): Method calculates the parameters μ and σ and

saves them as internal objects.

2.Transform(): Method using these calculated parameters

apply the transformation to a particular dataset.

3.Fit_transform(): joins the fit() and transform() method

for transformation of dataset. 28
# Step 6 - Building Neural Network Model

Build the neural network model using the scikit-

learn library's object, 'Multi-Layer Perceptron
Classifier'.

(a) import 'MLPClassifier'

from sklearn.neural_network import

MLPClassifier

29
(b) Instantiate the model

'hidden_layer_sizes’ = three layers,

(which has the same number of neurons as the
count of features in the dataset).
‘Rectified linear unit (ReLU) is the activation function and
'adam' is the solver for weight optimization.

mlp = MLPClassifier(hidden_layer_sizes=(8,8,8),
activation='relu', solver='adam', max_iter=500)

30
(c) fit the model to the training data

mlp.fit(X_train,y_train)

(d) use the trained model to generate predictions on the test

dataset

predict_test = mlp.predict(X_test)

31
Step 7: Evaluate performance of model
from sklearn.metrics import
classification_report,confusion_matrix

print(confusion_matrix(y_train,predict_train))

print(classification_report(y_train,predict_train))

32
 ASSIGNMENT NO. 5 updated
L.D = 10.06.2020
1. What is activation function.
2. Describe any five commonly used activation
functions.
3. Which activation function you would select
among above five and why?
4. Use MLP for provided dataset ‘bank.csv’ . It is
abridged form of:
(https://archive.ics.uci.edu/ml/datasets/bank+mark
eting) dataset. Explain dataset. Produce
confusion matrix and classification report.
33
THE END

34