Predictive Analytics using

Supervised Learning

Classification using
Artificial Neural Networks

8/12/2021 BDM MBA 57 Term IV 2021 1

 Why Artificial Neural Network?
• Ease of classification when cause-and-effect can be captured
– Example problem: Whether a student would get admission
in some IIM
– Decision tree suits well
• Difficulty of classification when cause-and-effect is extremely difficult
to capture, if at all even though patterns seem to exist!
– Example problem: Whether the signature in the cheque is
‘same’ as the signature the bank has against the account in
its database
– Decision tree use is more of a contradiction

8/12/2021 BDM MBA 57 Term IV 2021 2

 Session Plan

• Concept of an Artificial Neural Network

• Demonstration of how it works (with a
small dataset) using Excel
• Use of software tool
• Potential business applications
• Issues in using neural networks

8/12/2021 BDM MBA 57 Term IV 2021 3

 Artificial Neural Network (ANN)

Artificial Neural Network (ANN) is a

computational paradigm inspired by the
structure of biological neural networks and
their way of encoding and solving
problems

8/12/2021 BDM MBA 57 Term IV 2021 4

 Biological inspirations
• Some numbers…
– The human brain contains about 10 billion nerve cells
(neurons)
– Each neuron is connected to the others through about
10000 synapses

• Properties of the brain

– It can learn, reorganize itself from experience
– It adapts to the environment
– It is robust and fault tolerant

8/12/2021 BDM MBA 57 Term IV 2021 5

 Biological Neural Networks
• Human brain contains several billion nerve-cells
called neurons
• A neuron receives electrical signals through its
dendrites
• The accumulated effect of several signals
received simultaneously is linearly additive
• Output is non-linear (all or none) type of output
signal
• Connectivity (no of neurons connected to a
neuron) varies from 1 to 105. For the cerebral
cortex it’s about 103.
8/12/2021 BDM MBA 57 Term IV 2021 6
 One Single Biological Neuron
synapse axon

nucleus

cell body

dendrites

• Dendrites sense input

• Axons transmit output
• Information flows from dendrites to axons via cell body
• Axon connects to other dendrites via synapses
– Interactions of neurons
– synapses can be excitatory or inhibitory
– synapses vary in strength
• How can the above biological characteristics be modeled in an artificial
system?
8/12/2021 BDM MBA 57 Term IV 2021 7
 Human Brain (Billions of Neurons!)
Hand
A

B C
I

….

G
H
J D Lip

E
F
Eye
Ear ….

8/12/2021 BDM MBA 57 Term IV 2021 8

 Artificial implementation using a
computer ?
• Input
– Accepting external input is simple and commonplace
• Axons transmit output
– Output mechanisms too are well known
• Information flows from dendrites to axons via cell body
– Information flow is doable
• Axon connects to other dendrites via synapses
– Interactions of neurons (how? What kind of graph or network??)
– synapses can be excitatory or inhibitory (1/0 ? Continuous?)
– synapses vary in strength (weighted average?)

8/12/2021 BDM MBA 57 Term IV 2021 9

 Interconnections?
Feed Forward Neural Network
• Information is fed at the
Value to be interpreted as Classification
input
• Computations done at the
Output layer
hidden layers
2nd hidden • Deviations of computed
layer results from desired goals
retunes computations
1st hidden • Network thus ‘learns’
layer
• Computation is terminated
once the learning is
assumed acceptable or
resources earmarked for
x1 x2 …Input layer.. xn computation get exhausted
8/12/2021 BDM MBA 57 Term IV 2021 10
Computation at a neuron p. NOTATION: Ip: Net input to neuron p, Op:
Computed output of p, Wip:Synapse weight factor connecting i to p, θp: Bias
of p
i m
Oi=0.6

Wip=0.2 θp=0.5 Wpm=.1

.6*.2=.12
j Oj=0.3 Wjp=0.8 p
Op=f(Ip)=f(0.23)=0.557
Op=f(Ip)=f(0.23)=0.557
Ip Wpn=-.4)
Wkp=-0.7

Ok=0.9
k n

Ip = (Oi*Wip) + (Oj*Wjp) + (Ok*Wkp) +θp =

(.6*.2)+(.3*.8)+(.9*(-.7)) +.5= 0.23

8/12/2021 BDM MBA 57 Term IV 2021 11

 Typical Excitement or Activation
Function at a Neuron (Sigmoid
or Logistic curve)

Logistic
1
y  f ( x)  x
1 e

8/12/2021 BDM MBA 57 Term IV 2021 12

Neuron J is an output layer neuron. Shows how “learning from mistake” works

J O : As computed by J
T : True output for the input record

…… e = O (1-O )(T -O )
Example: T = 1 (that is, defaulter. Truth)
O = 0.63 (as predicted by ANN)
Error e = 0.63*(1-0.63)*(1-0.63) = .086247

8/12/2021 BDM MBA 57 Term IV 2021 13

Neuron J is an output layer neuron. Shows how self correction of bias works.

θ =0.2

J O : As computed by J
T : True output for the input record

…… e = O (1-O )(T -O )
Example: T = 1 (that is, defaulter. Truth)
O = 0.63 (as predicted by ANN)
Error e = 0.63*(1-0.63)*(1-0.63) = 0.086247
Δ θ = L* e , L is the Learning Rate (=.9 say)
=0.9*.086247 = 0.077622
θ =θ +Δ θ = 0.2+0.9*0.086247=0.277622

8/12/2021 BDM MBA 57 Term IV 2021 14

Suppose, p is any hidden layer neuron. Shows how “owning up mistake”
works through back-propagation by considering all outgoing links from p
m

Wpm=-.3 e =.012

p
O =.7 .....

Wpn=.6

e =O (1-O )∑ (W *e ) e =-.003
=.7 * (1-.7) * (-0.3*0.012 + 0.6(-0.003))=0.2046

8/12/2021 BDM MBA 57 Term IV 2021 15

Suppose, p is any hidden layer neuron. Shows how incremental correction
of weight and bias works. Assume L=0.9
m

θ =0.05 Wpm=-.3 e =.012

p
O =.7, ep=0.2046 .....

Wpn=.6

n
ΔW = L*O *e =0.9*0.7*0.012=.00756
Wpm = Wpm + ΔWpm = -0.3 + .00756 =-0.29244
Similarly, ∆Wpn, and new value of Wpn
e =-.003
Δ θp = L* e = 0.9*0.2046 = 0.18414
θp = θp + Δ θp = 0.05+0.18414 = 0.23414

8/12/2021 BDM MBA 57 Term IV 2021 16

 How does the network “learn”?
• Inputs and outputs are known for past data
records in the training set.
• The network “tunes” its weights and biases so
that network-wide computation transforms input
records to corresponding outputs as closely as
possible at an aggregate level
• Mismatches between truth and prediction about
the output of a past record helps in incrementally
finetuning the weights and biases!
• It’s a long iterative process
8/12/2021 BDM MBA 57 Term IV 2021 17
 Characteristics of ANN
• Supervised networks are good approximators
• Bounded functions can be approximated by an
ANN to any precision
• Does self-learning by adapting weights to
environmental needs
• Can work with incomplete data
• The information is distributed across the network.
If one part gets damaged the overall performance
may not degrade drastically

8/12/2021 BDM MBA 57 Term IV 2021 18

 What do we need to use ANN ?
• Determination of pertinent inputs and output
• Collection of data for the learning, validating,
and testing
• Finding the number of hidden layers and nodes
• Managing learning parameters
• Evaluating the performance of the network and
iterating by fine-tuning network parameters, if
necessary

8/12/2021 BDM MBA 57 Term IV 2021 19

 Applications of ANN
• Dares to address difficult problems where
cause-effect relationship of input-output is
very hard to quantify
– Stock market predictions
– Face recognition
– Time series prediction
– Process control
– Optical character recognition
– Optimization

8/12/2021 BDM MBA 57 Term IV 2021 20

 Concluding remarks on Neural
Networks
• Neural networks are utilized as statistical tools
– Adjust non linear functions to fulfill a task
– Need of multiple and representative examples but
fewer than in other methods
• Neural networks enable to model complex phenomena
• NN are good classifiers BUT
– Good representations of data have to be formulated
– Training vectors must be statistically representative of
the entire input space
• Effective use of ANN needs a good comprehension of
the problem and a good grip on underlying mathematics

8/12/2021 BDM MBA 57 Term IV 2021 21