Artificial Neural Networks

Day 5
LET US NOW LOOK AT A DIFFERENT BREED
OF PROBLEMS
 We can become extremely fast with practice
• We do not even know how our brain is solving
them (no algorithms)
 Features of an algorithmic solutions
• What are the features of an algorithmic solutions
– We are bad at instantly developing solutions in
our mind
– We do not become significantly faster/ better
with time
– We can clearly explain how we come to a
conclusion once the solution is given in a
repeatable manner
LET US UNDERSTAND BRAIN
 LET US UNDERSTAND BRAIN
Neuron: The Building Block of the Brain
 A Simplified Neuron

X1W1+ X2W2+ X3W3+ ...+XmWm

Squashing functions
Question?

What is the output for a sigmoid function?

LIMITATIONS OF PERCEPTRON AND HOW TO
OVERCOME
A 3 layer network can learn any function
 Method
• Gradient Descent (Back Propagation)
• The Input Layer in NN should have as many
nodes as we have the number of attributes.
• The output layer in NN should have as many
classes we have to predict.
• For regression problem, there is only one
output node.
 Method
• Gradient Descent (Back Propagation)
• The Input Layer in NN should have as many
nodes as we have the number of attributes.
• The output layer in NN should have as many
classes we have to predict.
• For regression problem, there is only one
output node.
• Hidden layer to be max. Of 2, good to have only one layer.
Number of nodes in hidden layer is cal. By
.5 – 2 times (of Input nodes)
Sqrt(I *O) I- input nodes, O-output nodes
H(I+O)+H+O = (No. Of Data points)/100
N nodes <= N Data points * N Error Rate
---------------------------------
N input nodes * N Output nodes
• Example: Input nodes = 10, output nodes = 5, data points = 10000,
error rate = 2%
.5 – 2 times = 5-20 nodes (experiment)
Sqrt(10*5) = ~ 7
H(10+5)+H+5 = 10000/100 => 16H+5 = 100 => H = 95/16
10000*.02 / 10*5 => 200/50 => 4
• Pain aspects of NN:
a. Black box
b. Overfits
c. Too many parameters to experiment with i.e. nodes,
learning rate etc..
• NN has very high variance, so it overfits