Unit 1

Lecture 3
Topics
• Basics of ANN
• Simple Neural Network with one layer
• Biological Neural Network
• Main parts of a biological neuron
• Terminology relationships between both the neurons
• Mathematical representation of the biological neuron
 Artificial Neural Network
• Models the human brain
• Perform several functions like pattern matching, classification, optimization,
approximation etc.
• Consists of large no. of highly interconnected processing elements called
neurons or artificial neurons
• Operates in parallel
• Each neuron connected to the other using a connection link viz associated
with weights
• Weights gives information about the input signals
• Features: ability to learn, recall and generalize patterns similar to human brain
Simple Neural Network

X1 and X2 are the input neurons which transmits the signals

Y is the output neuron which receives the signals
x1 and x2 are the inputs and w1 and w2 are the weights
Net input to Yin = x1w1+x2w2
Output at Y = function f(net input to Yin)
= function f(x1w1+x2w2)
• Function f is called the activation function
• Many types: Sigmoid, tanh, ReLu etc.
• The net input at Y = x1w1+x2w2 is similar to linear regression equation
(y=mx+c)
• When x and y varies linearly slope m also varies linearly
• For the pure linear equation y=mx+c the neural net will be

• To get the output y slope m is directly multiplied with the input x

 Biological Neural Network
• 3 main parts: Cell body, Dendrites, Axon
• Axon-carries
Signal from
Neuron
• Synapse-
Bulb like organ,
Used for passing
Signals to other
neurons

Approximately 1011 such neurons

104 synapses per neuron
• Electric pulses transferred between synapse and dendrites
• Electric transmission involves chemical reactions
• Specific transmitting substances are released from the sending side
• Signals strength if higher than a certain threshold receiving cells fires a pulse
or an action through the axon.
• After firing neuron waits for a certain period of time before firing again
(refractory period)
• Synapse is said to be inhibitory(if they don’t let the passing impulse fire an
action) or excitatory (if it lets the passing impulse fire an action)
Terminology relationships between biological and artificial neuron

Biological Neuron Artificial Neuron

Cell Neuron
Dendrites Weights or interconnections
Soma Net input
Axon Output
 Mathematical representation of the chemical process

Net input is
• Weights represent the strength of the synapse connecting the input and the
output neurons
• Positive weight corresponds to an excitatory synapse and negative weights
correspond to inhibitory synapse