ARTIFICIAL NEURON

Biological Neuron

• The brain may be thought of as a complex computer

• It has the following amazing characteristics:

• The ability to perform complex tasks (e.g. pattern

recognition, perception & motor control) much faster than
any computer

• The brain has the ability to solve several problems

simultaneously using distributed parts of the brain

1
 ARTIFICIAL NEURON

Biological Neuron

• The ability to learn, memorize and still generalize

• The brain is Fault Tolerant in two respects

- It is able to recognize many input signals that are

somewhat different from any signal we have seen before
(e.g. recognition of person from different pictures)

- It is able to tolerate damage to the neural system itself.

Most of the neurons are not replaced when they die. In
spite of continuous loss of neurons, we continue to learn

2
 ARTIFICIAL NEURON

Biological Neuron

• These characteristics prompted research in algorithmic

modeling of brain (biological neural systems)

• Is it possible to truly model the human brain?

Not at the present. Current successes in neural modeling

are for small artificial NNs aimed at solving a specific task

3
 ARTIFICIAL NEURON

Biological Neuron
• The basic building blocks of biological neural systems
are nerve cells, referred to as neurons
• Each consists of : SOMA, DENDRITES, AXON, and
SYNAPSE
• About 100 billion neuron in the human brain

4
 ARTIFICIAL NEURON

Biological Neuron

• The main body of the cell collects the incoming signals

from the other neurons through its dendrites

• The incoming signals are constantly being summed in the

cell body

• If the result of the summation crosses a certain threshold,

the cell body emits a signal of its own (called firing of the
neuron)

• This signal passes through the neuron’s axon, from where

the dendrites of other neurons pick it up
5
 ARTIFICIAL NEURON

Biological Neuron

• There are 1,000 to 10,000 dendrites in each neuron (few

millimeters long).

• There is only one axon (several centimeters long)

• The connection between dendrites and axon is

electrochemical and it is called synapse

• The synapses modify (enhance or inhibit) the signal while

passing it on to dendrites

6
 ARTIFICIAL NEURON

Biological Neuron

• The human learning is stored in these synapses, and the

connection of neurons with other neurons

• If stimulus at a dendrite causes the neuron to fire, then the

connection between that dendrite and axon is
strengthened

• If the arrival of stimulus does not cause the neuron to fire,

the connection weakens over time

7
ARTIFICIAL NEURON

Artificial Neuron Model

8
 ARTIFICIAL NEURON

• An artificial neuron has three components:

– Input connections (x1,x2..) with weights (w1,w2..) (the equivalent of
dendrites)
– An output (y) (the equivalent of an axon eventually linked with other
neurons’ “dendrites”.
– A computing unit (the equivalent of a living neuron’s soma). The
computing unit constantly sums the weighted inputs to produce a net
out of which the output is calculated using a non-linear activation
function (f) :
y  f ( wi xi )
 ARTIFICIAL NEURON

Artificial Neuron Model

It receives a vector X of I input signals,

X = (x1, x2, …, xI)

either from the environment or from other artificial neurons

Each input signal xi is multiplied by a weight wi to strengthen

or weaken it

The neuron computes the weighted

sum of the input signals
10
 Artificial Neural Systems
• Artificial neurons are analogous to their biological inspirers

• Here the neuron is actually a processing unit, it calculates

the weighted sum of the input signal to the neuron to
generate the activation signal a, given by

N
a   wi xi where wi is the strength of the synapse connected
i 1 to the neuron, xi is an input feature to the neuron
 Artificial Neural Systems
• The activation signal is passed through a
transformation function to produce the output of the
neuron, given by
 
y  f (a )

• The transformation function also called Activation

function can be linear, or non-linear,  
• For a linear function, the output y is proportional to
the activation signal a.
 ARTIFICIAL NEURON

Artificial Neuron Model

The weighted sum is usually called the activation of the

neuron

An activation function is applied on this weighted sum to

produce the output of the neuron

y = f(activation)

If the activation function is the unit step function, we can say

that an artificial neuron implements a nonlinear mapping
from a vector of real numbers to [-1, 1]

13