ARTIFICIAL

NEURAL NETWORK

Neural Networks NN 1 1
 SEQUENCE
• Brief History of ANN Evolution
• What are ANN
• Comparison between human brain and
neuron

Neural Networks NN 1 2
 What are Neural Networks?

• Simple computational elements forming a large network

– Emphasis on learning (pattern recognition)
– Local computation (neurons)
• Configured for a particular application
– Pattern recognition/data classification
• ANN algorithm
– Modeled after brain
• Brain € 1 0 0 , 0 0 0 times slower response
– Complex tasks (image, sound recognition, motion con)
– 10,000,000,000 times efficient in energy consumption/op
• Definition of NNs is vague
– Often | but not always | inspired by biological brain

Neural Networks NN 1 3
 Why use ANN?
• Remarkable ability to derive meaning from
complicated or imprecise data
• Extract patterns and detect trends that are too
complex to be noticed by either humans or other
computer techniques
• “Expert" in the category of information it has been
given to analyse
• Generalization ability
• Adaptive learning
• Self-Organization € c a n create its own ong o
iza
rn
ti or
representation of the information
• Real Time Operation
• Fault Tolerance via Redundant Information Coding
Neural Networks NN 1 4
 Human and Artificial Neurons
(Investigating similarities)
• The human brain is the most sophisticated device.
• One of the most important features of the brain is the
ability to learn from previous experience.
• Thus humans have a great success in dealing with
unforeseen situations utilising the knowledge gained
from previously experienced similar situations.
• This is not true in systems based on computer
technology as they rely entirely on human pre-
defined instructions, spelling out each step for all
tasks. Thus bugs, in these instructions may cause all
sorts of unexpected results.

Neural Networks NN 1 5
 BRAIN vs CPTR
• Brain
– Operating by 100-mV nerve
impulses lasting nearly a
millisecond. Neuron takes 4
ms to complete a firing cycle.
– It is robust and fault tolerant.
Nerve cells in the brain die
every day without affecting
its performance significantly.
– It accepts fuzzy, noisy, poorly
conditioned inputs and
produces an approximate
output.
– It is highly parallel due to
massive inter-connectivity
between neurons.
– Connection with other
elements via synapse.
Neural Networks
• Computer
– Technology Operating on 5-V
signal levels switching at
nanosecond intervals. Computers
with 33 MHz takes about 40 ns
to execute a single instruction.
Super computers take about 3
ns to complete a single
instruction.
– The destruction of even a single
transistor may cause complete
loss of functionality.
– It can only handle precise data
fed in properly.
– The conventional computer are
totally sequential. (Few
connections between its basic
elements)
– Links/weights
NN 1 6
 ANN vs CPTR
• Like human brain • Algorithmic approach
• Interconnected • A cognitive approach:
neurons – the way the problem is to solved
• Learn by must be known and stated in
examples small unambiguous instructions.
• Un- – These instructions are then
programmable converted to a high level
• Careful selection of language program and then into
machine code that the
examples
computer can understand.
• O p s € e au
b
cn
lte
p
d
ri
• Predictable; if anything goes
wrong is due to a software or
Neural Networks
haNNrd1 ware 7
 TANK BENEATH THE TREE

Neural Networks NN 1 8
 ANN vs CPTR
• ANNs and conventional algorithmic computers
– Not in competition
– Complement each other.
– Tasks =a l g o r i t h m i c approach like
arithmetic operations
– Tasks that are more suited to ANN.
– Large number of tasks
• Require systems that use a combination
• A conventional computer is used to supervise the neural
network
• Neural networks do not perform miracles. But if
used sensibly they can produce some
amazing results.
Neural Networks NN 1 9
 Human and Artificial Neurons
(FUNCTIONALITIES)
• There are hundreds different types of
neurons, each with its characteristic function,
shape and location, but the main features of a
neuron of any type are its cell body, called
soma, synapses, dendrites and the axon.
• Information is transferred from one neuron to
another at specialised junctions, called
synapses. The dendrites act as the input units
of external signals to the neuron and the
axon acts as the output unit.
Neural Networks NN 1 10
 Human and Artificial Neurons
(FUNCTIONALITIES)
• Much is still unknown
– How brain train itself/process information
– Theories abound
DENDRITES

CELL BODY

AXON
CELL BODY
SYNAPSE

AXON

Neural Networks NN 1 11
 Human and Artificial Neurons
(FUNCTIONALITIES)
• Dendrites
• Cell Body
– soma
• Axon
– Electrical pulses
• Synapses
– junction
• Biochemical activities
• Brain's functionality
– Thought, emotion and cognition

Neural Networks NN 1 12
 Human and Artificial Neurons
(FUNCTIONALITIES)
• Neurons receive inputs
– synapses
• Outputs are sent
– axons
• Membrane potential
– Exceeds a threshold,
– the neuron can propagate an all-or-nothing
action potential down its axon
• Many neurons can be active simultaneously
– state vector.

Neural Networks NN 1 13
 Course Topics
Learning Tasks

Supervised Unsupervised
Data: Data:
Labeled examples Unlabeled examples
(input , desired output) (different realizations of the
input)
Tasks:
classification Tasks:
pattern recognition clustering
regression
ANN models: ANN
perceptron models:
adaline self-organizing maps (SOM)
feed-forward NN Hopfield networks
radial basis function
support vector
machines
Neural Networks NN 1 14
 ANNs: goal and design
– Knowledge about the learning task is given in the
form of a set of examples (dataset) called training
examples.
– An ANN is specified by:
• an architecture: a set of neurons and links connecting
neurons. Each link has a weight,
• a neuron model: the information processing unit of the
ANN,
• a learning algorithm: used for training the ANN by
modifying the weights in order to solve the particular
learning task correctly on the training examples.
The aim is to obtain an ANN that generalizes well,
that is, that behaves correctly on new examples of
the learning task.
Neural Networks NN 1 15
 From Human neurons to
artificial neurons (AN)
• We conduct AN by deducing essential
feature and interconnections of human
neurons
• Prog a cptr to simulate them
• Knowledge is incomplete and
computation power is limited
– Gross idealization of real
neurons
Neural Networks NN 1 16
 A Neuron Model

Neural Networks NN 1 17