Artificial Neural Networks

Department of Computer Science

And Information Technology
University of AJ&K
Muzaffarabad
E-mail: kh_wajid@yahoo.com
Artificial Neural Networks
 An Artificial Neural network is an
information processing paradigm inspired by
biological nervous systems, such as our brain
 Structure: Composed of large number of
highly interconnected processing elements
(neurons) working together
 Like people, they learn from experience (by
example)
Artificial Neural Networks
 Neural networks are configured for a specific
application, such as pattern recognition or
data classification, through a learning
process
 In a biological system, learning involves
adjustments to the synaptic connections
between neurons
 same for artificial neural networks (ANNs)
 Artificial Neural Networks
 A neural network resembles brain in two respects:
 Knowledge is acquired by the network through a
learning process.
 Inter-neuron connection strengths known as synaptic
weights are used to store knowledge.
 An artificial neuron is characterized by:
 Architecture (Connection between neurons)
 Training or learning (Determining weights on the
connection)
 Activation function.
 Historical Development
 1943 ― McCulloch and Pitts: start modern era
of neural networks.
 1949 ― Hebb’s book: The organization of
behavior.
 1958 ― Rosenblatt introdes perceptron
 1960 ― Introduced Adaline and Madline
 1972 ― Kohonen’s self organization Maps
 1982 ― John Hopfield’s network
 Historical Development
 1985 ― Parker, 1986 ― Rumelhart, Hinton and Williams: Back
propagation algorithm for training multilayer perceptron
 1987, 1990 ― Carpenter and Gross: invented adaptive resonance theory
(ART).
 1988 ― Grossberg: developed leaning similar to that Kohonen, which is
used in Counter propagation net.
 1988 ― Broomhead and Lowe: invented adaptive resonance
theory.
 1990 ― Vapnick: developed support vector machine (SVM


Why Artificial Neural Networks
 Neural networks, with their remarkable ability to derive
meaning from complicated or imprecise data, can be used
to extract patterns and detect trends that are too complex
to be noticed by either humans or other computer
techniques.

 A trained neural network can be thought of as an "expert"

in the category of information it has been given to analyze.

 This expert can then be used to provide projections given

new situations of interest and answer "what if“ questions.
 Why Artificial Neural Networks
 Other advantages include:
 Adaptive learning: An ability to learn how to do tasks based on
the data given for training or initial experience.
 Self-Organisation: An ANN can create its own organisation or
representation of the information it receives during learning time.
 Real Time Operation: ANN computations may be carried out in
parallel, and special hardware devices are being designed and
manufactured which take advantage of this capability.
 Fault Tolerance via Redundant Information Coding: Partial
destruction of a network leads to the corresponding degradation
of performance. However, some network capabilities may be
retained even with major network damage.
Biological Neuron

A biological neuron or a nerve cell consists of

Synapse, Dendrites, the cell body or soma, and the
axon
Artificial Neuron

The association of biological neuron with artificial

neuron is shown in figure and table below
Biological Neural Artificial Neural
Network Network
Dendrite Net Input
Synapse Weights or
interconnections
Soma Processing
(summing and then
applying nonlinear
function)
Axon Output
Computers vs. Neural Networks
Standard Computer Neural Network
One CPU Highly parallel processing

Fast Processing units Slow processing units

Reliable Unreliable

Static Infrastructure Dynamic Infrastructure

 Comparison between Biological and
Artificial Neural Networks
Characteristics Artificial NN Biological NN
Speed Faster in processing Slower in processing
(operate on mill second (operate on mill second
range range
Processing Instructions execute in a Biological NN can perform
sequential mode on a massively parallel
conventional computer operations
Size and Do not involve as much The brain has about 1011
Complexity neurons, and are neurons and around 1015
unable to perform connections. This gives
complex pattern brain power to perform
recognition tasks. complex pattern
recognition tasks.
 Comparison between Biological and Artificial
Neural Networks

Characteristics Artificial NN Biological NN

Storage Information is stored in the Information is adaptable and is
memory. Any new stored in the strengths of the
information stored at same inter-connections. New
location destroys old information is added by
information. It is strictly adjusting the interconnection
replaceable. strengths, without destroying
the old information.
Fault Tolerance Inherently not fault tolerant , Exhibit fault tolerance, since is
since information is distributed in the connections
corrupted in the memory throughout the network.
cannot be retrieved. Information is still preserved due
the distributed nature of the
encoded information.
Control Mechanism There is a Control unit No central control for processing
which monitors all activities information in brain.
Basic Building Blocks of ANN

 The building blocks of the artificial neural

network are:
 Network Architecture
 Setting the weights
 Activation function
Network Architecture
The arrangement of neurons into layers and
pattern of connection within and in between
layers is called as the architecture of the net.
Various types of network architecture are:
 Feed Forward Architectures
 Feedback Architectures
 Simple Recurrent Networks
 Fully Recurrent Net
 Competitive Net etc.