Artificial Neural Network

Ms. Swetha R.
Assistant Professor
Department of Electronics &
Communication Engg.

Dept. of ECE, PES University, Bangalore 1

 ARTIFICIAL NEURAL NETWORK

Introduction Class

Ms. Swetha R.
Department of Electronics and Communication Engineering
Dept. of ECE, PES University, Bangalore 2
Artifical Neural Network
Content

1. Syllabus
2. Reference Books
3. Evaluation of the Course
4. Introduction to the course

Dept. of ECE, PES University, Bangalore 3

Syllabus

Unit 1: Introduction

• What is a neural network? Human brain; models of a neuron;

n e u ra l n e t wo rks a s d i re c te d g ra p h s ; fe e d b a c k ; n e t wo rk
architectures;
• Linear Neuron: Adaptive filtering problem; unconstrained
optimization technique, linear least-squares filters; least-mean-
square algorithm.

Unit 2: Perceptron

• Single-layer FFNN:Nonlinear neuron: McMulloch-Pitts model;

perceptron convergence theorem; limitations.
Dept. of ECE, PES University, Bangalore 4
Syllabus

Multilayer FFNN: Back propagation algorithm (BPA): for supervised

learning (sequential and batch processing); heuristics; generalization;
cross-validation; network pruning; limitations. Supervised learning as
an optimization problem.

Unit 3: Radial-Basis Function Networks

• Cover’s theorem on the separability of patterns, interpolation

problem, supervised learning as an ill-posed hyersurface
reconstruction problem, regularization theory, regularization
networks, generalized radial-basis function networks

Dept. of ECE, PES University, Bangalore 5

Syllabus

• XOR problem revisited, estimation of the regularization parameter,

approximation properties of RBF networks, comparison of RBF
networks and multilayer perceptrons; learning strategies.

Unit 4: Learning Processes:

• Error-correction learning; memory-based learning; Hebbian

learning; competitive learning; Boltzmann learning; credit-
assignment problem;learning with and without a teacher.
• Principal Component Analysis: Some intuitive principles of self-
organization; principal components analysis.
• Hebbian-based maximum eigenfilter; Hebbian-based principal
components analysis.
Dept. of ECE, PES University, Bangalore 6
Syllabus

Unit 5: Self Organization Map

• Feature-mapping models; self-organizing map; properties of the

feature map; learning vector quantization.
• Recurrent Networks: Recurrent network architectures; state-space
model.
• Computational power of recurrent networks; learning algorithms;
back-propagation through time; real-time recurrent learning.
Applications of Neural Networks in Engineering.

Dept. of ECE, PES University, Bangalore 7

Syllabus

Text Book:
• S. Haykin, (2003), “Neural Networks: A Comprehensive Foundation”,
2nd edition, Prentice Hall of India.

Reference Books:
1. S. Haykin, (2009), “Neural Networks and Learning Machines”, 3rd
edition, Prentice Hall of India.
2. Alan Murray, (1995), “Applications of Neural Networks”, Springer
Science+Business Media, New York.
3. T. Hagan, H. B. Demuth and M. Beale, (2002), Neural Network
Design, Thomson Learning.
4. M. M. Gupta, L. Jin and N. Homma, (2003), Static and Dynamic
Neural Networks: From Fundamentals to Advanced Theory, John
Wiley-IEEE Press.
Dept. of ECE, PES University, Bangalore 8
Evaluation of the Course

1. T1 -15 Marks
2. T2 - 15 Marks
3. Term-Project- 20 Marks

Dept. of ECE, PES University, Bangalore 9

Introduction

Identify the following image:

Dept. of ECE, PES University, Bangalore 10

Introduction

Dept. of ECE, PES University, Bangalore 11

Introduction

Dept. of ECE, PES University, Bangalore 12

Introduction

Dept. of ECE, PES University, Bangalore 13

Introduction

?????

Dept. of ECE, PES University, Bangalore 14

Introduction

Human
Nervous System

Dept. of ECE, PES University, Bangalore 15

Introduction

Human Nervous System:

• The nervous system is the most complex and highly organized body system.
• It receives information from the sensory organs via nerves, transmits the
information through the spinal cord, and processes it in the brain.
• Then, we perceive the information and we act/respond accordingly.

Dept. of ECE, PES University, Bangalore 16

Introduction

Human Nervous System:

• The human nervse sysem may be viewed as a three stage as depicted in the
below block diagram:

Dept. of ECE, PES University, Bangalore 17

Introduction

Human Nervous System:

• Receptors: Rods and Cones of eyes, Pain, touch, hot and cold receptors of
skin, Stretch receptors of muscles
• Effectors: Muscles and glands, speech generators
• All nervous tissue, from the brain to the spinal cord to the furthest nerve
branch, includes cells called .

Dept. of ECE, PES University, Bangalore 18

Introduction

• Neurons are charged cells:

they conduct
to through
the body
• A typical neuron consists of a
cell body, dendrites, and an
axon with an axon terminal.

Dept. of ECE, PES University, Bangalore 19

Introduction

• Main part of the “

” is Network of
.

Dept. of ECE, PES University, Bangalore 20

Introduction

• The dendrites receive signals from body tissues or other neurons and
pass them into the cell body.
• If an outgoing signal is produced, it zips down the axon to the axon
terminal and passes to the next neuron or target cell.
• This conductive capability sends information up and down nerve
pathways and through the central nervous system at incredible speed.
• Some 100 billion neurons give the brain its awesome processing
power.

Dept. of ECE, PES University, Bangalore 21

Introduction

LOGIC

INPUT OUTPUT

A TRAINED BRAIN
Dept. of ECE, PES University, Bangalore 22
Introduction

Is machine can learn like Human

Beings??

Dept. of ECE, PES University, Bangalore 23

Introduction

It is possible by making the machine

to learn from the given data

Dept. of ECE, PES University, Bangalore 24

Introduction

LETS TEACH MACHINE HOW

TO LEARN FROM DATA

Dept. of ECE, PES University, Bangalore 25

Introduction

Machine Learning!!!

Dept. of ECE, PES University, Bangalore 26

Introduction

To minimize the difference between

predicted output & actual output

INPUTS

Learns the parameters

OLD OUTPUT of the function – An Algorithm

(x)

Dept. of ECE, PES University, Bangalore 27

Introduction

To minimize the difference between

predicted output & actual output

INPUTS OUTPUT

f(x)- A Trained Statistical

Function

Dept. of ECE, PES University, Bangalore 28

Introduction

• Machine Learning is a subfield of artificial intelligence.

• It involved the development of self-learning algorithms to gain knowledge
from that data in order to make predictions.
• Instead of requiring humans to manually derive rules and build models from
analysing large amounts of data, machine learning offers a more efficient
alternative for capturing the knowledge in data to gradually improve the
performance of predictive models, and make data-driven decisions.

Dept. of ECE, PES University, Bangalore 29

Introduction

• Intelligence: The ability to acquire

and apply knowledge and skills.

Natural
Intelligence

Dept. of ECE, PES University, Bangalore 30

Introduction

What am I???

Dept. of ECE, PES University, Bangalore 31

Introduction

Can You
define AI for
me.. ?

Dept. of ECE, PES University, Bangalore 32

Introduction

Dept. of ECE, PES University, Bangalore 33

Introduction

• Deep Learning (DL) is a particular kind

of Machine Learning (ML) algorithm
that is inspired by the functionality of
our brain cells called neurons which led
to the concept of Artificial Neural
Network (ANN).

Dept. of ECE, PES University, Bangalore 34

 Introduction
ARTIFICIAL INTELLIGENCE MACHINE
AI stands for Artificial intelligence, where intelligence
is defined acquisition of knowledge intelligence is
defined as a ability to acquire and apply knowledge.
The aim is to increase chance of success
and not accuracy.
It work as a computer program that does
smart work
The goal is to simulate natural intelligence
to solve complex problem
AI is decision making.
It leads to develop a system to mimic human to
respond behave in a circumstances.
AI will go for finding the optimal solution.
Dept. of ECE, PES University, Bangalore
MACHINE LEARNING
ML stands for Machine Learning which is
defined as the acquisition of knowledge or
skill
The aim is to increase accuracy, but it does
not care about success
It is a simple concept machine takes data
and learn from data
The goal is to learn from data on certain task to
maximize the performance of machine on this task.
ML allows system to learn new things from data.
It involves in creating self learning algorithms.
ML will go for only solution for that whether it
is optimal or not.
35
Introduction

Dept. of ECE, PES University, Bangalore 36

 Applications

SELF DRIVING CAR HUMANOID ROBOT

Dept. of ECE, PES University, Bangalore 37

Applications

• SNAPCHAT FILTERS Netflix — The Application that knows

what you want: The Netflix
recommendation system is powered by
Machine Learning (ML) algorithms and
that is what makes their system so good
in predicting the movies that you are
most likely to watch and hence show
them as suggestions to you.

Dept. of ECE, PES University, Bangalore 38

Introduction

Dept. of ECE, PES University, Bangalore 39

Conclusion

• In todays class we come across the following:

• Syllabus(Lesson Plan)
• Assessment
• Introduction

Dept. of ECE, PES University, Bangalore 40

References

• Neural Networks: A Comprehensive Foundation S.

Haykin, 2ndEdition, Prentice Hall of India, 2003.
• Google. com

Dept. of ECE, PES University, Bangalore 41

THANK YOU

Swetha R.
Department of Electronics and
Communication Engineering
swethar@pes.edu
+91 80 2672 1983 Extn 753