Artificial Neural Networks - 11

Dr. Aditya Abhyankar

Last Time!
ART!!

Algorithm

Algorithm

Algorithm

Example
Use ART1 Net to cluster (1 1 0 0), (0 0 1 1),
(1 0 1 0), (0 1 0 1) in atmost three clusters

Today
PCA
BP

PCA

Principal Component Analysis

Statistical tool for better data representation
Mean
Std deviation
Variance and covariance
Eigenvalues and Eigenvectors

Statistics
Mean, std dev, var always on 1-D
n!
Co-variance between 2-D
(n 2)!* 2
E.g. 2 3
2 1

Try

3
2

PCA
MATLAB Example
Eigenvectors direction !
Strongest eigenvalue of cov Principal
component
Eigenvectors (exist only for square
matrices) are orthogonal if and ONLY if it is
a real symmetric matrix (Gilbert Strang)

Back-Propagation (BP)
Aims at balancing memorization and
generalization
Stage 1: Feedforward I/p training pattern
Stage 2: calculation and backpropagation of
associated error
Stage 3: Adjustments of the weights

Architecture

Hidden
Layer

Nomenclature

Nomenclature

Activation Function
Characteristics:

Continuous
Differentiable
Monotonically non-decreasing
Easily differentiable

Binary Sigmoid Function

Binary Sigmoid Function

range (0,1)

Bipolar Sigmoid Function

range (-1,1)

Algorithm: Training

Algorithm

Algorithm

Algorithm

Algorithm

Application

Example
X-or Problem (linearly not separable) using
2-4-1 backprop Net
Initial weights to
Initial Weights to
o/p layer
hidden layer