Neural Networks and Fuzzy Systems

ADALINE stands for an adaptive linear neuron

ADALINE has a linear activation function, i.e. f(net)=net;




Learning rule(Widrow-Hoff learning rule):



ADALINE
Learning objective:
net= d(X)
1 ) ( = '
j
net f
0 , ) 1 ( ) ( > + = c x c j w j w
i j i i
o
Neural Networks and Fuzzy Systems
Learning Algorithm
c
x
x
x
Neural Networks and Fuzzy Systems
ADLINE is a linear approximator. As a linear classifier, it
often connects with a hard-limiter at the output
ADALINE
Learning may fall into a local minimum, i.e. 0 but ; 0 = =
c
c
E
w
E
i
Neural Networks and Fuzzy Systems
Model identification of a linear system
Adaptive Control;
Market Prediction
Applications of ADALINE
Controller
Adaline Model
Plant
Controller Parameters
u
y
plant

u
c

... ) 1 ( ) (
... ) 2 ( ) 1 ( ) (
1
2 1
+ + +
+ + =

t u w t u w
t y w t y w t y
m m
Neural Networks and Fuzzy Systems
Stabilization of an invert pendulum by learning from
demonstration
Applications of ADALINE
... ) 2 ( ) 1 ( ) (
2 1
+ O + O = O t w t w t
ARM ARM C
Neural Networks and Fuzzy Systems
Adaptive Noise Cancellation :
Applications of ADALINE
Neural Networks and Fuzzy Systems
Can only deal with linear function approximation;
If an attempt is made to implement a non-linear function,
the solution that it arrives at will contain errors but will be a
kind of best fit, in the sense that it will be the best that can
be achieved using just a single layer NN;
In order to deal with non-linear approximation:
1. Introduce nonlinear high order terms for generating a
nonlinear separation line:
2. A solution is to have a layer of ADALINEs (with hard
limiter) feeding into a logic device such as an AND gate.
These multi-layered networks are called MADALINE.
ADALINE

+ =
ij i ijk i ik k
x x w x w y
Neural Networks and Fuzzy Systems
Summary
Adaline
Madaline
Readings
Pictons book: Chapter 2 (Adaline)
Haykins book: Chapter 4 (if you want to know convergence
proof of the learning law)
For the reading week
Widrow, 30 years of adaptive neural networs: perceptron,
madaline and backpropagation
Breazeal,Challenges in building robots that imitate people