Professional Documents
Culture Documents
Artificial Neural Networks - : Introduction
Artificial Neural Networks - : Introduction
- Introduction -
Peter Andras
peter.andras@ncl.ac.uk
Overview
1. Biological inspiration
2. Artificial neurons and neural networks
3. Learning processes
4. Learning with artificial neural networks
Biological inspiration
Animals are able to react adaptively to changes in their
external and internal environment, and they use their nervous
system to perform these behaviours.
Axon
Biological inspiration
dendrites
axon
synapses
i 1 y
w3
..
. wn-1
xn-1
wn
xn
The McCullogh-Pitts model
Artificial neurons
Output
Inputs
The young animal learns that the green fruits are sour,
while the yellowish/reddish ones are sweet. The learning
happens by adapting the fruit picking behavior.
Output
Inputs
y 1 f ( x , w1 ) 4
4 4 4
Neural network mathematics
y out F ( x, W )
r ( x) r (|| x c ||)
|| x w|| 2
Example: f ( x) e 2a 2 Gaussian RBF
|| x w1,k || 2
4
y out wk2 e 2( ak ) 2
x
k 1 yout
Neural network tasks
control
classification These can be reformulated
in general as
prediction
FUNCTION
approximation
APPROXIMATION
tasks.
Task specification:
Error measure:
1 N
E
N
( F ( xt ; W ) y t ) 2
t 1
E
wi c j
(W )
wi j
wi j , new
wi wi
j j
Learning:
E (t ) ( w(t )T x t yt ) 2
wi (t 1) wi (t ) c wi (t ) c
wi wi
wi (t 1) wi (t ) c ( w(t )T x t yt ) xit
m
w(t ) x w j (t ) x tj
T
j 1
k 1
1 2 N
Data: ( x , y1 ), ( x , y 2 ),..., ( x , yN )
|| x t w1,k || 2
M
Error: E (t ) ( y (t ) out yt ) ( wk2 (t ) e
2 2 ( ak ) 2
yt ) 2
k 1
Learning: E (t )
w (t 1) w (t ) c
2
i
2
i
wi2
|| x t w1,i || 2
E (t )
2 ( ai ) 2
2 ( F ( x t
, W (t )) yt ) e
wi2
M
1
y out F ( x;W ) w 2
k w1,kT x a k
k 1 1 e
Learning with general optimisation
Synaptic weight change rules for the output neuron:
E (t )
w (t 1) w (t ) c
2
i
2
i
wi2
E (t ) 1
2 ( F ( x t
, W (t )) y t )
wi2
1,iT t
1 e w x ai
Synaptic weight change rules for the neurons of the
hidden layer: w (t 1) w (t ) c Ew(t )
1, i
j
1, i
j 1, i
j
E (t ) 1
2 ( F ( x t
, W (t )) y t )
w1j,i w1j,i
1,iT t
w x ai
1 e
1,iT
x t ai
e w
1
w1,iT x t ai
w1j,i 1 e
w 1,iT t
x ai
1 e w1,iT x t ai
2
w j
1, i
w j
1, i
w1,iT x t ai x tj
1,iT
x t ai
ew
w (t 1) w (t ) c 2 ( F ( x , W (t )) yt )
1, i 1, i t
( x tj )
j j
1 e w 1,iT t
x ai
2
New methods for learning with
neural networks
Bayesian learning:
the distribution of the neural network
parameters is learnt